Discussions

• 4 You must be logged in to vote

  🙏

  How to load dataset downloaded with huggingface cli?

  minmie asked Apr 17, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  How to combine resumable IterableDataset with accelerate & transformers

  yzhangcs asked Jun 16, 2024 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  how to expose parameters to the return object of load_dataset

  aihao2000 asked Apr 13, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Datasets try to load locally although streaming is set to True in Kaggle notebook.

  RitchieP asked Apr 1, 2024 in Q&A · Closed · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Loading Dataset with .jsonl Metadata.

  nighting0le01 asked Apr 4, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  How can I delete a split wrongly named "all"

  yuanze1024 asked Mar 13, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  Sharing datasets cache across different users in linux

  halean asked Jun 9, 2021 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  How to load large dataset from s3 using load_from_disk?

  zotroneneis asked Jan 29, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  How to create dataset from array of strings and pass it to Trainer ?

  tempdeltavalue asked Jan 24, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Caching of same but differently loaded dataset

  Revist asked Jan 15, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  Data transforms for specific format (e.g. torch)?

  Modexus asked Nov 16, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Dataset.data property with ArrowTable (aka. MemoryMappedTable) not updated after filter?

  alvarobartt asked Nov 14, 2023 in Q&A · Answered

  

  

  1
• 2 You must be logged in to vote

  #️⃣

  MNIST loading incredibly slow with JAX

  ddrous asked Oct 23, 2023 in General · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  support automatic bucketed batching with dynamic batch sizes for multi-GPU

  wangleiofficial asked Jun 15, 2023 in Q&A · Closed · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  dataset = load_dataset("imagefolder", data_dir= data_dir,drop_labels = False)

  cobaltautomationdev asked Sep 19, 2023 in Q&A · Closed · Answered

  

  

  4
• 1 You must be logged in to vote

  🙏

  How to avoid recomputation

  aihao2000 asked Sep 17, 2023 in Q&A · Closed · Answered

  

  

  3
• 1 You must be logged in to vote

  🙏

  my computer hardisk almostfull how can I delete all download dataset ?

  bomb1980 asked Sep 20, 2023 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Understanding the behaviour of load_from_disk

  nihaljn asked Jul 1, 2022 in Q&A · Closed · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  How to speed up training for large batchsize

  aihao2000 asked Sep 19, 2023 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Moving my hugginface dataset under my organization

  Fatima-Haouari asked Jul 28, 2022 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  In the "my_datasets.py" code, how do you get the location of a local dataset.In the "my_datasets.py" code, how to get the location of the local dataset. My approach doesn't work anymore in the new version

  aihao2000 asked Sep 12, 2023 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  How to speed up "Generating train split"

  aihao2000 asked Sep 4, 2023 in Q&A · Closed · Answered

  

  

  9
• 1 You must be logged in to vote

  🙏

  Whether can be in "datasets. GeneratorBasedBuilder. _generate_examples" load model in data processing. When using non-streaming load_dataset, whether the model can be removed to avoid VRAM occupation

  aihao2000 asked Sep 3, 2023 in Q&A · Closed · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Loading large dataset to cloud storage batch-wise

  tillwenke asked Aug 23, 2023 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  #️⃣

  set_transform seem process all the sample on the fly, not batch_size

  maowandong asked Jul 19, 2023 in General · Closed · Unanswered

  

  

  2