huggingface/datasets: 1.7.0
Quentin Lhoest, Patrick von Platen, Thomas Wolf, Moral, Albert Villanova Del, Yacine Jernite, Suraj Shinde Patil, Mariama Drame, Julien Chaumond, Julien Plu, Joe Davison, Simon Brandeis, Teven Le Scao, Victor Sanh, Kevin Canwen Xu, Lewis Tunstall, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Lysandre Debut · Zenodo (CERN European Organization for Nuclear Research) · 2021
Dataset Changes New: NLU evaluation data #2238 (@dkajtoch) New: Add SLR32, SLR52, SLR53 to OpenSLR #2241, #2311 (@cahya-wirawan) New: Bbaw egyptian #2290 (@phiwi) New: GooAQ #2260 (@bhavitvyamalik) New: SubjQA #2302 (@lewtun) New: Ascent KB #2341, #2349 (@phongnt570) New: HLGD #2325 (@tingofurro) New: Qasper #2346 (@cceyda) New: ConvQuestions benchmark #2372 (@PhilippChr) Update: Wikihow - Clarify how to load wikihow #2240 (@albertvillanova) Update multi_woz_v22 - update checksum #2281 (@lhoestq) Update: OSCAR - Set encoding in OSCAR dataset #2321 (@albertvillanova) Update: XTREME - Enable auto-download for PAN-X / Wikiann domain in XTREME #2326 (@lewtun) Update: GEM - the DART file checksums in GEM #2334 (@yjernite) Update: web_science - fixed download link #2338 (@bhavitvyamalik) Update: SNLI, MNLI- README updated for SNLI, MNLI #2364 (@bhavitvyamalik) Update: conll2003 - correct labels #2369 (@philschmid) Update: offenseval_dravidian - update citations #2385 (@adeepH) Update: ai2_arc - Add dataset tags #2405 (@OyvindTafjord) Fix: newsph_nli - test data added, dataset_infos updated #2263 (@bhavitvyamalik) Fix: hyperpartisan news detection - Remove getchildren #2367 (@ghomasHudson) Fix: indic_glue - Fix number of classes in indic_glue sna.bn dataset #2397 (@albertvillanova) Fix: head_qa - Fix keys #2408 (@lhoestq) Dataset Features Implement Dataset add_item #1870 (@albertvillanova) Implement Dataset add_column #2145 (@albertvillanova) Implement Dataset to JSON #2248, #2352 (@albertvillanova) Add rename_columnS method #2312 (@SBrandeis) add desc to tqdm in Dataset.map() #2374 (@bhavitvyamalik) Add env variable HF_MAX_IN_MEMORY_DATASET_SIZE_IN_BYTES #2399, #2409 (@albertvillanova) Metric Changes New: CUAD metrics #2273 (@bhavitvyamalik) New: Matthews/Pearson/Spearman correlation metrics #2328 (@lhoestq) Update: CER - Docs, CER above 1 #2342 (@borisdayma) General improvements and bug fixes Update black #2265 (@lhoestq) Fix incorrect update_metadata_with_features calls in ArrowDataset #2258 (@mariosasko) Faster map w/ input_columns & faster slicing w/ Iterable keys #2246 (@norabelrose) Don't use pyarrow 4.0.0 since it segfaults when casting a sliced ListArray of integers #2268 (@lhoestq) Fix query table with iterable #2269 (@lhoestq) Perform minor refactoring: use config #2253 (@albertvillanova) Update format, fingerprint and indices after add_item #2254 (@lhoestq) Always update metadata in arrow schema #2274 (@lhoestq) Make tests run faster #2266 (@lhoestq) Fix metadata validation with config names #2286 (@lhoestq) Fixed typo seperate->separate #2292 (@laksh9950) Allow collaborators to self-assign issues #2289 (@albertvillanova) Mapping in the distributed setting #2298 (@TevenLeScao) Fix conda release #2309 (@lhoestq) Fix incorrect version specification for the pyarrow package #2317 (@cemilcengiz) Set default name in init_dynamic_modules #2320 (@albertvillanova) Fix duplicate keys #2333 (@lhoestq) Add note about indices mapping in save_to_disk docstring #2332 (@lhoestq) Metadata validation #2107 (@theo-m) Add Validation For README #2121 (@gchhablani) Fix overflow issue in interpolation search #2336 (@mariosasko) Datasets cli improvements #2315 (@mariosasko) Add key type and duplicates verification with hashing #2245 (@NikhilBartwal) More consistent copy logic #2340 (@mariosasko) Update README vallidation rules #2353 (@gchhablani) normalized TOCs and titles in data cards #2355 (@yjernite) simpllify faiss index save #2351 (@Guitaricet) Allow "other-X" in licenses #2368 (@gchhablani) Improve ReadInstruction logic and update docs #2261 (@mariosasko) Disallow duplicate keys in yaml tags #2379 (@lhoestq) maintain YAML structure reading from README #2380 (@bhavitvyamalik) add dataset card title #2381 (@bhavitvyamalik) Add tests for dataset cards #2348 (@gchhablani) Improve example in rounding docs #2383 (@mariosasko) Paperswithcode dataset mapping #2404 (@julien-c) Free datasets with cache file in temp dir on exit #2403 (@mariosasko) Experimental and work in progress: Cast a dataset for specific tasks Task casting for text classification & question answering #2255 (@SBrandeis) Add check for task templates on dataset load #2390 (@lewtun) Add args description to DatasetInfo #2384 (@lewtun) Improve task api code quality #2376 (@mariosasko)