Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
100 Open 1,262 Closed
100 Open 1,262 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

fix DeprecationWarning: invalid escape sequence '\s' for whitespace filter
#2560 opened Dec 11, 2024 by baberabb Loading…
batch loglikelihood_rolling across requests
#2559 opened Dec 11, 2024 by baberabb Loading…
1
Adding new subtask to SCORE tasks: non greedy robustness
#2558 opened Dec 10, 2024 by rimashahbazyan Loading…
add llama3 tasks
#2556 opened Dec 10, 2024 by baberabb Loading…
add option to add an assistant_prefix
#2545 opened Dec 5, 2024 by baberabb • Draft
[MM] Chartqa
#2544 opened Dec 5, 2024 by baberabb • Draft
[MM] Ai2d
#2542 opened Dec 5, 2024 by baberabb • Draft
New arabicmmlu
#2541 opened Dec 5, 2024 by bodasadallah Loading…
1
Update KorMedMCQA: ver 2.0
#2540 opened Dec 5, 2024 by GyoukChu Loading…
1
add Darija (Moroccan dialects) tasks including darijammlu. darijahellaswag and darija_bench
#2521 opened Nov 28, 2024 by hadi-abdine Loading…
2
Add --examples Argument for Fine-Grained Task Evaluation in lm-evaluation-harness. This feature is the first step towards efficient multi-prompt evaluation with PromptEval [1,2]
#2520 opened Nov 26, 2024 by felipemaiapolo Loading…
1
max_length not used
#2515 opened Nov 25, 2024 by lintangsutawika Loading…
3
Added small fix to split by eos_token_id before decoding
#2512 opened Nov 24, 2024 by EtashGuha Loading…
1
AraDICE task config file
#2507 opened Nov 19, 2024 by firojalam Loading…
1
fixed mmlu generative response extraction
#2503 opened Nov 18, 2024 by RawthiL Loading…
3
Added regex filter for bbh fewshot
#2502 opened Nov 18, 2024 by RawthiL Loading…
1
Add GigaChat API
#2495 opened Nov 15, 2024 by seldereyy Loading…
Yaml crowspairs tasks
#2488 opened Nov 14, 2024 by NAM00 Loading…
1
Biology ds
#2486 opened Nov 13, 2024 by deema-A Loading…
2
MILU dataset from AI4Bharat for Indic LLM eval
#2482 opened Nov 12, 2024 by abhinand5 Loading…
1
Update citation
#2474 opened Nov 8, 2024 by Sypherd Loading…
1
Use global filter alias
#2473 opened Nov 8, 2024 by Sypherd Loading…
allow fewshots for multimodal tasks
#2450 opened Nov 1, 2024 by artemorloff Loading…
2
Add Aggregation for Kobest Benchmark
#2446 opened Oct 31, 2024 by tryumanshow Loading…
fix tmlu tmlu_taiwan_specific_tasks tag
#2420 opened Oct 22, 2024 by nike00811 Loading…
3
ProTip! What’s not been updated in a month: updated:<2024-11-12.