Search results

Longbench评测使用模型以及参数设置问题

Status: Open.

#112 In THUDM/LongBench;

· Xiaoqi-Zhou916 opened on Feb 17, 2025

Reproducibility Issues

Status: Open.

#111 In THUDM/LongBench;

· Hisham-Cohere opened on Feb 7, 2025

The difference between longbench v2 and helmet?

Status: Open.

#110 In THUDM/LongBench;

· leileilin opened on Feb 6, 2025

w/ cot mode for "thinking" models

Status: Open.

#109 In THUDM/LongBench;

· olive-jy-song opened on Feb 6, 2025

Potential Error in Benchmark Data – Incorrect Answer for Question ID 6701cda0bb02136c067cb6eb

Status: Open.

#108 In THUDM/LongBench;

· Wangmerlyn opened on Feb 5, 2025

LongBench v2 Leaderboard Submission Request: Qwen2.5-14B & Gemini2.0 Flash Experimental Results

Status: Open.

#107 In THUDM/LongBench;

· iclal07 opened on Feb 3, 2025

What are the accurate scores of the task-level Radar graph

Status: Open.

#106 In THUDM/LongBench;

· hzhua opened on Jan 23, 2025

bug in pred.py <<def build_chat(tokenizer, prompt, model_name)>> func

Status: Open.

#105 In THUDM/LongBench;

· zswodegit opened on Jan 20, 2025

Error code: 400 - {'object': 'error', 'message': "This model's maximum context length is 131072 tokens. However, you requested 351430 tokens. Please reduce the length of the messages or completion.", 'type': 'BadRequestError', 'param': None, 'code': 400

Status: Open.

#104 In THUDM/LongBench;

· guanzy2012 opened on Jan 18, 2025

Token indices sequence length is longer than the specified maximum sequence length for this model (1113927 > 128000). Running this sequence through the model will result in indexing errors

Status: Open.

#103 In THUDM/LongBench;

· guanzy2012 opened on Jan 18, 2025

Error code: 404 - {'object': 'error', 'message': 'The model /dev/shm/glm-4-9b-chat/ does not exist.', 'type': 'NotFoundError', 'param': None, 'code': 404}

Status: Open.

#102 In THUDM/LongBench;

· guanzy2012 opened on Jan 18, 2025

Any Implementation of new models like Meta-Llama-3.1-8B , Qwen2.5-7B?

Status: Open.

#101 In THUDM/LongBench;

· WeiweiZhang1 opened on Jan 17, 2025