Skip to content

Issues: modelscope/evalscope

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

25 Open 79 Closed
25 Open 79 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

使用压测工具报错 perf
#236 opened Dec 11, 2024 by yangyu6
3 tasks
@Yunnglin
1
请问压测工具能不能添加tpot的p99，p95这些指标的监控 enhancement New feature or request perf
#235 opened Dec 11, 2024 by everloom
@Yunnglin
4
问答评估 native
#233 opened Dec 9, 2024 by lizhen-lizhen
1
使用opencompass评测后端结果和官网结构不一样 opencompass
#229 opened Dec 5, 2024 by yosure
4 tasks
@wangxingjun778
1
当模型推理结果为空时会报错 native
#228 opened Dec 3, 2024 by ChavesLiu
4
评估越来越慢，gpu利用率低 native
#227 opened Dec 3, 2024 by ChavesLiu
2 tasks done
@wangxingjun778
1
ragas eval error rageval
#226 opened Dec 2, 2024 by BiaoZhouZz
4
采用 Openai server 测试，能够产生 inference 结果，但是 eval 输出结果为空 opencompass
#208 opened Nov 22, 2024 by HaoWuSR
9 tasks
@wangxingjun778
2
为啥提高并发和请求数反而速度会变快，正常吗？ perf question Further information is requested
#207 opened Nov 21, 2024 by z1054136399
9 tasks
3
perf统计性能数据问题 documentation Improvements or additions to documentation perf
#204 opened Nov 14, 2024 by weibingo
5 tasks
@Yunnglin
4
bbh测试集之前可以测，现在测不了了 bug Something isn't working native
#202 opened Nov 14, 2024 by charliedream1
2
使用python代码如何统一指定离线data目录？ documentation Improvements or additions to documentation
#201 opened Nov 13, 2024 by charliedream1
opencompass数据集中safety板块数据集支持 enhancement New feature or request
#197 opened Nov 12, 2024 by A-Zi-cong-xiao-jiu-hen-ke-ai
2
评估提示模板 enhancement New feature or request
#195 opened Nov 12, 2024 by lizhen-lizhen
1
OpenCompass打印的测试集列表和网址给出的不一致，导致无法测试，比如cmmlu enhancement New feature or request
#191 opened Nov 9, 2024 by charliedream1
@wangxingjun778
11
能否给个longwriter和toolbench使用openai接口测试的例子 documentation Improvements or additions to documentation
#189 opened Nov 9, 2024 by charliedream1
1
module 'evaluate' has no attribute 'load' duplicate This issue or pull request already exists
#188 opened Nov 9, 2024 by charliedream1
9 tasks
@wangxingjun778
3
输出结果没有分数 bug Something isn't working opencompass
#181 opened Nov 8, 2024 by Leo20100307
9 tasks
@wangxingjun778
6
数据库写入不进去 perf
#150 opened Oct 12, 2024 by LeoCeasar
@Yunnglin
23
访问api 服务，当并发数大于等于500时，出现请求失败 perf
#142 opened Sep 29, 2024 by LHB-kk
@lxline
3
evalscope perf 测试sglang 部署的openai api server 无法输出结果 perf question Further information is requested
#128 opened Sep 12, 2024 by hetian127
@wangxingjun778
4
请问，评估支持使用昇腾910NPU嘛？ good first issue Good for newcomers
#123 opened Sep 9, 2024 by yiyayieryo
4
Baseline模型对比模式结果错误
#118 opened Aug 29, 2024 by stay-leave
2
请教一下，如何使用openai兼容格式的大模型作为评估模型来进行两模型在自定义数据集上的评估任务呢？
#115 opened Aug 26, 2024 by EvilCalf
3
model_id为local ckpt路径时，infer template映射和解析出现问题
#24 opened Mar 26, 2024 by wangxingjun778
ProTip! Follow long discussions with comments:>50.