Skip to content

evaluation

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

evaluation

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
simcse
simcse
 
 
README.md
README.md
 
 
eval.py
eval.py
 
 
eval_g.py
eval_g.py
 
 
eval_r.py
eval_r.py
 
 
get_remote_score.py
get_remote_score.py
 
 

README.md

Graph-R1 Evaluation

Preparation

First, to evaluate Graph-R1, we should use evaluation as the working directory.

cd evaluation

Then, we need to set openai api key in openai_api_key.txt file.

Eval for Graph-R1

python get_remote_score.py --dir  ../expr_results/Qwen2.5-3B-Instruct_2WikiMultiHopQA_grpo
# python get_remote_score.py --dir  ../expr_results/Qwen2.5-3B-Instruct_HotpotQA_grpo
# python get_remote_score.py --dir  ../expr_results/Qwen2.5-3B-Instruct_Musique_grpo
# python get_remote_score.py --dir  ../expr_results/Qwen2.5-3B-Instruct_NQ_grpo
# python get_remote_score.py --dir  ../expr_results/Qwen2.5-3B-Instruct_PopQA_grpo
# python get_remote_score.py --dir  ../expr_results/Qwen2.5-3B-Instruct_TriviaQA_grpo