Update README.md

Updated
nyunAI · Feb 16, 2024 · c84de00 · c84de00
1 parent 23cfb40
commit c84de00
Showing 1 changed file with 10 additions and 4 deletions.
diff --git a/README.md b/README.md
@@ -1,10 +1,16 @@
-To reproduce results only the baseline.py and evaluate.py are required. 
+To reproduce results:
 
 **Step 1 :**
-Use the baseline.py script to instantiate a model instance, decompose it as per the required budget using the chosen dataset and save the model. (No GPU required)
+Use run decomposer.py script to instantiate a model instance, decompose all its layers into low rank matrices of maximum rank and create a checkpoint. (No GPU required)
 
 **Step 2:**
-Using the evaluate.py script, employ the LLM-Harness to benchmark the model saved in the previous step. Result will be saved in the log path. 
+For surgical rank search based on commonsense reasoning datasets, use the path of the checkpoint from the previous step as an argument for surgical.py and run it. This will run the script along with continuous evaluation and checkpoints for both the disjoint splits (Search split and Test split). A log file will be created to track the progress of the rank search as well as the evaluation metrics.
+**Step 3 (To run rank search based on perplexity):**
+Run the perplexity_test.py script providing the path of the checkpoint from Step 1 as an arguement. Logs will be created similar to Step 2 and evaluation on common sense reasoning tasks will be done on the entire test dataset.
 
 **Comments :**
-The implementation of the proposed method is present in layers.py. 
+To be able to run the evaluation functions present in our repository it is neccessary to pull the master branch from the llm-evaluation-harness: https://github.com/EleutherAI/lm-evaluation-harness/tree/master and run the command 'pip install -e' from inside the pulled repository.
+
+In *Step 1* a different dataset names can be passed as arguements as well as an arguement as 'combination' can be passed depending on which the low rank decomposition will be done. 
+
+Similarly in *Step 2* the rank search can be done using different datasets which can be passed as arguements to the script.