(주)미디어그룹사람과숲과 (주)마커의 LLM 연구 컨소시엄에서 개발된 모델입니다

Sakura-SOLAR project;
I noted that all most of things about the Sakura-SOLAR models which is the global LLM Rank 1 on December, 2023.
I hope, the open-source more and more develop!😄😄

• Model Performance
• Training code
• Hyperparameters & Prompt
• References

• 🌸kyujinpy/Sakura-SOLAR-Instruct
• 🌸kyujinpy/Sakura-SOLAR-Instruct-DPO-v1
• 🌸kyujinpy/Sakura-SOLAR-Instruct-DPO-v2
• 🌸kyujinpy/Sakura-SOLRCA-Instruct-DPO🐋
• 🌸🐋kyujinpy/Sakura-SOLRCA-Math-Instruct-DPO-v1📐
• 🌸🐋kyujinpy/Sakura-SOLRCA-Math-Instruct-DPO-v2📐

• I created the 🌸kyujinpy/Sakura-SOLAR-Instruct LLM, which is Open LLM Rank 1.
• I love open-source, I want to share everything about the model that won first rank.
• I hope this GitHub helps a lot of people.😎😎

• 2023.12.28
  • Rank1 (Open LLM leaderboard): 🌸kyujinpy/Sakura-SOLAR-Instruct

Follow up as link.

1. First, donwload mergekit.
2. Implement below command for merge.

# Example)
mergekit-yaml ./config.yml ./Sakura-SOLAR [--cuda]

1. Implement below code for dpo.

# Example)
python DPO.py \
    --base_model kyujinpy/Sakura-SOLAR-Instruct \
    --data-path  kyujinpy/orca_math_dpo \
    --output_dir [...output_dir...] \
    --num_epochs [...epoch...] \
    --batch_size [...batch_size...] \
    --micro_batch_size [...micro_batch...] \
    --learning_rate [...learning_rate...] \
    --lora_r 16 \
    --lora_alpha 16 \
    --lora_dropout 0.05 \
    --lora_target_modules [...target_modules...] \
    --lr_scheduler 'linear' \
    --warmup_ratio 0.1 \
    --cutoff_len 4096 \

2. Merge: model + LoRA layer

python merge.py \
    --base_model_name_or_path kyujinpy/Sakura-SOLAR-Instruct \
    --peft_model_path [...output_dir...] \
    --output_dir [...output_final_dir...]

• 😎kyujinpy/Sakura-SOLAR-Instruct

slices:
  - sources:
      - model: VAGOsolutions/SauerkrautLM-SOLAR-Instruct
        layer_range: [0, 48]
      - model: upstage/SOLAR-10.7B-Instruct-v1.0
        layer_range: [0, 48]
        
merge_method: slerp
base_model: upstage/SOLAR-10.7B-Instruct-v1.0

parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
tokenizer_source: union
    
dtype: float16

• 😎kyujinpy/Sakura-SOLAR-Instruct-DPO-v1

### User:

### Assistant: 

Prompting

• 😎kyujinpy/Sakura-SOLAR-Instruct-DPO-v2

### User:

### Assistant:

• 😎kyujinpy/Sakura-SOLRCA-Instruct-Dpo

### User:

### Assistant:

• 😎kyujinpy/Sakura-SOLRCA-Math-Instruct-Dpo-v1

### User:

### Assistant:

• 😎kyujinpy/Sakura-SOLRCA-Math-Instruct-Dpo-v2

### User:

### Assistant:

Prompting

• Share code
• Share hyperparameters
• Share datasets

• Platypus
• DPO code
• Mergekit
• upstage/SOLAR-10.7B-Instruct-v1.0
• VAGOsolutions/SauerkrautLM-SOLAR-Instruct