Creative-Emporium

Creative-Emporium / Mamba-gpt-w-sub-word.py

Created June 24, 2024 06:59 — forked from thistleknot/Mamba-gpt-w-sub-word.py

Mamba Gpt w Sub Word tokenizer

	# -- coding: utf-8 --
	"""SimplerMambaSSM.ipynb
	Automatically generated by Colaboratory.
	#pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121
	Original file is located at
	https://colab.research.google.com/drive/1g9qpeVcFa0ca0cnhmqusO4RZtQdh9umY
	"""
	#!pip install mamba-ssm causal-conv1d
	#!wget https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
	#!mkdir differentattention

Creative-Emporium / llm_layer_compare.ipynb

Created June 24, 2024 05:58 — forked from maldevide/llm_layer_compare.ipynb

A comparison of the delta tensors in the first two layers of llama3 finetunes.

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

Creative-Emporium / tinyllama-merge.ipynb

Created June 24, 2024 05:57 — forked from maldevide/tinyllama-merge.ipynb

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

Creative-Emporium / merge_qlora_with_quantized_model.py

Created June 24, 2024 05:55 — forked from ChrisHayduk/merge_qlora_with_quantized_model.py

Merging QLoRA weights with quantized model

	"""

	The code below combines approaches published by both @eugene-yh and @jinyongyoo on Github.

	Thanks for the contributions guys!

	"""

	import torch
	import peft