All Questions
Tagged with fine-tuning peft
10
questions
0
votes
0
answers
94
views
The issue of bitsandbytes package supporting CUDA 12.4 version
when running the peft fine-tuning program, execute the following code:
model = get_peft_model(model, peft_config)
report errors:
Could not find the bitsandbytes CUDA binary at WindowsPath('D:/Users/1/...
- 1
0
votes
0
answers
29
views
QLora using peft from HF and custom class for binary classification
I am fine-tuning an mistral-7B LLM model for binary classification. I realize it may be an overkill; but we are running some experiments.
So far, I have used HuggingFace libraries like peft and ...
- 1,992
0
votes
0
answers
78
views
Creating a Dataset for Fine tuning LLM using PEFT and SFT Trainer?
I have a dataset of 1000 records with 3 columns "question" ,"step by step answer"," single word answer" in CSV format. I tried to fine tune an LLM (gemma) on this dataset ...
- 95
0
votes
0
answers
239
views
Issues finetuning LORA of 8bit Llama3 on custom dataset
I have been trying to fine tune a QLora version of Llama3-8B-IT model on kaggle notebook on a custom dataset of about 44 questions. However I am not getting good results in all of the responses. The ...
- 38
1
vote
0
answers
114
views
Peft model from checkpoint leading into size missmatch
I have trained peft model and saved it in huggingface. No i want to merge it with base model.
i have used following code.
from peft import PeftModel, PeftConfig,AutoPeftModelForCausalLM
from ...
1
vote
0
answers
150
views
PEFT LLM model loading is using all available RAM when connected to a GPU on Google Colab?
I am getting the all available RAM has been used error when running the following code (I haven't actually performed any fine-tuning as of yet here):
model_path = 'openlm-research/open_llama_3b_v2'
...
- 19
2
votes
1
answer
2k
views
What's the difference between PeftModel.from_pretrained & get_peft_model in initiating a peft model?
In the examples from PEFT source code, I found two ways to load the model:
model = PeftModel.from_pretrained(model, peft_model_id, device_map="auto", max_memory=max_memory)
model = ...
- 21
1
vote
0
answers
220
views
Questions about distributed finetuning of transformers model (chatglm) with Accelerate in Kaggle GPUs
I am trying to finetune the chatglm-6b model using LoRA with transformers and peft in Kaggle GPUs (2*T4). The model structure:
The traditional loading method (AutoModel.from_pretrained) needs to load ...
- 21
24
votes
3
answers
24k
views
Target modules for applying PEFT / LoRA on different models
I am looking at a few different examples of using PEFT on different models. The LoraConfig object contains a target_modules array. In some examples, the target modules are ["query_key_value"]...
- 1,153
1
vote
0
answers
449
views
How to improve the output of fine tuned Open Llama 7b model for text generation?
I am trying to fine tune a openllama model with huggingface's peft and lora. I fine tuned the model on a specific dataset. However, the output from the model.generate() is very poor for the given ...
