Questions tagged [peft]
PEFT stands for Parameter-Efficient Fine Tuning. Use this tag for PROGRAMMING questions only regarding implementing fine tuning. For general help with fine tuning, go to https://stats.stackexchange.com instead.
peft
67
questions
0
votes
0
answers
94
views
The issue of bitsandbytes package supporting CUDA 12.4 version
when running the peft fine-tuning program, execute the following code:
model = get_peft_model(model, peft_config)
report errors:
Could not find the bitsandbytes CUDA binary at WindowsPath('D:/Users/1/...
- 1
0
votes
0
answers
49
views
How to generate output of HuggingFace PEFT model with previous message history as context?
I am trying to generate text from my fine-tuned Llama3 model which uses the PEFT AutoPeftModelForCausalLM library while also passing in previous message history.
This is how I am currently generating ...
- 27
0
votes
0
answers
30
views
untimeError: cutlassF: no kernel found to launch! while generating text using finetuned Peft model
peft_model_id = "/finetuned_deep_seek/transformers/deepseek_finetuned/1"
peft_model = AutoModelForCausalLM.from_pretrained(peft_model_id)
peft_model = PeftModel.from_pretrained(model, ...
- 95
0
votes
0
answers
29
views
QLora using peft from HF and custom class for binary classification
I am fine-tuning an mistral-7B LLM model for binary classification. I realize it may be an overkill; but we are running some experiments.
So far, I have used HuggingFace libraries like peft and ...
- 1,992
0
votes
0
answers
78
views
Creating a Dataset for Fine tuning LLM using PEFT and SFT Trainer?
I have a dataset of 1000 records with 3 columns "question" ,"step by step answer"," single word answer" in CSV format. I tried to fine tune an LLM (gemma) on this dataset ...
- 95
3
votes
1
answer
2k
views
AttributeError: 'TrainingArguments' object has no attribute 'model_init_kwargs'
While finetuning Gemma2B model using QLoRA i'm getting error as AttributeError: 'TrainingArguments' object has no attribute 'model_init_kwargs'
Code:
Loading the libraries
from enum import Enum
from ...
- 182
0
votes
0
answers
239
views
Issues finetuning LORA of 8bit Llama3 on custom dataset
I have been trying to fine tune a QLora version of Llama3-8B-IT model on kaggle notebook on a custom dataset of about 44 questions. However I am not getting good results in all of the responses. The ...
- 38
1
vote
1
answer
631
views
How to fix error `OSError: <model> does not appear to have a file named config.json.` when loading custom fine-tuned model?
Preface
I am new to implementing the NLP model. I have successfully fine-tuned LLaMA 3-8B variants with QLORA and uploaded them to HuggingFace.
The directories are filled with these files:
- ....
- 128
1
vote
0
answers
114
views
Peft model from checkpoint leading into size missmatch
I have trained peft model and saved it in huggingface. No i want to merge it with base model.
i have used following code.
from peft import PeftModel, PeftConfig,AutoPeftModelForCausalLM
from ...
0
votes
1
answer
223
views
Can I dynamically add or remove LoRA weights in the transformer library like diffusers
I see that in the diffuser library, there is this feature to dynamically add and remove LoRA weights based on this article https://github.com/huggingface/blog/blob/main/lora-adapters-dynamic-loading....
0
votes
1
answer
134
views
half() is not supported for quantized model when using FineTuned
I have fine tuned a Llama-3 model ( model_name="meta-llama/Meta-Llama-3-8B") in standard way per this notebook https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g?usp=...
- 994
0
votes
0
answers
9
views
Can I add a composition Block of different adapter types in adapterhub library?
I am using adapterhub library for implementing PEFT methods for fine-tuning. I have a question please:
Can i stack prefix tuning with bottleneck adapter? something like config unioin but as ...
- 33
0
votes
1
answer
253
views
Diffrence between gguf and lora
Does the gguf format perform model quantization even though it's already quantized with LORA?
Hello ! im new to Llms ,and l've fine-tuned the CODELLAMA model on kaggle using LORA.I've merged and ...
- 3
0
votes
0
answers
37
views
KeyError: 'input_ids' arise when I used prompt-tuned Codet5 for
I successfully prompt tuned codet5, however, I can't use the fine-tuned model for inference.
It shows Key error 'input_ids':
Traceback (most recent call last):
File "/home/liangpeng/project/...
- 1
0
votes
0
answers
347
views
LoRA in PEFT can't reduce too much GPU memory consumption as we expected
I try to use lora to finetune a VIT for image classification:
I download the vit-base-patch16-224-in21k model from huggingface
I use the peft in huggingface to implement lora with ViT
Code could run ...
