All Questions
Tagged with large-language-model peft
35
questions
0
votes
0
answers
49
views
How to generate output of HuggingFace PEFT model with previous message history as context?
I am trying to generate text from my fine-tuned Llama3 model which uses the PEFT AutoPeftModelForCausalLM library while also passing in previous message history.
This is how I am currently generating ...
- 27
0
votes
0
answers
30
views
untimeError: cutlassF: no kernel found to launch! while generating text using finetuned Peft model
peft_model_id = "/finetuned_deep_seek/transformers/deepseek_finetuned/1"
peft_model = AutoModelForCausalLM.from_pretrained(peft_model_id)
peft_model = PeftModel.from_pretrained(model, ...
- 95
0
votes
0
answers
78
views
Creating a Dataset for Fine tuning LLM using PEFT and SFT Trainer?
I have a dataset of 1000 records with 3 columns "question" ,"step by step answer"," single word answer" in CSV format. I tried to fine tune an LLM (gemma) on this dataset ...
- 95
3
votes
1
answer
2k
views
AttributeError: 'TrainingArguments' object has no attribute 'model_init_kwargs'
While finetuning Gemma2B model using QLoRA i'm getting error as AttributeError: 'TrainingArguments' object has no attribute 'model_init_kwargs'
Code:
Loading the libraries
from enum import Enum
from ...
- 182
0
votes
0
answers
239
views
Issues finetuning LORA of 8bit Llama3 on custom dataset
I have been trying to fine tune a QLora version of Llama3-8B-IT model on kaggle notebook on a custom dataset of about 44 questions. However I am not getting good results in all of the responses. The ...
- 38
1
vote
1
answer
627
views
How to fix error `OSError: <model> does not appear to have a file named config.json.` when loading custom fine-tuned model?
Preface
I am new to implementing the NLP model. I have successfully fine-tuned LLaMA 3-8B variants with QLORA and uploaded them to HuggingFace.
The directories are filled with these files:
- ....
- 128
1
vote
0
answers
113
views
Peft model from checkpoint leading into size missmatch
I have trained peft model and saved it in huggingface. No i want to merge it with base model.
i have used following code.
from peft import PeftModel, PeftConfig,AutoPeftModelForCausalLM
from ...
0
votes
1
answer
134
views
half() is not supported for quantized model when using FineTuned
I have fine tuned a Llama-3 model ( model_name="meta-llama/Meta-Llama-3-8B") in standard way per this notebook https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g?usp=...
- 994
0
votes
0
answers
9
views
Can I add a composition Block of different adapter types in adapterhub library?
I am using adapterhub library for implementing PEFT methods for fine-tuning. I have a question please:
Can i stack prefix tuning with bottleneck adapter? something like config unioin but as ...
- 33
0
votes
1
answer
252
views
Diffrence between gguf and lora
Does the gguf format perform model quantization even though it's already quantized with LORA?
Hello ! im new to Llms ,and l've fine-tuned the CODELLAMA model on kaggle using LORA.I've merged and ...
- 3
0
votes
0
answers
36
views
KeyError: 'input_ids' arise when I used prompt-tuned Codet5 for
I successfully prompt tuned codet5, however, I can't use the fine-tuned model for inference.
It shows Key error 'input_ids':
Traceback (most recent call last):
File "/home/liangpeng/project/...
- 1
0
votes
0
answers
26
views
Seek help for a more efficient way to use caching to train my lora model
When I was fine-tuning llama2 model using lora, I came across a problem.
The instruction dataset goes something like this:
"Here's the background to the problem... (1000 identical words)... Now ...
- 1
1
vote
0
answers
106
views
What is the difference between merging LORA weight with base model and not merging the weight in LLAMA2 (LLM)?
The question is regarding LLM(Large language model). I want to understand it from LLAMA2 perspective.
Can someone explain why the final outcome is almost same without combining weights? Additionally, ...
- 111
1
vote
2
answers
1k
views
Huggingface transformer train function throwing Device() received an invalid combination of arguments
I was trying to train a model with peft qLora training. Lora config and peft training args are like below:
lora_config = LoraConfig(
r=8,
lora_alpha=16,
target_modules=[
"...
1
vote
0
answers
377
views
How do I save a huggingface LLM model into shards?
I am following the fine tuning guide on the following website:
https://www.labellerr.com/blog/hands-on-with-fine-tuning-llm/
I have successfully fine tuned the Falcon-7b model on a dataset from ...
