Questions tagged [fine-tuning]
The fine-tuning tag has no usage guidance.
fine-tuning
186
questions with no upvoted or accepted answers
6
votes
1
answer
4k
views
Fine-tuning BERT sentence transformer model
I am using a pre-trained BERT sentence transformer model, as described here https://www.sbert.net/docs/training/overview.html , to get embeddings for sentences.
I want to fine-tune these pre-trained ...
- 301
4
votes
0
answers
299
views
Huggingface: Fine-tuning (not enough values to unpack (expected 2, got 1))
I'm trying to fine-tune erfan226/persian-t5-paraphraser paraphrase generator model for Persian sentences. I used the Persian dataset of tapaco and reformatted it to match the glue (mrpc) dataset which ...
- 71
3
votes
0
answers
2k
views
How do we fine-tune Llama 2 model for Custom Intent Classification?
How do we fine-tune Llama 2 model for text classification? Additionally, what should the training data be like for Custom Intent Classification?
I aim to fine-tune the llama2 model to detect and ...
- 41
3
votes
1
answer
943
views
Getting missing pandas error while trying to fine-tune GPT3
I'm using the following command :
openai tools fine_tunes.prepare_data -f ./data.jsonl
and I'm getting the following error:
Analyzing...
Traceback (most recent call last):
File "/Users/...
- 51
2
votes
0
answers
379
views
llama2 multiclass multioutput training dataset preparation
I am trying to fine tune my dataset which is a multiclass multioutput probelm. The dataset to model is:
a1y1 + a2y2 + a3y3 = b1*f(x1) + b2*f(x2) + b3*f(x3)
The test data looks
Are there any python ...
- 753
2
votes
1
answer
2k
views
How to fine tune gpt 3.5 on conversational data?
My app provides emotional support to our end users. We have listeners who listen to the end users via chat. Now I have chat threads of one particular listener let say A, and he had talked with 1000 ...
- 178
2
votes
0
answers
2k
views
Finetune Llama 2-7B using PDF document in Sagemaker
Can someone give me ideas on how to fine-tune the Llama 2-7B model in Sagemaker using multiple PDF documents, please?
For now, I used pypdf and extracted the text from PDF but I don't know how to ...
- 53
2
votes
0
answers
299
views
GPT2 LLM fine-tuned model not generating expected answer
I am finetuning gpt2 model to answer questions with given faq.json.
There is some issue with the answer generated by below code. I am assuming I have not done encoding/decoding of questions and ...
- 443
2
votes
1
answer
122
views
How to refine a trained model in gpt2?
Im currently trying to work on text generation with my own text. I have trained my model with gpt2 with my own text. But it is giving random answers. For some questions it is giving me relevant ...
2
votes
1
answer
3k
views
LMM Fine Tuning - Supervised Fine Tuning Trainer (SFTTrainer) vs transformers Trainer
When should one opt for the Supervised Fine Tuning Trainer (SFTTrainer) instead of the regular Transformers Trainer when it comes to instruction fine-tuning for Language Models (LLMs)? From what I ...
2
votes
0
answers
352
views
How to fine-tune spacy-experimental "en-coreference-web-trf" model on my own custom domain dataset
I have a custom dataset of conversational data specific to farming domain. The spacy-experimental coreference model (en-coreference-web-trf) does perform okish in coreference resolution but does not ...
2
votes
1
answer
843
views
Look for good ways to prepare customized dataset for training controlnet with huggingface diffusers
I want to personally train the controlnet, but I find it inconvenient to prepare the datasets. As I follow the huggingface tutorial available at this link: https://huggingface.co/blog/train-your-...
- 21
2
votes
1
answer
1k
views
validation loss shows 'no log' during fine-tuning model
I'm finetuning QA models from hugging face pretrained models using huggingface Trainer, during the training process, the validation loss doesn't show. My compute_metrices function returns accuracy and ...
- 21
2
votes
0
answers
579
views
Torchvision RetinaNet predicts unwanted class background
I want to train the pretrained RetinaNet from torchvision with my custom dataset with 2 classes (without background). To train with RetinaNet, I did follow modifications:
num_classes = 3 # num of ...
- 51
2
votes
0
answers
1k
views
Finetuning Transformers in PyTorch (BERT, RoBERTa, etc.)
Alright. So there are multiple methods to fine tune a transformer:
freeze transformer's parameters and only its final outputs are fed into another model (user trains this "another" model),
...
- 31