I am new to LLM programming in Python and I am trying to fine-tune the instructlab/merlinite-7b-lab model on my Mac M1. My goal is to teach this model to a new music composer Xenobi Amilen I have invented.
The text of this composer is here.
Using the new Ilab CLI from RedHat I created this training set for the model. It is a JSONL file with 100 questions/answers about the invented composer.
I wrote this Python script to train the model. I tested all the parts related to the tokenizer, datasets and it seems to work. However, the final train got this error:
RuntimeError: Placeholder storage has not been allocated on MPS device!
0%| | 0/75 [00:00<?, ?it/s]
I found a lot of articles about this error on Google and also StackOverflow like this, for example. The problem seems that in addition to the model I have to send to mps also the input parameters, but it's not clear to me how to change my code to do that.
I tried several fixes but had no luck. Can anyone can help?
model.to(device)
?Trainer
should use the device of the model. Does it work if you run on cpu instead of the mps?