All Questions
Tagged with large-language-model python
566
questions
0
votes
1
answer
36
views
AzureChatOpenAI only uses one tool at a time
LangChain with AzureChatOpenAI is only ever calling one tool at a time.
When prompting the model to multiply and add two sets of numbers, I expect two tool calls, however only one tool is called, ...
- 1
-4
votes
0
answers
18
views
I want the model to generate an exact number of tokens, no more, no less [closed]
Are there any tips or best practices to achieve this? I have tried few-shot prompting
are there any open source models which can perform this?
I have tried few-shot prompting it was not giving best ...
1
vote
0
answers
28
views
TRL SFTTrainer clarification on truncation
I am currently finetuning LLama models using SFTTrainer in huggingface. However, I came up with a question, I can not answer through the documentations (atleast, it is a bit ambigious).
My dataset ...
-2
votes
0
answers
20
views
training help hybrid based model that integrates contextual and numerical features for a classification problem [closed]
I want a critical production RISK analysis problem. So, based on a record I want to risk rank each record from 0 to 5. The training set is fairly imbalanced.
> "0.0 964
> 1.0 393
&...
0
votes
0
answers
47
views
'LlamaForCausalLM' object has no attribute 'max_seq_length'
I'm fine-tuning llama3 using unsloth , I trained my model and saved it successfully but when I tried loading using AutoPeftModelForCausalLM.from_pretrained ,then I used TextStreamer from transformer ...
-3
votes
0
answers
33
views
Integrating web scraping and LLMs [closed]
I wanted to extract some information about a specific drug (lets say Rolvedon) from this site.
I tried using BeautifulSoup and Scrapy but they seem to be very format dependent. I want the code to be ...
0
votes
0
answers
46
views
How to improve response time of Phi-3-medium-128k serverless API?
I have deployed the Phi-3-medium-128k model using Azure AI Studio (serverless deployment). I am using the v1/chat/completions API to get chat completions and I am streaming the response. The time to ...
2
votes
1
answer
139
views
Saving Fine-tune Falcon HuggingFace LLM Model
I'm trying to save my model so it won't need to re-download the base model every time I want to use it but nothing seems to work for me, I would love your help with it.
The following parameters are ...
- 1,334
0
votes
0
answers
24
views
Export a teknium/OpenHermes-2.5-Mistral-7B model to ONNX
I am trying to export teknium/OpenHermes-2.5-Mistral-7B to ONNX,
This is my code:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
import onnx
model_name = "teknium/...
0
votes
0
answers
39
views
DSPy: How to get the number of tokens available for the input fields?
This is a cross-post from Issue #1245 of DSPy GitHub Repo. There were no responses in the past week, am I am working on a project with a tight schedule.
When running a DSPy module with a given ...
- 86
0
votes
0
answers
22
views
Dockerized OpenSearch not returning any hits for queries
I recently tried to move the chatbot project that I am working on over to OpenSearch. At first, my search function was working, but after dockerizing OpenSearch, I've run into the issue where my ...
1
vote
1
answer
65
views
How do I persist FAISS indexes?
In the langchain wiki of FAISS, https://python.langchain.com/v0.2/docs/integrations/vectorstores/faiss/, it only talks about saving indexes to files.
db.save_local("faiss_index")
new_db = ...
- 11
0
votes
0
answers
41
views
AI Stops Abruptly with Langchain and CTransformers
I am facing an issue with my AI application in Python. I am using the chainlit library along with langchain and CTransformers to generate AI responses. However, the AI often stops abruptly before ...
- 1
-1
votes
0
answers
33
views
AssertionError: Unexpected kwargs: {'use_flash_attention_2': False}
I'm using EvolvingLMMs-Lab/lmms-eval to evaluate LLaVa model
after running accelerate launch --num_processes=8 -m lmms_eval --model llava --model_args pretrained="liuhaotian/llava-v1.5-7b" ...
- 41
1
vote
0
answers
77
views
Passing Additional Information in LangChain abatch Calls
Given an abatch call for a LangChain chain, I need to pass additional information, beyond just the content, to the function so that this information is available in the callback, specifically in the ...
- 631