Questions tagged [bert-language-model]
BERT, or Bidirectional Encoder Representations from Transformers, is a method of pre-training language representations which obtains state-of-the-art results on a wide array of Natural Language Processing (NLP) tasks. BERT uses Transformers (an attention mechanism that learns contextual relations between words or sub words in a text) to generate a language model.
bert-language-model
1,800
questions
0
votes
0
answers
26
views
CUDA error: device-side assert triggered Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions
i am trying to convert my text into its embeddings using a bert model , when i apply this to my my dataset it works fine for some of my inputs then stops and gives that error
i have set ...
-1
votes
0
answers
18
views
Multitasking bert for multilabel classification of 5 classes [duplicate]
I built 5 BioClinicalBERT-based models (finetuned bert) to predict labels for medical records for the following categories:
specialties = ["aud","den","oph","oto&...
-1
votes
0
answers
11
views
Hybridized collaborative filtering and sentence similarity-based system for doctor recommendation based on user input of symptoms and location
I'm trying to solve a problem of recommending a doctor based on a user's symptoms and location using a hybridized collaborative filtering and sentence similarity-based recommender system that follow ...
1
vote
0
answers
22
views
Multitasking bert for multilabel classification of 5 categories
I built and finetuned 5 BioClinicalBERT-based models (finetuned bert) to predict labels for medical records for the following categories:
specialties = ["aud","den","oph",...
0
votes
0
answers
35
views
Separating text into smaller chunks based on meaning
I am working on a project involving approximately 8,000 job advertisements in CSV format. I have extracted job titles, IDs, descriptions, and other relevant information and saved it in a PostgreSQL ...
- 1
-3
votes
1
answer
18
views
how to match job title with vacancies name or vacancy descriptions? [closed]
How to match 400 professions to 10,000 job vacancies? I have two files: one contains the profession names and the sector to which they belong, and the second file is 10,000 vacancies from hh.kz, ...
1
vote
2
answers
62
views
Identify starting row of actual data in Pandas DataFrame with merged header cells
My original df looks like this -
df
Note in the data frame:
The headers are there till row 3 & from row 4 onwards, the values for those headers are starting.
The numbers of rows & columns ...
- 11
0
votes
0
answers
18
views
Finetuning BERT on classification task, tensor device mismatch error
I'm having trouble on fine-tuning a BERT model on a classification task, as I'm quite new to this. My data is composed of two columns, "item_title" (my input) and "meta_categ_id" (...
0
votes
0
answers
42
views
BERT embedding cosine similarities look very random and useless
I thought you can use BERT embeddings to determine semantic similarity. I was trying to group some words in categories using this, but the results were very bad.
E.g. here is a small example with ...
- 93
0
votes
0
answers
34
views
The Impact of Pretraining on Fine-tuning and Inference
I am working on a binary prediction classification task, primarily focusing on fine-tuning a BERT model to learn the association between CVEs and CWEs. I've structured my task into three phases: first,...
- 19
0
votes
0
answers
32
views
The accuracy from pretraining is worse than without pretraining
My current task is to classify the association between CVEs and CWEs. However, I've noticed that using BertModel.from_pretrained('bert-base-uncased') in the fine-tuning stage results in lower accuracy ...
- 19
1
vote
0
answers
21
views
Why am I seeing unused parameters in position embeddings when using relative_key in BertModel?
I am training a BERT model using pytorch and HuggingFace's BertModel. The sequences of tokens can vary in length from 1 (just a CLS token) to 128. The model trains fine when using absolute position ...
- 11
-1
votes
0
answers
23
views
BERT: how to get a quoted string as token
I eventually managed to train a model, based on BERT (bert-base-uncased) and TensorFlow, to extract intents and slots for texts like this:
create a doc document named doc1
For this text, my model ...
- 1,526
-1
votes
0
answers
17
views
Checking semantic meaning of 2 texts while considering the order of the texts
I am doing a task related to checking the semantic meaning similarity between 2 texts. There I used BERT sentence-transformers/all-MiniLM-L6-v2 model.
Input 1 - "Object moves in uniform ...
- 194
0
votes
0
answers
35
views
Exporting a Bert-based PyTorch model to CoreML. How can I make the CoreML model work for any input?
I use the code below to export a Bert-based PyTorch model to CoreML.
Since I used
dummy_input = tokenizer("A French fan", return_tensors="pt")
the CoreML model only works with ...
- 81.6k