Questions tagged [tfrecord]
TensorFlow Record Format. A TFRecord file represents a sequence of (binary) strings. The format is not random access, so it is suitable for streaming large amounts of data but not suitable if fast sharding or other non-sequential access is desired.
tfrecord
422
questions
0
votes
0
answers
13
views
tensorflow tfreocrd dataset load slow,how can I sovle this bottleneck,changing params like num_parallel_calls seems no use
I have tried to change almost all the params tf.Dataset offered,but all my effort seems no useful.Loading data is my bottleneck.
Here show my code and other details.
def feture_name(column, ...
- 1
0
votes
0
answers
34
views
How can I create sequences (windows) from data in a TFRecord?
I have around 1800 short videos. Each video is around 30 seconds. I trained a VAE to encode each frame into a latent vector (of size 200).
Using this VAE, then I created a TFrecord with an entry for ...
- 1,638
0
votes
0
answers
39
views
How to properly creates TFRecords for multilabel Object-Detection?
I am writing .tfrecord-files to train an object detection model using TensorFlow in google colab. The data I use comes in the coco-json format.
There are multiple objects (labels) in each image from ...
- 11
0
votes
1
answer
57
views
Excessive RAM usage for TFRecord file while feeding into any model
Problem
I have a *.tfrecords file that I want to feed in a ConvLSTM2D model, created using Tensorflow. Here is the model structure.
model = Sequential([
ConvLSTM2D(64, (3, 3), activation='relu', ...
- 75
0
votes
0
answers
41
views
InvalidArgumentError: can't parse serialized example when using model.fit() on keras model
I am using Google Colab
with Datasets from Roboflow (.tfrecord format)
and am trying to train an object detection model.
whenever i run the following method:
model.fit(x = train_dataset, epochs=...
- 11
0
votes
1
answer
35
views
Unrecognized Arguments Error in TensorFlow generate_tfrecord.py Script
I am facing an issue while trying to use the generate_tfrecord.py script in TensorFlow to create TFRecord files for my object detection dataset. The error message I'm receiving is as follows:
usage: ...
0
votes
0
answers
52
views
Sample by index from TFRecordDataset
I have data stored in TFRecord files, each file holding 500 consecutive observations. My current pipeline reads each line in a TFRecord file, parses it and returns the unbatched Tensor with shape (512,...
- 31
0
votes
0
answers
111
views
Generate tfrecord.py : UnicodeDecodeError: ‘utf-8’ codec can’t decode byte
I converted the xml files I created in the label img file to csv and I want to convert them to a tf record file from there. I tried many ways and looked at many forums and videos, but I still cannot ...
user22776335
0
votes
0
answers
25
views
Reading AudioSet data I not all record seems to be present
I am reading data from AudioSet dataset and I find less record than there are supposed to be. The data are save as TFrecord.
To read the Audioset tfrecord and select a thunder sound (label= 287) I ...
0
votes
0
answers
21
views
Why tensorflow skip val_acc check when I use big bacthsize?
I train my tensorflow model based on many tfrecord files. And I read the data_train and data_val as follows:
data = get_dataset(trainFile, btsize*10) ...
- 150
0
votes
1
answer
54
views
TFRecord - Diffrentiate between Example & SequenceExample
If I have a TFRecord file, how do I find if the underlying proto is Example vs SequenceExample?
- 124
2
votes
0
answers
18
views
2
votes
1
answer
134
views
`py_function` causes `ragged_batch()` not working in `tf.data`
I'm working on an object detection project, and use tf.data.Dataset input pipeline to load local data. Because object detection requires not only image but also annotations, and the different ...
- 591
0
votes
1
answer
70
views
time series in tensorflow with tfrecords
I am currently writing a tensorflow model to predict the next value of a label depending on a temporal sequence of items.
I created a series of tfrecords files, each corresponding to a single trial ...
- 27
0
votes
1
answer
172
views
Tensorflow TFRecord: Can't parse serialized example when loop through tensor
I am trying to read a tfrecord but keep getting this error
InvalidArgumentError: Key: sign. Can't parse serialized Example.
Here's what I did:
def decode_tfrec(record_bytes):
features = tf.io....
- 53