Questions tagged [apache-kafka]
Apache Kafka is a distributed streaming platform that is used for building real-time data pipelines and streaming applications.
apache-kafka
33,301
questions
0
votes
0
answers
4
views
Quickfix/n App - Kafka - Missing messages in Kafka
First of all if I need to separate the post in multiple questions please tell me how to do it by preserving the diagram and general information. thanks
I implemented a dropcopy fix reader and ...
- 93
0
votes
0
answers
18
views
org.apache.kafka.common.errors.SerializationException: Error registering Avro schema
https://i.sstatic.net/jtpBxVPF.png
I send a message from procuder to kafkaya and consumer listens to it.but i can't save the schema in shema registry
2024-07-26T10:22:03.461+03:00 INFO 19888 --- [nio-...
0
votes
0
answers
8
views
How fix Kafka error The producer attempted to update a transaction while another concurrent operation on the same transaction was ongoing?
The work process is as follows: the consumer receives a task, we create a producer, create a transaction, a unique transaction id is required, we do all sorts of calculations and do an offset and a ...
- 11
0
votes
0
answers
22
views
Kafka dissconnecting after few days
We are using Kafka for data exchange between microservices in NestJS and Java. We have an API Gateway in NestJS and a user service in Java. We are facing an issue with Kafka disconnecting after a few ...
- 139
0
votes
0
answers
23
views
Error Handling in Spring boot kafka in Batch Listener
Working on a Spring boot application where it listens to kafka topic, process and inserts to DB. Currently we are manually acknowledging the messages and using batchlistener as "true". We ...
- 53
0
votes
1
answer
15
views
kafka consumer class not found exception
my Spring Boot kafka producer works fine. i tested it using postman. the sent messages are <String,Complaint>, So i used string serializer for the key and Json serializer for the value.
in the ...
-2
votes
0
answers
13
views
How to install kafka in my jdeveloper 12c version 12.2.1.4? can you guide me step by steps
How to install osb-kafka transport in my jdeveloper 12c version 12.2.1.4? can you guide me step by steps
I have installed Apache Kafka and then Apache Ant.
I got stuck in setting the domain ...
1
vote
0
answers
9
views
How to delete offset when message got consumed using franz-go?
I'm using franz-go library to implement the kafka
Case want to achieve :- I want to delete the message from kafka when it get consumed using adminClient
Error :- map[] request key is unknown
Code
func ...
- 117
0
votes
0
answers
6
views
Kafka and Elastic search Certs Manager on GCP
We have a requirememt whereby we are deploying our clusters on GCP along with deploying Kakfa clusters, brokers, topics and partition on Google Cloud Platform.
Now we need to have a monitoring service ...
- 3,133
0
votes
0
answers
19
views
kafka consumer.poll taking different time across parallel threads
I'm running 8 threads in parallel. Each thread is given a list of partitions to process and will process all messages between a given start and end timestamp.
While most of the threads run fine, one ...
- 143
-2
votes
0
answers
10
views
Javadoc failure with Kafka Test
I am trying to run ./gradlew clients:test --tests KafkaProducerTest on the Apache Kafka repository downloaded from github.
However, when I do it, I get the following error:
starting a Gradle Daemon (...
- 361
0
votes
0
answers
13
views
How to have at least once guarantee for fanout in beam with flink runner?
I have a pipeline in beam java using flink
It run as follow
1. kafka topic (event_type1)
2. fanout (1 event_type1 map to 1000 event_type2)
3. repartition
4. process event_type2
5. notify event_type1 ...
- 327
0
votes
0
answers
21
views
Kafka Producer Key Shows Incorrect Value After Sending Message
I am working on a Spring Boot application where I am using Kafka to send messages to a topic. I am setting a specific key for each message, but after successfully sending the message, when I check the ...
- 95
0
votes
0
answers
11
views
Kafka Connect connector won't link up to a topic
In my work project I have some ksqlDB queries that rely on data provided by a topic that's supposed to be populated by a connector. Said connector isn't created through the ksqlDB CLI, but through the ...
0
votes
0
answers
13
views
Why does Flink Table with Kafka Connector not return results for window-based aggregation operations?
I created a table with
create table Bid(
auction BIGINT,
bidder BIGINT,
price BIGINT,
channel VARCHAR,
url VARCHAR,
dateTime TIMESTAMP(3),
...
- 3