All Questions
Tagged with java apache-kafka
6,039
questions
0
votes
0
answers
14
views
KafkaConsumer is not able to process 2 million records
I am facing issue while fetching 2 million records from kafka topic through a consumer.
My requirement is to fetch the record and store those records into a Map and return that Map to other methods so ...
- 1
0
votes
0
answers
20
views
org.apache.kafka.common.errors.SerializationException: Error registering Avro schema
https://i.sstatic.net/jtpBxVPF.png
I send a message from procuder to kafkaya and consumer listens to it.but i can't save the schema in shema registry
2024-07-26T10:22:03.461+03:00 INFO 19888 --- [nio-...
0
votes
0
answers
26
views
Error Handling in Spring boot kafka in Batch Listener
Working on a Spring boot application where it listens to Kafka a topic, processes, and inserts to DB. Currently, we are manually acknowledging the messages and using batchlistener as true. We are ...
- 53
0
votes
0
answers
19
views
kafka consumer.poll taking different time across parallel threads
I'm running 8 threads in parallel. Each thread is given a list of partitions to process and will process all messages between a given start and end timestamp.
While most of the threads run fine, one ...
- 143
-2
votes
0
answers
10
views
Javadoc failure with Kafka Test
I am trying to run ./gradlew clients:test --tests KafkaProducerTest on the Apache Kafka repository downloaded from github.
However, when I do it, I get the following error:
starting a Gradle Daemon (...
- 361
0
votes
0
answers
21
views
Kafka Producer Key Shows Incorrect Value After Sending Message
I am working on a Spring Boot application where I am using Kafka to send messages to a topic. I am setting a specific key for each message, but after successfully sending the message, when I check the ...
- 95
-2
votes
0
answers
22
views
Unable to use Avro default value at Kafka producer [closed]
I am trying to use Avro Schema with Kafka.
My Question is suppose I have a schema with 2 fields and have mentioned defaults in both of them :
"fields": [
{
"name": "lastName&...
0
votes
1
answer
25
views
Kafka-streams filter messages using headers
We're trying to use kafka streams for our project to read data from one topic and write to another, and we have a use case to use KafkaHeaders as a mechanism to filter our certain records.
Example, in ...
- 2,503
0
votes
2
answers
53
views
"Microservices Using Database Per Service" pattern in practice
I am working on a project where I would like to use this pattern. I have read for days, but I am still not sure how to implement this pattern properly and keep the multiple DBs in sync.
Let's suppose ...
- 21.9k
0
votes
0
answers
14
views
Unexpected Kafka request of type METADATA during SASL handshake while trying to connect with kafka server by a consumer
I'm trying to setup SASL plain authentication in kafka. But when I try to read a message or send a message, the Kafka Server logs the following error:
INFO [SocketServer listenerType=ZK_BROKER, nodeId=...
- 3
-1
votes
0
answers
18
views
Custom vault kafka configProvider for Mongo sink Connector
I implemented a custom vault configProvider with dynamic credentials rotation using configData(Map<String, String> data, Long ttl) for a MongoSink Kafka connect.
The first version of plugin ...
- 1
0
votes
0
answers
36
views
spring boot kafka listener consuming messages multiple times
I have a spring boot + kafka application, which consumes the messages from kafka, process and update DB. We have set "manual auto commit" and the max poll records as "500" and max ...
- 53
0
votes
1
answer
31
views
Getting Timeout Kafka exception org.apache.kafka.common.errors.TimeoutException: Call(callName=deleteRecords(api=METADATA)
I am facing kafka timeout exception. Error is:
partition: B3-0 error: org.apache.kafka.common.errors.TimeoutException: Call(callName=deleteRecords(api=METADATA), deadlineMs=1721138713548, tries=721498,...
- 155
0
votes
0
answers
24
views
How Does Java process exits gracefully by shutting kafka consumers and producers during pod deletion in Kubernetes
When the kubectl pod delete command is issued, it isn't waiting for the shutdown hook in a Java Kafka consumer and producer application to complete processing any unprocessed events. What steps need ...
0
votes
0
answers
34
views
My kafka streams is failing to connect to the upstash schema registry
Curling this https://right-boa-11231-eu1-rest-kafka.upstash.io/schema-registry/schemas/ids/8?fetchMaxId=false&subject=test1-value returns
{"schema": "{\"type\":\"...
- 1