Kafka Poll详解
what happens when you call poll on kafka consumer?
- Below is the sequence of steps to fetch the first batch of records. As you see in the first poll we fetch cluster topology, discover our group coordinator, ask it to join the group, start heartbeat thread, initialize offsets and finally fetch the records.
-
What can we conclude from inspecting the first poll of Kafka consumer?
-
Instantiating a new consumer and subscribing for topics does not create any new connection or thread.
-
Every consumer ensures its initialization on every poll. It creates any threads necessary, connects to servers, joins the group, etc.
-
Consumer is not thread safe - you can’t call its methods from different threads at the same time or else you’ll get an exception.
-
You have to call poll once in a while to ensure it is alive and connected to Kafka.
-
There is a heartbeat thread that notifies cluster about consumer liveness. It is created within poll method if it does not exist.
-