Kafka Rebalance详解

what happens when a new consumer join the group in kafka ?

• There are 5 group states, and transitions between them look like this (Dead state omitted):

  • Rebalancing consists of two phases - collecting the consumers and assigning the partitions.

  • After triggering rebalancing, Kafka waits for max(rebalance_timeout) for consumers to join.

  • One of the consumers is chosen to be a leader, who is responsible for assigning partitions to all the members.

  • Assignment strategy is a consumer-side piece of code, thus application developers can easily change it.

• At last, here is the icing on the cake - the whole rebalancing as a sequence diagram.

参考链接