Here’s a basic Kafka tutorial in Java:
Step 1: Set up a Kafka cluster
- Download and extract the Kafka binaries from the Apache Kafka website.
- Start the ZooKeeper service by running the following command
bin/zookeeper-server-start.sh config/zookeeper.properties
- Start the Kafka broker service by running the following command
bin/kafka-server-start.sh config/server.properties
Step 2: Create a Kafka topic
- Run the following command to create a topic named “test” with a single partition and one replica factor
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
Step 3: Produce messages to the Kafka topic
- Create a Java project in your IDE of choice and add the Kafka client library as a dependency.
- Use the following code to create a Kafka producer and send messages to the “test” topic
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | Properties props = new Properties(); props.put("bootstrap.servers", "localhost:9092"); props.put("acks", "all"); props.put("retries", 0); props.put("batch.size", 16384); props.put("linger.ms", 1); props.put("buffer.memory", 33554432); props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer"); props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer"); Producer<String, String> producer = new KafkaProducer<>(props); for (int i = 0; i < 10; i++) { String message = "Message " + i; producer.send(new ProducerRecord<String, String>("test", message)); } producer.close(); |
Step 4: Consume messages from the Kafka topic
- Use the following code to create a Kafka consumer and receive messages from the “test” topic
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | Properties props = new Properties(); props.put("bootstrap.servers", "localhost:9092"); props.put("group.id", "test-group"); props.put("enable.auto.commit", "true"); props.put("auto.commit.interval.ms", "1000"); props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); Consumer<String, String> consumer = new KafkaConsumer<>(props); consumer.subscribe(Collections.singletonList("test")); while (true) { ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100)); for (ConsumerRecord<String, String> record : records) { System.out.println("Received message: " + record.value()); } } |
That’s it! You now have a basic Kafka setup that can produce and consume messages in Java. Note that there are many more advanced features and configuration options available in Kafka, but this should be enough to get started.