We will use the following file in a new directory. Call the file docker-compose.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | --- version: '3' services: zookeeper: image: confluentinc/cp-zookeeper:7.3.2 container_name: zookeeper environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 broker: image: confluentinc/cp-kafka:7.3.2 container_name: broker ports: # To learn about configuring Kafka for access across networks see # https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/ - "9092:9092" depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181' KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://broker:29092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 |
Next we can create the Docker container by using the following command which calls docker compose to create a new container based on the confluent.io image of Kafka. The -d tells Docker to run it in the background
1 | docker compose up -d |

Next we create a topic to write and read messages to and from
1 2 3 4 | docker exec broker \ kafka-topics --bootstrap-server broker:9092 \ --create \ --topic quickstart |

Once have a topic lets set up a listener to read messages that are published on the topic
1 2 3 | docker exec --interactive --tty broker \ kafka-console-producer --bootstrap-server broker:9092 \ --topic quickstart |
This will create a consumer, however all you will see is a ‘>’ which is the Kafka prompt waiting for messages

Now we can start writing reading using a producer with the following command, In a separate terminal use the following commands
1 2 3 4 | docker exec --interactive --tty broker \ kafka-console-consumer --bootstrap-server broker:9092 \ --topic quickstart \ --from-beginning |
Go back to the producer in the first terminal and start writing messages, each line is a new message and appears in the consumer which as follows

Use Ctrl+C to exit both the consumer and the producer. Then to shutdown the container use
1 | docker compose down |
