Kafka MCP Server
A Message Context Protocol (MCP) server that integrates with Apache Kafka to provide publish and consume functionalities for LLM and Agentic applications.
Overview
This project implements a server that allows AI models to interact with Kafka topics through a standardized interface. It supports:
- Publishing messages to Kafka topics
- Consuming messages from Kafka topics
Prerequisites
- Python 3.8+
- Apache Kafka instance
- Python dependencies (see Installation section)
Installation
- Clone the repository: ```bash git clone <repository-url> cd <repository-directory> ```
- Create a virtual environment and activate it: ```bash python -m venv venv source venv/bin/activate # On Windows, use: venv\Scripts\activate ```
- Install the required dependencies: ```bash pip install -r requirements.txt ``` If no requirements.txt exists, install the following packages: ```bash pip install aiokafka python-dotenv pydantic-settings mcp-server ```
Configuration
Create a `.env` file in the project root with the following variables:
Usage
Running the Server
You can run the server using the provided `main.py` script:
Available transport options:
- `stdio`: Standard input/output (default)
- `sse`: Server-Sent Events
Integrating with Claude Desktop
To use this Kafka MCP server with Claude Desktop, add the following configuration to your Claude Desktop configuration file:
Replace `<PATH TO PROJECTS>` with the absolute path to your project directory.
Project Structure
- `main.py`: Entry point for the application
- `kafka.py`: Kafka connector implementation
- `server.py`: MCP server implementation with tools for Kafka interaction
- `settings.py`: Configuration management using Pydantic
Available Tools
kafka-publish
Publishes information to the configured Kafka topic.
kafka-consume
consume information from the configured Kafka topic.
- Note: once a message is read from the topic it can not be read again using the same groupid