multi-llm api gateway.com
multi-llm api gateway.com logo

Multi-LLM API Gateway

Provides a unified FastAPI server for interacting with multiple language model APIs, enabling seamless switching between...

Created byApr 23, 2025

MCP Server

[ ](README.zh-CN.md)

Project Overview

Built on FastAPI and MCP (Model Context Protocol), this project enables standardized context interaction between AI models and development environments. It enhances the scalability and maintainability of AI applications by simplifying model deployment, providing efficient API endpoints, and ensuring consistency in model input and output, making it easier for developers to integrate and manage AI tasks.
MCP (Model Context Protocol) is a unified protocol for context interaction between AI models and development environments. This project provides a Python-based MCP server implementation that supports basic MCP protocol features, including initialization, sampling, and session management.

Features

  • **JSON-RPC 2.0**: Request-response communication based on standard JSON-RPC 2.0 protocol
  • **SSE Connection**: Support for Server-Sent Events connections for real-time notifications
  • **Modular Design**: Modular architecture for easy extension and customization
  • **Asynchronous Processing**: High-performance service using FastAPI and asynchronous IO
  • **Complete Client**: Includes a full test client implementation

Project Structure

Installation

  1. Clone the repository:
  1. Install dependencies:

Usage

Starting the Server

By default, the server will start on `127.0.0.1:12000`. You can customize the host and port using environment variables:

Running the Client

Run the client in another terminal:
If the server is not running at the default address, you can set an environment variable:

API Endpoints

The server provides the following API endpoints:
  • **Root Path** (`/`): Provides server information
  • **API Endpoint** (`/api`): Handles JSON-RPC requests
  • **SSE Endpoint** (`/sse`): Handles SSE connections

MCP Protocol Implementation

Initialization Flow

  1. Client connects to the server via SSE
  1. Server returns the API endpoint URI
  1. Client sends an initialization request with protocol version and capabilities
  1. Server responds to the initialization request, returning server capabilities

Sampling Request

Clients can send sampling requests with prompts:
The server will return sampling results:

Closing a Session

Clients can send a shutdown request:
The server will gracefully shut down:

Development Extensions

Adding New Methods

To add new MCP methods, add a handler function to the `MCPServer` class and register it in the `_register_methods` method:

Integrating AI Models

To integrate actual AI models, modify the `handle_sample` method:

Troubleshooting

Common Issues

  1. **Connection Errors**: Ensure the server is running and the client is using the correct server URL
  1. **405 Method Not Allowed**: Ensure the client is sending requests to the correct API endpoint
  1. **SSE Connection Failure**: Check network connections and firewall settings

Logging

Both server and client provide detailed logging. View logs for more information:

References

  • [MCP Protocol Specification](https://www.claudemcp.com/specification)
  • [FastAPI Documentation](https://fastapi.tiangolo.com/)
  • [JSON-RPC 2.0 Specification](https://www.jsonrpc.org/specification)
  • [SSE Specification](https://html.spec.whatwg.org/multipage/server-sent-events.html)

License

This project is licensed under the MIT License. See the LICENSE file for details.