Unity MCP with Ollama Integration

Overview

1. Unity (via C#)

2. A Python MCP server

3. Local LLMs running through Ollama

• **Asset Management**: Create, import, and manipulate Unity assets programmatically

• **Scene Control**: Manage scenes, objects, and their properties

• **Material Editing**: Modify materials and their properties

• **Script Integration**: View, create, and update Unity scripts

• **Editor Automation**: Control Unity Editor functions like undo, redo, play, and build

Supported Models

• **deepseek-r1:14b** - A 14 billion parameter model with strong reasoning capabilities

• **gemma3:12b** - Google's 12 billion parameter model with good general capabilities

Installation (Asset Method)

Prerequisites

• Unity 2020.3 LTS or newer

• Python 3.10 or newer

• [Ollama](https://ollama.ai/) installed on your system

• The following LLM models pulled in Ollama: - `ollama pull deepseek-r1:14b` - `ollama pull gemma3:12b`

Step 1: Download and Install Editor Scripts

1. Download or clone this repository: ``` git clone https://github.com/ZundamonnoVRChatkaisetu/unity-mcp-ollama.git ```

2. Create a folder in your Unity project's Assets directory: ``` Assets/UnityMCPOllama ```

3. Copy the `Editor` folder from the cloned repository to your Unity project: ``` # Copy the entire Editor folder [Repository]/Editor Assets/UnityMCPOllama/Editor ```

4. Verify the folder structure is correct: ``` Assets/ UnityMCPOllama/ Editor/ MCPEditorWindow.cs UnityMCPBridge.cs ```

5. Let Unity import and compile the scripts

Step 2: Set Up Python Environment

1. Create a folder for the Python environment (outside your Unity project): ``` mkdir PythonMCP cd PythonMCP ```

2. Copy the Python folder from the cloned repository: ``` cp -r [Repository]/Python . ```

3. Create and activate a virtual environment: ```bash # Create a virtual environment python -m venv venv # Activate the virtual environment # On Windows: venv\Scripts\activate # On macOS/Linux: source venv/bin/activate ```

4. Install dependencies: ```bash cd Python pip install -e . ```

Step 3: Configure Ollama

1. Ensure Ollama is installed and running on your system

2. Pull the supported models: ```bash ollama pull deepseek-r1:14b ollama pull gemma3:12b ```

3. Start Ollama server: ```bash ollama serve ```

Using Unity MCP with Ollama

Step 1: Start Unity Bridge

1. Open your Unity project

2. Navigate to `Window > Unity MCP` to open the MCP window

3. Click the **Start Bridge** button to start the Unity bridge

Step 2: Start Python Server

1. Open a command prompt or terminal

2. Navigate to your Python environment: ```bash cd PythonMCP ```

3. Activate the virtual environment: ```bash # On Windows: venv\Scripts\activate # On macOS/Linux: source venv/bin/activate ```

4. Navigate to the Python directory and start the server: ```bash cd Python python server.py ```

Step 3: Configure Ollama Settings

1. In the Unity MCP window, locate the **Ollama Configuration** section

2. Verify or update the following settings: - **Host**: localhost (default) - **Port**: 11434 (default) - **Model**: Select either `deepseek-r1:14b` or `gemma3:12b` - **Temperature**: Adjust as needed (0.0-1.0)

3. Click **Apply Ollama Configuration**

Step 4: Use the Chat Interface

1. Click the **Show Chat Interface** button in the Unity MCP window

2. Type your instructions in the message field

3. Click **Send** to process your request

• "Create a red cube at position (0, 1, 0)"

• "Add a sphere to the scene and apply a blue material"

• "List all objects in the current scene"

• "Write a simple movement script and attach it to the cube"

Connection Status Indicators

• **Python Server Status**: Indicates whether the Python server is running - Green: Connected - Yellow: Connected but with issues - Red: Not connected

• **Unity Bridge Status**: Shows if the Unity socket server is running - Running: Unity is listening for connections - Stopped: Unity socket server is not active

• **Ollama Status**: Shows the connection status to Ollama - Connected: Successfully connected to Ollama server - Not Connected: Unable to connect to Ollama

Troubleshooting

Common Issues

1. **"Not Connected" Status for Python Server** - Ensure the Python server is running (`python server.py`) - Check for errors in the Python console - Verify the Unity Bridge is running

2. **Cannot find Unity MCP menu** - Make sure the Editor scripts are properly imported in your project - Check the Unity console for any errors - Restart Unity if necessary

3. **Ollama Connection Issues** - Verify Ollama is running with `ollama serve` - Check that models are properly pulled - Ensure no firewall is blocking port 11434

4. **MCP Command Execution Fails** - Check Python console for detailed error messages - Verify that the Unity Bridge is running - Make sure the prompt is clear and specific

Explicit Setup Instructions for Python Environment

1. Install Python 3.10 or newer

2. Install Ollama from [ollama.ai](https://ollama.ai/)

3. Create a dedicated directory for the Python environment: ``` mkdir C:\PythonMCP cd C:\PythonMCP ```

4. Clone or download this repository and copy the Python folder: ``` git clone https://github.com/ZundamonnoVRChatkaisetu/unity-mcp-ollama.git copy unity-mcp-ollama\Python . ```

5. Create a virtual environment: ``` python -m venv venv ```

6. Activate the virtual environment: ``` venv\Scripts\activate ```

7. Install dependencies: ``` cd Python pip install -e . ```

8. Run the server: ``` python server.py ```

Performance Considerations

• For **deepseek-r1:14b**: Recommended minimum 12GB VRAM

• For **gemma3:12b**: Recommended minimum 10GB VRAM

• CPU-only operation is possible but will be significantly slower

Contributing

License

Acknowledgments

• Based on [justinpbarnett/unity-mcp](https://github.com/justinpbarnett/unity-mcp)

• Uses [Ollama](https://ollama.ai/) for local LLM integration