Mobile Next - MCP server for Mobile Development and Automation | iOS, Android, Simulator, Emulator, and physical devices

Mobile MCP Roadmap: Building the Future of Mobile

Main use cases

• Native app automation (iOS and Android) for testing or data-entry scenarios.

• Scripted flows and form interactions without manually controlling simulators/emulators or physical devices (iPhone, Samsung, Google Pixel etc)

• Automating multi-step user journeys driven by an LLM

• General-purpose mobile application interaction for agent-based frameworks

• Enables agent-to-agent communication for mobile automation usecases, data extraction

Main Features

• Fast and lightweight: Uses native accessibility trees for most interactions, or screenshot based coordinates where a11y labels are not available.

• LLM-friendly: No computer vision model required in Accessibility (Snapshot).

• Visual Sense: Evaluates and analyses what s actually rendered on screen to decide the next action. If accessibility data or view-hierarchy coordinates are unavailable, it falls back to screenshot-based analysis.

• Deterministic tool application: Reduces ambiguity found in purely screenshot-based approaches by relying on structured data whenever possible.

• Extract structured data: Enables you to extract structred data from anything visible on screen.

Mobile MCP Architecture

Wiki page

Installation and configuration

Prerequisites

• Xcode command line tools

• Android Platform Tools

• node.js

• MCP supported foundational models or agents, like Claude MCP, OpenAI Agent SDK, Copilot Studio

Simulators, Emulators, and Physical Devices

• iOS Simulators on macOS/Linux

• Android Emulators on Linux/Windows/macOS

• Physical iOS or Android devices (requires proper platform tools and drivers)

Running in "headless" mode on Simulators/Emulators

1. Start an emulator (avdmanager / emulator command).

2. Run Mobile MCP with the desired flags

• xcrun simctl list

• xcrun simctl boot "iPhone 16"

Mobile Commands and interaction tools

mobile_list_apps

• Description: List all the installed apps on the device

• Parameters:

mobile_launch_app

• Description: Launches the specified app on the device/emulator

• Parameters:

mobile_terminate_app

• Description: Terminates a running application

• Parameters:

mobile_get_screen_size

• Description: Get the screen size of the mobile device in pixels

• Parameters: None

mobile_click_on_screen_at_coordinates

• Description: Taps on specified screen coordinates based on coordinates.

• Parameters:

mobile_list_elements_on_screen

• Description: List elements on screen and their coordinates, with display text or accessibility label.

• Parameters: None

mobile_element_tap

• Description: Taps on a UI element identified by accessibility locator

• Parameters:

mobile_tap

• Description: Taps on specified screen coordinates

• Parameters:

mobile_press_button

• Description: Press a button on device (home, back, volume, enter, power button.)

• Parameters: None

mobile_open_url

• Description: Open a URL in browser on device

• Parameters:

mobile_type_text

• Description: Types text into a focused UI element (e.g., TextField, SearchField)

• Parameters:

mobile_element_swipe

• Description: Performs a swipe gesture from one UI element to another

• Parameters:

mobile_swipe

• Description: Performs a swipe gesture between two sets of screen coordinates

• Parameters:

mobile_press_key

• Description: Presses hardware keys or triggers special events (e.g., back button on Android)

• Parameters:

mobile_take_screenshot

• Description: Captures a screenshot of the current device screen

• Parameters: None

mobile_get_source

• Description: Fetches the current device UI structure (accessibility snapshot) (xml format)

• Parameters: None

Thanks to all contributors

We appreciate everyone who has helped improve this project.

Mobile Next - MCP server for Mobile Development and Automation | iOS, Android, Simulator, Emulator, and physical devices

Mobile MCP Roadmap: Building the Future of Mobile

Main use cases

• Native app automation (iOS and Android) for testing or data-entry scenarios.

• Scripted flows and form interactions without manually controlling simulators/emulators or physical devices (iPhone, Samsung, Google Pixel etc)

• Automating multi-step user journeys driven by an LLM

• General-purpose mobile application interaction for agent-based frameworks

• Enables agent-to-agent communication for mobile automation usecases, data extraction

Main Features

• Fast and lightweight: Uses native accessibility trees for most interactions, or screenshot based coordinates where a11y labels are not available.

• LLM-friendly: No computer vision model required in Accessibility (Snapshot).

• Visual Sense: Evaluates and analyses what s actually rendered on screen to decide the next action. If accessibility data or view-hierarchy coordinates are unavailable, it falls back to screenshot-based analysis.

• Deterministic tool application: Reduces ambiguity found in purely screenshot-based approaches by relying on structured data whenever possible.

• Extract structured data: Enables you to extract structred data from anything visible on screen.

Mobile MCP Architecture

Wiki page

Installation and configuration

Prerequisites

• Xcode command line tools

• Android Platform Tools

• node.js

• MCP supported foundational models or agents, like Claude MCP, OpenAI Agent SDK, Copilot Studio

Simulators, Emulators, and Physical Devices

• iOS Simulators on macOS/Linux

• Android Emulators on Linux/Windows/macOS

• Physical iOS or Android devices (requires proper platform tools and drivers)

Running in "headless" mode on Simulators/Emulators

1. Start an emulator (avdmanager / emulator command).

2. Run Mobile MCP with the desired flags

• xcrun simctl list

• xcrun simctl boot "iPhone 16"

Mobile Commands and interaction tools

mobile_list_apps

• Description: List all the installed apps on the device

• Parameters:

mobile_launch_app

• Description: Launches the specified app on the device/emulator

• Parameters:

mobile_terminate_app

• Description: Terminates a running application

• Parameters:

mobile_get_screen_size

• Description: Get the screen size of the mobile device in pixels

• Parameters: None

mobile_click_on_screen_at_coordinates

• Description: Taps on specified screen coordinates based on coordinates.

• Parameters:

mobile_list_elements_on_screen

• Description: List elements on screen and their coordinates, with display text or accessibility label.

• Parameters: None

mobile_element_tap

• Description: Taps on a UI element identified by accessibility locator

• Parameters:

mobile_tap

• Description: Taps on specified screen coordinates

• Parameters:

mobile_press_button

• Description: Press a button on device (home, back, volume, enter, power button.)

• Parameters: None

mobile_open_url

• Description: Open a URL in browser on device

• Parameters:

mobile_type_text

• Description: Types text into a focused UI element (e.g., TextField, SearchField)

• Parameters:

mobile_element_swipe

• Description: Performs a swipe gesture from one UI element to another

• Parameters:

mobile_swipe

• Description: Performs a swipe gesture between two sets of screen coordinates

• Parameters:

mobile_press_key

• Description: Presses hardware keys or triggers special events (e.g., back button on Android)

• Parameters:

mobile_take_screenshot

• Description: Captures a screenshot of the current device screen

• Parameters: None

mobile_get_source

• Description: Fetches the current device UI structure (accessibility snapshot) (xml format)

• Parameters: None

Thanks to all contributors

We appreciate everyone who has helped improve this project.