Web Search MCP Server

An MCP server that provides Google search capabilities and web content viewing with advanced bot detection avoidance.

Features

Google Custom Search with advanced filtering

Web content viewing with markdown conversion

Rate limiting and caching

Browser instance pooling

Bot detection avoidance using rebrowser-puppeteer

Prerequisites

Bun runtime v1.0 or higher

Google API credentials (API key and Search Engine ID)

Installation

Configuration

Cookie Setup

For authenticated site access, you'll need to:

Install the [Get cookies.txt LOCALLY](https://chromewebstore.google.com/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpdbgdldlbecc) Chrome extension

Visit the sites you want to authenticate with and log in

Use the extension to export your cookies in JSON format

Store the exported cookies file in a secure location

Set the `BROWSER_COOKIES_PATH` environment variable to the absolute path of your cookies file

MCP Server Configuration

Add the server configuration to your MCP settings file:

For Cline: `%APPDATA%\Code\User\globalStorage\rooveterinaryinc.roo-cline\settings\cline_mcp_settings.json`

For Claude Desktop: - MacOS/Linux: `~/Library/Application Support/Claude/claude_desktop_config.json` - Windows: `%APPDATA%\Claude\claude_desktop_config.json`

Replace `/ABSOLUTE/PATH/TO/web_search_mcp` with the absolute path to your server directory.

Logging Configuration

The following environment variables control logging behavior:

`LOG_LEVEL`: Sets the logging level (error, warn, info, debug). Default: info

`NO_COLOR`: Disables colored output when set to "1"

`BUN_FORCE_COLOR`: Controls colored output in Bun runtime (set to "0" to disable)

`FORCE_COLOR`: Controls colored output globally (set to "0" to disable)

Bot Detection Avoidance

This server uses rebrowser-puppeteer to avoid bot detection:

Runtime.Enable Leak Prevention: - Uses the addBinding technique to avoid Runtime.Enable detection - Works with web workers and iframes - Maintains access to the main world context

Source URL Masking: - Changes Puppeteer's sourceURL to look like a legitimate script - Helps avoid detection of automation tools

Utility World Name: - Uses a generic utility world name - Prevents detection through world name patterns

Browser Launch Configuration: - Disables automation flags - Uses optimized Chrome arguments - Configures viewport and window settings

Using with Claude Desktop

Make sure you have Claude Desktop installed and updated to the latest version

Open your Claude Desktop configuration file: - MacOS/Linux: `~/Library/Application Support/Claude/claude_desktop_config.json` - Windows: `%APPDATA%\Claude\claude_desktop_config.json`

Add the server configuration as shown in the [Configuration](#configuration) section above.

Restart Claude Desktop

Look for the hammer icon ![](https://mintlify.s3.us-west-1.amazonaws.com/mcp/images/claude-desktop-mcp-hammer-icon.svg) to confirm the tools are available

Available Tools

1. Search Tool

2. View URL Tool

Troubleshooting

Claude Desktop Integration Issues

Check the logs: ```bash # MacOS/Linux tail -n 20 -f ~/Library/Logs/Claude/mcp*.log # Windows type %APPDATA%\Claude\Logs\mcp*.log ```

Common issues: - Server not showing up: Check configuration file syntax and paths - Tool calls failing: Check server logs and restart Claude Desktop - Path issues: Ensure you're using absolute paths

For more detailed troubleshooting, refer to the [MCP debugging guide](https://modelcontextprotocol.io/docs/tools/debugging).

Development

Important Notes

Bot Detection: - The bot detection avoidance features help prevent most common detection methods - However, additional measures like proper proxies and user agents may be needed - Some websites may still detect automation through other means

Performance: - Browser instances are pooled and reused - Idle browsers are automatically cleaned up - Resource limits prevent overloading

License

MIT

Google Search