Projects / rsearch

git clone https://molodetz.nl/retoor/rsearch.git copy

Files

rsearch

Author: retoor retoor@molodetz.nl

Multi-source search aggregator API that queries multiple search engines and returns unified results without requiring API keys.

Features

• 7 search providers with automatic fallback
• No API keys required (HTML scraping + public APIs)
• Async architecture for performance
• Unified JSON response format
• Fixed provider ordering by result quality
• Comprehensive integration tests

Search Providers

| Provider | Type | Description | |----------|------|-------------| | Brave | Scraping | High quality web results | | DuckDuckGo HTML | Scraping | Reliable lightweight version | | Bing | Scraping | Microsoft search engine | | Mojeek | Scraping | Independent search index | | DuckDuckGo | API | Instant answers | | Wikipedia | API | Encyclopedia reference | | Wikidata | API | Structured knowledge base |

Installation

Install dependencies:

pip install -r requirements.txt

Install as package (development mode):

pip install -e .

Install with test dependencies:

pip install -e ".[test]"

Or using make:

make dev

Usage

Run as module:

python -m rsearch

Or after installation:

rsearch

Or using make:

make run

Command Line Options

usage: rsearch [-h] [-H HOST] [-p PORT] [-l {DEBUG,INFO,WARNING,ERROR}] [-v]

options:
 -h, --help show help message
 -H, --host HOST Host to bind to (default: 0.0.0.0)
 -p, --port PORT Port to listen on (default: 8080)
 -l, --log-level Log level: DEBUG, INFO, WARNING, ERROR (default: INFO)
 -v, --version show version number

Examples:

rsearch --port 9000 # Run on port 9000
rsearch --host 127.0.0.1 --port 3000 # Bind to localhost:3000
rsearch --log-level DEBUG # Enable debug logging

Testing

Run integration tests:

make test

Or directly with pytest:

pytest tests/test_providers.py -v

Quick API test (requires running server):

make test-quick

API Endpoints

Search

GET /search?query=&count=

Parameters:

• query : Search term (required)
• count : Number of results (default: 10, max: 100)

Response:

{
 "query": "python",
 "source": "brave",
 "count": 3,
 "results": [
 {
 "title": "Welcome to Python.org",
 "url": "https://www.python.org/",
 "description": "The official home of the Python Programming Language",
 "source": "brave",
 "extra": {}
 }
 ],
 "timestamp": "2024-01-01T12:00:00.000000Z",
 "success": true,
 "error": null
}

Health Check

GET /health

Response:

{
 "status": "ok",
 "services": ["brave", "duckduckgo_html", "bing", "mojeek", "duckduckgo", "wikipedia", "wikidata"],
 "timestamp": "2024-01-01T12:00:00.000000Z"
}

Project Structure

rsearch/
├── rsearch/
│ ├── __init__.py
│ ├── __main__.py
│ └── app.py
├── tests/
│ ├── __init__.py
│ └── test_providers.py
├── requirements.txt
├── pyproject.toml
├── Makefile
└── README.md

License

MIT

Files