# Provider endpoints Exact request shapes transcribed from the rsearch service. All calls go through stealthii in the job container. Titles cap at 200 chars, descriptions at 500. A provider returning nothing is a failure for scoring, never an error for the job. ## HTML scraping Brave: `GET https://search.brave.com/search` with `q` and `source=web`. Select `div[data-type="web"][data-pos]`, link from `a[href^="http"]`, title from `div[class*="title"]` (attribute `title` first, else text), description from `div.generic-snippet div.content`, falling back to `div[class*="snippet-description"]`. DuckDuckGo HTML: `POST https://html.duckduckgo.com/html/` with `q`. Parse result blocks for link, title, snippet. Yahoo: `GET https://search.yahoo.com/search` with `p` set to the query. Parse web result blocks. Mojeek: `GET https://www.mojeek.com/search` with `q`. Parse result blocks. Yandex: `GET https://yandex.com/search/` with `text` set to the query. Parse result blocks. Bing (reserve only): `GET https://www.bing.com/search` with `q`. Enable when two or more primary HTML providers fail in a row. ## Public APIs DuckDuckGo instant: `GET https://api.duckduckgo.com/` with `q`, `format=json`, `no_html=1`, `skip_disambig=1`. Read `AbstractText` plus `RelatedTopics` URLs. SearXNG: `GET {instance}/search` with `q`, `format=json`, `categories=general`, 10 second timeout each. Try in order: `https://sx.xo.st/search`, `https://search.lumy.live/search`, `https://search.mectov.my.id/search`. Read `results` items (`title`, `url`, `content`, `engine`, `score`). Qwant: `GET https://api.qwant.com/v3/search/web` with `q`, `count` (max 10), `locale=en_US`, `offset=0`, `device=desktop`, plus `Origin: https://www.qwant.com` and `Referer: https://www.qwant.com/` headers. Walk `data.result.items.mainline`, keep `type=web` sections, read `title`, `url`, `desc`. Jina: `GET https://s.jina.ai/{query}`. Read the returned items. StackExchange: `GET https://api.stackexchange.com/2.3/search` with `order=desc`, `sort=relevance`, `intitle` set to the query, `site=stackoverflow`, `pagesize` max 30. Record score, answer count, answered flag, and up to 5 tags per item. Wikipedia: `GET https://en.wikipedia.org/w/api.php` with `action=query`, `list=search`, `srsearch` set to the query, `format=json`. Build article URLs as `https://en.wikipedia.org/wiki/{title with _ for spaces}`. Wikidata: `GET https://www.wikidata.org/w/api.php` with `action=wbsearchentities`, `search` set to the query, `language=en`, `format=json`. ## Paid last resort Exa: `POST https://api.exa.ai/search` with header `x-api-key: $EXA_API_KEY` and JSON body `{"query": ..., "num_results": ..., "contents": {"text": true}}`. Use only for hard queries after keyless providers come back thin. Never the default, never the first call.