01 / method
A benchmark is only as honest as its setup.
querySelection
Stratified sample across intent, language, freshness and tail length. The production corpus and query list are still being locked.
relevance
0 = not relevant, 1 = partially useful, 2 = directly answers the need.
raters
3 independent raters per query
blind
Engine names hidden during judging; result order preserved.
testDate
2026-10-27 (placeholder run)
engines
LYNX v0.1.0, Google, Bing, Brave Search, Mojeek and DuckDuckGo; public versions recorded at run time.
hardware
LYNX: 8 vCPU, 32 GiB RAM, NVMe SSD, local deployment.
02 / relevance quality
LYNX against the field.
Quality metrics
Higher is better · placeholder comparison
Zero-result rate
Lower is better
| Engine | NDCG@10 | Precision@10 | MRR | Zero-result rate | Independence |
|---|---|---|---|---|---|
| LYNX | 0.966 | 0.84 | 1 | 4.2% | Independent |
| 0.92 | 0.81 | 0.89 | 1.8% | Commercial | |
| Bing | 0.88 | 0.77 | 0.84 | 3.1% | Commercial |
| Brave | 0.82 | 0.71 | 0.76 | 5.4% | Independent |
| Mojeek | 0.74 | 0.65 | 0.68 | 9.7% | Independent |
| DuckDuckGo | 0.86 | 0.75 | 0.81 | 2.8% | Commercial |
03 / query types
The average hides the edges.
Navigational · NDCG@10
04 / index & crawler
The honest weakness is coverage.
LYNX has a small index compared with commercial engines. That limits recall, especially for fresh, local and non-English queries. This benchmark does not turn a small corpus into a claim of parity. Robots compliance is an observable property of the crawler.
05 / performance
Fast on the machine it was tested on.
Hardware: 8 vCPU, 32 GiB RAM, NVMe SSD, local deployment. These are small-corpus results, not a promise about production scale.
06 / privacy audit
Privacy is a set of observable properties.
| Engine | Cookies | Third-party requests | Trackers | Query logging | Note |
|---|---|---|---|---|---|
| LYNX | 0 | 0 | 0 | Documented retention | Not anonymous; not zero logs. |
| Varies | Detected | Detected | Policy-dependent | Measured values require a locked browser run. | |
| Bing | Varies | Detected | Detected | Policy-dependent | Measured values require a locked browser run. |
| Brave | Varies | Low | Low | Policy-dependent | Independent peer. |
| Mojeek | Varies | Low | Low | Policy-dependent | Independent peer. |
| DuckDuckGo | Varies | Detected | Detected | Policy-dependent | Measured values require a locked browser run. |
LYNX does not claim to be anonymous or “zero logs”. Its commitment is narrower and documented: no user profile, no selling or sharing query data, and explicit retention rules.
07 / ranking transparency
Ask why a result ranked where it did.
explain=true · ranking-config v0.1.0
LYNX result signal vector
Signal profile
Illustrative explain output from the placeholder record.
08 / Astra answers
Citation verification is not truth verification.
Astra is off by default. In v0.1, verification is sentence-level: a citation must resolve to a retrieved source. That does not prove the source is true, complete or unbiased, and it is not comparable to a claim of general factual accuracy.
09 / query explorer
Open the test set, one query at a time.
Selected query
how does bm25 ranking work
Top-10 result payloads will be linked here when the raw comparison export is published.
11 / reproduce it
Run the same evaluation locally.
Lock the toolchain
Use the repository’s pinned Rust and Node versions. Record engine versions and hardware before a run.
Run the judged set
Keep query selection, raters and relevance instructions unchanged.
Publish raw output
Attach CSV/JSON, config, date, corpus snapshot and failure notes.
12 / limitations & fairness
What this page does not claim.
The index is small compared with commercial engines.
The sample size and rater pool can move scores.
Search engines change; comparisons expire.
Scores remain provisional until the evaluation is complete.
13 / roadmap & changelog
A benchmark that can change its mind.
v0.1.0 release rehearsal
56 test suites; small-corpus quality and latency run.
Lock public comparison corpus
Publish raw queries, rater guide and engine snapshots.
Scale and distribution
Multi-shard index, larger crawl and regional replication.