defuss-lexicalsearch

defuss-lexicalsearch

This page

GitHub
v0.2.0No server, no API key, nothing leaves your browser

Lexical search that finds the exact word, symbol or phrase

with BM25 relevance ranking that matches exactly where it matters that forgives typos and fragments in 12 kB of gzipped JavaScript with zero runtime dependencies in your browser and Node.js

defuss-lexicalsearch indexes your documents field by field and answers keyword, phrase, prefix, typo and literal queries while you type. Exact-first ranking puts the literal match on top, and field-weighted BM25 orders everything after it. Indexing and search never touch the network.

Try it now (live) How it works

Search algorithms and engineering brought to you by Aron Homberg (kyr0) (opens in a new tab) · Open Source (MIT (opens in a new tab))

Exact where it matters

Literal modes compare the original strings: e.g. C++ never matches C#, and operator == never matches operator !=. Exact-first ranking keeps a whole-field match above every partial one, whatever the corpus statistics say.

Ranked like a search engine

Field-weighted BM25 with an analyzer per field and derived fields, so one symbol can be a raw keyword and a set of code words at once. With exact-first ranking, results come in strict tiers: literal, keyword, identifier-prefix, term, partial, fuzzy and incomplete.

Forgiving input

Prefix completion, suffix and inside-word matching, Levenshtein or transposition-aware typos, completion despite a typo (Kül finds Kühlschrank), quoted phrases, AND and OR, field restrictions and exclusions. Every expansion is exact: no cap silently drops a match.

Small enough to ship

The core is 12 kB of gzipped JavaScript with zero runtime dependencies, as ESM and CommonJS with TypeScript types. An index serializes to a JSON snapshot and restores with every structure rebuilt.

Private by design

Indexing and search run in your browser tab or Node.js process: documents and queries never leave the device, and no third party ever sees them. Only fromURL fetches, and only when you call it.

Many languages, many kinds of text

Unicode words, code identifiers, raw keywords and whitespace symbols, plus English, German, Spanish and French stemming, stopwords and phonetic codes, and Japanese word segmentation.

Analyze.

Index.

Rank.

Live demo

Finding a needle in a haystack

The complete defuss-shadcn documentation, 410 MDX pages with 8.08 MB of text, downloaded and indexed right here in your browser. Type and the results follow every keystroke. Open any page with ⌘K, or let the demo hide a needle: a word that occurs on exactly one page. In the tab you build an index of your own.

Runtime

Downloading the documentation corpus...

corpus
Corpus · defuss-shadcn documentation
410 MDX pages, gzipped JSON, unpacked by the browser
Index · built in a Web Worker
four fields per page: path as raw keyword, title, description and body as Unicode words

Pages

indexed documents

Vocabulary

distinct terms

Indexed in

download, unpack, index

Last query

postings visited

Worker log 
starting the search worker
Match

Search 410 documentation pages as you type

Try one of these, each showing a different kind of match, or let the demo hide a needle and find it again.

    How it works

    Analyze once, then match and rank every query

    Each field runs its own analyzer into postings with word positions and a vocabulary trie. A query is parsed into words and quoted phrases, analyzed the same way, expanded where you asked for it, and ranked by BM25; with exact-first ranking, the tier decides first.

    Index · browser or Node.js
    Search · browser or Node.js
    01 · Text
    02 · Analyze
    03 · Structures
    04 · Result
    Documents Fields per document stable IDs · JSON metadata
    Analyzer per field Tokenize, normalize, stem keyword · Unicode · code · language
    Per field Postings and trie positions · BM25 statistics
    Index memory · JSON snapshot
    Query words · "phrases" · symbols
    Same analyzers Parse and expand completion · typos · inside words
    Cheapest clause first Match and score phrases by position · BM25
    Top k, ranked tier, then score, then ID
    1. text
    2. terms
    3. structures
    4. lookups
    5. text
    6. terms
    7. top k

    Exact-first tiers, strongest first

    With exact-first ranking, the weakest clause of a result decides its tier. A literal match always outranks a keyword match, however rare the words of the other result are; BM25 orders results within one tier.

    1. literal the whole original field equals the query, whitespace included
    2. keyword an exact analyzed keyword, such as a case-folded identifier
    3. identifier-prefix a keyword starts or ends with the query
    4. term the analyzed words of the query occur exactly
    5. partial a word starts with, ends with or contains the query
    6. fuzzy a word lies within the edit distance, or starts within it while you type
    7. incomplete under OR, at least one query word is missing

    Measured on your device, just now

    Your browser downloaded, unpacked and indexed the documentation above. The search worker then timed seven typical queries: after three warm-up runs, the median of 21 samples, each a batch of searches lasting about 2 ms, since browsers round timers to 0.1 ms. These figures come from this page load; for Node.js, make bench writes the same kind of report.

    to download the 1,368 kB corpus
    to index 410 pages in a Web Worker
    median search for a rare word
    median search with typo tolerance

    Median latency per query type

    Top 12 of 410 pages. The first run of an inside-word search also builds its trigram index once.

    Query typeQueryMedianFirst runResults
    Runs once the index is ready...
    Portrait of Aron Homberg

    Need search tuned to your data?

    Aron Homberg, freelance and independent AI researcher: search and retrieval, embedding models and on-device AI, offers consulting services worldwide.

    Talk to Aron on LinkedIn (opens in a new tab)
    Use it

    Add it to your own app

    Write the five lines yourself, or hand one prompt to your coding agent.

    Package

    Install it from npm with npm install defuss-lexicalsearch, or bun add defuss-lexicalsearch. Then, in Node.js or the browser:

    search.js
    import { LexicalSearch } from "defuss-lexicalsearch";const index = new LexicalSearch({ fields: { title: { weight: 3 }, body: {} } });index.addAll([{ id: "dialog", fields: { title: "Dialog", body: "A modal window for focused tasks." } }]);const hits = index.search("dialgo", { fuzzy: true, transpositions: true });console.log(hits[0].id); // "dialog", despite the typo

    Vibe coding

    Paste this prompt into your coding agent. It links the repository, so the agent works from the current README.

    Integrate defuss-lexicalsearch into this app: https://github.com/kyr0/defuss-lexicalsearch
    Install the npm package defuss-lexicalsearch and read its README first. Index documents with new LexicalSearch({ fields }) and addAll(), then search with search(query, options): { prefix: true, substring: true, fuzzy: true, transpositions: true } for a search box, mode: "exact" for identifiers.
    For code symbols, start from apiSearchPreset, which keeps programming languages apart; for German, English, Spanish or French text, import the language analyzer.
    Index large collections in a Web Worker, and persist them with serialize() and LexicalSearch.fromJSON().
    send

    Exact and ranked search anywhere, for free

    BM25 ranking, literal symbol matching, phrases and typo tolerance run wherever JavaScript runs: no search server, no hosted index, no per-query bill.

    View on GitHub (opens in a new tab) Read the architecture (opens in a new tab)

    MIT licensed · demo corpus credited below

    Documentation pages