Language: English | 日本語 | 한국어 | 简体中文 | 繁體中文

Packaged for Linux (RPM, DEB, tarball), Windows (zip), and macOS (dmg).
Interfaces are stable; breaking changes are avoided where possible.

DocuBrowse turns a messy pile of documents into something you can actually search. Point it at your files — PDFs, ebooks, Word docs, notes, whatever — and it builds a smart index that understands not just keywords, but meaning. Ask for "that contract about the lease renewal" and find it even if those exact words never appear. Click any result for an instant AI summary before you even open the file. PII aware and works with multiple document directories.

DocuBrowse runs entirely on your own machine using local AI models — no internet connection required, no accounts, no API keys, and no per-query costs eating into a token budget. Your data. Your AI.

Under the hood: SQLite FTS5 keyword search plus AI-powered semantic similarity and synopsis generation (Ollama + nomic-embed-text + dolphin3). Supports multiple document and source code types.

Docker (experimental — not recommended yet): A containerized deployment is being tried out on the docker-experiment branch. Because of Docker/OS limitations and DocuBrowse's security model, a fully working model has not been achieved yet — notably, opening a document in your desktop app does not work from a browser-based container (the server is headless and the browser is sandboxed). It's available on that branch for anyone who wants to experiment, but is not recommended for use. See docker/README.md on the branch.

↑ Top

• Keyword Search — fast full-text search via SQLite FTS5 (title, author, subject, tags, snippet)
• Semantic Search — AI-powered similarity via Ollama embeddings (nomic-embed-text:latest). Semantic search identifies which documents are relevant to your query; Deep Links (below) then pinpoints where inside a document the match is.
• Hybrid Mode (default, "Both") — keyword and semantic merged: each document takes the stronger of its two scores (with a small boost when both hit), and must clear a keyword hit or a semantic floor to appear. See Search Tips and Tricks.

• From any keyword or semantic result, click Deep Links to find the matching passages inside that document — on demand, no reindex, no schema change.
• Each passage shows a short sample and a location label (page, line, or section); clicking one opens the passage with the matched text highlighted.
• Mode follows the search: a semantic search finds passages by meaning; a keyword (or hybrid) search finds them by term.
• Prose formats: PDF, TXT, HTML, Markdown, DOCX, RTF, ODT, e-books (EPUB, MOBI, AZW3), DjVu, plus the SGML/XML family (XHTML, XML, DocBook, RSS/Atom, OPML) and other text/markup formats (reST, AsciiDoc, LaTeX, config files, JSON/YAML, email, source code). Non-prose (spreadsheets, presentations, diagrams) fall back to opening in their reader.
• Deep Links may return "Not Found" if it does it's likely because the document type has not yet been coded into the mix. It picked up the keyword already assigned to the document but is as of yet still unable to actually read the document and create the deep links. I'm working on expanding this as much as possible and where it gives value (deep search in a PDF that is a picture of the Mona Lisa will likely never get searched this way)

• Click any document title for a Kindle-style book-jacket synopsis, generated on demand via Ollama (dolphin3:latest) and cached in the database after first generation. The first synopsis for a given document may take a while on minimal hardware (CPU-only, no GPU) — subsequent requests are instant since the result is cached. Semantic search embeddings are produced by a second local model (nomic-embed-text:latest)

• Formats: PDF, DOCX, PPTX, XLSX, ODT, ODS, ODP, OTT/OTS/OTP (ODF templates), VSDX/VSDM, VSD/VSS/VST (legacy Visio), VDX (Visio 2003 XML), draw.io/diagrams.net (.drawio/.dio), PlantUML (.puml/.plantuml), Mermaid (.mmd), SGML/XML family (.xml/.xhtml/.sgml/.sgm), DocBook (.docbook/.dbk), SVG, feeds (.rss/.atom/.opml), reStructuredText (.rst), AsciiDoc (.adoc/.asciidoc), LaTeX (.tex/.latex), Email (.eml), RTF (.rtf), CSV / TSV, EPUB, MOBI, AZW3, AZW, DjVu (.djvu/.djv), HTML, TXT, Markdown, plus config-ish plain text (.ini/.conf/.cfg/.log/.lst)
• PDF intelligence: pdfplumber (preferred) with pypdf fallback for bloated-object files; layout=False retry for complex layouts; scanned (image-only) PDFs detected and routed to ocr_list_pdfs.txt
• Word documents: python-docx extracts paragraphs, tables, and core properties (title, author, subject)
• Presentations: python-pptx extracts slide text, notes, and core properties
• Spreadsheets: openpyxl extracts cell values and sheet names
• OpenDocument: ODF text documents (.odt), spreadsheets (.ods), and presentations (.odp), plus their template variants (.ott/.ots/.otp) — paragraphs, headings, lists, tables, cell values, and slide text extracted via Python stdlib (zipfile + xml.etree.ElementTree). Templates route to the matching extractor by mimetype prefix. Metadata (title, author, subject, description, keywords) read from meta.xml. No extra dependency required.
• Visio and diagrams:
  • Modern Visio (.vsdx/.vsdm) — OOXML zip parsed with the stdlib; shape text, page names, and core properties (title/author/subject/keywords) extracted with no third-party dependency.
  • Legacy Visio (.vsd/.vss/.vst) — binary compound document; requires the optional vsd2xml tool from libvisio-tools (sudo dnf install libvisio-tools / sudo apt install libvisio-tools). Without it, legacy files are indexed metadata-only (filename as title, no body) and the path is appended to visio_legacy_missing.txt so a rescan after install picks them up.
  • draw.io / diagrams.net (.drawio/.dio) — both plain mxfile XML and compressed (deflate + base64 + URL-encoded) diagrams are supported; every mxCell label and object label is extracted, plus page names. Stdlib-only.
  • Text-based diagrams — PlantUML (.puml/.plantuml) and Mermaid (.mmd) source is indexed as plain text and tagged diagram.
• Markup family (all stdlib, no extra dependency):
  • XML/SGML (.xml/.xhtml/.sgml/.sgm), DocBook (.docbook/.dbk), SVG (also tagged diagram), and feeds (.rss/.atom/.opml) are tag-stripped — DOCTYPE, comments, CDATA wrappers, <script> and <style> blocks are removed and remaining tags dropped; entities are unescaped. Title/author/subject are sniffed from well-known elements (<title>, <dc:title>, <author>, <dc:creator>) before stripping, so DocBook, Atom, RSS, and SVG all surface useful metadata.
  • reStructuredText (.rst), AsciiDoc (.adoc/.asciidoc), and LaTeX (.tex/.latex) are indexed verbatim (all markup becomes searchable text) with a per-format title heuristic: reST underline-style titles, AsciiDoc level-0 = Title, LaTeX \title{} / \section{}. LaTeX \author{} is captured and line-% comments are stripped.
  • Every markup file is tagged markup for browse/filter convenience. .vdx (Visio 2003 XML) also gets a diagram tag.
• Email, RTF, tabular, and config-ish text:
  • Email (.eml) — parsed with the stdlib email package. Subject becomes the title, From becomes the author, To/Cc/Date and the plain-text body (or HTML body tag-stripped) become searchable content. Attachment filenames are appended so name-searches still hit.
  • RTF (.rtf) — decoded via striprtf (pure Python, MIT). Without it the file is indexed metadata-only and the path is appended to rtf_missing_striprtf.txt; installing striprtf and rescanning picks up the body.
  • CSV / TSV — first ~500 rows are indexed with pipe-delimited row rendering; the header row lands in the description field so column names weigh in keyword search. Stdlib-only; delimiter auto-detected.
  • Config-ish plain text — .ini, .conf, .cfg, .log, .lst route through the standard text path with the same 200 KB read cap.
• E-books: ebooklib for EPUB; mobi package + Calibre ebook-convert fallback for MOBI/AZW3 text extraction; DRM-encrypted AZW files indexed with metadata only (title/author visible, body not searchable)
• DjVu: .djvu/.djv text layer extracted via DjVuLibre (djvutxt/djvused, external non-pip tool). Without it, DjVu files are indexed metadata-only and paths appended to djvu_missing_djvulibre.txt; install DjVuLibre and rescan for body text. Image-only DjVu with no text layer is indexed metadata-only (OCR not performed, same as scanned PDFs)
• Platform: all extraction libraries are pure Python or have wheels for both x86_64 and ARM64. 32-bit systems are not supported.
• Metadata: title, author, subject extracted from document metadata fields; auto-generated tags from directory structure and content keywords
• PII protection: post-ingest scanner detects SSN, credit card, bank routing/account number, DOB, MRN, driver license, passport patterns; removes matching documents and permanently blacklists them

• Dark/Light theme toggle
• Paginated results (50 docs/page) with Back/Next controls
• Alphabetic index bar (A–Z, 0–9) for quick navigation, with state preserved across page loads; Home button to reset
• Tag cloud for filtering by topic
• Relevance score badges (0–100%) on every result
• Open button on each result card — launches the file in your default app via xdg-open
• Click document title for an AI synopsis; 📋 copies path to clipboard; 🗑 deletes file from disk and index (with confirmation)
• Moved/deleted documents: clicking a doc whose file no longer exists shows a dismissable modal (and removes it from the index on dismiss) if its filesystem is mounted, or a toast (no index change) if the filesystem can't be verified (e.g. unmounted drive)

• General panel: document directory (with live directory browser), any number of additional scan directories (added/removed under the same panel — automatically included in scan/rescan, no extra command needed), working directory, and port
• Ignored Directories panel: browse to add a directory to ignore_dirs.txt, with a confirmation prompt before purging already-indexed documents under it, and a confirmation before removing an entry

• Search latency: <150ms typical
• Parallel PDF extraction with ProcessPoolExecutor (physical-core-aware worker count)
• Memory-safe: kernel-enforced RLIMIT_AS (6 GB/worker) + pause/resume on free-RAM threshold

↑ Top

DocuBrowse has three search modes (toggle top-right): Keyword, Semantic, and Both (the default hybrid). Search runs when you press Enter — clearing the box shows all documents again. The same rules apply inside a document when you click Deep Links; Deep Links follows whichever mode your search used.

• Keyword — literal text via SQLite FTS5. Each word is prefix-matched and the words are OR-ed, so budget report finds documents containing budget… or report… (not necessarily both). Ranked by where the hit lands — matches in the title and author outrank matches in the body or tags. Best when you know a word, name, or code that's actually in the document.
• Semantic — meaning, via AI embeddings. Finds documents that are about your query even when they don't contain your exact words. Ranked by conceptual closeness. Best for "documents about X" when you don't know the exact wording.
• Both (default) — runs both and keeps, per document, the stronger of the two scores (with a small bump when a document scores on both). A document has to earn a keyword hit or a meaningful semantic score to show up, so weak semantic noise doesn't bury solid keyword matches.

Wrap words in "..." (or '...') to require that exact consecutive phrase (matching is case-insensitive). "machine learning" matches only documents where those two words appear together in that order, not documents that merely mention machine and learning separately. This works in every mode — in Semantic/Both it acts as a presence filter first, then ranks the phrase-containing set by meaning. You can mix forms: golang "import fmt" means the phrase "import fmt" OR the loose word golang.

Concretely, the quotes decide whether the small words count. "man in the middle" searches for that whole phrase, every word included. man in the middle (no quotes) in Semantic mode drops the article the before matching — it searches on man, in, middle — because only articles (a/an/the) and conjunctions (and/or/but/nor/for/so/yet) are treated as filler. Quote the phrase when the exact wording matters.

• Know the exact word, name, or error code → Keyword.
• Looking for documents about a topic → Semantic or Both.
• Want an exact phrase → quote it (any mode).
• Searching a name → Keyword beats Semantic (names embed fuzzily).
• Semantic can surface a document for a concept it never literally names — e.g. an MIT-licensed file shows up for freedom because the license text reads that way. That's semantic working as intended, not a bug.

↑ Top

Click any thumbnail to view full size.

Dark Mode	Light Mode

Settings	AI Synopsis

From any keyword or semantic result, Deep Links finds the matching passages inside that document, jumps to one, and highlights the matched text.

The three stages — the Deep Links button on a result, the modal listing the matching passages, and the chosen passage highlighted:

Semantic search results	Matching passages	Highlighted passage

Keyword search results	Matching passages	Highlighted passage

Deep Links renders the document's extracted text with the matched snippet in yellow, labelled by its location (page / line / section). Mode follows the search: semantic search opens semantic passages, keyword opens keyword passages.

Settings is a standalone page at /settings (opened in a new tab via the gear icon). The General panel covers the document directory (with live directory browser, plus any number of additional scan directories), working directory, and port; the Ignored Directories panel manages scan exclusions, each with a directory browser, add/clear controls, and confirmation before removal.

↑ Top

Minimum: 8 GB RAM, x86_64 or ARM64 CPU, 2 GB free disk (plus space for your documents and the index). Works without a GPU — synopsis generation will be slower but functional. 32-bit systems are not supported (Ollama does not ship 32-bit builds).

Recommended: 12 GB RAM, 4 GB+ vRAM (NVIDIA or Apple Silicon). GPU acceleration significantly speeds up synopsis generation and embedding.

• Python 3.9+
• pdfplumber, pypdf — PDF extraction
• python-docx — Word documents
• python-pptx — PowerPoint presentations
• openpyxl — Excel spreadsheets
• ebooklib, beautifulsoup4, mobi — E-books
• striprtf — RTF text extraction (pure Python; without it .rtf is indexed metadata-only)
• psutil — cross-platform process and hardware detection
• Calibre — E-book metadata and conversion (required for MOBI/AZW3/AZW indexing): sudo dnf install calibre or sudo apt install calibre
• libvisio-tools — optional; only needed to extract body text from legacy binary Visio (.vsd/.vss/.vst). Without it, those files are still indexed metadata-only. sudo dnf install libvisio-tools or sudo apt install libvisio-tools
• DjVuLibre — optional; only needed to extract text from DjVu (.djvu/.djv). Without it, those files are still indexed metadata-only. sudo dnf install djvulibre / sudo apt install djvulibre-bin / brew install djvulibre / choco install djvu-libre
• Ollama — installed automatically by docubrowser start if missing
• Modern browser (Chrome, Firefox, Safari, Edge)

See INSTALL.md for a full step-by-step guide.

DocuBrowse ships as RPM, DEB, tarball, Windows zip, and macOS dmg packages. Download the appropriate package from the Releases page.

# Fedora / RHEL
sudo dnf install ./docubrowser-foss-<VERSION>-<RELEASE>.noarch.rpm

# Debian / Ubuntu / Mint
sudo apt install ./docubrowser-foss_<VERSION>-<RELEASE>_all.deb

# Any Linux (tarball)
tar xzf docubrowser-foss-<VERSION>-<RELEASE>.tar.gz
cd docubrowser-foss-<VERSION>-<RELEASE>
sudo ./install.sh

Windows: Extract the zip, then double-click Install.bat. Requires Python 3.9+ and Ollama to be pre-installed. Installs to %USERPROFILE%\DocuBrowse with a Start Menu shortcut — no admin required. You may need to log out and back in for the shortcut to appear.

macOS: Open the dmg, then double-click Install.command (right-click → Open the first time — the scripts are unsigned). Requires Python 3.9+. Installs to ~/Applications/DocuBrowse/ with a Python virtualenv, CLI wrappers at /usr/local/bin/docubrowser and /usr/local/bin/docuback (sudo prompted; falls back to ~/bin/ if declined), and a DocuBrowse.app launcher that starts the server and opens the web UI in Terminal.

All Linux methods install to /opt/docubrowser/ with a Python virtualenv, CLI wrappers at /usr/bin/docubrowser and /usr/bin/docuback, a desktop menu entry under Office, and all Python dependencies from requirements.txt.

Once installed, the CLI is the docubrowser command — drop the ./ and .py from every example below. For example, docubrowser start and docubrowser rescan. The ./docubrowser.py <cmd> form shown throughout the rest of this README is the dev / cloned-repo path (running directly out of a checkout).

To uninstall: sudo dnf remove docubrowser-foss (RPM), sudo apt remove docubrowser-foss (DEB), sudo ./uninstall.sh (tarball), double-click Uninstall.bat (Windows), or double-click Uninstall.command (macOS — on the dmg or in ~/Applications/DocuBrowse/).

cd /path/to/DocuBrowse

# Scan and index your documents
./docubrowser.py rescan

# Start the server
./docubrowser.py start

# Open the UI
./docubrowser.py open

On an installed system, use the docubrowser command instead, e.g. docubrowser rescan / docubrowser start / docubrowser open.

docubrowser.py start automatically verifies Ollama is installed, running, and has both required models — nomic-embed-text:latest (embeddings) and dolphin3:latest (synopsis generation) — installing/starting/pulling as needed.

↑ Top

Usage: docubrowser.py <command> [options]

--db PATH      SQLite database path (overrides config)
--port PORT    Server port (overrides config)
--config FILE  Config file path

# Server management
./docubrowser.py start
./docubrowser.py start --port 9000
./docubrowser.py status
./docubrowser.py stop
./docubrowser.py stopall

# Scanning (scan and rescan are identical)
./docubrowser.py scan                          # scan, index, and embed all types
./docubrowser.py scan pdf                      # PDFs only
./docubrowser.py scan pdf txt                  # PDFs and plain text
./docubrowser.py scan --limit 100              # first 100 unindexed files only
./docubrowser.py scan --workers 4              # 4 extraction workers
./docubrowser.py scan --no-embed               # scan without embedding step
./docubrowser.py scan --doc-dir /data/docs

# Single-file indexing (useful for retrying blacklisted files)
./docubrowser.py scan-file --file /path/to/document.pdf
./docubrowser.py scan-file --file /path/with spaces/doc.pdf   # no quoting needed
./docubrowser.py scan-file --file /path/to/doc.pdf --no-embed

# Reporting and maintenance
./docubrowser.py report                         # file-type breakdown, no DB changes
./docubrowser.py embed                          # embed any un-embedded docs
./docubrowser.py purge --dry-run               # preview PII matches (safe)
./docubrowser.py purge                         # remove PII documents (prompts)

# Excluding directories from scanning
./docubrowser.py ignore add ~/Documents/myWorkDocs   # exclude + purge indexed docs under it
./docubrowser.py ignore list                                  # show ignored directories
./docubrowser.py ignore remove ~/Documents/myWorkDocs # re-allow (rescan to re-index)

# Duplicate detection and cleanup
./docubrowser.py duplist                       # find exact SHA256 duplicates
./docubrowser.py duplist --near-dups           # also find near-duplicates (cosine ≥97%)
./docubrowser.py duplist --near-dups --threshold 0.95
./docubrowser.py dupclean                      # interactive Keep A/Keep B/Keep Both TUI
./docubrowser.py dupclean --near-dups          # include near-duplicates in cleanup

# Cleaning up moved/deleted documents (opt-in, not run automatically)
./docubrowser.py scan-missing --dry-run        # report counts only, no DB changes
./docubrowser.py scan-missing                  # delete rows for genuinely-missing files

# Removing dotfiles indexed by an older version (standalone one-off tool)
python3 purge_dotfiles.py                       # dry-run: page the full list, no changes
python3 purge_dotfiles.py --apply               # delete them (cascade-safe)
python3 purge_dotfiles.py --db /path/du.db --roots /docs /extra --apply

purge_dotfiles.py is a transitional migration tool, run directly (not a docubrowser subcommand). The scanner already skips dotfiles going forward (D-6); this purges rows left behind by older versions. It uses the same roots-aware hidden-path check as the scanner (a deliberately-scanned dot-directory root is exempt) and the cascade-safe delete path, so FTS rows, tags, and embeddings are cleaned up correctly. Dry-run by default; --apply to delete; --db / --roots to override the defaults.

Types: pdf  txt  md  html  (default: all supported)

Examples:
  scan pdf               PDFs only
  scan pdf txt           PDFs and plain text
  scan                   all supported types (prompts if unfiltered)

scan-file is designed for retrying individual problem files:

• Removes the file from scan_blacklist.txt if listed (explicit retry)
• Refuses files in pii_blacklist.txt (permanent PII block)
• Detects scanned (image-only) PDFs → adds to ocr_list_pdfs.txt
• Paths with spaces work without quoting: --file accepts multiple tokens and rejoins them

↑ Top

DocuBrowse reads the first config file it finds:

1. /etc/docubrowse.config (system-wide)
2. ./docubrowse.config (next to docubrowser.py)

If neither exists, built-in defaults apply — except doc_dir, which has no default. Until a document directory is configured (via the Settings gear icon in the web UI, or by setting doc_dir in docubrowse.config), the web UI shows a banner prompting you to configure one, and CLI commands that need a document directory (rescan, report, scan) exit with an error explaining how to set it.

# docubrowse.config
doc_dir      = ~/Documents
db_path      = /home/user/DocuBrowse/du-docs.db
port         = 8643
work_dir     = /home/user/DocuBrowse
lang         = en

Environment variables override the matching config-file keys (useful for containers). CLI flags still win when provided.

doc_search.py also accepts DOCUBROWSE_DB / DOCUBROWSE_PORT when argv is omitted. OLLAMA_HOST is read by doc_search.py, embed_docs.py, and ensure_ollama.py — set it when Ollama runs on another host.

DocuBrowse FOSS is loopback-only: the server binds 127.0.0.1 and accepts connections only from the local machine. There is no remote-access mode and no user login — it is a single-user local application. (Remote / multi-user deployment is an Enterprise feature and is not part of FOSS.)

↑ Top

A DocuBrowse install serves one language at a time — the interface, the FTS tokenizer, the embedding model, and the synopsis model are all selected for that one language. This is a per-install setting, not a per-document one: DocuBrowse assumes the configured document directory is overwhelmingly one language, and does not support mixed-language corpora or per-document language detection.

Currently supported: English (en, default), Japanese (ja), Korean (ko), Simplified Chinese (zh), and Traditional Chinese (zh-hant). Japanese, Korean, and both Chinese variants use the bge-m3 multilingual embedding model (English uses nomic-embed-text); Korean's synopsis model is exaone3.5, and both Chinese variants use ornith-1.5:9b (Traditional is driven by a prompt that forces 繁體/正體字 output). For keyword (FTS5) search, CJK text (Japanese, Korean, and Chinese) is indexed with the standard unicode61 tokenizer plus app-side character-bigram segmentation (see cjk.py) rather than FTS5's built-in trigram tokenizer — Japanese has no spaces between words, Korean is agglutinative, and Chinese has no word boundaries at all, so both index-time and query-time text are pre-split into overlapping 2-character bigrams before FTS5 ever sees them. The same segmenter covers Simplified and Traditional characters (both live in the CJK Unified range), so no separate handling is needed. This is not a morphological segmenter (no MeCab/jieba/konlpy dependency); it's a zero-dependency stand-in that trades linguistic correctness for indexing every 2+ character CJK substring reliably. A single CJK character will not exact-match in Keyword mode (by design — 1-character matches are too noisy), but still surfaces via Both/semantic search.

Korean alphabet index bar: Korean is a true alphabet (unlike Japanese, whose kana and kanji have no first-letter ordering), so the document-list index bar is enabled for Korean and shows the 14 basic leading consonants (choseong): ㄱ ㄴ ㄷ ㄹ ㅁ ㅂ ㅅ ㅇ ㅈ ㅊ ㅋ ㅌ ㅍ ㅎ. Each button filters documents whose title begins with a syllable led by that consonant (tense consonants fold into their base, e.g. ㄲ→ㄱ). English uses the A–Z/0–9 bar; the bar is hidden for Japanese and for Chinese (both Simplified and Traditional) — Chinese is not alphabetic and has no first-letter ordering, the same reason it is hidden for Japanese.

Choosing a language:

• At install time — install.sh asks for the language on a fresh install. Answer non-interactively with DOCUBROWSE_LANG=en, =ja, =ko, =zh, or =zh-hant. Upgrades never re-prompt — an existing lang value in docubrowse.config is always preserved. (The platform installers — Windows/macOS/RPM/DEB — default new installs to lang = en; change it afterward via Settings.)
• At start time — docubrowser ko start (or docubrowser start ja) starts the server in that language; the code may appear in either position. The choice is written to docubrowse.config, so it persists for later plain docubrowser start runs until you pick another.
• After install — open the web UI, click the Settings (gear) icon, and use the language dropdown in the General panel (calls POST /api/language). Switching to a language that uses a different embedder or FTS tokenizer (e.g. English ↔ Japanese/Korean) shows a warning: existing documents keep their old embeddings and FTS index until you re-run docubrowser rescan (or embed_docs.py) to rebuild them for the new language — search quality is degraded for previously-indexed documents until then. (Japanese ↔ Korean share the bge-m3 embedder and the same tokenizer, so no rebuild is required between those two.)

Not yet supported: mixed-language or per-document language corpora, languages beyond English/Japanese/Korean/Chinese (the LANG_MODELS table in lang_models.py plus a locales/<code>.json file is the whole mechanism, so adding one is a data change, not a code change), a kana/reading-based (or pinyin) index bar for Japanese/Chinese document lists (neither has first-letter ordering; Korean uses the choseong bar described above), and Japanese "My Number" PII detection (only US PII patterns are implemented today).

↑ Top

┌─────────────────────────────────────┐
│  docubrowser.py  (CLI entry point)  │
│  ensure_ollama.py (prereq check)    │
└──────────┬──────────────────────────┘
           │ subprocess / direct call
           ↓
┌──────────────────────┐  ┌─────────────────────────────────┐
│  scan_docs.py        │  │  doc_search.py  (HTTP :8643)    │
│  ProcessPoolExecutor │  │  GET /  /api/search             │
│  pdf_extractor.py    │  │  GET /api/stats  /api/tags      │
│  embed_docs.py       │  │  GET /api/open  /api/config     │
│                      │  │  GET /api/delete /api/synopsis  │
│                      │  │  GET /api/deep-links            │
└──────────┬───────────┘  └──────────────┬──────────────────┘
           │                             │
           └──────────┬──────────────────┘
                      ↓
        ┌──────────────────────────────────────────────────┐
        │  du-docs.db  (SQLite FTS5)                       │
        │  Ollama (nomic-embed-text + dolphin3)            │
        └──────────────────────────────────────────────────┘

↑ Top

The HTTP API is a private implementation detail of the bundled local UI — not a public or remote API. It is reachable only from 127.0.0.1, has no authentication, and offers no stability guarantee. Do not build external, remote, or third-party clients against it; the endpoints below are documented for transparency and local debugging only. Remote / multi-user access is an Enterprise feature, not part of FOSS.

Base URL: http://localhost:8643

🔒 = state-changing or filesystem-exposing; requires the per-process X-CSRF-Token header and a loopback Origin/Referer. The token is injected into the served HTML, so only the first-party UI can call these. See Security. All requests are also rejected unless the Host header is localhost/127.0.0.1/[::1] (DNS-rebinding protection).

If the indexed path no longer exists on disk, /api/open returns one of:

{"ok": false, "error": "missing", "message": "..."}
{"ok": false, "error": "unmounted", "message": "..."}

missing means the file's filesystem is mounted and the file is genuinely gone — the UI shows a dismissable modal and deletes the document from the index (and disk-adjacent DB rows) when dismissed. unmounted means the path's filesystem can't currently be verified (likely an unmounted drive) — the UI shows a toast and makes no DB changes. See scan-missing for the equivalent batch cleanup.

GET /api/search?q=QUERY&offset=0&mode=both

{
  "documents": [
    {
      "id": 1,
      "name": "doc.pdf",
      "title": "Document Title",
      "author": "Jane Smith",
      "subject": "Cloud Security",
      "description": "First 500 chars of content...",
      "path": "~/Documents/doc.pdf",
      "tags": ["pdf", "security", "cloud"],
      "modified_at": "2026-06-07T14:30:00",
      "score": 0.95,
      "fts_score": 0.8,
      "sem_score": 0.98
    }
  ],
  "query": "cloud security",
  "count": 50,
  "total": 312,
  "offset": 0,
  "has_more": true,
  "mode": "both"
}

# Read endpoints are plain GETs:
curl "http://localhost:8643/api/stats"
curl "http://localhost:8643/api/search?q=kubernetes&mode=both"
curl "http://localhost:8643/api/search?q=&offset=50"

Mutating endpoints (/api/delete, /api/open, the POST config/dir routes) and /api/browse require the per-process CSRF token and a loopback origin, so they aren't easily exercised with a bare curl — drive them from the UI, or pass -X POST -H "X-CSRF-Token: <token>" -H "Origin: http://localhost:8643" where <token> is read from the <meta name="csrf-token"> tag in /.

↑ Top

A non-empty query is scored two ways and merged; metadata is then fetched only for the requested page (the whole corpus is no longer loaded per request).

final_score = 0.3 × keyword_score + 0.7 × semantic_score   (mode=both)

• Backed by the SQLite FTS5 index via MATCH + bm25() — not a Python substring scan.
• Query tokens are quoted and prefix-matched ("tok"*), OR-combined, so arbitrary input (operators, quotes, C++, &) can't break the query.
• Per-column BM25 weights echo the old field priorities (name 6, title 8, author 7, subject 5, description 3, content_snippet 3, tags 4); the result is normalized to 0–1.
• Orphan doc_fts rowids (contentless FTS has no FK cascade) are pruned against the live document set so they can't inflate totals.

• Cosine similarity between the query embedding and each document embedding.
• Computed against an in-process, L2-normalized embedding matrix (one vectorized NumPy matrix-vector product), cached and invalidated when the embeddings table changes — instead of reloading every BLOB per request.
• Range 0.0–1.0; minimum threshold (semantic-only mode): 0.30.
• Embeddings: 768-dimensional float32 vectors (nomic-embed-text:latest).

↑ Top

DocuBrowse binds to localhost and is intended for single-user local use, but it is hardened so a malicious web page you happen to visit can't reach it:

• Host-header allow-list — every request is rejected unless Host is localhost/127.0.0.1/[::1] (optionally with the serving port). Defeats DNS-rebinding against the localhost-bound server.
• CSRF tokens on mutations — /api/delete and /api/open are POST-only; they plus the POST config/dir routes and the filesystem-exposing /api/browse require a per-process X-CSRF-Token (injected into the served HTML, so only the first-party UI has it) and a loopback Origin/Referer.
• No stored-data injection — document fields are escaped for HTML and card actions use data-* attributes + delegated listeners (no inline onclick built from document data), closing a stored-XSS vector.
• PII purge validates structurally before deleting: SSNs against SSA allocation rules, credit cards by length + issuer prefix + Luhn, bank routing numbers by ABA checksum + Federal Reserve prefix — so it both catches more real PII and avoids deleting docs over incidental number groups.

The server is loopback-only: it binds 127.0.0.1 so the port is not exposed on any external interface, and verify_request() drops any connection whose source IP is outside 127.0.0.0/8 (or ::1) at the TCP-accept level — before a byte of HTTP is read. No authentication is needed because only the local user can reach the server, and access control does not rely on the host firewall.

There is no remote-access mode in FOSS. The HTTP API is a private implementation detail of the bundled local UI, not a public or remote API — do not build external or remote clients against it. Remote / multi-user deployment is an Enterprise feature.

↑ Top

DocuBrowse/
├── docubrowser.py          # CLI entry point (all commands)
├── ensure_ollama.py        # Ollama prerequisite checker/installer
├── doc_search.py           # HTTP search server (port 8643)
├── docubrowse_db.py        # SQLite schema and migrations
├── scan_docs.py            # Scanner: discovery, extraction, DB writes
├── pdf_extractor.py        # PDF extraction (pdfplumber + pypdf fallback)
├── docx_extractor.py       # Word document extraction (python-docx)
├── odf_extractor.py        # OpenDocument (.odt/.ods/.odp) extraction (stdlib)
├── visio_extractor.py      # Visio (.vsdx/.vsdm/.vsd) + draw.io (.drawio/.dio)
├── markup_extractor.py     # SGML/XML family + reST/AsciiDoc/LaTeX (stdlib)
├── eml_extractor.py        # Email (.eml) — stdlib `email`
├── csv_extractor.py        # CSV / TSV — stdlib `csv`
├── rtf_extractor.py        # RTF via striprtf (graceful degradation)
├── ebook_extractor.py      # EPUB/MOBI/AZW3/AZW extraction (ebooklib + Calibre)
├── hardware_utils.py       # CPU/GPU/RAM detection, worker formula
├── embed_docs.py           # Embedding generation pipeline
├── purge_pii.py            # PII scanner and purge tool
├── purge_dotfiles.py       # One-off: remove indexed dotfiles from the DB
├── dup_detect.py           # Exact (SHA256) and near-duplicate detection
├── platform_paths.py       # Cross-platform paths and process management
├── index.html              # Frontend UI (single-file, dark/light theme)
├── du-docs.db              # SQLite database (gitignored)
├── du-docs.db.example      # Empty schema for new installs
├── scan_blacklist.txt      # Failed-extraction skiplist (gitignored)
├── pii_blacklist.txt       # PII-removed files — permanent (gitignored)
├── ocr_list_pdfs.txt       # Image-only PDFs needing OCR (gitignored)
├── visio_legacy_missing.txt # Legacy .vsd files seen without vsd2xml (gitignored)
├── rtf_missing_striprtf.txt # .rtf files seen without striprtf (gitignored)
├── ignore_dirs.txt         # Directories excluded from scanning (gitignored)
├── scan_dirs.txt           # Additional scan directories (gitignored)
├── docubrowse.config       # Local config (optional, gitignored)
├── INSTALL.md              # Step-by-step install guide
├── README.md               # This file
├── LICENSE                 # GPL-3.0
├── packaging/              # RPM spec, DEB control, build scripts, installers
│   ├── build_packages.sh   # Builds RPM, DEB, and tarball (Linux)
│   ├── build_windows_zip.sh # Builds Windows zip
│   ├── docubrowser-foss.spec  # RPM spec
│   ├── docubrowser.desktop # Desktop menu entry (Linux)
│   ├── install.sh          # Tarball installer (Linux)
│   ├── uninstall.sh        # Tarball uninstaller (Linux)
│   ├── windows/            # Windows installer/uninstaller scripts
│   │   ├── Install.bat     # Double-click to install
│   │   ├── Uninstall.bat   # Double-click to uninstall
│   │   ├── install.ps1     # PowerShell installer
│   │   └── uninstall.ps1   # PowerShell uninstaller
│   └── macos/              # macOS installer/uninstaller scripts
│       ├── build_macos_dmg.sh   # Builds the macOS dmg
│       ├── Install.command      # Double-click to install
│       └── Uninstall.command    # Double-click to uninstall
├── systemd/
│   └── docubrowser.service # systemd unit file
├── status_docs/            # Project planning and decision logs
│   ├── project_status.md   # Current version, session history
│   └── DECISIONS.md        # Deferred decisions and known issues
└── test_pdfs_live/         # 100 sample PDFs for testing

↑ Top

Large document collections may trigger:

OSError: [Errno 28] inotify watch limit reached

This is a Linux kernel limit, not a disk space issue. It's safe to ignore these warnings — but if you'd rather not see them, raise the limit for the duration of the scan:

1. Edit /etc/sysctl.conf and find the line:

   fs.inotify.max_user_instances=128
   

2. Raise it to 256 or 512:

   fs.inotify.max_user_instances=256
   

3. Apply the change without rebooting:

   sudo sysctl -p

Once ingestion is finished, you can set the value back to 128 (edit the file again and re-run sudo sysctl -p) — or just leave it raised.

Some PDFs cause pdfminer to hang. These are auto-detected and blacklisted. If a specific file is causing problems, check:

# How many PDF objects does it have? (>8000 is abnormal)
pdfinfo /path/to/file.pdf | grep -i objects

# Retry the file after it has been blacklisted
./docubrowser.py scan-file --file /path/to/file.pdf

Files with >8,000 PDF objects (usually caused by repeated ExifTool metadata updates) are automatically routed through pypdf instead of pdfminer.

PDFs with no extractable text are detected and added to ocr_list_pdfs.txt. They are indexed with a placeholder ([scanned PDF — OCR required]) so they appear in browse but won't match keyword or semantic searches until OCR is run.

Check the log:

tail -f /var/log/docubrowser.log
# or
tail -f ~/.local/share/docubrowser/docubrowser.log

ollama serve                       # start manually
ollama list                        # verify both models are present
ollama pull nomic-embed-text:latest              # embeddings, if missing
ollama pull dolphin3:latest                      # synopsis generation, if missing

↑ Top

↑ Top

DocuBrowse now runs entirely in Chinese — both Simplified (zh) and Traditional (zh-hant). See Languages.

• Simplified (zh) and Traditional (zh-hant). Both use the bge-m3 multilingual embedder and the ornith-1.5:9b synopsis model; the two share one model and differ only by prompt (Traditional forces 繁體/正體字 output, empirically verified). Full UI translations (locales/zh.json, locales/zh-hant.json) and README translations (README-zh.md, README-zh-hant.md).
• CJK keyword search covers Chinese. The existing character-bigram segmentation already spans the CJK Unified range, so Simplified and Traditional both index and match with no cjk.py change; 2+ character terms match reliably, single characters surface via Both/semantic.
• No index bar for Chinese. Chinese is not alphabetic and has no first-letter ordering, so the document-list index bar is hidden (same as Japanese).
• Packaging. All README translations (en/ja/ko/zh/zh-hant) now ship in every package type (RPM, DEB, tarball, Windows, macOS).

Bug-fix release addressing CJK search and language-configuration issues found after v1.5.0.

• Fixed HTTP 500 on semantic search after switching languages. Semantic scoring now compares only vectors built by the active embedder, so an index built under a different language degrades to keyword search instead of crashing.
• Deep Links now find CJK passages. Keyword Deep Links tokenize the query into the same character bigrams the search index uses, so any document surfaced by search yields matching passages (previously "No matching passages" even on a 100% match).
• Configured language is honored at every step. docubrowser scan, embedding, and database init now resolve the same language as the server, so a configured ko/ja no longer silently indexes under English. docubrowser ko scan is no longer needed once the language is set.

DocuBrowse now runs entirely in Korean, and the interface localization is complete (search page and Settings page). See Languages.

• Korean (ko). Full language stack: bge-m3 embeddings, exaone3.5 synopsis, Hangul character-bigram keyword search, and a complete Korean UI translation (locales/ko.json).
• Korean alphabet index bar. Because Hangul is a true alphabet, the document-list index bar is enabled for Korean and browses by the 14 basic leading consonants (choseong) ㄱ–ㅎ; tense consonants fold into their base (ㄲ→ㄱ). English keeps the A–Z/0–9 bar; the bar stays hidden for Japanese.
• Settings page fully internationalized. Every label, description, button, placeholder, status line, confirm dialog, toast, and alert on the Settings page is now localized (English/Japanese/Korean).
• Start in a chosen language from the CLI. docubrowser ko start (or docubrowser start ja) starts the server in that language and persists the choice to docubrowse.config; the code may appear in either position.
• Localized messages. Synopsis-modal errors, the document/tag counters, the tag categories (code/diagram/markup), the scroll-to-top button, and network/error messages are now translated.
• Fixes. Fresh-instance "Error adding directory" (a blank work_dir line no longer overrides the default); the CLI now reads the same config the Settings UI writes, so scan targets the configured directory; HTML is served no-cache so UI/localization changes always reload fresh.

DocuBrowse can now run entirely in a second language. See Languages.

• Per-install language. One install serves one language — interface strings, embedding model, synopsis model, and FTS tokenizer are all selected together. English (en, default) and Japanese (ja) are supported today; adding a language is a data change (lang_models.py row + locales/<code>.json), not new code.
• Localized UI. All interface strings are served from per-language locale files (locales/en.json, locales/ja.json) and resolved client-side; the active locale ships with GET /api/config.
• Language-appropriate AI. Japanese uses the bge-m3 multilingual embedder and a Japanese synopsis model; models are pulled from Ollama on demand at first run (and when switching languages), nothing is bundled.
• CJK keyword search uses app-side character-bigram segmentation over the unicode61 tokenizer (not trigram), so 2+ character Japanese terms (e.g. 2-character kanji compounds) match reliably; single-character queries surface via Both/semantic. Zero dependency, and it applies uniformly to Chinese/Korean when they land.
• Choosing a language. Asked once on a fresh install (upgrades never re-prompt — an existing lang is preserved), or switched later from the Settings gear (POST /api/language), which warns that a rescan is needed to rebuild embeddings/index for the new language.
• Reasoning-model synopsis fix. Hybrid reasoning synopsis models (e.g. the Japanese nemotron) no longer return blank summaries; the synopsis timeout is also raised for large cold models.

• Deep Links across many more formats — HTML, Markdown, EPUB/MOBI/AZW3, DjVu, the SGML/XML family, email, JSON/YAML, source code, and other text/markup formats now support in-document passage search; non-prose (spreadsheets, presentations, diagrams) falls back to opening in its reader.
• Semantic search tuning — articles and conjunctions are stripped from the query before embedding so filler words don't dilute the match; Deep Links gained a semantic relevance floor and drops contentless passages (comment markers, bare numbers), and its semantic embedding is bounded so large documents no longer time out.
• Search on Enter — search runs when you press Enter instead of as-you-type; clearing the box shows all documents again.
• Dotfiles skipped (D-6) — files with a dot-prefixed path component (.env, .git/, .venv/) are no longer indexed. The new one-off purge_dotfiles.py tool removes dotfiles left in the DB by older versions (dry-run by default).
• Docs — new Search Tips and Tricks section; quoted-vs-unquoted phrase search and case-insensitivity clarified.

• Deep Links — in-document passage search. From any keyword or semantic result, find the matching passages inside that document, each with a location label (page / line / section), and jump to one with the matched text highlighted. Computed on demand — no reindex, no schema change, no new dependency. Prose formats: PDF, TXT, HTML, Markdown, DOCX, RTF, ODT, e-books (EPUB/MOBI/AZW3), DjVu, plus the SGML/XML family, email, JSON/YAML, source code, and other text/markup formats; non-prose falls back to opening in the reader. New endpoint GET /api/deep-links.
• Header logo links to the project on GitHub.
• Fix: --db / --port are now honored when placed before the subcommand (docubrowser --db PATH start) — previously they were silently dropped and the default database was used.
• Includes the Intel Arc GPU detection (via xpu-smi, with nvidia-smi fallback) and tar-traversal restore hardening first shipped in v1.0.3.1.

• Configurable Ollama host — OLLAMA_HOST (or DOCUBROWSE_OLLAMA_HOST) points the embedder, search server, and prerequisite checker at a remote or sidecar Ollama. A bare host:port is accepted; the scheme defaults to http://. Default stays http://localhost:11434.
• Path/port/DB via environment — DOCUBROWSE_DOC_DIR, DOCUBROWSE_DB / DOCUBROWSE_DB_PATH, DOCUBROWSE_PORT, and DOCUBROWSE_WORK_DIR overlay the config file (env overrides file; CLI still wins), so a container can run with no docubrowse.config. GET /api/config reflects these too, and doc_search.py can start from DOCUBROWSE_DB / DOCUBROWSE_PORT alone.
• Opt-in private-network access — DOCUBROWSE_TRUSTED_CIDRS and DOCUBROWSE_ALLOWED_HOSTS let a private reverse proxy / BFF reach the API past the loopback-only gate. Trusted ranges are capped at /24 (IPv4) / /120 (IPv6) — a /32 host is preferred — so a stray /8 can't grant the whole network unauthenticated API access. Default remains loopback-only.
• Docs — README, INSTALL, and the Administrator Guide document the full environment-variable set and the trusted-peer security model; a repeatable end-to-end feature test (test_features.py) and testing notes were added.

• DjVu support (.djvu/.djv) — text-layer extraction via DjVuLibre (djvutxt/djvused, optional external tool). Without it, DjVu files are indexed metadata-only and paths logged to djvu_missing_djvulibre.txt; install DjVuLibre and rescan for body text. Image-only DjVu with no text layer is indexed metadata-only (OCR deferred, same as scanned PDFs).
• ODF template support (.ott/.ots/.otp) — the OpenDocument template variants now index through the existing ODF extractor, routed by mimetype prefix to the matching text/spreadsheet/presentation handler. No new dependency.
• Packaging — djvu_extractor.py added to all packaging manifests (RPM, DEB, tarball, Windows, macOS).

DocuBrowse reaches v1.0.0: feature-complete, production-tested, and packaged for all three desktop platforms. This release marks the culmination of 7 weeks of development from MVP (v0.1.0) through format expansion, security hardening, and code quality.

• 40+ supported file formats across 15 extractor modules — PDF, DOCX, PPTX, XLSX, ODF, Visio, draw.io, SGML/XML family, email, RTF, CSV/TSV, EPUB, MOBI, AZW, HTML, Markdown, LaTeX, reST, AsciiDoc, and config-ish plain text.
• Pylint 10.00/10 on all 28 Python source files.
• Python 3.9 through 3.14 tested — including a fix for Python 3.14's stricter keyword argument handling.
• Packaged for Linux (RPM, DEB, tarball), Windows (zip), and macOS (dmg).
• Security hardened — CSRF protection, Host-header allowlist, no unauth GET mutations, PII detection with SSN/CC/routing number validation.
• AI-powered — semantic search, hybrid keyword+semantic mode, and on-demand AI synopsis generation, all running locally via Ollama.

Massive coverage bump. Adds 40+ new file extensions across five new extractor modules, all pure-Python / stdlib where possible.

• Visio & diagrams — new visio_extractor.py handles modern Visio (.vsdx/.vsdm), legacy binary Visio (.vsd/.vss/.vst — needs optional libvisio-tools for body text; degrades to metadata-only otherwise), and draw.io / diagrams.net (.drawio/.dio) — both plain and compressed (deflate + base64 + URL-encoded) mxfile variants. Shape text, page names, and core properties captured. PlantUML (.puml/.plantuml) and Mermaid (.mmd) source is indexed as text and tagged diagram.
• SGML/XML markup family — new markup_extractor.py handles .xml/.xhtml/.sgml/.sgm, DocBook (.docbook/.dbk), SVG (also tagged diagram), Visio 2003 XML (.vdx, tagged diagram), feeds (.rss/.atom/.opml), reStructuredText (.rst), AsciiDoc (.adoc/.asciidoc), and LaTeX (.tex/.latex). Schema-agnostic tag-strip with title/author sniffing from common local-names (<title>, <dc:title>, <author>, <dc:creator>). Stdlib-only.
• Email, RTF, tabular — new eml_extractor.py, csv_extractor.py, and rtf_extractor.py. Email (.eml) via stdlib email package (Subject → title, From → author, To/Cc/Date → subject field, plain or tag-stripped HTML body). CSV/TSV via stdlib csv with delimiter auto-sniffing and BOM-safe reading; header row → description. RTF via optional striprtf (added to requirements.txt); degrades to metadata-only with a rtf_missing_striprtf.txt sidecar if missing — mirrors the visio_legacy_missing.txt pattern.
• Config-ish plain text — .ini, .conf, .cfg, .log, .lst route through the existing text path.
• Tagging — every diagram file gets a diagram tag; every markup file gets a markup tag. Auto-keyword tag generation extended to cover all the new formats.
• CLI — 30+ new _TYPE_MAP entries so scan vsdx drawio eml rtf etc. all work naturally.
• New sidecar files (gitignored, informational): visio_legacy_missing.txt, rtf_missing_striprtf.txt.

• All 28 Python source files score pylint 10.00/10 — two cleanup passes addressing unused imports, missing docstrings, broad exception catches, line length, and naming conventions. No behavioral changes.
• Python 3.14 compatibility fix — Python 3.14 enforces that parameters prefixed with _ cannot be passed by a different keyword name. scan_docs.py and embed_docs.py called wait_for_memory(is_tty=...) but the function defines _is_tty. Fixed — scans no longer crash on Python 3.14+.

• Broken symlinks and permission errors no longer crash scans — is_file() and stat() calls during directory walks now catch OSError instead of propagating it. Affected code paths: pre-scan file count, report command, main file collection in scan_docs.py, and both data-grooming dedup scripts. Inaccessible entries (broken symlinks, NTFS junctions/reparse points copied to ext4, permission-denied paths on sshfs/network mounts) are skipped and counted in the summary output.

• ODF scanning — .odt (text), .ods (spreadsheet), and .odp (presentation) files are now indexed. Extraction uses only Python stdlib (zipfile + xml.etree.ElementTree) — no extra dependency required. Metadata (title, author, subject, description, keywords) is read from meta.xml; body text is extracted from content.xml with full namespace handling for paragraphs, headings, lists, tables, and slide frames.
• Version bump — all packaging files updated to 0.9.2.

• Relevance score capped at 100% — the hybrid search merge formula max(fts, sem) + 0.1 × min(fts, sem) could exceed 1.0 when both keyword and semantic scores were high, producing relevance percentages above 100%. Now clamped to 1.0.

DocuBrowse is now packaged for Linux and Windows with native installers.

• RPM, DEB, tarball, and Windows zip packages — build_packages.sh produces Linux packages; build_windows_zip.sh produces the Windows zip. Linux installs to /opt/docubrowser/ with a Python virtualenv, CLI wrappers at /usr/bin/docubrowser and /usr/bin/docuback, and a desktop menu entry under Office. Windows installs to %USERPROFILE%\DocuBrowse with a Start Menu shortcut (no admin required).
• Windows installer — Install.bat / install.ps1 detects Python, creates a virtualenv, installs dependencies, and creates a Start Menu shortcut. Uninstall.bat reverses everything.
• Cross-platform path abstraction — new platform_paths.py centralizes all runtime path selection (PID files, log files, backup directory) and process management (kill, find-by-script, kill-port). Linux paths are unchanged; Windows paths use %USERPROFILE%\DocuBrowse\.
• Windows compatibility — guarded all Unix-only constructs (resource, SIGALRM, os.killpg, /proc access) behind platform checks. Process management uses psutil with /proc fallback on Linux.
• Backup/restore — backup_restore.py supports Windows privilege checks (IsUserAnAdmin) and handles missing pwd module gracefully.
• Desktop menu entry — .desktop file uses xdg-terminal-exec for reliable terminal launching across all desktop environments; categorized under Office.
• Systemd service file — included for system-level deployments on Linux.
• dist/ pruning — build script keeps only the latest 2 releases per format.
• macOS dmg installer — packaging/macos/build_macos_dmg.sh produces a dmg with double-clickable Install.command / Uninstall.command scripts. Installs to ~/Applications/DocuBrowse/ (no sudo for the app itself) with a Python virtualenv, CLI wrappers, and a DocuBrowse.app launcher whose icon is generated from icons/icon-512.png via sips/iconutil.

Removes unused code paths and experimental features that were never part of the FOSS release, leaving a cleaner and more focused codebase.

• Leaner server — doc_search.py reduced by ~300 lines; removed unused network configuration, protocol negotiation, and handler code that had accumulated during development.
• Leaner CLI — docubrowser.py reduced by ~240 lines; removed the setup-tls command and related helpers that are not applicable to a localhost application.
• Stale files removed — deleted branding.json.example and other development-only files that were never used in production.
• Documentation updated — README, INSTALL, and architecture notes cleaned up to accurately reflect the current feature set.

• Hide documents from view — each card now has a 🙈 Hide icon that tags the document as "hidden" and fades it out of the listing. Hidden documents remain in the database and can be restored at any time.
• "Show 🙈" toggle button — added next to the page count in all views (All Documents, letter filter, search results). Clicking it reveals all hidden cards alongside normal ones; the button label switches to "Hide 🙈" to toggle them back off.
• Unhide (👀) icon — when hidden cards are visible, they display a 👀 icon instead of 🙈. Clicking it removes the "hidden" tag server-side, swaps the icon back to 🙈, and removes the "hidden" tag chip from the card.
• New API endpoint: POST /api/remove-tag — removes a single tag from a document. Parameters: path (URL-encoded file path), tag (tag name). Returns the updated tag list. CSRF-protected.
• Card action icons restyled — all icons (📋 🔖 🙈 ❌) now use solid, colorful emoji at full opacity. No more faded/dim icons in dark mode. ↑ Top

• Dark & light mode palettes redesigned — new CSS variable theming with data-theme attribute switching. Dark mode uses deep navy/purple tones with cyan, orange, and violet accents. Light mode uses clean whites with darkened accent variants for readability. Tag colors cycle through five distinct hues via nth-child selectors. Score badges, mode buttons, and action buttons all use the new palette.
• Trash icon now opens a 4-option modal instead of immediately deleting: (1) Remove from index only (file stays on disk, re-scanned next run), (2) Remove & blacklist (file stays, future scans skip it), (3) Remove & delete file from disk (with double-confirmation), (4) Cancel. Server API updated: POST /api/delete?path=...&mode=db_only|blacklist|delete_file (defaults to db_only for backward compatibility).
• Search scoring fixed in "both" mode — keyword matches were previously buried under thousands of low-similarity semantic results. Now applies a semantic floor (SEM_FLOOR=0.30) and uses max(fts, sem) scoring instead of a weighted average that capped keyword-only hits at 0.3.
• scan command now embeds by default (same as rescan) — new installations get working semantic search out of the box. Added --no-embed and --embed-workers flags for opt-out.
• Security hardening — CSRF protection added to /api/synopsis; exception leak to client suppressed.

• Open action button replaces the old clickable file path link on each result card — launches the file in your default app via xdg-open.
• Button styling updated — buttons use accent-colored border and text with a filled hover state, replacing the previous dim/greyed appearance.

• du-docs.db.example regenerated with current schema — fresh installs no longer hit an HTTP 500 ("no such column: d.subject") on first page load. The old example was built against an older schema (pre-author/subject/synopsis columns and pre-full-FTS5 index), causing the lazy migration to race against the first search request. New example has the correct schema from the start.

• Settings moved to a standalone page (/settings, opened in a new tab via the gear icon) — replaces the old modal. Full-width layout, a header "Done" button that saves config and returns to the search tab, and a redesigned Ignored Directories panel (description text, "Add a directory to exclude" row, "Currently excluded directories" list with inline ✕ remove buttons, confirm-before-purge on add and rescan-reminder on remove).
• Alpha index bar (0-9, A-Z) is now a true global filter — clicking a letter queries /api/search?letter=X for all matching documents (not just the loaded page), paginated by the existing page-size preference; Next/Back and page-size changes work while filtered, and clicking the active letter again returns to All Documents. A "Home" button (left of "0-9") returns to All Documents from anywhere, and the index bar now persists across every view (All Documents, letter-filtered, search, paginated).
• Multiple document directories confirmed fully automatic: resolve_doc_dirs() unifies the configured docPath with scan_dirs.txt into one ordered list; scan/rescan loop over every directory into the single shared database and run embedding once at the end — no manual per-directory rescan needed.
• Removed the "N embedded" count from the header stats bar (now "N docs · N tags").
• friendlyError() helper in index.html gives a clear "Cannot reach the DocuBrowse service" message (instead of a generic network error) if the server is down while a page is loaded — applied to search, filter, pagination, synopsis, open, and delete.
• Synopsis modal's "Generating synopsis..." message now updates at 6s/25s with reassuring text so a slow cold-start Ollama request (up to ~90s) doesn't look hung.

• doc_dir/docPath no longer default to ~/Documents — an unconfigured document directory is now a valid state across the CLI (docubrowser.py), API (doc_search.py /api/config), and install.sh's generated config.
• CLI commands that need a document directory (rescan, report, scan) now exit with a clear error pointing to the Settings gear, docubrowse.config, or --doc-dir if none is configured.
• index.html shows a banner — "No document directory configured yet. Click the Settings (gear) icon..." — whenever /api/config reports an empty docPath.
• Added uninstall.sh, mirroring install.sh's user/system mode detection: stops/disables/removes the systemd unit, removes the CLI wrapper and install directory, cleans up pid/log files, and (system mode, separate confirmation) can remove the dedicated docubrowse user/group.

• Installer: rewritten install.sh/uninstall.sh with a clean user vs. system split — user mode installs to ~/.docubrowse (own venv, wrapper at ~/.local/bin/docubrowser, no root, no systemd); system mode installs to /opt/docubrowse as a dedicated docubrowse user with a (not auto-enabled) docubrowser.service systemd unit and /usr/local/bin/docubrowser wrapper.
• Pre-flight checks: the installer validates all prerequisites up front (python3 ≥ 3.9 + venv/ensurepip, rsync, curl, tar, calibre, ollama, plus getent/useradd/groupadd/systemctl in system mode) and reports everything missing at once before making any changes.
• CLI: the launcher is now installed as the docubrowser command (no .py).
• requirements.txt added and installed via pip install -r requirements.txt — now includes previously-missing deps (numpy, python-pptx, openpyxl) alongside pdfplumber, pypdf, python-docx, ebooklib, beautifulsoup4, mobi.
• Fresh installs start empty: du-docs.db.example now ships empty, so new installs begin with no documents indexed.
• Multiple document directories: Settings now shows a single "Document directories" list (the old separate docPath + "additional directories" panels are merged). rescan/scan index every listed directory; an explicit --doc-dir still targets just one. doc_dir is now optional.

Remediation of a full code-quality + security audit (details in status_docs/DECISIONS.md). Highlights:

• Security: Host-header allow-list (anti DNS-rebinding); /api/delete and /api/open moved to POST and, with the POST config/dir routes and /api/browse, gated by a per-process CSRF token + loopback origin; stored-XSS vector closed (data-attribute + delegated listeners); PII purge now validates with SSA rules + Luhn/IIN.
• Search: keyword path now uses the FTS5 bm25() index and semantic scoring uses a cached NumPy embedding matrix instead of loading the whole corpus per request (keyword ~4ms, both ~55ms); server-side semantic search fixed (was silently returning nothing).
• Reliability: INSERT … ON CONFLICT upsert (re-indexing no longer wipes tags/embeddings/synopsis); worker-death "suspect isolation" blacklists only the real offender; schema init runs once per process; scan/embed commit on a ~2s time budget so the server isn't blocked; dupclean no longer corrupts the disk/DB on its main path; precise /proc-based worker termination; single shared document-delete helper; assorted medium/low fixes.
• UI: pagination Back/Next correct at all page sizes; a newer search now supersedes an in-flight page load (no stale results).

• /api/open now returns {"ok": false, "error": "missing"|"unmounted", "message": ...} for files that no longer exist, instead of a generic error.
• The UI shows a dismissable modal for missing files (and removes them from the index on dismiss), or a toast for unmounted files (filesystem can't be verified, no index change).
• New opt-in scan-missing [--dry-run] CLI command batch-cleans missing rows across the whole index without touching unmounted rows.

• Fixed "Open file" (/api/open) silently doing nothing — the server's environment was missing DBUS_SESSION_BUS_ADDRESS/DISPLAY/XAUTHORITY/ XDG_RUNTIME_DIR, so xdg-open exited successfully without launching the default app. handle_open now reconstructs the desktop session environment and prefers gio open for reliable launches.

↑ Top

• ✅ DOCX extractor (python-docx)
• ✅ EPUB/MOBI/AZW3/AZW extraction (ebooklib + Calibre)
• No-extension file classification (magic bytes)
• Scale to 10K+ documents

• ✅ duplist / dupclean — exact + near-duplicate detection and interactive cleanup
• ✅ Config read/write via Settings UI (port, docPath, workDir)
• Sliding window ETA for progress bar
• File-type filter in search UI

• Config persistence
• Advanced filtering (date range, type, author)
• Result export (CSV/JSON)
• OCR integration for scanned PDFs

• API key authentication
• Document similarity clustering
• Docker deployment

↑ Top

DocuBrowse is developed with Claude as an active coding partner. To resume a session with full context, load these files at the start:

# Print all three for copy/paste into any AI assistant
cat .claude/CLAUDE.md status_docs/project_status.md status_docs/DECISIONS.md

↑ Top

GNU General Public License v3.0 or later (GPL-3.0-or-later).

Copyright (C) 2026 James Sparenberg

See LICENSE or https://www.gnu.org/licenses/gpl-3.0.html.

DocuBrowse v1.5.2 — Fast, local, AI-powered document search.