TUI multiline fuzzy document finder

Think interactive grep for text files, PDFs, DOCXs, etc, but word/token based instead of regex and line based, so you can search across lines easily.

Docfd aims to provide good UX via integration with common text editors and PDF viewers, so you can jump directly to a search result with a single key press.

Navigating repo:

Quick search with non-interactive mode:

Navigating PDF and opening it to the closest location to the selected search result via PDF viewer integration:

• Multithreaded indexing and searching

• Multiline fuzzy search of multiple files or a single file

• Content view pane that shows the snippet surrounding the search result selected

• Text editor and PDF viewer integration

• Command history - rewrite/plan your actions in text editor

• Search scope narrowing - limit scope of next search based on current search results

• Swap between multi-file view and single file view on the fly

Statically linked binaries for Linux and macOS are available via GitHub releases.

Docfd is also packaged on the following platforms for Linux:

• opam
• AUR (as docfd-bin) by kseistrup
• Nix (as docfd) by chewblacka

The only way to use Docfd on Windows right now is via WSL.

Notes for packagers: Outside of the OCaml toolchain for building (if you are packaging from source), Docfd also requires the following external tools at run time for full functionality:

• pdftotext from poppler-utils for PDF support
• pandoc for support of .epub, .odt, .docx, .fb2, .ipynb, .html, and .htm files
• fzf for file selection menu

command | docfd

Docfd uses single file view when source of document is piped stdin.

No paths should be supplied as arguments in this case. If any paths are specified, then stdin is ignored.

In this case, the default cache soft limit might not be enough or you might want to keep a stable cache for this collection of files

The following script template may be handy in this situation for creating a collection specific cache

#!/usr/bin/env bash

docfd --cache-dir /large/collection/.cache --cache-soft-limit 20000 /large/collection

docfd [PATH]...

The list of paths can contain directories. Each directory in the list is scanned recursively for files with the following extensions by default:

• For multiline search mode:
  • .txt, .md, .pdf, .epub, .odt, .docx, .fb2, .ipynb, .html, .htm
• For single line search mode:
  • .log, .csv, .tsv

You can change the file extensions to use via --exts and --single-line-exts, or add onto the list of extensions via --add-exts and --single-line-add-exts.

If the list PATHs is empty, then Docfd defaults to scanning the current directory . unless any of the following is used: --paths-from, --glob, --single-line-glob.

If exactly one file is specified in the list of paths, then Docfd uses single file view. Otherwise, Docfd uses multi-file view.

docfd [PATH]... ?

The ? can be in any position in the path list. If any of the path is ?, then file selection of the discovered files via fzf is invoked.

docfd [PATH]... --paths-from paths.txt

The final list of paths used is then the concatenation of PATHs and paths listed in paths.txt, which has one path per line.

docfd --glob 'relative/path/glob' --glob '/absolute/path/glob'

Resolution of relative globs starts at current working directory.

It matches the common file globbing syntax

• ? matches any character, e.g. R?ADME.md matches README.md
• * matches any number of characters excluding /
• ** matches any number of characters including /
• [x-y] matches any character between x and y, e.g. [A-R]EADME.md matches README.md
• a matches character a
• \a matches character a, unless the character is c (see below)

Additional markers:

• If \c is present at any point, then the glob is treated as case-insensitive, e.g. re\cadme.md matches README.md

  • Note that the escape rule is applied first, i.e. \\c matches the string \c, and is not treated as a marker

The search field takes a search expression as input. A search expression is one of:

• Search phrase
• ?expression (optional)
• (expression)
• expression | expression (or), e.g. go ( left | right )

To use literal ?, (, ) or |, a backslash (\) needs to be placed in front of the character.

A search phrase is a sequence of tokens where a token is one of:

• Unannotated (fuzzy match, e.g. hello means fuzzy match hello)
• 'tok (' prefix means exact match the token)
• ^tok (^ prefix means prefix match the token)
• tok$ ($ suffix means suffix match the token)
• ~ (explicit spaces, i.e. contiguous sequence of spaces, tabs, etc)

Tokens that are not separated by spaces, operators, or parentheses are treated specially, we call these linked tokens. For example, 12, :, 30 are linked in 12:30, but not in 12 : 30. Linked tokens have a much stricter search distance by default, e.g. in 12:30, Docfd will search for : only up to a few tokens away from 12, and so on. This allows user to state intention of reduced fuzziness.

To link spaces to tokens, one needs to be make use of ~. For example, to search for "John Smith" ("John" and "Smith" separated by some number of spaces), one can use John~Smith to establish linkage.

For ', ^, $ to be considered annotation markers, there cannot be space between the marker and token, e.g. ^abc means "prefix match abc", but ^ abc means "fuzzy match ^ and fuzzy match abc".

Annotated linked tokens are also treated specially:

• ^12:30 is equivalent to '12 ': ^30
• '12:30 is equivalent to '12 ': '30
• 12:30$ is equivalent to 12$ ': '30

But with even stricter search restriction than the normal linked tokens, namely the next matching token must follow immediately from the current match, e.g. ^12:3 will not match 12 : 30 but will match 12:30

Search is asynchronous, specifically:

• Editing of search field is not blocked by search progress
• Updating/clearing the search field cancels the current search and starts a new search immediately

Navigation mode

• Switch to search mode
  • /
• Exit Docfd
  • Esc
  • Ctrl+C
• Enter print mode to print items to stderr
  • p

Search mode

• Search field is active in this mode
• Enter to confirm search expression and exit search mode

The default TUI is divided into four sections:

• Left is the list of documents which satisfy the search expression
• Top right is the content view of the document which tracks the search result selected
• Bottom right is the ranked search result list
• Bottom pane consists of:
  • Status bar
  • Key binding info
  • File path filter bar
  • Search bar

File path filter bar consists of the file path filter status indicator and the file path filter field. The file path filter status indicator shows one of the following values:

• OK
  • File path filter glob was applied successfully
• ERR
  • Docfd failed to parse the glob in the field

Search bar consists of the search status indicator and the search field. The search status indicator shows one of the following values:

• OK
  • Docfd is idle/search is done
• ...
  • Docfd is still searching
• ERR
  • Docfd failed to parse the search expression in the field

If the specified path to Docfd is not a directory, then single file view is used.

In this view, the TUI is divided into only three sections:

• Top is content view
• Middle is ranked search result list
• Bottom pane is the same as the one displayed in multi-file view, but with different key binding info

• File auto-reloading is not supported for PDF files, as PDF viewers are invoked in the background via shell. It is possible to support this properly in the ways listed below, but requires a lot of engineering for potentially very little gain:

  • Docfd waits for PDF viewer to terminate fully before resuming, but this prohibits viewing multiple search results simultaneously in different PDF viewer instances.

  • Docfd manages the launched PDF viewers completely, but these viewers are closed when Docfd terminates.

  • Docfd invokes the PDF viewers via shell so they stay open when Docfd terminates. Docfd instead periodically checks if they are still running via the PDF viewers' process IDs, but this requires handling forks.

  • Outside of tracking whether the PDF viewer instances interacting with the files are still running, Docfd also needs to set up file update handling either via inotify or via checking file modification times periodically.

• Big thanks to @lunacookies and @jthvai for the many UI/UX discussions and suggestions
• Demo gifs and some screenshots are made using vhs.
• ripgrep-all was used as reference for text extraction software choices
• Marc Coquand (author of Stitch) for discussions and inspiration of results narrowing functionality
• Part of the search syntax was copied from fzf