I previously asked this in 2020 [1] and I think many of us learned of some upcoming and interesting tools.
I use “non-standard” loosly here. We’re looking for CLI utilities that are definitely not part of the POSIX required or optional utilities, and more coloquiallly not considered to be standard BSD or *nix fare.
[1] https://lobste.rs/s/eprvjp/what_are_your_favorite_non_standard_cli
I actually picked up a few from the last thread that I continue to use today
devenvandessfor local development secrets/settingsSome new ones I’ve started using since last time
env.samplein sync with.env, so secrets and settings get documented. I use it in a post-commit hook.kittywindows withaercand typicallydi-tuinixcliThe ones I’ve been using for many years
passTools typically only used inside projects (via devenv dependency)
I’ll keep editing as I think of more.
If you’re familiar with SQL, I’ve found that
duckdbhas pretty much replaced my usage ofxsvandcsvkit.Slightly off topic, but I find this alias really handy for quickly qr encoding stuff I want to access on other devices:
e.g.
quickqr lobste.rs[update]
Even more off topic – you’re probably a lot better off using qrencode’s
-t ansiutf8switch to print the qr directly to your terminal. Either I missed this switch years ago, or it didn’t exist when I created this alias.I’ve updated it to the much more sensible
I’ve been using
qrencode -t ANSI256UTF8 blahblahfor copying OTPs from my Authy export to my ReinerSCT doodad. Super handy.I use devenv(lorri,flakes) with the flakes version as a fallback. Lorri has some really nice features when it comes to flakes.
I haven’t used it yet, but a lot of people sound enthusiastic about mise as a superset of this functionality.
IME, mise is a nicer asdf, with a good UI. I’ve used it in the past for controlling versions of software tools, but as a general tool, it’s sometimes not the best for more complicated setups. E.g., I prefer uv for my Python needs.
And unlike nix (which devenv is based off), it can’t control nearly as many things if you need super-fine-grained and/or extensive versioning.
Thank you!
I didn’t see this one mentioned in the last thread, so I’ll post it here:
fq is basically jq for binary formats. Whilst not so useful for programming it tends to be very useful for quick reverse engineering of formats or if you need to inspect some metadata information stored in binary data.
🥳
I tried
fqon some binary file and was disappointed by the gobbledygook, but it turns out /usr/bin/fq on ubuntu is a log viewer for nq/tq jobs … ie. install it from github, not apt :):D yeap sadly there is a name collision :( hope the intended fq gave more resonable output
it did! It’s actually surprisingly interesting to just
fq .random binaries :-D🥳 i also casually poke around with fq at times, good way to get motivation when something unknown looks interesting :)
In alphabetical order, here are a few that come to mind:
entr: run a command when files changeffmpeg: convert audio/video (see also: ffmpeg buddy)ncdu: analyze disk spacepwgen: generate passwordsshellcheck: lint shell scriptstig:git loginterface I prefer (among other features)tmux: windows and splits in your terminal (among other features)tree: print directory trees nicelyyt-dlp: download audio/video from many sourceszopfli: make gzip-compatible files that are smaller thangzip. Also seezopflipngncduis one I can never remember when I need to go byte hunting. I’ll add it to my nixos config this time :)Consider trying duc : http://duc.zevv.nl/
One I use often is xh as an httpie replacement without depending on python. It has (in my opinion) a more intuitive command interface and can optionally print out the equivalent curl invocation for you
Thank you! I’ve been looking for a competent HTTPie replacement for a while, and xh looks good.
Nobody yet has mentioned ugrep, my favorite grep.
The
ug -Qcommand opens up an interactive regex-builder that searches for results as you edit the regex. It’s like those websites that teach you how to write regex queries, except inside your own file system, and it can compose with other CLIs.I also love the ELFKickers tool suite. They are obscure but I use them all the time. Especially
sstrip(a strip-ier strip) andelfls, but the others are also great.I am a caveman that doesn’t really stray outside of the posix toolset that often, but there’s some invaluable stuff I’ve picked up
and some more stuff that mainly appeals to media nuts/archivists like me:
yt-dlp properly handles Bandcamp albums now.
it still only downloads the surface-level mp3 128k files from bandcamp–it can’t navigate the buttons and whatnot that need to be pushed to get a flac download link
I didn’t realise
free-bandcamp-downloadercould get the FLACs from my account! Thank you, this is great news!lswhich generates nicer output thanls.lfbut I feel that yazi is being recently more actively developed. It renders images in the terminal out-of-the-box.cd.jj — a Git compatible version control system.
acme (plan9port) — a text editor from Plan 9.
yaegi — a Go interpreter.
rc (plan9port) — a shell from Plan 9.
tcc — a tiny C compiler. Featuring the -run flag.
aunpack (atool) — an archive unpacker.
rlwrap — a readlinifier. So your Emacs key bindings work.
alejandra — a Nix formatter.
cloc — a lines of code counter.
oooh acme and rc :)
for context, i am very hesitant to pick up new tools in general, since standard CLI utilities/applications get me 90% of the way there for most things - to introduce a tool into my kit, it really has to be exceptional in my mind.
so, tools i can’t live without:
(can you tell i interact with json a lot?)
highly recommend step, it’s laid out a lot better than
opensslit handles MOST things i need to do w/ certificatesit would be
step certificate inspect "https://google.com"can also provide
--format=jsonwhich is lovelypv, the pipe viewer, is super handy. I also use parallel sometimes. These aren’t mentioned much in the thread so far.
+1 for
pvandparallel.parallelis especially helpful when you’re mangling filenames with its{}twiddlers (and escape to Perl if you need something esoteric.)For mac users,
pbcopy+pbpasteis a great little trick to get stuff in and out of your clipboardAlso
caffeinatewill prevent your Mac going to sleep.pv, moreutils, kakoune are the ones I use that aren’t already here
Several I have not seen mentioned (in order of what I most frequently use):
Also not sure if this one counts since it’s not really a CLI tool but does enhance my terminal experience:
xqfor doing the same with XML.otpauth://URL as text, and I want to use pass otp as my authenticator.Annoyingly, moreutils conflicts with a couple of these:
ts, which conflicts with taskspooler (Debian renames the latter totspfor this reason)parallelcommand which conflicts with GNU Parallel’s. I always prefer the latter, so I mark it “high priority” in my Nix configs so itsbin/parallelsymlink takes precedence.Eventually I’m tossing around the idea of an AWK implementation that uses Pike’s structural regex and capture groups instead of FS, but we’ll see if I get to it.
ayy i wrote pa - any reason for wanting to swap it out ooc?
I haven’t decided if I will or not. I’m likely looking to unify my usage with agenix and maybe bitwarden for syncing. We’ll see though, I haven’t investigated too much yet.
pais nice and simple, so it works for scripts really well. It’s not quite clear to me how it decrypts the vault though? I haven’t dug through the code, but the fact that I can just dopa show ...without any sort of decryption/password step concerns me slightly and has prevented me from putting more in it.Thanks for introducing me to tsk, I like it a lot!
Glad you like it! (I’m the author)
Link?
The original plan9 version has docs here.
I usually use this version, though I found some bugs with child processes getting orphaned improperly that might push me to either fork it or try out the plan9port version.
rcis much nicer to script than (ba)sh IMO, and I control all of my systems /shrugjj: Has replaced the git CLI for me in the last few monthsjq: tbh, I’m not sure how much longer this one counts as non-standard, but structured JSON at the terminal is supported by so many tools now and is nicer thanawkzoxideis basically for jumping to directories based on partial name matches as acdsupplementbatiscatwith line numbers and syntax highlightingezahas nicer presentation out of the box and I also had an alias that used it as agit statusreplacement before moving tojjrofiis nice for minimal GUI from shell scripts`when you need a picker or text inputfzfis similar for TUII personally wrote quite a few that I still use extensively today:
I also couldn’t live in a terminal anymore without those:
I don’t use
jqbecause my main shell is nushell and it can wrangle JSON and other formats like YAML, CSV, TOML, even INI with plugins!I wrote wrappers for commands I use frequently that output tabular data or benefit from autocomplete like docker, kubectl and so on. I have even more scripts strewn about my work laptop to do stuff that’s more specific to my company’s infra. I build pipelines to parse and filter and aggregate data very frequently. I’m just generally very happy with nushell, even though it still has its rough edges.
Other stuff:
.envrcfile in each repository that I add to.git/info/excludeso it doesn’t accidentally end up in my coworkers’ computers.irbaren’t going away any time soon for me when I need to do some quick thinking.I probably picked a few up from previous threads, so feel free to take credit for what you like from here :)
On my day to day at home and work:
At home I also use often:
Most of the stuff I use will already be mentioned here, so I will add two slightly less common things:
Oh, and the most important one, the shell: fish.
Everything on https://github.com/leahneukirchen/leahutils of course. ;)
Since you’ve broken the taboo, my own collection is here :)
http://chriswarbo.net/git/warbo-utilities
kubectlcommandsTilde. By far my favourite console/shell text editor, as I have written about.
The search is over! Finally (apparently I am 13 years late though) a proper MSEDIT Clone.
I can’t tell if you’re kidding or not. :-)
Anyway, yes. I don’t write code; I tweak config files a bit.
I don’t want any additional functionality aimed at writing code. I want my editor not to have extensions for programmers. I want it small, simple, and I want a familiar UI, ideally one that is close to the UI I normally work with on the desktop.
This simple and reasonable position appears to baffle and bewilder most in the xNix and FOSS world.
:-) I was serious. I learned about computers by opening files using MSEdit on MSDOS and try to figure out what’s what.
I think discoverability is underrated. I’ve been a Linux user for 25 years. It was painful to get started in the beginning and there was this anti-microsoft mindset that refused to look at positive things Microsoft was doing.
MSDOS gets little credit and is pretty much forgotten. But I think bill gates and his employees were really good at giving people what they wanted/needed. And they mastered the art of making things intuitive.
With MSEdit, one only needed to know that you press alt plus the underlined leter. From that point on, you could explore the possibilities yourself.
I remember some 10 years ago or so, emacs making a release highlighting the great breakthrough that the menus finally dropped down on the terminal. We are talking about 3 decades of delay comparing to Microsoft.
Great to hear it. I completely agree!
Trying to only list some that aren’t already listed:
Some of my own:
It is bundled with vim and so was installed on every *NIX system I used and I didn’t realise until recently that it wasn’t standard: xxd. Hex dump, with a reverse mode (so you can turn any text editor into a hex editor by piping the buffer through it and back). Can also generate a C header containing the binary, which is useful until
#embedworks everywhere.I’m not sure how non-standard it is, but I like git-delta a lot for viewing diffs.
Lotta common ones with folks in here but I use:
jq- When I just need to script something withcurlto do an actioneza- I use this on macOS + Linux because it looks as good aslsoutput does on windows with powershellop- 1password CLI tool I use for.envfiles.Not much else :v
screen (might be standard, I don’t know)
I have nothing more to add to what everyone else said; the only one that might be useful is this small program I wrote to bookmark files and open them easily.
Nix, fish, jj, Helix, tmux, ripgrep, eza, entr, HTTPie, jq, tio, age and sops.
I try to use GUI for most things, but lazygit is still my favorite git tool.
devbox gives me everything I want out of a package manager on my Mac. That combined with direnv gives me per project repeatable dependency installation and I don’t have to use homebrew.
I wrote septum to interactively search large codebases (>1 million lines). It looks for multi-line contexts of a specific width which match filters of terms to include or exclude. It works well on big projects like Unreal Engine and LLVM.
I use my own tools which are part of the Offpunk package (disclaimer: I’m the main author)
More rarely:
https://offpunk.net/
The other day I learned Visidata can handle JSON (if it’s sufficiently tabular). There are things that I don’t like about the project, but I haven’t found anything nicer. (It has replaced most uses of spreadsheets for me.)
Also, https://github.com/ihabunek/toot
batch
qmvfromrenameutilsis underappreciated - batch rename files using your usual text editor!