project: December Adventure

Michael F. Lamb

2023-12-02

In lieu of doing the Advent of Code I aim to do a little bit of programming every day in December.

This is the December Adventure, invented by a notably awesome internet person named Eli.

I tried the December Adventure in 2022, my first time, and despite poor diligence keeping a log, I had fantastic results: I worked together with my buddy m455 to create a minimalist static git forge, which let me Give Up GitHub!

It works great! I use it all the time. And it was great to work together on a project with a friend. I have Eli to thank for this!

I’m excited to give December Adventure another try this year.

Dec 01 (Fri)

I lost my job! 😱

I’m definitely in for an adventure now.

It was sad, because I really liked this employer.

But it wasn’t a terrible shock: my employer was going through a restructuring brought on by Covid-19. I found out a week ago that today would be my last day.

And it wasn’t traumatic: everyone at work was effusive about my skills and were sorry to see me go. My boss was clear that if there were any way to keep me on, they would. And all my colleagues offered assistance and references for my job search.

Anyway, I didn’t do any adventuring today other than maybe thinking about what I might want to work on.I’m feeling very reluctant to mention those ideas here because it’s somewhat embarassing to lay out my plans and then end up doing something completely different. But I’m giving myself permission.

 Here’s the top of mind stuff that I think I might want to attempt:

  1. A SyncAdaptor for TiddlyWiki for nginx’s DAV module. TiddlyWiki is, so far, the best knowledge-management tool I have, and I get quite a lot of use out of it. However, for a hosted TiddlyWiki to load and save pages efficiently, it requires you to run a slightly customized DAV server, implemented in JavaScript using nodejs. I dislike nodejs, I’m already running nginx which includes a builtin DAV module, and I suspect those customizations aren’t acutally necessary.

  2. A web-based editor for my website. As much as I enjoy the power of a text editor, pandoc, and the static site generator that I wrote, I think I would update my website more often if I could do so “live,” through the web, like a wiki, like my TiddlyWiki.

    Also, I’d like my website to give the impression that it intends to be a space to tinker and explore ideas, not an unchanging collection of carefully-revised publications of strongly-held opinions.

    One possible way to implement this might be to implement idea #1 and then make my whole website into a TiddlyWiki.

  3. Rust Keyboard Firmware. I designed and assembled a circuit board for a mechanical keyboard. I want to implement its firmware using Rust but I haven’t yet learned enough to do more than make it blink an LED.

  4. Git forge bugfix. The git forge that I built last year has a flaw: it doesn’t work with clients based on libgit, in particular guix, because libgit doesn’t support git’s static http transport. I can work around this by making nginx proxy-pass to git-http-backend. But I don’t feel great about it; it moves me away from the goal of an extremely low-overhead service.

  5. ActivityPub-based website comments. There’s a mechanism to enable user comments on a static website like this one by serving them from a Mastodon server. I’d like to try to get that going for this website.

  6. Human-scale search and recommendations. I’ve got some ideas for a thing that would be like linkhut or pinboard mashed up with a search engine like marginalia.nu, that recommends web pages weighted by the strength of your connections to other users in the network.

All time consumed by final day of work and preparing for joblessness. No code yet.

P.S. hire me?

Dec 02 (Sat)

Wrote the initial draft of this page. No code yet.

Dec 03 (Sun)

🤷 Weekend family stuff. No code yet.

Dec 04 (Mon)

Spent some time wandering toward idea #1, a TiddlyWiki SyncAdaptor for nginx DAV. Refamiliarizing myself with TiddlyWiki’s code, its Extended Persistence mechanisms, and what efforts and difficulties I encountered the last time I looked at this.

Recalled some really interesting related projects:

Oh also, remember how “Rust Keyboard Firmware” is my December Adventure project #3?

While learning about TiddlyPWA, I learned that its author, Val Packett, taught themself KiCad in service of manufacturing a circuit board for their own custom keyboard, with Rust firmware, just like I did! But they had way more success than me, and thoroughly documented many of the finer exciting details!

Now with all this great info close at hand I’m eager to tinker on mine some more. I am most interested in studying their design around the USB-C connector, reverse polarity protection, and the super-efficient hardware mechanism to watch for keypresses they found from another blog.

I also want to congratulate myself for doing a few chores regarding employment, but that’s just my vocation now, not my fun December Adventure.

Bunch of research. No code yet.

Dec 05 (Tue)

Yesterday’s writeup, that’s it. 🤷

Dec 06 (Wed)

I’ve been attempting a different diet, possibly incorrectly, and while it has been mildly successful (I’ve lost a few pounds) I’ve been feeling very low-energy in the morning. Then, I have errands in the afternoon and evening. Then, today, I decided to spend some time in company of family instead of in my dark cave, computer-touching. By the time everybody went to bed, brain didn’t want to brain anymore and I did little more than read a bit.

So, once again: 🤷 no code yet.

December Adventure is low-key, low-stress, I’m supposed to be kind to myself and not feel bad about an off-day. Reading through others’ logs is enough proof that it’s fine, and everybody does it. But in spite of myself I do feel bad about having no real output yet.

Dec 07 (Thu)

I tuned up my e-mail filters a bit today with some hand-generated sieve rules, to quiet my inbox, which is a very good and useful thing in the long run. But with no code yet under the December Adventure category it felt almost like procrastination.

🌩️️

🌩️️

Dec 10 (Sun)

FeatherWiki, when it is configured to sync to an upstream server, does so like TiddlyWiki’s Saver module does: it compiles a new bundle of all its pages into itself and HTTP PUTs that single big ball of code to the server. This isn’t great for me to build from; it is more likely to result in conflicts if more than one person (or one person in multiple devices or tabs) is editing the wiki concurrently, even if the edits are on different wiki pages.

TiddlyPWA is super cool but, for my purposes, I don’t need its encryption: I’m not attempting to build a hostable-anywhere private wiki; I want a public-readable wiki that some authorized users may also edit through-the-web.

I do like its use of a IndexedDB that gets synchronized, but I don’t want to require a complex server on the backend.

So it looks like maybe I should focus on a modification to standard TiddlyWiki to do what I want.

One thing I dislike about TiddlyWiki’s syncer module is that it polls the server for changes. I recently learned about nchan, a module for nginx that looks like the easiest way I’ve seen yet to add websockets and server-pushed events to any web application. But its killer feature is that it’s trivially available via Debian which runs on my webserver, just a sudo apt install away.

I’d have to not just implement a SyncAdaptorModule but also replace or override parts of TiddlyWiki’s Syncer to implement server-pushed updates with nchan, but that sure would be cool.

There are of course lots of other websocket server implementations out there, but I’m drawn to this nginx module because, since I’m already running nginx to serve pages, I get them “for free” in terms of resources and complexity. Almost.

New project on the pile: nchan + auth proof-of-concept. A few weeks ago, I managed to get client certificates working for user authentication on orbital.rodeo. Can I build a little webpage that can distinguish between authenticated and unauthenticated visitors using nchan-based pub-sub? Preferrably by client certificates, but if not, then by http basic auth?

Dec 11 (Mon)

Dec 12 (Tue)

I think I got a proof of concept working for nchan, with authentication by tls client certificates!

This is sufficient to allow unauthenticated visitors to get a read-only websocket and authenticated users to get a read-write websocket:

ssl_client_certificate /opt/system/etc/ssl/certs/ca.crt;
ssl_verify_client optional;
location /pubsub-test/pubsub {
  nchan_pubsub;
  nchan_channel_id default;
  if ($ssl_client_i_dn != "CN=orbital.rodeo certificate authority") {
    return 403 "Forbidden";
  }
}
location /pubsub-test/sub {
  nchan_subscriber;
  nchan_channel_id default;
}

It was a bit of a slog to get here. In particular, attempting to use nchan_authorize_request to an @-style internal location that returns an HTTP status code causes segfaults?

To Do

Dec 13 (Wed)

I’ve actually done some work on this TiddlyWiki effort back in May of 2022. At that time I got tiddler saving and loading via nginx DAV to work somewhat okay. Instead of the nodejs server dynamically replying with a filtered list of tiddlers, I had a shell script using inotify and jq to prerender a similar list as a static file whenever a tiddler was modified.

But it was a fast and loose “tweak things and see what happens” hack without a lot of understanding. So I was left with two outstanding problems (that I know of):

  1. At first load, the StoryList is not applied; you always see the default GettingStarted tiddler.

  2. While editing a tiddler, tiddlers will be refreshed in the background on a timer. In my version, when that happened, focus jumps from the edit box to the title. Which is super annoying while typing.

Today was mostly loading this work back into my brain and making some observations:

Next steps

Ideas

I’m modifying TiddlyWiki because it is big and loaded with features and I have a lot of notes already written in it. But, it is old, and where it reduces complexity I’d prefer to use more modern front-end tech. It’s also a large codebase that’s not easy to wrap my brain around. So I daydream about how I might build a wiki-like thing from scratch.

I really enjoy that the server-side can be described like a handful of legos:

And then, stretch goal, looking at TiddlyPWA for inspiration,

That set of features feel really compelling. It’s very lightweight and not tied in any strong ways to a particular language or design. I can imagine other, non-wiki web projects using the exact same (dare I call it) framework.

As for the data format,

I probably wouldn’t be able to use htmx together with the barebones nginx DAV, because htmx expects the response from a PUT request to be in HTML format, to replace some element on the page. DAV PUT responses are empty.

“But Javascript is Bad!” Many people disable JavaScript in their browsers, for good reason. But it’s pretty much required for the stuff above to work. I’m fond of projects like SourceHut who can claim, “all features work without Javascript.”

The way I square this is to give myself permission to use JavaScript, as long as I do so in a human-friendly way:

Dec 14 (Thu)

Read some more TiddlyWiki source code.

Feels like TiddlyWiki assumes it will have all of its composite tiddlers in memory from the first view, and I don’t want that. I want maybe a list of pages and attributes (so-called “skinny tiddlers”) and to load tiddler payload from the server as-needed.

I was hoping the scope of this project would be limited to “create a storage adapter” not “rewrite the whole thing” along with the resultant “fight against every piece of the system that assumes it works a different way than I want it to.”

So if it begins to feel like a slog, maybe the “build from scratch” idea might actually be the way to go. I’d probably end up with a very rudimentary editor mechansim and no cool things like a plugin architecture, but I could be okay with that for now.

But on second thought: if it is assuming it has all of its tiddler payloads in memory, why do “skinny tiddlers” exist? Need to study more source code.

Ahead

Dec 15 (Fri)

Dec 16 (Sat)
Dec 17 (Sun)

Dec 18 (Mon)
Dec 19 (Tue)
Dec 20 (Wed)
Dec 21 (Thu)
Dec 22 (Fri)

Dec 23 (Sat)
Dec 24 (Sun)

Dec 25 (Mon)
Dec 26 (Tue)
Dec 27 (Wed)
Dec 28 (Thu)
Dec 29 (Fri)

Dec 30 (Sat)
Dec 31 (Sun)