Using image-dired to browse the latest screenshots from multiple directories

| emacs, image, org, link

Since A+ and I play lots of Minecraft together, I figured it's a good opportunity to slowly get her into the idea of documenting learning. Besides, I can always practise it myself. Screenshots are handy for that. In Minecraft Java, F1 hides the usual heads-up display, and F2 takes the screenshot. Usually, when I start taking screenshots. A+ starts taking screenshots too. I want to build on her enthusiasm by including her screenshots in notes. To make it easy to incorporate her pictures into our notes, I've shared her GDLauncher folder and her Videos folder with my computer using Syncthing so that I can grab any screenshots or videos that she's taken.

In Emacs, image-dired makes it easy to see thumbnails. The neat thing is that it doesn't just work with a single directory. Just like Dired, you can give it a cons cell with a directory in the first part and a list of files in the second part as the first argument to the function, and it will display those files. This means I can use directory-files-recursively to make a list of files, sort it to show most recent screenshots first, limit it to the most recent items, and then display a buffer with thumbnails. image-dired-show-all-from-dir reports a small error when you do this (I need to send a patch upstream), so we hush it with condition-case in my-show-combined-screenshots.

(defvar my-screenshot-dirs
  '("~/recordings"
    "~/.var/app/org.prismlauncher.PrismLauncher/data/PrismLauncher/instances/"
    "~/sync/gdlauncher-instances/"))
(defvar my-recent-screenshot-limit 50)

(defun my-combined-screenshots (&optional limit)
  (seq-take
   (sort
    (seq-mapcat (lambda (dir)
                  (directory-files-recursively dir "^[0-9][0-9][0-9][0-9]-.*\\.\\(png\\|webm\\)"))
                my-screenshot-dirs)
    :key #'file-name-base
    :lessp #'string<
    :reverse t)
   (or limit my-recent-screenshot-limit)))

(defun my-latest-screenshot ()
  (car (my-combined-screenshots)))

(defun my-show-combined-screenshots (&optional limit)
  "Show thumbnails for combined screenshots."
  (interactive (list (when current-prefix-arg (read-number "Limit: "))))
  (condition-case nil
      ;; ignore errors from image-dired trying to set default-directory
      (image-dired-show-all-from-dir
       (cons (car my-screenshot-dirs) (my-combined-screenshots limit)))
    (error nil)))
2025-01-02_08-19-32.png
Figure 1: The result of my-show-combined-screenshots

In the *image-dired* buffer created by my-show-combined-screenshots, I can use m (image-dired-mark-thumb-original-file) to mark images and C-u w (image-dired-copy-filename-as-kill) to copy their absolute paths.

To make it easier to create links to a file by using org-store-link (which I've bound to C-c l) and org-insert-link (C-c C-l in an Org buffer), I can define a link-storing function that takes the original filename:

(defun my-org-image-dired-store-link ()
  (when (and (derived-mode-p 'image-dired-thumbnail-mode)
             (get-text-property (point) 'original-file-name))
    (org-link-store-props
     :link (concat "file:" (get-text-property (point) 'original-file-name)))))

(with-eval-after-load 'org
  (org-link-set-parameters
   "image-dired"
   :store #'my-org-image-dired-store-link))

To make it easier to insert the marked links that have been copied as absolute paths:

(defun my-org-yank-file-links-from-kill-ring ()
  (interactive)
  (dolist (file (read (concat "(" (current-kill 0) ")")))
      (insert (org-link-make-string (concat "file:" file)) "\n")))

I usually want to copy those files to another directory anyway. I have a my-org-copy-linked-files function in Copy linked file and change link that copies the files and rewrites the Org links. This means that I can copy my notes to an index.org in a directory I share with A+, save the images to an images subdirectory, and export the index.html so that she can read the notes any time she likes.

This is part of my Emacs configuration.
View org source for this post

Cuberventures: Early game Create Mod

| minecraft

A+ likes exploring different Minecraft Java modpacks in creative mode. We're currently playing FXNT Create 2. I like challenging myself to figure things out in survival mode, so I'm getting a lot of practice in building early-game survival farms with Create. My setup has changed a little bit over the last few modpacks/worlds. I usually start by mining enough iron and zinc to make a mechanical press so that I can make the iron sheets used in other machines. Then I make a mechanical mixer so that I can make andesite nuggets more efficiently. Then I make a millstone and an encased fan so that I can process cobblestone and wash the resulting gravel manually.

Once that's in place, I can start making more elaborate things.

2025-01-01_16.47.59.png
Figure 1: A simple radial tree farm uses a mechanical saw.
2025-01-01_16.47.03.png
Figure 2: This time, I experimented with putting my crop farm under the tree farm.

I used to have the encased fans just blowing horizontally. After I learned that I can use encased fans to blow upwards through a depot, I switched to setting up my blasting/smoking/washing stations vertically so that I didn't burn myself as often.

2025-01-01_16.36.12.png
Figure 3: My blasting/smoking/washing stations

The most recent thing I added to this world is this cobblestone-gravel-iron nugget machine.

2025-01-01_13.27.19.png
Figure 4: The cobblestone generator…
2025-01-01_13.28.06.png
Figure 5: … feeds into a millstone/washer
2025-01-01_13.29.02.png
Figure 6: All together now
2025-01-01_16.07.40.png
Figure 7: I changed the cobblestone output to feed into two millstones, dropping the results with chutes

I needed to upgrade to a large waterwheel to drive the two millstones along with the fans.

Next things I'll probably try:

  • The lava farm is filling up nicely, so I can put some pipes underneath it and have a pump move lava into a basin for making andesite. That'll give me something to do with the flint from the iron nugget farm. I'll need to mine some copper first, though.
  • I can start mining for diamonds so that I can get obsidian and go to the Nether. I made a rope pulley + drills mining contraption, but (a) I think it got stuck after going past some lava and water, and (b) the shaft it made uncovered a deep dark biome, so… that's a little scary. Maybe I'll do some branch mining, or cave mining when the world is set to peaceful.
View org source for this post

Editing videos with Emacs and subed-record.el

| emacs, subed, video

I want to document more of my Minecraft adventures with A+. Video is a natural way to do this. It builds on her familiarity with the tutorials and streams she enjoys watching. I set up OBS on her laptop and plugged in my Blue Yeti microphone. We did our first interview yesterday. I edited and subtitled it (because why not!), uploaded it as an unlisted YouTube video, and shared it with her dad, sister, and cousins.

I did the video editing in Emacs with subed-record. First, I used WhisperX to transcribe the video, and I used subed-align to fix the timestamps with aeneas. I normalized the audio with Audacity and I exported the .opus file for use in subed-record.el. Then I added NOTE #+SKIP before times I wanted to remove, like when she asked for a retake. Here's what that subtitle markup looks like:

WEBVTT

NOTE #+SKIP

00:00:00.000 --> 00:00:16.679
And then I'll record in my side also
and we'll just put it in somehow.
Somehow. Okay. We can edit that, right?
Yeah, we'll learn how to edit things.
It'll be great.

NOTE
Introduction
#+AUDIO: cuberventures-001.opus
[[file:intro.webm]]
#+OUTPUT: cuberventures-001-fxnt-create-2-windmill-home-cafe-trains-hotel-half-underwater.webm

00:00:16.680 --> 00:00:19.399
Okay, so now we're here with <username>.

00:00:19.400 --> 00:00:23.039
I want to find out what you like about Minecraft and

00:00:23.040 --> 00:00:26.079
all the cool things that you have been building lately.

This was a little different from my usual video creation workflow, where I record the audio and the video separately. When I wrote subed-record.el, I assumed I'd edit the audio first, choose images/GIFs/videos that were already ready to go, and then combine those visuals with sections of audio, speeding things up or slowing things down as needed. Now I wanted to apply the same edits to the video as I did to the audio. A+ did a great job of looking at stuff in Minecraft while talking about them, so I wanted to keep her narration in sync. I added some code to allow me to specify a same-edits keyword for the visuals. That meant that I would use the same selection list that I used for cutting the audio. Here's what that subtitle markup looks like:

NOTE
[[file:2024-12-31 10-35-14.mkv]]
#+OPTIONS: same-edits

00:00:43.860 --> 00:00:45.941
Shall we take a tour of my world?

00:00:45.942 --> 00:00:50.079
Sure, let's tell people which mod pack this is.

00:00:50.080 --> 00:00:55.639
This is FXNT Create 2, also known as FoxyNoTail Create 2.

NOTE Windmill

00:00:55.640 --> 00:00:58.239
I've got this little bit of path leading to the interview

00:00:58.240 --> 00:01:01.839
room. This is my unfinished windmill. I've been meaning to

This workflow lets me cut out segments in the middle of the video, like this:

00:17:30.200 --> 00:17:33.119
great start for a tour. I'm looking forward to seeing what

00:17:33.120 --> 00:17:34.112
you will build next.

NOTE #+SKIP

00:17:34.113 --> 00:18:02.379
Do you have any last words before
we try to figure out this video editing thing?
Yeah. We'll cut that last part out.
Let's just do a retake on that last part.
Someday. Out here. Okay. There you go.
This is a beautiful view.

00:18:02.380 --> 00:18:08.119
The last things I want to say about this world is there'll be

I also wanted to start the video with a segment from my recording, so we could see her avatar on screen during the introduction. She kept her computer on first-person POV instead of changing the camera. I used mpv to figure out the timestamps for the start and end of the part that I wanted to use, then I used ffmpeg to cut that clip. I added a comment with a link to that video in order to use it before the main video. That's the [[file:intro.webm]] in the first section's comments.

After testing a small section of the transcript by selecting a region and using subed-record-compile-video, I deselected the region and used subed-record-compile-video to produce the whole video.

I also modified subed-record-compile-subtitles to include the other non-directive comments, so I can include the section headings in the raw VTT file and have them turn up in the exported version. Then I can use the new subed-section-comments-as-chapters command to copy those as chapters for the YouTube description.

We're not going to share that particular video yet, but I'm looking forward to trying that technique with videos about stuff I'm figuring out in Minecraft or Emacs. It's also tempting me to think about ways to specify transitions like crossfades and other fancy effects like overlays.

I like using the transcript as the starting point for video editing. It just makes sense to me to work with it as text. I also like this experiment with documenting more of our Minecraft experimentation. It seems to get her talking and encourages her to build more. I'm looking forward to learning more about Minecraft and making videos too.

We did another video today using the new shortcuts I've just set up for toggling OBS recording. This time we didn't even need to do any editing. I used Org Export to make her a little HTML file that had the two videos on it, so she can review it any time. Onward!

View org source for this post

2024-12-30 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Hacker News, lobste.rs, kbin, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at [email protected]. Thank you!

View org source for this post

Monthly review: November 2024

| monthly, review

This month, I experimented with doing my daily drawings on a calendar grid. Since that meant I had a nice neat one-page summary of the month right there, I figured I might as well resume writing these monthly reviews.

Most of my discretionary time was taken up by preparations for EmacsConf, which was a lot of fun. The main things were adding to our organizers notebook and figuring out our own BigBlueButton installation.

We still had plenty of time to get outside to the playground, go for walks and bike adventures, go skating, and make wontons. On indoor days, we mostly played Minecraft, Ni No Kuni, and Supermarket Together. Now that her usual playgroup's shifting mostly indoors (and tend to be pretty cough-y when they're outdoors), we've been going to the ice rink instead, and have even had a couple of playdates with new friends.

A+ definitely craves more stimulation during virtual school. The teacher suggested micro:bit programming, and we've been having fun making simple programs to run on actual hardware. A+ has also been learning turtle graphics via Python programming, and she's quite proud of programming by typing instead of using blocks. She passed the first stage of the gifted identifaction process in the public school board, so we had a couple of meetings and I scrambled to do some research. I don't think it'll change much. The Toronto District School Board doesn't offer virtual placements for gifted students even if they do identify an exceptionality, and there probably isn't anything in their budget for extra resources for the self-contained virtual school they're setting up next year. Ah well. We're planning to take a very chill, non-tiger-parenting approach to the whole thing, and we'll just have to see how things work out.

We set up a new desk for A+ near the window, which let her enjoy more sunlight during the day. In return, I got to have her old desk setup, so now I can sometimes get computer time with an extra monitor (at least when I'm not helping her stave off boredom).

It was the last month before W- retired, so we squeezed in a few dental and medical appointments to take advantage of the remaining coverage. Now we get to figure out what our days could be like!

Blog posts

Sketches

Time

Category Previous month % This month % Diff % h/wk Diff h/wk
A+ 30.3 39.3 8.9 63.9 15.0
Personal 8.3 10.9 2.6 17.8 4.3
Discretionary - Play 0.2 0.0 -0.2 0.0 -0.3
Unpaid work 3.5 2.9 -0.5 4.8 -0.9
Discretionary - Family 1.0 0.2 -0.8 0.3 -1.3
Sleep 35.8 33.5 -2.3 54.6 -3.8
Business 4.1 0.3 -3.7 0.5 -6.3
Discretionary - Productive 16.9 12.9 -4.0 21.0 -6.7
View org source for this post

Linking to Org Babel source in a comment, and making that always use file links

| emacs, org

I've been experimenting with these default header args for Org Babel source blocks.

(setq org-babel-default-header-args
      '((:session . "none")
        (:results . "drawer replace")
        (:comments . "link")  ;; add a link to the original source
        (:exports . "both")
        (:cache . "no")
        (:eval . "never-export") ;; explicitly evaluate blocks instead of evaluating them during export
        (:hlines . "no")
        (:tangle . "no"))) ;; I have to explicitly set up blocks for tangling

In particular, :comments link adds a comment before each source block with a link to the file it came from. This allows me to quickly jump to the actual definition. It also lets me use org-babel-detangle to copy changes back to my Org file.

I also have a custom link type to make it easier to link to sections of my configuration file (Links to my config). Org Mode prompts for the link type to use when more than one function returns a link for storing, so that was interrupting my tangling with lots of interactive prompts. The following piece of advice ignores all the custom link types when tangling the link reference. That way, the link reference always uses the file: link instead of offering my custom link types.

(advice-add #'org-babel-tangle--unbracketed-link
            :around (lambda (old-fun &rest args)
                      (let (org-link-parameters)
                        (apply old-fun args))))
This is part of my Emacs configuration.
View org source for this post

EmacsConf 2024 notes

Posted: - Modified: | emacs, emacsconf
  • [2025-01-02 Thu]: Add numbers based on latest Linode invoice.
  • [2024-12-28 Sat]: Added talk and Q&A count, added note about BBB max simultaneous users, added note about BBB, added thanks

The videos have been uploaded, thank-you notes have been sent, and the kiddo has decided to play a little Minecraft on her own, so now I get to write some quick notes on EmacsConf 2024.

Stats

Talks 31
Hours 10.7
Q&A web conferences 21
Hours 7.8
  • Saturday:
    • gen: 177 peak + 14 peak lowres
    • dev: 226 peak + 79 peak lowres
  • Sunday:
    • gen: 89 peak + 10 peak lowres

Server configuration:

meet 16GB 8core dedicated peak 409% CPU (100% is 1 CPU), average 69.4%
front 32GB 8core shared peak 70.66% CPU (100% is 1 CPU)
live 64GB 16core shared peak 552% CPU (100% is 1 CPU) average 144%
res 46GB 12core peak 81.54% total CPU (100% is 12 CPUs); each OBS ~250%), mem 7GB used
media 3GB 1core  

YouTube livestream stats:

Shift Peak Avg
Gen Sat AM 46 28
Gen Sat PM 24 16
Dev Sat AM 15 7
Dev Sat PM 20 12
Gen Sun AM 28 17
Gen Sun PM 26 18

Timeline

Call for proposals [2024-06-30 Sun]
CFP deadline [2024-09-20 Fri]
Speaker notifications [2024-09-27 Fri]
Publish schedule [2024-10-25 Fri]
Video target date [2024-11-08 Fri]
EmacsConf [2024-12-07 Sat]-[2024-12-07 Sat]

We did early acceptances again this year. That was nice. I wasn't sure about committing longer periods of time early in the scheduling process, so I usually tried to nudge people to plan a 20-minute video with the option of possibly doing more, and I okayed longer talks once we figured out what the schedule looked like.

There were 82 days between the call for proposals and the CFP deadline, another 49 days from that to the video target date, and 29 days between the video target date and EmacsConf. It felt like there was a good amount of time for proposals and videos. Six videos came in before or on the target date. The rest trickled in afterwards, which was fine because we wanted to keep things low-pressure for the speakers. We had enough capacity to process and caption the videos as they came in.

Data

We continued to use an Org file to store the talk information. It would be great to add some validation functions:

  • Check permissions and ownership for files
  • Check case sensitivity for Q&A type detection
  • Check BBB redirect pages to make sure they exist
  • Check transcripts for ` because that messes up formatting; consider escaping for the wiki
  • Check files are public and readable
  • Check captioned by comment vs caption status vs captioner

Speakers uploaded their files via PsiTransfer again. I didn't get around to setting up the FTP server. I should probably rename ftp-upload.emacsconf.org to upload.emacsconf.org so that people don't get confused.

Communication

As usual, we announced the EmacsConf call for proposals on emacs-tangents, Emacs News, emacsconf-discuss, emacsconf-org, https://reddit.com/r/emacs. System Crafters, Irreal, and Emacs APAC, mentioned it, and people also posted about EmacsConf on Mastodon, X, BlueSky, and Facebook. @[email protected] suggested submitting EmacsConf to https://foss.events, so I did. There was some other EmacsConf-related discussions in r/emacs. 200ok and Ardeo organized an in-person meetup in Switzerland, and emacs.si got together in Ljubljana.

For communicating with speakers and volunteers, I used lots of mail merge (emacsconf-mail.el). Most of the templates only needed a little tweaking from last year's code. I added a function to help me double-check delivery, since the batches that I tried to send via async sometimes ran into errors.

Next time, I think it could be interesting to add more blog posts and Mastodon toots.

Also, maybe it would be good to get in touch with podcasts like

to give a heads up on EmacsConf before it happens and also let them know when videos are available.

We continued to use Mumble for backstage coordination. It worked out well.

Schedule

The schedule worked out to two days of talks, with two tracks on the first day, and about 15-20 minutes between each talk. We were able to adapt to late submissions, last-minute cancellations, and last-minute switches from Q&A to live.

We added an open mic session on Sunday to fill in the time from a last-minute cancellation. That worked out nicely and it might be a good idea to schedule in that time next year. It was also good to move some of the usual closing remarks earlier. We were able to wrap up in a timely manner, which was great for some hosts and participants because they didn't have to stay up so late.

Sunday was single-track, so it was nice and relaxed. I was a little worried that people might get bored if the current talk wasn't relevant to their interests, but everyone managed just fine. I probably should have remembered that Emacs people are good at turning extra time into more configuration tweaks.

Most of the scheduling was determined by people's time constraints, so I didn't worry too much about making the talks flow logically. I accidentally forgot to note down one speaker's time constraints, but he caught it when we e-mailed the draft schedule and I was able to move things around for a better time for him.

There was a tiny bit of technical confusion because the automated schedule publishing on res had case-sensitive matching (case-fold-search was set to nil), so if a talk was set to "Live" Q&A, it didn't announce it as a live talk because it was looking for live. Whoops. I've added that configuration setting to my emacsconf-stream-config.el, so the ansible scripts should get it next time.

I asked Leo and Corwin if they wanted to manually control the talks this year. They opted to leave it automatically managed by crontab so that they wouldn't have to worry as much about timekeeping. It worked reliably. Hooray for automation! The only scheduling hiccup was because I turned off the crontab so that we could do Saturday closing remarks when we wanted to and I forgot to reenable autopilot the next day. We noticed when the opening remarks didn't start right on the dot, and I got everything back on track.

Like last year, I scheduled the dev track to start a little later than the gen track. That made for a less frantic morning. Also, this year we scheduled Sunday morning to start with more IRC Q&A instead of live Q&A. We didn't notice any bandwidth issues on Sunday morning this time.

It would be nice to have Javascript countdowns in some kind of web interface to make it easier for hosts, especially if we can update it with the actual time the current video will end in MPV.

I can also update the emacsconf-stream.el code to make it easier to automatically count down to the next talk or to a specific talk.

We have Javascript showing local time on the individual talk pages, but it would be nice to localize the times on all the schedule/watch pages too.

Most of my stuff (scheduling, publishing, etc.) is handled by automation with just a little bit of manual nudging every so often, so it might be possible to organize an event that's more friendly to Europe/APAC timezones.

Recorded videos

As usual, we strongly encouraged speakers to record videos to lower everyone's stress levels and allow for captioning by volunteers, so that's what most speakers did. We were able to handle a few last-minute submissions as well as a live talk. Getting videos also meant we could publish them as each talk went live, including automatically putting the videos and transcripts on the wiki.

We didn't have obvious video encoding cut-offs, so re-encoding in a screen was a reliable way to avoid interruptions this year. Also, no one complained about tiny text or low resolution, so the talk preparation instructions seem to be working out.

Automatically normalizing the audio with ffmpeg-normalize didn't work out, so Leo Vivier did a last-minute scramble to normalize the audio the day before the conference. Maybe that's something that volunteers can help with during the lead-up to the conference, or maybe I can finally figure out how to fit that into my process. I don't have much time or patience to listen to things, but it would be nice to get that sorted out early.

Next year we can try remixing the audio to mono. One of the talks had some audio moving around, which was a little distracting. Also, some people listen to the talks in one ear, so it would be good to drop things down to mono for them.

We think 60fps videos stressed the res server a bit, resulting in dropped frames. Next year, we can downsample those to 30fps and add a note to the talk preparation instructions. The hosts also suggested looking into setting up streaming from each host's computer instead of using our shared VNC sessions.

There was some colour smearing and weirdness when we played some videos with mpv on res. Upgrading MPV to v0.38 fixed it.

Some people requested dark mode (light text on dark background), so maybe we can experiment with recommending that next year.

I did a last-minute change to the shell scripts to load resources from the cache directory instead of the assets/stream directory, but I didn't get all of the file references, so sometimes the test videos played or the introductions didn't have captions. On the plus side, I learned how to use j in MPV to reload a subtitle file.

Sometimes we needed to play the videos manually. If we get the hang of starting MPV in a screen or tmux session, it might be easier for hosts to check how much time is left, or to restart a video at a specific point if needed. Leo said he'll work on figuring out the configuration and the Lua scripts.

I uploaded all the videos to YouTube and scheduled them. That was nice because then I didn't have to keep updating things during the conference. It turns out that Toobnix also has a way to schedule uploads. I just need to upload it as unlisted first, and then choose Scheduled from the visibility. I wonder if peertube-cli can be extended to schedule things. Anyway, since I didn't know about that during the conference, I just used emacsconf-publish-upload-talk function to upload videos.

It was fun playing Interview with an Emacs Enthusiast in 2023 [Colorized] - YouTube at lunch. I put together some captions for it after the conference, so maybe we can play it with captions next year.

Recorded introductions

We record introductions so that hosts don't have to worry about how to say things on air. I should probably send the intro check e-mail earlier–maybe on the original video target date, even if speakers haven't submitted their videos yet. This will reduce the last-minute scramble to correct intros.

When I switched the shell scripts to use the cache directory, I forgot to get it to do the intros from that directory as well, so some of the uncorrected intros were played.

I forgot to copy the intro VTTs to the cache directory. This should be handled by the subed-record process for creating intros, so it'll be all sorted out next year.

Captioning

We used WhisperX for speech-to-text this year. It did a great job at preparing the first drafts of captions that our wonderful army of volunteer captioners could then edit. WhisperX's built-in voice activity detection cut down a lot on the hallucinations that OpenAI Whisper had during periods of silence in last year's captions, and there was only one instance of WhisperX missing a chunk of text from a speaker that I needed to manually fill in. I upgraded to a Lenovo P52 with 64GB RAM, so I was able to handle last-minute caption processing on my computer. It might be handy to have a smaller model ready for those last-minute requests, or have something ready to go for the commercial APIs.

The timestamps were a little bit off. It was really helpful that speakers and volunteers used the backstage area to check video quality. I used Aeneas to re-align the text, but Aeneas was also confused by silences. I've added some code to subed so that I can realign regions of subtitles using Aeneas or WhisperX timestamps, and I also wrote some code to skim timestamps for easy verification.

Anush V experimented with using machine learning for subtitle segmentation, so that might be something to explore going forward.

BigBlueButton web conference

This year we set up a new BigBlueButton web conferencing server. The server with our previous BigBlueButton instance had been donated by a defunct nonprofit, so it finally got removed on October 27. After investigating whether Jitsi or Galene might be a good fit for EmacsConf, we decided to continue with BigBlueButton. There were some concerns about non-free Mongo for BBB versions >= 2.3 and < 3, so I installed BBB 3.0. This was hard to get working on a Docker on the existing res server. We decided it was worth spinning up an additional Linode virtual private server. It turned out that BBB refused to run on anything smaller than 8GB/4core, so I scaled up to that during testing, scaled back down to 1GB/1core in between, and scaled up to 16GB/8core dedicated during the conference.

I'm still not 100% sure I set everything up correctly or that everything was stable. Maybe next year BBB 3.0 will be better-tested, someone more sysad-y can doublecheck the setup, or we can try Galene.

One of the benefits of upgrading to BBB 3.0 was that we could use the smart layout feature to drag the webcam thumbnails to the side of the shared screen. This made shared screens much easier to read. I haven't automated this yet, but it was easy enough for us to do via the shared VNC session.

On the plus side, it was pretty straightforward to use the Rails console to create all the rooms. We used moderator access codes to give all the speakers moderator access. Mysteriously, superadmins didn't automatically have moderator access to all the rooms even if they were logged in, so we needed to add host access by hand so that they could start the recordings.

Since we self-hosted and were budgeting more for the full-scale node, I didn't feel comfortable scaling it up to production size until a few days before the conference. I sent the access codes with the check-in e-mails to give speakers time to try things out.

Compared to last year's stats:

  2023 2024
Max number of simultaneous users 62 107
Max number of simultaneous meetings 6 7
Max number of people in one meeting 27 25
Total unique people 84 102
Total unique talking 36 40

(Max number of simultaneous users wasn't deduplicated, since we need that number for server load planning)

Tech checks and hosting

FlowyCoder did a great job getting everyone checked in, especially once I figured out the right checklist to use. We used people's emergency contact information a couple of times.

Corwin and Leo were able to jump in and out of the different streams for hosting. Sometimes they were both in the same Q&A session, which made it more conversational especially when they were covering for technical issues. We had a couple of crashes even though the tech checks went fine, so that was weird. Maybe something's up with BBB 3.0 or how I set it up.

Next time, we can consider asking speakers what kind of facilitation style they like. A chatty host? Someone who focuses on reading the questions and then gets out of the way? Speakers reading their own questions and the host focusing on timekeeping/troubleshooting?

Streaming

I experimented with setting up the live0 streaming node as a 64GB 32core dedicated CPU server, but that was overkill, so we went back down to 64GB 16core and it still didn't approach the CPU limits.

The 480p stream seemed stable, hooray! I had set it up last year to automatically kick in as soon as I started streaming to Icecast, and that worked out. I think I changed a loop to be while true instead of making it try 5 times, so that probably helped.

I couldn't get Toobnix livestreaming to work this year. On the plus side, that meant that I could use OBS to directly stream to YouTube instead of trying to set up multicasting. I set up one YouTube livestreaming event for each shift and added the RTMP keys to our shift checklists so that I could update the settings before starting the stream. That was pretty straightforward.

This year, I wrote a little randomizer function to display things on the countdown screen. At first I just dumped in https://www.gnu.org/fun/jokes/gnuemacs.acro.exp.en.html, but some of those were not quite what I was looking for. (… Probably should've read them all first!) Then I added random packages from GNU ELPA and NonGNU ELPA, and that was more fun. I might add MELPA next time too. The code for dumping random packages is probably worth putting into a different blog post, since it's the sort of thing people might like to add to their dashboards or screensavers.

I ran into some C-s annoyances in screen even with flow control turned off, so it might be a good idea to switch to tmux instead of screen.

Next year, I think it might be a good idea to make intro images for each talk. Then we can use that as the opening slide in BigBlueButton (unless they're already sharing something else) as well as a video thumbnail.

Publishing

The automated process for publishing talks and transcripts to the wiki occasionally needed nudging when someone else had committed a change to the wiki. I thought I had a git pull in there somewhere, but maybe I need to look at it some more.

I forgot to switch the conference publishing phase and enable the inclusion of Etherpads, but fortunately Ihor noticed. I did some last-minute hacking to add them in, and then I remembered the variables I needed to set. Just need to add it to our process documentation.

Etherpad

We used Etherpad 1.9.7 to collect Q&A again this year. I didn't upgrade to Etherpad v2.x because I couldn't figure out how to get it running within the time I set aside for it, but maybe that's something for next year.

I wrote some Elisp to copy the current ERC line (unwrapped) for easier pasting into Etherpad. That worked out really well, and it let me keep up with copying questions from IRC to the pad in between other bits of running around. (emacsconf-erc-copy in emacsconf-erc.el)

Next year, I'll add pronouns and pronunciations to the Etherpad template so that hosts can refer to them easily.

If I rejig the template to move the next/previous links so that notes can be added to the end, I might be able to use the Etherpad API to add text from IRC.

IRC

We remembered to give the libera.chat people a heads-up before the conference, so we didn't run into usage limits for https://chat.emacsconf.org. Yay!

Aside from writing emacsconf-erc-copy (emacsconf-erc.el) to make it easier to add text from IRC to the Etherpad, I didn't tinker much with the IRC setup for this year. It continued to be a solid platform for discussion.

I think a keyboard shortcut for inserting a talk's URL could be handy and should be pretty easy to add to my Embark keymap.

Extracting the Q&A

We sometimes forgot to start the recording for the Q&A until a few minutes into the talk. I considered extracting the Q&A recordings from the Icecast dump or YouTube stream recordings in order to get those first few minutes, but decided it wasn't worth it since people could generally figure out the answers.

Getting the recordings off BigBlueButton was easier this year because I configured it with video as an additional processing format, so we could grab one file per session instead of combining the different streams with ffmpeg.

I did a quick pass of the Q&A transcripts and chat logs to see if people mentioned anything that they might want to take out. I also copied IRC messages and the pads, and I copied over the answers from the transcripts using the new emacsconf-extract-subed-copy-section-text function.

Audio mixing was uneven. It might be nice to figure out separate audio recordings just in case (#12302, bigbluebutton-dev). We ended up not tinkering with the audio for the Q&A, so next time, I can probably upload them without waiting to see if anyone wants to fiddle with the audio.

Trimming the Q&A was pretty straightforward. I added a subed-crop-media-file function to subed so that I can trim files easily.

Thanks to my completion functions for adding section headings based on comments, it was easy to index the Q&A this year. I didn't even put it up backstage for people to work on.

Nudged by @ctietze, I'm experimenting with adding sticky videos if Javascript is enabled so that it's easier to navigate using the transcript. There's still a bit of tinkering to do, but it's a start.

I added some conference-related variables to a .dir-locals.el file so that I can more easily update things even for past conferences. This is mostly related to publishing the captions on the wiki pages, which I do with Emacs Lisp.

Budget and donations

Costs (USD, not including 13% tax):

52.54 Extra costs for hosting in December
3.11 Extra costs for BBB testing in November
120 Hosting costs year-round (two Linode nanodes)

Total of USD 175.65 + tax, or USD 198.48 for 2024.

The Free Software Foundation also provided media.emacsconf.org for serving media files. Ry P provided res.emacsconf.org for OBS streaming over VNC sessions.

Amin Bandali was away during the conference weekend and no one else knew how to get the list of donors and current donation stats from the FSF Working Together program on short notice. Next time, we can get that sorted out beforehand so that we can thank donors properly.

Documentation and time

I think my biggest challenge was having less time to prepare for EmacsConf this year because the kiddo wanted more of my attention. In many ways, the automation that I'd been gradually building up paid off. We were able to pull together EmacsConf even though I had limited focus time.

Here's my Emacs-related time data (including Emacs News and tweaking my config):

Year Jan Feb March April May June July Aug Sept Oct Nov Dec Total
2023 23.4 15.9 16.2 11.2 4.4 11.5 6.5 13.3 36.6 86.6 93.2 113.0 432
2024 71.2 12.0 5.6 6.6 3.3 9.6 11.0 4.7 36.0 40.3 52.3 67.7 320

(and here's a longer-term analysis going back to 2012.)

I spent 92.6 hours total in October and November 2024 doing Emacs-related things, compared to 179.8 hours the previous year – so, around half the time. Part of the 2023 total was related to preparing my presentation for EmacsConf, so I was much more familiar with my scripts then. Apparently, there was still a lot more that I needed to document. As I scrambled to get EmacsConf sorted out, I captured quick tasks/notes for the things I need to add to our organizers notebook. Now I get to go through all those notes in my inbox. Maybe next year will be even smoother.

On the plus side, all the process-related improvements meant that the other volunteers could jump in pretty much whenever they wanted, including during the conference itself. I didn't want to impose firm commitments on people or bug them too much by e-mail, so we kept things very chill in terms of scheduling and planning. If people were available, we had stuff people could help with. If people were busy, that was fine, we could manage. This was nice, especially when I applied the same sort of chill approach to myself.

I'd like to eventually get to the point of being able to mostly follow my checklists and notes from the start of the conference planning process to the end. I've been moving notes from year-specific organizer notebooks to the main organizers' notebook. I plan to keep that one as the main file for notes and processes, and then to have specific dates and notes in the yearly ones.

Thanks

  • Thank you to all the speakers, volunteers, and participants, and to all those other people in our lives who make it possible through time and support.
  • Thanks to Leo Vivier and Corwin Brust for hosting the sessions, and to FlowyCoder for checking people in.
  • Thanks to our proposal review volunteers James Howell, JC Helary, and others for helping with the early acceptance process.
  • Thanks to our captioning volunteers: Mark Lewin, Rodrigo Morales, Anush, annona, and James Howell, and some speakers who captioned their own talks.
  • Thanks to Leo Vivier for fiddling with the audio to get things nicely synced.
  • Thanks to volunteers who kept the mailing lists free from spam.
  • Thanks to Bhavin Gandhi, Christopher Howard, Joseph Turner, and screwlisp for quality-checking.
  • Thanks to shoshin for the music.
  • Thanks to Amin Bandali for help with infrastructure and communication.
  • Thanks to Ry P for the server that we're using for OBS streaming and for processing videos.
  • Thanks to the Free Software Foundation for Emacs itself, the mailing lists, the media.emacsconf.org server, and handling donations on our behalf through the FSF Working Together program. https://www.fsf.org/working-together/fund
  • Thanks to the many users and contributers and project teams that create all the awesome free software we use, especially: BigBlueButton, Etherpad, Icecast, OBS, TheLounge, libera.chat, ffmpeg, OpenAI Whisper, WhisperX, the aeneas forced alignment tool, PsiTransfer, subed, and many, many other tools and services we used to prepare and host this years conference
  • Thanks to everyone!

Overall

Good experience. Lots of fun. I'd love to do it again next year. EmacsConf feels like a nice, cozy get-together where people share the cool things they've been working on and thinking about. People had fun! They said:

  • "emacsconf is absolutely knocking it out of the park when it comes to conference logistics"
  • "I think this conference has defined the terms for a successful online conference."
  • "EmacsConf is one of the big highlights of my year every year. Thank you a ton for running this 😊"

It's one of the highlights of my year too. =) Looking forward to the next one!

In the meantime, y'all can stay connected via Emacs News, meetups (online and in person), Planet Emacslife, and now emacs.tv. Enjoy!

p.s. I'd love to learn from other people's conference blog posts, EmacsConf or otherwise. I'm particularly interested in virtual conferences and how we can tinker with them to make them even better. I'm having a hard time finding posts; please feel free to send me links to ones you've liked or written!

View org source for this post