Jekyll2023-05-06T01:01:43+00:00http://salewski.email/feed.xmlAlan D. Salewski (writings)Al's misc. thoughts, notes, and other writings Left-hand ENTER key2020-09-22T00:00:00+00:002020-09-22T00:00:00+00:00http://salewski.email/2020/09/22/left-hand-enter-key<p>(Updated: 2023-05-05 to fix some typos)</p> <h1 id="remap-caps-lock-key-as-left-hand-enter">Remap “Caps Lock” key as left-hand ENTER</h1> <p>[ This post is relevant primarily to right-handed folks, and to left-handed folks who use a right-handed computer setup (you use the mouse with your right hand). ]</p> <p>I found myself working on a one-off task that involved a fair amount of copy/paste work in my terminal emulator, building up command lines with little tidbits of data gathered from here and there, without any typing (except to hit the ENTER key to run the command).</p> <p>I am using <a href="https://en.wikipedia.org/wiki/X_Window_System" title="wikipedia: X Window System">X11</a>, so selecting the source text tidbits is enough to “copy” it, and clicking the middle mouse button pastes it in the target window at the location of the cursor. So far, so good. Once the command line was ready I pressed the <code class="language-plaintext highlighter-rouge">ENTER</code> key (with my right hand), and then moved on to the next one.</p> <p>I found that moving my right hand from the mouse to the <code class="language-plaintext highlighter-rouge">ENTER</code> key and then back to the mouse again was slowing me down, and the motion was generally irritating.</p> <p>It occurred to me that it would be nice if there were an <code class="language-plaintext highlighter-rouge">ENTER</code> key within reach of my left hand, and that the <code class="language-plaintext highlighter-rouge">Caps Lock</code> key would be a good candidate for remapping for that purpose.</p> <hr /> <h1 id="tldr">TL;DR</h1> <p>To turn your <code class="language-plaintext highlighter-rouge">Caps Lock</code> key into an <code class="language-plaintext highlighter-rouge">ENTER</code> key, run these two commands in a shell within your X11 session. If you are happy with the result, then add the commands to your <code class="language-plaintext highlighter-rouge">~/.xsession</code> file (or your local system equivalent) to have them take effect the next time you restart <code class="language-plaintext highlighter-rouge">X</code>.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -e 'clear Lock' $ xmodmap -e 'keycode 66 = Return NoSymbol Return' </code></pre></div></div> <p>For best results, read the rest of the article first, though; <em>the <code class="language-plaintext highlighter-rouge">Caps Lock</code> key on your keyboard might not be mapped to keycode 66</em>. If that is the case, the above will just disable your <code class="language-plaintext highlighter-rouge">Caps Lock</code> key and turn whatever key that is mapped to keycode 66 into an <code class="language-plaintext highlighter-rouge">ENTER</code> key. Also described below is an approach to preserving the mapping when keyboards are hotplugged to the system (e.g., USB keyboards).</p> <hr /> <h1 id="background">Background</h1> <p>A typical 104- or 105-key US keyboard has <em>two</em> <code class="language-plaintext highlighter-rouge">ENTER</code> keys reachable with your right hand:</p> <ol> <li> <p>one to the right side of the home row, and</p> </li> <li> <p>one on the right edge of the number pad.</p> </li> </ol> <p>However, there are not any reachable from your left hand.</p> <p><strong>Since the <code class="language-plaintext highlighter-rouge">Caps Lock</code> key is useless anyway, it is a good candidate to re-map as a left-hand <code class="language-plaintext highlighter-rouge">ENTER</code> key. Having that allows you to keep your right hand on the mouse and use your left hand to press the <code class="language-plaintext highlighter-rouge">ENTER</code> key, when needed.</strong></p> <hr /> <h1 id="gather-tools-and-data">Gather tools and data</h1> <p>In order to remap the keyboard <code class="language-plaintext highlighter-rouge">Caps Lock</code> key, you will need a couple of pieces of information and two programs: <a href="https://manpages.debian.org/xev" title="xev(1)"><code class="language-plaintext highlighter-rouge">xev(1)</code></a> and <a href="https://manpages.debian.org/xmodmap" title="xmodmap(1)"><code class="language-plaintext highlighter-rouge">xmodmap(1)</code></a>. On <a href="https://www.debian.org/" title="www.debian.org">Debian</a>-based systems, the <code class="language-plaintext highlighter-rouge">xev</code> program is provided by the <a href="https://tracker.debian.org/pkg/x11-utils" title="pkg: x11-utils"><code class="language-plaintext highlighter-rouge">x11-utils</code></a> package, and the <code class="language-plaintext highlighter-rouge">xmodmap</code> program is provided by the <a href="https://tracker.debian.org/pkg/x11-xserver-utils" title="pkg: x11-xserver-utils"><code class="language-plaintext highlighter-rouge">x11-xserver-utils</code></a> package. You may have one or both of them already, but if not:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> # apt-get -u install x11-utils x11-xserver-utils </code></pre></div></div> <p>The pieces of information needed include the “keycodes” for the “Caps Lock” and “Enter” keys, as well as the mapping (to “keysyms”) in effect for the “Enter” key (so we can copy it for use with the “Caps Lock” key).</p> <p>Use <code class="language-plaintext highlighter-rouge">xev</code> to obtain the keycodes. Here’s what I saw initially for my “Caps Lock” key:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>KeyPress event, serial 31, synthetic NO, window 0xd400001, root 0x1ab, subw 0x0, time 4049429559, (169,-13), root:(309,938), state 0x10, keycode 66 (keysym 0xffe5, Caps_Lock), same_screen YES, XLookupString gives 0 bytes: XmbLookupString gives 0 bytes: XFilterEvent returns: False KeyRelease event, serial 34, synthetic NO, window 0xd400001, root 0x1ab, subw 0x0, time 4049429670, (169,-13), root:(309,938), state 0x12, keycode 66 (keysym 0xffe5, Caps_Lock), same_screen YES, XLookupString gives 0 bytes: XFilterEvent returns: False </code></pre></div></div> <p>So that tells us that our “Caps Lock” key is mapped to keycode 66. A similar query for the “Enter” key told me it was mapped to keycode 36.</p> <p>Next we will use <code class="language-plaintext highlighter-rouge">xmodmap</code> to obtain the keycode to keysyms mapping, and also the current state of the <code class="language-plaintext highlighter-rouge">Lock</code> modifier (from the “modifier map”).</p> <p>Since we will make reference below to clearing the lock modifier, it is useful to note the initial state of it:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -pm | grep '^lock' lock Caps_Lock (0x42) </code></pre></div></div> <p><em>[ The <code class="language-plaintext highlighter-rouge">-pm</code> option causes <code class="language-plaintext highlighter-rouge">xmodmap</code> to show us the current modifier map. ]</em></p> <p>The next thing to find out was what the xmodmap setting for the “Enter” key looked like. That was done by browsing the output of the command:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -pke | less </code></pre></div></div> <p><em>[ The <code class="language-plaintext highlighter-rouge">-pke</code> option causes <code class="language-plaintext highlighter-rouge">xmodmap</code> to print on <code class="language-plaintext highlighter-rouge">stdout</code> the current keymap table, in the form of expressions that can be fed back into to <code class="language-plaintext highlighter-rouge">xmodmap</code>. ]</em></p> <p>The two lines in that output of interest here were:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> keycode 36 = Return NoSymbol Return keycode 66 = Caps_Lock NoSymbol Caps_Lock </code></pre></div></div> <p>The first shows the setting for the “Enter” key, and the second is the initial setting for the “Caps Lock” key.</p> <hr /> <h1 id="lets-do-it">Let’s do it</h1> <p>To turn our “Caps Lock” key into a left-hand “Enter” key, we need to do two things:</p> <ol> <li> <p>Remove the lock modifier for the “Caps Lock” key.</p> </li> <li> <p>Re-map the “Caps Lock” key’s keycode to the “keysyms” appropriate for an “Enter” key.</p> </li> </ol> <p>Step (1) is accomplished on the live system by using the command<sup><a href="#aside-why-non-surgical">1</a></sup>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -e 'clear Lock' </code></pre></div></div> <div style="margin: auto; max-width: 80%; padding: 20px; font-size: smaller; text-align: justify; border: 1px solid #e5e5e5; background-color: #e4e6e8;"> <aside> <p> <a id="aside-why-non-surgical"></a> [1] Note that a more surgical approach to the above action would be to run the command:</p> <pre style="font-size: smaller;"> $ xmodmap -e 'remove Lock = Caps_Lock' </pre> <p>Indeed, that was what I was using initially, but it came with complications.</p> <p>The first (and easily dealt with) complication is that upon (re)attaching a USB keyboard to the machine, we could not know in advance the state of the lock modifier. We might find that the lock modifier is not applied to any keysym, or maybe it is applied to the 'Return' keysym, or perhaps applied to both 'Caps_Lock' and 'Return' keysyms, or some other variation. Just blindly running the above command would fail if the lock modifier was not currently set for the 'Caps_Lock' keysym.</p> <p>So to use the more specific command requires first detecting the current state and then taking appropriate action. For example (in bash):</p> <pre style="font-size: smaller;"> current_lock_assignment=$(xmodmap -pm | grep -i '^lock[[:space:]]') shopt -q -s nocasematch if [[ "${current_lock_assignment}" =~ Caps_Lock ]]; then xmodmap -e 'remove Lock = Caps_Lock' fi if [[ "${current_lock_assignment}" =~ Return ]]; then xmodmap -e 'remove Lock = Return' fi </pre> <p>Depending on other factors in your machine configuration, the above may work for you "as is". If so, it is probably a better approach than the approach I ultimately took because it expresses precisely the desired configuration.</p> <p>In my environment, however, I have another complication: There is something else in the mix also responding to the "keyboard enabled" events, and it is conflicting with my use of the above. Oddly enough, it only affects me <em>every other time</em> I plug the keyboard in. The observed behavior is that the lock modifier is applied to the 'Return' keysym, and though the "Caps Lock" key still behaves as an "Enter" key, it also still toggles the letter case with each press.</p> <p>I don't have time at the moment to track down the competing mechanism, but it does not rear its head (in any obvious way) when using the <code style="font-size: smaller;">'clear Lock'</code> expression. So that is what I am going with, at least for now. Also, using <code style="font-size: smaller;">'clear Lock'</code> is not a compromise of any functionality; I did not use lock for anything other than "Caps Lock" (the "Num Lock" key uses a different modifier (<code style="font-size: smaller;">'mod2'</code>)).</p> </aside> </div> <p><br /></p> <p>Step (2) is accomplished on the live system by using the command:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -e 'keycode 66 = Return NoSymbol Return' </code></pre></div></div> <p>Congratulations, your “Caps Lock” key now functions as an “Enter” key. At this point, your “Caps Lock” should have stopped having any letter case-altering effect, and should have no locking (“toggle”) effect, either. If your keyboard has an LED light that used to turn on when your “Caps Lock” key was in the lock position, you will find that the LED is no longer lit when the “Caps Lock” key is pressed (or even held down).</p> <hr /> <h1 id="make-permanent">Make permanent</h1> <p>The above settings are transient as currently configured. If you were to restart <code class="language-plaintext highlighter-rouge">X</code> or reboot your machine (or something similar) the old settings would come back when <code class="language-plaintext highlighter-rouge">X</code> gets reinitialized. To have the new settings take effect automatically in your next <code class="language-plaintext highlighter-rouge">X</code> session, there are two different approaches you might consider:</p> <ol> <li> <p>The first (and simpler) approach would be to add the commands exactly as typed above to your <a href="https://unix.stackexchange.com/questions/47359/what-is-xsession-for" title="What is '.xsession' for?"><code class="language-plaintext highlighter-rouge">~/.xsession</code></a> file (or local system equivalent). If your keyboard is always plugged into the machine (typical for a desktop) then that approach is good enough.</p> </li> <li> <p>The second, more elaborate, approach would be to take additional steps to allow the settings to take effect not only upon re-initializing <code class="language-plaintext highlighter-rouge">X</code>, but also when hotplugging the keyboard.</p> </li> </ol> <hr /> <h1 id="make-dynamic">Make dynamic</h1> <p>If your keyboard is not always plugged into the machine (e.g., a USB keyboard plugged into a laptop machine, sometimes) then you will find that approach (1) above is not adequate. The reason is that, in X11, those keyboard setting overrides would applied only to the devices attached to the system at the time the settings are applied – that is, when initializing <code class="language-plaintext highlighter-rouge">X</code>.</p> <p>If, for example, a USB keyboard is unplugged and then re-plugged, it will get default settings, not the override settings that we would have applied during X11 initialization.</p> <p>One solution to this problem is the userspace <a href="https://manpages.debian.org/inputplug" title="inputplug(1)"><code class="language-plaintext highlighter-rouge">inputplug(1)</code></a> daemon, available in the <a href="https://tracker.debian.org/pkg/inputplug" title="pkg: inputplug">‘inputplug’</a> Debian package or from the GitHub repo <a href="https://github.com/andrewshadura/inputplug" title="GitHub: andrewshadura/inputplug">andrewshadura/inputplug</a>. Because <code class="language-plaintext highlighter-rouge">inputplug</code> monitors the system for a particular class of <a href="https://cgit.freedesktop.org/xorg/proto/inputproto/tree/specs/XI2proto.txt" title="The X Input Extension 2.x">XInput2</a> events, it can determine when a new device has been (re)added to the system. When such an event is received, it invokes a user-provided program with the event metadata; that gives us a chance to (re)apply our override settings.</p> <p><em>[ The approach described below is an adaptation of the technique presented by <a href="https://unix.stackexchange.com/">unix.stackexchange.com</a> user <a href="https://unix.stackexchange.com/users/308316/mosvy">mosvy</a> in <a href="https://unix.stackexchange.com/a/523959">an answer</a> in the discussion <a href="https://unix.stackexchange.com/questions/523635/prevent-keyboard-layout-reset-when-usb-keyboard-is-plugged-in/523959">“Prevent keyboard layout reset when USB keyboard is plugged in”</a>. ]</em></p> <p>This approach uses four small pieces acting on concert.</p> <ol> <li> <p><strong><code class="language-plaintext highlighter-rouge">my-keyboard-setup</code></strong> – A script with the above <code class="language-plaintext highlighter-rouge">xmodmap</code> commands in it</p> </li> <li> <p><strong><code class="language-plaintext highlighter-rouge">on-new-kbd</code></strong> – A script invoked by <code class="language-plaintext highlighter-rouge">inputplug</code>; calls <code class="language-plaintext highlighter-rouge">my-keyboard-setup</code></p> </li> <li> <p><strong><code class="language-plaintext highlighter-rouge">~/.xsession</code> (1st)</strong> – Modify to call <code class="language-plaintext highlighter-rouge">my-keyboard-setup</code> (at X init time)</p> </li> <li> <p><strong><code class="language-plaintext highlighter-rouge">~/.xsession</code> (2nd)</strong> – Modify to launch <code class="language-plaintext highlighter-rouge">inputplug</code> daemon (at X init time)</p> </li> </ol> <h4 id="piece-1-of-4-my-keyboard-setup">Piece 1 of 4: my-keyboard-setup</h4> <p>The first is a small script that runs the <code class="language-plaintext highlighter-rouge">xmodmap</code> commands we have been using above. The reason for putting them in a dedicated script is that it will allow us to trigger the functionality from two different contexts.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ mkdir -p ~/bin/ $ cat - &lt;&lt;EOF &gt; ~/bin/my-keyboard-setup &gt; #!/bin/bash - &gt; xmodmap -e 'clear Lock' &gt; xmodmap -e 'keycode 66 = Return NoSymbol Return' &gt; EOF $ chmod 0755 ~/bin/my-keyboard-setup </code></pre></div></div> <h4 id="piece-2-of-4-on-new-kbd">Piece 2 of 4: on-new-kbd</h4> <p>The second is a script intended to be invoked by <code class="language-plaintext highlighter-rouge">inputplug</code> that will decide when to invoke the above command (basically, only when a new keyboard has been enabled). There is a more complete version in <a href="https://gist.github.com/salewski/abc0c9dfc6657fec4bcadc55fc5c0017" title="GitHub Gist: on-new-kbd">the on-new-kbd Gist</a>, but the basic idea looks like this:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#!/bin/bash - declare -r PROG='on-new-kbd' # When invoked by 'inputplug', four command line parameters will be # provided: # # event-type device-id device-type device-name # # See inputplug(1) for details. event_type=$1 device_id=$2 device_type=$3 device_name=$4 # Examples: # ========= # # Note that we get two different events when we unplug the keyboard, and # two different events when we plug the keyboard back in. # # Unplug events: # # ----------------------------------------------- # event_type: XIDeviceDisabled XIDeviceDisabled # device_id: 10 10 # device_type: XISlaveKeyboard XISlaveKeyboard # device_name: is_unset_or_null # ----------------------------------------------- # event_type: XISlaveRemoved XISlaveRemoved # device_id: 10 10 # device_type: is_unset_or_null # device_name: is_unset_or_null # ----------------------------------------------- # # # Plug event: # # ----------------------------------------------- # event_type: XISlaveAdded XISlaveAdded # device_id: 10 10 # device_type: XIFloatingSlave XIFloatingSlave # device_name: Microsoft Natural Keyboard Elite Microsoft Natural Keyboard Elite # ----------------------------------------------- # event_type: XIDeviceEnabled XIDeviceEnabled # device_id: 10 10 # device_type: XISlaveKeyboard XISlaveKeyboard # device_name: Microsoft Natural Keyboard Elite Microsoft Natural Keyboard Elite # ----------------------------------------------- case "${event_type} ${device_type}" in 'XIDeviceEnabled XISlaveKeyboard') my-keyboard-setup ;; *) : uninteresting event -- ignore ;; esac </code></pre></div></div> <h4 id="piece-3-of-4-xession-call-to-my-keyboard-setup">Piece 3 of 4: ~/.xession call to my-keyboard-setup</h4> <p>To have our keyboard mapping overrides take effect during X11 initialization, add a line such as the following to your <code class="language-plaintext highlighter-rouge">~/.xsession</code> file (or equivalent):</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> "${HOME}/bin/my-keyboard-setup" </code></pre></div></div> <h4 id="piece-4-of-4-xession-run-inputplug1-daemon">Piece 4 of 4: ~/.xession run inputplug(1) daemon</h4> <p>Also done during X11 initialization, arrange for the <code class="language-plaintext highlighter-rouge">inputplug</code> daemon to run and to invoke the <code class="language-plaintext highlighter-rouge">on-new-kbd</code> script for device add/remove and enable/disable events. To do so, add a line such as the following to <code class="language-plaintext highlighter-rouge">~/.xsession</code> beneath the line we just added in the previous step:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> /usr/bin/inputplug -c "${HOME}/bin/on-new-kbd" </code></pre></div></div> <hr /> <h1 id="summary">Summary</h1> <p>At this point you should have a working left-hand <code class="language-plaintext highlighter-rouge">ENTER</code> key that is both permanent and dynamic. Enjoy!</p>(Updated: 2023-05-05 to fix some typos)initialysm (definition)2020-09-17T00:00:00+00:002020-09-17T00:00:00+00:00http://salewski.email/2020/09/17/word-def-initialysm<dl> <dt><strong>initialysm:</strong> (<a href="https://www.merriam-webster.com/dictionary/noun" alt="noun"><em>noun</em></a>)</dt> <dd> <p>[<em>portmanteau of </em><a href="https://www.merriam-webster.com/dictionary/initially">initially</a> + <a href="https://www.merriam-webster.com/dictionary/initialism">initialism</a> <br /> &nbsp;<em>syllables:</em> <code>ini·​tial·​ysm</code> <br /> &nbsp;<em>pronunciation:</em> <code>i-ˈni-shə-ˌli-zəm</code> <em>(pronounced identically to "initialism")</em>]</p> <p>A word interpreted as an initialism that predates its own expansion; such a word that was born backwards. As in, "Initially it was just the initials, but later each letter was made to stand-in for a word for mnemonic effect."</p> <p>Like a <a href="http://catb.org/jargon/html/B/backronym.html">backronym</a>, but is always pronounced as a series of letters.</p> <p><em>Etymology</em><br /> The word <em>initialysm</em> was coined in order to explain the name of the <strong><code>aagg</code></strong> program, a command line program for use on <a alt="Unix (wikipedia article)" href="https://en.wikipedia.org/wiki/Unix">Unix</a>-like systems.</p> <p>The name of the program is pronounced "a a gee gee", with the individual letters standing for the expansion phrase "<strong>A</strong>WS-<strong>a</strong>-<strong>g</strong>o-<strong>g</strong>o".</p> <p>The program was initially named by finding a small number of characters that combined to form an easy to type, <a href="http://catb.org/jargon/html/G/grep.html">grep</a>-friendly sequence on a standard <a alt="QWERTY (jargon file)" href="http://catb.org/jargon/html/Q/QWERTY.html">QWERTY</a> keyboard. Later, when it came time to document the program for use by others, the expansion phrase was retrofitted as a mnemonic device.</p> </dd> </dl>initialysm: (noun)Debian 9.x “stretch” APT pinning change2017-06-25T00:00:00+00:002017-06-25T00:00:00+00:00http://salewski.email/2017/06/25/debian-9.x-stretch-apt-pinning-change<p>Last week (on <code class="language-plaintext highlighter-rouge">2017-06-17</code>), Debian GNU/Linux 9.x (codenamed “stretch”) <a href="https://www.debian.org/News/2017/20170617">was released</a>. While preparing to update one of my systems I came across the following statement in <a href="https://www.debian.org/releases/stable/amd64/release-notes/ch-information.en.html#apt-issues">section 5.3.2.2 of the release notes</a> (“New requirements for APT repository”):</p> <blockquote> <p>5.3.2.2. New APT pinning engine</p> <p>APT 1.1 introduced a new pinning engine that now matches the description in the manual page.</p> <p>The old engine assigned one pin priority per package; the new one assigns pin priorities per version. It then picks the version with the highest pin that is not a downgrade or that has a pin &gt; 1000.</p> <p>This changes the effect of some pins, especially negative ones. Previously, pinning a version to -1 effectively prevented the package from being installed (the package pin was -1); it now only prevents the version of this package from being installed.</p> </blockquote> <p>I run my <a href="https://www.debian.org/">Debian</a> systems <a href="http://without-systemd.org/">without systemd</a>, and my initial reading of the above text gave me pause because I use APT pinning to prevent systemd from becoming my init system. My initial reading of the above left me with the misunderstanding that I would need to apply a different mechanism to keep my current init setup (<code class="language-plaintext highlighter-rouge">'/sbin/init'</code> provided by <code class="language-plaintext highlighter-rouge">sysvinit-core</code> package).</p> <p>The <em>default</em> <a href="http://blog.darknedgy.net/technology/2015/09/05/0/">init system</a> on Debian (since Debian 8.x, codenamed “jessie”) has been <code class="language-plaintext highlighter-rouge">systemd</code>, in which <code class="language-plaintext highlighter-rouge">'/sbin/init'</code> is provided by the <code class="language-plaintext highlighter-rouge">systemd-sysv</code> package. This continues to be the case with Debian 9.x.</p> <p>My existing setups use a customized set of APT preferences which, among other things, contains the file:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> /etc/apt/preferences.d/LOCAL-avoid-systemd.pref </code></pre></div></div> <p>whose meat is:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> Package: systemd-sysv Pin: release o=Debian Pin-Priority: -1 </code></pre></div></div> <p>The effect of the <code class="language-plaintext highlighter-rouge">'Pin-Priority'</code> of <code class="language-plaintext highlighter-rouge">-1</code> is to prevent APT from <em>ever</em> installing the <code class="language-plaintext highlighter-rouge">'systemd-sysv'</code> package. And indeed, this is how the mechanism continues to work in Debian 9.x “stretch”. I did not realize this without closer study, however.</p> <p>After spending more time than I care to admit searching for and reading related info on the ‘Net, I came to realize that the statement in the release notes above applies very specifically to how APT treats pinning of <em>versions</em>. The above APT configuration, in contrast, is based on the separate concept of <em>release origin</em>.</p> <p>The above config file snippet applies the <code class="language-plaintext highlighter-rouge">-1</code> pin priority to any <code class="language-plaintext highlighter-rouge">'systemd-sysv'</code> package in which the “Release file” declares the <em>origin</em> as “Debian” (the <code class="language-plaintext highlighter-rouge">release o=Debian</code> bit). It <em>does not</em> contain any bits related to the package <em>version</em>.</p> <p>In retrospect, this is all rather obvious, and I’m embarrassed to think how much time I spent researching it. It was also quite helpful to read the motivation for the release notes blurb, which is contained in <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852967">bug #852967</a>, against the <a href="https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=release-notes">‘release-notes’</a> <a href="https://www.debian.org/Bugs/pseudo-packages">pseudo-package</a>.</p> <p>Other than mucking around with my <code class="language-plaintext highlighter-rouge">'sources.list'</code> file (and related), I tend to go many years between having to think about my APT configuration at all – so it is easy to become hazy on the details. Hence my focus on the <code class="language-plaintext highlighter-rouge">-1</code> pin priority but glossing over the specific language in the release notes of “pinning a version” (which it is now clear that I am not doing).</p> <p>I hope the above saves somebody from the need to research the same concerns I had.</p>Last week (on 2017-06-17), Debian GNU/Linux 9.x (codenamed “stretch”) was released. While preparing to update one of my systems I came across the following statement in section 5.3.2.2 of the release notes (“New requirements for APT repository”):Terraform HOWTO: delete a non-empty AWS S3 bucket2017-04-30T00:00:00+00:002017-04-30T00:00:00+00:00http://salewski.email/2017/04/30/terraform-howto-delete-a-non-empty-aws-s3-bucket<p>The Amazon <a href="https://aws.amazon.com/">AWS</a> <a href="https://aws.amazon.com/s3/">S3 service</a> provides <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/delete-or-empty-bucket.html">a number of different ways</a> to delete a non-empty S3 bucket; some of the approaches involve “emptying” the bucket prior to deleting it. The process can also vary a bit depending on whether or not the bucket has <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html">versioning</a> enabled.</p> <p>When the <a href="https://www.terraform.io/docs/providers/aws/index.html">“aws”</a> provider is used, the <a href="https://github.com/hashicorp/terraform">Terraform</a> program acts as a client to the AWS service, so has a number of available approaches it can take when deleting S3 buckets.</p> <p>When managing your infrastructure using Terraform, one common way to get rid of an infrastructure resource (cause it to be destroyed) is to simply remove it from your Terraform configuration (either by commenting-out it’s configuration block or by deleting it from the configuration file entirely).</p> <p>Non-empty S3 buckets throw a monkeywrench into that process. The data stored as S3 objects within the bucket can be considered as separate (possibly precious!) artitfacts, so a little extra convincing is needed to let Terraform know that you really do want it to delete an S3 bucket resource <em>and</em> any data objects it contains.</p> <p>If you simply get rid of the configuration block for the bucket, the <code class="language-plaintext highlighter-rouge">terraform plan</code> command will succeed in telling you that it would remove the bucket (as you might expect):</p> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform plan Refreshing Terraform state <span class="k">in</span><span class="nt">-memory</span> prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to <span class="nb">local </span>or remote state storage. aws_s3_bucket.your_tf_s3_bucket_resource_name: Refreshing state... <span class="o">(</span>ID: your-bucket-name<span class="o">)</span> The Terraform execution plan has been generated and is shown below. Resources are shown <span class="k">in </span>alphabetical order <span class="k">for </span>quick scanning. Green resources will be created <span class="o">(</span>or destroyed and <span class="k">then </span>created <span class="k">if </span>an existing resource exists<span class="o">)</span>, yellow resources are being changed <span class="k">in</span><span class="nt">-place</span>, and red resources will be destroyed. Cyan entries are data sources to be read. Note: You didn<span class="s1">'t specify an "-out" parameter to save this plan, so when "apply" is called, Terraform can'</span>t guarantee this is what will execute. - aws_s3_bucket.your_tf_s3_bucket_resource_name Plan: 0 to add, 0 to change, 1 to destroy. </code></pre></div></div> <p>However, if you were then run <code class="language-plaintext highlighter-rouge">terraform apply</code> you might be surprised by the error:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ terraform apply aws_s3_bucket.your_tf_s3_bucket_resource_name: Refreshing state... (ID: your-bucket-name) aws_s3_bucket.your_tf_s3_bucket_resource_name: Destroying... (ID: your-bucket-name) Error applying plan: 1 error(s) occurred: * aws_s3_bucket.your_tf_s3_bucket_resource_name (destroy): 1 error(s) occurred: * aws_s3_bucket.your_tf_s3_bucket_resource_name: Error deleting S3 Bucket: BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket. status code: 409, request id: &lt;request-id-value&gt;, host id: &lt;host-id-value&gt; "your-bucket-name" Terraform does not automatically rollback in the face of errors. Instead, your Terraform state file has been partially updated with any resources that successfully completed. Please address the error above and apply again to incrementally change your infrastructure. </code></pre></div></div> <p>Despite the “delete all versions in the bucket” language of the error message, the above error will appear regardless of whether or not the bucket has versioning enabled.</p> <h1 id="the-solution">The Solution</h1> <p>Terraform <em>will</em> happily delete all of the objects in the bucket for you, but you have to explicitly tell it to do so, and you have to know how to ask.</p> <p>Let’s say you have the following as your S3 bucket configuration block in your Terraform configuration file:</p> <div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nx">resource</span> <span class="s2">"aws_s3_bucket"</span> <span class="s2">"my_s3_bucket_resource"</span> <span class="p">{</span> <span class="nx">bucket</span> <span class="p">=</span> <span class="s2">"my-bucket-name"</span> <span class="nx">acl</span> <span class="p">=</span> <span class="s2">"private"</span> <span class="nx">versioning</span> <span class="p">{</span> <span class="nx">enabled</span> <span class="p">=</span> <span class="kc">true</span> <span class="p">}</span> <span class="nx">tags</span> <span class="p">{</span> <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"whatev-name"</span> <span class="nx">Environment</span> <span class="p">=</span> <span class="s2">"whatev-env"</span> <span class="p">}</span> <span class="nx">lifecycle</span> <span class="p">{</span> <span class="c1"># Any Terraform plan that includes a destroy of this resource will</span> <span class="c1"># result in an error message.</span> <span class="c1">#</span> <span class="nx">prevent_destroy</span> <span class="p">=</span> <span class="kc">true</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>You’ll want to do two things to allow the above S3 bucket to be deleted:</p> <ol> <li> <p>Comment-out (or remove) the <code class="language-plaintext highlighter-rouge">'prevent_destroy'</code> setting</p> </li> <li> <p>Add a <code class="language-plaintext highlighter-rouge">'force_destroy'</code> setting</p> </li> </ol> <p>Here’s our new version:</p> <div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nx">resource</span> <span class="s2">"aws_s3_bucket"</span> <span class="s2">"my_s3_bucket_resource"</span> <span class="p">{</span> <span class="nx">bucket</span> <span class="p">=</span> <span class="s2">"my-bucket-name"</span> <span class="nx">force_destroy</span> <span class="p">=</span> <span class="kc">true</span> <span class="nx">acl</span> <span class="p">=</span> <span class="s2">"private"</span> <span class="nx">versioning</span> <span class="p">{</span> <span class="nx">enabled</span> <span class="p">=</span> <span class="kc">true</span> <span class="p">}</span> <span class="nx">tags</span> <span class="p">{</span> <span class="nx">Name</span> <span class="p">=</span> <span class="s2">"whatev-name"</span> <span class="nx">Environment</span> <span class="p">=</span> <span class="s2">"whatev-env"</span> <span class="p">}</span> <span class="c1"># lifecycle {</span> <span class="c1">#</span> <span class="c1"># # Any Terraform plan that includes a destroy of this resource will</span> <span class="c1"># # result in an error message.</span> <span class="c1"># #</span> <span class="c1"># prevent_destroy = true</span> <span class="c1"># }</span> <span class="p">}</span> </code></pre></div></div> <p>Now you can see the plan for destroying that bucket (and its dependencies) with the command:</p> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform plan <span class="nt">-destroy</span> <span class="nt">-target</span><span class="o">=</span>aws_s3_bucket.my_s3_bucket_resource </code></pre></div></div> <p>And then to actually have the bucket, its content, and its dependencies destroyed:</p> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform destroy <span class="nt">-target</span><span class="o">=</span>aws_s3_bucket.my_s3_bucket_resource <span class="o">[</span>Terraform will prompt you to confirm, warning that there is no <span class="s1">'undo'</span> <span class="k">for </span>the action] </code></pre></div></div> <p>Here’s what it looks like in action:</p> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform destroy <span class="nt">-target</span><span class="o">=</span>aws_s3_bucket.my_s3_bucket_resource Do you really want to destroy? Terraform will delete the following infrastructure: aws_s3_bucket.my_s3_bucket_resource There is no undo. Only <span class="s1">'yes'</span> will be accepted to confirm Enter a value: <span class="nb">yes </span>aws_s3_bucket.my_s3_bucket_resource: Refreshing state... <span class="o">(</span>ID: my-bucket-name<span class="o">)</span> aws_s3_bucket.my_s3_bucket_resource: Destroying... <span class="o">(</span>ID: my-bucket-name<span class="o">)</span> aws_s3_bucket.my_s3_bucket_resource: Destruction <span class="nb">complete </span>Destroy <span class="nb">complete</span><span class="o">!</span> Resources: 1 destroyed. </code></pre></div></div> <h1 id="q-what-would-happen-if-we-did-not-comment-out-prevent_destroy">Q: What would happen if we did not comment-out <code class="language-plaintext highlighter-rouge">'prevent_destroy'</code>?</h1> <p>You may be wondering what would happen if we did not comment-out the <code class="language-plaintext highlighter-rouge">'lifecycle'</code> setting <code class="language-plaintext highlighter-rouge">prevent_destroy = true</code> while having <code class="language-plaintext highlighter-rouge">force_destroy = true</code> set at the same time. The answer is that Terraform would do what you would want it to do: it would refuse to delete the S3 bucket:</p> <div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform plan <span class="nt">-destroy</span> <span class="nt">-target</span><span class="o">=</span>aws_s3_bucket.my_s3_bucket_resource Refreshing Terraform state <span class="k">in</span><span class="nt">-memory</span> prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to <span class="nb">local </span>or remote state storage. aws_s3_bucket.my_s3_bucket_resource: Refreshing state... <span class="o">(</span>ID: my-bucket-name<span class="o">)</span> Error running plan: 1 error<span class="o">(</span>s<span class="o">)</span> occurred: <span class="k">*</span> aws_s3_bucket.my_s3_bucket_resource: aws_s3_bucket.my_s3_bucket_resource: the plan would destroy this resource, but it currently has lifecycle.prevent_destroy <span class="nb">set </span>to true. To avoid this error and <span class="k">continue </span>with the plan, either disable lifecycle.prevent_destroy or adjust the scope of the plan using the <span class="nt">-target</span> flag. </code></pre></div></div> <h1 id="terraform-version-094--released-2017-04-26">Terraform version: 0.9.4 (released 2017-04-26)</h1> <p>The examples in this post all used <a href="https://github.com/hashicorp/terraform/blob/v0.9.4/CHANGELOG.md">terraform-0.9.4</a> (released 2017-04-26), but are likely to work just fine with both earlier and later versions. The earliest version expected to work (<em>not</em> tested!) is <a href="https://github.com/hashicorp/terraform/blob/v0.5.3/CHANGELOG.md">terraform-0.5.3</a> (the version that introduced the <code class="language-plaintext highlighter-rouge">'force_destroy</code>’ parameter for AWS S3 bucket resources).</p> <h1 id="additional-reading">Additional Reading</h1> <p>The rationale for Terraform not blindly deleting S3 objects was discussed in <a href="https://github.com/hashicorp/terraform/issues/1977">hashicorp/terraform#1977</a>; the discussion there includes the need for some sort of “force” option.</p> <p>The <code class="language-plaintext highlighter-rouge">'force_destroy'</code> option was implemented in <a href="https://github.com/hashicorp/terraform/pull/2007">hashicorp/terraform#2007</a>.</p> <p>The <a href="https://www.terraform.io/docs/providers/aws/r/s3_bucket.html#force_destroy">‘force_destroy’</a> option is also documented in the Terraform under the <a href="https://www.terraform.io/docs/providers/aws/index.html">“aws”</a> provider.</p>The Amazon AWS S3 service provides a number of different ways to delete a non-empty S3 bucket; some of the approaches involve “emptying” the bucket prior to deleting it. The process can also vary a bit depending on whether or not the bucket has versioning enabled.Al’s GitHub Pages with Jekyll cheatsheet2016-06-18T00:00:00+00:002016-06-18T00:00:00+00:00http://salewski.email/2016/06/18/github-pages-with-jekyll-cheatsheet<p>(Updated: 2020-05-05 to add notes on creating the orphan ‘gh-pages’ branch)</p> <h2 id="preliminaries">Preliminaries</h2> <h3 id="create-an-orphan-git-branch-named-gh-pages">Create an “orphan” git branch named ‘gh-pages’</h3> <p><a href="https://pages.github.com/">GitHub Pages</a> will allow you to publish your project’s site from a few different “sources”. Here we are concerned only with one of those approaches: an “orphan” git branch named <code class="language-plaintext highlighter-rouge">'gh-pages'</code> within the repo.</p> <p>The name <code class="language-plaintext highlighter-rouge">'gh-pages'</code> is not optional; it is treated specially by GitHub Pages. You cannot choose an arbitrary branch name from which to publish your project’s site; the branch must be called <code class="language-plaintext highlighter-rouge">'gh-pages'</code>.</p> <p>Note that your project’s settings page in the GitHub web site will only show the <code class="language-plaintext highlighter-rouge">'gh-pages'</code> branch as an option in the GitHub Pages “Source” drop-down if the branch already exists in your project’s repository.</p> <p>For the purposes of publishing your project’s site on GitHub Pages, you’ll want the <code class="language-plaintext highlighter-rouge">'gh-pages'</code> branch to be an “orphan”. That is, you want it to have a history that is entirely disconnected from the other existing branches in your repository. To achieve that, we are going to use the <code class="language-plaintext highlighter-rouge">--orphan</code> option to <a href="https://git-scm.com/docs/git-checkout" title="git-checkout(1)">git-checkout(1)</a>:</p> <pre> $ <strong>git checkout --orphan gh-pages</strong> Switched to a new branch 'gh-pages' </pre> <p>At this point, a <code class="language-plaintext highlighter-rouge">git status</code> would show that all of the repo’s files that were being tracked on the branch source point are now staged in the index; git does this in anticipation of the user wanting to create a new, clean go-forward history for the “as is” state of the files. That is not our use case, however; we are going to use an entirely different set of files on our new branch, so we can just clear out the git index:</p> <pre> $ <strong>git rm -fr .</strong> $ <strong>git status</strong> On branch gh-pages No commits yet nothing to commit (create/copy files and use "git add" to track) </pre> <p>Now you have a clean <code class="language-plaintext highlighter-rouge">'gh-pages'</code> branch with no commits. Read on to see how we want to populate it, starting with a file named <code class="language-plaintext highlighter-rouge">'Gemfile'</code>.</p> <h3 id="install-jekyll-and-other-related-ruby-based-tooling">Install Jekyll and other related Ruby-based tooling</h3> <p>In your brand new working directory for a site you intend to generate with <a href="https://jekyllrb.com/">Jekyll</a> and host with <a href="https://pages.github.com/">GitHub Pages</a>, you will want to have some tooling available locally to allow you to veiw the Jekyll-based site generation prior to pubishing it on GitHub Pages.</p> <p>The Jekyll software is written in <a href="https://www.ruby-lang.org">Ruby</a>, and (as is true of most Ruby software) is released in the form of a <a href="https://www.ruby-lang.org/en/libraries/">“gem”</a>.</p> <pre> $ <strong>cat - &lt;&lt;EOF &gt; Gemfile</strong> source 'http://rubygems.org' gem 'github-pages', group: :jekyll-plugins EOF </pre> <p>This will install (into the <code class="language-plaintext highlighter-rouge">'local-ruby-gems'</code> subdir) the dependencies specified in the <code class="language-plaintext highlighter-rouge">'Gemfile'</code> (see <a href="https://bundler.io/man/gemfile.5.html">Gemfile(5)</a>), along with all of <em>their</em> dependencies.</p> <pre> $ <strong>bundle install --path=./local-ruby-gems --verbose</strong> </pre> <pre> $ <strong>cat - &lt;&lt;EOF &gt; _config.yml title: my-project-name by Alan D. Salewski description: blah blah blah markdown: kramdown kramdown: input: GFM # Enable GitHub Flavored Markdown (fensed code blocks) hard-wrap: false highlighter: rouge exclude: - local-ruby-gems/ # avoid processing jekyll itself; see also https://github.com/jekyll/jekyll/issues/2938 EOF</strong> </pre> <h2 id="regular-usage">Regular usage</h2> <h3 id="prior-to-publishing-edit---review---commit-cycle">Prior to publishing: edit -&gt; review -&gt; commit cycle</h3> <p>Once you have your tooling setup in your project’s git working directory, you will basically just edit the <a href="https://daringfireball.net/projects/markdown/">Markdown</a> files of individual articles in the <code class="language-plaintext highlighter-rouge">'_drafts'</code> subdirectory. The edit -&gt; review -&gt; commit cycle works almost the same as any other software project, with commits happening wherever makes sense for the particular article content.</p> <p>The “review” phase mainly involves just rereading the article content and making adjustments as necessary. Before committing, however, you should view the Jekyll-rendered form of the article in your web browser. To do that, you <em>could</em> just <code class="language-plaintext highlighter-rouge">'git push'</code> to GitHub Pages and hope for the best, but since you’ve gone through the trouble of setting up your local Jekyll-based tooling you should launch the local web server built into Jekyll for previewing your changes.</p> <p>This will generate the static site and render it the way it will be published when you <code class="language-plaintext highlighter-rouge">'git push ...'</code> your changes up to GitHub:</p> <pre> $ <strong>bundle exec jekyll serve --watch</strong> Configuration file: /path/to/project/_config.yml Configuration file: /path/to/project/_config.yml Source: /path/to/project Destination: /path/to/project/_site Incremental build: disabled. Enable with --incremental Generating... done in 0.389 seconds. Auto-regeneration: enabled for '/path/to/project' Configuration file: /path/to/project/_config.yml Server address: http://127.0.0.1:4000/ Server running... press ctrl-c to stop. </pre> <p>The following variation of the same command (adds the <code class="language-plaintext highlighter-rouge">'--drafts'</code> option) will generate the static site and render it with all of your unpublished drafts integrated into the site:</p> <pre> $ <strong>bundle exec jekyll serve --watch --drafts</strong> Configuration file: /path/to/project/_config.yml Configuration file: /path/to/project/_config.yml Source: /path/to/project Destination: /path/to/project/_site Incremental build: disabled. Enable with --incremental Generating... done in 0.639 seconds. Auto-regeneration: enabled for '/path/to/project' Configuration file: /path/to/project/_config.yml Server address: http://127.0.0.1:4000/ Server running... press ctrl-c to stop. </pre> <p>In addition to the normal posts (everything beneath the <code class="language-plaintext highlighter-rouge">'_posts'</code> subdir), the above command also includes everything beneath the <code class="language-plaintext highlighter-rouge">'_drafts/'</code> subdir.</p> <p><strong>Q: Should I check the Gemfile in?</strong></p> <p>(Concerned that it might have an adverse effect if interpreted by the remote GitHub site builder.)</p> <p><strong>A:</strong> Yes, empirical testing shows that the remote site builder does not break.</p> <h3 id="publishing">Publishing</h3> <p>Once you are happy with the content of an article, you can cause GitHub Pages to render it by moving it from the <code class="language-plaintext highlighter-rouge">'_drafts/'</code> subdir to the <code class="language-plaintext highlighter-rouge">'_posts/writings/'</code> subdir, with a publication date embedded in the file name:</p> <pre> $ <strong>git mv _drafts/my-article.md _posts/writings/YYYY-MM-DD-my-article.md</strong> </pre> <p>Push your locally committed changes to GitHub to have them rendered by GitHub Pages:</p> <pre> $ <strong>git push --follow-tags origin master</strong> ... </pre>(Updated: 2020-05-05 to add notes on creating the orphan ‘gh-pages’ branch)