Jekyll2023-05-06T01:01:43+00:00http://salewski.email/feed.xmlAlan D. Salewski (writings)Al's misc. thoughts, notes, and other writings
Left-hand ENTER key2020-09-22T00:00:00+00:002020-09-22T00:00:00+00:00http://salewski.email/2020/09/22/left-hand-enter-key<p>(Updated: 2023-05-05 to fix some typos)</p>
<h1 id="remap-caps-lock-key-as-left-hand-enter">Remap “Caps Lock” key as left-hand ENTER</h1>
<p>[ This post is relevant primarily to right-handed folks, and to
left-handed folks who use a right-handed computer setup (you use the
mouse with your right hand). ]</p>
<p>I found myself working on a one-off task that involved a fair amount of
copy/paste work in my terminal emulator, building up command lines with
little tidbits of data gathered from here and there, without any typing
(except to hit the ENTER key to run the command).</p>
<p>I am using <a href="https://en.wikipedia.org/wiki/X_Window_System" title="wikipedia: X Window System">X11</a>, so selecting the source text tidbits is
enough to “copy” it, and clicking the middle mouse button pastes it in
the target window at the location of the cursor. So far, so good. Once
the command line was ready I pressed the <code class="language-plaintext highlighter-rouge">ENTER</code> key (with my right
hand), and then moved on to the next one.</p>
<p>I found that moving my right hand from the mouse to the <code class="language-plaintext highlighter-rouge">ENTER</code> key and
then back to the mouse again was slowing me down, and the motion was
generally irritating.</p>
<p>It occurred to me that it would be nice if there were an <code class="language-plaintext highlighter-rouge">ENTER</code> key
within reach of my left hand, and that the <code class="language-plaintext highlighter-rouge">Caps Lock</code> key would be a
good candidate for remapping for that purpose.</p>
<hr />
<h1 id="tldr">TL;DR</h1>
<p>To turn your <code class="language-plaintext highlighter-rouge">Caps Lock</code> key into an <code class="language-plaintext highlighter-rouge">ENTER</code> key, run these two commands
in a shell within your X11 session. If you are happy with the result,
then add the commands to your <code class="language-plaintext highlighter-rouge">~/.xsession</code> file (or your local system
equivalent) to have them take effect the next time you restart <code class="language-plaintext highlighter-rouge">X</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -e 'clear Lock'
$ xmodmap -e 'keycode 66 = Return NoSymbol Return'
</code></pre></div></div>
<p>For best results, read the rest of the article first, though; <em>the
<code class="language-plaintext highlighter-rouge">Caps Lock</code> key on your keyboard might not be mapped to keycode 66</em>. If
that is the case, the above will just disable your <code class="language-plaintext highlighter-rouge">Caps Lock</code> key and
turn whatever key that is mapped to keycode 66 into an <code class="language-plaintext highlighter-rouge">ENTER</code> key. Also
described below is an approach to preserving the mapping when keyboards
are hotplugged to the system (e.g., USB keyboards).</p>
<hr />
<h1 id="background">Background</h1>
<p>A typical 104- or 105-key US keyboard has <em>two</em> <code class="language-plaintext highlighter-rouge">ENTER</code> keys reachable
with your right hand:</p>
<ol>
<li>
<p>one to the right side of the home row, and</p>
</li>
<li>
<p>one on the right edge of the number pad.</p>
</li>
</ol>
<p>However, there are not any reachable from your left hand.</p>
<p><strong>Since the <code class="language-plaintext highlighter-rouge">Caps Lock</code> key is useless anyway, it is a good candidate to
re-map as a left-hand <code class="language-plaintext highlighter-rouge">ENTER</code> key. Having that allows you to keep your
right hand on the mouse and use your left hand to press the <code class="language-plaintext highlighter-rouge">ENTER</code> key,
when needed.</strong></p>
<hr />
<h1 id="gather-tools-and-data">Gather tools and data</h1>
<p>In order to remap the keyboard <code class="language-plaintext highlighter-rouge">Caps Lock</code> key, you will need a couple
of pieces of information and two programs: <a href="https://manpages.debian.org/xev" title="xev(1)"><code class="language-plaintext highlighter-rouge">xev(1)</code></a> and
<a href="https://manpages.debian.org/xmodmap" title="xmodmap(1)"><code class="language-plaintext highlighter-rouge">xmodmap(1)</code></a>. On <a href="https://www.debian.org/" title="www.debian.org">Debian</a>-based systems, the
<code class="language-plaintext highlighter-rouge">xev</code> program is provided by the <a href="https://tracker.debian.org/pkg/x11-utils" title="pkg: x11-utils"><code class="language-plaintext highlighter-rouge">x11-utils</code></a> package,
and the <code class="language-plaintext highlighter-rouge">xmodmap</code> program is provided by the
<a href="https://tracker.debian.org/pkg/x11-xserver-utils" title="pkg: x11-xserver-utils"><code class="language-plaintext highlighter-rouge">x11-xserver-utils</code></a> package. You may have one or
both of them already, but if not:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> # apt-get -u install x11-utils x11-xserver-utils
</code></pre></div></div>
<p>The pieces of information needed include the “keycodes” for the “Caps
Lock” and “Enter” keys, as well as the mapping (to “keysyms”) in effect
for the “Enter” key (so we can copy it for use with the “Caps Lock”
key).</p>
<p>Use <code class="language-plaintext highlighter-rouge">xev</code> to obtain the keycodes. Here’s what I saw initially for my “Caps Lock”
key:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>KeyPress event, serial 31, synthetic NO, window 0xd400001,
root 0x1ab, subw 0x0, time 4049429559, (169,-13), root:(309,938),
state 0x10, keycode 66 (keysym 0xffe5, Caps_Lock), same_screen YES,
XLookupString gives 0 bytes:
XmbLookupString gives 0 bytes:
XFilterEvent returns: False
KeyRelease event, serial 34, synthetic NO, window 0xd400001,
root 0x1ab, subw 0x0, time 4049429670, (169,-13), root:(309,938),
state 0x12, keycode 66 (keysym 0xffe5, Caps_Lock), same_screen YES,
XLookupString gives 0 bytes:
XFilterEvent returns: False
</code></pre></div></div>
<p>So that tells us that our “Caps Lock” key is mapped to keycode 66. A similar
query for the “Enter” key told me it was mapped to keycode 36.</p>
<p>Next we will use <code class="language-plaintext highlighter-rouge">xmodmap</code> to obtain the keycode to keysyms mapping, and
also the current state of the <code class="language-plaintext highlighter-rouge">Lock</code> modifier (from the “modifier map”).</p>
<p>Since we will make reference below to clearing the lock modifier, it is
useful to note the initial state of it:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -pm | grep '^lock'
lock Caps_Lock (0x42)
</code></pre></div></div>
<p><em>[ The <code class="language-plaintext highlighter-rouge">-pm</code> option causes <code class="language-plaintext highlighter-rouge">xmodmap</code> to show us the current modifier map. ]</em></p>
<p>The next thing to find out was what the xmodmap setting for the “Enter” key
looked like. That was done by browsing the output of the command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -pke | less
</code></pre></div></div>
<p><em>[ The <code class="language-plaintext highlighter-rouge">-pke</code> option causes <code class="language-plaintext highlighter-rouge">xmodmap</code> to print on <code class="language-plaintext highlighter-rouge">stdout</code> the current
keymap table, in the form of expressions that can be fed back into to
<code class="language-plaintext highlighter-rouge">xmodmap</code>. ]</em></p>
<p>The two lines in that output of interest here were:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> keycode 36 = Return NoSymbol Return
keycode 66 = Caps_Lock NoSymbol Caps_Lock
</code></pre></div></div>
<p>The first shows the setting for the “Enter” key, and the second is the
initial setting for the “Caps Lock” key.</p>
<hr />
<h1 id="lets-do-it">Let’s do it</h1>
<p>To turn our “Caps Lock” key into a left-hand “Enter” key, we need to do
two things:</p>
<ol>
<li>
<p>Remove the lock modifier for the “Caps Lock” key.</p>
</li>
<li>
<p>Re-map the “Caps Lock” key’s keycode to the “keysyms” appropriate
for an “Enter” key.</p>
</li>
</ol>
<p>Step (1) is accomplished on the live system by using the
command<sup><a href="#aside-why-non-surgical">1</a></sup>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -e 'clear Lock'
</code></pre></div></div>
<div style="margin: auto; max-width: 80%; padding: 20px; font-size: smaller; text-align: justify; border: 1px solid #e5e5e5; background-color: #e4e6e8;">
<aside>
<p>
<a id="aside-why-non-surgical"></a>
[1] Note that a more surgical approach to the above action would be to
run the command:</p>
<pre style="font-size: smaller;">
$ xmodmap -e 'remove Lock = Caps_Lock'
</pre>
<p>Indeed, that was what I was using initially, but it came with
complications.</p>
<p>The first (and easily dealt with) complication is that upon
(re)attaching a USB keyboard to the machine, we could not know in
advance the state of the lock modifier. We might find that the lock
modifier is not applied to any keysym, or maybe it is applied to the
'Return' keysym, or perhaps applied to both 'Caps_Lock' and 'Return'
keysyms, or some other variation. Just blindly running the
above command would fail if the lock modifier was not currently set for
the 'Caps_Lock' keysym.</p>
<p>So to use the more specific command requires first detecting the
current state and then taking appropriate action. For example (in
bash):</p>
<pre style="font-size: smaller;">
current_lock_assignment=$(xmodmap -pm | grep -i '^lock[[:space:]]')
shopt -q -s nocasematch
if [[ "${current_lock_assignment}" =~ Caps_Lock ]]; then
xmodmap -e 'remove Lock = Caps_Lock'
fi
if [[ "${current_lock_assignment}" =~ Return ]]; then
xmodmap -e 'remove Lock = Return'
fi
</pre>
<p>Depending on other factors in your machine configuration, the above
may work for you "as is". If so, it is probably a better approach than
the approach I ultimately took because it expresses precisely the
desired configuration.</p>
<p>In my environment, however, I have another complication: There is
something else in the mix also responding to the "keyboard enabled"
events, and it is conflicting with my use of the above. Oddly enough, it
only affects me <em>every other time</em> I plug the keyboard in. The
observed behavior is that the lock modifier is applied to the 'Return'
keysym, and though the "Caps Lock" key still behaves as an "Enter" key,
it also still toggles the letter case with each press.</p>
<p>I don't have time at the moment to track down the competing
mechanism, but it does not rear its head (in any obvious way) when using
the <code style="font-size: smaller;">'clear Lock'</code> expression. So
that is what I am going with, at least for now. Also, using <code style="font-size: smaller;">'clear Lock'</code> is not a compromise of
any functionality; I did not use lock for anything other than "Caps
Lock" (the "Num Lock" key uses a different modifier (<code style="font-size: smaller;">'mod2'</code>)).</p>
</aside>
</div>
<p><br /></p>
<p>Step (2) is accomplished on the live system by using the command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ xmodmap -e 'keycode 66 = Return NoSymbol Return'
</code></pre></div></div>
<p>Congratulations, your “Caps Lock” key now functions as an “Enter”
key. At this point, your “Caps Lock” should have stopped having any
letter case-altering effect, and should have no locking (“toggle”)
effect, either. If your keyboard has an LED light that used to turn on
when your “Caps Lock” key was in the lock position, you will find that
the LED is no longer lit when the “Caps Lock” key is pressed (or even
held down).</p>
<hr />
<h1 id="make-permanent">Make permanent</h1>
<p>The above settings are transient as currently configured. If you were to
restart <code class="language-plaintext highlighter-rouge">X</code> or reboot your machine (or something similar) the old
settings would come back when <code class="language-plaintext highlighter-rouge">X</code> gets reinitialized. To have the new
settings take effect automatically in your next <code class="language-plaintext highlighter-rouge">X</code> session, there are
two different approaches you might consider:</p>
<ol>
<li>
<p>The first (and simpler) approach would be to add the commands
exactly as typed above to your <a href="https://unix.stackexchange.com/questions/47359/what-is-xsession-for" title="What is '.xsession' for?"><code class="language-plaintext highlighter-rouge">~/.xsession</code></a>
file (or local system equivalent). If your keyboard is always
plugged into the machine (typical for a desktop) then that
approach is good enough.</p>
</li>
<li>
<p>The second, more elaborate, approach would be to take additional
steps to allow the settings to take effect not only upon
re-initializing <code class="language-plaintext highlighter-rouge">X</code>, but also when hotplugging the keyboard.</p>
</li>
</ol>
<hr />
<h1 id="make-dynamic">Make dynamic</h1>
<p>If your keyboard is not always plugged into the machine (e.g., a USB
keyboard plugged into a laptop machine, sometimes) then you will find
that approach (1) above is not adequate. The reason is that, in X11,
those keyboard setting overrides would applied only to the devices
attached to the system at the time the settings are applied – that is,
when initializing <code class="language-plaintext highlighter-rouge">X</code>.</p>
<p>If, for example, a USB keyboard is unplugged and then re-plugged, it
will get default settings, not the override settings that we would have
applied during X11 initialization.</p>
<p>One solution to this problem is the userspace
<a href="https://manpages.debian.org/inputplug" title="inputplug(1)"><code class="language-plaintext highlighter-rouge">inputplug(1)</code></a> daemon, available in the
<a href="https://tracker.debian.org/pkg/inputplug" title="pkg: inputplug">‘inputplug’</a> Debian package or from the GitHub repo
<a href="https://github.com/andrewshadura/inputplug" title="GitHub: andrewshadura/inputplug">andrewshadura/inputplug</a>. Because <code class="language-plaintext highlighter-rouge">inputplug</code>
monitors the system for a particular class of <a href="https://cgit.freedesktop.org/xorg/proto/inputproto/tree/specs/XI2proto.txt" title="The X Input Extension 2.x">XInput2</a>
events, it can determine when a new device has been (re)added to the
system. When such an event is received, it invokes a user-provided
program with the event metadata; that gives us a chance to (re)apply our
override settings.</p>
<p><em>[ The approach described below is an adaptation of the technique presented by
<a href="https://unix.stackexchange.com/">unix.stackexchange.com</a> user
<a href="https://unix.stackexchange.com/users/308316/mosvy">mosvy</a> in
<a href="https://unix.stackexchange.com/a/523959">an answer</a> in the discussion
<a href="https://unix.stackexchange.com/questions/523635/prevent-keyboard-layout-reset-when-usb-keyboard-is-plugged-in/523959">“Prevent keyboard layout reset when USB keyboard is plugged in”</a>. ]</em></p>
<p>This approach uses four small pieces acting on concert.</p>
<ol>
<li>
<p><strong><code class="language-plaintext highlighter-rouge">my-keyboard-setup</code></strong> – A script with the above <code class="language-plaintext highlighter-rouge">xmodmap</code> commands in it</p>
</li>
<li>
<p><strong><code class="language-plaintext highlighter-rouge">on-new-kbd</code></strong> – A script invoked by <code class="language-plaintext highlighter-rouge">inputplug</code>; calls <code class="language-plaintext highlighter-rouge">my-keyboard-setup</code></p>
</li>
<li>
<p><strong><code class="language-plaintext highlighter-rouge">~/.xsession</code> (1st)</strong> – Modify to call <code class="language-plaintext highlighter-rouge">my-keyboard-setup</code> (at X init time)</p>
</li>
<li>
<p><strong><code class="language-plaintext highlighter-rouge">~/.xsession</code> (2nd)</strong> – Modify to launch <code class="language-plaintext highlighter-rouge">inputplug</code> daemon (at X init time)</p>
</li>
</ol>
<h4 id="piece-1-of-4-my-keyboard-setup">Piece 1 of 4: my-keyboard-setup</h4>
<p>The first is a small script that runs the <code class="language-plaintext highlighter-rouge">xmodmap</code> commands we have
been using above. The reason for putting them in a dedicated script is
that it will allow us to trigger the functionality from two different
contexts.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ mkdir -p ~/bin/
$ cat - <<EOF > ~/bin/my-keyboard-setup
> #!/bin/bash -
> xmodmap -e 'clear Lock'
> xmodmap -e 'keycode 66 = Return NoSymbol Return'
> EOF
$ chmod 0755 ~/bin/my-keyboard-setup
</code></pre></div></div>
<h4 id="piece-2-of-4-on-new-kbd">Piece 2 of 4: on-new-kbd</h4>
<p>The second is a script intended to be invoked by <code class="language-plaintext highlighter-rouge">inputplug</code> that will
decide when to invoke the above command (basically, only when a new
keyboard has been enabled). There is a more complete version in <a href="https://gist.github.com/salewski/abc0c9dfc6657fec4bcadc55fc5c0017" title="GitHub Gist: on-new-kbd">the
on-new-kbd Gist</a>, but the basic idea looks like
this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#!/bin/bash -
declare -r PROG='on-new-kbd'
# When invoked by 'inputplug', four command line parameters will be
# provided:
#
# event-type device-id device-type device-name
#
# See inputplug(1) for details.
event_type=$1
device_id=$2
device_type=$3
device_name=$4
# Examples:
# =========
#
# Note that we get two different events when we unplug the keyboard, and
# two different events when we plug the keyboard back in.
#
# Unplug events:
#
# -----------------------------------------------
# event_type: XIDeviceDisabled XIDeviceDisabled
# device_id: 10 10
# device_type: XISlaveKeyboard XISlaveKeyboard
# device_name: is_unset_or_null
# -----------------------------------------------
# event_type: XISlaveRemoved XISlaveRemoved
# device_id: 10 10
# device_type: is_unset_or_null
# device_name: is_unset_or_null
# -----------------------------------------------
#
#
# Plug event:
#
# -----------------------------------------------
# event_type: XISlaveAdded XISlaveAdded
# device_id: 10 10
# device_type: XIFloatingSlave XIFloatingSlave
# device_name: Microsoft Natural Keyboard Elite Microsoft Natural Keyboard Elite
# -----------------------------------------------
# event_type: XIDeviceEnabled XIDeviceEnabled
# device_id: 10 10
# device_type: XISlaveKeyboard XISlaveKeyboard
# device_name: Microsoft Natural Keyboard Elite Microsoft Natural Keyboard Elite
# -----------------------------------------------
case "${event_type} ${device_type}" in
'XIDeviceEnabled XISlaveKeyboard') my-keyboard-setup ;;
*)
: uninteresting event -- ignore
;;
esac
</code></pre></div></div>
<h4 id="piece-3-of-4-xession-call-to-my-keyboard-setup">Piece 3 of 4: ~/.xession call to my-keyboard-setup</h4>
<p>To have our keyboard mapping overrides take effect during X11
initialization, add a line such as the following to your <code class="language-plaintext highlighter-rouge">~/.xsession</code>
file (or equivalent):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> "${HOME}/bin/my-keyboard-setup"
</code></pre></div></div>
<h4 id="piece-4-of-4-xession-run-inputplug1-daemon">Piece 4 of 4: ~/.xession run inputplug(1) daemon</h4>
<p>Also done during X11 initialization, arrange for the <code class="language-plaintext highlighter-rouge">inputplug</code> daemon
to run and to invoke the <code class="language-plaintext highlighter-rouge">on-new-kbd</code> script for device add/remove and
enable/disable events. To do so, add a line such as the following to
<code class="language-plaintext highlighter-rouge">~/.xsession</code> beneath the line we just added in the previous step:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> /usr/bin/inputplug -c "${HOME}/bin/on-new-kbd"
</code></pre></div></div>
<hr />
<h1 id="summary">Summary</h1>
<p>At this point you should have a working left-hand <code class="language-plaintext highlighter-rouge">ENTER</code> key that is
both permanent and dynamic. Enjoy!</p>(Updated: 2023-05-05 to fix some typos)initialysm (definition)2020-09-17T00:00:00+00:002020-09-17T00:00:00+00:00http://salewski.email/2020/09/17/word-def-initialysm<dl>
<dt><strong>initialysm:</strong> (<a href="https://www.merriam-webster.com/dictionary/noun" alt="noun"><em>noun</em></a>)</dt>
<dd>
<p>[<em>portmanteau of </em><a href="https://www.merriam-webster.com/dictionary/initially">initially</a> + <a href="https://www.merriam-webster.com/dictionary/initialism">initialism</a>
<br />
<em>syllables:</em> <code>ini·tial·ysm</code>
<br />
<em>pronunciation:</em> <code>i-ˈni-shə-ˌli-zəm</code>
<em>(pronounced identically to "initialism")</em>]</p>
<p>A word interpreted as an initialism that predates its own expansion;
such a word that was born backwards. As in, "Initially it was just
the initials, but later each letter was made to stand-in for a word
for mnemonic effect."</p>
<p>Like a <a href="http://catb.org/jargon/html/B/backronym.html">backronym</a>,
but is always pronounced as a series of letters.</p>
<p><em>Etymology</em><br />
The word <em>initialysm</em> was coined in order to explain the name
of the <strong><code>aagg</code></strong> program, a command line
program for use on
<a alt="Unix (wikipedia article)" href="https://en.wikipedia.org/wiki/Unix">Unix</a>-like systems.</p>
<p>The name of the program is pronounced "a a gee gee", with the
individual letters standing for the expansion phrase
"<strong>A</strong>WS-<strong>a</strong>-<strong>g</strong>o-<strong>g</strong>o".</p>
<p>The program was
initially named by finding a small number of characters that
combined to form an easy to type,
<a href="http://catb.org/jargon/html/G/grep.html">grep</a>-friendly
sequence on a standard
<a alt="QWERTY (jargon file)" href="http://catb.org/jargon/html/Q/QWERTY.html">QWERTY</a>
keyboard. Later, when it came time to
document the program for use by others, the expansion phrase
was retrofitted as a mnemonic device.</p>
</dd>
</dl>initialysm: (noun)Debian 9.x “stretch” APT pinning change2017-06-25T00:00:00+00:002017-06-25T00:00:00+00:00http://salewski.email/2017/06/25/debian-9.x-stretch-apt-pinning-change<p>Last week (on <code class="language-plaintext highlighter-rouge">2017-06-17</code>), Debian GNU/Linux 9.x (codenamed “stretch”)
<a href="https://www.debian.org/News/2017/20170617">was released</a>. While preparing to update one of
my systems I came across the following statement in
<a href="https://www.debian.org/releases/stable/amd64/release-notes/ch-information.en.html#apt-issues">section 5.3.2.2 of the release notes</a> (“New
requirements for APT repository”):</p>
<blockquote>
<p>5.3.2.2. New APT pinning engine</p>
<p>APT 1.1 introduced a new pinning engine that now matches the description in
the manual page.</p>
<p>The old engine assigned one pin priority per package; the new one assigns pin
priorities per version. It then picks the version with the highest pin that is
not a downgrade or that has a pin > 1000.</p>
<p>This changes the effect of some pins, especially negative ones. Previously,
pinning a version to -1 effectively prevented the package from being installed
(the package pin was -1); it now only prevents the version of this package
from being installed.</p>
</blockquote>
<p>I run my <a href="https://www.debian.org/">Debian</a> systems
<a href="http://without-systemd.org/">without systemd</a>, and my initial reading of the above
text gave me pause because I use APT pinning to prevent systemd from becoming
my init system. My initial reading of the above left me with the
misunderstanding that I would need to apply a different mechanism to keep my
current init setup (<code class="language-plaintext highlighter-rouge">'/sbin/init'</code> provided by <code class="language-plaintext highlighter-rouge">sysvinit-core</code> package).</p>
<p>The <em>default</em> <a href="http://blog.darknedgy.net/technology/2015/09/05/0/">init system</a> on Debian (since
Debian 8.x, codenamed “jessie”) has been <code class="language-plaintext highlighter-rouge">systemd</code>, in which <code class="language-plaintext highlighter-rouge">'/sbin/init'</code> is
provided by the <code class="language-plaintext highlighter-rouge">systemd-sysv</code> package. This continues to be the case with
Debian 9.x.</p>
<p>My existing setups use a customized set of APT preferences which, among other
things, contains the file:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> /etc/apt/preferences.d/LOCAL-avoid-systemd.pref
</code></pre></div></div>
<p>whose meat is:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> Package: systemd-sysv
Pin: release o=Debian
Pin-Priority: -1
</code></pre></div></div>
<p>The effect of the <code class="language-plaintext highlighter-rouge">'Pin-Priority'</code> of <code class="language-plaintext highlighter-rouge">-1</code> is to prevent APT from <em>ever</em>
installing the <code class="language-plaintext highlighter-rouge">'systemd-sysv'</code> package. And indeed, this is how the mechanism
continues to work in Debian 9.x “stretch”. I did not realize this without
closer study, however.</p>
<p>After spending more time than I care to admit searching for and reading
related info on the ‘Net, I came to realize that the statement in the release
notes above applies very specifically to how APT treats pinning of
<em>versions</em>. The above APT configuration, in contrast, is based on the separate
concept of <em>release origin</em>.</p>
<p>The above config file snippet applies the <code class="language-plaintext highlighter-rouge">-1</code> pin priority to any
<code class="language-plaintext highlighter-rouge">'systemd-sysv'</code> package in which the “Release file” declares the <em>origin</em> as
“Debian” (the <code class="language-plaintext highlighter-rouge">release o=Debian</code> bit). It <em>does not</em> contain any bits related
to the package <em>version</em>.</p>
<p>In retrospect, this is all rather obvious, and I’m embarrassed to think how
much time I spent researching it. It was also quite helpful to read the
motivation for the release notes blurb, which is contained in
<a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852967">bug #852967</a>, against the
<a href="https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=release-notes">‘release-notes’</a>
<a href="https://www.debian.org/Bugs/pseudo-packages">pseudo-package</a>.</p>
<p>Other than mucking around with my <code class="language-plaintext highlighter-rouge">'sources.list'</code> file (and related), I tend
to go many years between having to think about my APT configuration at all –
so it is easy to become hazy on the details. Hence my focus on the <code class="language-plaintext highlighter-rouge">-1</code> pin
priority but glossing over the specific language in the release notes of
“pinning a version” (which it is now clear that I am not doing).</p>
<p>I hope the above saves somebody from the need to research the same concerns I
had.</p>Last week (on 2017-06-17), Debian GNU/Linux 9.x (codenamed “stretch”) was released. While preparing to update one of my systems I came across the following statement in section 5.3.2.2 of the release notes (“New requirements for APT repository”):Terraform HOWTO: delete a non-empty AWS S3 bucket2017-04-30T00:00:00+00:002017-04-30T00:00:00+00:00http://salewski.email/2017/04/30/terraform-howto-delete-a-non-empty-aws-s3-bucket<p>The Amazon <a href="https://aws.amazon.com/">AWS</a> <a href="https://aws.amazon.com/s3/">S3 service</a> provides
<a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/delete-or-empty-bucket.html">a number of different ways</a> to delete a non-empty S3
bucket; some of the approaches involve “emptying” the bucket prior to deleting
it. The process can also vary a bit depending on whether or not the bucket has
<a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html">versioning</a> enabled.</p>
<p>When the <a href="https://www.terraform.io/docs/providers/aws/index.html">“aws”</a> provider is used, the
<a href="https://github.com/hashicorp/terraform">Terraform</a> program acts as a client to the AWS service, so has
a number of available approaches it can take when deleting S3 buckets.</p>
<p>When managing your infrastructure using Terraform, one common way to get rid
of an infrastructure resource (cause it to be destroyed) is to simply remove
it from your Terraform configuration (either by commenting-out it’s
configuration block or by deleting it from the configuration file entirely).</p>
<p>Non-empty S3 buckets throw a monkeywrench into that process. The data stored
as S3 objects within the bucket can be considered as separate (possibly
precious!) artitfacts, so a little extra convincing is needed to let Terraform
know that you really do want it to delete an S3 bucket resource <em>and</em> any data
objects it contains.</p>
<p>If you simply get rid of the configuration block for the bucket, the
<code class="language-plaintext highlighter-rouge">terraform plan</code> command will succeed in telling you that it would remove the
bucket (as you might expect):</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform plan
Refreshing Terraform state <span class="k">in</span><span class="nt">-memory</span> prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to <span class="nb">local </span>or remote state storage.
aws_s3_bucket.your_tf_s3_bucket_resource_name: Refreshing state... <span class="o">(</span>ID: your-bucket-name<span class="o">)</span>
The Terraform execution plan has been generated and is shown below.
Resources are shown <span class="k">in </span>alphabetical order <span class="k">for </span>quick scanning. Green resources
will be created <span class="o">(</span>or destroyed and <span class="k">then </span>created <span class="k">if </span>an existing resource
exists<span class="o">)</span>, yellow resources are being changed <span class="k">in</span><span class="nt">-place</span>, and red resources
will be destroyed. Cyan entries are data sources to be read.
Note: You didn<span class="s1">'t specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can'</span>t guarantee this is what will execute.
- aws_s3_bucket.your_tf_s3_bucket_resource_name
Plan: 0 to add, 0 to change, 1 to destroy.
</code></pre></div></div>
<p>However, if you were then run <code class="language-plaintext highlighter-rouge">terraform apply</code> you might be surprised by the
error:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> $ terraform apply
aws_s3_bucket.your_tf_s3_bucket_resource_name: Refreshing state... (ID: your-bucket-name)
aws_s3_bucket.your_tf_s3_bucket_resource_name: Destroying... (ID: your-bucket-name)
Error applying plan:
1 error(s) occurred:
* aws_s3_bucket.your_tf_s3_bucket_resource_name (destroy): 1 error(s) occurred:
* aws_s3_bucket.your_tf_s3_bucket_resource_name: Error deleting S3 Bucket: BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket.
status code: 409, request id: <request-id-value>, host id: <host-id-value> "your-bucket-name"
Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.
</code></pre></div></div>
<p>Despite the “delete all versions in the bucket” language of the error message,
the above error will appear regardless of whether or not the bucket has
versioning enabled.</p>
<h1 id="the-solution">The Solution</h1>
<p>Terraform <em>will</em> happily delete all of the objects in the bucket for you, but
you have to explicitly tell it to do so, and you have to know how to ask.</p>
<p>Let’s say you have the following as your S3 bucket configuration block in your
Terraform configuration file:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nx">resource</span> <span class="s2">"aws_s3_bucket"</span> <span class="s2">"my_s3_bucket_resource"</span> <span class="p">{</span>
<span class="nx">bucket</span> <span class="p">=</span> <span class="s2">"my-bucket-name"</span>
<span class="nx">acl</span> <span class="p">=</span> <span class="s2">"private"</span>
<span class="nx">versioning</span> <span class="p">{</span>
<span class="nx">enabled</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>
<span class="nx">tags</span> <span class="p">{</span>
<span class="nx">Name</span> <span class="p">=</span> <span class="s2">"whatev-name"</span>
<span class="nx">Environment</span> <span class="p">=</span> <span class="s2">"whatev-env"</span>
<span class="p">}</span>
<span class="nx">lifecycle</span> <span class="p">{</span>
<span class="c1"># Any Terraform plan that includes a destroy of this resource will</span>
<span class="c1"># result in an error message.</span>
<span class="c1">#</span>
<span class="nx">prevent_destroy</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>You’ll want to do two things to allow the above S3 bucket to be deleted:</p>
<ol>
<li>
<p>Comment-out (or remove) the <code class="language-plaintext highlighter-rouge">'prevent_destroy'</code> setting</p>
</li>
<li>
<p>Add a <code class="language-plaintext highlighter-rouge">'force_destroy'</code> setting</p>
</li>
</ol>
<p>Here’s our new version:</p>
<div class="language-hcl highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nx">resource</span> <span class="s2">"aws_s3_bucket"</span> <span class="s2">"my_s3_bucket_resource"</span> <span class="p">{</span>
<span class="nx">bucket</span> <span class="p">=</span> <span class="s2">"my-bucket-name"</span>
<span class="nx">force_destroy</span> <span class="p">=</span> <span class="kc">true</span>
<span class="nx">acl</span> <span class="p">=</span> <span class="s2">"private"</span>
<span class="nx">versioning</span> <span class="p">{</span>
<span class="nx">enabled</span> <span class="p">=</span> <span class="kc">true</span>
<span class="p">}</span>
<span class="nx">tags</span> <span class="p">{</span>
<span class="nx">Name</span> <span class="p">=</span> <span class="s2">"whatev-name"</span>
<span class="nx">Environment</span> <span class="p">=</span> <span class="s2">"whatev-env"</span>
<span class="p">}</span>
<span class="c1"># lifecycle {</span>
<span class="c1">#</span>
<span class="c1"># # Any Terraform plan that includes a destroy of this resource will</span>
<span class="c1"># # result in an error message.</span>
<span class="c1"># #</span>
<span class="c1"># prevent_destroy = true</span>
<span class="c1"># }</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now you can see the plan for destroying that bucket (and its dependencies)
with the command:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform plan <span class="nt">-destroy</span> <span class="nt">-target</span><span class="o">=</span>aws_s3_bucket.my_s3_bucket_resource
</code></pre></div></div>
<p>And then to actually have the bucket, its content, and its dependencies
destroyed:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform destroy <span class="nt">-target</span><span class="o">=</span>aws_s3_bucket.my_s3_bucket_resource
<span class="o">[</span>Terraform will prompt you to confirm, warning that there is no <span class="s1">'undo'</span> <span class="k">for </span>the action]
</code></pre></div></div>
<p>Here’s what it looks like in action:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform destroy <span class="nt">-target</span><span class="o">=</span>aws_s3_bucket.my_s3_bucket_resource
Do you really want to destroy?
Terraform will delete the following infrastructure:
aws_s3_bucket.my_s3_bucket_resource
There is no undo. Only <span class="s1">'yes'</span> will be accepted to confirm
Enter a value: <span class="nb">yes
</span>aws_s3_bucket.my_s3_bucket_resource: Refreshing state... <span class="o">(</span>ID: my-bucket-name<span class="o">)</span>
aws_s3_bucket.my_s3_bucket_resource: Destroying... <span class="o">(</span>ID: my-bucket-name<span class="o">)</span>
aws_s3_bucket.my_s3_bucket_resource: Destruction <span class="nb">complete
</span>Destroy <span class="nb">complete</span><span class="o">!</span> Resources: 1 destroyed.
</code></pre></div></div>
<h1 id="q-what-would-happen-if-we-did-not-comment-out-prevent_destroy">Q: What would happen if we did not comment-out <code class="language-plaintext highlighter-rouge">'prevent_destroy'</code>?</h1>
<p>You may be wondering what would happen if we did not comment-out the
<code class="language-plaintext highlighter-rouge">'lifecycle'</code> setting <code class="language-plaintext highlighter-rouge">prevent_destroy = true</code> while having
<code class="language-plaintext highlighter-rouge">force_destroy = true</code> set at the same time. The answer is that Terraform
would do what you would want it to do: it would refuse to delete the S3
bucket:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>terraform plan <span class="nt">-destroy</span> <span class="nt">-target</span><span class="o">=</span>aws_s3_bucket.my_s3_bucket_resource
Refreshing Terraform state <span class="k">in</span><span class="nt">-memory</span> prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to <span class="nb">local </span>or remote state storage.
aws_s3_bucket.my_s3_bucket_resource: Refreshing state... <span class="o">(</span>ID: my-bucket-name<span class="o">)</span>
Error running plan: 1 error<span class="o">(</span>s<span class="o">)</span> occurred:
<span class="k">*</span> aws_s3_bucket.my_s3_bucket_resource: aws_s3_bucket.my_s3_bucket_resource: the plan would destroy this resource, but it currently has lifecycle.prevent_destroy <span class="nb">set </span>to true. To avoid this error and <span class="k">continue </span>with the plan, either disable lifecycle.prevent_destroy or adjust the scope of the plan using the <span class="nt">-target</span> flag.
</code></pre></div></div>
<h1 id="terraform-version-094--released-2017-04-26">Terraform version: 0.9.4 (released 2017-04-26)</h1>
<p>The examples in this post all used <a href="https://github.com/hashicorp/terraform/blob/v0.9.4/CHANGELOG.md">terraform-0.9.4</a>
(released 2017-04-26), but are likely to work just fine with both earlier and
later versions. The earliest version expected to work (<em>not</em> tested!) is
<a href="https://github.com/hashicorp/terraform/blob/v0.5.3/CHANGELOG.md">terraform-0.5.3</a> (the version that introduced the
<code class="language-plaintext highlighter-rouge">'force_destroy</code>’ parameter for AWS S3 bucket resources).</p>
<h1 id="additional-reading">Additional Reading</h1>
<p>The rationale for Terraform not blindly deleting S3 objects was discussed in
<a href="https://github.com/hashicorp/terraform/issues/1977">hashicorp/terraform#1977</a>; the discussion there includes
the need for some sort of “force” option.</p>
<p>The <code class="language-plaintext highlighter-rouge">'force_destroy'</code> option was implemented in
<a href="https://github.com/hashicorp/terraform/pull/2007">hashicorp/terraform#2007</a>.</p>
<p>The <a href="https://www.terraform.io/docs/providers/aws/r/s3_bucket.html#force_destroy">‘force_destroy’</a> option is also documented
in the Terraform under the <a href="https://www.terraform.io/docs/providers/aws/index.html">“aws”</a> provider.</p>The Amazon AWS S3 service provides a number of different ways to delete a non-empty S3 bucket; some of the approaches involve “emptying” the bucket prior to deleting it. The process can also vary a bit depending on whether or not the bucket has versioning enabled.Al’s GitHub Pages with Jekyll cheatsheet2016-06-18T00:00:00+00:002016-06-18T00:00:00+00:00http://salewski.email/2016/06/18/github-pages-with-jekyll-cheatsheet<p>(Updated: 2020-05-05 to add notes on creating the orphan ‘gh-pages’ branch)</p>
<h2 id="preliminaries">Preliminaries</h2>
<h3 id="create-an-orphan-git-branch-named-gh-pages">Create an “orphan” git branch named ‘gh-pages’</h3>
<p><a href="https://pages.github.com/">GitHub Pages</a> will allow you to publish your project’s
site from a few different “sources”. Here we are concerned only with one of
those approaches: an “orphan” git branch named <code class="language-plaintext highlighter-rouge">'gh-pages'</code> within the repo.</p>
<p>The name <code class="language-plaintext highlighter-rouge">'gh-pages'</code> is not optional; it is treated specially by GitHub
Pages. You cannot choose an arbitrary branch name from which to publish your
project’s site; the branch must be called <code class="language-plaintext highlighter-rouge">'gh-pages'</code>.</p>
<p>Note that your project’s settings page in the GitHub web site will only show
the <code class="language-plaintext highlighter-rouge">'gh-pages'</code> branch as an option in the GitHub Pages “Source” drop-down if
the branch already exists in your project’s repository.</p>
<p>For the purposes of publishing your project’s site on GitHub Pages, you’ll
want the <code class="language-plaintext highlighter-rouge">'gh-pages'</code> branch to be an “orphan”. That is, you want it to have a
history that is entirely disconnected from the other existing branches in your
repository. To achieve that, we are going to use the <code class="language-plaintext highlighter-rouge">--orphan</code> option to
<a href="https://git-scm.com/docs/git-checkout" title="git-checkout(1)">git-checkout(1)</a>:</p>
<pre>
$ <strong>git checkout --orphan gh-pages</strong>
Switched to a new branch 'gh-pages'
</pre>
<p>At this point, a <code class="language-plaintext highlighter-rouge">git status</code> would show that all of the repo’s files that
were being tracked on the branch source point are now staged in the index; git
does this in anticipation of the user wanting to create a new, clean
go-forward history for the “as is” state of the files. That is not our use
case, however; we are going to use an entirely different set of files on our
new branch, so we can just clear out the git index:</p>
<pre>
$ <strong>git rm -fr .</strong>
$ <strong>git status</strong>
On branch gh-pages
No commits yet
nothing to commit (create/copy files and use "git add" to track)
</pre>
<p>Now you have a clean <code class="language-plaintext highlighter-rouge">'gh-pages'</code> branch with no commits. Read on to see how
we want to populate it, starting with a file named <code class="language-plaintext highlighter-rouge">'Gemfile'</code>.</p>
<h3 id="install-jekyll-and-other-related-ruby-based-tooling">Install Jekyll and other related Ruby-based tooling</h3>
<p>In your brand new working directory for a site you intend to generate with
<a href="https://jekyllrb.com/">Jekyll</a> and host with <a href="https://pages.github.com/">GitHub Pages</a>, you
will want to have some tooling available locally to allow you to veiw the
Jekyll-based site generation prior to pubishing it on GitHub Pages.</p>
<p>The Jekyll software is written in <a href="https://www.ruby-lang.org">Ruby</a>, and (as is true of most
Ruby software) is released in the form of a <a href="https://www.ruby-lang.org/en/libraries/">“gem”</a>.</p>
<pre>
$ <strong>cat - <<EOF > Gemfile</strong>
source 'http://rubygems.org'
gem 'github-pages', group: :jekyll-plugins
EOF
</pre>
<p>This will install (into the <code class="language-plaintext highlighter-rouge">'local-ruby-gems'</code> subdir) the dependencies
specified in the <code class="language-plaintext highlighter-rouge">'Gemfile'</code> (see <a href="https://bundler.io/man/gemfile.5.html">Gemfile(5)</a>), along with all
of <em>their</em> dependencies.</p>
<pre>
$ <strong>bundle install --path=./local-ruby-gems --verbose</strong>
</pre>
<pre>
$ <strong>cat - <<EOF > _config.yml
title: my-project-name by Alan D. Salewski
description: blah blah blah
markdown: kramdown
kramdown:
input: GFM # Enable GitHub Flavored Markdown (fensed code blocks)
hard-wrap: false
highlighter: rouge
exclude:
- local-ruby-gems/ # avoid processing jekyll itself; see also https://github.com/jekyll/jekyll/issues/2938
EOF</strong>
</pre>
<h2 id="regular-usage">Regular usage</h2>
<h3 id="prior-to-publishing-edit---review---commit-cycle">Prior to publishing: edit -> review -> commit cycle</h3>
<p>Once you have your tooling setup in your project’s git working directory, you
will basically just edit the <a href="https://daringfireball.net/projects/markdown/">Markdown</a> files of individual
articles in the <code class="language-plaintext highlighter-rouge">'_drafts'</code> subdirectory. The edit -> review -> commit cycle
works almost the same as any other software project, with commits happening
wherever makes sense for the particular article content.</p>
<p>The “review” phase mainly involves just rereading the article content and
making adjustments as necessary. Before committing, however, you should view
the Jekyll-rendered form of the article in your web browser. To do that, you
<em>could</em> just <code class="language-plaintext highlighter-rouge">'git push'</code> to GitHub Pages and hope for the best, but since
you’ve gone through the trouble of setting up your local Jekyll-based tooling
you should launch the local web server built into Jekyll for previewing your
changes.</p>
<p>This will generate the static site and render it the way it will be published
when you <code class="language-plaintext highlighter-rouge">'git push ...'</code> your changes up to GitHub:</p>
<pre>
$ <strong>bundle exec jekyll serve --watch</strong>
Configuration file: /path/to/project/_config.yml
Configuration file: /path/to/project/_config.yml
Source: /path/to/project
Destination: /path/to/project/_site
Incremental build: disabled. Enable with --incremental
Generating...
done in 0.389 seconds.
Auto-regeneration: enabled for '/path/to/project'
Configuration file: /path/to/project/_config.yml
Server address: http://127.0.0.1:4000/
Server running... press ctrl-c to stop.
</pre>
<p>The following variation of the same command (adds the <code class="language-plaintext highlighter-rouge">'--drafts'</code> option)
will generate the static site and render it with all of your unpublished
drafts integrated into the site:</p>
<pre>
$ <strong>bundle exec jekyll serve --watch --drafts</strong>
Configuration file: /path/to/project/_config.yml
Configuration file: /path/to/project/_config.yml
Source: /path/to/project
Destination: /path/to/project/_site
Incremental build: disabled. Enable with --incremental
Generating...
done in 0.639 seconds.
Auto-regeneration: enabled for '/path/to/project'
Configuration file: /path/to/project/_config.yml
Server address: http://127.0.0.1:4000/
Server running... press ctrl-c to stop.
</pre>
<p>In addition to the normal posts (everything beneath the <code class="language-plaintext highlighter-rouge">'_posts'</code> subdir),
the above command also includes everything beneath the <code class="language-plaintext highlighter-rouge">'_drafts/'</code> subdir.</p>
<p><strong>Q: Should I check the Gemfile in?</strong></p>
<p>(Concerned that it might have an adverse effect if interpreted by the remote
GitHub site builder.)</p>
<p><strong>A:</strong> Yes, empirical testing shows that the remote site builder does not break.</p>
<h3 id="publishing">Publishing</h3>
<p>Once you are happy with the content of an article, you can cause GitHub Pages
to render it by moving it from the <code class="language-plaintext highlighter-rouge">'_drafts/'</code> subdir to the
<code class="language-plaintext highlighter-rouge">'_posts/writings/'</code> subdir, with a publication date embedded in the file
name:</p>
<pre>
$ <strong>git mv _drafts/my-article.md _posts/writings/YYYY-MM-DD-my-article.md</strong>
</pre>
<p>Push your locally committed changes to GitHub to have them rendered by GitHub
Pages:</p>
<pre>
$ <strong>git push --follow-tags origin master</strong>
...
</pre>(Updated: 2020-05-05 to add notes on creating the orphan ‘gh-pages’ branch)