For me, a good programmer is someone who can influence their peers. A good programmer’s style and approach to solving problems is initially imitated by others until at some point it becomes “common sense” for them. Such influence can result in both horrible and excellent programming practices.
I use hetzner cloud (they operate in de and fi) and scaleway (fr and nl). Both are similar if you just need compute, but scaleway can be interesting if you want arm-based servers or a blobstore.
vultr is American, but they have datacenters globally, so that could also be an option. They also support BSDs ootb.
Self-hosted on OpenBSD with OpenSMTPD and dovecot. Self-hosting my emails for over a decade so I’ve been through all ups and downs. I like to run my own stuff, have a maximum level of privacy and always learn new stuff. On the downside, I nearly lost my complete inbox twice (restored from backups, so take backups!), learned very fast that having a primary and a backup MX is different from having two primaries.
I am also self-hosting using OpenBSD, OpenSMTPD and dovecot for a number of years. I’ve got a primary and a secondary server with SPF and DKIM. My netblock was blacklisted by outlook.com but was easy enough to fix by filling into an online form.
I think it’s really cool that you are self-hosted but I have to ask; how are your delivery rates? Do you have DKIM and SPF records? I know it’s quite the challenge to develop a good sending reputation so I am always curious to see how others fare.
I have SPF records (mainly to make google happy) but no DKIM. However, DKIM is not a hassle to set up. There are plenty of good howtos out there.
I cannot complaint about reputation, it seems all my email reach the recipient (and yes, also the ones at gmail). I once had some trouble with outlook.com and German Telekom when I had a system at Hetzner because their IP addresses have a very bad reputation. Once I moved away, everything works fine.
It jumps out to me that you had to include an estimate of your connection speed in the configuration. What’s the behavior if Comcast either gives you a free performance boost, or runs in a degraded state with lower performance?
If the actual bandwidth is more than what’s specified, it won’t hurt but you will be artificially limiting yourself. If it is less than what’s specified, then it probably won’t work very well because even though the buffers on your router will remain empty (LAN<->WAN both at 1Gbps), the next device in sequence will start buffering and that’s outside of your control at this point. In other words, you want the router that is doing QoS to be the bottleneck.
On OpenBSD, if you don’t specify the bandwidth param, then it will default to whatever rate the NICs are running at (10/100/1000Mbps for example).
Note that SMT doesn’t necessarily have a posive effect on performance; it highly depends on the workload. In all likelyhood it will actually slow down most workloads if you have a CPU with more than two cores.
In case you’re wondering, this refers to OpenBSD’s giant-locked kernel. Some parts of this kernel are now unlocked (e.g. network stack) but for some workloads 2 CPUs can be faster than 3 or more due to lock contention.
Per my understanding, every “physical” CPU can have many cores, and each core can have multiple hardware thread if SMT is supported. So every “hardware thread” is a “logical” CPU. For OpenBSD kernel, does it do special operations according to physical CPU, core and hardware thread? Or just consider “logic” CPU? Thanks!
If I understand correctly, disable SMT means cut half the “logical” CPU, right? For example, if the server has one CPU, 2 cores, and every core has 2 hardware threads, in theory, the server has 4 “logical” CPUs. Assume my workload has 4 thread, and every thread is independent and computing-intensive (mostly user-space computation, not involved kernel part, such as syscall, or accessing network, etc.). Currently the workload can occupy the whole 4 “logical” CPUs. But now, if the count of “logical” CPU is halved, and my workload’s 4 thread need to contend for 2 “logical” CPUs. So in this scenario, the workload’s performance should be downgraded.
At least when HT was new, it also meant the caches would be halved unless you disabled HT in bios. So if your threads are doing different things they might suffer from it.
I have tried surf and ran for for quite some days, but there were a number of problems I had with it.
It is unstable. It crashed way too often for me to be used as my day-to-day browser.
I know no reliable way to do adblocking with it. Just fiddling with /etc/hosts is not really enough; many pages look weird if you do that. Adblockers in Firefox have never shown this problem for me. And keeping /etc/hosts up-to-date is a pain.
Enabling JavaScript for a page only on demand does not work. I want it off by default.
surf rejects a number of SSL websites Firefox accepts for no obvious reason (especially bad with lets-encrypt sites). In contrast to what the article says, surf does support SSL, though. Just not in the stable version I have found.
I have no idea how to create the facility of a plugin I very much like, Flagfox. It displays a little country flag in Firefox' URL bar depending on where it thinks the IP is from.
Bookmarks. I know I can manage them with scripts, but until now I have been too lazy for that, since a proper script would allow me to search the bookmark list and then directly follow the link. I often find myself remembering the title of an article I read and bookmarked, but not the page. So it is required that the bookmark link is stored together with the title and can be selected by either title or URL. As said, possible, but I’m just too lazy for that.
How to forbid 3rd party cookies? How to delete all cookies after quitting the last surf instance?
There were probably more points which I don’t remember anymore.
Now I’m back on Firefox and have turned on the start-search-by-typing option. This gives me the required level of keyboard navigation I need – I can just type in the text of a link and Firefox will select it. There is a surprising amount of useful keyboard shortcuts in Firefox that is a little bit hidden (for example, by typing ‘ [single quote] with the start-search-by-typing option enabled you search only the links of a page, very useful).
The SSL woes are more than likely because Firefox caches sub-CAs it sees in the wild to handle all the badly configured webservers that do not serve the whole certificate chain when connecting.
I really wish browsers did not do this as it masks a problem that to the sysadmin running the site looks just like a temporary glitch in the matrix that they can ignore.
surf rejects a number of SSL websites Firefox accepts for no obvious reason (especially bad with lets-encrypt sites). In contrast to what the article says, surf does support SSL, though. Just not in the stable version I have found.
There are some issues with TLS, at least on my Fedora system (visiting the badssl.com dashboard). For example, no host matching, no check for expired certificate, etc.
I really like these kind of writeups, both tedu’s but also the post mentioned from poolp.org. I do think it’s an unfortunate trend that all these lovely things are buried away from openbsd.org or undeadly. Maybe the world needs a ‘Planet OpenBSD’ where all the developer’s blogs are syndicated?
planet.openbsd.org doesn’t appear to currently be a thing.
I’d rather your feed had a single but fulltext entry than 10 but abbreviated ones. (At least as long as you don’t post twice within half a day or so… which I don’t remember seeing.)
Do you happen to know which readers replace content when it changes? That was my other concern, that i update something, but readers cache a frozen version.
Don’t all of them? I can’t remember seeing one that doesn’t. No doubt they do exist, but I doubt they are at all common. I can remember ones all the way out at the opposite extreme, where they version they content and offer diffs in the UI. NewsBlur has that in some capacity, and there was a desktop one on the Mac that did this – probably old NetNewsWire.
Frozen caches really happen when items get updates after falling off the bottom of the feed. Obviously aggregators won’t see content you didn’t put in the feed… so item inclusion for the feed must be based on update date rather than creation date, if that’s a concern.
(Btw, while we’re here… could you use proper <category>s in the item, instead of putting a line with <p>tagged: at the bottom of your description and then me having to sed your feed to fix that?)
Yup. I recommend http://www.rssboard.org/rss-profile for reference, which is lamentably difficult to stumble upon serendipitously. It includes recommendations based on surveys of publishers and aggregators in the wild… well, from 10 years ago, but still.
Hm, if that peril is also the reason you don’t have a <guid>… that would be nice, because in absence of it, aggregators must guess how to identify an item as being the same one throughout edits. For flak you can just switch the <link> to <guid> I think (you never change those URLs, right?)… or have both if you worry about edge-case aggregators. For inks, I’ve noticed you number the blocks in the HTML, so you already have an identifier to reuse – keep the <link> and add a <guid isPermaLink="false">, probably with a tag: URL, maybe tag:www.tedunangst.com,2016:inks:37 (where only the trailing number varies; the date is just any point in time you controlled the domain, it can be constant). That would go a long way to ensuring that your updates to items do come through as updates, rather than showing up as dupes. (That’s part of the reason I sed your feed – I’d get dupes all the time when you edited your inks tags, which you do quite a bit, whereas metadata doesn’t figure into the deduping in Liferea, so now I only get dupes anymore when you actually update the item description.)
I missed that announcement :( I had heard rumours that it was nearing the door, but didn’t realise it was going to be so soon. Guess it’s the passing of an era as it was Theo’s massive patchset for NetBSD/sparc that was key during the lead up to the fork (for those who’ve never read it, coremail is a fascinating read - lobsters story).
I still have a few 32-bit SPARC systems (not used for anything productive - I’m a huge fan of the SPARCstation 20) - I guess NetBSD is the only viable option now.
Keep them. My best recommendation for dealing with potential NSA subversion was putting root of trust on old, esp ancient, hardware that likely predated subversion. One can put a trusted interface in front of them to force simple, mediated communication to the app. Yet, gotta make sure hardware itself isn’t bacdoored. Odds strongly against that on a SPARCstation 20 or a VAX. ;)
Note: Another benefit is in chasing the holy grail of automated generation of correct, secure, and portable software. Need lots of ISA’s and machines to test such tooling on. A tool with 10 implementations running full coverage testing on 50 machines from mutually-suspicious countries with same, correct output for every input inspires much confidence. For me at least.
Note 2: Intel’s i960 should be on that list. It’s still available in watered-down form. The original was one of their best designs. They’re the assholes that locked up Alpha’s, too. Briefly licensed by them and Samsung. They need to FOSS the last Alpha implementation if they still have it given OpenPOWER and OpenSPARC. I wan’t PALcode damnit! :)
Looks too complicated to be a useful starting point for anyone not comfortable writing their own Makefile. I think make(1) should be studied in the same way one studies sh(1), yacc(1) etc. Once the main points are understood it is fairly trivial to write a minimal Makefile that gets the job done.
Yep. The whole mess was started at the time I was not quite enjoying makefile. easymake has gave me some sweet time. It’s not quit generic or extensible, though.
“It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.” - Edsger W. Dijkstra
QuickBASIC, of course, has almost nothing other than a common set of keywords and sigils to do with the language that Dijkstra was railing against. It’s structured in exactly the sense he supported. It still supports GOTO, but you’ll find it rarely if at all in idiomatic QuickBASIC code.
Just blindly installed it yesterday on a old PC (after I failed trying to reinstall 9front.. 3 times), I was not expecting to find 5.9 but I haven’t been keeping up with the release cycles so I thought I’d just have forgotten about it.
Sadly it’s old enough to miss both VT-x, amd64 and UEFI, so I don’t get to try any of the new goodness.. thought I guess “pledge” works?
Follow the recommended security practices. This guide is only useful to show the details of performing the installation on that particular machine without attempting to replicate information found in the OpenBSD manpages or FAQ.
Relevant: http://www.catb.org/~esr/faqs/smart-questions.html
For me, a good programmer is someone who can influence their peers. A good programmer’s style and approach to solving problems is initially imitated by others until at some point it becomes “common sense” for them. Such influence can result in both horrible and excellent programming practices.
I use hetzner cloud (they operate in de and fi) and scaleway (fr and nl). Both are similar if you just need compute, but scaleway can be interesting if you want arm-based servers or a blobstore.
vultr is American, but they have datacenters globally, so that could also be an option. They also support BSDs ootb.
I also use hetzner cloud (their cheapest offering) and I run OpenBSD 6.4 on it with IPv4 and IPv6, no issues at all.
J.C.R. Licklider and most of the Bell Lab folks.
I have a ThinkPad x250. It is a small, portable machine and runs quite well. You can find them cheap on ebay. Previously I had a t420 and an x200.
Self-hosted on OpenBSD with OpenSMTPD and dovecot. Self-hosting my emails for over a decade so I’ve been through all ups and downs. I like to run my own stuff, have a maximum level of privacy and always learn new stuff. On the downside, I nearly lost my complete inbox twice (restored from backups, so take backups!), learned very fast that having a primary and a backup MX is different from having two primaries.
I am also self-hosting using OpenBSD, OpenSMTPD and dovecot for a number of years. I’ve got a primary and a secondary server with SPF and DKIM. My netblock was blacklisted by outlook.com but was easy enough to fix by filling into an online form.
I also recommend to get yourself onto whitelists like https://www.dnswl.org/.
I think it’s really cool that you are self-hosted but I have to ask; how are your delivery rates? Do you have DKIM and SPF records? I know it’s quite the challenge to develop a good sending reputation so I am always curious to see how others fare.
I have SPF records (mainly to make google happy) but no DKIM. However, DKIM is not a hassle to set up. There are plenty of good howtos out there.
I cannot complaint about reputation, it seems all my email reach the recipient (and yes, also the ones at gmail). I once had some trouble with outlook.com and German Telekom when I had a system at Hetzner because their IP addresses have a very bad reputation. Once I moved away, everything works fine.
Did the same 4/5 years ago. Never looked back and would not go back to a third-party provider for a million bucks.
It jumps out to me that you had to include an estimate of your connection speed in the configuration. What’s the behavior if Comcast either gives you a free performance boost, or runs in a degraded state with lower performance?
There’s not much you can do about that. You could always monitor it with a cron job and update the params/rules with a script I suppose.
Right, but what happens? How does the system behave in those circumstances?
If the actual bandwidth is more than what’s specified, it won’t hurt but you will be artificially limiting yourself. If it is less than what’s specified, then it probably won’t work very well because even though the buffers on your router will remain empty (LAN<->WAN both at 1Gbps), the next device in sequence will start buffering and that’s outside of your control at this point. In other words, you want the router that is doing QoS to be the bottleneck.
On OpenBSD, if you don’t specify the bandwidth param, then it will default to whatever rate the NICs are running at (10/100/1000Mbps for example).
Thanks!
In case you’re wondering, this refers to OpenBSD’s giant-locked kernel. Some parts of this kernel are now unlocked (e.g. network stack) but for some workloads 2 CPUs can be faster than 3 or more due to lock contention.
Per my understanding, every “physical” CPU can have many cores, and each core can have multiple hardware thread if SMT is supported. So every “hardware thread” is a “logical” CPU. For OpenBSD kernel, does it do special operations according to physical CPU, core and hardware thread? Or just consider “logic” CPU? Thanks!
As far as I know the SMT threads were simply exposed as additional CPUs to the scheduler.
@stsp Thanks for your response!
If I understand correctly, disable SMT means cut half the “logical” CPU, right? For example, if the server has one CPU, 2 cores, and every core has 2 hardware threads, in theory, the server has 4 “logical” CPUs. Assume my workload has 4 thread, and every thread is independent and computing-intensive (mostly user-space computation, not involved kernel part, such as syscall, or accessing network, etc.). Currently the workload can occupy the whole 4 “logical” CPUs. But now, if the count of “logical” CPU is halved, and my workload’s 4 thread need to contend for 2 “logical” CPUs. So in this scenario, the workload’s performance should be downgraded.
Is it correct? Thanks in advance!
At least when HT was new, it also meant the caches would be halved unless you disabled HT in bios. So if your threads are doing different things they might suffer from it.
As far as I understand, it doesn’t mean that all 4 threads can progress in parallel, it will depend on which unit in the CPU each thread is utilizing.
I have a TODO file in my home directory for generic tasks and separate TODO files in the target project directories.
I have tried surf and ran for for quite some days, but there were a number of problems I had with it.
There were probably more points which I don’t remember anymore.
Now I’m back on Firefox and have turned on the start-search-by-typing option. This gives me the required level of keyboard navigation I need – I can just type in the text of a link and Firefox will select it. There is a surprising amount of useful keyboard shortcuts in Firefox that is a little bit hidden (for example, by typing ‘ [single quote] with the start-search-by-typing option enabled you search only the links of a page, very useful).
The SSL woes are more than likely because Firefox caches sub-CAs it sees in the wild to handle all the badly configured webservers that do not serve the whole certificate chain when connecting.
I really wish browsers did not do this as it masks a problem that to the sysadmin running the site looks just like a temporary glitch in the matrix that they can ignore.
I hate the web.
For adblocking, you could use http://git.codemadness.nl/surf-adblock/ with surf-webkit2.
There are some issues with TLS, at least on my Fedora system (visiting the badssl.com dashboard). For example, no host matching, no check for expired certificate, etc.
I really like these kind of writeups, both tedu’s but also the post mentioned from poolp.org. I do think it’s an unfortunate trend that all these lovely things are buried away from openbsd.org or undeadly. Maybe the world needs a ‘Planet OpenBSD’ where all the developer’s blogs are syndicated?
planet.openbsd.org doesn’t appear to currently be a thing.
https://bronevichok.ru/openbsd-planet/
Argh, no full text RSS feed. Why do people persist in doing that (and making me jump through [minor] hoops to work around it)?
In my case, because it would push tons of unnecessary traffic.
I’d rather your feed had a single but fulltext entry than 10 but abbreviated ones. (At least as long as you don’t post twice within half a day or so… which I don’t remember seeing.)
Do you happen to know which readers replace content when it changes? That was my other concern, that i update something, but readers cache a frozen version.
Don’t all of them? I can’t remember seeing one that doesn’t. No doubt they do exist, but I doubt they are at all common. I can remember ones all the way out at the opposite extreme, where they version they content and offer diffs in the UI. NewsBlur has that in some capacity, and there was a desktop one on the Mac that did this – probably old NetNewsWire.
Frozen caches really happen when items get updates after falling off the bottom of the feed. Obviously aggregators won’t see content you didn’t put in the feed… so item inclusion for the feed must be based on update date rather than creation date, if that’s a concern.
(Btw, while we’re here… could you use proper
<category>s in the item, instead of putting a line with<p>tagged:at the bottom of your description and then me having tosedyour feed to fix that?)Oh? category is a thing? that seems doable. the perils of writing everything from scratch.
Yup. I recommend http://www.rssboard.org/rss-profile for reference, which is lamentably difficult to stumble upon serendipitously. It includes recommendations based on surveys of publishers and aggregators in the wild… well, from 10 years ago, but still.
Hm, if that peril is also the reason you don’t have a
<guid>… that would be nice, because in absence of it, aggregators must guess how to identify an item as being the same one throughout edits. For flak you can just switch the<link>to<guid>I think (you never change those URLs, right?)… or have both if you worry about edge-case aggregators. For inks, I’ve noticed you number the blocks in the HTML, so you already have an identifier to reuse – keep the<link>and add a<guid isPermaLink="false">, probably with atag:URL, maybetag:www.tedunangst.com,2016:inks:37(where only the trailing number varies; the date is just any point in time you controlled the domain, it can be constant). That would go a long way to ensuring that your updates to items do come through as updates, rather than showing up as dupes. (That’s part of the reason Isedyour feed – I’d get dupes all the time when you edited your inks tags, which you do quite a bit, whereas metadata doesn’t figure into the deduping in Liferea, so now I only get dupes anymore when you actually update the item description.)Ah, cool. My understanding of RSS readers is heavily influenced by the one I wrote, which is also odd in its own way.
Hey, thanks for all the fixes! Much appreciated.
I’m still disappointed that VAX support is no longer present, but pulling it was the right decision. I guess we’ll always have 4.3 Quasijarus!
I just hope SPARC isn’t next to go…
sparc was just removed on OpenBSD -current. sparc64 is still there.
I missed that announcement :( I had heard rumours that it was nearing the door, but didn’t realise it was going to be so soon. Guess it’s the passing of an era as it was Theo’s massive patchset for NetBSD/sparc that was key during the lead up to the fork (for those who’ve never read it, coremail is a fascinating read - lobsters story).
I still have a few 32-bit SPARC systems (not used for anything productive - I’m a huge fan of the SPARCstation 20) - I guess NetBSD is the only viable option now.
Keep them. My best recommendation for dealing with potential NSA subversion was putting root of trust on old, esp ancient, hardware that likely predated subversion. One can put a trusted interface in front of them to force simple, mediated communication to the app. Yet, gotta make sure hardware itself isn’t bacdoored. Odds strongly against that on a SPARCstation 20 or a VAX. ;)
Got a list of them here: https://www.schneier.com/blog/archives/2013/09/surreptitiously.html#c1762647
Note: Another benefit is in chasing the holy grail of automated generation of correct, secure, and portable software. Need lots of ISA’s and machines to test such tooling on. A tool with 10 implementations running full coverage testing on 50 machines from mutually-suspicious countries with same, correct output for every input inspires much confidence. For me at least.
Note 2: Intel’s i960 should be on that list. It’s still available in watered-down form. The original was one of their best designs. They’re the assholes that locked up Alpha’s, too. Briefly licensed by them and Samsung. They need to FOSS the last Alpha implementation if they still have it given OpenPOWER and OpenSPARC. I wan’t PALcode damnit! :)
is this an old text? This fact should be mentioned in the title, shouldn’t it ?
I’ve amended the title to include the date.
Looks too complicated to be a useful starting point for anyone not comfortable writing their own Makefile. I think make(1) should be studied in the same way one studies sh(1), yacc(1) etc. Once the main points are understood it is fairly trivial to write a minimal Makefile that gets the job done.
Yep. The whole mess was started at the time I was not quite enjoying makefile. easymake has gave me some sweet time. It’s not quit generic or extensible, though.
“It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.” - Edsger W. Dijkstra
QuickBASIC, of course, has almost nothing other than a common set of keywords and sigils to do with the language that Dijkstra was railing against. It’s structured in exactly the sense he supported. It still supports
GOTO, but you’ll find it rarely if at all in idiomatic QuickBASIC code.One of the hazards of quoting a bon mot without pausing to understand it.
https://arcetera.moe/git/pg/
no more, no less, just a pager
Cool. What do you use to host your git projects?
That’s stagit: http://git.2f30.org/stagit/
On OpenBSD I use ksh. On my workstation at work, I use mksh.
Just blindly installed it yesterday on a old PC (after I failed trying to reinstall 9front.. 3 times), I was not expecting to find 5.9 but I haven’t been keeping up with the release cycles so I thought I’d just have forgotten about it.
Sadly it’s old enough to miss both VT-x, amd64 and UEFI, so I don’t get to try any of the new goodness.. thought I guess “pledge” works?
vmm(4) is not enabled in 5.9 so you are not missing out on that.
Skipping signify verification I see. What’s the point in that?
Follow the recommended security practices. This guide is only useful to show the details of performing the installation on that particular machine without attempting to replicate information found in the OpenBSD manpages or FAQ.
The author should consider hosting his own server and using whatever markdown implementation is comfortable with instead of relying on github.