The YouTube channel here seems to be a person who needs to be dramatic for view reasons. I think the actual content, and the position of the Ghostty author here on this topic, is pretty mild.
An actual bit from the video:
Guest: “…I don’t know, I’m questioning everything about Go’s place in the stack because […reasonable remarks about design tradeoffs…]”
Host: “I love that you not only did you just wreck Go […]”
Aside… In the new year I’ve started reflexively marking videos from channels I follow as “not interested” when the title is clickbait, versus a succinct synopsis of what the video is about. I feel like clickbait and sensationalism on YouTube is out of control, even among my somewhat curated list of subscribed channels.
This is why I can’t stand almost any developer content on YouTube and similar platforms. They’re way too surface-level, weirdly obsessed with the inane horse race of finding the “best” developer tooling, and clickbait-y to a laughable degree. I have >20 years of experience, I’m not interested in watching someone blather on about why Go sucks when you could spend that time on talking about the actual craft of building things.
But, no, instead we get an avalanche of beginner-level content that lacks any sort of seriousness.
This is why I really like the “Developer Voices” channel. Great host, calm and knowledgeable. Interesting guests and topics. Check it out if you don’t know it yet.
I’m in a similar boat. Have you found any decent channels that aren’t noob splooge? Sometimes I’ll watch Asahi Lina, but I haven’t found anything else that’s about getting stuff done. Also, non-OS topics would be nice additions as well.
7 (7!) Years ago LTT made a video about why their thumbnails are so… off putting and it essentially boiled down to “don’t hate the player; hate the game”. YouTube rewards that kind of content. There’s a reason why nearly every popular video these days is some variant of “I spent 50 HOURS writing C++” with the thumbnail having a guy throwing up. If your livelihood depends on YouTube, you’re leaving money on the table by not doing that.
It’s not just “Youtube rewards it”, it’s that viewers support it. It’s a tiny, vocal minority of people who reject those thumbnails. The vaaaaast majority of viewers see them and click.
I don’t think you can make a definitive statement either way because YouTube has its thumb on the scales. Their algorithm boosts videos on factors other than just viewer click through or retention rates (this has also been a source of many superstitions held by YouTubers in the past) and the way the thumbnail, title and content metas have evolved make me skeptical that viewers as a whole support it.
What is the alternative? That they look at the image and go “does this person make a dumb face” ? Or like “there’s lots of colors” ? I think the simplest explanation is that people click on the videos a lot.
…or it’s just that both negative and positive are tiny slices compared to neutrals but the negative is slightly smaller than the positive.
(I use thumbnails and titles to evaluate whether to block a channel for being too clickbait-y or I’d use DeArrow to get rid of the annoyance on the “necessary evil”-level ones.)
I am quite happy to differ in opinion to someone who says ‘great content’ unironically. Anyway your response is obviously a straw man, I’m not telling Chopin to stop composing for a living.
Your personal distaste for modern culture does not make it any higher or lower than Chopin, nor does it invalidate the fact that the people who make it have every right to make a living off of it.
They literally don’t have a right to make a living from Youtube, this is exactly the problem. Youtube can pull the plug and demonetise them at any second and on the slightest whim, and they have absolutely no recourse. This is why relying on it to make a living is a poor choice. You couldn’t be more diametrically wrong if you tried. You have also once again made a straw man with the nonsense you invented about what I think about modern culture.
How’s that any different from the state of the media industry at any point in history? People have lost their careers for any reason in the past. Even if you consider tech or any other field, you’re always building a career on top of something else. YouTube has done more to let anyone make a living off content than any other stage in history, saying you’re choosing poorly to make videos for YouTube is stupid.
You have also once again made a straw man with the nonsense you invented about what I think about modern culture
You’re the one who brought it up:
I am quite happy to differ in opinion to someone who says ‘great content’ unironically
Isn’t this kind of a rigid take? Why is depending on youtube a poor choice? For a lot of people, I would assume it’s that or working at a fast-food restaurant.
Whether that’s a good long-term strategy, or a benefit to humanity is a different discussion, but it doesn’t have to necessarily be a poor choice.
Not really?
I mean sure if you’ve got like 1000 views a video then maybe your livelihood depending on YouTube is a poor choice.
There’s other factors that come into this, but if you’ve got millions of views and you’ve got sponsors you do ad-reads for money/affiliate links then maybe you’ll be making enough to actually “choose” YouTube as your main source of income without it being a poor choice (and it takes a lot of effort to reach that point in the first place).
We’ve been seeing this more and more. You can, and people definitely do, make careers out of YouTube and “playing the game” is essential to that.
Heh - I had guessed who the host would be based on your comment before I even checked. He’s very much a Content Creator (with all the pandering and engagement-hacking that implies). Best avoided.
Your “ghostty author” literally built a multibillion dollar company writing Go for over a decade, so Im pretty sure his opinion is not a random internet hot take.
Yup. He was generally complimentary of Go in the interview. He just doesn’t want to use it or look at it at this point in his life. Since the Lobsters community has quite an anomalous Go skew, I’m not surprised that this lack of positivity about Go would be automatically unpopular here.
And of course the title was click-baity – but can we expect from an ad-revenue-driven talk show?
I was able to get the incremental re-builds down to 3-5 seconds on a 20kloc project with a fat stack of dependencies which has been good enough given most of that is link time for a native binary and a wasm payload. cargo check via rust-analyzer in my editor is faster and does enough for my interactive workflow most of the time.
Don’t be a drama queen ;-) You can, all you want. That’s what most people do.
The host actually really likes Go, and so does the guest. He built an entire company where Go was the primary (only?) language used. It is only natural to ask him why he picked Zig over Go for creating Ghostty, and it is only natural that the answer will contrast the two.
There’s a lot of stuff that would be easy for the right person which I don’t know well, such as modifying my bash prompt, writing a simple web scraper (all my experience is with Data Science/Data Engineering), or working with the latest AWS library – they’ll often give me code that sets up a basic example and almost works, that I can quickly turn into something that does work.
LLM-based tab completion is quite nice, if you’re ok with a lot of false positives. I’ve pretty much stopped using vim macros and multiple cursors, because doing something once and letting the LLM suggest the remaining changes is so much faster. And the false positives are offset by the number of typos that it’s caught for me.
The major downside is code quality – any significant chunk of code ends up being extremely verbose and often redundant, to the point where I’ll essentially rewrite all the code once I understand it. This makes code generation useless for anything I’m halfway decent at.
Yeah, to me an LLM is like autotune for a different domain. It’ll get something passable out the door but won’t impress somebody who can actually sing.
Google has their own internal AI codey help thingy. I use the auto-complete a lot, as it’s really quite good at figuring out what I am trying to do and so I can get from 5 typed characters to 25 auto-completed characters quite fast. It’s really removed a lot of boilerplate, especially in Java which is boilerplate city.
Apparently, I can do all sorts of cool things like ask the LLM to generate me test cases and such. I just don’t trust it at the end of the day. The autocomplete-y has burned me a few times on hallucinations that I then had to manually find and patch up. Would I have introduced more or less bugs this way? I don’t know. I do feel like the hallucinations generate more tricky to find bugs, but I have no numbers. Would it generate good unit tests? Maybe. Maybe I am just old and view my job as a creative writer rather than a proof-reader for an LLM. Maybe the current generation of CS undergrads will have this new focus. Maybe not.
It’s just a tool, and you either find value in it or you don’t. Copilot and stuff was not good enough when it came out, so I am not surprised people that were burned by it aren’t interested in using it again.
I know engineers who don’t know how nor care to change their editor colors and font. They appear to be doing their job just fine. Let people use what they want and they need, and don’t bother judging them for it.
Yeah I was just going to write something along these lines – I think we need some terminology to describe when AI is useful, along the lines of when StackOverflow is useful. There are some cases where copying from StackOverflow is bad engineering, and some where it can be good.
Personally I find StackOverflow to be extremely useful … I always write tests for any snippets I get from there, even 1 line solutions, and that helps me understand it. Overall it can still really accelerate software creation
I think there’s an asymmetry of the “cost to be wrong”
writing new production code - if the AI hallucinates, then you may be shipping buggy code
the cost of shipping buggy code varies a lot, but in some cases it kills people, etc.
writing test cases - if the AI hallucinates, then your code may grow bugs without you noticing
Of course, that can also happen if you don’t write tests yourself. Unfortunately I don’t think the status quo as far as developers writing tests is very good. Testing is an underappreciated art.
I think the best case is that it helps you find cases that you MISSED. But unfortunately people might use it to avoid thinking about testing in the first place.
Writing one-off scripts to aid software development:
I saw one post where Andreas Kling said he used ChatGPT to analyze object leaks in a garbage collector
if the AI is wrong here, then at worst you’ve wasted your time.
Generating pull request descriptions
Personally I do NOT want this. I don’t want my co-workers generating summaries of commits. Because the act of writing descriptions actually clarifies what I want to do.
Half the time I STOP in the middle of writing a description, and realize I need to do something else, change something, or just realize that my assumptions aren’t fully examined. I write the caveats in the description too.
I’ve also received pull requests where the submitter seemed confused about what their goal was
Writing bash scripts for deployment - I saw a recent post about this
I can see a problem where these become permanent technical debt that nobody understands! Also, I believe LLMs kinda copy the status quo of bash coding, which is BAD. They can propagate existing bad practices
Using AI for auto-complete:
If you are working in an unfamiliar language
If you are not a good typist
(Personally I am a fast typer in the languages I know, and I try to stick to that happy / flow state path. So I don’t use LSPs at all, but I occasionally use AI. I might use LSPs if I were learning an unfamiliar stack/language, though I actually find over-reliance on LSPs to be a “smell”. Similarly I might regard over-reliance on AIs to be a smell.)
Other reasons:
Using Claude/ChatGPT as search, because Google is declining, and the results pages are riddled with ads and SEO bullshit (this is kicking the can down the road, because Claude/ChatGPT aren’t sustainable, but it’s also practical in many cases …)
I wonder if anyone has broken down all these use cases – and summarized the pitfalls. I’d be interested in reading experiences
There might be an analogy to “Type I” and “Type II” errors in statistics (which are the most horrible namess, as recently pointed out here)
But I think there should be some words/memes for this distinction
Unfortunately the discourse seems to be very polarized now – “AGI is going to make programmers obsolete” vs. “AI is snake Oil”
But as usual the answer is somewhere in between …. At some point it will calm down, and we will still have these tools, and we’ll learn when to use them, and when not to use them
Or judging by the industry for the last ~10 years, we can write pretty bad code for a long time, and people still have to use it, which is scary …
writing new production code - if the AI hallucinates, then you may be shipping buggy code
the cost of shipping buggy code varies a lot, but in some cases it kills people, etc.
Are you assuming that the programmer isn’t reading/testing the code output by an LLM? Plenty of the code I get from LLMs is buggy, but that doesn’t mean it isn’t useful on net – so far, none of those bugs have made it into production.
I saw one post where Andreas Kling said he used ChatGPT to analyze object leaks in a garbage collector
if the AI is wrong here, then at worst you’ve wasted your time.
I’ve found LLMs to be much better at generating hypotheses for obscure bugs than Googling, and like you said the cost is very small
Generating pull request descriptions
Personally I do NOT want this. I don’t want my co-workers generating summaries of commits. Because the act of writing descriptions actually clarifies what I want to do.
Agree
I wonder if anyone has broken down all these use cases – and summarized the pitfalls
I think one category you’re missing under is stuff like editor/command line configuration. I’ve used ChatGPT to configure my bash prompt after unsuccessfully spending an hour trying to do the same. And it’s very easy to verify the results.
Using AI for auto-complete:
If you are working in an unfamiliar language
If you are not a good typist
I think this understates the case – I’m a very fast typist, pretty experienced with my IDE & vim macros, and Cursor’s tab-complete is still faster than my muscle memory.
It’s hard to give a short, comprehensive argument because the strength of any autocomplete depends on a lot of little things. But as one tiny example, in Julia if I have a function function f(job_id::Int, args...) and change it to job_ids, Cursor will reliably tab-complete the type annotation, the variable name, and all the places where it’s used to broadcast operations over a Vector instead of a single Int. Most importantly, this lets me not have to jump down a level of abstraction.
Jujutsu is and will remain a completely unusable project for me until it has support for pre-commit hooks, unfortunately. I enjoyed what I saw when I demoed it a bit to learn, but every repo I’ve worked with in my 12-year career has had at least one pre-commit hook in it (including personal projects which generally have 2-3!), and running them manually completely defeats the entire purpose of them especially in a professional setting.
shows the completely different universe the devs of jj live in, and it leads me to believe that this feature just won’t get much priority because obviously they never use it, and all projects like this are (rightfully, usually) targeting fixing problems for the people making it.
I have my hopes still, but until then…. back to git.
I’m curious why people like pre-commit hooks. I run my makefile far more frequently than I commit, and it does the basic testing and linting. And the heavyweight checks done on the server after pushing. So there doesn’t seem much point to me in adding friction to a commit when the code has already been checked, and will be checked and reviewed again.
To take an example from my jj-style patch-manipulation-heavy workflows:
I often I abort (“stash”) work and switch to a new speculative approach, which means committing it before it’s been checked. Indeed, at this stage, it’s not expected to pass checks, so triggering a pre-commit hook would be a waste of time.
I often significantly modify the graph structure (combine and split various commits, discard some, etc.), and then need to re-run checks on the whole patch stack/tree. Therefore, I rely primarily on post-hoc checks that run across all the commits in the stack/tree, rather than pre-commit checks.
I often want to re-run checks for each commit in my stack after rebasing on top of the latest main commit (in a trunk-based development workflow).
One should definitely distinguish between “checks that should run on each commit” and “pre-commit checks”.
Most pre-commit checks represent checks that should run on each commit, but it doesn’t mean that before committing is the optimal time to run the check. That depends on one’s specific local workflows and how they use commits.
In your case, you are already running your checks before committing, and presumably you don’t modify the commits afterwards, so an actual pre-commit hook seems unlikely to help your workflows.
I use pre-commit hooks extensively, to ensure that the code I’m pushing meets all kinds of project requirements. I use formatters, and linters, and check everything that can be checked. For one thing, it does away with the endless battles over meaningless nonsense, like where commas belong in SQL statements, or how many spaces to use for indenting. Another is it just reduces the load on the CI/CD systems, trading a small fraction of my time locally for expensive time we pay for by the cycle.
I’ll never go without them again.
ETA: but, based on sibling comments, it seems that the jj folks are On It, and yeah, it won’t be a “pre-commit” hook, the same way, but as long as it can be automagically run … ok, I’m in.
As others here have stated, I think the fundamental issue is that commits in jj are essentially automatic every time you save. There are a few consequences to this such as:
no predefined “significant event” to sensibly attach a pre-hook to (I’d make them pre-push hooks, maybe?)
inability to save any secrets to disk as those immediately get committed, meaning they’re in the history, meaning you have to rewrite history locally to remove them before pushing, or Lord help you
Not everyone uses an IDE, or the same IDE, or the same settings in the IDE. I think that computers should automatically do what they can, and using pre-commit hooks (or whatever the jj equivalent will be) is a way to guarantee invariants.
I totally agree with you all that stuff is super important to run before changes make it to the repo (or even a PR). The problem is that pre-commit hooks (with a lower case “p”) fundamentally don’t mesh with Jujutsu’s “everything is committed all the time model”. There’s no index or staging area or anything. As soon as you save a file, it’s already committed as far as Jujutsu is concerned. That means there’s no opportunity for a tool to insert itself and say “no, this commit shouldn’t go through”.
The good news is that anything you can check in a pre-commit hook, works just as well in a pre-push hook, and that will work once the issues 3digitdev linked are fixed. In the meantime, I’ve made myself a shell alias that runs pre-commit -a && jj git push and that works well enough for me shrug
Not everything runs that easily. Eg https://github.com/cycodehq/cycode-cli has a pre_commit command that is designed specifically to run the security scan before commit. It doesn’t work the same before push because at that point the index doesn’t contain the stuff you need to scan.
Hm, I’m not sure if cycode-cli would work with Jujutsu in that case. I’m sure if there’s enough demand someone would figure out a way to get it working. Even now, I’ve seen people MacGyvering their own pre-commit hooks by abusing their $EDITOR variable.. E,g export EDITOR="cycode-cli && vim" ><
In my years I’ve had exactly one repo that used makefiles.
The issue with makefiles has nothing to do with makefiles – the issue without pre-commit is deliberate action.
If I’m on a team of 12 developers, and we have a linter and a formatter which must be run so we can maintain code styles/correctness, I am NOT going to hope that all 12 developers remembered to run the linter before they made their PR. Why? because I FORGET to do it all the time too. Nothing irritates me more and wastes more money than pushing to a repo, making a PR, and having it fail over and over until I remember to lint/format. Why bother with all of that? Setup pre-commit hooks once, and then nobody ever has to think about it ever again. Commit, push, PR, etc, add new developers, add new things to run – it’s all just handled.
Can you solve this with a giant makefile that makes it so you have just one command to run before a PR? Yes. But that’s still a point of failure. A point of failure that could be avoided with existing tools that are genuinely trivial to setup (most linters/etc have pre-commit support and hooks pre-built for you!). Why let that point of failure stand?
Edit: Also, keep in mind the two arent mutually exclusive. You like your makefile, fine keep it. Run it however many times you want before committing. But if everyone must follow this step at least once before they commit…..why wait for them to make a mistake? Just do it for them.
Generally, I believe jj developers and users are in favor of the idea of defining and standardizing “checks” for each commit to a project, but the jj model doesn’t naturally lend itself to running them specifically at pre-commit time. The main problems with running hooks at literally pre-commit time:
Tends to pessimize and preclude the patch-manipulation workflows that jj is designed to facilitate.
In particular, an arbitrary commit is likely not to be in a working state, so the pre-commit hook would fail.
Running pre-commit hooks for each commit serially is often slower. There are ways to parallelize many formatting and linting tasks (for multiple simultaneous commits), but you can only do that once you’ve actually written all the commits that need to be validated.
Unclear how it would work with all the jj commands that do in-memory operations.
It’s not necessarily possible to run pre-commit hooks on those commits since they’ve never been materialized in a working copy.
For example, if you rebase commits from one place to another, those commits may no longer pass pre-commit checks. A lot of the technical and UX advantages of jj rely on “commits” always succeeding (including rebases and operations that may introduce merge conflicts); adding a way to make committing fail due to pre-commit hooks would sacrifice a lot of the benefits.
The current jj fix command can run on in-memory commits, I believe, but this also means that it’s limited in capability and doesn’t support arbitrary commands.
The hook situation for jj is not fully decided, but I believe the most popular proposal is:
Support “pre-push” hooks that run immediately before pushing.
These can run the checks on each commit about to be pushed, and either fix them or reject the push as appropriate.
Rather than support “pre-commit” hooks, delegate the checks to commands like the following, which run post-commit, even on commits that aren’t the ones currently checked-out. Then the user can run these commands manually if they want, and the pre-push hook can invoke them automatically as well.
(present) jj fix: primarily for single-file formatters and linters
(future) jj run: can run arbitrary commands by provisioning a full working copy
For the workflows you’ve specified, I think the above design would still work. You can standardize that your developers run certain checks and fixes before submitting code for review, but in a way that works in the jj model better, and might also have better throughput and latency considerations for critical workflows like making local commits.
We had at least one conversation at a Mercurial developer event discussing how to make hg configs (settings, extensions, hooks, etc) distributed by the server so clones would get reasonable, opinionated behavior by default. (It’s something companies care about.)
We could never figure out how to solve the trust/security issues. This feature is effectively a reverse RCE vulnerability. We thought we could allowlist certain settings. But the real value was in installing extensions. Without that, there was little interest. Plus if you care this much about controlling the client endpoint, you are likely a company already running software to manage client machines. So “just” hook into that.
I’m not entirely sure I follow what you’re asking. My guess here is that you’re unfamiliar with pre-commit, so I’ll answer from that perspective. Sorry if I’m assuming wrong
Pre-commit hooks aren’t some individualized separate thing. They are managed/installed inside the repo, and defined by a YAML file at the root of your repo. If you add a new one, it will get installed (once), then run the next time you run git commit by default.
As long as you have pre-commit installed on your system, you can wipe away the repo folder and re-clone all you want, and nothing has to change on your end.
If a new dev joins, all they have to do is clone, install pre-commit (single terminal command, run once), and then it just…works.
If a pre-commit hook changes (spoiler: They basically never do…) or is added/removed, you just modify the YAML, make a PR to the repo, and its merged. Everyone gets latest master and boom, they have the hook.
There is no ‘management’. No need to really document even (although you can). They should be silent and hidden and just run every commit, making it so nobody ever has to worry about them unless their code breaks the checks (linters, etc).
I take it the former runs code on pre-commit for any cloned git repo (trusted or not) when installed locslly, while the latter needs setup after initially cloning to work?
So it changes git behavior to run code pre-commit on every repo - but doesn’t directly execute code, rather parses a yaml file?
Yes I am talking about the former. To be clear though I don’t know exact internals of pre-commit, but I don’t THINK it modifies git behavior directly. Instead it just has a hook that runs prior to the actual git commit command executing, and it DOES execute code, but does so in a containerized way I believe. The YAML file is just a config file that acts sorta like a helm chart does; providing configuration values, what command to run – it’s different for each hook.
If you’re curious, you can see an example in one of my personal repos where I am defining a pre-commit hook to run the “Ruff” tool that does linting/formatting on my project.
Also, a note: Pre-commit hooks only execute against the files inside the commit too! So they tend to be quite performant. My ruff checks add maybe like….100ms to the runtime of the commit command. This can obviously vary – I’ve had some that take a few seconds, but in any case they never feel like they’re “blocking” you.
FWIW I’d consider 100ms to be quite bad given that jj commits tend to take < 10ms.
I agree with folks in general that in jj, almost no commits made by developers will pass pre-commit checks so a rethinking/different model is required. (For most of my professional life I’ve just used a combination of IDE/LSP feedback and CI for this, a long with a local script or two to run. I also locally commit and even push a lot of broken/WIP code.)
I’m also curious how the developer experience is maintained with pre-commit hooks. I was on the source control at Meta for many years and there was an intense, data-driven focus on performance and success metrics for blocking operations like commit. Do most authors of pre-commit hooks bring a similar level of care to their work?
There has been some interesting discussion in response to my question about pre-commit hooks.
The thing I’m (still) curious about is how to choose the point in the workflow where the checks happen. From the replies, I get the impression that pre-commit hooks are popular in situations where there isn’t a build step, so there isn’t a hook point before developers run the code. But do programmers in these situations run the tests before commit? If so, why not run the lints as part of the tests? If the lints are part of the standard tests, then you can run the same tests in CI, which suggests to me that the pre-commit hook configuration is redundant.
I’m asking these questions because pre-commit hooks imply something about the development workflow that I must be missing. How can you get to the point of pushing a branch that hasn’t gone through a build and test script? Why does it make sense to block a commit that no-one else will see, instead of making the developer’s local tests fail, and blocking the merge request because of the lint failure?
For auto-formatting in particular, I don’t trust it to produce a good result. I would not like a pre-commit hook that reformats code and commits a modified version that I had not reviewed first. (The likelihood of weird reformatting depends somewhat on the formatter: clang-format and rustfmt sometimes make a mess and need to be persuaded to produce an acceptable layout with a few subtle rephrasings.)
For auto-formatting you can choose to either have it silently fix things, or block the commit and error out. But yes, generally you can never rely on pre-commit hooks to prevent checks because there’s no way to enforce all developers use it. You need to rely on CI for that either way.
The benefit to running the checks before pushing to a pull request for instance, is that it reduces notifications, ensures reviewers don’t waste time, reduces the feedback / fix loop, etc. Generally just increases development velocity.
But all those benefits can also be achieved with pre-push. So I see the argument for running the checks prior to pushing to the remote. I fail to see the argument for running the checks prior to committing. Someone elsewhere in the thread pointed out a checker that apparently inherently needs to run pre-commit, so I guess there’s that? I don’t know the specifics of that checker, but seems like it’s poorly designed to me.
We always have one dev that introduces Husky to the repo and then everybody else has to add --no-verify to their commits because that’s easier than going into an argument with the kind of dev that would introduce Husky to the repo.
Most devs underestimate (as in have absolutely no idea as to the amount of rigor necessary here) what goes into creating pre-commit checkers that are actually usable and improve the project without making the developer experience miserable.
I mentioned this in another comment, but pre-commit (the hook) will never be feasible for the simple fact that “committing” isn’t something you explicitly do in Jujutsu. It’s quite a mindshift for sure, but it’s one of those things that once you get used to it, you wonder how you ever did things the old way.
The good news is that pre-push is totally possible, and as you noted, there is work underway to make the integration points there nicer. AIUI the issues you linked are getting closer to completion (just don’t expect them being fixed to mean that you’ll have the ability to run pre-commit hooks).
Find a rust capable dev, start hacking on it and engage with the Jujutsu crowd on IRC (#jujutsu at libera) or Discord. The developers are very approachable and can use additional PRs. That way everyone can eventually benefit from a completed hooks implementation.
With jj, it seems like you constantly have to shuffle around these logical commit IDs that have no meaning at all.
If I rebase a stacked series of branches with --update-refs, I just identify the fixup line, move it up after it’s corresponding commit (identified by its description, of course), and change it to f for fixup. Because jj doesn’t have interactive rebasing, it seems like you can’t do this?
The interactive rebase is like a GUI, it shows me things I don’t need all the time. Jujutsu seems like it insists on me shuffling around its logical IDs. But I’d rather just have my own good names, i.e. branch names.
You may’ve already, but if not I’d say give it a legit try. Maybe for a few weeks.
I’ve fully switched over at this point. And, FWIW (n=1, here), I don’t feel like I”m “constantly having to shuffle around” change IDs.
If I wanted, I could use bookmark (ie branch) names. But, really, for short-lived branches (which is nearly all I need), it just feels unnecessary in jj.
The workflow, once I broke free from the git garbage (poison, one might argue) in my muscle memory, feels streamlined and productive and simpler, while remaining at least as functional.
I gave jj a try and found it really nice to use on clean, well established projects. But on my personal, experimental projects it was quite a hassle. I don’t think it’s fault of jj, as it was not built with such use cases in mind: namely, a lot of uncommitted files, half written tests that will eventually be merged, code that is used in the repo, but must never be committed to the repo, and so on.
I do similar stuff by using a squash workflow: I have a “local” change that has a lot of random stuff I don’t really want to push, and run a lot of code from there. When I’m ready to push, I create a change before the local, and squash the things I actually want to add to the repo into that change, and push that as a PR.
The benefit of this over just untracked files is that I still get the revision history, so if I end up wanting to revert one of the changes to my local files it’s easy to do so.
But the split command needed to selectively take pieces of a change is IMO such a subpar experience compared to git add -p, at least out of the box, where it ended up launching the extremely clunky (for me) emacs-ediff.
Would be great if something incremental like git add -p eventually landed in jj.
I don’t use split, because I don’t love the UI. Instead, I use squash: I create an empty Change before the one I’m working on, and squash the files I want into the (new) parent. And jj squash --interactive lets me select specific sections of files if I need to.
Definitely agree it would be nice to have something with the ui of git add -p
If you have specific feedback about how the built-in jj split UI could be improved vs git add -p, you could leave it in Issues or Discussions for the scm-record repo.
Speaking personally, I think you already captured some of my issues in your README (discoverability, predictability). But more than that, it’s the fact that, being used to git add -p, I want something like that, instead of a worse version of magit (worse mainly because of the different key-bindings compared to my editor, I must say).
Specifically, it looks like the way to disable auto-track is to configure snapshot.auto-track to be none(). Then you would use jj file track to track files.
If you only want to ignore files in certain directories or files with predictable names, you can use a .gitignore file instead.
Out of curiosity, in your example you move a fixup to the proper place and change the type to f for fixup.
Are you aware of git rebase -i --autosquash which does it automatically or am I missing something?
Of course, with the interactive rebase you can do much more than that but I was wondering.
I know about it but it’s not a major burden so I haven’t looked into it. And people are telling me to just learn Jujutsu so maybe I shouldn’t be learning new Git features?
I’m not the person you asked, but I am an example of someone who knows about --autosquash yet still prefers the workflow of git rebase -i and manually typing the f. The main reason is that, for my project at work, I don’t like how the required “fixup! ” prefix makes the commit stand out in git log:
SOMEPROJ-1234: update dependency
fixup! SOMEPROJ-1234: fix failing test
SOMEPROJ-1234: add support for foo
Given this project’s requirement to mention a ticket in each commit, the prefix looks too out of place. So I prefer to write “(squash)” after the ticket name in the message and then search for lines with that phrase by eye in my git rebase -i editor:
Good question – I had to think a bit to remember. It’s because my workflow sometimes includes pushing the fixup commits as-is to the remote branch. I do that when I was pairing with another dev on my computer and I want to make it easy for them to see the history of the work we accomplished that day, which would be harder if I squashed before pushing. When working on large changesets like that, I only squash (and then force push) after my coworker and I think we won’t need more changes and want to consider the final breakdown of commits.
My problem with switching away from git is I need to know git well in order to fix others’ git mess ups. It’s part of the job to use this particular tool. Otherwise, I’d be using Pijul.
The most wonderful thing about jj is that you don’t have to switch away from git - it can be used either as a wrapper around git, or colocated with git (docs). So I wouldn’t think of it as “switching” from Git - it can be used alongside git to improve your own workflow.
I started using jj two weeks ago, it’s really cheap to try. You can continue using all the normal git commands and slowly try the jj bits if you find them helpful. And jj’s undo feature is worth it even if you decide to otherwise stick to the git model.
I wasn’t aware that this was a default action in VSCode! I’ve fiddled with manually setting up my splits and jumping around code before, so this’ll be a nice little time save
For those curious, the default binding is “Ctrl-K F12” (I had trouble finding it, the command name is “Open Definition to the Side” in the command palette). I might try to find a better binding, but I don’t think I can sacrifice my comma key for it!
But I feel there is a really nice general purpose language in there somewhere.
True. Julia is essentially a modern Dylan on top of LLVM. It’s a really great language, and it makes easy to write fast code for HPC while retaining all benefits from dynamic typing. I’ve written a lot of Julia, and it’s hard to return to Python or R, where the two-language problem is so evident.
Furthermore, libraries compose really well. It’s lots of small libraries, Julia all the way down. When you do mathematics, this is really satisfying because you can easily peek into the internals in case some results look odd. That contrasts with Python, where there are exceptionally good libraries, but it’s hard to compose them (big silos) and inspect (tons of low level C/C++/FORTRAN).
I do agree that macros should be used a lot less often, but when they’re good they’re amazing. We have a ~70 line macro that dramatically changes the evaluation strategy of code (adding desired outputs to a DAG and then calculating them all at once), and doing something similar without macros would be about 3,000 lines of code – more importantly, it would introduce a lot of duplication that would be difficult to keep in sync.
Unfortunately no. But the idea is straightforward: the macro takes a function that’s written to operate on the graph, and creates a few other versions of it, including one that operates on Julia expressions. This lets us write code like add_expr_to_graph(slope(input_expr, num_bars)), where input_expr and slope(input_expr, num_bars) will get added as nodes to the graph to be evaluated later (as well as any sub-expressions).
The major benefit is that if input_expr is used multiple times, we only calculate it once, and this makes a big difference for our use case. It also allows us to evaluate everything in parallel. Standard DAG stuff overall, but it means that our researchers can write code that looks like normal computational code, and we can execute it efficiently in production.
Hm interesting … though is that example really better than:
it("should have an age greater than or equal 18") do
expect (user.age > 18)
end
seems like a lot less typing and just as clear!
BTW in Oils we do
assert [user.age > 18]
to give a “lazy/unevaluated expression”, which can be destructured and understood by the test framework. This allows you to print the LHS and RHS, which I mentioned here
I disagree with a lot of what Stroustrup says, but I do like one comment from him (paraphrasing from memory):
There are two kinds of technologies, the ones everyone hates and the ones that no one uses.
Any useful technology is going to have to make a load of compromises to be useful. I consider it a good rule of thumb that, if you couldn’t sit down and write a few thousand words about all of the things you hate about a technology, you probably don’t understand it well enough to recommend it.
I’ve come to dislike that comment, and would put it in the same category as “everything is a tradeoff”. It’s a thought-terminating cliche that’s used to fend off criticism and avoid introspection.
There are such a thing as bad engineering decisions. Not everything was made with perfect information, implemented flawlessly to spec, with optimal allocation of resources. In fact many decisions are made on the bases of all kinds of biases, including personal preferences, limited knowledge, unclear goals, a bunch of gut feeling, etc.
And even a technology with good engineering decisions can turn a lot worse over time, e.g. when fundamental assumptions change.
I agree with you, but I’d like to defend the phrase “everything is a tradeoff” please!
To me, the natural corollary is that you should decide which set of tradeoffs are best for you.
All of the things you said are true but you can avoid a lot of pitfalls by being aware of what you are optimising for, what you are giving up, and why that might be appropriate in a given situation.
you should decide which set of tradeoffs are best for you.
(neo)Confucianism teaches that solutions don’t exist, rather just sets of trade offs. E.g. you can choose to have your current problem, or the problem which will occur if “solve” it.
What’s your background with Confucianism? I would say it’s fairly optimistic about human perfectibility. It’s maybe not utopian, but a sage king can rightly order the empire with the Mandate of Heaven at least. Or do you mean more contemporary Confucian inspired thought not classical (c300BCE) or neo (c1000CE)?
Neoconfucianism (in Chinese, study of “li”) synthesized it with Daoism and Buddhism (the 3 teachings). Wu wei is an important aspect of that li (logos, natural law) channeling the Zhuangzi’s pessimism. Yangmingzi riffs on this, believing in random action (experimenting?) but not consciously planning/acting towards plans. You’re to understand the trade offs etc. and map the different ways destiny may flow, but not act on them. Original Confucianism had a more limited focus (family first) which Zhang Zai extended, by treating everything as a bigger family, allowing Confucian approaches to apply to other domains.
One 4 character parable/idiom (which means “blessing in disguise”) has:
lose horse - poor
horse comes back with horse friends - richer
break leg - bad
don’t get drafted - good
background
Wing-tsit Chan and Lin Yutang made great translations and discussions on the history of ideas in Chinese thought. Though I may read Chen Chun, it’s really through their lenses as my Chinese isn’t yet up to snuff.
Okay. I wasn’t sure if by “neo” you meant like Daniel Bell’s New Confucianism.
I would say Wang Yangming is pretty optimistic about solutions. He’s against theoretical contemplation, but for the unity of knowledge and conduct, so ISTM the idea is you solve problems by acting intuitively. I’m sure if pressed he would acknowledge there are some tradeoffs, but I don’t see him as having a very pessimistic view or emphasizing the tradeoffs versus emphasizing perfecting your knowledge-conduct.
Thank you, I dug into this a bit deeper. I believe you are right and I have been misunderstanding some aspect of will./intention, which I struggle to articulate. Laozi and everything later building on it do seem to focus on (attempts to) control backfiring. I’m not sure if my pick-tradeoffs-lens is a productive innovation or missing the point. (How broadly/narrowly should we apply things?)
I’m also tired of hearing “everything is a trade-off” for that reason. I definitely like the phrase “thought-terminating cliche”.
It’s also not true. “Everything is a trade-off” implies that everything is already Pareto-optimal, which is crazy. Lots of things are worse than they could be without making any compromises. It even feels arrogant to say that anything is any kind of “optimal”.
There are such a thing as bad engineering decisions.
Of course, I think people mostly mean that there are bad approaches, but no perfect ones. (Or, more mathematically, the “better than” relation is often partial.)
Both sides phrases are inportant and meaningful, yes people can overuse them, and people can also fail to understand that “changing this behavior to be ‘sensible’” also is a trade off as changing behaviour can break existing stuff.
We can look at all sorts of things where the “trade off” being made is not obvious:
lack of safety in C/C++: yay performance! Downside: global performance cost due to myriad mitigations in software (aslr, hardened allocators, …) and hardware (pointer auth, mte, cheri, …) cost performance (power and speed) for everything
myriad weird bits of JS - mean lots of edge cases in the language, though in practice the more absurd cases aren’t hit and basic changes to style choices mitigate most of the remainder, so the cost of removing the behavior is unbounded and leaving it there has little practical cost
removing “print” statements from Python 3: made the language more “consistent” but imo was one of the largest contributors to just how long the 2->3 migration took, but was also entirely unnecessary from a practical point of view as a print statement is in practice distinguishable from a call
At the end of the day you might disagree with my framing/opinion of the trade offs being made, but they’re still trade offs, because trade offs are a fundamental part of every design decision you can ever make.
There’s nothing thought terminating about “everything is a trade off”, claiming that it is is itself thought terminating: it implies a belief that the decisions being made are not a trade off and that a decision is either right or wrong. That mentality leads to inflexibility, and arguably incorrect choices because it results in a design choices that don’t consider the trade offs being made.
“changing this behavior to be ‘sensible’” also is a trade off as changing behaviour can break existing stuff.
But what about the time when the decisions were actually made? What technical, calculated trade-offs did JS make when implementing its numerous inconsistencies, that are collectively seen as design failures?
claiming that it is is itself thought terminating: it implies a belief that the decisions being made are not a trade off and that a decision is either right or wrong
I definitely think some decisions can be right or wrong.
But what about the time when the decisions were actually made? What technical, calculated trade-offs did JS make when implementing its numerous inconsistencies, that are collectively seen as design failures?
The key tradeoff made in the development of JavaScript was to spend only a week and a half on it, sacrificing coherent design and conceptual integrity in exchange for a time-to-market advantage.
This isn’t really true. The initial prototype was made in 10 days but there were a lot of breaking changes up to Javascript 1.0 which was released a year later. Still a fairly short time frame for a new language but not exactly ten days.
At least there the weird quirks would’ve (hopefully) gotten fixed as bugs, because it has an actual language spec. OTOH, it might also have begotten generations of Scheme-haters and parenthophobes, or Microsoft’s Visual Basicscript would’ve taken off and we’d all be using that instead. Not sure what’s worse…
But what about the time when the decisions were actually made? What technical, calculated trade-offs did JS make when implementing its numerous inconsistencies, that are collectively seen as design failures?
Some behaviors are not the result of “decisions”, they’re just happenstance of someone writing code at the time without considering the trade offs because at the time they did not recognize that they were making a decision that had trade offs.
You’re saying there are numerous inconsistencies that were implemented, but that assumes that the inconsistencies were implemented, rather than an unexpected interaction of reasonable behaviors, without knowing the exact examples you’re thinking of I can’t speak to anything.
I definitely think some decisions can be right or wrong.
With the benefit of hindsight, or with a different view of the trade offs. Do you have examples of things where the decision was objectively wrong and not just a result of the weight of trade offs changing over time, such that the trade offs made in the past would not be made now?
A good example that happens all the time in the small is doing redundant work, mostly because you’re not aware it’s happening. Cloning data structures too often, verifying invariants multiple times, etc. I’ve seen a lot of cases where redundancies could be avoided with zero downside, if the author had paid more attention.
This makes me think how beautiful it is that crypto developers have managed to make NFT’s into not only something everyone hates but something nobody uses, at the same time
I consider it a good rule of thumb that, if you couldn’t sit down and write a few thousand words about all of the things you hate about a technology, you probably don’t understand it well enough to recommend it.
As a core element of good critical thinking, one should hypothetically be able to write such a criticism about anything they are a fan of. In fact, I encourage everyone to try this out as often as possible and push through the discomfort.
Notice I used the dreaded word “fan” there- which is the point of this comment: There should be a key distinction between someone who is a “fan” of a technology based on a critical evaluation of its pros and cons and someone who is a “fan” of a technology based on a relatively rosy assessment of its pros and a relatively blind assessment of its cons.
I think the OP blogger is really complaining about the latter. And, all other things being equal, I believe a developer using a technology chosen via critical assessment by a fan will always lead to superior work relative to a technology chosen via critical assessment by a non-fan. The fan, for example, will be motivated to know and understand things like the niche micro-optimizations to use that don’t make the code less readable (I’m thinking of, for example, the “for” construct in Elixir), and will likely use designs that align closer to the semantics of that particular language’s design than to languages in general.
One of the reasons I left Ruby and went to Elixir is that the “list of valid and impactful criticisms” I could come up with was simply shorter (and significantly so) with Elixir. (Perhaps I should blogpost a critical assessment of both.) And yes, I went from being a “fan” of Ruby to a “fan” of Elixir, but I can also rattle Elixir’s faults off the top of my head (slowish math, can’t compile to static binary, complex deployment, depends on BEAM VM/Erlang, still a bit “niche”, functional semantics more difficult to adopt for new developers, wonky language server in VSCode, no typing (although that’s about to change somewhat), not as easy to introspect language features as Ruby, etc.)
The other point I’d like to make is that even though “everything is a compromise,” there are certainly locally-optimal maxima with the more correct level of abstraction and the more correct design decisions. Otherwise we should all just code in Brainfuck because of its simple instruction set or in assembly because of its speed.
I think the distinction would be that I wouldn’t really call you a “fan” of Ruby or Elixir if you’re making these considered decisions, weighing the trade-offs, and considering whether they’re appropriate more-or-less dispassionately. You can certainly like languages, but I think if you call someone a “fan” of something, there’s an implication of a sort of blind loyalty. By analogy to sports fans, where a fan always supports their team, no matter who they’re up against, a fan of a particular technology is someone who always supports their particular technology, rails against those who criticize it, and hurls vitriol against “opponents” of their tool of choice.
Alright. Interesting distinction/clarification that gave me an idea.
So, in thinking of my Apple “fandom” that has been pretty consistent since I was 12 in 1984 when my family was simultaneously the last one in the neighborhood to get a family computer and the first ones to get a Mac (128k) and I was absolutely fucking enthralled in a way I cannot describe… which persisted through the near-death of 1997 and beyond into the iPod and iPhone era…
I think it has to do with “love”, frankly. If you “love” something, you see it through thick and thin, you stick around through difficulties, and you often (in particular if you contribute directly to the good quality of the thing, or the quality of its use, or its “evangelism”, or its community) literally believe the thing into a better version of itself over time.
The “likers”, in essence, value things based on the past and current objective value while the “lovers” (the fans) value things based on the perceived intrinsic and future value.
And the latter is quite irrational and thus indefensible and yet is the fundamental instrument of value creation.
But can also lead to failure. As we all know. The things the “likers” use are less risky.
One factor is that many software development projects are much closer to bike sheds than to skyscrapers. If someone is a fan of, say, geodesic domes (as in, believes that their current and/or future value is underrated), there is no reason not to try that in constructing a bike shed — it’s unlikely that whatever the builder is a fan of will completely fail to work for the intended purpose. The best outcome is that the technology will be proved viable or they will find ways to improve it.
If people set out to build a skyscraper from the start, then sure, they must carefully evaluate everything and reject unacceptable tradeoffs.
When people build a social network for students of a single college using a bikeshed-level technology stack because that’s what allowed them to build it quickly on zero budget and then start scaling it to millions users, it’s not the same problem as “started building a skyscraper from plywood”.
OTOH, no sane architect or engineer would expand a bikeshed into a skyscraper by continuing with the same materials and techniques. They’d probably trash the bikeshed and start pouring a foundation, for starters…
Exactly. When managers or VCs demand that a bikeshed is to be expanded into a skyscraper using the same materials, it’s not an engineering problem. Well, when engineers choose to do that, it definitely is. But a lot of the time, that’s not what happens.
Thank you! The “critical thinking” aspect is I think muddied by the article by setting up an ingroup/outgroup dichotomy with engineers on one side and fans on the other.
It’s normal to be a fan of something while also being fully aware of its trade-offs. Plus, sometimes an organization’s inertia needs the extra energetic push of advocacy (e.g. from a fanatic) to transition from a “good enough / nobody got fired for buying IBM” mentality into a better local optimum.
The mindset of “everything is a trade-off” is true but can also turn into a crutch and you end up avoiding thinking critically because oh well it’s just some trade-offs I can’t be bothered to fully understand.
“Engineers” and “fans” don’t look at the same trade-offs with different-colored glasses, they actually see different sets of trade-offs.
if you couldn’t sit down and write a few thousand words about all of the things you hate about a technology, you probably don’t understand it well enough to recommend it.
I would add: you should also be able to defend the options you didn’t choose. If someone can give a big list of reasons why Go is better than Rust, yet they still recommend Rust for this project, I’m a lot more likely to trust them.
There are two kinds of technologies, the ones everyone hates and the ones that no one uses.
This is true. I remember people hating Java, C++ and XML. Today I more often meet people hating Python, Rust and YAML. Sometimes it is the same people. Older technologies are well established and the hatred has run out. People have got used to it and take it for what it is. Hyped technologies raise false hopes and unrealistic expectations, which then lead to disappointment and hate.
I always struggle with the YAGNI concept because, in my experience, writing generic code is not more complex than writing over-specialised code and can often lead to simplifications for the specific use case. I then often end up being able to either copy code from one project to another, or factor it out into a library that both can depend on.
The one constant in software engineering is that your requirements will have changed by the time you’ve implemented them. Writing code that’s flexible in the presence of changing requirements is software engineering.
I like to distinguish “generic” from “general”, whereby “generic” means “independent of irrelevant details” and “general” means “handling many cases”. I believe in YAGN generality, but like you, I value genericity.
I tend to break YAGNI defensively. If, early in the implementation work, my gut tells me that the current “thin” specs are the result of an inability to formulate use cases in sufficient depth on the product management side, then there’s a very high chance that I am, in fact, GNI. In that case, I will totally write the more generic version, because there are two things that can happen one year down the line:
We actually NI, and most of it is already done.
We don’t actually NI and, worst-case scenario, I spend an hour on a refactoring that drops features, rather than several hours implementing and testing new features, in a module with high potential for regressions, on a tight deadline.
What if I don’t have time for that refactoring? Haha. If I can’t find an hour for a simple feature-dropping refactoring, I definitely won’t find the way more refactoring hours required to morph the code that implemented what we thought we needed a year ago into the code that implements what we actually understand we need today.
The undelying assumption in the “but what if you don’t have time to refactor it?” is that, as more requirements are formulated, code only needs to be “extended” to cover them, you just “add” things to it. That’s one of the worst sermons of the engineering management cargo cult.
Also, code written by successive generations of YAGNI people is some of the worst there is. It never evolves into a generic solution, it just snowballs into umpteen particular solutions held together by some inconsequential generics.
For me, YAGNI is not necessarily saying you shouldn’t write generic code. I’ll try to explain with an example from my team’s current project. We’ve been writing a program to manage hardware and the software running on it because none of the infrastructure management software we evaluated fit our needs well enough. So we have a daemon on our edge hardware and a service running in the cloud that coordinates that hardware and hands out configuration. That service connects to a database, which in my team’s case is Postgres. The engineer who wrote the initial draft of this system, however, added in a constant and some code to be able to work with any SQL database. However, it is highly unlikely that we would ever use a database besides Postgres since we’re already using it. This is the sort of thing YAGNI is about - you’re working on a team that has infrastructure in place, you know what you’re writing is internal and not meant to be consumed/built on by others. Why would you put in the work to support different types of infrastructure than you need to?
So yeah, requirements will change, but what you’re building on top of usually will not. So don’t try to support more than what your team uses unless you know with certainty that you need more.
The deciding factor for me is usually indirection. If the more specific approach lets me avoid a layer or two of indirection, I’ll usually pick that. If it doesn’t, I’ll go with the broader version.
At my previous job, I saw a feature added that wasn’t once used in the 11 years I was there. It stared out as a way to segment SS7 traffic to different backends, and a later addition of SIP had to support the same concept, yet it was never used. It was discussed from time to time to use it, but never came to fruition. And it would take more work, and coordination with another development team and ops, to fully remove the feature.
writing generic code is not more complex than writing over-specialised code
I think YAGNI is against adding feature or making code more complex in anticipation of future value. I have never seen anything against just making the code just as simple (or even simpler) by using a more general version.
I’ve heard this called “implement for today, design for tomorrow” and I think it is fundamentally at odds with YAGNI. The problem is it is (by definition?) more complex to read, and therefore maintain, tomorrow. There’s a quote about it:
[YAGNI] is making a bet. It is betting that it is better to do a simple thing today and pay a little more tomorrow to change it if needs it, than to do a more complicated thing today that may never be used anyway.
The quote is from the XP book, but I think in the context of this thread we can safely replace the word with YAGNI. Have you made the bet in the past and it hasn’t paid off, or have you just not tried it yet?
The bet is also that when you do need to make that change, you will have more information, so you will be able to do a better job then than you will be by anticipating those needs now.
With the understanding that I might just have been lucky, and / or working for a company seen as a more valuable customer … in a previous role I was a director of engineering for a (by Australian standards) large tech company. We had two teams run into difficulties, one with AWS, and another with GCP.
One team reached out to Amazon, and our AWS TAM wound up giving us some great advice around how we’d be better off using different AWS tech, and put us in touch with another AWS person who provided some deeper technical guidance. We wound up saving a tonne of money on our AWS bill with the new implementation, and delighting our internal customers.
The other team reached out to Google, and Google wouldn’t let a human speak to us until we could prove our advertising spend, and as it wasn’t high enough, we never got a straight answer to our question.
Working with Amazon feels like working with a company that actually values its customers; working with Google feels like working with a company that would prefer to deal with its customers solely through APIs and black-box policies.
I’m still confused. What do you have to do in AWS that you don’t have to do in GCP or is significantly easier? I have very little experience with GCP and a lot with AWS, so interested to learn more about how GCP compares.
“Trust me bro” is a difficult sell; but I am here to back up the sentiment. There are simply too many little things that by themselves are not deal breakers at all - but make the experience much more frustrating.
GCP feels like if engineers made a cloud platform for themselves with nearly infinite time and budget (and some idiots tried to bolt crud on the side, like oracle, vmware and various AI stuff).
AWS feels like if engineers were forced to write adjacently workable services without talking to each other and on an obscenely tight deadline - by people who didnt really know what they wanted, and then accidentally made something everyone started to use.
I’m going yo write my own blog post about this, I used to have a list of all my minor frustrations so I could flesh it out.
I’m still working on it, there’s some missing points and I want to bring up a major point about cloud being good for prototyping, but here: https://blog.dijit.sh/gcp-the-only-good-cloud/
The thing you are describing I should have only needed to learn once for each cloud vendor and put in a script: The developer-time should amortise to basically zero for either platform as long as the cloud vendor doesn’t change anything.
The developer-time should amortise to basically zero for either platform as long as the cloud vendor doesn’t change anything.
Yes and no.
Some constructs you don’t even have to worry about on GCP, so, it is best for fast deployments.
However, if you spend 500+ hours a year doing cloud work, then your point stands.
Indeed, if you are doing 5 hours a week of cloud infra work, your application is likely a full-time job or equivalent. I do believe you made the right choice with AWS.
No.
If you are spending 5 hours a week on infrastructure, then your application would be worth spending 40-hours on. Or is infra the only component of your application?
Can you do an actual comparison of the work? I’d be curious. Setting up a web service from a Docker image is a few minutes. Env vars are no extra work. Secrets are trivial. A cert would take maybe 10 minutes?
Altogether, someone new should be able to do this in maybe 30 minutes to an hour. Someone who’s done it before could likely get it done in 10 minutes or less, some of that being downtime while you wait for provisioning.
You’ve posted a lot in this thread to say, roughly ‘they’re different, AWS is better, but I can’t tell you why, trust me’. This doesn’t add much to the discussion. It would help readers (especially people like me, who haven’t done this on either platform) if yo u could slow down a bit and explain the steps in AWS and the steps in GCP and why there’s more friction with the latter.
No, please don’t trust me.
Here’s an exercise for yourself.
Assuming you control a domain domain1.com.
Deploy a dockerized web service (pull one from the Internet if you don’t know how to write one).
Deploy it on gcp1.domain1.com,
Deploy another on aws1.domain1.com.
Compare how many steps it takes and how long it takes.
Here’s how I deploy it on GCP. I never open-sourced my AWS setup but I am happy to see a faster one.
As I said, I have not used AWS or GCP so I have no idea how many steps it requires on either platform. I can’t judge whether your command is best practice for CGP and have no idea what the equivalent is on AWS (I did once do this on Azure and the final deploy step looked similar, from what I recall, but there was some setup to create the Azure Container thingy instance, but I can’t tell from your example if this is not needed on GCP of if you’ve simply done it already). If I tried to do it on AWS, I’d have no idea if I were doing the most efficient thing or some stupid thing that most novices would manage to avoid.
You have, apparently, done it on both. You are in a position to tell me what steps are needed on AWS but not on GCP. Or what abstractions are missing on AWS but are better on GCP. Someone from AWS might even read your message and improve AWS. But at the moment I just see that a thing on GCP is one visible step plus at least zero setup steps, whereas on AWS it is at least one step.
You’ve posted ten times so far in this story to say that GCP is better, but not articulated how or why it’s better. As someone with no experience with either, nothing you have said gives me any information to make that comparison. At least one of the following is true:
GCP has more efficient flows than AWS.
You are more familiar with GCP than AWS and so you are comparing an efficient flow with GCP that you’ve learned to use to an inefficient flow with AWS.
You are paid by GCP to market for them (not very likely).
It sounds as if you believe the first is true and the second two are not but (again) as someone reading your posts who understands the problem but is not familiar with either of the alternatives in any useful level of detail, I cannot judge for myself from your posts.
If you wrote ‘GCP has this set of flows / abstractions that have no equivalent on AWS’ then someone familiar with AWS could say ‘actually, it has this, which is as good’ or ‘you are correct, this is missing and it’s annoying’. But when you write:
I have done it for myself and I’m speaking from experience.
That doesn’t help anyone reading the thread understand what is better or why.
I find AWS to be better organized and documented but much larger that any Google product I’ve seen. There is more to learn because it’s a large, modular system. And it’s trivial to do an easy things the hard way.
I don’t have much direct experience with Google’s hosting but if any of their service products are similar, the only advantage is they do less, which means you can skimp on organization and documentation without too many problems.
In terms of big players I would recommend GCP still, but only because I mostly work with Kubernetes and it’s best there. From smaller players Fly.io is actually works well for me.
I find the opposite of this is far more often the problem. Just let the data be. Having two arguments of the same type is not a good enough reason to complicate your data representation. The solution to that is keyword arguments, or passing a record with named fields, or just naming the function in a way that suggests the order.
I’ve also gone back to mostly using primitive types, after having spent a few years going really heavy on specific types. The major advantages are staying close to the data, having an easier time with serialization/deserialization, and having an easier time with big refactors (because you don’t have a large type hierarchy that may need to change.) I usually stick to a handful of Structs to represent cleaned up versions of input/output data, with no nested Structs, and no special Structs to represent intermediate values.
I like Zach Tellman’s generalization of this, which is (paraphrasing) that your cores should contain code that makes strong assumptions. Those assumptions should buy you things like simplicity and performance. Your shell should generally try to conform data to the assumptions of a relevant core.
Synchronous code is a reasonable core, and so is code that runs on the GPU for a machine learning pipeline – these are both restrictive sets of assumptions that buy you something.
i wonder if i’ll live to see the day where we can talk about a language without putting a different language down
The YouTube channel here seems to be a person who needs to be dramatic for view reasons. I think the actual content, and the position of the Ghostty author here on this topic, is pretty mild.
An actual bit from the video:
Guest: “…I don’t know, I’m questioning everything about Go’s place in the stack because […reasonable remarks about design tradeoffs…]”
Host: “I love that you not only did you just wreck Go […]”
Aside… In the new year I’ve started reflexively marking videos from channels I follow as “not interested” when the title is clickbait, versus a succinct synopsis of what the video is about. I feel like clickbait and sensationalism on YouTube is out of control, even among my somewhat curated list of subscribed channels.
This is why I can’t stand almost any developer content on YouTube and similar platforms. They’re way too surface-level, weirdly obsessed with the inane horse race of finding the “best” developer tooling, and clickbait-y to a laughable degree. I have >20 years of experience, I’m not interested in watching someone blather on about why Go sucks when you could spend that time on talking about the actual craft of building things.
But, no, instead we get an avalanche of beginner-level content that lacks any sort of seriousness.
This is why I really like the “Developer Voices” channel. Great host, calm and knowledgeable. Interesting guests and topics. Check it out if you don’t know it yet.
Very nice channel indeed. Found it accidentally via this interview about Smalltalk and enjoyed it very much.
Do you have other channel recommendations?
I found Software Unscripted to pretty good too. Not quite as calm as Developer Voices, but the energy is positive.
Thanks! Didn’t know Richard Feldman hosted a podcast, he’s a good communicator.
Signals and Threads is another great podcast, albeit doesn’t seem to have a scheduled release
Thanks for the suggestion. I will check it out!
I’m in a similar boat. Have you found any decent channels that aren’t noob splooge? Sometimes I’ll watch Asahi Lina, but I haven’t found anything else that’s about getting stuff done. Also, non-OS topics would be nice additions as well.
As someone else said, Developer Voices is excellent, and the on the opposite end of the spectrum from OP.
The Software Unscripted podcast publishes on YouTube too, and I enjoy it a fair bit at least in the audio only format.
Two more:
Book Overflow, which focuses on reading a software book about once every two weeks and talking about it in depth.
7 (7!) Years ago LTT made a video about why their thumbnails are so… off putting and it essentially boiled down to “don’t hate the player; hate the game”. YouTube rewards that kind of content. There’s a reason why nearly every popular video these days is some variant of “I spent 50 HOURS writing C++” with the thumbnail having a guy throwing up. If your livelihood depends on YouTube, you’re leaving money on the table by not doing that.
It’s not just “Youtube rewards it”, it’s that viewers support it. It’s a tiny, vocal minority of people who reject those thumbnails. The vaaaaast majority of viewers see them and click.
I don’t think you can make a definitive statement either way because YouTube has its thumb on the scales. Their algorithm boosts videos on factors other than just viewer click through or retention rates (this has also been a source of many superstitions held by YouTubers in the past) and the way the thumbnail, title and content metas have evolved make me skeptical that viewers as a whole support it.
What is the alternative? That they look at the image and go “does this person make a dumb face” ? Or like “there’s lots of colors” ? I think the simplest explanation is that people click on the videos a lot.
…or it’s just that both negative and positive are tiny slices compared to neutrals but the negative is slightly smaller than the positive.
(I use thumbnails and titles to evaluate whether to block a channel for being too clickbait-y or I’d use DeArrow to get rid of the annoyance on the “necessary evil”-level ones.)
then you have chosen poorly.
No, I think it’s okay for people to make great content for a living.
I am quite happy to differ in opinion to someone who says ‘great content’ unironically. Anyway your response is obviously a straw man, I’m not telling Chopin to stop composing for a living.
Your personal distaste for modern culture does not make it any higher or lower than Chopin, nor does it invalidate the fact that the people who make it have every right to make a living off of it.
They literally don’t have a right to make a living from Youtube, this is exactly the problem. Youtube can pull the plug and demonetise them at any second and on the slightest whim, and they have absolutely no recourse. This is why relying on it to make a living is a poor choice. You couldn’t be more diametrically wrong if you tried. You have also once again made a straw man with the nonsense you invented about what I think about modern culture.
How’s that any different from the state of the media industry at any point in history? People have lost their careers for any reason in the past. Even if you consider tech or any other field, you’re always building a career on top of something else. YouTube has done more to let anyone make a living off content than any other stage in history, saying you’re choosing poorly to make videos for YouTube is stupid.
You’re the one who brought it up:
Isn’t this kind of a rigid take? Why is depending on youtube a poor choice? For a lot of people, I would assume it’s that or working at a fast-food restaurant.
Whether that’s a good long-term strategy, or a benefit to humanity is a different discussion, but it doesn’t have to necessarily be a poor choice.
Not really?
I mean sure if you’ve got like 1000 views a video then maybe your livelihood depending on YouTube is a poor choice.
There’s other factors that come into this, but if you’ve got millions of views and you’ve got sponsors you do ad-reads for money/affiliate links then maybe you’ll be making enough to actually “choose” YouTube as your main source of income without it being a poor choice (and it takes a lot of effort to reach that point in the first place).
We’ve been seeing this more and more. You can, and people definitely do, make careers out of YouTube and “playing the game” is essential to that.
Heh - I had guessed who the host would be based on your comment before I even checked. He’s very much a Content Creator (with all the pandering and engagement-hacking that implies). Best avoided.
Your “ghostty author” literally built a multibillion dollar company writing Go for over a decade, so Im pretty sure his opinion is not a random internet hot take.
Yup. He was generally complimentary of Go in the interview. He just doesn’t want to use it or look at it at this point in his life. Since the Lobsters community has quite an anomalous Go skew, I’m not surprised that this lack of positivity about Go would be automatically unpopular here.
And of course the title was click-baity – but can we expect from an ad-revenue-driven talk show?
My experience is that Lobste.rs is way more Rust leaning than Go leaning, if anything.
We have more time to comment on Lobsters because our tools are better ;)
Waiting for compile to finish, eh?
Hahahahaha. Good riposte!
I was able to get the incremental re-builds down to 3-5 seconds on a 20kloc project with a fat stack of dependencies which has been good enough given most of that is link time for a native binary and a wasm payload.
cargo checkviarust-analyzerin my editor is faster and does enough for my interactive workflow most of the time.Yeah, Haskell is so superior to Rust that’s not even fun at this point.
It’s funny you say that because recently it seems we get a huge debate on any Go-related post :D
First thought was “I bet it’s The Primeagen.” Was not disappointed when I clicked to find out.
Don’t be a drama queen ;-) You can, all you want. That’s what most people do.
The host actually really likes Go, and so does the guest. He built an entire company where Go was the primary (only?) language used. It is only natural to ask him why he picked Zig over Go for creating Ghostty, and it is only natural that the answer will contrast the two.
i can’t upvote this enough
I’ve found LLMs helpful in a few specific cases:
There’s a lot of stuff that would be easy for the right person which I don’t know well, such as modifying my bash prompt, writing a simple web scraper (all my experience is with Data Science/Data Engineering), or working with the latest AWS library – they’ll often give me code that sets up a basic example and almost works, that I can quickly turn into something that does work.
LLM-based tab completion is quite nice, if you’re ok with a lot of false positives. I’ve pretty much stopped using vim macros and multiple cursors, because doing something once and letting the LLM suggest the remaining changes is so much faster. And the false positives are offset by the number of typos that it’s caught for me.
The major downside is code quality – any significant chunk of code ends up being extremely verbose and often redundant, to the point where I’ll essentially rewrite all the code once I understand it. This makes code generation useless for anything I’m halfway decent at.
Yeah, to me an LLM is like autotune for a different domain. It’ll get something passable out the door but won’t impress somebody who can actually sing.
Google has their own internal AI codey help thingy. I use the auto-complete a lot, as it’s really quite good at figuring out what I am trying to do and so I can get from 5 typed characters to 25 auto-completed characters quite fast. It’s really removed a lot of boilerplate, especially in Java which is boilerplate city.
Apparently, I can do all sorts of cool things like ask the LLM to generate me test cases and such. I just don’t trust it at the end of the day. The autocomplete-y has burned me a few times on hallucinations that I then had to manually find and patch up. Would I have introduced more or less bugs this way? I don’t know. I do feel like the hallucinations generate more tricky to find bugs, but I have no numbers. Would it generate good unit tests? Maybe. Maybe I am just old and view my job as a creative writer rather than a proof-reader for an LLM. Maybe the current generation of CS undergrads will have this new focus. Maybe not.
It’s just a tool, and you either find value in it or you don’t. Copilot and stuff was not good enough when it came out, so I am not surprised people that were burned by it aren’t interested in using it again.
I know engineers who don’t know how nor care to change their editor colors and font. They appear to be doing their job just fine. Let people use what they want and they need, and don’t bother judging them for it.
Yeah I was just going to write something along these lines – I think we need some terminology to describe when AI is useful, along the lines of when StackOverflow is useful. There are some cases where copying from StackOverflow is bad engineering, and some where it can be good.
Personally I find StackOverflow to be extremely useful … I always write tests for any snippets I get from there, even 1 line solutions, and that helps me understand it. Overall it can still really accelerate software creation
I think there’s an asymmetry of the “cost to be wrong”
Writing one-off scripts to aid software development:
Using AI for auto-complete:
(Personally I am a fast typer in the languages I know, and I try to stick to that happy / flow state path. So I don’t use LSPs at all, but I occasionally use AI. I might use LSPs if I were learning an unfamiliar stack/language, though I actually find over-reliance on LSPs to be a “smell”. Similarly I might regard over-reliance on AIs to be a smell.)
Other reasons:
I wonder if anyone has broken down all these use cases – and summarized the pitfalls. I’d be interested in reading experiences
There might be an analogy to “Type I” and “Type II” errors in statistics (which are the most horrible namess, as recently pointed out here)
But I think there should be some words/memes for this distinction
Unfortunately the discourse seems to be very polarized now – “AGI is going to make programmers obsolete” vs. “AI is snake Oil”
But as usual the answer is somewhere in between …. At some point it will calm down, and we will still have these tools, and we’ll learn when to use them, and when not to use them
Or judging by the industry for the last ~10 years, we can write pretty bad code for a long time, and people still have to use it, which is scary …
This is a good breakdown!
Are you assuming that the programmer isn’t reading/testing the code output by an LLM? Plenty of the code I get from LLMs is buggy, but that doesn’t mean it isn’t useful on net – so far, none of those bugs have made it into production.
I’ve found LLMs to be much better at generating hypotheses for obscure bugs than Googling, and like you said the cost is very small
Agree
I think one category you’re missing under is stuff like editor/command line configuration. I’ve used ChatGPT to configure my bash prompt after unsuccessfully spending an hour trying to do the same. And it’s very easy to verify the results.
I think this understates the case – I’m a very fast typist, pretty experienced with my IDE & vim macros, and Cursor’s tab-complete is still faster than my muscle memory.
It’s hard to give a short, comprehensive argument because the strength of any autocomplete depends on a lot of little things. But as one tiny example, in Julia if I have a function
function f(job_id::Int, args...)and change it tojob_ids, Cursor will reliably tab-complete the type annotation, the variable name, and all the places where it’s used to broadcast operations over a Vector instead of a single Int. Most importantly, this lets me not have to jump down a level of abstraction.Jujutsu is and will remain a completely unusable project for me until it has support for pre-commit hooks, unfortunately. I enjoyed what I saw when I demoed it a bit to learn, but every repo I’ve worked with in my 12-year career has had at least one pre-commit hook in it (including personal projects which generally have 2-3!), and running them manually completely defeats the entire purpose of them especially in a professional setting.
I keep checking in on the issue and the tracker for it, but still no luck.
I think given that the original issue starts with
shows the completely different universe the devs of
jjlive in, and it leads me to believe that this feature just won’t get much priority because obviously they never use it, and all projects like this are (rightfully, usually) targeting fixing problems for the people making it.I have my hopes still, but until then…. back to
git.I’m curious why people like pre-commit hooks. I run my makefile far more frequently than I commit, and it does the basic testing and linting. And the heavyweight checks done on the server after pushing. So there doesn’t seem much point to me in adding friction to a commit when the code has already been checked, and will be checked and reviewed again.
To take an example from my jj-style patch-manipulation-heavy workflows:
maincommit (in a trunk-based development workflow).One should definitely distinguish between “checks that should run on each commit” and “pre-commit checks”.
I use pre-commit hooks extensively, to ensure that the code I’m pushing meets all kinds of project requirements. I use formatters, and linters, and check everything that can be checked. For one thing, it does away with the endless battles over meaningless nonsense, like where commas belong in SQL statements, or how many spaces to use for indenting. Another is it just reduces the load on the CI/CD systems, trading a small fraction of my time locally for expensive time we pay for by the cycle.
I’ll never go without them again.
ETA: but, based on sibling comments, it seems that the jj folks are On It, and yeah, it won’t be a “pre-commit” hook, the same way, but as long as it can be automagically run … ok, I’m in.
As others here have stated, I think the fundamental issue is that commits in jj are essentially automatic every time you save. There are a few consequences to this such as:
I care about these things too, but they’re tested in CI, once I am ready to integrate them, rather than locally.
Horses for courses. I’d rather use my compute than The Cloud but I am notoriously a cranky greybeard.
For sure, I’m just saying it is possible to care about those things and not use hooks. You should do what’s best for you, though.
Aren’t those all already available in the ide? I get my red squiggles as I write, instead of waiting for either the precommit or the ci.
Not everyone uses an IDE, or the same IDE, or the same settings in the IDE. I think that computers should automatically do what they can, and using pre-commit hooks (or whatever the jj equivalent will be) is a way to guarantee invariants.
Pre-commit hooks are really easy to enforce across a large team, while any sort of IDE settings are not. Some I’ve used before:
You can do all of these in other ways, but pre-commit makes it easy to do exactly the same thing across the entire team
I totally agree with you all that stuff is super important to run before changes make it to the repo (or even a PR). The problem is that pre-commit hooks (with a lower case “p”) fundamentally don’t mesh with Jujutsu’s “everything is committed all the time model”. There’s no index or staging area or anything. As soon as you save a file, it’s already committed as far as Jujutsu is concerned. That means there’s no opportunity for a tool to insert itself and say “no, this commit shouldn’t go through”.
The good news is that anything you can check in a pre-commit hook, works just as well in a pre-push hook, and that will work once the issues 3digitdev linked are fixed. In the meantime, I’ve made myself a shell alias that runs
pre-commit -a && jj git pushand that works well enough for me shrugNot everything runs that easily. Eg https://github.com/cycodehq/cycode-cli has a
pre_commitcommand that is designed specifically to run the security scan before commit. It doesn’t work the same before push because at that point the index doesn’t contain the stuff you need to scan.Hm, I’m not sure if
cycode-cliwould work with Jujutsu in that case. I’m sure if there’s enough demand someone would figure out a way to get it working. Even now, I’ve seen people MacGyvering their own pre-commit hooks by abusing their $EDITOR variable.. E,gexport EDITOR="cycode-cli && vim"><Many people never use an editor to write a commit message, they use the
-mflag directly on the command line 🙂But yeah, at that point we could mandate that we have to use a shell script wrapper for
git committhat does any required checks.Jujutsu has at least a large file settings: https://martinvonz.github.io/jj/latest/config/#maximum-size-for-new-files
In my years I’ve had exactly one repo that used makefiles.
The issue with makefiles has nothing to do with makefiles – the issue without pre-commit is deliberate action.
If I’m on a team of 12 developers, and we have a linter and a formatter which must be run so we can maintain code styles/correctness, I am NOT going to hope that all 12 developers remembered to run the linter before they made their PR. Why? because I FORGET to do it all the time too. Nothing irritates me more and wastes more money than pushing to a repo, making a PR, and having it fail over and over until I remember to lint/format. Why bother with all of that? Setup pre-commit hooks once, and then nobody ever has to think about it ever again. Commit, push, PR, etc, add new developers, add new things to run – it’s all just handled.
Can you solve this with a giant makefile that makes it so you have just one command to run before a PR? Yes. But that’s still a point of failure. A point of failure that could be avoided with existing tools that are genuinely trivial to setup (most linters/etc have pre-commit support and hooks pre-built for you!). Why let that point of failure stand?
Edit: Also, keep in mind the two arent mutually exclusive. You like your makefile, fine keep it. Run it however many times you want before committing. But if everyone must follow this step at least once before they commit…..why wait for them to make a mistake? Just do it for them.
Generally, I believe jj developers and users are in favor of the idea of defining and standardizing “checks” for each commit to a project, but the jj model doesn’t naturally lend itself to running them specifically at pre-commit time. The main problems with running hooks at literally pre-commit time:
jj fixcommand can run on in-memory commits, I believe, but this also means that it’s limited in capability and doesn’t support arbitrary commands.The hook situation for jj is not fully decided, but I believe the most popular proposal is:
jj fix: primarily for single-file formatters and lintersjj run: can run arbitrary commands by provisioning a full working copyFor the workflows you’ve specified, I think the above design would still work. You can standardize that your developers run certain checks and fixes before submitting code for review, but in a way that works in the jj model better, and might also have better throughput and latency considerations for critical workflows like making local commits.
This does sound promising.
@arxanas is speaking from experience and you can see this in action today if you want: it’s built into git-branchless (which he built) as
git test: https://github.com/arxanas/git-branchless/wiki/Command:-git-testgit-branchless is a lovely, git-compatible ui like jj, inspired by mercurial but with lots of cool things done even better (like
git test)Once on every clone, across the entire team if a hook changes or is added?
How do you manage hooks? Just document them in the top level readme?
We had at least one conversation at a Mercurial developer event discussing how to make hg configs (settings, extensions, hooks, etc) distributed by the server so clones would get reasonable, opinionated behavior by default. (It’s something companies care about.)
We could never figure out how to solve the trust/security issues. This feature is effectively a reverse RCE vulnerability. We thought we could allowlist certain settings. But the real value was in installing extensions. Without that, there was little interest. Plus if you care this much about controlling the client endpoint, you are likely a company already running software to manage client machines. So “just” hook into that.
I’m not entirely sure I follow what you’re asking. My guess here is that you’re unfamiliar with pre-commit, so I’ll answer from that perspective. Sorry if I’m assuming wrong
Pre-commit hooks aren’t some individualized separate thing. They are managed/installed inside the repo, and defined by a YAML file at the root of your repo. If you add a new one, it will get installed (once), then run the next time you run
git commitby default.As long as you have pre-commit installed on your system, you can wipe away the repo folder and re-clone all you want, and nothing has to change on your end.
If a new dev joins, all they have to do is clone, install pre-commit (single terminal command, run once), and then it just…works.
If a pre-commit hook changes (spoiler: They basically never do…) or is added/removed, you just modify the YAML, make a PR to the repo, and its merged. Everyone gets latest
masterand boom, they have the hook.There is no ‘management’. No need to really document even (although you can). They should be silent and hidden and just run every commit, making it so nobody ever has to worry about them unless their code breaks the checks (linters, etc).
Thank you.
You’re talking about?:
https://pre-commit.com/
Not?:
https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
I take it the former runs code on pre-commit for any cloned git repo (trusted or not) when installed locslly, while the latter needs setup after initially cloning to work?
So it changes git behavior to run code pre-commit on every repo - but doesn’t directly execute code, rather parses a yaml file?
Yes I am talking about the former. To be clear though I don’t know exact internals of pre-commit, but I don’t THINK it modifies git behavior directly. Instead it just has a hook that runs prior to the actual
git commitcommand executing, and it DOES execute code, but does so in a containerized way I believe. The YAML file is just a config file that acts sorta like a helm chart does; providing configuration values, what command to run – it’s different for each hook.If you’re curious, you can see an example in one of my personal repos where I am defining a pre-commit hook to run the “Ruff” tool that does linting/formatting on my project.
Also, a note: Pre-commit hooks only execute against the files inside the commit too! So they tend to be quite performant. My ruff checks add maybe like….100ms to the runtime of the commit command. This can obviously vary – I’ve had some that take a few seconds, but in any case they never feel like they’re “blocking” you.
FWIW I’d consider 100ms to be quite bad given that jj commits tend to take < 10ms.
I agree with folks in general that in jj, almost no commits made by developers will pass pre-commit checks so a rethinking/different model is required. (For most of my professional life I’ve just used a combination of IDE/LSP feedback and CI for this, a long with a local script or two to run. I also locally commit and even push a lot of broken/WIP code.)
I’m also curious how the developer experience is maintained with pre-commit hooks. I was on the source control at Meta for many years and there was an intense, data-driven focus on performance and success metrics for blocking operations like commit. Do most authors of pre-commit hooks bring a similar level of care to their work?
There has been some interesting discussion in response to my question about pre-commit hooks.
The thing I’m (still) curious about is how to choose the point in the workflow where the checks happen. From the replies, I get the impression that pre-commit hooks are popular in situations where there isn’t a build step, so there isn’t a hook point before developers run the code. But do programmers in these situations run the tests before commit? If so, why not run the lints as part of the tests? If the lints are part of the standard tests, then you can run the same tests in CI, which suggests to me that the pre-commit hook configuration is redundant.
I’m asking these questions because pre-commit hooks imply something about the development workflow that I must be missing. How can you get to the point of pushing a branch that hasn’t gone through a build and test script? Why does it make sense to block a commit that no-one else will see, instead of making the developer’s local tests fail, and blocking the merge request because of the lint failure?
For auto-formatting in particular, I don’t trust it to produce a good result. I would not like a pre-commit hook that reformats code and commits a modified version that I had not reviewed first. (The likelihood of weird reformatting depends somewhat on the formatter: clang-format and rustfmt sometimes make a mess and need to be persuaded to produce an acceptable layout with a few subtle rephrasings.)
For auto-formatting you can choose to either have it silently fix things, or block the commit and error out. But yes, generally you can never rely on pre-commit hooks to prevent checks because there’s no way to enforce all developers use it. You need to rely on CI for that either way.
The benefit to running the checks before pushing to a pull request for instance, is that it reduces notifications, ensures reviewers don’t waste time, reduces the feedback / fix loop, etc. Generally just increases development velocity.
But all those benefits can also be achieved with pre-push. So I see the argument for running the checks prior to pushing to the remote. I fail to see the argument for running the checks prior to committing. Someone elsewhere in the thread pointed out a checker that apparently inherently needs to run pre-commit, so I guess there’s that? I don’t know the specifics of that checker, but seems like it’s poorly designed to me.
We always have one dev that introduces Husky to the repo and then everybody else has to add
--no-verifyto their commits because that’s easier than going into an argument with the kind of dev that would introduce Husky to the repo.Most devs underestimate (as in have absolutely no idea as to the amount of rigor necessary here) what goes into creating pre-commit checkers that are actually usable and improve the project without making the developer experience miserable.
I mentioned this in another comment, but
pre-commit(the hook) will never be feasible for the simple fact that “committing” isn’t something you explicitly do in Jujutsu. It’s quite a mindshift for sure, but it’s one of those things that once you get used to it, you wonder how you ever did things the old way.The good news is that
pre-pushis totally possible, and as you noted, there is work underway to make the integration points there nicer. AIUI the issues you linked are getting closer to completion (just don’t expect them being fixed to mean that you’ll have the ability to runpre-commithooks).Find a rust capable dev, start hacking on it and engage with the Jujutsu crowd on IRC (#jujutsu at libera) or Discord. The developers are very approachable and can use additional PRs. That way everyone can eventually benefit from a completed hooks implementation.
With jj, it seems like you constantly have to shuffle around these logical commit IDs that have no meaning at all.
If I rebase a stacked series of branches with
--update-refs, I just identify the fixup line, move it up after it’s corresponding commit (identified by its description, of course), and change it tofforfixup. Because jj doesn’t have interactive rebasing, it seems like you can’t do this?The interactive rebase is like a GUI, it shows me things I don’t need all the time. Jujutsu seems like it insists on me shuffling around its logical IDs. But I’d rather just have my own good names, i.e. branch names.
You may’ve already, but if not I’d say give it a legit try. Maybe for a few weeks.
I’ve fully switched over at this point. And, FWIW (n=1, here), I don’t feel like I”m “constantly having to shuffle around” change IDs.
If I wanted, I could use bookmark (ie branch) names. But, really, for short-lived branches (which is nearly all I need), it just feels unnecessary in
jj.The workflow, once I broke free from the git garbage (poison, one might argue) in my muscle memory, feels streamlined and productive and simpler, while remaining at least as functional.
I gave jj a try and found it really nice to use on clean, well established projects. But on my personal, experimental projects it was quite a hassle. I don’t think it’s fault of jj, as it was not built with such use cases in mind: namely, a lot of uncommitted files, half written tests that will eventually be merged, code that is used in the repo, but must never be committed to the repo, and so on.
I do similar stuff by using a squash workflow: I have a “local” change that has a lot of random stuff I don’t really want to push, and run a lot of code from there. When I’m ready to push, I create a change before the local, and squash the things I actually want to add to the repo into that change, and push that as a PR.
The benefit of this over just untracked files is that I still get the revision history, so if I end up wanting to revert one of the changes to my local files it’s easy to do so.
Sounds like a nice workflow.
But the split command needed to selectively take pieces of a change is IMO such a subpar experience compared to
git add -p, at least out of the box, where it ended up launching the extremely clunky (for me) emacs-ediff.Would be great if something incremental like
git add -peventually landed in jj.I don’t use split, because I don’t love the UI. Instead, I use squash: I create an empty Change before the one I’m working on, and squash the files I want into the (new) parent. And
jj squash --interactivelets me select specific sections of files if I need to.Definitely agree it would be nice to have something with the ui of
git add -pIf you have specific feedback about how the built-in jj split UI could be improved vs
git add -p, you could leave it in Issues or Discussions for thescm-recordrepo.Speaking personally, I think you already captured some of my issues in your README (discoverability, predictability). But more than that, it’s the fact that, being used to
git add -p, I want something like that, instead of a worse version of magit (worse mainly because of the different key-bindings compared to my editor, I must say).Just my 2 cents :)
You can disable auto add of new files as of a few weeks.
Specifically, it looks like the way to disable auto-track is to configure
snapshot.auto-trackto benone(). Then you would usejj file trackto track files.If you only want to ignore files in certain directories or files with predictable names, you can use a
.gitignorefile instead.Do private commits work for the case you’re describing here?
https://martinvonz.github.io/jj/latest/config/#set-of-private-commits
Out of curiosity, in your example you move a fixup to the proper place and change the type to f for fixup. Are you aware of
git rebase -i --autosquashwhich does it automatically or am I missing something? Of course, with the interactive rebase you can do much more than that but I was wondering.I know about it but it’s not a major burden so I haven’t looked into it. And people are telling me to just learn Jujutsu so maybe I shouldn’t be learning new Git features?
I’m not the person you asked, but I am an example of someone who knows about
--autosquashyet still prefers the workflow ofgit rebase -iand manually typing thef. The main reason is that, for my project at work, I don’t like how the required “fixup! ” prefix makes the commit stand out ingit log:Given this project’s requirement to mention a ticket in each commit, the prefix looks too out of place. So I prefer to write “(squash)” after the ticket name in the message and then search for lines with that phrase by eye in my
git rebase -ieditor:Why include the ticket number in the fixup commit’s message? It gets absorbed into the previous commit; your history will not include it post rebase.
Good question – I had to think a bit to remember. It’s because my workflow sometimes includes pushing the fixup commits as-is to the remote branch. I do that when I was pairing with another dev on my computer and I want to make it easy for them to see the history of the work we accomplished that day, which would be harder if I squashed before pushing. When working on large changesets like that, I only squash (and then force push) after my coworker and I think we won’t need more changes and want to consider the final breakdown of commits.
My problem with switching away from git is I need to know git well in order to fix others’ git mess ups. It’s part of the job to use this particular tool. Otherwise, I’d be using Pijul.
The most wonderful thing about
jjis that you don’t have to switch away from git - it can be used either as a wrapper around git, or colocated with git (docs). So I wouldn’t think of it as “switching” from Git - it can be used alongside git to improve your own workflow.I started using jj two weeks ago, it’s really cheap to try. You can continue using all the normal git commands and slowly try the jj bits if you find them helpful. And jj’s undo feature is worth it even if you decide to otherwise stick to the git model.
I wasn’t aware that this was a default action in VSCode! I’ve fiddled with manually setting up my splits and jumping around code before, so this’ll be a nice little time save
For those curious, the default binding is “Ctrl-K F12” (I had trouble finding it, the command name is “Open Definition to the Side” in the command palette). I might try to find a better binding, but I don’t think I can sacrifice my comma key for it!
Oh this is quite helpful. Command Palette + “defside” uniquely identifies this command, and I’ll probably use that rather than set a new binding.
I explored Julia recently and created a BDD style testing framework for it called BeeDeeDee.
Some thoughts from my limited experience.
Data science and scientific computing in general. Lots of advanced stuff in Julia you would have a hard time finding in any other ecosystem.
True. Julia is essentially a modern Dylan on top of LLVM. It’s a really great language, and it makes easy to write fast code for HPC while retaining all benefits from dynamic typing. I’ve written a lot of Julia, and it’s hard to return to Python or R, where the two-language problem is so evident.
Furthermore, libraries compose really well. It’s lots of small libraries, Julia all the way down. When you do mathematics, this is really satisfying because you can easily peek into the internals in case some results look odd. That contrasts with Python, where there are exceptionally good libraries, but it’s hard to compose them (big silos) and inspect (tons of low level C/C++/FORTRAN).
I do agree that macros should be used a lot less often, but when they’re good they’re amazing. We have a ~70 line macro that dramatically changes the evaluation strategy of code (adding desired outputs to a DAG and then calculating them all at once), and doing something similar without macros would be about 3,000 lines of code – more importantly, it would introduce a lot of duplication that would be difficult to keep in sync.
Sounds awesome, is the code open source? I’d love to see it.
Unfortunately no. But the idea is straightforward: the macro takes a function that’s written to operate on the graph, and creates a few other versions of it, including one that operates on Julia expressions. This lets us write code like
add_expr_to_graph(slope(input_expr, num_bars)), whereinput_exprandslope(input_expr, num_bars)will get added as nodes to the graph to be evaluated later (as well as any sub-expressions).The major benefit is that if
input_expris used multiple times, we only calculate it once, and this makes a big difference for our use case. It also allows us to evaluate everything in parallel. Standard DAG stuff overall, but it means that our researchers can write code that looks like normal computational code, and we can execute it efficiently in production.The example code you posted is a delight to read. Thanks for sharing it and your work!
Hm interesting … though is that example really better than:
seems like a lot less typing and just as clear!
BTW in Oils we do
to give a “lazy/unevaluated expression”, which can be destructured and understood by the test framework. This allows you to print the LHS and RHS, which I mentioned here
https://old.reddit.com/r/ProgrammingLanguages/comments/1fg5ae2/a_retrospective_on_the_oils_project/ln14y5g/
Picture: https://app.oilshell.org/picdir/uploads/1nmmxpl__ysh-assert-short.png
I think that is the reason they might be doing that over the simpler
>That is, the
|>gives you something whereuser.ageand18can be understood separately by the test framework perhaps?It doesn’t seem like the conventional usage of
|>, which is for data transformations via function chainingThe pipe allows for chaining.
Sure, but my point is that chaining usually used for data transformation, and that is not happening in the example
And also that it’s longer than writing out
user.age > 18There could be other instances where a test framework makes good use of
|>for chaining, but I don’t see itWe need a style that helps spot the mismatch between textual description and test code 🫣
It’s suppose to read like an English sentence. Totally up to you if you prefer your way.
I disagree with a lot of what Stroustrup says, but I do like one comment from him (paraphrasing from memory):
Any useful technology is going to have to make a load of compromises to be useful. I consider it a good rule of thumb that, if you couldn’t sit down and write a few thousand words about all of the things you hate about a technology, you probably don’t understand it well enough to recommend it.
I’ve come to dislike that comment, and would put it in the same category as “everything is a tradeoff”. It’s a thought-terminating cliche that’s used to fend off criticism and avoid introspection.
There are such a thing as bad engineering decisions. Not everything was made with perfect information, implemented flawlessly to spec, with optimal allocation of resources. In fact many decisions are made on the bases of all kinds of biases, including personal preferences, limited knowledge, unclear goals, a bunch of gut feeling, etc.
And even a technology with good engineering decisions can turn a lot worse over time, e.g. when fundamental assumptions change.
I agree with you, but I’d like to defend the phrase “everything is a tradeoff” please!
To me, the natural corollary is that you should decide which set of tradeoffs are best for you.
All of the things you said are true but you can avoid a lot of pitfalls by being aware of what you are optimising for, what you are giving up, and why that might be appropriate in a given situation.
You said it already
That is better than “everything is a tradeoff” and makes the pithy statement less pithy and more actionable.
(neo)Confucianism teaches that solutions don’t exist, rather just sets of trade offs. E.g. you can choose to have your current problem, or the problem which will occur if “solve” it.
What’s your background with Confucianism? I would say it’s fairly optimistic about human perfectibility. It’s maybe not utopian, but a sage king can rightly order the empire with the Mandate of Heaven at least. Or do you mean more contemporary Confucian inspired thought not classical (c300BCE) or neo (c1000CE)?
Neoconfucianism (in Chinese, study of “li”) synthesized it with Daoism and Buddhism (the 3 teachings). Wu wei is an important aspect of that li (logos, natural law) channeling the Zhuangzi’s pessimism. Yangmingzi riffs on this, believing in random action (experimenting?) but not consciously planning/acting towards plans. You’re to understand the trade offs etc. and map the different ways destiny may flow, but not act on them. Original Confucianism had a more limited focus (family first) which Zhang Zai extended, by treating everything as a bigger family, allowing Confucian approaches to apply to other domains.
One 4 character parable/idiom (which means “blessing in disguise”) has:
Wing-tsit Chan and Lin Yutang made great translations and discussions on the history of ideas in Chinese thought. Though I may read Chen Chun, it’s really through their lenses as my Chinese isn’t yet up to snuff.
Okay. I wasn’t sure if by “neo” you meant like Daniel Bell’s New Confucianism.
I would say Wang Yangming is pretty optimistic about solutions. He’s against theoretical contemplation, but for the unity of knowledge and conduct, so ISTM the idea is you solve problems by acting intuitively. I’m sure if pressed he would acknowledge there are some tradeoffs, but I don’t see him as having a very pessimistic view or emphasizing the tradeoffs versus emphasizing perfecting your knowledge-conduct.
Thank you, I dug into this a bit deeper. I believe you are right and I have been misunderstanding some aspect of will./intention, which I struggle to articulate. Laozi and everything later building on it do seem to focus on (attempts to) control backfiring. I’m not sure if my pick-tradeoffs-lens is a productive innovation or missing the point. (How broadly/narrowly should we apply things?)
I’m also tired of hearing “everything is a trade-off” for that reason. I definitely like the phrase “thought-terminating cliche”.
It’s also not true. “Everything is a trade-off” implies that everything is already Pareto-optimal, which is crazy. Lots of things are worse than they could be without making any compromises. It even feels arrogant to say that anything is any kind of “optimal”.
That was exactly my point, thanks for nailing it concisely.
that pareto-optimality explanation site is fantastic
Of course, I think people mostly mean that there are bad approaches, but no perfect ones. (Or, more mathematically, the “better than” relation is often partial.)
Both sides phrases are inportant and meaningful, yes people can overuse them, and people can also fail to understand that “changing this behavior to be ‘sensible’” also is a trade off as changing behaviour can break existing stuff.
We can look at all sorts of things where the “trade off” being made is not obvious:
lack of safety in C/C++: yay performance! Downside: global performance cost due to myriad mitigations in software (aslr, hardened allocators, …) and hardware (pointer auth, mte, cheri, …) cost performance (power and speed) for everything
myriad weird bits of JS - mean lots of edge cases in the language, though in practice the more absurd cases aren’t hit and basic changes to style choices mitigate most of the remainder, so the cost of removing the behavior is unbounded and leaving it there has little practical cost
removing “print” statements from Python 3: made the language more “consistent” but imo was one of the largest contributors to just how long the 2->3 migration took, but was also entirely unnecessary from a practical point of view as a print statement is in practice distinguishable from a call
At the end of the day you might disagree with my framing/opinion of the trade offs being made, but they’re still trade offs, because trade offs are a fundamental part of every design decision you can ever make.
There’s nothing thought terminating about “everything is a trade off”, claiming that it is is itself thought terminating: it implies a belief that the decisions being made are not a trade off and that a decision is either right or wrong. That mentality leads to inflexibility, and arguably incorrect choices because it results in a design choices that don’t consider the trade offs being made.
But what about the time when the decisions were actually made? What technical, calculated trade-offs did JS make when implementing its numerous inconsistencies, that are collectively seen as design failures?
I definitely think some decisions can be right or wrong.
The key tradeoff made in the development of JavaScript was to spend only a week and a half on it, sacrificing coherent design and conceptual integrity in exchange for a time-to-market advantage.
This isn’t really true. The initial prototype was made in 10 days but there were a lot of breaking changes up to Javascript 1.0 which was released a year later. Still a fairly short time frame for a new language but not exactly ten days.
I often wonder what development would be like now if Brendan Eich had said “no, I can’t complete it in that time, just embed Python”.
I don’t think Python even had booleans at that point. IMHO the contemporary embedding language would have been Tcl, of all things!
The starting point was Scheme, so that would probably have been the default choice if not implementing something custom.
At least there the weird quirks would’ve (hopefully) gotten fixed as bugs, because it has an actual language spec. OTOH, it might also have begotten generations of Scheme-haters and parenthophobes, or Microsoft’s Visual Basicscript would’ve taken off and we’d all be using that instead. Not sure what’s worse…
When was Microsoft adding VBScript to IE?
I’m not sure there’s an actual trade-off there. Don’t you think it’s possible to come up with a more coherent design in that timeframe?
Unlikely given the specific constraints Eich was operating under at the time.
Some behaviors are not the result of “decisions”, they’re just happenstance of someone writing code at the time without considering the trade offs because at the time they did not recognize that they were making a decision that had trade offs.
You’re saying there are numerous inconsistencies that were implemented, but that assumes that the inconsistencies were implemented, rather than an unexpected interaction of reasonable behaviors, without knowing the exact examples you’re thinking of I can’t speak to anything.
With the benefit of hindsight, or with a different view of the trade offs. Do you have examples of things where the decision was objectively wrong and not just a result of the weight of trade offs changing over time, such that the trade offs made in the past would not be made now?
A good example that happens all the time in the small is doing redundant work, mostly because you’re not aware it’s happening. Cloning data structures too often, verifying invariants multiple times, etc. I’ve seen a lot of cases where redundancies could be avoided with zero downside, if the author had paid more attention.
This makes me think how beautiful it is that crypto developers have managed to make NFT’s into not only something everyone hates but something nobody uses, at the same time
The quote you refer to is: “There are only two kinds of programming languages: those people always bitch about and those nobody uses.”
Another good one is “For new features, people insist on LOUD explicit syntax. For established features, people want terse notation.”
I’ve never heard this one before! It’s really good.
Here’s the primary source on that one: https://www.thefeedbackloop.xyz/stroustrups-rule-and-layering-over-time/
Looks like there’s some sort of error though. I’m on my phone so I’m not getting great diagnostics hah.
As a core element of good critical thinking, one should hypothetically be able to write such a criticism about anything they are a fan of. In fact, I encourage everyone to try this out as often as possible and push through the discomfort.
Notice I used the dreaded word “fan” there- which is the point of this comment: There should be a key distinction between someone who is a “fan” of a technology based on a critical evaluation of its pros and cons and someone who is a “fan” of a technology based on a relatively rosy assessment of its pros and a relatively blind assessment of its cons.
I think the OP blogger is really complaining about the latter. And, all other things being equal, I believe a developer using a technology chosen via critical assessment by a fan will always lead to superior work relative to a technology chosen via critical assessment by a non-fan. The fan, for example, will be motivated to know and understand things like the niche micro-optimizations to use that don’t make the code less readable (I’m thinking of, for example, the “for” construct in Elixir), and will likely use designs that align closer to the semantics of that particular language’s design than to languages in general.
One of the reasons I left Ruby and went to Elixir is that the “list of valid and impactful criticisms” I could come up with was simply shorter (and significantly so) with Elixir. (Perhaps I should blogpost a critical assessment of both.) And yes, I went from being a “fan” of Ruby to a “fan” of Elixir, but I can also rattle Elixir’s faults off the top of my head (slowish math, can’t compile to static binary, complex deployment, depends on BEAM VM/Erlang, still a bit “niche”, functional semantics more difficult to adopt for new developers, wonky language server in VSCode, no typing (although that’s about to change somewhat), not as easy to introspect language features as Ruby, etc.)
The other point I’d like to make is that even though “everything is a compromise,” there are certainly locally-optimal maxima with the more correct level of abstraction and the more correct design decisions. Otherwise we should all just code in Brainfuck because of its simple instruction set or in assembly because of its speed.
I think the distinction would be that I wouldn’t really call you a “fan” of Ruby or Elixir if you’re making these considered decisions, weighing the trade-offs, and considering whether they’re appropriate more-or-less dispassionately. You can certainly like languages, but I think if you call someone a “fan” of something, there’s an implication of a sort of blind loyalty. By analogy to sports fans, where a fan always supports their team, no matter who they’re up against, a fan of a particular technology is someone who always supports their particular technology, rails against those who criticize it, and hurls vitriol against “opponents” of their tool of choice.
Alright. Interesting distinction/clarification that gave me an idea.
So, in thinking of my Apple “fandom” that has been pretty consistent since I was 12 in 1984 when my family was simultaneously the last one in the neighborhood to get a family computer and the first ones to get a Mac (128k) and I was absolutely fucking enthralled in a way I cannot describe… which persisted through the near-death of 1997 and beyond into the iPod and iPhone era…
I think it has to do with “love”, frankly. If you “love” something, you see it through thick and thin, you stick around through difficulties, and you often (in particular if you contribute directly to the good quality of the thing, or the quality of its use, or its “evangelism”, or its community) literally believe the thing into a better version of itself over time.
The “likers”, in essence, value things based on the past and current objective value while the “lovers” (the fans) value things based on the perceived intrinsic and future value.
And the latter is quite irrational and thus indefensible and yet is the fundamental instrument of value creation.
But can also lead to failure. As we all know. The things the “likers” use are less risky.
Does this distinction make sense?
One factor is that many software development projects are much closer to bike sheds than to skyscrapers. If someone is a fan of, say, geodesic domes (as in, believes that their current and/or future value is underrated), there is no reason not to try that in constructing a bike shed — it’s unlikely that whatever the builder is a fan of will completely fail to work for the intended purpose. The best outcome is that the technology will be proved viable or they will find ways to improve it.
If people set out to build a skyscraper from the start, then sure, they must carefully evaluate everything and reject unacceptable tradeoffs.
When people build a social network for students of a single college using a bikeshed-level technology stack because that’s what allowed them to build it quickly on zero budget and then start scaling it to millions users, it’s not the same problem as “started building a skyscraper from plywood”.
OTOH, no sane architect or engineer would expand a bikeshed into a skyscraper by continuing with the same materials and techniques. They’d probably trash the bikeshed and start pouring a foundation, for starters…
Exactly. When managers or VCs demand that a bikeshed is to be expanded into a skyscraper using the same materials, it’s not an engineering problem. Well, when engineers choose to do that, it definitely is. But a lot of the time, that’s not what happens.
Thank you! The “critical thinking” aspect is I think muddied by the article by setting up an ingroup/outgroup dichotomy with engineers on one side and fans on the other.
It’s normal to be a fan of something while also being fully aware of its trade-offs. Plus, sometimes an organization’s inertia needs the extra energetic push of advocacy (e.g. from a fanatic) to transition from a “good enough / nobody got fired for buying IBM” mentality into a better local optimum.
The mindset of “everything is a trade-off” is true but can also turn into a crutch and you end up avoiding thinking critically because oh well it’s just some trade-offs I can’t be bothered to fully understand.
“Engineers” and “fans” don’t look at the same trade-offs with different-colored glasses, they actually see different sets of trade-offs.
I would add: you should also be able to defend the options you didn’t choose. If someone can give a big list of reasons why Go is better than Rust, yet they still recommend Rust for this project, I’m a lot more likely to trust them.
This is true. I remember people hating Java, C++ and XML. Today I more often meet people hating Python, Rust and YAML. Sometimes it is the same people. Older technologies are well established and the hatred has run out. People have got used to it and take it for what it is. Hyped technologies raise false hopes and unrealistic expectations, which then lead to disappointment and hate.
I always struggle with the YAGNI concept because, in my experience, writing generic code is not more complex than writing over-specialised code and can often lead to simplifications for the specific use case. I then often end up being able to either copy code from one project to another, or factor it out into a library that both can depend on.
The one constant in software engineering is that your requirements will have changed by the time you’ve implemented them. Writing code that’s flexible in the presence of changing requirements is software engineering.
I like to distinguish “generic” from “general”, whereby “generic” means “independent of irrelevant details” and “general” means “handling many cases”. I believe in YAGN generality, but like you, I value genericity.
I tend to break YAGNI defensively. If, early in the implementation work, my gut tells me that the current “thin” specs are the result of an inability to formulate use cases in sufficient depth on the product management side, then there’s a very high chance that I am, in fact, GNI. In that case, I will totally write the more generic version, because there are two things that can happen one year down the line:
What if I don’t have time for that refactoring? Haha. If I can’t find an hour for a simple feature-dropping refactoring, I definitely won’t find the way more refactoring hours required to morph the code that implemented what we thought we needed a year ago into the code that implements what we actually understand we need today.
The undelying assumption in the “but what if you don’t have time to refactor it?” is that, as more requirements are formulated, code only needs to be “extended” to cover them, you just “add” things to it. That’s one of the worst sermons of the engineering management cargo cult.
Also, code written by successive generations of YAGNI people is some of the worst there is. It never evolves into a generic solution, it just snowballs into umpteen particular solutions held together by some inconsequential generics.
For me, YAGNI is not necessarily saying you shouldn’t write generic code. I’ll try to explain with an example from my team’s current project. We’ve been writing a program to manage hardware and the software running on it because none of the infrastructure management software we evaluated fit our needs well enough. So we have a daemon on our edge hardware and a service running in the cloud that coordinates that hardware and hands out configuration. That service connects to a database, which in my team’s case is Postgres. The engineer who wrote the initial draft of this system, however, added in a constant and some code to be able to work with any SQL database. However, it is highly unlikely that we would ever use a database besides Postgres since we’re already using it. This is the sort of thing YAGNI is about - you’re working on a team that has infrastructure in place, you know what you’re writing is internal and not meant to be consumed/built on by others. Why would you put in the work to support different types of infrastructure than you need to?
So yeah, requirements will change, but what you’re building on top of usually will not. So don’t try to support more than what your team uses unless you know with certainty that you need more.
The deciding factor for me is usually indirection. If the more specific approach lets me avoid a layer or two of indirection, I’ll usually pick that. If it doesn’t, I’ll go with the broader version.
At my previous job, I saw a feature added that wasn’t once used in the 11 years I was there. It stared out as a way to segment SS7 traffic to different backends, and a later addition of SIP had to support the same concept, yet it was never used. It was discussed from time to time to use it, but never came to fruition. And it would take more work, and coordination with another development team and ops, to fully remove the feature.
I think YAGNI is against adding feature or making code more complex in anticipation of future value. I have never seen anything against just making the code just as simple (or even simpler) by using a more general version.
I’ve heard this called “implement for today, design for tomorrow” and I think it is fundamentally at odds with YAGNI. The problem is it is (by definition?) more complex to read, and therefore maintain, tomorrow. There’s a quote about it:
The quote is from the XP book, but I think in the context of this thread we can safely replace the word with YAGNI. Have you made the bet in the past and it hasn’t paid off, or have you just not tried it yet?
The bet is also that when you do need to make that change, you will have more information, so you will be able to do a better job then than you will be by anticipating those needs now.
It is hard to recommend [anything but AWS]
sad but true for large cloud providers
more niche cloud providers like cloudflare and fly.io and the like are making inroads, though
AWS is reliable but too coarse compared to the Google Cloud Platform The engineering productivity cost is much higher with AWS than with GCP.
With the understanding that I might just have been lucky, and / or working for a company seen as a more valuable customer … in a previous role I was a director of engineering for a (by Australian standards) large tech company. We had two teams run into difficulties, one with AWS, and another with GCP.
One team reached out to Amazon, and our AWS TAM wound up giving us some great advice around how we’d be better off using different AWS tech, and put us in touch with another AWS person who provided some deeper technical guidance. We wound up saving a tonne of money on our AWS bill with the new implementation, and delighting our internal customers.
The other team reached out to Google, and Google wouldn’t let a human speak to us until we could prove our advertising spend, and as it wasn’t high enough, we never got a straight answer to our question.
Working with Amazon feels like working with a company that actually values its customers; working with Google feels like working with a company that would prefer to deal with its customers solely through APIs and black-box policies.
Indeed. If you think you will need a human to get guidance, then GCP is an inferior option by a huge margin.
Where in my experience, guidance can include “why is your product behaving in this manner?”.
What does “too coarse” mean?
I’m not sure I believe you that the “engineering productivity cost” is higher with AWS: How exactly did you measure that?
I have a Docker image (Python/Go/static HTML) that I want to deploy as a web server or a web service.
GCP is far superior than AWS on this measure.
I’m still confused. What do you have to do in AWS that you don’t have to do in GCP or is significantly easier? I have very little experience with GCP and a lot with AWS, so interested to learn more about how GCP compares.
I have a good enough experience with both platforms so this is something you have to try and you will see the difference.
GCP primitives are different and much better than AWS.
“Trust me bro” is a difficult sell; but I am here to back up the sentiment. There are simply too many little things that by themselves are not deal breakers at all - but make the experience much more frustrating.
GCP feels like if engineers made a cloud platform for themselves with nearly infinite time and budget (and some idiots tried to bolt crud on the side, like oracle, vmware and various AI stuff).
AWS feels like if engineers were forced to write adjacently workable services without talking to each other and on an obscenely tight deadline - by people who didnt really know what they wanted, and then accidentally made something everyone started to use.
I’m going yo write my own blog post about this, I used to have a list of all my minor frustrations so I could flesh it out.
Thanks, I would love to link it.
I’m still working on it, there’s some missing points and I want to bring up a major point about cloud being good for prototyping, but here: https://blog.dijit.sh/gcp-the-only-good-cloud/
Thanks for sharing.
If it helps I have the same professional experience with AWS and GCP. GCP is easier to work with but an unreliable foundation.
Correct me if I’m wrong but you can do all of the above with Elastic Beanstalk yes? Maybe ECS as well?
The trickiest part would be using AWS Secrets Manager to store/fetch the keys, which has a friendly enough UI through their web, or CLI.
You can definitely do all of this with EKS easily, but that requires k8s knowledge which is a whole other bag of worms.
The thing you are describing I should have only needed to learn once for each cloud vendor and put in a script: The developer-time should amortise to basically zero for either platform as long as the cloud vendor doesn’t change anything.
GCP however, likes to change things, so…
Yes and no. Some constructs you don’t even have to worry about on GCP, so, it is best for fast deployments. However, if you spend 500+ hours a year doing cloud work, then your point stands.
500 hours a year is only about five hours a week.
My application has run for over ten years. It paid for my house.
You’re damn right my point stands.
Indeed, if you are doing 5 hours a week of cloud infra work, your application is likely a full-time job or equivalent. I do believe you made the right choice with AWS.
Five hours a week is a full-time job?
You’ve got some strange measures friend…
read the post you replied to again, it does not say that.
No. If you are spending 5 hours a week on infrastructure, then your application would be worth spending 40-hours on. Or is infra the only component of your application?
Can you do an actual comparison of the work? I’d be curious. Setting up a web service from a Docker image is a few minutes. Env vars are no extra work. Secrets are trivial. A cert would take maybe 10 minutes?
Altogether, someone new should be able to do this in maybe 30 minutes to an hour. Someone who’s done it before could likely get it done in 10 minutes or less, some of that being downtime while you wait for provisioning.
I have done it for myself and I’m speaking from experience.
You’ve posted a lot in this thread to say, roughly ‘they’re different, AWS is better, but I can’t tell you why, trust me’. This doesn’t add much to the discussion. It would help readers (especially people like me, who haven’t done this on either platform) if yo u could slow down a bit and explain the steps in AWS and the steps in GCP and why there’s more friction with the latter.
No, please don’t trust me. Here’s an exercise for yourself.
Assuming you control a domain
domain1.com.Deploy a dockerized web service (pull one from the Internet if you don’t know how to write one). Deploy it on
gcp1.domain1.com, Deploy another onaws1.domain1.com. Compare how many steps it takes and how long it takes.Here’s how I deploy it on GCP. I never open-sourced my AWS setup but I am happy to see a faster one.
As I said, I have not used AWS or GCP so I have no idea how many steps it requires on either platform. I can’t judge whether your command is best practice for CGP and have no idea what the equivalent is on AWS (I did once do this on Azure and the final deploy step looked similar, from what I recall, but there was some setup to create the Azure Container thingy instance, but I can’t tell from your example if this is not needed on GCP of if you’ve simply done it already). If I tried to do it on AWS, I’d have no idea if I were doing the most efficient thing or some stupid thing that most novices would manage to avoid.
You have, apparently, done it on both. You are in a position to tell me what steps are needed on AWS but not on GCP. Or what abstractions are missing on AWS but are better on GCP. Someone from AWS might even read your message and improve AWS. But at the moment I just see that a thing on GCP is one visible step plus at least zero setup steps, whereas on AWS it is at least one step.
You’ve posted ten times so far in this story to say that GCP is better, but not articulated how or why it’s better. As someone with no experience with either, nothing you have said gives me any information to make that comparison. At least one of the following is true:
It sounds as if you believe the first is true and the second two are not but (again) as someone reading your posts who understands the problem but is not familiar with either of the alternatives in any useful level of detail, I cannot judge for myself from your posts.
If you wrote ‘GCP has this set of flows / abstractions that have no equivalent on AWS’ then someone familiar with AWS could say ‘actually, it has this, which is as good’ or ‘you are correct, this is missing and it’s annoying’. But when you write:
That doesn’t help anyone reading the thread understand what is better or why.
Given my experience with AWS, my vote is #2.
I find AWS to be better organized and documented but much larger that any Google product I’ve seen. There is more to learn because it’s a large, modular system. And it’s trivial to do an easy things the hard way.
I don’t have much direct experience with Google’s hosting but if any of their service products are similar, the only advantage is they do less, which means you can skimp on organization and documentation without too many problems.
IMO, it’s hard to recommend AWS as well.
So what do you recommend then? 🙂
In terms of big players I would recommend GCP still, but only because I mostly work with Kubernetes and it’s best there. From smaller players Fly.io is actually works well for me.
Why is it “best” there? I use EKS on aws and have had no issues with it…?
in Kubernetes territory, Google is the elder. Also cleaner and more intuitive UI.
not shocking. aren’t they the original contributors?
I don’t touch EKS’s ui much/ever so I honestly don’t really care about that. Usually use aws via terraform/pulumi.
I find the opposite of this is far more often the problem. Just let the data be. Having two arguments of the same type is not a good enough reason to complicate your data representation. The solution to that is keyword arguments, or passing a record with named fields, or just naming the function in a way that suggests the order.
I’ve also gone back to mostly using primitive types, after having spent a few years going really heavy on specific types. The major advantages are staying close to the data, having an easier time with serialization/deserialization, and having an easier time with big refactors (because you don’t have a large type hierarchy that may need to change.) I usually stick to a handful of Structs to represent cleaned up versions of input/output data, with no nested Structs, and no special Structs to represent intermediate values.
I like Zach Tellman’s generalization of this, which is (paraphrasing) that your cores should contain code that makes strong assumptions. Those assumptions should buy you things like simplicity and performance. Your shell should generally try to conform data to the assumptions of a relevant core.
Synchronous code is a reasonable core, and so is code that runs on the GPU for a machine learning pipeline – these are both restrictive sets of assumptions that buy you something.