You’ll note the build script tries to tail the container logs to nearly-automate the copying, but I’m not sure if it actually works – I haven’t taken the time to troubleshoot it.
What you could do instead is:
Remove the infinite sleep and grep logic to try to extract the path to the binary;
Simply call cp "$(cabal list-bin $EXECUTABLE_NAME)" bin after the containerised build finishes.
Another small improvement would be to add caches outside of the image:
That way we can avoid rebuilding everything between runs. I suppose we could reuse the host’s cache here, but prefer to keep them separate.
Thanks for writing this up by the way, I found it useful. I guess what would be really neat is if we could have a nix config (with system dependencies) and then derive the alpine dockerfile from that (that way we wouldn’t need to update system dependencies in two places.)
Although Haskell is a compiled language and the default build step results in a single binary for a Yesod site, it is not a deployable single binary, because it may still depend on libraries and resources on the machine it was built.
I’m genuinely curious—and I mean overall, and not just in Haskell, where you need IO to read files from disk—what’s the big advantage in having a single binary rather than a folder with a binary and data files?
I’m not talking about static linking here, just about where static resources are stored.
Distributing the typeface separately along with the binary would complicate deployment, because then we need to care for search paths and file system privileges.
It doesn’t seem very complicated to set the correct WorkingDirectory for your systemd unit. As for filesystem privileges… I guess if your service can write to the disk, there’s always the possibility of a path traversal vulnerability, which would allow an attacker to overwrite the service’s own static files. Having them read-only could help as a mitigation. (But that would not prevent the attacker from overriding any files the server needs writable, such as SQLite databases.)
In my experience the contrast between developing with golang vs ruby is wild, they are two ends of a spectrum. On one end you have a runtime, dynamic system libraries, dependencies with linked dependencies, your actual project’s source files, and potentially a ton of non-ruby source files (data, config, assets, etc). On the other end, you have.. a binary and maybe a few config/data files? And with embed you can just have a single binary?
The difference between deployment and distribution (as a server bin or user facing cli) is where this difference shows up for me. I simply cannot distribute scripts to people or servers as readily as I can a single binary.
From what I can tell, this is the ideal the author is working toward. The difference between a single binary and a binary plus a few assets isn’t much, but far down on the opposite end of that spectrum quite is unwieldy.
Even if there is heavy runtime io, it is solving the distribution problem for me.
what’s the big advantage in having a single binary rather than a folder with a binary and data files?
Apparently more and more people are not used to file systems, folders, and specifically file system hierarchies, plausibly because the computers people interact with the most, their phones, hide this structure.
I can assure you familiarity is not the problem in this case. It’s a function of caring a lot about the availability of my side projects combined with having a full time job, small children, and many competing spare time interests that prevent dropping all prior obligations when something goes wrong.
You know… for my personal projects the pendulum has swung back and forth between “self host everything on a VM” to “put it in docker containers and host it in K8s” a few times.
Your comment makes me smile. Producing single-binary builds and pushing them to VMs is probably the happy medium between those two things. K8s has generally worked ok but sometimes things would break in the complexity (eg the nginx-based LetsEncrypt auto-certificate stuff, resulting in a painful debug experience because I hadn’t touched it in a year). The self-host thing would sometimes break because e.g. a system Python package would change or an OS upgrade would be required to get a library version for a new project.
Containers get easier too when everything is built into a single file.
If it can be a static asset, there should at least be an option to stick it in the binary for easier management.
Unbidden advice: always make it easy to host your http application at an arbitrary route, not relying on being the top level. That makes life easier too.
Unbidden advice: always make it easy to host your http application at an arbitrary route, not relying on being the top level. That makes life easier too.
for an example of a single-binary Ocaml web app (eeven without requiring root to install) in action, see https://seppo.social/aseppototry
What you could do instead is:
cp "$(cabal list-bin $EXECUTABLE_NAME)" binafter the containerised build finishes.Another small improvement would be to add caches outside of the image:
That way we can avoid rebuilding everything between runs. I suppose we could reuse the host’s cache here, but prefer to keep them separate.
Thanks for writing this up by the way, I found it useful. I guess what would be really neat is if we could have a nix config (with system dependencies) and then derive the alpine dockerfile from that (that way we wouldn’t need to update system dependencies in two places.)
Ohh, yeah, that’s really clever. Thanks!
I’m genuinely curious—and I mean overall, and not just in Haskell, where you need
IOto read files from disk—what’s the big advantage in having a single binary rather than a folder with a binary and data files?I’m not talking about static linking here, just about where static resources are stored.
It doesn’t seem very complicated to set the correct
WorkingDirectoryfor your systemd unit. As for filesystem privileges… I guess if your service can write to the disk, there’s always the possibility of a path traversal vulnerability, which would allow an attacker to overwrite the service’s own static files. Having them read-only could help as a mitigation. (But that would not prevent the attacker from overriding any files the server needs writable, such as SQLite databases.)I don’t know of a big advantage, but just one less thing that can go out of sync with expectations.
In my experience the contrast between developing with golang vs ruby is wild, they are two ends of a spectrum. On one end you have a runtime, dynamic system libraries, dependencies with linked dependencies, your actual project’s source files, and potentially a ton of non-ruby source files (data, config, assets, etc). On the other end, you have.. a binary and maybe a few config/data files? And with embed you can just have a single binary?
The difference between deployment and distribution (as a server bin or user facing cli) is where this difference shows up for me. I simply cannot distribute scripts to people or servers as readily as I can a single binary.
From what I can tell, this is the ideal the author is working toward. The difference between a single binary and a binary plus a few assets isn’t much, but far down on the opposite end of that spectrum quite is unwieldy.
Even if there is heavy runtime io, it is solving the distribution problem for me.
[Comment removed by author]
Apparently more and more people are not used to file systems, folders, and specifically file system hierarchies, plausibly because the computers people interact with the most, their phones, hide this structure.
I can assure you familiarity is not the problem in this case. It’s a function of caring a lot about the availability of my side projects combined with having a full time job, small children, and many competing spare time interests that prevent dropping all prior obligations when something goes wrong.
You know… for my personal projects the pendulum has swung back and forth between “self host everything on a VM” to “put it in docker containers and host it in K8s” a few times.
Your comment makes me smile. Producing single-binary builds and pushing them to VMs is probably the happy medium between those two things. K8s has generally worked ok but sometimes things would break in the complexity (eg the nginx-based LetsEncrypt auto-certificate stuff, resulting in a painful debug experience because I hadn’t touched it in a year). The self-host thing would sometimes break because e.g. a system Python package would change or an OS upgrade would be required to get a library version for a new project.
Thanks for the idea!
Containers get easier too when everything is built into a single file.
If it can be a static asset, there should at least be an option to stick it in the binary for easier management.
Unbidden advice: always make it easy to host your http application at an arbitrary route, not relying on being the top level. That makes life easier too.
Sigh. Yes. 100% yes.
distribution and especially onboarding. It may be only copying one single file. It is with https://seppo.social/en/support/#installation, a single-binary Ocaml web application.