Does anybody know a cross-platform language-agnostic, environment-oriented package manager? Cross-platform means it can be used on Linux, Mac, and Windows (natively, not Cygwin, WSL, etc). Language-agnostic means it can be used for arbitrary libraries and binaries. Environment-oriented means packages are installed into some folder like Python virtualenv and not system- or user-wide.
The is a list on Wikipedia. The “hybrid systems” are the closest ones. Apart from GoFish they focus on Unix-like though. GoFish looks interesting but is not environment-oriented.
I could say Docker and not be sure if I’m trolling or being realistic.
It sounds like a defeat: giving up on trying to find a bridge between the various operating systems.
But targetting both Windows and Linux makes no sense, as they do not have the same syscalls on the first place. This will reduce the packages to only the ones that Windows can support Windows, Mac OS, Linux.
Windows approach to this was to adapt syscalls through an emulation layer. Why go against Microsoft in its own effort to improve portability?
I should have said: makes no sense [to me].
Now I realize: maybe the aim of the author is to package software that already supports Windows and Mac OS explicitly…
Hm.
Docker would be a way to define complete environments and deploy them. It provides no help for the integration.
Have you heard the good word about pkgsrc?
I don’t seem to see Windows mentioned as supported on the linked page, while I it’s listed as an explicit need by the OP?
oops, missed the negation.
They do support windows via cygwin. Not ideal, definitely. Also I’m sure it works under WSL(but have never tried it).
Also, it comes ready to go with 22.5k packages, and can handle env oriented(as it does on macOS) or OS oriented(as it does on NetBSD and friends) installations..
This is difficult:
There is a huge semantic gap between
Most existing tools for one of those categories suck. There are only very few tools that don’t suck in the category they target.
You are basically asking for a tool that does both, and doesn’t suck.
Such a tool sadly doesn’t exist, but https://distr1.org/ is trying an interesting combination of new things which might be interesting to you.
git? What you’re really describing though is kinda the holy grail of software atm. I don’t know of anything that really does it, or if it’s really even possible to do it.
Unfortunately, git sucks for maintaining and deploying larger binaries. My experience with git-lfs was not good.
http://0install.net/ maybe?
Interesting. I remember zero install as Linux only (like FlatPack, Snap, etc). However, they provide a Windows-specific .Net implementation.
Looked some more as I was missing the “environment-oriented” part. Found 0env to enable it.
You said “Environment-oriented means packages are installed into some folder like Python virtualenv and not system- or user-wide”, but it’s not quite clear to me what you’re trying to achieve with that.
If you just want to avoid conflicts (e.g. ~/project1 and ~/project2 need different versions of a library) then 0install will handle that by default even with a user-wide cache.
If you really want completely isolated caches, you can set
$ZEROINSTALL_PORTABLE_BASEto the directory to use.BTW, we’re currently in the process of updating the documentation - you might find http://docs.0install.net/ slightly more up-to-date.
I think Nix is this. Nix works rather well most of the time.
Except that it does not work natively on Windows. Without knowing more details, it does tick off the other requirements:
cross-platform language-agnostic,
nixpkgs has the usual applications, C libraries, and language-specific package/derivation sets. Of the listed platforms, it supports Linux well, macOS reasonably well (stuff breaks fairly frequently on
master, but work well on stable releases).environment-oriented package manager? […] Environment-oriented means packages are installed into some folder like Python virtualenv and not system- or user-wide.
The nix store (
/nix) is system-wide, but users decide what packages to install in their own profile (and thus do not occur in the profile of other users). More over, you can usenix runornix-shellto get a shell with a specific set of dependencies that are only visible within that shell. Moreover, for projects, you can crate ashell.nixthat specifies dependencies (to use withnix-shell) or set updirenvto automatically useshell.nixwhen switching to a project directory/subdirectory.For me the latter is one of the ‘superpowers’ of Nix:
cd alpinocorpusand I have an environment CMake, make, Boost, dbxml, Xerces-C, XQilla, etc.cd ../reductiveand I have an environment with a Rust compiler, cargo, and Intel MKL.Of course, a system is never perfect, Nix also has many sharp edges. Moreover, it becomes particularly useful if you are willing to learn the Nix language (which is a small, lazy functional language).
I found no evidence that Nix works well on Windows. It only seems to be possible with some Unix emulation (Msys, Cygwin, WSL, etc).
I was kinda trying to port a part of Nix to native (non-WSL) Windows (as one of my too many hobby projects). One thing I found out, that I believe is especially annoying there for such an effort, is Nix’s heavy use of symbolic links. AFAIK, in default configuration of Windows, creating those requires Administrator prompt/elevated permissions.
Maybe you want conda and the packages on conda-forge?
Yes, conda looks good. The fact that it is advertised for Python and R suggests some genericity. It is hard to find documentation about anything not Python though.
an example from bioconda (derived from conda-forge), using Rust: https://github.com/bioconda/bioconda-recipes/blob/33c024c852e1292d4548407a3b3c2884f24acc93/recipes/rust-bio-tools/meta.yaml
pkgsrc https://pkgsrc.org
Handles 23 supported platforms. It doesn’t natively handle Windows, but does support cygwin. pkgsrc can install to it’s own place(does on macOS) or can install to system places like it does on NetBSD.
Comes ready to go with 22,500 packages.
There’s
esy, which runs natively on Linux, Mac, and Windows (but does require admin privileges on Windows because it uses symlinks). Right now it only has JS and OCaml packages on it, but their goal is to support pretty much any language in the future (and I think they’re targeting C++ next).This is difficult:
There is a huge semantic gap between
Most existing tools for one of those categories suck. There are only very few tools that don’t suck in the category they target.
You are basically asking for a tool that does both, and doesn’t suck.
Such a tool sadly doesn’t exist, but distri is trying an interesting combination of new things which might be interesting to you.
Why would you want that? Other than being really complex to implement I don’t see any reason why you’d want to handle development libraries globally instead of on a per-project basis using a separate manager that vendors them like npm or ruby’s bundler.
What are your needs? Why do you need all of these requirements?
A product, even, open-source, like this, would have a massive testing matrix and footprint.
Another “closest you’ll get” is the JVM ecosystem. It’s cross-platform and can be environment-oriented (e.g. tell maven, gradle, or sbt and friends where to put libraries), but language agnosticism is limited to JVM languages (Java, Scala, Kotlin, Clojure, Jython, JRuby, Freige (Haskell), Javascript, Groovy, etc.) and languages that can expose internals via FFI and JNI (C, C++, Rust easily; Go, Nim probably).
Well, ZeroInstall seems to meet the requirements. We’ll see if it works in a practice test.
I’m willing to relax on convenience, for example. Pip/Conda are probably easier for Python packages, but the advantage to also handle C++, Perl, Binaries with a single tool might outweigh that.
Package managers are more than package formats. Windows is the blocker here, and spack documents it well although I think the same applies to brew, nix and distri, and the inverse would apply to chocolatey.
https://github.com/spack/spack/issues/9323
ZeroInstall looks pretty cool. I can say that I’ve never had a reason to install ZeroInstall, flatpak, or anything like it. I’ve used snappy only to try it out on a Raspberry Pi installation of Ubuntu Server.
Keep in mind though that a user will hesitate to install another package management system, especially if it’s just to install your software. It’ll be a source of friction. It’s not clear what your intended audience is, so that might not really affect you or you may desire that friction to scare off users for whom you’re unprepared to support. ZeroInstall may be adequate for now but if you intend to grow a userbase, it may become a liability or a blocker to adoption unless your use provides adequate momentum to encourage users to install ZeroInstall first.
The more you meet a user in the middle, the more likely they are to use your software. Consider using existing automation that packages your software for distribution in existing formats. nebula os-package or setupbuilder in the Gradle world handle this without much effort for all of your target OSes. Maybe there’s something in your ecosystem that handles this for you.
Ah, I do not care so much about the user but about our development environment here. ;)
The use case is development of embedded products in a large company. Development requires a range of tools where some are Open Source, some internal, some bought. Essentially we write C++, but something like Conan is not fit to deliver simulation or modelling tools for example. Reproducibility is important, so the precise Python, Perl, bash, etc versions are necessary.
Currently, we use a Windows-only home-grown installation tool which cannot even do dependencies or uninstall. We increasingly use Linux for our testing infrastructure and this brings a second configuration tooling. Using the same would be an improvement.
(Using a monorepo is also considered but that is much more disruptive a people would have to change their version control system.)
Yes, I have just written one - If you are interested in trying it you can message me, it is still private. Windows support is wip however. I haven’t ruled it out.
You reminded me of gx. Seems to fit the bill, but I’m not sure how far along they are. It was originally meant for Golang and uses IPFS.
While I can almost certainly say that this isn’t what you actually want, there is an option that checks all of the boxes list. Wasmer.
It is
So Wasmer, or it packaging system wapm, works by compiling into web-assembly application that runs in a runtime with OS independent abstractions. There are actually a surprising number of applications that has a wasmer version including nginx & sqlite.
So about that caveat I said earlier. LLVM can target web-assembly, so most compiled languages are supported. But because of how this works dynamic languages can’t compile to a web-assembly binary. Ruby and python are supported though by basically shipping the interpreter with the application. While this is no prohibitively expensive it does add weight.
pacman, ArchLinux’ package manager, runs on Windows as well. MSYS2 uses it.
Like FlatPack but for other systems than Linux?
Like encrypted filesystems, these seem to be very OS specific.