In the spirit of 'scratch your own itch', I actually got it working myself after (quite some) tinkering. If anyone else bumps into this issue, here's how I got it working on Ubuntu 22.04:

1. Install dependencies for Rust & solo2-cli:
   sudo apt install curl build-essential gcc make pkg-config libudev-dev libpcsclite-dev -y
2. Install Rust
   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
3. Activate Rust path in current shell (or restart your shell)
   source "$HOME/.cargo/env"
4. Install solo2-cli
   cargo install solo2
5. Fix udev rules (running solo2-cli with sudo didn't work for me)

cd /etc/udev/rules.d/
sudo wget https://github.com/solokeys/solo2-cli/raw/main/70-solo2.rules
sudo chmod g-w 70-solo2.rules

6. Insert solo2 key and see if it's detected
   solo2 ls
7. If properly detected, update the firmware (will be downloaded automagically)
   solo2 update
8. Follow instructions as prompted, done.