I assumed using WebP or AVIF for favicons would be completely unsupported in browsers, however, it turns out Firefox 119.0 handles them just fine.

I don't know how widely supported this is, searching for favicon.webp returns very limited results unfortunately. My initial guess is that any browser that supports these image formats overall would support them as favicons, but I doubt this is always true across all browsers.