Description
Intro
Showing images in egui is currently very cumbersome. There is egui_extras::RetainedImage, but it requires you to store the RetainedImage image somewhere and is obviously not very immediate mode.
Ideally we want users to be able to write something like:
ui.image("file://image.png");
ui.image("https://www.example.com/imag.png");We also want 3rd party crates like egui_commonmark to be able to use the same system, without having to implement their own image loading.
Desired features
egui is designed to have minimal dependencies and never doing any system calls, and I'd like to keep it that way. Therefor the solution needs to be some sort of plugin system with callbacks. This is also the best choice for flexibility.
There is three steps to loading an image:
- Getting the image bytes
- file://
- https://
- From a static list of
include_bytes!("my_icon.png") - …
- Decoding the image
- png, jpg, …
- svg rasterization
- …
- Uploading it to a texture
In most cases we want the loaded image to be passed to egui::Context::load_texture, which wil hand the image off to whatever rendering backend egui is hooked up to. This will allow the image loaded to work with any egui integration.
We should also allow users to have more exotic ways of loading images. At Rerun we have a data store where we store images, and it would be great if we could just reference images in that store with e.g. ui.image(rerun://data/store/path);.
In some cases however, the user may want to refer to textures that they themselves have uploaded to the GPU, i.e. return a TextureId::User. We do this at Rerun.
Proposal
I propose we introduce three new traits in egui:
BytesLoaderImageLoaderTextureLoaader
The user can then mix-and-match these as they will.
Users can register them with ctx.add_bytes_loader, ctx.add_image_loader, ctx.add_texture_loader,
and use them with ctx.load_bytes, ctx.load_image, ctx.load_texture.
We will supply good default implementations for these traits in egui_extras. Most users will never need to look at the details of these.
All these traits are designed to be immediate, i.e. callable each frame. It is therefor up to the implementation to cache any results.
They are designed to be able to suppor background loading (e.g. slow downloading of images).
For this they return Pending when something is being loaded. When the loading is done, they are responsible for calling ctx.request_repaint so that the now-loaded image will be shown.
They can also return an error.
Pending will be shown in egui using ui.spinner, and errors with red text.
Common code
enum LoadError {
/// This loader does not support this protocol or image format.
///
/// Try the next loader instead!
NotSupported,
/// A custom error string (e.g. "File not found: foo.png")
Custom(String),
}
/// Given as a hint. Used mostly for rendering SVG:s to a good size.
///
/// All variants will preserve the original aspect ratio.
///
/// Similar to `usvg::FitTo`.
pub enum SizeHint {
/// Keep original size.
Original,
/// Scale to width.
Width(u32),
/// Scale to height.
Height(u32),
/// Scale to size.
Size(u32, u32),
}BytesLoader
// E.g. from file or from the web
trait BytesLoader {
/// Try loading the bytes from the given uri.
///
/// When loading is done, call `ctx.request_repaint` to wake up the ui.
fn load(&self, ctx: &egui::Context, uri: &str) -> Result<BytesPoll, LoadError>;
/// Allow implementations to evict the cache
fn end_frame(&self, frame_index: usize) { }
}
enum BytesPoll {
/// Data is being loaded,
Pending {
/// Set if known (e.g. from a HTTP header, or by parsing the image file header).
size: Option<Vec2u>,
},
/// Bytes are loaded.
Ready {
/// Set if known (e.g. from a HTTP header, or by parsing the image file header).
size: Option<Vec2u>,
/// File contents, e.g. the contents of a `.png`.
bytes: Arc<u8>,
},
}ImageLoader
// Will usually defer to an `Arc<dyn BytesLoader>`, and then decode the result.
trait ImageLoader {
fn load(
&self,
ctx: &egui::Context,
uri: &str,
size_hint: SizeHint,
) -> Result<TexturePoll, TextureLoadError>;
/// Allow implementations to evict the cache
fn end_frame(&self, frame_index: usize) { }
}
trait ImagePoll {
/// Image is loading.
Pending {
/// Set if known (e.g. from parsing the image header).
size: Option<Vec2u>,
},
Ready {
image: epaint::ColorImage, // Yes, only color images for now. Keep it simple.
}
}TextureLoader
// Will usually defer to an `Arc<dyn ImageLoader>`,
// and then just pipe the results to `egui::Context::laod_texture`.
trait TextureLoader {
fn load(
&self,
ctx: &egui::Context,
uri: &str,
texture_options: TextureOptions,
size_hint: FitTo,
) -> Result<TexturePoll, TextureLoadError>;
/// Allow implementations to evict the cache
fn end_frame(&self, frame_index: usize) { }
}
struct SizedTexture {
pub id: TextureId,
pub size: Vec2u,
}
enum TexturePoll {
/// Texture is loading.
Pending {
/// Set if known (e.g. from parsing the image header).
size: Option<Vec2u>,
},
/// Texture is ready.
Ready(SizedTexture),
}Common usage
egui_extras::install_texture_loader(ctx);
ui.image("file://foo.png");Advanced usage:
egui_ctx.add_texture_loader(Box::new(MyTextureLoader::new(…)));
let texture: Result<TexturePoll, _> = egui_ctx.load_texture(uri);
ui.add(egui::Image::from_uri("file://example.svg").fit_to_width());Implementation
For version one, let's ignore cache eviction, and lets parse all images inline (on the main thread). We can always improve this in future PRs.
in egui
We just have the bare traits here, plus:
impl Context {
pub fn add_bytes_loader(&self, loader: Arc<dyn BytesLoader>) { … }
pub fn add_image_loader(&self, loader: Arc<dyn ImageLoader>) { … }
pub fn add_texture_loader(&self, loaded: Arc<dyn TextureLoaader>) { … }
// Uses the above registered loaders:
pub fn load_bytes(&self, uri: &str) {
for loaders in &self.bytes_loaders {
let result = loader.load(uri);
if matches!(result, Err(LoadError::NotSupported)) {
continue; // try next loader
} else {
return result;
}
}
}
pub fn load_image(&self, uri: &str, size_hint: FitTo) { … }
pub fn load_texture(&self, uri: &str, texture_options: TextureOptions, size_hint: FitTo) { … }
}
/// a bytes loader that loads from static sources (`include_bytes`)
struct IncludedBytesLoader {
HashMap<String, &'static [u8]>,
}
impl DefaultTextureLoader { }
impl TextureLoaader for DefaultTextureLoader {
fn load(
&self,
ctx: &egui::Context,
uri: &str,
texture_options: TextureOptions,
size_hint: FitTo,
) -> Result<TexturePoll, TextureLoadError>
{
let img = ctx.load_image(uri, size_hint)?;
match img {
ImagePoll::Pending { size } => Ok(TexturePoll::Pending { size }),
ImagePoll::Ready { image } => Ok(ctx.load_texture(image, texture_options)),
}
}
}in egui_extras
fn install_texture_loader(ctx: &egui::Context) {
#[cfg(not(target_os == "wasm32"))]
ctx.egui_add_bytes_loader(Arc::new(FileLoader::new()));
#[cfg(feature = "ehttp")]
ctx.egui_add_bytes_loader(Arc::new(EhttpLoader::new()));
#[cfg(feature = "image")]
ctx.add_texture_loader(Arc::new(ImageLoader::new()));
#[cfg(feature = "svg")]
ctx.add_texture_loader(Arc::new(SvgLoader::new()));
}
/// Just uses `std::fs::read` directly
#[cfg(not(target_os == "wasm32"))]
struct FileLoader {
cached: HashMap<String, Arc<[u8]>,
}
/// Uses `ehttp` to fetch images from the web in background threads/tasks.
#[cfg(feature = "ehttp")]
struct EhttpLoader {
cached: HashMap<String, poll_promise::Promise<Result<Arc<[u8]>>>>
}
#[cfg(feature = "image")]
struct ImageCrateLoader {
cached: HashMap<String, ColoredImage>, // loaded parsed with `image` crate
}
impl ImageLoader for ImageCrateLoader { … }
#[cfg(feature = "svg")]
struct SvgLoader {
cached: HashMap<(String, SizeHint), ColoredImage>,
}
impl ImageLoader for SvgLoader { … }Implementation notes
- A lot of the necessary code can be found in https://github.com/lampsitter/egui_commonmark/pull/8/files?diff=split&w=0
- For how to handle SVGs, see
egui_extras::RetainedImage, or the above url