Files
Oxide/docs/ui.md
T
Homer Simpson 9eead719b0 Import Oxide engine (Stages 0–10) under MIT license
Full project snapshot migrated to new Gitea remote without history:
engine, editor, physics, script, examples, tests, docs, and assets.
Relicensed from GPLv3 to MIT and updated repo URLs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 20:41:02 +02:00

861 lines
38 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# UI System
The `oxide_engine::ui` module is the engine's **in-game** UI system — what an
exported game uses to draw menus, HUDs, and tools. It is intentionally
separate from the editor's `egui` (which stays editor-only): a shipped game
cannot link `egui`, so the runtime owns its own widget tree, lays it out,
batches it through the Stage-5 render pipeline, and routes input through the
Stage-7 model.
Stage 8 ships in pieces. This document covers what is live today and tells
you where the rest is going.
## What's live today
- **Piece 1 — widget tree + layout** (data structures, three layout modes,
pure-logic layout function). See [below](#whats-in-piece-1--widget-tree--layout).
- **Piece 2 — styling & theming** (per-widget visual overrides, named-style
themes, RON cascade). See [below](#whats-in-piece-2--styling--theming).
- **Piece 3 — text shaping & glyph atlas** (TTF loading via `ab_glyph`,
shelf-packed R8 atlas, multi-font line wrapping with alignment + DPI
scaling). See [below](#whats-in-piece-3--text-shaping--glyph-atlas).
- **Piece 4a — screen-space overlay render pass** (`paint` turns a laid-out
tree into draw commands; `UiOverlayPass` batches them through wgpu with one
R8 atlas and one alpha-blended pipeline). See [below](#whats-in-piece-4a--screen-space-overlay-render-pass).
- **Piece 4b — world-space UI panels** (`UiPanel` carries a `Widget` tree +
pixel/world sizes; `UiBatch::world_space(...)` composes the MVP that
places the UI on a 3D quad through a perspective camera). See [below](#whats-in-piece-4b--world-space-ui-panels).
- **Piece 5 — input routing** (`Router` walks the `LayoutTree` against the
Stage-7 `InputState`, tracks hover / press / focus per widget, and emits
events plus capture flags the host uses to decide whether the game also
receives the input). See [below](#whats-in-piece-5--input-routing).
- **Piece 6 — events + data binding** (immediate-mode queries on
`RouterFrame``clicked_left("play")` etc. — plus typed `WidgetValue`s
on the tree so game state and widget state round-trip each frame). See
[below](#whats-in-piece-6--events--data-binding).
- **Piece 7 — `examples/ui_menu`** (runnable main menu + settings panel
built entirely from the Stage-8 stack: themed buttons, a draggable
volume slider, a clickable invert-Y checkbox, Back/Quit navigation).
Run with `cargo run -p oxide-examples --bin ui_menu`.
- **Piece 8 — `examples/ui_hud`** (a game HUD composited on top of a live
3D scene: the Stage-4 `ForwardPass` renders the spinning cube/sphere/
plane, then a screen-space `UiOverlayPass` draws corner-anchored HP/Ammo
chips, a minimap stand-in with an orbiting dot, and a centre crosshair —
with animated digits that demonstrate the glyph-atlas cache reaching
steady state). Run with `cargo run -p oxide-examples --bin ui_hud`. See
[below](#whats-in-piece-8--examplesui_hud).
- **Editor UI canvas** (Stage 8.5 piece 7) — the editor's **UI Canvas** panel
authors a `UiPanel` document visually: a widget-tree view (positional
[`WidgetPath`](../engine/src/ui/widget.rs) addressing), an Add palette
(Leaf/Row/Column/Grid/Anchor), a scaled live preview, and a property panel
(id, text, colors, font size, **font-asset picker**, layout sizing). Edits are
undoable and the document saves as a `ui/` asset — the same RON the runtime
loads. The picker writes [`VisualStyle::font_asset`](../engine/src/ui/visual.rs),
resolved through the [asset database](assets.md).
## What's in piece 1 — widget tree + layout
Piece 1 is pure-logic: data structures + a deterministic layout function. No
GPU, no input, no async. Every test runs headlessly.
- **`Widget`** — one node in a tree. Holds an [`id`](#widget-ids), a
[`LayoutStyle`](#layoutstyle), and a [`WidgetKind`](#widgetkinds).
- **`WidgetKind`** — what the node is:
- `Leaf { intrinsic: Vec2 }` — childless node sized by an intrinsic logical
extent. Interactive widgets (label, button, image, slider, …) layer on
top of this in later pieces.
- `Stack(Stack)` — row or column container with a per-stack `gap`,
`direction`, and `main_align`.
- `Grid(Grid)` — equal-cell `cols × rows` container with a `gap: Vec2`.
- `Anchor(AnchorGroup)` — container that positions each child via the
**child's** own [`Anchor`](#anchor).
- **`LayoutStyle`** — sizing, padding, margin, alignment, and (for anchor
children) the anchor itself. The same flat struct on every widget.
- **`layout(root, viewport, scale) -> LayoutTree`** — the layout function.
Returns a `LayoutTree` of `LayoutNode`s (one per widget, root at index 0)
with each node's resolved `rect`, `content_rect` (padding-inset), and the
indices of its direct children.
The whole module lives under
[`engine/src/ui/`](../engine/src/ui/) and is re-exported through the engine
prelude under disambiguated names (`UiSizing`, `UiAnchor`, `Widget`, …) so it
doesn't collide with the Stage-1 math types.
## Building a widget tree
The `Widget::row()`, `Widget::column()`, `Widget::grid(cols, rows)`,
`Widget::anchor()`, and `Widget::leaf(intrinsic)` constructors plus the
`with_*` builder methods produce trees declaratively. Builder methods that
only make sense on certain kinds (`with_gap` on a stack, `with_grid_gap` on a
grid, `with_child` on any container) panic with a clear message when called
on the wrong kind — catching author mistakes during construction instead of
producing a silently misshapen UI at layout time.
```rust
use oxide_engine::math::Vec2;
use oxide_engine::ui::{Insets, LayoutStyle, Sizing, Widget};
let toolbar = Widget::row()
.with_id("toolbar")
.with_gap(8.0)
.with_style(LayoutStyle {
width: Sizing::Grow(1.0),
height: Sizing::Fixed(32.0),
padding: Insets::all(4.0),
..Default::default()
})
.with_child(Widget::leaf(Vec2::new(64.0, 24.0)).with_id("file"))
.with_child(Widget::leaf(Vec2::new(64.0, 24.0)).with_id("edit"));
```
## Sizing
`Sizing` controls how a widget asks to be sized along one axis.
| Variant | Behavior |
|---------|----------|
| `Fixed(f32)` | Fixed logical size; multiplied by the layout scale factor. |
| `Grow(f32)` | Take a share of the parent's leftover space, weighted by `f32`. Two siblings with `Grow(1.0)` split evenly; `Grow(2.0)` next to `Grow(1.0)` takes 2/3. A non-positive weight contributes nothing. |
| `FitContent` (default) | Fit the widget's intrinsic content size — leaves use their `intrinsic`, containers use the recursive content extent. |
The defaults of `FitContent × FitContent` are intentional: leaves are sized
by what they contain, containers are sized by what they wrap. A root widget
that wants to **fill the viewport** must opt in with
`Sizing::Grow(_)` on both axes (or set `Fixed` extents) — the layout function
makes no special root case.
## Padding, margin, alignment
- **`padding`** shrinks a widget's `content_rect`, the area inside which
children are arranged. Multiplied by the scale factor.
- **`margin`** reserves space *outside* the widget's rect, so siblings don't
touch it. In a stack, margin is added to the child's main-axis footprint
before grow accounting.
- **`align_horizontal` / `align_vertical`** position a widget within its
parent's slot when the widget's resolved size is **smaller** than the slot.
In a stack, cross-axis alignment lets a short child dock to the top,
middle, or bottom of its row. (The stack-level `main_align` does the
analogous thing on the main axis when there's no `Grow` child to absorb
leftover space.)
## Layout modes
### Stack (`StackDirection::Row` / `Column`)
1. Allocate each child's **main-axis** size:
- `Fixed(v)``v * scale`,
- `FitContent` → recursive intrinsic measurement,
- `Grow(w)` → reserved (zero first), then assigned a share of leftover
space proportional to `w`.
2. **Cross-axis** sizing happens during the child's own `arrange_in_slot`
pass: `Grow` fills the parent's cross extent; the other variants leave
space the child's `align_*` consumes.
3. With no `Grow` child, the stack's `main_align` (Start / Center / End)
positions the children's combined footprint inside the content rect.
### Grid
Equal-cell `cols × rows` layout. Cell size is computed from the parent's
content rect after subtracting `(cols - 1) * gap.x` and `(rows - 1) * gap.y`.
Children fill cells left-to-right, top-to-bottom; extras past `cols * rows`
are ignored. Within a cell the child's own `align_*` and sizing decide how it
positions itself — `Grow` fills the cell, `Fixed`/`FitContent` aligns inside
it.
More flexible grids (auto-sized rows/columns, spans) are a follow-up; the
equal-cell case covers the Stage-7 bindings preferences page and the Stage-8
settings examples.
### Anchor
Each child specifies its own `Anchor` in `LayoutStyle::anchor`. The anchor is
two normalized points in `[0, 1]²` (the anchor rectangle) plus per-corner
offsets in logical pixels:
```text
rect.min = parent.content.min + parent.content.size * anchor.min + offset_min * scale
rect.max = parent.content.min + parent.content.size * anchor.max + offset_max * scale
```
The Unity/Godot convention applies: the anchor is **authoritative**. An
anchor child's `width`, `height`, `margin`, and `align_*` are ignored along
the axes the anchor constrains; padding still applies (it's an
inside-the-rect concern). The
`Anchor::FILL`, `Anchor::TOP`, `Anchor::TOP_LEFT`, `Anchor::BOTTOM_RIGHT`, …
constants cover the common cases, and `Anchor::between(min, max)` +
`with_offsets(min, max)` is the escape hatch.
## DPI
All linear inputs (sizing, padding, margin, gap, anchor offsets) are in
**logical pixels** and multiplied by the `scale` factor passed to
[`layout`]. The widget tree is DPI-independent; the layout call is where the
display's scale factor enters. The same widget tree laid out at `scale=1.0`
inside a 800 × 600 viewport and at `scale=2.0` inside a 1600 × 1200 viewport
produces identically *proportioned* rects, with every dimension doubled —
verified by an integration test.
## Widget ids and lookups
`WidgetId(pub String)` is the author-facing identifier. UI documents ship
their string ids straight through RON (`"play"`, `"volume-slider"`), so a
visual editor, a hand-edited file, and game code all refer to the same
widget. The empty id (`""`) is the default and means "anonymous"; multiple
anonymous widgets are allowed and `LayoutTree::find` rejects lookups by empty
id.
`LayoutTree::find(id)` is a linear scan — fine for the dozens-of-widgets
trees Stage 8 currently targets; a hash-map index can be added if a profile
ever says it's hot.
## RON dual-edit
Every type in the module derives `Serialize + Deserialize` and round-trips
through RON. `Widget::to_ron()` produces the pretty-printed canonical form
the editor's UI canvas saves and the runtime loads; `Widget::from_ron(text)`
parses it. The Stage-8 integration suite verifies that the round-trip
**preserves layout** — the laid-out trees match — so an external editor or AI
agent can edit the same file the runtime loads.
## What's in piece 2 — styling & theming
Visual styling is intentionally **orthogonal** to layout — layout decides
where a widget is; visual styling decides what it looks like. Adding a
`VisualStyle` or `theme_style` to a widget never changes its laid-out rect.
The integration suite verifies this with a paired `layout()` call before and
after styling.
The data:
- **`VisualStyle`** — a flat struct of `Option<T>` fields: `background`,
`foreground`, `border` (color + width), `corner_radius`, `font`, and
`font_size`. `None` means *inherit*; `Some` means *override*. Every field
serializes via `skip_serializing_if = "Option::is_none"`, so an empty
visual style vanishes from RON entirely.
- **`Theme`** — `default: VisualStyle` plus `styles: BTreeMap<String,
VisualStyle>`. The `BTreeMap` (not `HashMap`) gives deterministic RON
output, important for diff-friendly UI documents and reproducible test
snapshots.
- **`Widget`** gains two fields: `visual: VisualStyle` (per-instance
overrides) and `theme_style: Option<String>` (opt-in name into the
theme's named map).
The cascade — implemented by `Theme::resolve(style_ref, override_with)` and
exposed on the widget as `Widget::resolve_visual(&theme)`:
1. Start with `theme.default`.
2. If the widget specifies `theme_style: Some(name)` and the theme has a
matching entry, merge it on top (a missing name is treated as "no
contribution", not an error).
3. Merge the widget's per-instance `visual` on top.
Each merge is field-by-field via `VisualStyle::merged(self, override_with)`:
right-hand `Some` wins, otherwise the left-hand value is kept. The same
primitive will drive runtime state overlays in piece 5 (hover, focus,
press).
`FontRef` carries `family`, `weight: FontWeight`, and `italic: bool`. The
descriptor stores **names**, not paths: portable across machines, and the
runtime (piece 3) is free to pick the platform's best match. `FontWeight`
exposes `opentype_value()` returning the OpenType 100900 weight scale.
`VisualStyle` also has a `font_asset: Option<AssetRef<Font>>` (Stage 8.5 piece
7): a reference to a **specific project font asset** under `assets/fonts/`,
chosen in the editor's UI canvas from the asset browser. When set it takes
precedence over the `font` descriptor — the renderer resolves the [`AssetRef`]
to a loaded face through the [asset database](assets.md) (a default-registered
`FontLoader` makes `.ttf`/`.otf` loadable via the `AssetServer`). `None` falls
back to the descriptor / theme path. This is the engine's first `AssetRef<T>`
field and the asset-picker's end-to-end target.
[`AssetRef`]: ../engine/src/asset/database.rs
### Quick example
```rust
use oxide_engine::math::{Color, Vec2};
use oxide_engine::ui::{Border, FontRef, Theme, VisualStyle, Widget};
let theme = Theme::new()
.with_default(VisualStyle {
foreground: Some(Color::BLACK),
background: Some(Color::WHITE),
font: Some(FontRef::regular("Inter")),
font_size: Some(14.0),
..VisualStyle::EMPTY
})
.with_style(
"button",
VisualStyle {
background: Some(Color::rgb(0.85, 0.85, 0.9)),
border: Some(Border::new(Color::rgb(0.6, 0.6, 0.7), 1.0)),
corner_radius: Some(4.0),
..VisualStyle::EMPTY
},
);
let play = Widget::leaf(Vec2::new(80.0, 24.0))
.with_id("play")
.with_theme_style("button")
.with_visual(VisualStyle {
background: Some(Color::rgb(0.2, 0.4, 0.8)), // primary-button accent
foreground: Some(Color::WHITE),
..VisualStyle::EMPTY
});
let resolved = play.resolve_visual(&theme);
assert_eq!(resolved.foreground, Some(Color::WHITE)); // per-instance wins
assert_eq!(resolved.corner_radius, Some(4.0)); // inherited from "button"
assert_eq!(resolved.font, Some(FontRef::regular("Inter"))); // inherited from default
```
### RON dual-edit
`Theme::to_ron` / `Theme::from_ron` round-trip themes through pretty-printed
RON, matching `Widget::to_ron` from piece 1. `BTreeMap`-ordered output keeps
named styles alphabetised so diffs are stable. Empty fields (`None` options,
empty maps, `FontWeight::Regular`, `italic: false`) skip serializing — the
default form of any of these structs is `()` in RON.
## What's in piece 3 — text shaping & glyph atlas
The text subsystem lives at `oxide_engine::ui::text` and splits into three
sub-modules that compose, but each is testable on its own:
- **`font`** — owns `Font` (a thin wrapper around `ab_glyph::FontVec`),
`FontId`, and `FontStore`. `Font::rasterize(glyph, size_px)` returns a
`RasterizedGlyph` with an alpha mask + per-glyph bearings + advance.
`FontStore::insert_with_descriptor(FontRef, Font)` indexes a font under a
piece-2 `FontRef`, so a theme's `font: Some(FontRef::bold("Inter"))`
resolves to a `FontId` the shaper can use.
- **`atlas`** — `GlyphAtlas::new(width, height)` allocates a single R8
(alpha-only) buffer; `get_or_rasterize(GlyphKey, &FontStore)` returns the
glyph's `AtlasEntry` (UV rect + size + bearing + advance), rasterizing
and packing on first miss and serving the cache forever after. The
packer is a **best-fit shelf packer** — simple, deterministic, and
near-optimal density for the typically-uniform glyph heights of one font
at one size. The `dirty()` flag tells the piece-4 render pass when the
texture needs re-upload.
- **`shape`** — `shape(text, style, &params, &fonts)` turns a string into
a `ShapedText { lines, size }` of positioned `ShapedGlyph`s. Each glyph
carries a `GlyphKey` the renderer feeds back into the atlas, and a
`position` at the **baseline** (not the top-left). Algorithm:
greedy line-break at ASCII whitespace, multi-font runs supported via
`shape_runs(&[TextRun])`, alignment within `max_width` (Left / Center /
Right), DPI scaling via `ShapeParams::scale`.
### The atlas is the cache
`GlyphAtlas` keys entries by `(FontId, GlyphId, size_px rounded to nearest
integer)`. Every glyph is rasterized **exactly once** per (font, glyph,
size) triple — a HUD that repaints `"HP: 1234 / 1500"` every frame
rasterizes the ten ASCII characters one time at startup and then runs
purely on textured quads. The integration suite verifies this:
`shaped_hud_text_is_cached_after_one_frame` shapes a three-line HUD,
walks every glyph through the atlas twice, and asserts the atlas's
`dirty` flag stays false on the second pass — i.e., zero new
rasterizations. The library choice (ab_glyph vs fontdue) only affects
the one-time miss cost, not steady-state.
### Quick example
```no_run
use oxide_engine::math::Vec2;
use oxide_engine::ui::text::{
shape, Font, FontStore, GlyphAtlas, ShapeParams, TextAlign, TextStyle,
};
let mut fonts = FontStore::new();
let id = fonts.insert(Font::from_path("/usr/share/fonts/.../Inter-Regular.ttf").unwrap());
let style = TextStyle { font: id, size_px: 14.0 };
let params = ShapeParams {
max_width: Some(300.0),
align: TextAlign::Center,
line_height: 1.4,
scale: 1.0,
};
let shaped = shape("Press F to pay respects", style, &params, &fonts);
let mut atlas = GlyphAtlas::new(1024, 1024);
for line in &shaped.lines {
for glyph in &line.glyphs {
// Render with the atlas's bearing offset; this is exactly the
// call piece 4's overlay pass will make per glyph per frame.
if let Some(entry) = atlas.get_or_rasterize(glyph.key, &fonts) {
let quad_top_left: Vec2 = glyph.position + entry.bearing;
let _quad_size: Vec2 = entry.size_px;
let _ = (quad_top_left, entry.uv_min, entry.uv_max);
}
}
}
```
### Limitations (deliberate, scoped to piece 3)
- One glyph per `char` — no ligatures, no combining marks, no complex-
script shaping (Arabic, Devanagari, Thai). The data path is ready for
a future `rustybuzz`-shaped intermediate; the current shaper just
doesn't invoke one.
- No BiDi or RTL — text flows left-to-right.
- No hyphenation or character-level break inside an over-wide word.
- ASCII whitespace only (`\t` and `\r` are treated as spaces).
- No bold/italic synthesis — each face is a separately-loaded `Font`.
### Font choice
The engine doesn't bundle a font; piece-3 tests use whichever sans-serif
they find on `/usr/share/fonts/` (or `/System/Library/Fonts` on macOS) via
`common_system_font_paths()`, skipping with `eprintln!("SKIP: …")` when no
candidate is present. The default UI font shipped with examples is a
piece-7 decision.
### Why ab_glyph
`ab_glyph` is a TTF parser + rasterizer only. It does not do layout —
which is fine because the shaper above already owns that. With
`fontdue` we would have gotten line wrapping for free at the cost of
living inside a fixed layout model; with `ab_glyph` we own every line-
break, kerning, and alignment decision. That control buys us a clean
path to richer features later: rich-text markup, per-character
animation, in-canvas editor caret positioning, and **SDF font
rendering** — a future follow-up where each glyph is rasterized once
as a signed-distance field and the shader scales it to any size for
free. SDF is on the Stage-8 backlog in [PLAN.md](../PLAN.md); it would
slot in beside `ab_glyph` without rewriting the shaper.
## What's in piece 4a — screen-space overlay render pass
Piece 4 splits the GPU work into two commits — **4a (screen-space, this
piece)** and **4b (world-space UI panels in 3D)**. Both share one render
pass, one shader, one R8 glyph atlas. The split is purely for review
size; the same `UiOverlayPass` handles both modes via per-batch MVP
matrices.
Two new pieces, both pure-CPU but the second one talks to wgpu:
- **`oxide_engine::ui::paint`** — `paint(&Widget, &LayoutTree, &Theme,
&FontStore, scale) -> PaintedFrame`. Walks the laid-out tree in
parent-then-children order; for each node, resolves the cascaded
[`VisualStyle`](#whats-in-piece-2--styling--theming) under the theme,
emits one `DrawCommand::Quad` if a background was resolved, and shapes
the widget's `text: Option<String>` inside its `content_rect` to emit
one `DrawCommand::Glyph` per laid-out glyph. Pure-logic; tests run
without a GPU and most without a font.
- **`oxide_engine::render::UiOverlayPass`** — implements
[`RenderPass`](render-pipeline.md) and slots into the Stage-5 pipeline
*after* the `ForwardPass`. Consumes `Vec<UiBatch>` per frame; each batch
pairs an MVP matrix with a `PaintedFrame`. For piece 4a the host builds
one batch with `UiBatch::screen_space(painted, target_size)` — an
orthographic projection from window pixels to NDC with y-down (origin at
the top-left).
### Vertex format and shader
One vertex format, one fragment path:
```text
struct UiVertex { position: vec2, uv: vec2, color: vec4 } // 32 bytes
```
The shader (`engine/src/render/shaders/ui.wgsl`) discriminates "solid quad
vs. glyph quad" by a sentinel UV: `uv.x < 0.0` skips the atlas sample. So
a solid red rectangle and a glyph from "Inter" pass through identical
pipeline state and live in the same vertex buffer — no state changes per
primitive, no separate textures. Alpha-blending is on; UI never reads
depth (it overlays).
### Atlas lifecycle
Each frame's `run`:
1. Walk every glyph in every batch, calling
`GlyphAtlas::get_or_rasterize(key, &fonts)` to ensure the entry is
cached. Misses rasterize once; hits do nothing.
2. If the atlas's `dirty` flag is set, re-upload the whole R8 buffer to
the GPU texture and clear the flag. Re-uploading the whole atlas (vs.
tracking dirty sub-rects) keeps the code simple; the buffer is small
(1 MB at 1024×1024) so this is fine. A dirty-region upload is a
straightforward follow-up if a profile says it's hot.
3. For each batch: serialize draw commands into vertices, write the MVP
uniform, set the viewport from `FrameContext::resolved_viewport()`,
and submit one draw call.
### Test strategy
The piece-4 tests live in three places:
- `engine/src/ui/paint.rs` — 5 lib tests verify the CPU paint logic: a
solid widget emits one quad at its rect, a text widget emits one glyph
command per visible char at the same baseline, layered widgets draw
parent-before-child, etc. No GPU required.
- `engine/src/render/ui_pass.rs` — 4 headless GPU pixel-readback tests:
a 20×20 red quad shows red at its centre and clear-color outside; an
empty batch list is a no-op; two quads in one batch both render to
their respective rects; the vertex buffer grows when a batch exceeds
the initial 4096-vertex capacity.
- `tests/src/lib.rs` mod stage8 — 1 integration test runs the whole
pipeline: `Widget` → `layout` → `paint` → `UiOverlayPass::run` →
pixel-readback, then asserts the centred 48×48 red panel is red in the
middle and clear-color in the gutter.
Both lib and integration GPU tests skip with `eprintln!("SKIP: ...")` if
no adapter is available, matching the Stage-4 pattern.
### Wiring it into an app
```rust,no_run
use oxide_engine::math::Color;
use oxide_engine::render::{RenderPipeline, UiBatch, UiOverlayPass};
use oxide_engine::ui::{layout, paint, FontStore, Theme, Widget};
# fn build_pipeline(device: &wgpu::Device, format: wgpu::TextureFormat) {
let mut pipeline = RenderPipeline::forward(device, format);
let ui_pass = UiOverlayPass::new(device, format);
pipeline.add_pass("ui", ui_pass);
# }
# fn each_frame(
# ui_pass: &mut UiOverlayPass,
# document: &Widget,
# theme: &Theme,
# fonts: &FontStore,
# viewport: oxide_engine::math::Rect,
# target_size: (u32, u32),
# ) {
let tree = layout(document, viewport, 1.0);
let painted = paint(document, &tree, theme, fonts, 1.0);
ui_pass.set_batches(vec![UiBatch::screen_space(painted, target_size)]);
// pipeline.render(&mut frame); — at next frame.
# }
```
## What's in piece 4b — world-space UI panels
`oxide_engine::ui::UiPanel` is a pure-data holder: a `Widget` tree plus two
sizes — `pixel_size` (the resolution the UI is laid out at) and
`world_size` (the panel's physical dimensions in world units). It does
*not* own the panel's `Transform`; that lives on the entity that hosts
the panel (eventually a hecs component), so the same panel can be
duplicated across many entities with different placements.
`UiBatch::world_space(painted, pixel_size, world_size, &panel_transform,
view_projection)` composes a single MVP that the existing piece-4a pass
uses unchanged:
```text
mvp = view_projection
* panel_transform // world placement
* scale(world.x / pixel.x, -world.y / pixel.y, 1) // pixels → world (y-flip)
* translate(-pixel.x / 2, -pixel.y / 2, 0) // recentre pixel origin
```
A pixel at `(0, 0)` in the painted frame lands at the panel's top-left
corner in world space; a pixel at `pixel_size` lands at the bottom-right.
Same pipeline, same shader, same atlas — only the MVP differs.
`UiPanel::build_batch(theme, fonts, &panel_transform, view_projection)`
is the convenience that lays out + paints + builds the batch in one call.
Hosts that want finer control compose the same three steps by hand.
### Overlay semantics
World-space panels in piece 4b render as **overlays**: no depth test, no
depth write — they draw on top of whatever's in the colour target. That
keeps the implementation simple and matches the common "always-visible"
use case (player nameplates, mission markers, editor canvas previews).
A future **depth-aware mode** (where a panel behind a wall is properly
hidden) is in PLAN.md's Stage-8 backlog and slots in by attaching the
depth target to a second pass of the same pipeline.
### Tests
- 4 lib tests on `UiPanel`: `build_batch` returns `None` on zero
`pixel_size`, succeeds on a valid panel, the panel round-trips through
RON for dual-edit, and the identity-MVP sanity check maps pixel
`(0, 0)` to world `(-world.x/2, +world.y/2)` (verifying the y-flip).
- 1 GPU pixel-readback lib test (`world_space_panel_renders_inside_its_projected_region`):
a 2 m × 2 m red panel at the origin under a 60° camera 3 m away,
asserts the framebuffer centre is red and corners stay clear.
- 1 integration test (`ui_panel_in_3d_renders_under_perspective_camera`):
exercises the full `UiPanel::build_batch` → `UiOverlayPass` path end
to end with a real perspective camera and a pixel-readback assertion.
### Quick example
```rust,no_run
use oxide_engine::math::{Color, Transform, Vec2, Vec3};
use oxide_engine::render::{Camera, UiBatch, UiOverlayPass};
use oxide_engine::ui::{FontStore, Theme, UiPanel, VisualStyle, Widget};
# fn each_frame(pass: &mut UiOverlayPass, panel: &UiPanel) {
let camera = Camera::perspective(60_f32.to_radians(), 0.1, 100.0);
let view_transform = Transform::looking_at(Vec3::new(0.0, 1.5, 4.0), Vec3::ZERO, Vec3::Y);
let view_projection = camera.view_projection(16.0 / 9.0, &view_transform);
// Where the panel sits in the world. Treat as if it were a Transform
// component on the entity hosting the panel.
let panel_transform = Transform::default();
let theme = Theme::new();
let fonts = FontStore::new();
let batch = panel
.build_batch(&theme, &fonts, &panel_transform, view_projection)
.expect("valid panel");
pass.set_batches(vec![batch]);
// pipeline.render(&mut frame); — at next frame.
# }
```
## What's in piece 5 — input routing
The UI must consume input *before* the game (PLAN.md): clicking a button
shouldn't also fire the game action bound to the same mouse button.
`oxide_engine::ui::routing` gives the host one object that does this
end-to-end:
```rust,no_run
use oxide_engine::prelude::*;
use oxide_engine::ui::Router;
# fn each_frame(router: &mut Router, tree: &UiLayoutTree, input: &InputState) {
let frame = router.process(tree, input);
if !frame.captured_mouse {
// game receives mouse this frame
}
if !frame.captured_keyboard {
// game receives keys this frame
}
for event in &frame.events {
// piece 6 will dispatch each event to the matching widget's callback
}
# }
```
### Hit-test
`hit_test(&LayoutTree, point) -> Option<&LayoutNode>` walks the laid-out
nodes in **reverse order** — the same order as paint (parents-then-
children, earlier siblings before later ones), so the topmost-drawn
widget is the first one tested. Anonymous widgets
(`WidgetId::default()`) are skipped so a decorative container doesn't
block clicks reaching the button inside it.
### State machine
The `Router` persists three pieces of state across frames:
- **hovered** — recomputed each frame from the cursor + hit-test.
- **focused** — set when the cursor presses over a widget; cleared when
the cursor presses outside any widget. Survives subsequent hover
changes so a focused text input keeps focus while the cursor moves.
- **pending presses** — per-button, the widget that received the
most-recent unreleased press. A press → release on the **same**
widget emits `Clicked`. Drag-off then release cancels the click.
### Events
`RouterFrame.events: Vec<RouterEvent>` collects everything that
happened: `Hovered` / `Unhovered`, `Pressed` / `Released` / `Clicked`
(per mouse button), `FocusGained` / `FocusLost`. Piece 6 will dispatch
each event to per-widget callbacks; piece 5 is purely the state machine
producing the event list.
### Tests
- 13 lib tests cover hit-test (topmost wins, anonymous skipped,
outside-root → None, padding gutter resolves to parent), hover/
unhover/swap-on-move, press → focus, press + release on the same
widget → click, drag-off cancels click, press outside clears focus,
captured-flag transitions, and cursor-unset → no hover.
- 1 integration test exercises the full hover → press → release →
click → move → press-outside-loses-focus sequence end-to-end with a
synthetic `InputState`.
All tests are pure-CPU; no GPU, no font, no window.
### What's deliberately not in piece 5
- **Keyboard focus navigation** (Tab / arrow keys to move focus) — a
small follow-up on top of the existing focus state.
- **Per-widget callbacks** — piece 6.
- **World-space hit-test** — clicking through a 3D panel needs a
ray-cast and an inverse-MVP. A follow-up that slots in by adding a
`Router::hit_test_world(ray, &UiPanel, &Transform)` helper.
## What's in piece 6 — events + data binding
Piece 6 takes the **immediate-mode** stance (same as Bevy UI and egui):
no callback storage, no `Rc<RefCell<...>>` for state, no lifetime
gymnastics — the host reads the `RouterFrame` each frame and acts
directly.
### Events: immediate-mode queries on `RouterFrame`
The piece-5 `RouterFrame` already carries the event list. Piece 6 adds
typed query methods that game code calls directly:
```rust,no_run
# fn each_frame(frame: oxide_engine::ui::RouterFrame) {
use oxide_engine::winit::event::MouseButton;
if frame.clicked_left("play") {
// start_game();
}
if frame.clicked("save", MouseButton::Right) {
// open_save_menu();
}
if frame.hovered_in("tooltip-target") {
// show_tooltip();
}
if frame.focus_gained("volume_slider") {
// ...
}
# }
```
The seven query methods — `clicked`, `clicked_left`, `pressed`,
`released`, `hovered_in`, `hovered_out`, `focus_gained`, `focus_lost`
— each take a widget id and (where applicable) a `MouseButton`, and
return `bool`. They scan the frame's event list, so the cost is linear
in the number of events emitted that frame — typically a handful.
### Data binding: `Widget::value: Option<WidgetValue>`
Every widget can carry typed state — a checkbox's bool, a slider's
float, a text input's string — independent of its `kind`. The
`WidgetValue` enum has variants `Bool(bool)` / `Int(i64)` /
`Float(f64)` / `Text(String)`, plus `From`-impls for `bool`, `i32`,
`i64`, `f32`, `f64`, `&str`, and `String`.
Per-widget access uses `Widget::value(&id)` and `Widget::set_value(&id,
v)` — both walk the subtree to find the widget by id:
```rust,no_run
use oxide_engine::ui::{Widget, WidgetValue};
# fn pull_then_push(root: &mut Widget, audio_volume: &mut f32) {
// Pull game state into the widget tree (typically at the start of frame).
root.set_value(&"volume".into(), *audio_volume);
// ... user interacts, slider widget updates its own value ...
// Push the widget tree's value back into game state (at end of frame).
if let Some(v) = root.value(&"volume".into()).and_then(|v| v.as_float()) {
*audio_volume = v as f32;
}
# }
```
For values that don't change between frames (e.g., a label's string),
no binding is needed — set it once.
### Why immediate-mode
The persistent-callback alternative (each widget owns a
`Box<dyn FnMut(...)>`) forces every callback to either:
- own its game state via `Rc<RefCell<...>>` (verbose, costs every
read), or
- borrow game state for `'static` (impossible), or
- defer to a queue (the same shape as immediate-mode, but indirected).
Immediate-mode skips all three: the widget tree is **data**, not a
network of callbacks. The host's main loop is the dispatcher; the
piece-6 queries are just convenient predicates over the event list.
### What's deliberately not in piece 6
- **Typed bindings helper** (`Bindings<T>` that registers per-field
getter/setter pairs and runs them automatically) — adds a `Box<dyn>`
abstraction over what's currently two lines of host code. Will land
alongside piece-7's settings example if the boilerplate becomes
painful.
- **Per-widget keyboard event delivery** (text input handling, hotkey
registration) — needs a focused-widget event-routing pass on top of
the piece-5 focus state. Either piece-7 or a follow-up.
### Tests
- 6 lib tests on `WidgetValue` cover accessor matching, `From`
conversions for every primitive, and RON round-trip for each variant.
- 4 lib tests on `Widget`: `find_by_id` / `find_by_id_mut` walk the
subtree, `set_value` updates a descendant by id, `with_value` builder
works, the value round-trips through `Widget`'s own RON.
- 2 lib tests on `RouterFrame`: query methods return true for matching
events, false for non-matching, across every event variant.
- 1 integration test (`settings_widget_tree_round_trips_game_state_each_frame`):
pulls game state into a settings panel, simulates user interaction +
an Apply click, pushes the widget values back into game state, and
asserts the round-trip is exact.
## What's in piece 8 — `examples/ui_hud`
`examples/src/bin/ui_hud.rs` is the second runnable Stage-8 example and
the first to **composite the UI over a 3D scene**. It reuses the
`hello_mesh` scene (spinning cube + sphere + ground plane through the
Stage-4 `ForwardPass`) and draws a HUD on top with a screen-space
`UiOverlayPass`.
### Compositing two passes on one surface
The window runner clears the surface to the configured clear color
*before* `render`. Both the forward pass and the UI overlay then use
`LoadOp::Load` for their color attachment, so each draws over whatever
is already there:
1. `pipeline.render(&mut frame)` runs the forward pass — 3D geometry
plus its own depth buffer (cleared each call).
2. `ui_pass.run(&mut frame)` runs the overlay — no depth, alpha
blending — so the HUD sits on top of the 3D image.
The host owns the `UiOverlayPass` separately from the `RenderPipeline`
(rather than `add_pass`-ing it) because the overlay needs `set_batches`
mutated every frame and the pipeline consumes pass ownership. Both
passes share the same `FrameContext`, so the example builds the 3D
objects and the painted HUD, then calls the two `run`s back to back.
### Corner anchoring
Each HUD element is an anchor child of a full-screen anchor root. A
corner-pinned, fixed-size widget is expressed as a corner `Anchor`
constant plus offsets that define its box — e.g. a top-left chip is
`Anchor::TOP_LEFT.with_offsets((M, M), (M + W, M + H))`, and a centred
crosshair is `Anchor::between((0.5, 0.5), (0.5, 0.5)).with_offsets(...)`.
The crosshair's two bars are themselves anchor children spanning one
axis and pinned thin on the other.
### Demonstrating the atlas cache
The HP and Ammo values animate every frame (HP oscillates down then up;
Ammo counts down as if firing, reloading at 0). The digits change
constantly, but the glyph atlas only ever rasterizes each character
**once** — after the digits `0``9` and the static label text have been
seen, the atlas stops growing and every later frame is a pure cache hit
(no rasterize, no GPU re-upload). The example logs each atlas growth and
the moment it reaches steady state, via two accessors added to the pass:
```rust
pass.atlas_glyph_count(); // distinct glyphs cached so far
pass.atlas_dirty(); // grew-this-run flag (false in steady state)
```
The `atlas_caches_glyphs_and_reaches_steady_state` GPU test in
`render::ui_pass` proves this property automatically: it draws the ten
digits one per frame (asserting the count grows by one each time), then
re-draws a cached digit and asserts the count holds and the dirty flag
stays clear.
Run it: `cargo run -p oxide-examples --bin ui_hud` (Esc quits).
## What's coming in the rest of Stage 8
- **Piece 9 — Editor UI canvas.** A new editor panel for visually
authoring `Widget` / `UiPanel` documents: a drag-from widget palette,
a canvas showing the document at target size with drag-resize handles,
a property inspector for `LayoutStyle` / `VisualStyle` / `text` /
`value`, RON save/load round-tripping the same format the runtime
loads, and a live preview rendered through the actual `UiOverlayPass`
(not egui). Likely splits into 9a (canvas + palette + inspector) and
9b (live preview + drag/resize handles).
The piece-1 data structures already accommodate the editor canvas: a
document is just a `Widget` tree, the inspector edits the same reflected
style structs the runtime uses, and the preview reuses the exact paint +
overlay pipeline the game ships with.