Bevy’s UI is built completely within its ECS system.
We control UI elements by spawning entities with components just like the
rest of our game. The entities representing our UI hold Node
components which
define their layout.
You might think ECS works best when we know the components can be processed independently of each other. In a UI context the components are tightly coupled and usually do a lot of async event handling.
However, by keeping the UI inside of the ECS, we gain some benefits:
- Keeping consistent with the ECS data, no transfers to a virtual world
- Able to build off of Bevy’s other successes
- Do more than draw boxes on a screen
- Avoid problems with the borrow checker and graphs of UI elements
For a deeper discussion about these trade-offs I highly suggest reading Alice’s post. Alice is a maintainer at Bevy and works in CAD software.
Layout in Bevy is powered by the taffy library which is a high performance rust-powered UI layout library.
The bevy_ui
crate uses an implementation of the
Flexbox
and
CSS-Grid
layout models. If you are familiar with web development, you will feel right at
home.
A hierarchy of these Node
components are passed to a special render
pipeline that is independent of our Camera
viewport. So our UI stays put when
we move the camera when we change what we are looking at.
Currently, these Node
elements can be cumbersome. To create rich interactive
UI’s there are many supporting third party libraries to make your life easier:
State of the art
The UI libraries are in a constant state of flux. In August, 2023 Cart posted about the direction he wants to take Bevy’s UI.
As of today, if your game requires significant UI design, Alice recommends starting with a newer library: sickle_ui.
It’s a component library built on top of bevy_ui
which should make migration
in the future easier while giving you a reduction in boilerplate today.
As for Cart’s direction it seems the near-term future work on the UI will be focused on adding abstractions and reducing boilerplate.
Drawing a box
To draw a box on the screen as part of our UI all we need to do
is use a NodeBundle
with a Style
component that determines how to lay
itself out within particular Window
.
Bevy calculates the sizes and layout of these components using a representation of your elements mirrored from the taffy crate.
To represent the hierarchy of our nodes, like in HTML pages, we use the
Parent
and Children
components provided by bevy_hierarchy
:
fn spawn_box(mut commands: Commands) {
let container = NodeBundle {
style: Style {
width: Val::Percent(100.0),
height: Val::Percent(100.0),
justify_content: JustifyContent::Center,
..default()
},
..default()
};
let square = NodeBundle {
style: Style {
width: Val::Px(200.),
border: UiRect::all(Val::Px(2.)),
..default()
},
background_color: Color::rgb(0.65, 0.65, 0.65).into(),
..default()
};
let parent = commands.spawn(container).id();
let child = commands.spawn(square).id();
commands.entity(parent).push_children(&[child]);
}
A quick aside, you may see other examples that nest components using with_child
and closures. I personally recommend you keep your nodes flat and maintain
build your hierarchy separately like in the example above.
We spawned an outer box and our child was placed inside it according to the layout rules of the parent.
All Children
of a node will set their Transform
to be relative to their
Parent
, so the NodeBundle
we spawned as a child will be placed in the center
of its parent.
Even though your nodes have Transform
and GlobalTransform
you should never
modify them directly. Always set your nodes position through the layout defined
in its Style
.
Displaying some text
Text can be rendered in two separate ways:
- As part of our game with
Text2dBundle
- As part of our UI with
TextBundle
When text is part of our game world then the text will be rendered according to
our Camera
rather than independently as part of our UI.
So to spawn text within our UI we use TextBundle
:
fn spawn_text(mut commands: Commands) {
let text = "Hello world!";
commands.spawn(
TextBundle::from_section(
text,
TextStyle {
font_size: 100.0,
color: Color::WHITE,
..default()
},
) // Set the alignment of the Text
.with_text_justify(JustifyText::Center)
// Set the style of the TextBundle itself.
.with_style(Style {
position_type: PositionType::Absolute,
bottom: Val::Px(5.0),
right: Val::Px(5.0),
..default()
})
);
}
A TextBundle
is just another NodeBundle
with some additional information
about the text based properties such as alignment.
If we had spawned this text as part of our game with a Text2dBundle
then when
we move our Camera
around the game world the text would position itself like
other entities and begin moving as well.
What should you choose? Depends on your use case:
Use case | Decision | Reason |
Floating combat text | Text2dBundle | This text would be relative to our entities |
Health meter | TextBundle | We want this in a fixed position regardless of our viewport |
Title screen | Either | Would depend on the effect you are going for |
Reacting to a button press
Adding interactivity happens through an Interaction
component. We simply query
using a Changed<Interaction>
filter.
First we need to use ButtonBundle
to create a button:
const NORMAL_BUTTON: Color = Color::rgb(0.15, 0.15, 0.15);
fn spawn_button(mut commands: Commands) {
let container_node = NodeBundle {
style: Style {
width: Val::Percent(100.0),
height: Val::Percent(100.0),
align_items: AlignItems::Center,
justify_content: JustifyContent::Center,
..default()
},
..default()
};
let button_node = ButtonBundle {
style: Style {
width: Val::Px(150.0),
height: Val::Px(65.0),
border: UiRect::all(Val::Px(5.0)),
// horizontally center child text
justify_content: JustifyContent::Center,
// vertically center child text
align_items: AlignItems::Center,
..default()
},
border_color: BorderColor(Color::BLACK),
background_color: NORMAL_BUTTON.into(),
..default()
};
let button_text_node = TextBundle::from_section(
"Button",
TextStyle {
font_size: 40.0,
color: Color::rgb(0.9, 0.9, 0.9),
..default()
},
);
let container = commands.spawn(container_node).id();
let button = commands.spawn(button_node).id();
let button_text = commands.spawn(button_text_node).id();
commands.entity(button).push_children(&[button_text]);
commands.entity(container).push_children(&[button]);
}
Then we can create a system that reacts to any change in the Interaction
component which got spawned in our ButtonBundle
:
fn button_system(
mut interaction_query: Query<
(
&Interaction,
&mut BackgroundColor,
&mut BorderColor,
&Children,
),
(
Changed<Interaction>,
With<Button>
),
>,
mut text_query: Query<&mut Text>,
) {
for (
interaction,
mut color,
mut border_color,
children
) in &mut interaction_query {
let mut text = text_query.get_mut(children[0]).unwrap();
match *interaction {
Interaction::Pressed => {
text.sections[0].value = "Press".to_string();
*color = PRESSED_BUTTON.into();
border_color.0 = Color::RED;
}
Interaction::Hovered => {
text.sections[0].value = "Hover".to_string();
*color = HOVERED_BUTTON.into();
border_color.0 = Color::WHITE;
}
Interaction::None => {
text.sections[0].value = "Button".to_string();
*color = NORMAL_BUTTON.into();
border_color.0 = Color::BLACK;
}
}
}
}
Depending on whether we hover, press or do nothing with the button, the background color and text will change.
Nodes
A Node
is a component that describes the size of the UI component, but not its
layout.
When we spawn a Node
, usually through a NodeBundle
, we can add a Style
component that hold information about its layout:
// https://github.com/bevyengine/bevy/blob/a830530be4398fb0832992e5b4baefc316d1f1d0/crates/bevy_ui/src/ui_node.rs#L128
pub struct Style {
display: Display,
position_type: PositionType,
overflow: Overflow,
direction: Direction,
left: Val,
right: Val,
top: Val,
bottom: Val,
width: Val,
height: Val,
min_width: Val,
min_height: Val,
max_width: Val,
max_height: Val,
aspect_ratio: None,
align_items: AlignItems,
justify_items: JustifyItems,
align_self: AlignSelf,
justify_self: JustifySelf,
align_content: AlignContent,
justify_content: JustifyContent,
margin: UiRect,
padding: UiRect,
border: UiRect,
flex_direction: FlexDirection,
flex_wrap: FlexWrap,
flex_grow: f32,
flex_shrink: f32,
flex_basis: Val,
row_gap: Val,
column_gap: Val,
grid_auto_flow: GridAutoFlow,
grid_template_rows: Vec,
grid_template_columns: Vec,
grid_auto_rows: Vec,
grid_auto_columns: Vec,
grid_column: GridPlacement,
grid_row: GridPlacement,
};
It holds all the normal CSS styles we use to automatically lay them out relative to each other.
The Val
items are an enum representing the different measurements we can use
for our value:
Type | Description |
Val::Auto |
Automatically determine the value based on the context and other properties |
Val::Px(f32) |
Pixel based values |
Val::Percent(f32) |
Percentage based along a specific axis which depends on other fields |
Val::Vw(f32) |
Percentage of the viewport width |
Val::Vh(f32) |
Percentage of the viewport height |
Val::VMin(f32) |
Percentage of the viewports smaller dimension |
Val::VMax(f32) |
Percentage of the viewports larger dimension |
To easily spawn nodes we would use a NodeBundle
:
// https://github.com/bevyengine/bevy/blob/0d23d71c19c784ceb1acfbb134dda9ce0c2adc61/crates/bevy_ui/src/node_bundles.rs#L75
#[derive(Bundle)]
pub struct NodeBundle {
// Describes the logical size of the node
pub node: Node,
// Styles which control the layout (size and position) of the node and it's children
// In some cases these styles also affect how the node drawn/painted.
pub style: Style,
// The background color, which serves as a "fill" for this node
pub background_color: BackgroundColor,
// The color of the Node's border
pub border_color: BorderColor,
// Whether this node should block interaction with lower nodes
pub focus_policy: FocusPolicy,
// The transform of the node
//
// This field is automatically managed by the UI layout system.
// To alter the position of the `NodeBundle`, use the properties of the [`Style`] component.
pub transform: Transform,
// The global transform of the node
//
// This field is automatically managed by the UI layout system.
// To alter the position of the `NodeBundle`, use the properties of the [`Style`] component.
pub global_transform: GlobalTransform,
// Describes the visibility properties of the node
pub visibility: Visibility,
// Algorithmically-computed indication of whether an entity is visible and should be extracted for rendering
pub computed_visibility: ComputedVisibility,
// Indicates the depth at which the node should appear in the UI
pub z_index: ZIndex,
}
Margin, padding and borders is implemented using UiRect
.
Currently a root Node
with no Parent
will lay itself out full width, but
this behaviour will change after 0.12
to require a width: Val::Percent(100.0)
.
Rendering order
The UI system happens in a set of defined stages:
- Layout: Where we update the layout state of the UI
- Focus: Where we handle input interactions with UI entities
- Stack: Where we create the
UiStack
component for ordering UI nodes by their depth
The UiStack
orders the UI nodes so that we can have stacking windows. The
first entry is the furthest node and the first to get rendered on the screen.
However the first node is also the last to receive any interactions so its actually the final node that would be interacted with.
// https://github.com/bevyengine/bevy/blob/0d23d71c19c784ceb1acfbb134dda9ce0c2adc61/crates/bevy_ui/src/stack.rs#L13
// The first entry is the furthest node from the camera
// and is the first one to get rendered, while the last
// entry is the first node to receive interactions.
#[derive(Debug, Resource, Default)]
pub struct UiStack {
// List of UI nodes ordered from back-to-front
pub uinodes: Vec<Entity>,
}
We generate this stack by iterating over the ZIndex
components of our Node
components.
Nodes are either added to either the parent or global contexts depending on if
the z-index is ZIndex::Global
. If it is we add it to the root context.
So what does a raw node look like in the ECS?
// Spawn a node in the global context with a z-index of 1:
commands.spawn(Node::default(), ZIndex::Global(1))
// Spawn a node relative to its parent in the local context
// with a z-index of 2:
commands.spawn(Node::default(), ZIndex::Local(2))
Nodes are placed using the Parent
and Child
node markings which lets
us link entities together in a parent <-> child relationship.
Relative Cursor Position
The Transform
of a Node
is not directly related to its actual position on
screen. To manually our cursors relative position of the Node
on screen we
would also need the Window
, the GlobalTransform
, and the CalculatedClip
.
To help make this easier, every Node
we spawn will have
a RelativeCursorPosition
component added to the same entity.
The RelativeCursorPosition
component stores the cursor position relative to
our node. If it is within the range of (0., 0.)
to (1., 1.)
then the cursor
is currently over the node.
If the cursor position is unknown (e.g we are alt+tabbed out of our game) then
the position will be None
.
We can query this component in our systems to get this info:
use bevy::ui::RelativeCursorPosition;
// This systems polls the relative cursor position
// and displays its value in a text component.
fn relative_cursor_position(
cursor_query: Query<&RelativeCursorPosition>,
) {
let cursor = cursor_query.single();
// This is an `Option<Vec2>` as an unknown cursor position
// will return `None`
if let Some(cursor) = cursor.normalized {
info!(
"({:.1}, {:.1})",
cursor.x, cursor.y
)
}
}
Interaction
Interaction is done by the ui_focus_system
which finds the cursors position
from any Camera
that has a UiCameraConfig
and updates Interaction
components onto UI nodes that have them.
// https://github.com/bevyengine/bevy/blob/0d23d71c19c784ceb1acfbb134dda9ce0c2adc61/crates/bevy_ui/src/focus.rs#L37
#[derive(Component)]
pub enum Interaction {
// The node has been pressed.
// Note: This does not capture click/press-release action.
Pressed,
// The node has been hovered over
Hovered,
// Nothing has happened
None,
}
The note on Interaction::Pressed
above means that it is activated when the
button is pushed down, not when it is released.
When updating these Interaction
components the ui_focus_system
iterates them
in a way where nodes that are on top capture the interaction and the nodes
underneath are set to Interaction::None
.
The UiSurface
manages this interaction between Bevy and taffy, acting as a
public interface for changing the internal representation of the UI that taffy
stores.
During rendering clippings are calculated and CalculatedClip
components are
updated.
Read more
- https://www.leafwing-studios.com/blog/ecs-gui-framework/
- https://github.com/bevyengine/bevy/blob/main/crates/bevy_ui/src/lib.rs
- https://github.com/bevyengine/bevy/discussions/8941
- https://www.gamedev.net/forums/topic/680812-ecs-gui-systems/
- https://github.com/bevyengine/bevy/blob/main/examples/ui/button.rs