Bevy UI
Bevy's UI is built completely within its ECS system.
We create UI elements by spawning entities with certain components that the UI part of the renderer cares about. The most important of these components being the Node
that controls a specific element's layout.
Now, you might be thinking:
ECS works best when we know the components can be processed independently of each other. In a UI context the components are tightly coupled and usually do a lot of async event handling.
However, by keeping the UI inside of the ECS, we stand to gain some benefits:
- Keeping consistent with the ECS data, no transfers to a virtual world
- Able to build off of Bevy's other successes
- Do more than draw boxes on a screen
- Avoid problems with the borrow checker and graphs of UI elements
For a deeper discussion about these trade-offs I highly suggest reading Alice's post. Alice is the lead maintainer at Bevy.
Layout in Bevy is powered by the taffy library which is a high performance rust-powered UI layout library.
The bevy_ui
crate uses an implementation of the Flexbox and CSS-Grid layout models. If you are familiar with web development, you will feel right at home.
A hierarchy of Node
components are passed to a special render pipeline that is usually independent of our Camera
viewport. So our UI stays put when we move the camera when we change what we are looking at.
Whether or not your UI follows the Camera
can be controlled by adding a UiTargetCamera
component to your Node
based entity.
Currently, these Node
elements can be cumbersome. To create rich interactive UI's there are many supporting third party libraries to make your life easier:
State of the art
The UI libraries are in a constant state of flux. In August, 2023 Cart posted about the direction he wants to take Bevy's UI.
Bevy has been moving towards some kind of reactive framework for their UI and so while that settles its kind of the wild west.
Drawing boxes
To draw a box on the screen as part of our UI all we need to do is use a Node
component that determines how to lay itself out within particular Window
.
The basic mental model you should have is this:
- A
Window
represents a space on your monitor - A
Camera
represents a region on aWindow
that it has a right to draw on - A
Node
represents a layout on theWindow
, on top of anyCamera
region
This is obviously not the full story. You can track the UI to a specific camera using the UiTargetCamera
component for example. But it is a solid model for the default state of things.
Bevy calculates the sizes and layout of these components using a representation of your elements mirrored from the taffy crate.
Nodes are components that hold layout information. Nodes that are children of other nodes lay themselves out relative to their parent. To represent the hierarchy of our nodes, Bevy uses ChildOf
and Children
components provided by bevy_hierarchy
:
fn spawn_box(mut commands: Commands) {
let container = Node {
width: Val::Percent(100.0),
height: Val::Percent(100.0),
justify_content: JustifyContent::Center,
..default()
};
let square = (
BackgroundColor(Color::srgb(0.65, 0.65, 0.65)),
Node {
width: Val::Px(200.),
border: UiRect::all(Val::Px(2.)),
..default()
},
);
commands.spawn((container, children![(square)]));
}
We spawned an outer box and our child was placed inside it according to the layout rules of the parent.
All Children
of a node will set their Transform
to be relative to their parent, so the Node
we spawned as a child will be placed in the center of its parent.
Even though your nodes have Transform
and GlobalTransform
you should never modify them directly. Always set your nodes position through the layout and style properties inside the Node
.
Displaying some text
Text can be rendered in two separate ways:
- As part of our game with
Text2d
- As part of our UI with
Text
So, to spawn text within our UI we use Text
:
fn spawn_text_in_ui(mut commands: Commands, assets: Res<AssetServer>) {
commands.spawn((
Node {
position_type: PositionType::Absolute,
bottom: Val::Px(5.0),
right: Val::Px(5.0),
..default()
},
Text::new("Here is some text"),
TextColor(Color::BLACK),
TextLayout::new_with_justify(JustifyText::Center),
));
}
A Text
component requires Node
, as well as some additional text property components like alignment.
If we had spawned this text as part of our game with a Text2d
then when we move our Camera
around the game world.
Color
All colors are created from a certain color space.
Each type of color space has its own dedicated struct. For example your standard RGBA looks like this:
struct Srgba {
red: f32,
green: f32,
blue: f32,
alpha: f32,
}
A Color
is a generic enum over all different color spaces, each with a separate implementation.
We can easily convert from one type to another using the From
and Into
traits:
let color: Hsla = Srgba::rgb(1.0, 0.0, 1.0).into();
Its very common to use the helper methods on the Color
enum:
let black = Color::srgb(0.0, 0.0, 0.0);
let red = Color::hsv(0., 1., 1.)
If we want to convert from one color to another we need to convert our color into the desired space and then perform our action before converting back:
let black = Color::srgb(0.0, 0.0, 0.0);
let srgba = Srgba {
blue: 1.0,
..Srgba::from(black),
};
let blue = Color::from(srgba);
To change the transparency we can use the color methods:
Color::set_alpha
Color::with_alpha
Color::alpha
Color::is_fully_transparent
There are a convenient set of color constants available inside the palettes::css
namespace:
use bevy::color::palettes::css::{BLACK, BLUE, WHITE};
These can be very useful for prototyping out your game without worrying about the specific color spaces.
Reacting to a button press
Adding interactivity happens through an Interaction
component. We simply query using a Changed<Interaction>
filter.
First we need to create a button:
const NORMAL_BUTTON: Color = Color::srgb(0.15, 0.15, 0.15);
fn spawn_button(mut commands: Commands) {
let container = Node {
width: Val::Percent(100.0),
height: Val::Percent(100.0),
align_items: AlignItems::Center,
justify_content: JustifyContent::Center,
..default()
};
let button = (
BorderColor(Color::BLACK),
BackgroundColor(NORMAL_BUTTON),
Node {
width: Val::Px(150.0),
height: Val::Px(65.0),
border: UiRect::all(Val::Px(5.0)),
justify_content: JustifyContent::Center,
align_items: AlignItems::Center,
..default()
},
children![(
Text::new("Button"),
TextColor(Color::srgb(0.9, 0.9, 0.9)),
TextFont::default().with_font_size(40.0),
)],
);
let button_entity = commands.spawn(button).id();
commands
.spawn(container)
.add_children(&[button_entity]);
}
Then we can create a system that reacts to any change in the Interaction
component which got spawned in our previous system:
use bevy::color::palettes::css::{BLACK, BLUE, WHITE};
fn button_system(
mut interactions: Query<
(
&Interaction,
&mut BackgroundColor,
&mut BorderColor,
&Children,
),
(Changed<Interaction>, With<Button>),
>,
mut texts: Query<&mut Text>,
) {
for (interaction, mut color, mut border_color, children) in &mut interactions
{
if let Ok(mut text) = texts.get_mut(children[0]) {
match *interaction {
Interaction::Pressed => {
text.0 = "Press".to_string();
*color = PRESSED_BUTTON.into();
border_color.0 = BLUE.into();
}
Interaction::Hovered => {
text.0 = "Hover".to_string();
*color = HOVERED_BUTTON.into();
border_color.0 = WHITE.into();
}
Interaction::None => {
text.0 = "Button".to_string();
*color = NORMAL_BUTTON.into();
border_color.0 = BLACK.into();
}
}
}
}
}
Depending on whether we hover, press or do nothing with the button, the background color and text will change.
Nodes
A Node
is a component that holds the layout and style properties.
Nodes are laid out with either a flexbox or CSS grid layout.
A Node
has a required component: ComputedNode
which contains the actually calculated size and position of your node. This is used by the renderer to actually lay the node out on the screen.
This is what the default style of a Node
looks like:
// https://github.com/bevyengine/bevy/blob/release-0.16.1/crates/bevy_ui/src/ui_node.rs#L625
impl Node {
pub const DEFAULT: Self = Self {
display: Display::DEFAULT,
box_sizing: BoxSizing::DEFAULT,
position_type: PositionType::DEFAULT,
left: Val::Auto,
right: Val::Auto,
top: Val::Auto,
bottom: Val::Auto,
flex_direction: FlexDirection::DEFAULT,
flex_wrap: FlexWrap::DEFAULT,
align_items: AlignItems::DEFAULT,
justify_items: JustifyItems::DEFAULT,
align_self: AlignSelf::DEFAULT,
justify_self: JustifySelf::DEFAULT,
align_content: AlignContent::DEFAULT,
justify_content: JustifyContent::DEFAULT,
margin: UiRect::DEFAULT,
padding: UiRect::DEFAULT,
border: UiRect::DEFAULT,
flex_grow: 0.0,
flex_shrink: 1.0,
flex_basis: Val::Auto,
width: Val::Auto,
height: Val::Auto,
min_width: Val::Auto,
min_height: Val::Auto,
max_width: Val::Auto,
max_height: Val::Auto,
aspect_ratio: None,
overflow: Overflow::DEFAULT,
overflow_clip_margin: OverflowClipMargin::DEFAULT,
row_gap: Val::ZERO,
column_gap: Val::ZERO,
grid_auto_flow: GridAutoFlow::DEFAULT,
grid_template_rows: Vec::new(),
grid_template_columns: Vec::new(),
grid_auto_rows: Vec::new(),
grid_auto_columns: Vec::new(),
grid_column: GridPlacement::DEFAULT,
grid_row: GridPlacement::DEFAULT,
};
}
It holds all the normal CSS styles we use to automatically lay them out relative to each other.
The Val
items are an enum representing the different measurements we can use for our value:
Type | Description |
---|---|
Val::Auto | Automatically determine the value based on the context and other properties |
Val::Px(f32) | Pixel based values |
Val::Percent(f32) | Percentage based along a specific axis which depends on other fields |
Val::Vw(f32) | Percentage of the viewport width |
Val::Vh(f32) | Percentage of the viewport height |
Val::VMin(f32) | Percentage of the viewports smaller dimension |
Val::VMax(f32) | Percentage of the viewports larger dimension |
All of these values are affected by the UiScale
resource. It will scale these Val
based values by the multiplier you set.
Margin, padding and borders is implemented using UiRect
.
Rendering order
The UI system happens in a set of defined stages:
- Layout: Where we update the layout state of the UI
- Focus: Where we handle input interactions with UI entities
- Stack: Where we create the
UiStack
component for ordering UI nodes by their depth
The UiStack
orders the UI nodes so that we can have stacking windows. The first entry is the furthest node and the first to get rendered on the screen.
However the first node is also the last to receive any interactions so its actually the final node that would be interacted with.
// https://github.com/bevyengine/bevy/blob/release-0.16.1/crates/bevy_ui/src/stack.rs#L16
// The first entry is the furthest node from the camera
// and is the first one to get rendered, while the last
// entry is the first node to receive interactions.
#[derive(Debug, Resource, Default)]
pub struct UiStack {
// List of UI nodes ordered from back-to-front
pub uinodes: Vec<Entity>,
}
We generate this stack by iterating over the ZIndex
components of our Node
components.
Nodes are added to either the parent or global contexts depending on if the z-index is GlobalZIndex
or just the local ZIndex
. If it is we add it to the root context.
Relative Cursor Position
The Transform
of a Node
is not directly related to its actual position on screen. To manually calculate our cursors relative position of the Node
on screen we would also need the Window
, the GlobalTransform
, and the CalculatedClip
.
To help make this easier, we can spawn a RelativeCursorPosition
component on each entity we care about that has a Node
component. Bevy will keep this updated automatically.
The RelativeCursorPosition
component stores the cursor position relative to our node. If it is within the range of (0., 0.)
to (1., 1.)
then the cursor is currently over the node.
If the cursor position is unknown (e.g we are alt+tabbed out of our game) then the position will be None
.
We can query this component in our systems to get this info:
use bevy::ui::RelativeCursorPosition;
// This systems polls the relative cursor position
// and displays its value in a text component.
fn relative_cursor_position(cursor_query: Query<&RelativeCursorPosition>) {
if let Ok(cursor) = cursor_query.single() {
// This is an `Option<Vec2>` as an unknown cursor position
// will return `None`
if let Some(cursor) = cursor.normalized {
info!("({:.1}, {:.1})", cursor.x, cursor.y)
}
}
}
Interaction
Interaction is done by the ui_focus_system
which finds the cursors position from any Camera
that has a UiCameraConfig
and updates Interaction
components onto UI nodes that have them.
// https://github.com/bevyengine/bevy/blob/release-0.16.1/crates/bevy_ui/src/focus.rs#L49
#[derive(Component)]
pub enum Interaction {
// The node has been pressed.
// Note: This does not capture click/press-release action.
Pressed,
// The node has been hovered over
Hovered,
// Nothing has happened
None,
}
The note on Interaction::Pressed
above means that it is activated when the button is pushed down, not when it is released.
When updating these Interaction
components the ui_focus_system
iterates them in a way where nodes that are on top capture the interaction and the nodes underneath are set to Interaction::None
.
The UiSurface
manages this interaction between Bevy and taffy, acting as a public interface for changing the internal representation of the UI that taffy stores.
During rendering clippings are calculated and CalculatedClip
components are updated.