Skip to content

Airfield

Airfield's own documentation

This page covers how the car's workspace uses airfield. For airfield itself, see its documentation website, airfield.io. Good places to start there are the Overview, the Quick Start and the CLI Reference.

The car's software, the roboracer_ws workspace, is built and launched with airfield. Airfield builds one container image per ROS 2 package, and launches a plan (a named set of packages) as a tmux session with one pane per program.

On an F1TENTH car, the setup script installs airfield and builds everything. This page explains how the workspace works once it's there.

How the workspace is laid out

~/roboracer_ws/                  the project (airfield.yaml at the top)
├── airfield.yaml                project settings: the base image, the list of package repos
├── packages/<name>/             one ROS 2 package, its own git repo -> one container image
│   └── airfield.yaml            its dependencies, devices, groups, build flags
├── dependencies/                how to install each dependency into an image
│   ├── xplatform/               recipes for every platform
│   └── arm64/ | x86_64/         platform-specific recipes, and the Jetson base image
├── plans/<name>.yaml            what to launch: navstack, teleop, car_calibration, ...
├── scripts/                     build / up / down helpers
├── .air                         this machine's settings (not in git)
└── .airfield/workspace/         the compiled code (not in git)
    └── build/ install/ log/

Two unrelated things are called "packages": ~/roboracer_ws/packages/ is the car's ROS code, and ~/.cache/airfield/packages/ is airfield's shared recipe book for installing system software (ROS libraries, OpenCV, ...) into images.

The workspace has to be at ~/roboracer_ws, directly in your home folder, because some of the code refers to that path.

Build once, launch many

Every package's container mounts the same compiled workspace, so the ROS 2 code is compiled once, into install/. Each pane then only sources that and runs its program. Compiling in every pane at once needs far more memory than the Orin's 8 GB, and runs it out of memory.

The compiled workspace has two paths:

  • On the car, it's ~/roboracer_ws/.airfield/workspace/, inside the project. Each project gets its own, so two projects that both have a package with the same name can't pick up each other's binaries.
  • Inside a container, it's always ~/workspace.

AIRFIELD_WORKSPACE moves the car-side folder, if you ever want several projects to share one. See What gets mounted on airfield.io for everything a container receives.

Building

cd ~/roboracer_ws
scripts/build                # everything the navstack plan uses
scripts/build ut_automata    # just this package and what it needs

scripts/build builds each package's image, then compiles the code one package at a time with limited parallelism, so the Orin doesn't run out of memory. The first build is slow, mostly downloading and installing software into the images. Later builds reuse that.

After you edit a package, build it again with scripts/build <package>. Panes only compile packages that are missing from install/, so without a rebuild the change never reaches the car. The usual symptom is a "file not found in the share directory of package" error.

When a pane starts, it checks whether its package is already compiled. If not, it compiles it first, one pane at a time, so launching a plan on a fresh workspace also works, just more slowly. Build flags for a package come from colcon_args: in its airfield.yaml (for example ut_automata's -DCMAKE_BUILD_MODE=Hardware).

To force a clean compile:

cd ~/roboracer_ws
rm -rf .airfield/workspace/build .airfield/workspace/install
scripts/build

Launching and stopping

cd ~/roboracer_ws
scripts/up               # the navstack plan
scripts/up teleop        # any plan in plans/
scripts/down             # stop everything

scripts/up clears leftovers from earlier runs, builds anything missing, then starts the plan. airfield project up <plan> and airfield project down do the same day to day. scripts/up and scripts/down also clean up after a crash: a stale VNC display lock, and a stuck camera service.

The navigation stack starts on the gdc_3n map. To use another one: MAP=<name> scripts/up. The maps are in packages/av_navigation.

In the tmux session:

  • Move between panes: Ctrl-b, then an arrow key.
  • Leave everything running and get your terminal back: Ctrl-b, then d. Return with tmux attach -t navstack (the plan's name).
  • A pane whose hardware isn't plugged in shows errors. The other panes keep working.

Always stop with scripts/down or airfield project down, not by closing tmux. Each pane stops its container when its tmux session ends normally, but killing the tmux server leaves containers running in the background. After a power loss or a hard crash, airfield project down --prune removes any leftover containers.

Plans live in plans/. Create a Plan on airfield.io explains how to write one.

To watch from your laptop, point Foxglove Studio at ws://<car>:8765, or a VNC viewer at <car>:5909 for rviz and the GUI.

Running a command in a package's container

cd ~/roboracer_ws
airfield package shell ut_automata                 # an interactive shell
airfield package cmd <package> -- <command>        # one command
airfield package run orin_rp2_csi mono_processor   # a named command from the package's airfield.yaml

Every command and option is in airfield's CLI Reference.

Per-machine settings

The .air file

.air, at the top of the project, lists folders on this machine that get shared into every container. It depends on the machine, so it isn't in git. The setup script writes it on a car. By hand:

cd ~/roboracer_ws
cat > .air <<'EOF'
mounts:
  - ~/.bash_history
  - ~/.ssh/authorized_keys
  # let Qt windows (ut_automata gui, rviz2) reach the touchscreen (:0) or the VNC display (:9)
  - /tmp/.X11-unix
  - /run/user/$UID/gdm
EOF

Keep $UID exactly as written: airfield fills in your user ID. A hardcoded number resolves to nothing on a machine whose user has a different ID, and the touchscreen GUI then can't draw. Folders that don't exist are skipped with a warning.

.air belongs in the project, not in packages/ut_automata, because ut_automata is shared course code that other robots use too.

Device groups

The containers reach the motor controller, the IMU and the controller through group numbers in packages/ut_automata/airfield.yaml:

devices:  [/dev/ttyACM0, /dev/i2c-7, /dev/input, /dev/ttyUSB0]
group_add: ["20", "108", "996"]   # dialout (VESC, RPLIDAR), i2c (IMU), input (controller)

Check a machine's numbers with getent group dialout i2c input. If they differ, change group_add: to match and don't commit that change. A device that isn't present is skipped with a warning, and the container still starts. If the motor controller or IMU is on a different device, change it both here and in vesc.lua.

Dependencies

A package's airfield.yaml lists its dependencies by name, and each name needs a recipe (a dependency manifest) that says how to install it. Airfield looks in the project's dependencies/ folder, then in its shared recipe book, a copy of airfield/packages in ~/.cache/airfield/packages.

airfield package dependencies check   # compare the project's recipes with the shared ones
airfield package dependencies pull    # get the newest shared recipes

A dependency with no recipe fails the build with an unresolved-dependency error. Add the recipe to the project's dependencies/xplatform/, or to the shared recipe book. Managing Dependencies on airfield.io explains how to write one.

The base image and JetPack

On the Orin, every package image starts from roboracer/l4t-jazzy:r39.2.1: ROS 2 Jazzy plus NVIDIA's camera and GPU libraries at exactly the car's JetPack version (L4T 39.2.1 is JetPack 7.2.1). The car's camera plugins are shared into the containers when they run, so the libraries inside have to match the car exactly, or the camera breaks in ways that are hard to trace.

The image isn't in any registry, so it's built on each car:

cd ~/roboracer_ws
dependencies/arm64/l4t-jazzy/build.sh

The script reads the car's L4T version, pins every NVIDIA package in the image to it, and tags the image with it. Nothing in the script is tied to one JetPack release. The project's airfield.yaml names the image every package uses (base_image:) and says not to download it (pull_base_image: false).

  • A car on the fleet's JetPack: run build.sh. The setup script does.
  • A car on another JetPack: build.sh still works, tags the image for that version, and warns that it isn't the one airfield.yaml names. Reflash the car to the fleet's version, or point base_image: at the tag it built, locally and uncommitted.
  • The whole fleet moving to a new JetPack: change base_image: once, then rebuild on every car.
  • build.sh fails its first check: NVIDIA has removed that L4T version from its package server. The error lists what's still available. L4T_VERSION=<version> dependencies/arm64/l4t-jazzy/build.sh builds with one of those, but the image then no longer matches the car exactly.

Updating

On an F1TENTH car, the setup script updates airfield, its recipes, and the code: see Keeping a car up to date.

On another machine:

pipx reinstall airfield                 # the newest commit of the branch it was installed from
airfield package dependencies pull      # the newest recipes, after updating airfield

Update airfield before its recipes: newer recipes can use a format an older airfield reads without complaint but installs nothing from. airfield system update compares version numbers and may say it's up to date when it isn't; airfield system update --force always reinstalls.

The first build after an airfield update rebuilds every image, because airfield copies itself into each one. To do that before you launch, run airfield package cmd <package> -- true for each package.

Setting up another machine

The F1TENTH cars use the setup script. On any other machine:

  1. Install Docker (on a Jetson, also NVIDIA's container runtime: sudo nvidia-ctk runtime configure --runtime=docker), and add yourself to the docker group.
  2. Install airfield: pipx install git+https://github.com/airfield/airfield.git, then run airfield doctor.
  3. Get the code: 
    cd ~
git clone https://github.com/ut-av/roboracer_ws.git
cd roboracer_ws
airfield subpackages checkout
git -C packages/ut_automata submodule update --init --recursive
  4. On a Jetson, build the base image.
  5. Create the .air file, then run scripts/build.

Troubleshooting

  • Foxglove bridge logs "Failed to load schemaDefinition ... not found". Harmless: those topics just can't be shown in Foxglove, because the bridge's image doesn't have their message packages. Add them to dependencies: in packages/foxglove_bridge/airfield.yaml to see them.
  • The GUI or rviz panes crash at startup. They draw on the VNC pane's display :9, and a crash can leave a stale lock for it. scripts/down then scripts/up clears it.
  • The camera pane says "Captured image is invalid or empty". See Raspberry Pi Camera troubleshooting.
  • nvidia-smi says "No devices" and the camera can't start. The GPU occasionally fails to start at boot. Reboot the car.
  • A change to a package doesn't show up. Run scripts/build <package>; see Building.