Skip to content

VNC and RViz

You watch the car's sensors, maps and plans in RViz, the standard ROS visualization tool. RViz runs on the car itself, on a remote desktop, and you open that desktop on your laptop with a VNC viewer. Nothing besides the viewer needs to be installed on your laptop.

The car's plans start everything for you: a vnc pane serves the desktop (display :9), and an rviz2 pane opens RViz on it.

Install the viewer

Install the TurboVNC viewer from the TurboVNC releases page. There are installers for Windows, macOS and Linux.

Connect

  1. Turn on your WireGuard tunnel.
  2. Launch a plan on the car, for example airfield project up teleop.

  3. Open the TurboVNC viewer and connect to your car's address followed by :5909, the port of display 9. For car 12 that is 10.1.0.112:5909, or car12:5909 if you've added the car to your hosts file.

    TurboVNC Client

There is no password. You'll see a desktop with the RViz window on it. The window can take up to a minute to appear after the plan starts.

What each plan shows

Plan On the desktop
teleop RViz with a model of the car, the camera, the lidar and the TF tree, from ~/roboracer_ws/rviz2/sensors.rviz
record_map RViz with the map as it is built, the car on it, the lidar, the camera and the TF tree, from navstack.rviz, and the GUI
navstack The same RViz view, plus the costmap, the planned path and the AMCL pose, and the GUI
imitation_inference RViz with the camera, from sensors.rviz, and the GUI
camera_calibration The camera calibration window
car_calibration, record_sensors The GUI

Using RViz

  • RViz draws with the car's GPU. That's what vglrun -d egl does in the rviz2 pane's command; without it, RViz would take about two of the car's six CPU cores.
  • If you close the RViz window, it opens again by itself a few seconds later.
  • Changes you make to the displays last until RViz closes. To keep them, use File → Save Config As and save into ~/roboracer_ws/rviz2/, then reopen your file later with File → Open Config.
  • To stop RViz, press Ctrl-C in its tmux pane.

To start RViz by hand, for example after stopping it, run this on the car:

cd ~/roboracer_ws
airfield package cmd rviz2 -- bash -lc 'DISPLAY=:9 vglrun -d egl rviz2 -d $HOME/roboracer_ws/rviz2/sensors.rviz'

Use navstack.rviz instead of sensors.rviz for the map view. This needs a plan with a vnc pane to be running.

Troubleshooting

The viewer can't connect

  • Check that WireGuard is on and that you can ssh to the car.
  • Check that the plan is running and that its vnc pane says VNC Server started.
  • Check the address ends in :5909.

The desktop is there but RViz isn't

Look at the rviz2 pane. It waits for the vnc pane's display before it starts, so it sits silently until the display is up. If it keeps waiting, or shows an error about connecting to display :9, tear the plan down with airfield project down --prune and launch it again.

The viewer asks for a password

The plans' VNC server never asks for one, so you have probably reached a different server. Check the port is 5909.

No password means anyone on the network can connect

Anyone who can reach port 5909 on the car can see the desktop and use it while a plan runs. Connect to the cars over the WireGuard VPN.