Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ every change goes through a short-lived branch and a pull request.
1. **Issue.** Every change starts with an issue: what and why. Take an existing one or create a new one.
2. **Branch** from `main`, named `<issue number>-<short-description>`, e.g. `57-imu-calibration`.
The *Create a branch* button in the issue (*Development* section) makes exactly this name.
3. **Commits** in the branch: small, each one builds. Messages are free-form in the branch, but should say
what changed.
3. **Commits** in the branch: small, each one builds. Write the messages in the same
[Conventional Commits](#pull-request-title) format as the PR title, e.g. `fix(hw): reconnect after a UART timeout`.
This is a recommendation, CI does not check it: the branch commits do not get into `main` (squash), but the
history of the PR is easier to review.
4. **Pull request** into `main` as soon as there is something to show, mark it as *Draft* while it is not ready.
The PR title follows [Conventional Commits](#pull-request-title), the description
[links the issue](#links-to-issues) (`Closes #57`).
Expand All @@ -27,7 +29,8 @@ topic: a fix found on the way goes to a separate PR.
## Pull request title

The PR title becomes the commit message in `main`, so it follows
[Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/). CI checks it.
[Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/). CI checks it. Use the same format for
the commit messages in the branch (recommended, not checked).

```
<type>(<scope>): <description>
Expand Down
38 changes: 34 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ and `log/` next to it.
| `src/external/` | external packages imported from `deps.repos` (not in git) |
| `deps.repos` | external packages that are not in apt, pinned to a commit |
| `docker/Dockerfile` | all images: `pc`, `robot` |
| `docker-compose.yaml` | PC side (service `pc`) |
| `docker-compose.yaml` | PC side (service `pc`, with GPU variants `pc-nvidia` and `pc-wsl`) |
| `scripts/rpi-setup/` | robot side: compose file, setup script, udev rules. Installed to `~/kpi-rover` on the robot |
| `rover.sh` | one entry point for everything below, `./rover.sh help` |

Expand Down Expand Up @@ -103,9 +103,36 @@ Image tags: `main` (current version), `<branch>` (built on request), `sha-<commi

## Simulation on a PC

Requirements: Linux with Docker ([without sudo](https://docs.docker.com/engine/install/linux-postinstall/)) and an X server.
Requirements: Linux (or Windows with WSL2, see below) with Docker ([without sudo](https://docs.docker.com/engine/install/linux-postinstall/)),
the Docker Compose v2 plugin (`docker compose version` must work) and an X server. Ubuntu's own `docker.io` package
does not include Compose: `sudo apt install docker-compose-v2` (with Docker CE from docker.com: `docker-compose-plugin`).

**Windows (WSL2).** Works with WSL2 and WSLg (Windows 11, or Windows 10 with WSL from the Microsoft Store: run
`wsl --update` in Windows). Check in WSL: `ls /mnt/wslg /dev/dxg` must show both. Install Docker inside WSL as
on Ubuntu (above). The GPU driver is the normal Windows driver, **do not install a Linux GPU driver or
nvidia-container-toolkit in WSL**. `rover.sh` detects WSL (`/dev/dxg`) and uses the service `pc-wsl`: Gazebo and
RViz render on the Windows GPU through Mesa's d3d12 driver, for NVIDIA, AMD and Intel. The renderer check below
shows `D3D12 (<your GPU>)`. A laptop with two GPUs may pick the integrated one, choose another by a part of
its name: `MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA ./rover.sh pc`.
Keep the repository in the WSL file system (e.g. `~/ros_sw`), not on a Windows drive (`/mnt/c/...`): there
building and Gazebo are several times slower.

**WSL and the real robot: mirrored networking.** By default WSL2 is behind NAT: ssh to the robot works, but
ROS 2 topics do not (the robot cannot reach the PC, multicast discovery does not pass). Simulation alone does not
need this. Switch WSL to the mirrored mode once: in Windows create `%UserProfile%\.wslconfig` with
```ini
[wsl2]
networkingMode=mirrored
```
then run `wsl --shutdown` in Windows and open WSL again. WSL then has the same IP address as Windows
(`hostname -I` in WSL shows the address of the Windows Wi-Fi/Ethernet). If the robot's topics are still not
visible, Windows Firewall blocks the incoming DDS traffic: allow it for the Hyper-V firewall of WSL once, in
PowerShell as administrator:
```powershell
Set-NetFirewallHyperVVMSetting -Name '{40E0AC32-46A5-438A-A0B2-2B479E8F2E90}' -DefaultInboundAction Allow
```

**NVIDIA GPU.** Without [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
**NVIDIA GPU (Linux).** Without [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
Gazebo and RViz render on the CPU and are very slow. Install it once (Ubuntu/Debian, the NVIDIA driver must already work: `nvidia-smi`):
```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
Expand Down Expand Up @@ -319,7 +346,10 @@ students can pull them without a login (organization package settings).

| Problem | Reason / solution |
|---|---|
| Gazebo is very slow | Check `./rover.sh doctor`: `renderer` must not be `llvmpipe` (CPU rendering). Intel/AMD: the user must be in the `video` and `render` groups. NVIDIA: install nvidia-container-toolkit. |
| `unknown flag: --rm` or `Docker Compose v2 is not installed` | The Compose plugin is missing: `sudo apt install docker-compose-v2` (Ubuntu `docker.io`) or `docker-compose-plugin` (Docker CE). |
| Gazebo is very slow | Check `./rover.sh doctor`: `renderer` must not be `llvmpipe` (CPU rendering). Intel/AMD: the user must be in the `video` and `render` groups. NVIDIA: install nvidia-container-toolkit. WSL: `wsl` must be shown in `doctor` with `WSLg: yes`, update the Windows GPU driver and run `wsl --update`. |
| WSL: the renderer is the wrong GPU (e.g. Intel instead of NVIDIA) | `MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA ./rover.sh pc` |
| WSL: ssh to the robot works, but no topics in RViz / `ros2 topic list` | WSL is behind NAT: switch to mirrored networking and allow inbound traffic, see [Simulation on a PC](#simulation-on-a-pc) (Windows (WSL2)). |
| RViz does not show the robot | Same network? `./rover.sh rviz` uses the robot's `ROS_DOMAIN_ID` and its address as a static peer, so multicast is not required. |
| The robot software restarts in a loop | `./rover.sh logs`. `device ... not found`: check the lidar cable and `./rover.sh doctor` (devices). |
| `./rover.sh use my-branch` cannot download the image | The workflow for the branch has not finished, or failed. Check *Actions*. |
Expand Down
18 changes: 18 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,21 @@ services:
environment:
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=all

# The same on Windows (WSL2 with WSLg). The GPU of Windows is reached through /dev/dxg and Mesa's d3d12 driver,
# for any vendor (NVIDIA, AMD, Intel); the GPU libraries come from Windows in /usr/lib/wsl/lib. No
# nvidia-container-toolkit is needed for rendering.
pc-wsl:
extends: pc
profiles: [wsl]
environment:
- LD_LIBRARY_PATH=/usr/lib/wsl/lib
- GALLIUM_DRIVER=d3d12
# Several GPUs (e.g. a laptop with Intel + NVIDIA): pick one by a part of its name, e.g. NVIDIA
# (passed only when set on the host)
- MESA_D3D12_DEFAULT_ADAPTER_NAME
volumes:
- /usr/lib/wsl:/usr/lib/wsl:ro
- /mnt/wslg:/mnt/wslg
devices:
- /dev/dxg:/dev/dxg
8 changes: 7 additions & 1 deletion rover.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ robot_env() { on_robot "sed -n 's/^$1=//p' .env"; }
# ---------- PC ----------

pc_env() {
docker compose version > /dev/null 2>&1 ||
die "Docker Compose v2 is not installed (\`docker compose\` does not work). On Ubuntu with docker.io: sudo apt install docker-compose-v2; with Docker CE: sudo apt install docker-compose-plugin"
export HOST_UID HOST_GID VIDEO_GID RENDER_GID
HOST_UID=$(id -u)
HOST_GID=$(id -g)
Expand All @@ -65,7 +67,10 @@ pc_env() {
xhost +local:docker > /dev/null
fi
PC_SERVICE=${PC_SERVICE:-pc}
if [ "$PC_SERVICE" = pc ] && command -v nvidia-container-runtime > /dev/null && docker info --format "{{json .Runtimes}}" 2> /dev/null | grep -q nvidia; then
# WSL2 exposes the Windows GPU as /dev/dxg (any vendor), rendering goes through Mesa's d3d12 driver
if [ "$PC_SERVICE" = pc ] && [ -e /dev/dxg ]; then
PC_SERVICE=pc-wsl
elif [ "$PC_SERVICE" = pc ] && command -v nvidia-container-runtime > /dev/null && docker info --format "{{json .Runtimes}}" 2> /dev/null | grep -q nvidia; then
PC_SERVICE=pc-nvidia
fi
}
Expand Down Expand Up @@ -221,6 +226,7 @@ cmd_doctor() {
echo "os: $(. /etc/os-release && echo "$PRETTY_NAME")"
echo "docker: $(docker version --format '{{.Server.Version}}' 2>&1)"
echo "compose: $(docker compose version --short 2>&1)"
[ -e /dev/dxg ] && echo "wsl: $(uname -r), WSLg: $([ -d /mnt/wslg ] && echo yes || echo no)"
echo "groups: video=$(getent group video | cut -d: -f3) render=$(getent group render | cut -d: -f3)"
command -v nvidia-smi > /dev/null && echo "gpu: $(nvidia-smi --query-gpu=name --format=csv,noheader 2>&1)"
if [ -n "${DISPLAY:-}" ]; then
Expand Down