Deployment & CI/CD

JOSHUA provides a comprehensive deployment infrastructure that supports cross-platform builds, Docker containerization, and automated continuous integration. This page covers every aspect of building, testing, and deploying the JOSHUA framework across multiple architectures and operating system versions.

Overview

JOSHUA targets cross-platform deployment across both AMD64 and ARM64 architectures. The framework uses Docker for environment isolation and reproducibility, Bazel for hermetic builds, and GitHub Actions for continuous integration and automated testing.

The deployment stack consists of the following layers:

• Docker Compose — Defines multi-service container configurations for development and production.
• Bazel Build System — Hermetic builds with cross-compilation support for ARM64 targets.
• GitHub Actions CI/CD — Automated build, test, and lint pipelines triggered on pull requests.
• Setup Scripts — Automated environment provisioning for bare-metal and container-based deployments.
• Code Quality Gates — Pre-commit hooks and linting enforce code standards before merge.

Docker Containers

JOSHUA uses Docker Compose to define a multi-service container topology. The docker-compose.yml file defines five services covering production, experimental, ARM64, and UI workloads. A separate docker-compose.dev.yml file provides development-specific overrides.

Service Definitions

The five services defined in docker-compose.yml are:

Development Containers

All development containers (joshua-u22, joshua-u24, and the ARM64 variants) share a common configuration profile optimized for robotics development:

• Privileged mode — Required for direct hardware access to USB devices (servo controllers, cameras, LiDAR).
• Host networking — Enables ROS2 DDS discovery and multicast communication without port mapping overhead.
• /dev mount — Bind-mounts the host /dev directory to allow access to serial ports, video devices, and input devices.
• Persistent Bazel cache volumes — Named Docker volumes are mounted for the Bazel output base and repository cache, preventing full rebuilds across container restarts.

# Example: Start the production development container
docker compose up -d joshua-u22

# Example: Start with development overrides
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d joshua-u22

# Example: Start the web UI container
docker compose up -d joshua-ui
# Accessible at http://localhost:3000

Web UI Container

The joshua-ui service runs the React web interface inside an nginx container. It serves the production build of the web UI on port 3000 and proxies WebSocket connections to the Zenoh bridge for real-time ROS2 topic streaming.

Development Compose File

The docker-compose.dev.yml file extends the base configuration with development-specific settings such as source code bind mounts, additional environment variables for debugging, and X11 forwarding for GUI applications (Qt6 control panel, MuJoCo visualization).

Cross-Compilation

JOSHUA supports cross-compilation for ARM64 targets, primarily the NVIDIA Jetson Orin Nano edge computing platform used for on-robot inference. The cross-compilation infrastructure is built on Bazel's platform system and conditional compilation features.

Bazel Platforms

Three Bazel platform definitions are provided for targeting different architectures:

Conditional Compilation

The .bazelrc file defines config settings that enable conditional compilation based on the target platform. This allows architecture-specific code paths (e.g., NEON SIMD on ARM64, AVX on x86_64) to be selected at build time:

# Build for Jetson Orin Nano
bazel build --config=jetson //joshua/robot:all

# Build for standard AMD64
bazel build --config=u22 //joshua/robot:all

Pre-Built Distribution Tarballs

For environments where building from source is impractical, JOSHUA provides pre-built distribution tarballs in the dist/ directory:

• dist/u22/x86/ — Ubuntu 22.04 AMD64 pre-built binaries
• dist/u22/arm64/ — Ubuntu 22.04 ARM64 pre-built binaries (Jetson Orin Nano)
• dist/u24/arm64/ — Ubuntu 24.04 ARM64 pre-built binaries

Build System (Bazel)

JOSHUA uses Bazel 8.3.1 (managed via Bazelisk) as its primary build system. Bazel provides hermetic, reproducible builds with built-in support for cross-compilation, remote caching, and multi-language projects (C++, Python, Protocol Buffers).

MODULE.bazel and Dependencies

The MODULE.bazel file declares all external dependencies using Bazel's Bzlmod system. All dependencies are resolved hermetially — they are downloaded and built from source at pinned versions, ensuring reproducibility across machines:

Bazel Rule Sets

JOSHUA uses the following Bazel rule sets to build different target types:

• rules_cc — C++ compilation rules for robot drivers, node generator, and control panel.
• rules_python — Python rules for AI inference, simulation scripts, and data processing.
• rules_proto — Protocol Buffers compilation for configuration schema and ROS2 message generation.
• rules_pkg — Packaging rules for creating distribution tarballs and Debian packages.

Hermetic Python Toolchain

Bazel manages two hermetic Python toolchains to match the target Ubuntu versions:

• Python 3.10 — Used with Ubuntu 22.04 (ROS2 Humble) builds via --config=u22.
• Python 3.12 — Used with Ubuntu 24.04 (ROS2 Jazzy) builds via --config=u24.

The Python toolchains are downloaded and managed by Bazel, ensuring that the correct interpreter version is always used regardless of what is installed on the host system.

OS-Specific Configurations

The .bazelrc file provides OS-specific build configurations that set the correct compiler flags, Python version, and ROS2 distribution paths:

# Ubuntu 22.04 with ROS2 Humble
bazel build --config=u22 //joshua/...

# Ubuntu 24.04 with ROS2 Jazzy
bazel build --config=u24 //joshua/...

# Run all tests
bazel test --config=u22 //...

C++ Standard and Compiler

All C++ targets are compiled with the C++17 standard using the system GCC compiler. The .bazelrc file sets --cxxopt=-std=c++17 globally. The system GCC is used (rather than a hermetic toolchain) to ensure ABI compatibility with system-installed ROS2 libraries.

CI/CD Pipeline

JOSHUA uses GitHub Actions for continuous integration. The pipeline is defined in .github/workflows/ci.yml and runs automated builds and tests on every pull request.

Trigger Conditions

• Pull requests targeting the develop branch — Every PR is validated before merge.
• Manual dispatch — The workflow can be triggered manually via the GitHub Actions UI for ad-hoc testing.

Runner Environment

The CI pipeline runs on ubuntu-22.04 GitHub-hosted runners. This matches the primary production target and ensures that tests run in an environment identical to deployment.

Pipeline Steps

The CI workflow executes the following steps in order:

1. Checkout — Clones the repository with full history for accurate change detection.
2. Disk Cleanup — Reclaims disk space on the GitHub runner by removing unused toolchains and SDKs. GitHub-hosted runners have limited disk space, and Bazel builds can consume significant storage.
3. Bazel Cache Restoration — Restores the Bazel build cache from GitHub Actions cache, keyed on MODULE.bazel.lock. When dependencies change, the cache key changes and a fresh cache is built. This dramatically reduces build times for PRs that don't modify dependencies.
4. Install Bazelisk — Downloads and installs Bazelisk, which automatically manages the correct Bazel version (8.3.1) as specified in the .bazelversion file.
5. Setup ROS2 Humble — Adds the ROS2 apt repository and installs the ros-humble-desktop meta-package along with development tools. Sources the ROS2 setup script to configure environment variables.
6. Install System Dependencies — Installs required system libraries:
   • Qt6 — For the desktop control panel GUI.
   • OpenCV — For camera capture and image processing.
   • libevdev — For gamepad/joystick input device handling.
7. Run Tests — Executes the full test suite with Bazel:

   bazel test //...

   This command discovers and runs all test targets in the repository, including C++ unit tests (Google Test), Python tests, and integration tests.

Setup Script

The scripts/setup.sh script automates environment provisioning for both development workstations and production runtime environments. It handles the complete installation of all system dependencies, build tools, and framework prerequisites.

Environment Modes

The script supports two environment modes selected via the --env flag:

OS Auto-Detection

The script automatically detects the Ubuntu version (22.04 or 24.04) and selects the appropriate package versions, ROS2 distribution (Humble or Jazzy), and Python version. Running on an unsupported OS version will produce a clear error message.

Installation Stages

The setup script executes the following stages:

1. ROS2 Installation — Adds the official ROS2 apt repository, installs the desktop meta-package (dev mode) or base meta-package (runtime mode), and configures the environment.
2. OpenCV — Installs OpenCV libraries and Python bindings for camera capture and image processing.
3. Python Dependencies — Installs Python packages via pip, including PyTorch, HuggingFace Transformers, JAX, and other ML framework dependencies.
4. Docker — Installs Docker Engine and Docker Compose plugin, configures the current user for rootless Docker access.
5. Bazel — Installs Bazelisk (which manages Bazel versioning) and verifies the installation.

NVIDIA Library Fixes

The setup script includes special handling for NVIDIA GPU libraries that are commonly misconfigured on Ubuntu systems:

• CuDNN — Resolves symlink issues and ensures the correct CuDNN version is linked for PyTorch and TensorRT.
• cusparseLt — Fixes missing library paths that cause PyTorch import failures on systems with CUDA 12.
• NCCL — Configures NCCL for multi-GPU training scenarios and resolves version conflicts between CUDA toolkit and PyTorch packages.

# Full development setup on Ubuntu 22.04
./scripts/setup.sh --env=dev

# Minimal runtime setup on Ubuntu 24.04
./scripts/setup.sh --env=runtime

Code Quality

JOSHUA enforces code quality standards through pre-commit hooks, automated linting, and standardized PR templates. These gates ensure consistent code style and catch common issues before they reach the main codebase.

Pre-Commit Hooks

Two pre-commit hooks run automatically before each commit:

• lint_check.sh — Runs C++ and Python linters on staged files. Blocks the commit if any linting errors are found.
• sample_check.py — Validates that configuration samples and example files are syntactically correct and conform to the Protocol Buffers schema.

C++ Linting

C++ code is formatted and linted using clang-format-14. The project includes a .clang-format configuration file at the repository root that defines the coding style:

# Run clang-format on all C++ files
find . -name "*.cc" -o -name "*.h" | xargs clang-format-14 -i

# Check formatting without modifying (used in CI)
find . -name "*.cc" -o -name "*.h" | xargs clang-format-14 --dry-run --Werror

Python Linting

Python code is linted using ruff, a fast Python linter written in Rust. Ruff enforces PEP 8 style, import ordering, and common Python anti-patterns:

# Run ruff linter
ruff check .

# Run ruff with auto-fix
ruff check --fix .

PR Template

The repository includes a pull request template (.github/PULL_REQUEST_TEMPLATE.md) that standardizes PR descriptions with the following sections:

• Summary — A brief description of what the PR changes and why.
• Test — Description of how the changes were tested, including specific test commands and results.

.clang-format Configuration

The .clang-format file at the repository root defines the C++ code style. All contributors must format their code according to this configuration before submitting a PR. The pre-commit hook (lint_check.sh) enforces this automatically.

Platform Support Matrix

The following table summarizes the supported platform configurations, their associated ROS2 distributions, Python versions, and current support status:

The Ubuntu 22.04 AMD64 configuration is the primary development and deployment target. It is the platform used in CI (ubuntu-22.04 runners), and all hardware deployments (desktop workstations, Jetson Orin Nano) are validated against this configuration first.