JOSHUA

Joint Open-Source Hub for Universal Automation

A modular, open-source robotics framework that bridges high-level AI with physical hardware through standardized middleware, enabling rapid prototyping of autonomous robotic systems.

Get Started View Architecture Source Code

Project Overview

JOSHUA transforms a single configuration file into a fully operational robot system using ROS2, Protocol Buffers, and pluggable AI inference.

JOSHUA System Architecture Diagram
Figure 1: JOSHUA System Architecture — Configuration-driven robot system from a single config file through ROS2 node orchestration to hardware operation.

Configuration-Driven Design

Define your entire robot system—hardware, sensors, AI models, and operation mode—in a single Protocol Buffers text configuration file. No code changes required to reconfigure the system.

Hardware Abstraction Layer

Plug-and-play support for servo motors (STS3215, Pybricks SPIKE), cameras (OpenCV), encoders, and LiDAR sensors with factory-pattern instantiation and mock drivers for testing.

AI Inference Pipeline

Extensible model architecture supporting SmolVLA (vision-language-action), decision transformers, and custom models. C++/Python interop via PyBind11 for real-time inference on edge devices.

ROS2 Middleware Integration

Built on ROS2 (Humble/Jazzy) with 91 standard message types, configurable QoS profiles, and automatic node lifecycle management through the Node Generator orchestrator.

MuJoCo Simulation

Physics-based simulation with interactive, passive, mirror (digital twin), and offscreen rendering modes. GPU-accelerated RL training via MuJoCo-XLA (MJX) and NVIDIA Isaac Sim integration.

Cross-Platform Deployment

Supports AMD64 and ARM64 (NVIDIA Jetson Orin Nano) with Docker containers for Ubuntu 22.04/24.04. Bazel build system for reproducible, hermetic builds with cross-compilation.

System Architecture

A layered, modular architecture where each component is independently configurable and replaceable.

Configuration Layer

Protocol Buffers schema defines the entire system: robot hardware, AI models, communication protocols, operation modes, and simulation settings. A single .pbtxt file configures the complete system.

Node Generator (Orchestrator)

Central process manager that parses configuration, validates integrity (port conflicts, dependency checks), determines C++/Python backend per node, and manages the full lifecycle of child processes.

Robot Layer (HAL)

Hardware Abstraction Layer with factory-pattern device instantiation. Supports multiple actuators, sensors, and communication protocols through standardized interfaces with both C++ and Python implementations.

AI Layer

Model registry with base class architecture for pluggable inference models. Supports vision-language-action models (SmolVLA), reinforcement learning (PPO via JAX/Flax), and custom model integration.

Simulation Engine

MuJoCo-based physics simulation with four operational modes. GPU-accelerated parallel training via MuJoCo-XLA (2048 parallel environments). NVIDIA Isaac Sim integration for industrial-grade simulation.

User Interfaces

Qt6 C++ desktop control panel for real-time monitoring and calibration. React/TypeScript web UI with Zenoh bridge for distributed monitoring, topology visualization, and configuration management.

Explore Full Architecture Documentation

Technology Stack

Built on industry-standard technologies for robotics, AI, and systems engineering.

Core Languages

C++17 Python 3.10/3.12 TypeScript Protocol Buffers

C++ for performance-critical robot drivers and node orchestration. Python for AI inference and rapid prototyping. TypeScript for the web monitoring UI.

Robotics & Middleware

ROS2 Humble ROS2 Jazzy Zenoh Boost.Asio

ROS2 for inter-process communication with configurable QoS. Zenoh bridge for distributed monitoring. Boost.Asio for async serial communication.

AI & Machine Learning

PyTorch HuggingFace JAX/Flax LeRobot Stable-Baselines3

PyTorch and HuggingFace Transformers for model inference. JAX/Flax with MuJoCo-XLA for GPU-parallel RL training. LeRobot for vision-language-action models.

Simulation

MuJoCo MuJoCo-XLA NVIDIA Isaac Sim Gymnasium

MuJoCo for physics simulation with 4 operational modes. MJX for GPU-accelerated training. Isaac Sim for industrial-grade simulation with USD assets.

Build & Infrastructure

Bazel Docker GitHub Actions PyBind11

Bazel for hermetic, reproducible builds with cross-compilation (ARM64). Docker multi-platform containers. CI/CD with automated testing and linting.

User Interface

Qt6 React 18 Tailwind CSS Radix UI Vite

Qt6 native desktop GUI for real-time control and calibration. React web UI with topology visualization and Zenoh SSE-based real-time monitoring.

Operation Modes

JOSHUA supports seven distinct operation modes, all selectable through configuration.

Teleoperation

Leader-follower arm control for manual robot manipulation. Supports Xbox controller input and keyboard control for data collection scenarios.

AI Inference

Real-time autonomous operation using trained AI models. Supports SmolVLA vision-language-action models with multi-camera input and natural language task descriptions.

Training

Reinforcement learning (PPO) and imitation learning pipelines. GPU-parallel training via MuJoCo-XLA with 2048 simultaneous environments.

Simulation

Physics-based simulation with interactive GUI, trajectory replay, ROS2 digital twin mirroring, and headless offscreen rendering for data generation.

Data Collection

Record teleoperation demonstrations with episode indexing for imitation learning. Supports HuggingFace datasets format (Parquet, CSV, JSONL) for standardized data storage.

Calibration

Servo motor calibration with operational limit detection. Publishes calibration data as ROS2 topics for integration with the control panel GUI.

Technical Documentation

Comprehensive documentation covering every layer of the JOSHUA framework.

System Architecture →

Layered architecture overview, data flow diagrams, node orchestration, and the dual-layer data type system (Protocol Buffers + ROS2).

Getting Started →

Installation guide for Ubuntu 22.04/24.04, Docker setup, Bazel build instructions, and running your first robot configuration.

Configuration System →

Protocol Buffers schema reference, configuration presets, operation modes, and the complete .pbtxt file specification.

Robot Hardware Layer →

Hardware abstraction, factory patterns, supported actuators (STS3215, Pybricks), sensors (camera, LiDAR, encoder), and communication protocols.

AI System →

Model registry, SmolVLA integration, reinforcement learning pipeline, training infrastructure, and the extensible inference architecture.

Simulation Engine →

MuJoCo physics engine, simulation modes, MJCF robot models, MuJoCo-XLA parallel training, and NVIDIA Isaac Sim integration.

Deployment & CI/CD →

Docker containerization, cross-compilation for ARM64, GitHub Actions CI, pre-commit hooks, and production deployment workflows.