Adding Cameras

Adamo encodes camera frames with a hardware H.264 encoder and publishes them as named tracks. You attach as many tracks as you want — one per camera — and they run in parallel.

There are two input methods:

1. V4L2 — direct device capture from /dev/videoN. Use this for USB webcams, RealSense, and any V4L2-compatible camera.
2. Shared memory (iceoryx2) — frames produced by another process on the same host (a custom driver, a perception pipeline, an external bridge), or pushed in from your own SDK code via the caller-fed video() track.

Those sources can carry raw pixels such as BGRA, NV12, or I420. They can also carry MJPEG frames or complete H.264/H.265 access units. Encoded H.264 and H.265 inputs can either be transcoded to the configured output codec or passed through unchanged.

USB / V4L2 Cameras

For USB webcams, Intel RealSense, and any V4L2-compatible device.

Python

import adamo

robot = adamo.Robot(api_key="ak_...", name="my-arm")
robot.attach_video(
    "main",
    device="/dev/video0",
    width=1280,
    height=720,
    fps=30,
    bitrate_kbps=4000,
)
robot.run()

Rust

use adamo::Robot;

fn main() -> adamo::Result<()> {
    let mut robot = Robot::new_default("ak_...", Some("my-arm"))?;
    robot.attach_v4l2("main", "/dev/video0", 1280, 720, 30, 4000, false)?;
    robot.run()
}

C

adamo_robot_attach_video_v4l2(
    robot, "main", "/dev/video0",
    /* width  */ 1280, /* height */ 720,
    /* fps    */ 30,   /* bitrate_kbps */ 4000,
    /* stereo */ false);

Finding the device

ls /dev/video*
v4l2-ctl --list-devices
v4l2-ctl -d /dev/video0 --list-formats     # what pixel formats it supports

On Jetson the SDK auto-detects the V4L2 pixel format (YUY2, NV12, …) so the encoder uses the optimal GPU colourspace path.

If the camera exposes an MJPEG mode, request it with pixel_format="mjpeg":

robot.attach_video(
    "usb_mjpeg",
    device="/dev/video0",
    width=1280,
    height=720,
    fps=30,
    bitrate_kbps=4000,
    pixel_format="mjpeg",
)

Multiple Cameras

Attach one track per camera. They run independently — different fps and bitrates per track is fine.

Python

robot.attach_video("front", device="/dev/video0", width=1920, height=1080, fps=60, bitrate_kbps=8000)
robot.attach_video("rear",  device="/dev/video2", width=1280, height=720,  fps=15, bitrate_kbps=2000)

Rust

robot.attach_v4l2("front", "/dev/video0", 1920, 1080, 60, 8000, false)?;
robot.attach_v4l2("rear",  "/dev/video2", 1280, 720,  15, 2000, false)?;

C

adamo_robot_attach_video_v4l2(robot, "front", "/dev/video0", 1920, 1080, 60, 8000, false);
adamo_robot_attach_video_v4l2(robot, "rear",  "/dev/video2", 1280, 720,  15, 2000, false);

Shared Memory (iceoryx2)

Use shared memory when frames come from somewhere V4L2 can’t see — a custom driver, a Python perception loop, an external bridge, or your own SDK code rendering frames in process.

Two patterns: consume an existing iceoryx2 service produced by another process, or publish frames yourself from a tight loop.

Consume an existing SHM service

Use this when another process already owns the camera or sensor and can publish raw frames into iceoryx2. The general flow is:

1. Your camera process captures or generates one complete raw frame.
2. It publishes that frame as a single iox2.Slice[ctypes.c_uint8] sample.
3. Adamo attaches to the same iceoryx2 service with robot.attach_video(..., shm=...).
4. The native Adamo pipeline reads the SHM frames, hardware-encodes them, and streams the video track.

The producer and Adamo bridge must agree on the iceoryx2 service name, frame width, frame height, pixel format, and frame rate. The SHM payload is raw frame bytes only. Do not prepend timestamps, headers, or metadata. For 1280x720 BGRA, each sample must contain exactly 1280 * 720 * 4 = 3,686,400 bytes.

Python

Run the publisher and Adamo bridge as separate processes on the same host:

export ADAMO_API_KEY="ak_..."
python zed_left_to_shm.py
python shm_to_adamo.py

The publisher below is a ZED-specific example: it captures the left image from the ZED SDK and publishes it as raw BGRA frames. For other camera types, keep the iceoryx2 publishing shape the same and replace only the frame acquisition code.

zed_left_to_shm.py

import ctypes

import iceoryx2 as iox2
import numpy as np
import pyzed.sl as sl

SERVICE = "camera/zed/left"
FPS = 60

params = sl.InitParameters()
params.camera_resolution = sl.RESOLUTION.HD720
params.camera_fps = FPS
params.depth_mode = sl.DEPTH_MODE.NONE

camera = sl.Camera()
status = camera.open(params)
if status != sl.ERROR_CODE.SUCCESS:
    raise RuntimeError(f"ZED open failed: {status}")

info = camera.get_camera_information()
resolution = info.camera_configuration.resolution
WIDTH = int(resolution.width)
HEIGHT = int(resolution.height)
PIXEL_FORMAT = "BGRA"
FRAME_SIZE = WIDTH * HEIGHT * 4

node = iox2.NodeBuilder.new().create(iox2.ServiceType.Ipc)
service = (
    node.service_builder(iox2.ServiceName.new(SERVICE))
    .publish_subscribe(iox2.Slice[ctypes.c_uint8])
    .enable_safe_overflow(True)
    .subscriber_max_buffer_size(2)
    .open_or_create()
)
publisher = service.publisher_builder().initial_max_slice_len(FRAME_SIZE).create()

runtime = sl.RuntimeParameters()
image = sl.Mat()

while True:
    if camera.grab(runtime) != sl.ERROR_CODE.SUCCESS:
        continue

    camera.retrieve_image(image, sl.VIEW.LEFT)
    frame = np.ascontiguousarray(image.get_data())
    payload = frame.tobytes()
    if len(payload) != FRAME_SIZE:
        continue

    sample = publisher.loan_slice_uninit(FRAME_SIZE)
    ctypes.memmove(sample.payload_ptr, payload, FRAME_SIZE)
    sample.assume_init().send()

shm_to_adamo.py

import os

import adamo

robot = adamo.Robot(
    api_key=os.environ["ADAMO_API_KEY"],
    name="my-robot",
)

robot.attach_video(
    "main",
    shm="camera/zed/left",
    width=1280,
    height=720,
    pixel_format="BGRA",
    fps=60,
    bitrate_kbps=4000,
)

robot.run()

The Adamo side is camera-agnostic. It only needs the service name and frame layout that your publisher uses.

If the shared-memory service publishes complete JPEG frames instead of raw pixels, set pixel_format="mjpeg":

robot.attach_video(
    "head",
    shm="head_cam_mjpeg",
    width=1280,
    height=720,
    fps=30,
    bitrate_kbps=4000,
    pixel_format="mjpeg",
)

Rust

Use Robot::attach_shm when another process already publishes complete frames to an iceoryx2 service:

use adamo::Robot;

fn main() -> adamo::Result<()> {
    let mut robot = Robot::new_default("ak_...", Some("my-robot"))?;

    robot.attach_shm(
        "main",
        "camera/zed/left",
        1280,
        720,
        "BGRA",
        60,
        4000,
        false,
    )?;

    robot.run()
}

For compressed Motion JPEG services, pass "mjpeg" as the pixel format:

robot.attach_shm("head", "head_cam_mjpeg", 1280, 720, "mjpeg", 30, 4000, false)?;

C

Use adamo_robot_attach_video_shm when another process already publishes complete frames to an iceoryx2 service:

adamo_robot_t *robot = adamo_robot_new_default("ak_...", "my-robot");

adamo_robot_attach_video_shm(
    robot,
    "main",
    "camera/zed/left",
    1280,
    720,
    "BGRA",
    60,
    4000,
    false
);

adamo_robot_run(robot);

For compressed Motion JPEG services, pass "mjpeg" as the pixel format.

Transcode H.264 or H.265 from shared memory

Use this mode when another process already produces encoded video, but the codec sent through Adamo needs to be different. For example, a camera process can publish H.265 into shared memory while Adamo decodes it and re-encodes it as H.264 for the frontend.

The producer and Adamo must use the same iceoryx2 service name. Each [u8] sample must contain exactly one complete Annex-B access unit, with no timestamp, size prefix, or application header prepended. Publish parameter sets (H.264 SPS/PPS or H.265 VPS/SPS/PPS) with keyframes so a decoder can join the stream. AVCC/HVCC length-prefixed samples must be converted to Annex-B first.

The important options are:

Option	Meaning
Source format	The bytes in shared memory: h264/avc or h265/hevc.
Output codec	The codec Adamo sends: normally h264 or h265.
Encoder	Optional host-specific encoder. Leave unset to auto-detect one.
Passthrough	Keep false to decode and re-encode. Set true only to forward the original codec unchanged.
Backend	hw_pipeline uses the native hardware path; gstreamer uses GStreamer.

Robot configuration

h265-to-h264.yaml

robot_name: "my-robot"
api_key: "ak_..."

video_tracks:
  - name: "front"
    source_type: "shm"
    shm_service: "camera/front/encoded"
    source_format: "h265"       # input in shared memory
    v4l2_capture_resolution: [1920, 1080]
    fps: 30
    encoder: "nvv4l2h264enc"    # H.264 output on Jetson
    hw_pipeline: true
    passthrough: false           # decode + re-encode
    bitrate: 4000                # kbit/s
    keyframe_distance: 2.0

Start the Adamo publisher, then start the process that writes access units:

adamo-network --config h265-to-h264.yaml

Use vtenc_h264 on Apple Silicon, nvh264enc on an NVIDIA desktop GPU, vah264enc on Intel/AMD Linux, or x264enc for CPU encoding. On Jetson, nvv4l2h264enc selects the hardware encoder shown above. To produce H.265, choose the corresponding H.265 encoder and set the output codec accordingly.

Rust SDK

Install the video-enabled SDK:

cargo add adamo --features video

use adamo::{Protocol, Robot, VideoBackend, VideoOptions};

fn main() -> adamo::Result<()> {
    let mut robot = Robot::new("ak_...", Some("my-robot"), Protocol::Quic)?;

    let options = VideoOptions {
        width: 1920,
        height: 1080,
        codec: "h264".into(),       // output sent through Adamo
        bitrate_kbps: 4000,
        fps: 30,
        backend: VideoBackend::HwPipeline,
        passthrough: false,
        ..VideoOptions::default()
    }
    .with_source_format("h265");    // input in shared memory

    robot.attach_shm_with_options(
        "front",
        "camera/front/encoded",
        &options,
    )?;
    robot.run()
}

VideoBackend::GStreamer selects the GStreamer path instead. The native hardware pipeline currently uses the host ffmpeg executable to decode encoded shared-memory input, so install FFmpeg and ensure it is on PATH. The GStreamer path needs the parser and decoder plugins for the input codec.

To pass an encoded stream through without transcoding, set passthrough: true and make the output codec match the input. For example, H.265 passthrough uses source_format: h265 and an H.265 codec/encoder tag. No decoder or encoder runs in passthrough mode.

Republish SHM frames to ROS 2

You can also bridge the same iceoryx2 service into ROS 2. This is useful when Adamo is the low-latency video transport, but another local ROS node still needs the raw camera stream.

The flow is:

1. A camera-specific producer publishes raw frames to iceoryx2.
2. A ROS republisher subscribes to that SHM service.
3. The republisher copies each raw frame into a sensor_msgs/Image message.

This single example assumes 1280x720 BGRA frames from camera/zed/left. For other camera types, change the constants at the top to match your producer.

source /opt/ros/humble/setup.bash
python shm_to_ros.py

shm_to_ros.py

import ctypes
import time

import iceoryx2 as iox2
import rclpy
from sensor_msgs.msg import Image

SERVICE = "camera/zed/left"
TOPIC = "/zed/left/image_raw"
WIDTH = 1280
HEIGHT = 720
ENCODING = "bgra8"
BYTES_PER_PIXEL = 4
FRAME_ID = "zed_left_camera"

EXPECTED_SIZE = WIDTH * HEIGHT * BYTES_PER_PIXEL
STEP = WIDTH * BYTES_PER_PIXEL

iox_node = iox2.NodeBuilder.new().create(iox2.ServiceType.Ipc)
service = (
    iox_node.service_builder(iox2.ServiceName.new(SERVICE))
    .publish_subscribe(iox2.Slice[ctypes.c_uint8])
    .enable_safe_overflow(True)
    .subscriber_max_buffer_size(2)
    .open_or_create()
)
subscriber = service.subscriber_builder().create()

rclpy.init()
node = rclpy.create_node("shm_image_republisher")
publisher = node.create_publisher(Image, TOPIC, 10)
node.get_logger().info(
    f"Republishing SHM '{SERVICE}' to ROS topic '{TOPIC}' "
    f"as {WIDTH}x{HEIGHT} {ENCODING}"
)

try:
    while rclpy.ok():
        sample = subscriber.receive()
        if sample is None:
            rclpy.spin_once(node, timeout_sec=0.0)
            time.sleep(0.0005)
            continue

        try:
            payload = bytes(sample.payload())
        finally:
            delete = getattr(sample, "delete", None)
            if delete is not None:
                delete()

        if len(payload) != EXPECTED_SIZE:
            node.get_logger().warn(
                f"dropping SHM sample: expected {EXPECTED_SIZE} bytes, "
                f"got {len(payload)}"
            )
            continue

        msg = Image()
        msg.header.stamp = node.get_clock().now().to_msg()
        msg.header.frame_id = FRAME_ID
        msg.height = HEIGHT
        msg.width = WIDTH
        msg.encoding = ENCODING
        msg.is_bigendian = 0
        msg.step = STEP
        msg.data = payload
        publisher.publish(msg)
        rclpy.spin_once(node, timeout_sec=0.0)
except KeyboardInterrupt:
    pass
finally:
    node.destroy_node()
    rclpy.shutdown()

Publish frames yourself

The SDK gives you a track handle; you push raw pixel buffers into it. Adamo allocates a private iceoryx2 service for the track and routes the frames into the encoder.

Python

import cv2
import adamo

robot = adamo.Robot(api_key="ak_...", name="my-arm")
track = robot.video("overlay", width=1280, height=720, pixel_format="BGRA",
                    fps=30, bitrate_kbps=4000)

cap = cv2.VideoCapture(0)
while True:
    ok, frame = cap.read()
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2BGRA)
    track.send(frame)

The encoder pipeline auto-starts on the first track.send().

track.send() is for fixed-size raw pixel buffers. For variable-size MJPEG frames, publish complete JPEG frames to an iceoryx2 service and consume that service with attach_video(..., shm=..., pixel_format="mjpeg").

Rust

use adamo::Robot;

fn main() -> adamo::Result<()> {
    let mut robot = Robot::new_default("ak_...", Some("my-arm"))?;
    let mut track = robot.video("overlay", 1280, 720, "BGRA", 30, 4000, false)?;
    std::thread::spawn(move || {
        loop {
            let frame: Vec<u8> = render_next_frame();  // your code
            track.send(&frame).unwrap();
        }
    });
    robot.run()
}

C

adamo_video_track_t *t = adamo_robot_video(
    robot, "overlay", 1280, 720, "BGRA", 30, 4000, false);
// push frames in a loop:
adamo_video_track_send(t, frame_bytes, frame_len);

Supported data types

Format	Payload size	Notes
BGRA, RGBA, BGRX, RGBX	4 bytes per pixel	Most convenient for OpenCV / numpy.
RGB, BGR	3 bytes per pixel
YUY2, UYVY	2 bytes per pixel
I420, NV12	1.5 bytes per pixel	Most efficient raw formats for the encoder.
MJPEG, MJPG, JPEG, image/jpeg	Variable	Compressed Motion JPEG. Each source sample must contain one complete JPEG frame. Supported for V4L2, GStreamer, and consume-side SHM sources.
H264, AVC, video/h264	Variable	One complete Annex-B H.264 access unit per SHM sample. Transcoded unless passthrough is enabled.
H265, HEVC, video/h265	Variable	One complete Annex-B H.265 access unit per SHM sample. Transcoded unless passthrough is enabled.

MJPEG sources are decoded and re-encoded into the configured output codec. On Jetson, Adamo avoids the Jetson nvjpegdec path for non-16-aligned MJPEG dimensions because that decoder can corrupt the bottom of frames; the stream still uses the Jetson hardware H.264 encoder after decode.

GStreamer Pipelines

Use a GStreamer pipeline when the camera is best accessed through a GStreamer element, such as zedsrc, nvarguscamerasrc, a hardware decoder, or another custom source. In Python, pass a source pipeline string with pipeline=:

import adamo

robot = adamo.Robot(api_key="ak_...", name="my-robot")
robot.attach_video(
    "main",
    pipeline=(
        "videotestsrc is-live=true "
        "! capsfilter caps=video/x-raw,format=BGRA,width=1280,height=720,framerate=30/1"
    ),
    width=1280,
    height=720,
    fps=30,
    pixel_format="BGRA",
    encoder="nvv4l2h264enc",
    bitrate_kbps=4000,
)
robot.run()

The pipeline string should describe a source bin, not a full Adamo output pipeline. It should end in raw video frames, or image/jpeg frames if you set pixel_format="mjpeg". Do not include Adamo’s encoder, parser, network sink, or final appsink; the SDK adds its own queue, hardware encoder, parser, and transport publisher.

Use explicit capsfilter caps=... elements rather than shorthand caps syntax. For example, use this:

zedsrc stream-type=2 camera-resolution=3 camera-fps=60 ! capsfilter caps=video/x-raw,format=BGRA,framerate=60/1

not this:

zedsrc stream-type=2 camera-resolution=3 camera-fps=60 ! video/x-raw,format=BGRA,framerate=60/1

The caps should match the frame layout you pass to attach_video: pixel format, width, height, and frame rate. If the GStreamer source negotiates BGRA 1280x720@30, pass pixel_format="BGRA", width=1280, height=720, and fps=30.

Reading from another pipeline’s tee

A GStreamer tee pad belongs to the pipeline that created it; a separate Adamo process cannot attach to that pad directly. If another process already owns the camera pipeline, tee one branch to a GStreamer shmsink, bridge that socket into an iceoryx2 service, then attach Adamo with shm=.

Producer pipeline branch:

... ! tee name=t \
  t. ! queue ! existing_sink \
  t. ! queue leaky=downstream max-size-buffers=1 \
     ! capsfilter caps=video/x-raw,format=BGRA,width=1280,height=720,framerate=30/1 \
     ! shmsink socket-path=/tmp/adamo-main.sock wait-for-connection=false sync=false

Bridge the GStreamer socket to an iceoryx2 video service:

target/release/shm-publisher --source gstreamer \
  --pipeline "shmsrc socket-path=/tmp/adamo-main.sock is-live=true do-timestamp=true ! capsfilter caps=video/x-raw,format=BGRA,width=1280,height=720,framerate=30/1" \
  --service camera/main

Then publish that service through Adamo:

import adamo

robot = adamo.Robot(api_key="ak_...", name="my-robot")
robot.attach_video(
    "main",
    shm="camera/main",
    width=1280,
    height=720,
    fps=30,
    pixel_format="BGRA",
    encoder="nvv4l2h264enc",
    bitrate_kbps=4000,
)
robot.run()

The raw buffers crossing the GStreamer shared-memory socket do not include a self-describing frame layout. Make sure the format, width, height, and framerate caps after shmsrc describe exactly what the tee branch writes into shmsink. The SDK width/height/fps/pixel_format values must describe that same layout.

Avoid passing shmsrc ... ! capsfilter ... directly as pipeline= for live external tee sources. The SDK currently probes GStreamer source caps before starting the real track, and live external shmsrc sources can stall there before video/<track>/alive is declared. The shmsink → shm-publisher → SDK shm= path above is the reliable path today.

Tuning Quality

Two knobs cover almost everything:

• bitrate_kbps — higher means better picture and more bandwidth. Start at 2000–4000 kbps for 720p, 6000–8000 for 1080p.
• fps — 60 if your camera supports it, 30 otherwise. On constrained links drop to 15.

If video looks blocky, raise the bitrate before anything else. If bandwidth is the bottleneck, lower fps first.

Encoder selection

Adamo picks the best available hardware H.264 encoder for the host at startup:

Host	Encoder used
Jetson (Orin, Thor, …)	On-chip NVENC
x86 with NVIDIA discrete GPU	NVENC
Intel / AMD integrated graphics	VA-API
macOS	VideoToolbox
Anything else	CPU (x264)

To see what the SDK picked, check the log line printed at connect.