MAMBA4D point cloud spatio-temporal scanning strategy on edge embedded device
Amizzuddin Amin Chan · DR-NTU (Nanyang Technological University) · 2026
This report presents a comprehensive technical investigation into the feasibility and efficiency of deploying the UST-SSM (Unified Spatio-Temporal State Space Model) — a Mamba-based deep learning architecture for 4D point cloud action recognition — on the NVIDIA Jetson AGX Xavier embedded edge device. The work spans two Git repositories: (1) sc4079, the main inference repository containing the Dockerfile, Docker Compose stack, Visual Studio Code DevContainer configuration, UST-SSM model code, ROS2 “action_recognition” package, and training pipeline; and (2) RealSense, a separate private repository containing the RealSense driver, quality of service (QoS) relay node, and monitoring tools for the Intel RealSense D435 depth camera. The sc4079 Dockerfile parametrically targets two architectures: x86-64 and ARM64 AGX Xavier. Both causal-conv1d and mamba-ssm require source-compilation with CUDA version guards and Triton dependency bypassed via sed patching. The CI/CD pipeline builds AMD64 images on standard GitHub Actions runners and cannot use QEMU for the AGX Xavier image due to Tegra-specific CUDA device dependencies; AGX Xavier images are built locally using the run script. The RealSense repository uses the standard multi-architecture image, enabling QEMU-based multi-arch CI/CD builds. Its Dockerfile default CMD runs rs_relay.launch.py via command.sh, but in the sc4079 dev_compose.yml the camera service overrides this command to launch only "ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true", starting the RealSense D435 driver at 640x480 6 FPS with PointCloud2 enabled and no QoS relay. The realsense2_camera node publishes PointCloud2 directly on /camera/depth/color/points with RELIABLE QoS. A monitoring_tool package provides topic_monitor (BEST_EFFORT rate measurement tool), image_decompressor, camera_info_relay, and pointcloud_constructor for debugging multi-machine communication when the image compression relay mode is active. Cyclone DDS is configured with disable "AllowMulticast" and unicast peer discovery to accommodate the TP Link HB810 mesh network where IP multicast is blocked. The UST-SSM model (~0.64 M parameters) was successfully deployed on AGX Xavier in float32 precision with the inference node configured at 2 Hz inference rate. No formal GPU benchmarking was conducted; runtime performance characterisation remains as future work.