236 Figure 1 – Experimental setup for autonomous dump pocket cleaning using imitation learning. Top left: teleoperated dataset (162 demonstrations). Top right: physical testbed with dual RGB cameras (top-view, wrist-mounted), dump pocket, and SO-ARM100 platform (follower and leader arms). Bottom: imitation learning pipeline receiving observations (camera streams, task prompt*, proprioceptive state), processing through policy network, and executing actions in closed loop. *Task prompt: “use the scoop-like end effector to gather material from the dump pocket and deposit it into the primary crusher at the center”, used only by VLA models (π0.5, SmolVLA) for language conditioning; ACT and Diffusion operate on vision and proprioception only. of the dump pocket, and deposits the material. This cycle repeats continuously to maximize removal rate. Success criteria: We define success operationally as (i) fully autonomous operation without human intervention during the 10-minute session, (ii) no safety-limit violations, and (iii) measurable material removal. All methods start from the same initial state (3 kg material mass), and the primary performance metric is total material removed (g/10 min). 3.2. Observations and Action Space We define the observation ot as the concatenation of two RGB streams and proprio- ception: • Wrist camera: 2MP camera (30 FPS) mounted on the follower arm end-effector (bucket), providing close-range perspective of material and manipulation area. • Overhead camera: Webcam providing global context view of the dump pocket andarm workspace. • Proprioception: Joint angles (6-DoF from STS3215 servos) and gripper state, syn- chronized with camera timestamps at 30 Hz. • Task prompt (VLA only): Natural language instruction: “use the scoop-like end effector to gather material from the dump pocket and deposit it into the primary crusher at the center”, used by π0.5 and SmolVLA for multimodal conditioning; ACT and Diffusion do not use language input. The action at is a vector of joint-position targets qtarget predicted by the policy and executed by the low-level servo controller. Actions are produced at 30 Hz control frequency. ACT uses action chunking with horizon H = 100 steps; π0.5, SmolVLA, and Diffusion Policy operate in receding-horizon mode with model-specific temporal windows as configured in the training library [9]. Table 1 summarizes the platform specifications and dataset statistics. Table 1 – Platform and dataset summary. Platform Dataset SO-ARM100, 6-DoF, bucket 0–18 g 162 eps (129/33 split), 73,944 frames Sub-$250, leader-follower 30 FPS, mean 15.18 s/ep (range 10.4–26.6 s) Dual RGB (wrist + overhead) 40.99 min total, expert rate 62.04 g/min 3.3. Models The control is governed by four imitation learning paradigms: ACT (Action Chunk- ing with Transformers) uses a CVAE to predict action chunks trained from scratch with an L1 + λ KL objective [7,
RkJQdWJsaXNoZXIy MTM0Mzk2