Observations and actions

Observation

Observe.observation is an Observation:

Field	Meaning
arms[]	One ArmState per commanded arm
images[]	One Image per negotiated camera stream
flags	StationFlags
t_capture_ns	Station monotonic time the arm states were read

ArmState

Per arm, joints in driver order. Which fields are filled follows the negotiated proprio set; request what you need in the manifest.

Field	Proprio field	Meaning
name		Arm name
joint_pos_rad[]	JOINT_POS	Absolute joint positions
joint_vel_rad_s[]	JOINT_VEL	Joint velocities
joint_torque_nm[]	JOINT_TORQUE	Measured joint torques
gripper_pos	GRIPPER_POS	Opening in [0, 1]; absent on an arm without a gripper
gripper_eff_nm	GRIPPER_EFF	Gripper effort
tcp_pose	TCP_POSE	Pose{frame_id, position_m[3], orientation_xyzw[4]} from forward kinematics, base frame
last_target_joint_pos_rad[], last_target_gripper_pos	LAST_TARGET	The target the station actually applied at capture time

Image

Field	Meaning
camera	Camera name
width, height	Size after the station's downscale, at most what was negotiated
encoding	IMAGE_ENCODING_JPEG
data	Encoded bytes
t_receive_ns	Receive time on the station's monotonic clock, not the shutter instant
seq	Frame counter

Images are always JPEG. The station never upscales.

StationFlags

Field	Meaning
authority	POLICY, HOLD, HOMING, OPERATOR_PAUSE, or NONE
hold_active	The arms are holding rather than following your targets
paused	The operator has paused
steps_since_valid_chunk	Consecutive steps without an accepted chunk

Actions

A Chunk holds 1 to negotiated.max_horizon Actions. Action k is the target at time t₀ + k·dt, where t₀ is when the station accepts the chunk and dt = 1 / negotiated.step_hz. Each Action holds one ArmAction per commanded arm.

joint_position_v1

The schema the station executes today.

message JointPositionAction {
  string arm = 1;
  repeated double joint_pos_rad = 2;   // absolute, driver joint order
  optional double gripper_pos = 3;     // absolute opening in [0, 1]
}

Build one time step with the helper:

policy.joint_action({"follower_l": (q_left, 1.0), "follower_r": (q_right, None)})

A None gripper leaves the opening unchanged.

tcp_delta_v1

Reserved. TcpDeltaAction{arm, translation_m[3], rotation_axis_angle_rad[3], gripper_pos} exists in the proto and is refused at negotiation until the station carries an inverse-kinematics term.

How the station executes a chunk

Each control cycle the station samples a time-indexed buffer at the current time by linear interpolation between the bracketing targets. A newer chunk overwrites every entry at or after its first offset. Past the last entry the arm holds it. Past the operator's hold grace period without a new entry the arms walk to the station's home pose, a rest pose away from the scene, and the episode is invalidated with HOLD_EXPIRED. Every episode ends with that walk; the arms return to the start pose before the next one begins.

At the start of each episode and after any hold, the first accepted chunk is entered through a bring-up ramp so the arm walks onto your first target instead of snapping to it.

Safety checks

Before a chunk is buffered, every action is checked per arm, in this order:

1. Finite check. Any NaN or infinity rejects the chunk.
2. Joint limit clamp, from the arm's limits in the station manifest.
3. Joint step limit. The first action is measured from the arm's measured joint position at submission, so a chunk that starts from the observation you were given always passes this rule; each later action is measured from the one before it.
4. Speed limit along the resulting path.
5. Workspace box on the forward-kinematics TCP.
6. Collision query against the other arm's target at the same offset.

A clamp keeps the action, limited, with the verdict CLAMPED. A rejection drops the whole chunk with the rule that fired. Both are counted in the episode manifest. A rejected chunk is recorded as a missed step. Consecutive missed steps beyond the station's budget invalidate the episode with MISS_BUDGET. The limit values are configured per task by the operator. The station manifest in Welcome carries the joint ranges, and the steps_since_valid_chunk field in every observation reports the current run of steps without an accepted chunk.

During execution the station also watches measured joint torque. Sustained excess holds the arms and invalidates the episode with TORQUE_CEILING.