Lesson 2.3.4 — In-Hand and Dexterous Manipulation

Week 15

Prerequisite lessons: 2.3.1 (grasp synthesis and quality), 2.3.2 (motion planning for manipulation), 2.3.3 (contact-rich manipulation), 2.2.3 (offline RL), 2.2.4 (offline-to-online fine-tuning), 2.1.5 (visual SLAM and odometry).

Prior results used in this lesson. From 2.3.1: force closure and the Ferrari–Canny $\varepsilon$ metric. From 2.3.3: the friction cone, impedance control, the momentum observer, and residual RL. From 2.2.3/2.2.4: the offline pretraining and online fine-tuning pipeline with the Q-shield. From 2.1.5: the eye-in-hand pose estimate and the kinematic prior. Each is restated at the point of use.

What this lesson hands forward. The grasp-matrix / hand-Jacobian formalism, the internal-force decomposition, and the teacher–student distillation pattern. Module 3.4 (sim-to-real) uses the distillation pattern as its template for privileged-information transfer; Module 4.2 uses the internal-force decomposition as the safety layer for multi-contact manipulation.

The running case in one sentence. The running arm, camera, table, cube, bin, and divider are unchanged, but for this lesson the parallel-jaw gripper is replaced by a three-finger hand: three fingers with two joints each (six joints, fingerpads with $\mu = 0.5$, maximum fingertip force 15 N, fingertip tactile arrays at 100 Hz), mounted on the same end-effector flange. The task changes with the hand: the cube must be picked up from $T_W^{Obj} = (R_z(\pi/4), [0.5, 0.2, 0.3])$, reoriented in the hand to a face-aligned orientation, and then placed in the bin.

§L.0 — Lesson map, objectives, and misconceptions

Observable outcomes. By the end of this lesson you can, without notes: (1) count the degrees of freedom of a hand and compare them with the controllable object DOF; (2) define underactuation and give the running case’s count; (3) write the hand Jacobian and the grasp matrix and state the velocity/force duality between them; (4) decompose contact forces into external and internal components and explain why internal forces are free; (5) solve the internal-force distribution as a quadratic program with friction-cone constraints; (6) write the rolling-contact kinematics of two smooth bodies; (7) compute a manipulability measure and diagnose a singular hand posture; (8) describe tactile servoing as a force-error-driven motion law; (9) design a teacher–student distillation for a dexterous policy with privileged state; (10) budget a 90° in-hand reorientation and integrate it into the 1.1.4 pipeline.

Misconception inventory.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

Q1. In 2.3.1, a grasp is force-closed when every external wrench can be resisted by contact forces inside the friction cones. What does that say about the dimension of the space of resistible wrenches?

Answer. It says the image of the cone-admissible force set under the grasp matrix is all of $\mathbb{R}^6$, i.e., dimension six. Used in §L.A.2 to separate force closure from manipulability.

Q2. In 2.3.3, the friction cone is $|\lambda_t| \le \mu\lambda_n$. Linearized, it becomes $\lambda_t = \sum_i \beta_i d_i$ with $\sum_i \beta_i \le \mu \lambda_n$. Which of these is a linear constraint?

Answer. The linearized form is a set of linear inequalities in $(\beta_i, \lambda_n)$, so it defines a polyhedral cone; the exact form defines a second-order cone. Used in §L.B.2 to formulate the internal-force QP.

Q3. In 2.3.3, the momentum observer converges to $J^\top F_{ext}$ without a wrist force sensor. For a three-finger hand, what does the finger’s joint-torque observer converge to, and why is that useful?

Answer. It converges to the fingertip contact force mapped through that finger’s Jacobian; with per-finger observers the hand can estimate the full contact-force vector from joint torque alone. Used in §L.C.2 as the fallback when tactile sensors saturate.

Q4. In 2.2.4, the fine-tuning pipeline keeps a Q-shield at runtime and tracks trough depth and length. Why is a runtime shield more important for a dexterous policy than for a reaching policy?

Answer. Because a bad dexterous action can drop or crush the object, whereas a bad reaching action usually just misses; the cost of an out-of-distribution action is higher, so the shield is worth its conservatism. Used in §L.D.2 when the learned reorientation policy is deployed.

Part A — The dexterity problem

§L.A.1 — Counting degrees of freedom honestly

A parallel-jaw gripper has one actuated DOF and grasps with two contacts. The three-finger hand of the running case has six actuated joints and grasps with three contacts. Naively, six joints against a six-DOF object sounds like a perfect match. It is not, and the reason is the central idea of dexterous manipulation.

Write $n_q$ for the number of actuated joints, $n_c$ for the number of contact constraints, and $k$ for the number of independent scalar constraints each contact imposes. For a point contact with friction (the model of 2.3.3), the contact constrains the relative velocity to zero in all three directions, so $k=3$ and the total number of scalar contact constraints is $3 n_c$. For a soft-finger contact, add one torsional constraint, so $k = 4$.

The object has six DOF. The hand has $n_q$ DOF. At a grasp where all contacts are maintained, the coupled system’s instantaneous motion satisfies

where $v_{obj} \in \mathbb{R}^6$ is the object twist, $\dot q \in \mathbb{R}^{n_q}$ the joint velocities, $J \in \mathbb{R}^{3n_c \times n_q}$ the hand Jacobian (stacking each finger’s contact-point Jacobian), and $G \in \mathbb{R}^{6 \times 3n_c}$ the grasp matrix. Equation (A.1) is the velocity-level statement that the contact points move with the object.

The dimension count follows. The combined velocity space has dimension $6 + n_q$. The constraints impose $\operatorname{rank}(J)$ independent scalar equations, so the space of compatible motions has dimension

Of those, at most six can move the object independently (the object has six DOF); the remaining $d - 6$ are internal motions that change the hand configuration without moving the object — finger repositioning, rolling, and regrasping.

Underactuation is the gap between the object’s DOF and what the hand can independently control at a given instant. The number of controllable object DOF is

For the three-finger hand with $n_q = 6$ and three point contacts, $J$ is $9 \times 6$, so $\operatorname{rank}(J) \le 6$. Then $d = 6 + 6 - 6 = 6$ when $J$ is full rank, and the hand has no internal motion at a fully constrained grasp. That is the fundamental problem: with six joints and three frictional contacts, the hand is exactly actuated at best and cannot roll the object without breaking and re-making contact. Real dexterous hands have $n_q \ge 9$ (three fingers with three joints, or five fingers with four joints), precisely to buy internal DOF.

This is the first misconception in numbers. Adding a finger adds three contact constraints and (say) two joints: the net change in the internal-motion dimension is $2 - 3 = -1$ unless the new finger’s Jacobian adds rank. Dexterity comes from excess actuation relative to contact constraints, not from contact count.

Self-explanation 1. A hand has 9 joints and three soft-finger contacts. Compute the internal-motion dimension when the hand Jacobian is full rank.

Answer. $k=4$ per contact, so $J$ is $12 \times 9$ with rank at most 9; $d = 6 + 9 - 9 = 6$, so there are zero internal motions at a fully constrained grasp. Adding one more joint gives $d=7$ and one internal DOF.

Concept check. Two hands both have 12 joints and three contacts. Hand A has all 12 joints on the fingers; hand B has 9 on the fingers and 3 on a wrist that moves the whole hand. Which has more in-hand dexterity, and what does the wrist DOF buy that the finger DOF does not?

Answer. Hand A, because wrist DOF move the object in the world but not relative to the palm; in-hand dexterity requires motion of the object relative to the hand. The wrist buys reach and orientation, not regrasping. [displaces M1]

§L.A.2 — The grasp matrix and the hand Jacobian are duals

The grasp matrix and the hand Jacobian are related but distinct, and confusing them is the second misconception.

The hand Jacobian $J$ is a kinematic object: it maps joint velocities to contact-point velocities,

Each block $J_i$ is the Jacobian of contact point $i$ expressed in the world frame: $J_i = [\, I \;\; -\widehat{p_i} \,] \, T_W^{E_i} \, J^{E_i}(q)$ where $p_i$ is the contact point and $J^{E_i}$ the finger’s end-effector Jacobian.

The grasp matrix $G$ is a static object: it maps contact forces to the net wrench on the object,

The duality is the principle of virtual work. The power delivered by contact forces through contact velocities equals the power delivered by the net wrench through the object twist:

Since this holds for all $f$, the kinematic compatibility condition is Eq. (A.1): $G^\top v_{obj} = J\dot q$. The two matrices are transposes of each other only in the sense that they appear transposed in the dual pairing; they have different dimensions and different physical meaning.

The second duality is the force relation. Let $\tau$ be the joint torques and $f$ the contact forces. Virtual work again:

Equation (A.7) is the mechanism by which a dexterous hand feels: the joint torques required to hold a contact force are $J^\top f$, and the momentum observer of 2.3.3 recovers exactly this.

Worked-example ladder.

WE-L.A.a (fully worked). Three point contacts on the running case’s 5 cm cube at $p_1 = [0.025, 0, 0]$, $p_2 = [-0.025, 0, 0]$, $p_3 = [0, 0.025, 0]$ in the cube frame. Write $G$ and compute its rank.

Solution. Each block is $3 \times 3$. The first three rows are $[I\ I\ I]$, rank 3. The last three rows are $[\widehat{p}_1\ \widehat{p}_2\ \widehat{p}_3]$. With $\widehat{p}_1 = [[0,0,0],[0,0,-0.025],[0,0.025,0]]$, $\widehat{p}_2 = -\widehat{p}_1$, and $\widehat{p}_3 = [[0,0,0.025],[0,0,0],[-0.025,0,0]]$, the moment rows span three dimensions. The full matrix has rank 6, so the grasp can, in principle, generate any wrench. Force closure additionally requires the signs to be admissible, which the friction cone decides.

WE-L.A.b (completion). Now place the three contacts all on the top face of the cube: $p_i = [x_i, y_i, 0.025]$ with distinct $(x_i, y_i)$. Compute $\operatorname{rank}(G)$ and explain the physical consequence.

Completion. The moment blocks $\widehat{p}_i$ all have the form $[[0, -z_i, y_i],[z_i, 0, -x_i],[-y_i, x_i, 0]]$ with $z_i = 0.025$; the $x$ and $y$ columns depend only on $z_i$ and are therefore identical across contacts. The moment rows span at most rank 2, so $G$ has rank at most 5. Physically, the grasp cannot generate a moment about the top-face normal: the cube can spin freely about $z$. The completion shows that contact placement, not contact count, decides grasp rank.

WE-L.A.c (faded). Derive Eq. (A.2) from Eq. (A.1) by counting dimensions, and specialize to the running case with $n_q = 6$, $n_c = 3$, $k = 3$. Faded step: the combined velocity has $6 + n_q$ components; the constraint is $\operatorname{rank}(J)$ independent equations; subtract. Completion: $d = 6 + 6 - 6 = 6$, so there are zero internal motions at a full-rank grasp. To reorient the cube, the hand must break the full-rank condition — lift one finger, roll on the other two — or the hand needs more joints.

WE-L.A.d (contrastive near-miss). A learner adds a fourth finger to the three-finger hand and expects the object to become easier to rotate in hand. In simulation, the object becomes harder to rotate, and the planner returns infeasible trajectories. Why it fails: the fourth contact adds three constraints but only two joints, so $d$ falls by one; the grasp is now over-constrained and the planner cannot find a rolling motion. The fix: either actuate the fourth finger fully (three joints per finger) or leave it passive/retracted. The near-miss teaches that contacts are constraints first and manipulation affordances second.

Comparative table: grasp and hand matrices.

Design principle P34.1. Count constraints against actuators before adding hardware. Dexterity is the dimension of the controllable space, $6 + n_q - \operatorname{rank}(J)$, and a finger that adds constraints faster than it adds actuation reduces it.

Consolidation checkpoint 1

(a) Three capabilities. You can compute $d$ from $n_q$, $n_c$, and $\operatorname{rank}(J)$. You can write $G$ for a given contact set and estimate its rank from the contact geometry. You can state the two dualities of Eq. (A.1) and Eq. (A.7).

(b) Transfer probe. A five-finger hand has 20 joints and grasps a sphere with five point contacts. Ignoring joint limits, what is $d$? Answer: $J$ is $15 \times 20$ with rank at most 15, so $d = 6 + 20 - 15 = 11$, giving five internal DOF for rolling and regrasping.

(c) Remediation rule. If you cannot state why adding a contact can reduce dexterity, re-derive Eq. (A.2) before Part B.

Part B — Contact kinematics, internal forces, manipulability

§L.B.1 — The null space of the grasp matrix

The wrench balance is

With $n_c = 3$ point contacts, $f \in \mathbb{R}^9$ and Eq. (B.1) provides six equations, so the solution set is a three-dimensional affine subspace:

The particular solution $f_p$ is the minimum-norm force distribution; the homogeneous term $f_0 \in \ker(G)$ is the internal force. Internal forces squeeze the object without changing its net wrench. They are free in the sense that the object’s motion does not determine them; they are not free physically, because they must stay inside the friction cones and below the actuators’ limits.

This is the third misconception in algebra. The object’s dynamics gives six numbers; the hand must supply nine. The three extra are a design choice, and choosing them badly is how hands crush objects and how hands drop them. Squeezing too hard saturates actuators and deforms the object; squeezing too little lets the object slip when an unexpected disturbance arrives.

The internal-force choice is a constrained optimization:

where $W$ weights the effort and $A_i$ encodes the linearized friction cone of Eq. (B.3) of 2.3.3. With the cone linearized by $m$ edges, Eq. (B.3) is a quadratic program with linear constraints, solvable at control rate (Boyd & Vandenberghe 2004, tier A, S; the formulation for grasping is in Bicchi & Kumar 2000, tier A, S).

A common practical choice is to parameterize the internal force by a scalar squeeze $s$, $f_0 = s \, n$, where $n$ is a unit vector in $\ker(G)$ pointing “inward” at every contact. Then the choice reduces to choosing $s$, which is exactly the grip-force budget of 2.3.3 generalized to multi-contact.

Self-explanation 2. Why can an internal force exist at all, given that the object is rigid and its wrench balance is fixed?

Answer. Because the contact forces are not uniquely determined by the object’s wrench; only their image under $G$ is. The null space of $G$ is the set of force patterns that cancel among themselves.

§L.B.2 — Rolling contact kinematics

The reorientation task requires the object to rotate relative to the palm. The cleanest way to do that without regrasping is rolling: the contact point moves continuously over both surfaces. The kinematics of rolling between two smooth bodies is given by Montana’s equations (Montana 1988, tier A, S).

Parameterize the finger surface by $(u_f, v_f)$ and the object surface by $(u_o, v_o)$. Let $R$ be the relative rotation between the contact frames. The contact-point velocities satisfy

where $M_f, M_o$ are the surface metric (first fundamental form) matrices, $\omega_x, \omega_y$ the tangential components of the relative angular velocity, and the no-slip condition forces the tangential velocity to zero. The normal component $\omega_z$ is the spin, which is resisted by torsional friction.

Equation (B.4) is the reason rolling is hard to control and easy to learn. Hard, because the map from relative twist to surface parameter rates is nonlinear and configuration-dependent; easy, because rolling is a continuous motion with no impacts, so a policy can discover it with gradient-based learning and a reward that asks for rotation.

For the running case’s three-finger hand, the canonical reorientation is a finger gait: two fingers hold the cube while the third lifts, moves to a new face, and re-establishes contact; then the roles rotate. Each gait cycle produces a finite rotation, typically 30–90°. This is discrete manipulation rather than continuous rolling, and it is what the running case uses because the hand’s six joints are exactly actuated (Part A) and cannot roll continuously.

Worked-example ladder.

WE-L.B.a (fully worked). For the three-contact grasp of WE-L.A.a with $w_{ext} = [0,0,-mg,0,0,0]^\top$ and $m = 0.1$ kg, compute $f_p = -G^+ w_{ext}$, then add an internal force and check the friction cones.

Solution. $|w_{ext}| = 0.981$ N. With $G$ rank 6 and 9 columns, $G^+ = G^\top(GG^\top)^{-1}$. The minimum-norm solution distributes roughly $0.327$ N normal at each contact (depending on the moment arms). Adding $f_0 = s\,n$ with $n$ the inward squeeze direction raises each normal force by $s$ while keeping the net wrench fixed. With $s = 2$ N, each normal is about 2.33 N and the tangential components remain small, so the ratio $|f_t|/\mu f_n$ is well below 1.

WE-L.B.b (completion). Sweep $s$ from 0 to 5 N and find the smallest $s$ at which the grasp survives a 2 m/s² acceleration along $x$. Completion: the required total friction scales with $m(g+a)$, and the friction available scales with $s$ plus the particular solution; solving the inequality gives $s_{min} \approx 1.4$ N. The standard operating point of $s = 2$ N leaves a 40% margin.

WE-L.B.c (faded). Write the QP (B.3) with a linearized cone and solve it for the three-contact grasp with a 2 N squeeze target. Faded step: the decision variable is the 9-vector $f$; the constraints are the six equality rows $Gf = -w_{ext}$, the nine cone rows, and the 15 N actuator upper bound; the objective is $|f|^2$. Completion: the QP returns $f$ with all constraints active or slack, and the active set tells you which contact is at its friction limit.

WE-L.B.d (contrastive near-miss). A learner commands a large positive $s$ to “be safe” and observes that the cube is crushed by 0.3 mm and the fingers saturate at 15 N. Why it fails: the internal force is not free in the actuator sense; the joint torques are $J^\top f$ and scale with $s$, and the finger’s mechanical advantage is finite. The fix: solve the QP with actuator limits, or parameterize $s$ by the actuator’s remaining torque budget rather than a fixed number.

Comparative table: internal-force choices.

Design principle P34.2. The object’s wrench balance fixes six numbers; the hand must choose the rest. Make that choice an explicit optimization with friction and actuator constraints, never a default.

Consolidation checkpoint 2

(a) Three capabilities. You can write the affine solution set of the wrench balance. You can state Montana’s rolling equations and identify the no-slip and spin terms. You can set up the internal-force QP.

(b) Transfer probe. A four-contact grasp has $f \in \mathbb{R}^{12}$ and a rank-6 grasp matrix. How many internal-force dimensions are there? Answer: six, so the internal-force choice is six-dimensional and the QP is genuinely necessary.

(c) Remediation rule. If you cannot explain why the minimum-norm solution is not automatically the right one, re-read §L.B.1 before Part C.

Part C — Planning and control for dexterous manipulation

§L.C.1 — Tactile servoing

Tactile servoing is the dexterous analogue of visual servoing (2.1.3). The controller drives a measured tactile signal to a desired one by moving the fingers, without an explicit model of the contact. The simplest form is

where $f_{meas}$ is the vector of fingertip force measurements, $f^{*}$ the target, $J_t$ the tactile-to-joint Jacobian (the map from joint velocity to change in measured tactile signal), and $K$ a gain. Like visual servoing, Eq. (C.1) closes the loop on the current measurement and does not integrate, so it does not drift.

The gain $K$ must be small enough that the loop bandwidth is well below the tactile sampling rate (100 Hz in the running case) and the contact dynamics (Part A of 2.3.3). With $K = 0.02$ m/N and a force error of 0.5 N, the commanded motion is 1 cm, which at a 10 ms control period is a velocity of 1 m/s — too fast. The gain must be expressed in velocity units: $\dot q = K_v J_t^{+} (f^* - f)$ with $K_v$ in m/(N·s).

Tactile servoing has three properties that matter for dexterity. First, it regulates contact force without a force/torque sensor. Second, it works on arbitrary contact geometry because $J_t$ is estimated online from the response of the signal to small motions. Third, it is robust to tactile sensor calibration drift if the target is expressed relative to the current reading rather than absolutely.

§L.C.2 — Finger gaiting and pivoting

The running case’s hand cannot roll continuously (Part A). It reorients by finger gaiting: a sequence of discrete grasp configurations, each connected to the next by a lift–move–place motion of one finger.

A gait cycle for a 90° reorientation has four phases:

1. Hold. Two fingers hold the cube; the third is in contact.
2. Release. The third finger lifts by 1 cm, breaking its contact. The momentum observer detects the release as a step in the residual.
3. Transfer. The third finger moves to a new face of the cube; the other two rotate the cube by 30° using a small rolling motion of their contact patches, which the soft finger pads permit even though the joint count is exact.
4. Regrasp. The third finger closes on the new face; the internal-force QP is re-solved; the squeeze returns to its nominal value.

Three cycles give 90°. The cycle time is dominated by the lift–move–place trajectory, not by the rotation itself: about 0.4 s per cycle, so 1.2 s for 90°. The planning problem is to choose the contact locations and the rotation per cycle so that the cube’s final orientation is exact and every intermediate grasp is force-closed with margin.

Model-predictive control over the gait is the natural formulation: a short-horizon optimization over the finger trajectories with the contact-mode sequence fixed by the gait planner, and the internal forces from the QP. The MPC cost penalizes orientation error and force-limit violation, and the horizon must cover at least one gait cycle so that the terminal state is a valid regrasp.

Self-explanation 3. Why does the running case’s hand need finger gaiting when a three-fingered hand with soft pads can, in principle, roll?

Answer. Because rolling requires the contact points to move over the surfaces, which consumes the internal DOF that the exact-actuation count does not provide; the soft pads allow a small roll but not a sustained one.

Concept check. A tactile-servo controller has $K_v = 0.05$ m/(N·s) and a force error of 2 N. What is the commanded fingertip velocity, and is it safe against a 15 N actuator limit?

Answer. 0.1 m/s. Safety is not about velocity but about the force the motion produces: against a contact stiffness of $10^5$ N/m, 0.1 m/s over a 10 ms period produces a 100 N force spike, so the gain is unsafe. The mechanism is the product of velocity, stiffness, and period. [displaces M4]

Worked-example ladder.

WE-L.C.a (fully worked). A gait cycle rotates the cube 30°. The cube’s half-width is 2.5 cm. Compute the arc length that the contact patch must roll over each finger and the number of cycles for 90°.

Solution. Rolling a cube by 30° about a vertical axis through its centre moves each contact point along an arc of radius 2.5 cm by $2.5 \times (\pi/6) = 1.31$ cm. Three cycles give 90°. The soft pad must accommodate 1.31 cm of tangential motion, which is within the pad’s 2 cm compliance.

WE-L.C.b (completion). Suppose each cycle rotates 25° instead of 30°. How many cycles for 90°, and what is the residual error if the cycles are integer? Completion: 4 cycles give 100°, overshooting by 10°; 3 cycles give 75°, undershooting by 15°. The planner must therefore either allow a final partial cycle or choose a per-cycle angle that divides 90° exactly (22.5° for four cycles).

WE-L.C.c (faded). Write the MPC cost for the gait with horizon one cycle: penalize terminal orientation error, peak normal force, and cycle time. Faded step: $J = |R_{final} - R_{target}|_F^2 + \lambda \max(0, f_{max} - f_{limit})^2 + \gamma T_{cycle}$. Completion: with $\lambda$ large, the optimizer chooses slower cycles that keep forces low; with $\gamma$ large, it chooses faster cycles that approach the force limit.

WE-L.C.d (contrastive near-miss). A learner runs tactile servoing with a target force of 5 N and observes the cube being ejected from the hand. Why it fails: the tactile target was applied to all three fingers simultaneously while the object was free, so the net force was unbalanced and the cube was pushed out. The fix: servo forces only in the null space of $G$ — that is, servo the internal force, not the net wrench — or hold the object with a fixed external force while servoing the squeeze.

Comparative table: dexterous control strategies.

Design principle P34.3. Servo the internal force, not the net wrench. The net wrench is determined by the task; only the null-space component is available for control.

Consolidation checkpoint 3

(a) Three capabilities. You can write the tactile-servo law and choose its gain in velocity units. You can lay out a four-phase gait cycle and compute the per-cycle rotation. You can write the MPC cost for a gait.

(b) Transfer probe. The cube’s face is slippery, $\mu = 0.2$. What changes in the gait? Answer: the per-cycle rotation must fall, because the tangential force that rolling requires scales with the normal force and the cone shrinks; more cycles are needed, and the squeeze $s$ rises.

(c) Remediation rule. If you cannot explain why servoing all three fingers to 5 N ejects the cube, re-read §L.C.1 before Part D.

Part D — Learning dexterous manipulation

§L.D.1 — What Dactyl showed

The landmark result is Dactyl (OpenAI et al. 2018, tier A, S), which learned in-hand manipulation of a cube on a Shadow hand using a combination of:

• Recurrent policies over proprioception and tactile sensing, so the policy can integrate contact history.
• Massive domain randomization of friction, object mass, actuator delay, sensor noise, and object shape.
• Distributed PPO (2.2.2) with tens of thousands of parallel environments.
• Automatic curriculum over goal difficulty: the target orientation is sampled from a distribution that widens as success improves.

The policies transferred to a real hand and could reorient the cube to arbitrary target orientations. Two follow-ups sharpened the picture. OpenAI et al. 2019 (tier A, S) solved a Rubik’s cube with a similar pipeline plus a vision-based cube-state estimator and a solver in the loop, demonstrating that the manipulation primitive and the reasoning layer can be separated. Andrychowicz et al. 2020 (tier A, S) showed that the learned primitives can be composed into a pick-and-place pipeline.

The architectural lesson is the one from 2.3.3 restated at larger scale: learning a residual or a primitive on top of a structured system beats end-to-end learning. The vision-based cube tracker, the kinematic model, and the state machine stayed; only the contact-rich control became learned.

§L.D.2 — Teacher–student distillation

Training directly on real hardware is too slow and too dangerous. Training directly on proprioception alone is too hard, because the policy must infer contact state from joint angles. The standard solution is a teacher–student architecture.

The teacher is trained in simulation with privileged information: full contact states, object pose, friction coefficient, and mass. It converges quickly because the hard inference problem has been removed. The student is trained to imitate the teacher’s actions from observations that are available on the real robot: proprioception, tactile signals, and camera images. The student loss is

where $s_t$ is the privileged state, $o_t$ the deployable observation, and $\mathcal{D}$ the distribution of states visited by the teacher. The critical detail is that $\mathcal{D}$ must be the teacher’s state distribution, not a random one: the student must learn to act on the states it will actually visit. This is the DAgger insight (Ross, Gordon, Bagnell 2011, tier A, S) specialized to distillation.

This pattern appears across embodied AI: Lee et al. 2020 (tier A, S) used it for quadruped locomotion with a privileged teacher and a proprioceptive student; Miki et al. 2022 (tier A, S) used it for perceptive locomotion with a depth-based student; and it recurs in Phase 3 for VLA policies that must run without privileged simulator state.

The teacher–student split also solves a safety problem. The teacher’s privileged state makes its actions well-conditioned; the student’s observation may be noisy, but the student is trained to reproduce the teacher’s actions on the teacher’s state distribution, so the composition is stable if the student’s error is small. The Q-shield of 2.2.4 catches the residual cases.

§L.D.3 — Sim-to-real for hands

Three parameters dominate the dexterous sim-to-real gap, and each has a specific treatment.

Contact parameters. Friction, stiffness, and damping vary by 20–50% between simulation and reality. Randomize them over the identification intervals, exactly as in 2.3.3. For the running case, $\mu \in [0.3, 0.7]$, contact stiffness $k_c \in [5\times10^4, 1.5\times10^5]$ N/m.

Actuator delay and dynamics. Real fingers have tendon compliance, backlash, and 5–20 ms of latency. Model the delay explicitly in simulation and randomize it; a policy trained without delay will oscillate at deployment. The passivity check: the closed-loop phase margin must exceed the delay’s phase lag at the loop crossover.

Object variability. The cube’s mass and inertia are known only approximately; randomize $\pm 20$. If the task involves multiple objects, randomize shape over the training set (OpenAI et al. 2018, tier A, S).

A parameter that is not on this list is sensor noise. Tactile sensors are noisy, but the noise is high-frequency and the policy integrates over time; randomizing noise helps less than randomizing delay. This is a case where intuition from vision (“add noise to images”) transfers poorly to touch.

Self-explanation 1. Why must the student be trained on the teacher’s state distribution rather than a uniformly sampled one?

Answer. Because the student only needs to be correct on the states the teacher visits; uniform sampling spends capacity on states the closed loop never reaches and underweights the boundary states where errors compound.

Concept check. A teacher policy is trained with full object pose and friction. The student is trained with only proprioception. On the real robot the student drops the cube during fast rotations. Name two mechanisms and the discriminating experiment.

Answer. (1) The student cannot infer the rotation state from proprioception alone, so its action lags; test by giving the student the object pose at deployment and observing whether the drops vanish. (2) The teacher’s state distribution under fast rotation differs from the student’s, a covariate shift; test by measuring the student’s action error as a function of rotation rate. [displaces M5]

Worked-example ladder.

WE-L.D.a (fully worked). The teacher’s action is a 6-vector of joint torque targets. The student observes 6 joint positions, 6 velocities, and 9 tactile readings, a 21-vector. Compute the input and output dimensions of the student network and the number of parameters in a two-hidden-layer MLP with 256 units per layer.

Solution. Input 21, hidden 256, hidden 256, output 6. Parameters: $21\cdot256 + 256 + 256\cdot256 + 256 + 256\cdot6 + 6 = 5376 + 256 + 65536 + 256 + 1536 + 6 = 72{,}966$. The network is small by modern standards; the difficulty is the data distribution, not the capacity.

WE-L.D.b (completion). The student’s action error is 0.05 Nm RMS on the teacher’s states but 0.4 Nm RMS on states visited after deployment. Diagnose the covariate shift and propose the fix.

Completion. The student is accurate on $\mathcal{D}$ but drifts off it; the fix is iterative data aggregation (DAgger): run the student, collect the states it visits, label them with the teacher, and retrain. Two or three rounds typically reduce the deployed error to within a factor of two of the on-distribution error.

WE-L.D.c (faded). Design the domain-randomization schedule for the reorientation policy: which parameters, what ranges, and how the curriculum widens the target orientation. Faded step: start with $\mu$ near nominal and target rotations within ±10°, widen to the full identification range and ±90° as success exceeds 80%. Completion: the curriculum prevents the policy from collapsing to a conservative “do nothing” behaviour early in training.

WE-L.D.d (contrastive near-miss). A learner trains the student with supervised learning on a fixed dataset of teacher rollouts and deploys it. The student is accurate for the first 0.5 s and then diverges. Why it fails: the fixed dataset covers the teacher’s trajectory, but the student’s small errors move it off that trajectory, into states with no training data. The fix: DAgger-style aggregation, or a residual policy on top of a stabilizing controller so the closed loop stays near the data.

Comparative table: learning architectures for dexterity.

Design principle P34.4. Train with privilege, deploy without it. The teacher may see the simulator’s hidden state; the student must not, and the gap between them is closed by distillation on the teacher’s state distribution.

Consolidation checkpoint 4

(a) Three capabilities. You can state the teacher–student loss and the DAgger correction. You can name the three dominant sim-to-real parameters for hands. You can size a student network and estimate its data needs.

(b) Transfer probe. The real hand’s tendon compliance is 30% higher than simulated. Which randomization parameter was mis-set, and what is the symptom? Answer: actuator dynamics/delay; the symptom is oscillation at the controller bandwidth, visible as a limit cycle in the joint trajectories.

(c) Remediation rule. If you cannot explain why a fixed teacher dataset produces a diverging student, re-read §L.D.2 before Part E.

Part E — The running case: the 90° reorientation

§L.E.1 — The reorientation budget

The running case’s task is to pick up the cube at $R_z(\pi/4)$, reorient it by 90° in the hand, and place it in the bin. The reorientation is executed as three 30° gait cycles, each 0.4 s, for a total of 1.2 s.

Force budget per cycle. During the transfer phase only two fingers hold the cube, so the grasp is a two-contact grasp with the same 45° misalignment as 2.3.3. From Eq. (B.5) of 2.3.3, the minimum finger force at rest is 1.39 N; with the cycle’s acceleration of about 1 m/s² the requirement rises to $0.1 \cdot 10.81/(2 \cdot 0.5 \cdot 0.7071) = 1.53$ N. The internal-force QP returns $s \approx 2$ N per finger, giving a 30% margin — thinner than the parallel-jaw case because only two contacts are active.

Rotation accuracy. Each cycle’s rotation is set by the rolling arc length, which depends on the contact point’s distance from the rotation axis. With a 2.5 cm half-width and a 1.31 cm arc, a 5% error in arc length gives a 1.5° error per cycle, 4.5° over three cycles. The final placement tolerates 10°, so the budget is adequate but not generous; the mitigation is a final visual correction using the 2.1.2 6D pose estimate before release.

Time budget. Pick 0.6 s, reorient 1.2 s, transport 1.5 s, place 0.8 s: 4.1 s total. The reorientation is 29% of the task time, and it is the phase with the highest failure probability, so the 1.1.4 state machine must treat it as a checkpoint: if the cube’s pose after reorientation is more than 10° from target, the state machine re-runs a cycle rather than proceeding.

§L.E.2 — Integration with the running threads

Four running threads meet at the reorientation.

The 1.1.4 state machine. A new REORIENT state sits between GRASP and TRANSPORT. Its entry condition is a successful grasp with the momentum-observer residual below threshold; its exit condition is the cube pose within 10° of target as measured by the 2.1.2 estimator. Its health checks are: internal-force QP feasible; all three fingertip forces within [0.5, 15] N; cube pose estimate fresh within 100 ms.

The 2.1.5 pose estimate. The eye-in-hand camera tracks the cube throughout. During the gait, one finger occludes the cube periodically; the pose estimator must handle 200 ms gaps. The 2.1.5 kinematic prior $T_W^C(q) = T_W^E(q) T_E^C$ keeps the camera pose known, so the cube pose can be propagated through the gap using the last valid measurement and the object’s (near-zero) velocity.

The 2.3.3 momentum observer. Each finger has its own observer; the three residuals give an independent estimate of the contact forces, which cross-checks the tactile arrays. A disagreement between the observer and the tactile reading larger than 2 N flags a sensor fault.

The 2.2.4 Q-shield. The learned reorientation policy (if used) is wrapped in the shield: actions whose predicted value falls below the deployed policy’s 10th percentile are replaced by the gait planner’s action. The shield’s threshold is set from the offline validation set, as in 2.2.4.

Worked-example ladder.

WE-L.E.a (fully worked). Compute the total rotation error over three cycles if each cycle’s arc length has a 5% error and the per-cycle nominal rotation is 30°.

Solution. Each cycle’s error is $0.05 \times 30° = 1.5°$; the errors are independent, so the RMS total is $\sqrt{3} \times 1.5° = 2.6°$, and the worst case is 4.5°. Both are within the 10° tolerance, but the worst case leaves only a factor of 2.2 margin.

WE-L.E.b (completion). The cube’s mass is 0.12 kg instead of 0.1 kg, a silent +20% error. Recompute the two-finger force requirement during transfer.

Completion. $F \ge 0.12 \cdot 10.81/(2 \cdot 0.5 \cdot 0.7071) = 1.83$ N. The QP’s $s=2$ N gives only a 9% margin, and a small friction drop would cause slip. The completion shows why the mass randomization range of §L.D.3 must include +20%.

WE-L.E.c (faded). Design the recovery sequence if the cube slips during the transfer phase: which sensors detect it, what the state machine does, and how long recovery takes. Faded step: the observer residual drops below threshold at the slipping finger; the tactile force on that finger falls to zero; the camera sees the cube rotate relative to the gripper. The state machine transitions to RECOVER_GRASP: re-close all three fingers, re-run the internal-force QP, and re-check the cube pose. Completion: recovery takes about 0.5 s, and if it fails twice the state machine returns to PICK from the table.

WE-L.E.d (contrastive near-miss). A learner increases the squeeze $s$ to 4 N “for safety” during the gait and finds that the third finger cannot lift clear of the cube during the transfer phase — the pad adheres and drags the cube. Why it fails: the higher normal force increases both the friction that holds the cube and the friction that resists the finger’s withdrawal; at $s=4$ N the withdrawal force needed exceeds the finger’s actuator budget. The fix: lower $s$ during the transfer phase and raise it only during the hold phases, or use a peeling motion for the withdrawal.

Comparative table: the reorientation phases and their budgets.

Design principle P34.5. Budget dexterity phase by phase; the force requirement of a three-finger grasp is set by the two fingers that remain during the gait, not by the three that hold at rest.

Consolidation checkpoint 5

(a) Three capabilities. You can compute the two-finger force requirement during a gait. You can size the rotation-error budget across cycles. You can integrate the reorientation state into the 1.1.4 machine with health checks.

(b) Transfer probe. The camera is occluded for 300 ms during the transfer phase. What does the pipeline do? Answer: propagate the cube pose with the kinematic prior and zero object velocity, inflate the pose covariance, and hold the gait; if the occlusion exceeds 500 ms, pause the gait and re-acquire.

(c) Remediation rule. If you cannot compute the two-finger force requirement, re-derive Eq. (B.5) of 2.3.3 before the lab.

Failure taxonomy

The table covers the three lab corruptions plus three failures not exercised in the lab: joint backlash, QP infeasibility, and delay-induced oscillation.

The lab

Task. Compute the grasp matrix and internal-force distribution for the three-contact grasp on the running case’s cube. Verify that the internal force lies in the null space of $G$ (leaves the object wrench unchanged), check all three contacts against the friction cone, and run one tactile-servo step that raises contact 2’s normal force by 0.5 N without ejecting the cube.

Conditions. Python 3.10+, NumPy only for the core computation; MuJoCo 3.x optional for the closed-loop extension. Cube half-width 2.5 cm, mass 0.1 kg, $\mu = 0.5$, three contacts on three faces as specified. Seed all RNGs.

Controls. (i) Minimum-norm force distribution with no internal force; (ii) minimum-norm plus a scalar squeeze $s = 2$ N; (iii) the QP (B.3) with a linearized cone; (iv) the QP plus a tactile-servo correction.

Metrics.

Pre-registered hypotheses.

H1. The internal force added by the scalar squeeze lies in $\ker(G)$ to numerical precision, because it is constructed from the null-space basis.

H2. The QP (B.3) produces a larger minimum friction margin than the fixed squeeze $s=2$ N, because it allocates force where the cone is widest.

H3. Servoing all three fingers toward the same force target ejects the cube, while servoing only the null-space component does not.

H4. The tactile-servo step reaches the 0.5 N target within 20 steps with the velocity-unit gain, and overshoots with a position-unit gain.

Break-it sequence.

1. Collapse the contact set silently. Move all three contacts onto the top face. The grasp matrix’s rank should fall to 5, and the friction check should reveal an unconstrained spin. Silent: the printed forces look normal; only the rank reveals the problem.
2. Saturate one finger silently. Set contact 3’s normal force to 15 N (the actuator limit) while the QP still assumes it can go higher. The QP’s solution should violate the limit and the friction margin should collapse at the other contacts. Silent: the total wrench is still balanced.
3. Inject a tactile bias. Add a constant 1 N bias to contact 1’s reading. Tactile servoing should drive a real force error while the measured error goes to zero. Silent: the measured signals converge perfectly.

Acceptance criteria.

What good looks like. A good artefact has five properties. (1) The grasp matrix, its rank, and its null-space basis are printed and checked, not assumed. (2) The internal force is visualized as vectors at the three contacts, so an inward squeeze is visibly distinct from a net push. (3) The friction-cone check reports the margin per contact, not a single pass/fail. (4) The break-it results are a table, so the silent rank collapse is visible. (5) The tactile-servo gain is expressed in velocity units with a stated stability argument.

A runnable core of the computation, seeded and annotated:

# lab_2_3_4.py — grasp matrix, internal forces, tactile servo (numpy only)
import numpy as np
np.random.seed(0)                                          # reproducibility

mu = 0.5                                                    # friction coefficient
cube_half = 0.025                                           # 5 cm cube
m, g = 0.1, 9.81                                            # mass, gravity

contacts = np.array([                                       # cube-frame contact points
    [ cube_half, 0.0, 0.0],                                 # face +x
    [-cube_half, 0.0, 0.0],                                 # face -x
    [0.0,  cube_half, 0.0],                                 # face +y
])
normals = np.array([                                        # inward normals
    [-1.0, 0.0, 0.0],
    [ 1.0, 0.0, 0.0],
    [0.0, -1.0, 0.0],
])

def skew(v):                                                # 3x3 skew-symmetric
    return np.array([[0, -v[2], v[1]], [v[2], 0, -v[0]], [-v[1], v[0], 0]])

def grasp_matrix(pts):                                      # Eq. (A.5)
    G = np.zeros((6, 3 * len(pts)))
    for i, p in enumerate(pts):
        G[:3, 3*i:3*i+3] = np.eye(3)                        # net force block
        G[3:, 3*i:3*i+3] = skew(p)                          # moment-arm block
    return G

G = grasp_matrix(contacts)
print("rank(G) =", np.linalg.matrix_rank(G))                # expect 6

$w_ext$ = np.array([0, 0, -m*g, 0, 0, 0])                     # gravity only
$f_p$ = np.linalg.pinv(G) @ ($-w_ext$)                          # particular solution

U, S, Vt = np.linalg.svd(G)
null_basis = Vt[len(S):].T                                  # columns span ker(G)
$f_0$ = null_basis @ np.array([2.0, 0.0, 0.0])                # 2 N squeeze, 1st null dir
f = $f_p$ + $f_0$

print("null-space residual =", np.linalg.norm(G @ $f_0$))     # expect ~1e-16

for i, n in enumerate(normals):                             # cone check per contact
    fi = f[3*i:3*i+3]
    fn = fi @ n                                             # normal component
    ft = np.linalg.norm(fi - fn * n)                        # tangential magnitude
    print(f"contact {i}: fn={fn:.2f} N, ft={ft:.2f} N, margin={mu*fn - ft:.2f} N")

$f_des$ = f.copy(); $f_des[6$] += 0.5                           # raise contact-2 normal by 0.5 N
Kv, dt = 0.02, 0.01                                         # m/(N s) gain, 10 ms period
dq = Kv * dt * ($f_des$ - f)                                  # null-space-safe servo step
print("servo dq (m):", np.round(dq, 5))

Line 6 fixes the friction coefficient that the cone check uses; line 7 fixes the geometry. Lines 9–16 encode the contact set and inward normals, which is where the silent rank collapse of break-it 1 is introduced. Line 24 builds $G$ from the moment arms. Line 33 projects out the particular solution; line 36 extracts the null-space basis; line 37 applies a 2 N squeeze along one null direction. Lines 39–45 verify the two properties the lab is about: the squeeze leaves the wrench unchanged, and the forces stay inside the cones. Lines 47–50 take one tactile-servo step in velocity units.

Cheat sheet

Exercises with solutions

E 2.3.4.1. A hand has $n_q = 12$ joints and four soft-finger contacts. Compute $d$ assuming full rank and state how many internal motions exist.

S 2.3.4.1. Soft-finger contacts give $k=4$, so $J$ is $16 \times 12$ with rank at most 12. $d = 6 + 12 - 12 = 6$, so there are zero internal motions at a fully constrained grasp; the hand is exactly actuated. Adding one joint gives $d=7$ and one internal DOF. The lesson: dexterity requires actuation in excess of contact constraints, and four contacts consume a lot of that excess.

E 2.3.4.2 (transfer). Three contacts lie on three mutually orthogonal faces of the cube. Is the grasp matrix full rank? Give the geometric argument.

S 2.3.4.2. Yes. The three inward normals span $\mathbb{R}^3$, so the force block is rank 3; the three moment arms span the remaining 3 directions because the contacts are non-coplanar. The lesson: rank is a geometric property of contact placement; non-coplanarity is the condition.

E 2.3.4.3. Write the affine solution set of $Gf = -w$ for $n_c = 3$ and explain in one sentence why the minimum-norm solution is not automatically the right one.

S 2.3.4.3. $f = -G^+w + N\lambda$ with $N$ spanning $\ker(G)$. The minimum-norm solution ignores the friction cones and actuator limits, so it can be infeasible or unsafe even though it balances the wrench. The lesson: the wrench balance is necessary, not sufficient.

E 2.3.4.4 (diagnostic). A three-finger hand cannot rotate the cube in hand, and the planner reports that the hand Jacobian has rank 6 with $n_q = 6$. Diagnose the problem and state the hardware or strategy fix.

S 2.3.4.4. With $d = 6 + 6 - 6 = 6$ there are no internal motions, so a continuous rotation is impossible without breaking contact. The fix is either hardware (more joints, e.g., three joints per finger) or strategy (finger gaiting with discrete regrasps). The lesson: exact actuation at a grasp is a hard ceiling on continuous manipulation.

E 2.3.4.5 (transfer). A tactile-servo controller has $K_v = 0.02$ m/(N·s), a 10 ms period, and a contact stiffness of $10^5$ N/m. Compute the force produced by a 1 N error in one step and decide whether the gain is safe against a 15 N limit.

S 2.3.4.5. The motion is $0.02 \times 0.01 \times 1 = 2 \times 10^{-4}$ m, which against $10^5$ N/m produces 20 N — exceeding the 15 N limit. The gain must fall to about $0.01$ m/(N·s) or lower. The lesson: tactile-servo gains are constrained by the product of contact stiffness and control period, not by the force error alone.

E 2.3.4.6. State the teacher–student loss of Eq. (D.1) and explain why the expectation is over the teacher’s state distribution.

S 2.3.4.6. $L = \mathbb{E}_{o\sim\mathcal{D}}[|\pi_s(o) - \pi_t(o,s)|^2]$ with $\mathcal{D}$ the teacher’s state distribution. The student must be correct where it will operate; the teacher’s closed loop defines that region, and uniform sampling spends capacity on states the loop never reaches. The lesson: distillation is distribution matching as much as function matching.

E 2.3.4.7 (diagnostic). A trained reorientation policy works in simulation but oscillates at 8 Hz on hardware. Name two causes and the discriminating test.

S 2.3.4.7. (1) Actuator delay not modelled: test by adding a matched delay in simulation and observing whether the same 8 Hz oscillation appears. (2) Contact stiffness higher than modelled: test by raising $k_c$ in simulation. The delay hypothesis is more likely when the frequency matches $1/(4\tau_d)$. The lesson: oscillation frequency is a fingerprint of the dominant unmodelled dynamics.

E 2.3.4.8. A gait cycle must rotate the cube by exactly 90° in an integer number of equal cycles. What per-cycle angle should be chosen for three, four, and five cycles, and which is preferred given a 1.5° per-cycle arc-length error?

S 2.3.4.8. Three cycles: 30°; four: 22.5°; five: 18°. RMS rotation error is $\sqrt{n} \times 1.5°$, giving 2.6°, 3.0°, and 3.35°. Three cycles minimizes error and time, and its 4.5° worst case fits the 10° tolerance. The lesson: more cycles buy smaller per-cycle rotations but accumulate error; choose the smallest cycle count that satisfies the force margin.

Exit test

You are ready for Module 3.1 when you can, without notes:

1. Compute $d = 6 + n_q - \operatorname{rank}(J)$ and interpret it.
2. Explain underactuation and give the running case’s count.
3. Write the hand Jacobian and the grasp matrix and state their dimensions.
4. Derive both dualities: $G^\top v = J\dot q$ and $\tau = J^\top f$.
5. Decompose contact forces into particular and internal parts and justify the split.
6. Set up the internal-force QP with linearized cone and actuator constraints.
7. State Montana’s rolling-contact equations and identify the spin term.
8. Compute a manipulability measure and diagnose a singular posture.
9. Write the tactile-servo law and choose its gain from contact stiffness and period.
10. Lay out a four-phase finger gait and compute the per-cycle rotation.
11. Write the MPC cost for a gait with force and time terms.
12. State the teacher–student loss and explain the DAgger correction.
13. Name the three dominant sim-to-real parameters for dexterous hands.
14. Budget a 90° reorientation across three cycles with force and time margins.
15. Integrate the REORIENT state into the 1.1.4 state machine with health checks.

Cross-references. 2.3.1 supplies force closure and the $\varepsilon$ metric restated in §L.A.2. 2.3.3 supplies the friction cone, the momentum observer, and residual RL restated in §L.B.1, §L.C.1, and §L.E.2. 2.2.3 and 2.2.4 supply the offline pretraining and the Q-shield consumed in §L.D.2. 2.1.5 supplies the eye-in-hand pose estimate restated in §L.E.2. Module 3.1 (VLA models) builds on the teacher–student pattern as the transfer mechanism for pre-trained policies, and Module 3.4 (sim-to-real) extends the randomization-and-distillation architecture of Part D to whole-body systems.

Ready. The next lesson to write is 3.1.1 — Introduction to VLAs: the paradigm shift from specialized to generalist policies. Say “next” to generate it. Note that Module 3.1 is marked proposed in the curriculum index; confirm you want Phase 3 to proceed before I write it.