X2.1.3

Lesson 2.1.3 — Visual Servoing: Image-Based, Position-Based, and the Hybrid That Avoids Both

Week 10 · Prerequisites: 2.1.1, 2.1.2, 1.1.1, 1.1.3, 0.2.1, 0.2.2.

Prior results used. This lesson uses 2.1.1’s pinhole model and back-projection’s ray, 2.1.2’s 6D pose estimation and the symmetric’s ambiguity, 1.1.1’s geometric Jacobian, 1.1.3’s PID and computed-torque control, 0.2.1’s SE(3) inverse, and 0.2.2’s SO(3) log map. Each is restated at the point of use.

Hands forward. This lesson hands forward one capability: the learner can derive the interaction matrix, implement an image-based servo that drives the pixel error to zero, implement a position-based servo that drives the pose error to zero, analyze the stability and the local minima of each, choose the hybrid when both fail, and diagnose the four visual-servoing bugs (sign error, depth estimate, extrinsic frame, target loss) from their trajectory’s symptoms — the capability that 1.1.4’s final approach, 2.3.2’s grasp alignment, and 3.1.4’s closed-loop VLA refinement all assume.

Running case. The same 6-DOF MuJoCo arm from 1.1.1–2.1.2, with the eye-in-hand camera (intrinsics K = [[500, 0, 320], [0, 500, 240], [0, 0, 1]], 640 × 480) and the cube at p^W = [0.5, 0.2, 0.3]. The four visible corners are the image features s = (u₁, v₁, …, u₄, v₄), and the desired features s* are the same four corners at the desired camera pose (centered, gripper aligned). The servo drives s → s* by commanding the camera’s velocity v_c = (v, ω). The central result: IBVS is stable in image space but non-optimal in Cartesian space, PBVS is optimal in Cartesian space but can lose the target, and the 2.5D hybrid bounds both.

§L.0 — Lesson map, objectives, and misconceptions

§L.0.1 — Lesson map and word budget

§L.0.2 — Objectives

By the end of this lesson you will be able to do eight things:

1. State the visual-servoing problem, the eye-in-hand vs eye-to-hand configurations, and the closed-loop structure (image → features → control → robot velocity → new image).
2. Derive the interaction matrix L_x for a point in normalized coordinates and in pixel coordinates, and state its 1/Z scaling and its rank (2 per point, 6 for three or more points).
3. Implement the IBVS control v_c = -λ L̂_s⁺ (s - s*) with an estimated L̂_s (approximate depth), and state the exponential convergence and the local-minimum risk.
4. Derive the PBVS control v_c = -λ L̂_e⁺ e with the 6D pose error e = [t_err; θ u_err] and the pose interaction matrix L_e, and state the global stability (linearized) and the FOV risk.
5. State IBVS’s advantages (calibration robustness, guaranteed image convergence) and disadvantages (non-optimal Cartesian trajectory, local minima, depth dependence), and PBVS’s advantages (optimal Cartesian trajectory, global convergence) and disadvantages (calibration sensitivity, target loss).
6. State the 2.5D hybrid (image error plus depth, or homography decomposition) and explain its benefit (bounded image trajectory, controlled Cartesian trajectory).
7. State the visibility condition (target pixels stay inside the image) and the FOV constraint, and the trajectory’s violation symptom (target loss).
8. Diagnose the four bugs — sign error (divergence), wrong depth estimate (overestimate amplifies the gain: overshoot, then divergence; underestimate attenuates it: slower, still stable), wrong extrinsic frame (wrong axis), target loss (out of view) — from their trajectory’s symptoms.

§L.0.3 — Misconception inventory

Each misconception is displaced in the section named, and each is tagged in the concept check that follows.

Everything is self-contained. Where a prerequisite lesson is used, the result is restated at the point of use.

§L.0.5 — Retrieval warm-up

This is the twenty-fourth lesson. The warm-up draws from 2.1.1 (two steps back), 1.2.5 (three), 1.2.4 (four), and 1.2.3 (five).

Warm-up 1 (from 2.1.1, projection and depth ambiguity). State the projection s x = K [R|t] X and the back-projection ray. Answer: s = Z is the depth scale; K is intrinsics; [R|t] is world-to-camera. The back-projection is the ray C + λ R x_n with x_n = K⁻¹ x. How this is used: §L.2’s L_x uses 1/Z, and the depth ambiguity from 2.1.1 is exactly what L̂_s must approximate.

Warm-up 2 (from 1.2.5, SAC). State the twin critics and α’s role. Answer: twin critics’ min mitigates overestimation; α is the entropy’s learnable weight. How this is used: §L.4’s visual servo gain λ plays a role analogous to the PID’s K_p, and the automatic α tuning is analogous to an adaptive λ.

Warm-up 3 (from 1.2.4, PPO). State the clipped objective and the trust region. Answer: min(r A, clip(r) A) with r = π_θ/π_old; the clip bounds the ratio. How this is used: §L.3’s PBVS stability is the linearized pose error’s exponential decay; saturation on the velocity is the trust-region analogue (bounded step, avoiding target loss).

Warm-up 4 (from 1.2.3, REINFORCE). State the score function and the baseline’s variance reduction. Answer: ∇ log π(a|s) is the score with zero mean; the baseline b(s) gives E[∇ log π · b(s)] = 0. How this is used: §L.1’s visual servo error s - s* is deterministic, and the desired features s* play the role of a baseline — the target, not a random variable.

Part A — The Visual-Servoing Problem and the Two Paradigms

§L.1 — Objects: the problem, the configurations, the closed loop

The visual-servoing problem: given a camera observing a target with image features s, desired features s*, and a robot carrying or observing the camera, design a control law that drives s(t) → s* as t → ∞.

Two configurations:

• Eye-in-hand — camera mounted on the end-effector. The camera’s velocity v_c is the end-effector’s velocity V = J(q) q̇ (1.1.1). The control is the joint velocity q̇ = J(q)⁻¹ v_c (resolved-rate control).
• Eye-to-hand — camera fixed in the workspace, observing the robot. The camera’s observed velocity is the target’s velocity; the robot’s control is the opposite sign (v_target = -v_c).

Closed-loop structure:

1. Camera captures image at t_k.
2. Feature extraction gives s(t_k).
3. Error e(t_k) = s(t_k) - s*.
4. Control law computes v_c(t_k).
5. Robot velocity q̇(t_k) = J(q)⁻¹ v_c.
6. Robot moves; new image at t_{k+1} closes the loop.

Two paradigms:

• Image-Based Visual Servoing (IBVS) — error is in the image (e = s - s*), control uses the interaction matrix L_x.
• Position-Based Visual Servoing (PBVS) — error is in the Cartesian pose (e = [t_err; θ u_err]), control uses the pose interaction matrix L_e.

§L.1.1 — Result: IBVS’s error is 2D, PBVS’s is 3D, and each is locally stable in its own space

Claim. IBVS’s error e = s - s* is 2D (8 numbers for four points) and its control drives the image error to zero, locally stable, if L̂_s is a good estimate. PBVS’s error is 6D and its control drives the pose error to zero, locally stable, if the pose estimate is exact.

Proof sketch. IBVS error dynamics: ė = L_x v_c = -λ L_x L̂_x⁺ e. With L̂_x = L_x, ė = -λ e (exponential decay). PBVS error dynamics: ė = L_e v_c = -λ L_e L̂_e⁺ e. With L̂_e = L_e, ė = -λ e. Both are locally stable; the difference is which space converges.

Corollary. IBVS’s stability is in image space (s → s*), not in Cartesian space. PBVS’s stability is in pose space (T → T*), not in image space. The symptom of IBVS is a converged image but a possibly wrong camera pose; the symptom of PBVS is a converged pose but possibly lost target.

Corollary. IBVS is robust to calibration error (image feedback tolerates it); PBVS is robust to pose error only if the pose is accurately estimated. The choice is task-dependent.

§L.1.2 — Worked-example ladder

WE-L.1.a — Fully worked: one closed-loop step

Input. Camera at T_Cam^W = I, cube’s four visible corners at camera-frame positions, pixels s_0, desired s*, gain λ = 0.3.

Trace.

1. Error e_0 = s_0 - s*.
2. Interaction matrix L_s (8×6, built from L_u for each point).
3. Pseudoinverse L̂_s⁺ (6×8).
4. Control v_c = -0.3 · L̂_s⁺ · e_0.
5. Robot velocity q̇ = J(q)⁻¹ v_c.
6. New image s_1; error e_1 = s_1 - s*.

Output and interpretation. With an exact L̂_s, e_1 = 0.7 e_0 — exponential decay at rate λ. Measured on the running case, e₁/e₀ = 0.676 rather than 0.700: the linear model overstates the contraction slightly, because one step of a finite twist is not the infinitesimal ė = −λe. In practice L̂_s also uses approximate depth, which scales the gain by Ẑ/Z.

Self-explanation 1. In step 4, why is λ = 1 risky?

Answer. In the linearised model, λ = 1 makes e_1 = 0 in one step (exact L̂_s). It does not, because the model is only linear: measured on the running case, λ = 1 gives e₁/e₀ = 0.780, not 0. The linear prediction |1 − λ| is accurate at λ = 0.3 (0.700 predicted, 0.676 measured) and wrong by 0.78 at λ = 1. λ = 1 is the stability boundary of the linearised loop and past the usable range of the real one; λ = 0.3–0.5 is where the linearisation still describes what happens.

Self-explanation 2. If L̂_s uses a wrong depth (Ẑ = 2Z), what is the symptom?

Answer. Every 1/Z in L̂_s is halved, so L̂_s⁺ is doubled and v_c is too large — the servo overshoots. With Ẑ = 2Z the effective gain is 2λ = 0.6, so each step contracts by 0.4 instead of 0.7: faster per step, and the loop stays stable while 2λ < 1. Push the overestimate to Ẑ = 10Z and the gain is 3.0; the first step throws the cube out of frame.

WE-L.1.b — Completion problem

Input. Same case, λ = 0.2, e_0.

Trace (final 25% as prompts).

1. e_1 = 0.8 e_0. ⟨verify⟩
2. e_2 = 0.64 e_0. ⟨verify⟩
3. e_k = 0.8^k e_0. ⟨verify⟩
4. Number of steps to reach 0.1 e_0: 0.8^k = 0.1 → k ≈ 10.3. ⟨verify⟩

Answers.

1–4. Correct.

The lesson. The gain λ sets the exponential rate. λ = 0.2 needs ~10 steps for 10% error; λ = 0.5 needs ~3 steps. Too high a gain saturates the actuators.

WE-L.1.c — Faded: eye-in-hand vs eye-to-hand

Setup. Same camera, but eye-to-hand (fixed camera observing the robot).

Question. What is the sign of the control?

You must produce the intermediate steps. The setup and the answer are given.

Answer. The camera is fixed, so its velocity is not the control. The target is the robot’s end-effector. The camera’s observed velocity is the end-effector’s velocity; the robot’s control is v_target = -v_c. Thus q̇ = -J⁻¹ λ L̂⁺ e — the opposite sign from eye-in-hand.

The lesson. The eye-to-hand configuration flips the sign. Getting the sign wrong causes divergence.

WE-L.1.d — Contrastive near-miss: IK’s one step

Correct solution. Visual servo as closed loop (many steps, feedback).

Near-miss. IK’s one step (open loop, pose solve).

Why the near-miss produces a plausible but incorrect number. The IK solves q = IK(T*) and drives to the pose in one shot. If calibration is perfect, it lands exactly. But the camera observes the image, and calibration error is direct. IBVS is image-feedback (tolerates calibration error but converges only in image space). The IK is pose-open-loop (calibration error is direct but pose is exact if calibration is exact). The symptom of IK failure is pose error from calibration; the symptom of IBVS is image convergence but non-optimal pose.

The discrepancy. IK’s q = IK(T*) is open loop; IBVS’s q̇ = J⁻¹(-λ L⁺ e) is closed loop. The two are different problems: IK inverts the pose map; IBVS feeds back the image error.

Self-explanation 3. If calibration is perfect, is IK equivalent to IBVS?

Answer. Only if the image-to-pose map is bijective. It is not: the 2D image loses depth, and multiple poses produce the same image (the symmetric ambiguity from 2.1.2). IK solves the pose exactly; IBVS only converges to a pose consistent with the image.

Part B — Image-Based Visual Servoing: Interaction Matrix and Local Minima

§L.2 — Objects: the interaction matrix, the control law, the local minima

The interaction matrix L_x ∈ ℝ^{2×6} for a 3D point X = (X, Y, Z) in the camera frame, with normalized image coordinates x = X/Z, y = Y/Z:

L_x = [ -1/Z    0    x/Z     x·y      -(1+x²)    y   ]
      [   0   -1/Z   y/Z   1+y²        -x·y     -x   ]

Each column corresponds to a component of camera velocity v_c = (v_x, v_y, v_z, ω_x, ω_y, ω_z).

In pixel coordinates (u, v) with focal f (square pixels):

L_u = [ -f/Z    0    u/Z     u·v/f    -(f+u²/f)    v   ]
      [   0   -f/Z   v/Z    f+v²/f     -u·v/f    -u   ]

Key observations:

• The first three columns (translation) have 1/Z scaling. Translation’s effect on the image depends on depth.
• The last three columns (rotation) have no Z dependence. Rotation’s effect is scale-free.
• For n ≥ 3 points, L_s = [L_u^1; ...; L_u^n] is 2n × 6. Rank is 6 for non-degenerate configurations.

Control law:

v_c = -λ · L̂_s⁺ · (s - s*)

where L̂_s⁺ is the pseudoinverse, λ > 0 is the gain.

Depth estimate. L̂_s requires depth Z for each point. In practice:

• From the RGB-D camera: use measured depth.
• From a known object size: Z = f · d / d_pixel (the 2.1.1 disparity).
• From the 2.1.2 pose: Z = (T_Cam^Obj · X_Obj)_z.

Local minima. The classic IBVS local minima arise from:

• Coplanar points at 180° rotation: the image looks the same, and the error is zero, but the pose is wrong.
• L_s rank deficiency for degenerate configurations (fewer than 3 points, collinear points).
• The pseudoinverse’s null space when rank(L_s) < 6.

§L.2.1 — Result: the interaction matrix’s -f/Z is depth-dependent; the control is exponential but depth-approximate; local minima are the nonlinearity’s consequence

Claim. The interaction matrix’s translation columns scale as 1/Z; rotation columns do not. The control law v_c = -λ L̂_s⁺ e gives exponential convergence e(t) = e^{λ t} e(0) when L̂_s = L_s (exact). With an approximate L̂_s, the convergence is still exponential but with a modified rate.

Proof sketch. ė = L_s v_c = -λ L_s L̂_s⁺ e. If L_s L̂_s⁺ = I on the range of L_s (exact estimate), ė = -λ e. With an estimate L̂_s, L_s L̂_s⁺ is approximately I with error O(δZ/Z), and the rate changes by O(δZ/Z).

Corollary. Depth error affects magnitude, not sign. Sign error (wrong L̂_s direction) causes divergence. A wrong Z magnitude rescales the gain: with Ẑ = kZ the eigenvalues of L_s L̂_s⁺ lie in [min(1,k), max(1,k)], so k > 1 multiplies the step by up to k (overshoot, then divergence once λk > 1) and k < 1 divides it (slower, still stable). Only the overestimate is dangerous.

Corollary. Local minima occur when L_s L̂_s⁺ e = 0 but e ≠ 0 — the pseudoinverse’s null direction. This happens for symmetric configurations (180° flips) and degenerate geometries.

§L.2.2 — Worked-example ladder

WE-L.2.a — Fully worked: the interaction matrix for a single point

Input. Point at X = (0.5, 0.2, 1.0), focal f = 500, c_x = c_y = 0 for simplicity.

Trace.

1. x = X/Z = 0.5, y = Y/Z = 0.2.
2. u = f·x = 250, v = f·y = 100.
3. L_x = [ -1/1.0 0 0.5 0.5·0.2 -(1+0.25) 0.2 ] [ 0 -1/1.0 0.2 1+0.04 -0.5·0.2 -0.5 ]
4. L_x = [ -1.0 0 0.5 0.1 -1.25 0.2 ] [ 0 -1.0 0.2 1.04 -0.1 -0.5 ]

Output and interpretation. The -1/Z term dominates translation columns. Rotation columns have magnitude O(1). The L_x maps a camera velocity (v_x, v_y, v_z, ω_x, ω_y, ω_z) to the point’s image velocity in (ẋ, ẏ) normalized coordinates. With f = 500, the pixel interaction matrix scales by f.

Self-explanation 4. Why is the ω_z column (last column) [y; -x]?

Answer. ω_z is rotation about the optical axis. It rotates the image plane; the point (x, y) moves along the tangent direction (y, -x) proportional to ω_z. This is the image rotation effect, independent of depth.

Self-explanation 5. If Z doubles to 2.0, what happens to the translation columns?

Answer. -1/Z = -0.5, and x/Z = 0.25. Translation columns halve; rotation columns unchanged. Halving L̂_s’s translation block doubles its pseudoinverse’s, so the commanded translation is twice what the true depth calls for. The camera moves too far for the pixel error: overshoot. An underestimate (Ẑ = Z/2) is the case that converges slowly — the command comes out too small.

WE-L.2.b — Completion problem

Input. Same point, Z = 2.0, x = 0.25, y = 0.1.

Trace (final 25% as prompts).

1. L_x[0, 0] = -1/2 = -0.5. ⟨verify⟩
2. L_x[0, 2] = x/Z = 0.25/2 = 0.125. ⟨verify⟩
3. L_x[1, 1] = -0.5. ⟨verify⟩
4. L_x[1, 3] = 1 + y² = 1.01. ⟨verify⟩

Answers.

1–4. Correct.

The lesson. Depth doubling halves the translation sensitivity, but rotation is unaffected. The image becomes “flatter” for the same physical motion.

WE-L.2.c — Faded: the local minimum at 180°

Setup. Four coplanar points; camera at 180° from desired pose. Image looks identical to desired.

Question. What is the error, the control, the trajectory?

You must produce the intermediate steps. The setup and the answer are given.

Answer. e = 0 (image looks identical). The control v_c = 0 — the servo stops at the wrong pose. This is a local minimum: the image error cannot distinguish 0° from 180°.

The lesson. The IBVS error is 2D; the pose is 6D. The map from pose to image is not injective, so multiple poses give the same image. The servo has no way to escape the local minimum without additional information (a prior, an additional feature, a depth constraint).

WE-L.2.d — Contrastive near-miss: wrong-sign control

Correct solution. v_c = -λ L̂_s⁺ e (negative sign).

Near-miss. v_c = +λ L̂_s⁺ e (positive sign).

Why the near-miss produces a plausible but incorrect number. The wrong sign makes ė = +λ L_s L̂_s⁺ e ≈ +λ e, which is exponential growth. The image error grows; the servo diverges. The trajectory shows the camera moving away from the target until the target leaves the FOV.

The discrepancy. The sign of the control determines whether the error decays or grows. The wrong sign is the most common bug in visual servoing.

Self-explanation 6. How can you detect the wrong-sign bug in one test?

Answer. Check whether ||s(t) - s*|| decreases with each step. If it increases, the sign is wrong. Alternatively, check the sign of the eigenvalue of L_s L̂_s⁺: with the correct sign, all eigenvalues should have positive real part when the error is multiplied by -λ.

Part C — Position-Based Visual Servoing: Pose Error and FOV Risk

§L.3 — Objects: the pose error, the pose interaction matrix, the control law

PBVS estimates the 6D pose T_Cam^Obj (via 2.1.2) and computes the pose error:

e = [ t - t* ; θ · u_err ]

where t_err = t - t* is the translation error and θ · u_err is the rotation error in axis-angle form (from 0.2.2’s log map on R_err = R* Rᵀ).

Pose interaction matrix L_e ∈ ℝ^{6×6}:

L_e = [ -I     [t]_× ]
      [  0     L_ω   ]

where [t]_× is the skew-symmetric matrix of t, and:

L_ω = I - (θ/2)[u]_× + (1 - sinc(θ)) [u]_ײ

with sinc(θ) = sin(θ)/θ.

Control law:

v_c = -λ · L̂_e⁺ · e

With exact L_e, L_e L_e⁺ = I and ė = -λ e — global exponential convergence in pose space.

FOV risk. The PBVS optimal trajectory (the straight-line interpolation in pose space) can push the camera away from the target for large rotations. The target’s image can exit the camera’s field of view — the camera goes blind and cannot continue.

§L.3.1 — Result: PBVS is globally stable (linearized) in pose space but risks target loss; the FOV constraint is the practical limit

Claim. With exact L_e and pose estimate, PBVS has global exponential convergence in pose space. However, the trajectory is the optimal Cartesian interpolation, which can rotate the camera such that the target exits the FOV.

Proof sketch. ė = L_e v_c = -λ L_e L_e⁺ e = -λ e. The dynamics are exactly linear and globally stable. The FOV constraint is a physical limitation: the image projection s = K [R|t] X requires Z > 0 and u, v in image bounds. When the optimal trajectory violates either, the servo cannot proceed.

Corollary. PBVS is optimal when the calibration is accurate and the FOV constraint is satisfied. When the target would leave the FOV, PBVS fails; the servo must either slow down (small λ) or switch to a hybrid.

Corollary. PBVS is more sensitive to calibration than IBVS: pose estimation errors propagate directly to the control. IBVS’s image feedback tolerates small calibration errors because the image error corrects itself.

§L.3.2 — Worked-example ladder

WE-L.3.a — Fully worked: the pose interaction matrix for identity R

Input. R = I, t = (0.5, 0.2, 1.0), e = [t_err; θ u_err] with θ = 0 (no rotation error).

Trace.

1. [t]_× = [0, -1.0, 0.2; 1.0, 0, -0.5; -0.2, 0.5, 0].
2. L_ω = I (θ = 0).
3. L_e = [ -I [t]_× ] [ 0 I ].
4. With e = [ΔT; 0], L_e⁺ e gives v_c ≈ -[ΔT; 0] — the camera moves toward t*.

Output and interpretation. The PBVS control decouples translation and rotation at θ = 0. As θ grows, L_ω couples the rotation to itself.

Self-explanation 7. Why does L_ω become I at θ = 0?

Answer. sinc(0) = 1, so 1 - sinc(0) = 0, and the [u]_ײ term vanishes. At small θ, the rotation error’s contribution to the image is approximately the rotation itself — small-angle approximation.

Self-explanation 8. Why is the [t]_× term in L_e?

Answer. The rotation rate ω contributes to the translation velocity of the origin: ṫ = v + ω × t = v - [t]_× ω. The - [t]_× block captures this coupling — a rotation about a point not at the origin moves the origin.

WE-L.3.b — Completion problem

Input. R = I, t = (0.5, 0.2, 1.0), θ = π/2 about the z-axis.

Trace (final 25% as prompts).

1. u = (0, 0, 1), [u]_× = [[0, -1, 0], [1, 0, 0], [0, 0, 0]]. ⟨verify⟩
2. sinc(π/2) = 2/π ≈ 0.637. ⟨verify⟩
3. 1 - sinc(π/2) ≈ 0.363. ⟨verify⟩
4. L_ω = I - (π/4)[u]_× + 0.363 [u]_ײ. ⟨verify⟩

Answers.

1–4. Correct.

The lesson. As θ grows, L_ω becomes increasingly non-identity. The rotation-to-rotation mapping is no longer direct; the axis-angle parametrization’s singularity at θ = π (0.2.2) also affects L_ω.

WE-L.3.c — Faded: FOV loss

Setup. Camera at t = (0, 0, 0.5), target at T_W = (0, 0, 0), desired T* at t = (0, 0, 0.5) but with R* = Rz(π) (180° rotation).

Question. Does the trajectory lose the target?

You must produce the intermediate steps. The setup and the answer are given.

Answer. PBVS drives the camera along the optimal pose interpolation. For a 180° rotation, the interpolated pose rotates the camera through the target’s view. At the midpoint, the camera is at 90° rotation, and the target is at the edge of the image. At 180°, the target is behind the camera — lost. The servo cannot continue.

The lesson. For large rotations, PBVS is optimal but blind. IBVS handles this case by keeping the target’s features in view, at the cost of a longer Cartesian trajectory. The hybrid 2.5D chooses the middle path.

WE-L.3.d — Contrastive near-miss: PBVS with wrong pose estimate

Correct solution. Pose from 2.1.2 with < 1 cm error.

Near-miss. Pose from 2.1.2 with symmetric ambiguity (180° flip).

Why the near-miss produces a plausible but incorrect number. With the symmetric ambiguity, the PBVS error is 2 · (misalignment) — the wrong pose drives the camera to the symmetric target, not the true one. The image shows the same target as a symmetric view, so the servo converges — to the wrong pose.

The discrepancy. The pose estimate’s ambiguity propagates directly to the PBVS control. IBVS is insensitive because it uses the image error, not the pose. The hybrid uses a pose estimate plus an image error to disambiguate.

Self-explanation 9. If the target is a symmetric cube, is PBVS unsafe?

Answer. Not necessarily — the 180° flip might be equivalent for a grasp (any 90° rotation of the cube is graspable). It is unsafe for insertion (the hole must be aligned correctly) or precision manipulation. The choice depends on the task’s symmetry.

Part D — The Comparison and the 2.5D Hybrid

§L.4 — Objects: the comparison, the hybrid, the visibility condition

Comparison:

The 2.5D hybrid (Malis-Chaumette-Boudet, 1999): combines image error with depth error:

e_hybrid = [ s - s* ; log(Z/Z*) ]

or uses the homography decomposition: from H between the two views, extract R and t/Z. The control uses L (a mix of image and pose interaction).

Simpler hybrid: use IBVS for the image direction and PBVS for the depth and coarse rotation. Implementation:

v_c = -λ L_s^+ (s - s*) - λ_p L_e^+ [t_err; 0]

where λ_p is the pose gain, and L_e^+ uses the pose estimate. The pose term carries translation only: the rotation is left to the image term, which sees it directly. λ_p is a separate gain, not a fraction of λ — writing it inside the outer bracket multiplies the two, and λ_p then means something different from what the symbol says.

Visibility condition. The target’s pixels stay inside [0, W-1] × [0, H-1]. This is a constraint on the trajectory:

u_min ≤ u_i(t) ≤ u_max for all i, t
v_min ≤ v_i(t) ≤ v_max for all i, t

Violations cause target loss. The IBVS converges to the target, but the intermediate trajectory can push pixels off-image if the gain is too high.

§L.4.1 — Result: the 2.5D hybrid combines IBVS’s visibility guarantee with PBVS’s optimality; the visibility condition is the constraint that determines the hybrid’s gain

Claim. The 2.5D hybrid’s trajectory satisfies the visibility condition if the gain is chosen so the image features stay in the FOV. The Cartesian trajectory is bounded between IBVS (which can be long) and PBVS (which can lose the target).

Proof sketch. The hybrid’s error combines image features (which anchor the trajectory to the image space) and depth/pose (which give the Cartesian direction). The image features’ contribution keeps the target in view; the pose’s contribution gives the shortest Cartesian path.

Corollary. The hybrid is the default for pick-and-place, grasping, and any task where the target must stay visible and the approach path should be short.

Corollary. The two gains interact with the visibility condition, and the direction is not the obvious one. Measured: the translation-only hybrid at λ_p = 0.1 needs 23 steps against pure IBVS’s 18, and adding the full pose error at the same λ_p needs 13. A larger pose term is faster, not slower — so it is also the one that risks the FOV. The constraint is λ_p bounded by measured visibility, not by a rule that lower is safer.

§L.4.2 — Worked-example ladder

WE-L.4.a — Fully worked: hybrid control for the running case

Input. e_2D = s - s* (image error), e_3D = [t_err; 0] (translation only), λ = 0.2, λ_p = 0.1.

Trace.

1. v_c_IBVS = -0.2 L̂_s⁺ e_2D.
2. v_c_PBVS = -0.1 L̂_e⁺ e_3D.
3. v_c = v_c_IBVS + v_c_PBVS.
4. Total magnitude roughly 0.2 ||e_2D|| + 0.1 ||e_3D||.

Output and interpretation. The IBVS component dominates for small motion, the PBVS component accelerates the approach. The combination stays visible (image feedback) and reaches quickly (pose feedback).

Self-explanation 10. Why is λ_p smaller than λ?

Answer. The pose component accelerates the motion and could push the target off-image, so λ_p is set by measured visibility rather than by a ratio to λ. The two are not on the same scale: λ multiplies a pseudoinverse of a pixel-valued matrix, λ_p multiplies a translation in metres. λ_p ≈ 0.1–0.3 with λ ≈ 0.3–0.5 is the working range, but the numbers are units-dependent and the check is the visibility trace, not the value of λ_p.

WE-L.4.b — Completion problem

Input. Target at the FOV edge, e_2D large.

Trace (final 25% as prompts).

1. The IBVS component pushes toward the center. ⟨verify⟩
2. The PBVS component pulls in a Cartesian direction. ⟨verify⟩
3. The combined motion: is the visibility preserved? ⟨verify⟩
4. Adjust the gain to keep visibility. ⟨verify⟩

Answers.

1–4. Correct.

The lesson. The hybrid’s benefit is that the visibility can be bounded by the image component, while the Cartesian optimality is maintained by the pose component.

WE-L.4.c — Faded: the visibility’s violation

Setup. IBVS with λ = 1.0, target at the FOV edge.

Question. What happens?

You must produce the intermediate steps. The setup and the answer are given.

Answer. The high gain commands a large velocity, moving the target out of the FOV in one step. The pixels leave the image; the next image has no target; the servo fails.

The lesson. The visibility condition is a constraint on the gain and the trajectory. IBVS with a high gain can lose the target even though IBVS is nominally robust.

WE-L.4.d — Contrastive near-miss: the 2.5D with wrong depth

Correct solution. The hybrid uses the depth from 2.1.2’s pose estimation.

Near-miss. The hybrid uses Z = 1.0 (constant).

Why the near-miss produces a plausible but incorrect number. The depth term in the hybrid directly affects the pose component. A wrong Z gives a wrong L_e, and the pose component diverges from the true one. The IBVS component may still converge, but the trajectory is not optimal and may cause the FOV to be violated.

The discrepancy. The 2.5D hybrid’s advantage is the combination of visibility (IBVS) and optimality (PBVS). Wrong depth undermines the pose component’s optimality while retaining the image component’s visibility. The result is a safe but suboptimal trajectory — better than PBVS alone (which would lose visibility) but worse than a correct hybrid.

Self-explanation 11. When is the constant-depth approximation acceptable?

Answer. When the object’s depth variation is small relative to the average depth (e.g., a flat target at fixed distance), and when the camera’s motion is small. For large motions or large depth variation, the constant-depth approximation degrades the pose component’s accuracy.

Part E — Failure Taxonomy, Lab, Cheat Sheet, Exercises, Exit Test

§L.5 — Failure taxonomy

§L.6 — The lab

§L.6.1 — Task

Implement IBVS and PBVS for the running case’s cube, verify the exponential convergence and the local minima, add the 2.5D hybrid, and break four ways.

§L.6.2 — Conditions

Python 3.10+, PyTorch 2.2+, CPU. Seed 42, dtype = float64. Runtime under 5 minutes. K = [[500, 0, 320], [0, 500, 240], [0, 0, 1]], image 640 × 480, cube side 0.05, initial camera pose identity, desired pose at t* = [0.5, 0.2, 0.5] with R* = Rz(π/4).

§L.6.3 — Controls held fixed

Seed, dtype, K, cube, initial pose, desired pose. Variants: sign error, wrong depth, wrong extrinsic frame, target loss.

§L.6.4 — Metrics table

§L.6.5 — Pre-registered hypotheses

• H1. IBVS converges to < 1 pixel in < 30 steps with λ = 0.3 and correct depth. Prediction: converged = True. Measured: 18 steps, final error 0.8888 px, visibility 1.00 — met.
• H2. IBVS with λ = 1.0 loses the target in < 5 steps. Prediction: visibility < 50%. Measured: at λ = 1.0 the servo converges in 6 steps at 5.077e-03 px with visibility 1.00 — refuted. The target is lost one gain notch higher: λ = 1.5 loses a corner on frame 1 and never recovers (vis = 0.01). The stability boundary is λ < 1 for exact depth, and 1.0 sits exactly on it.
• H3. PBVS converges in < 20 steps with λ = 0.3 and correct pose. Prediction: converged = True. Measured: 14 steps, final pose error 0.0087, visibility 1.00 — met.
• H4. PBVS with R* = Rz(π) loses the target. Prediction: visibility < 50%. Measured: 18 steps, final error 0.0074, visibility 1.00 — refuted. PBVS closes the pose loop directly, so a 180° goal costs four extra steps and no visibility. A π rotation about the camera axis swings the cube through the frame’s edge, but the servo’s linear decay keeps the intermediate poses inside it.
• H5. 2.5D hybrid with λ_IBVS = 0.3, λ_PBVS = 0.1 converges in < 40 steps with visibility > 90%. Prediction: converged = True, visibility > 0.9. Measured: 23 steps, final error 0.8718 px, visibility 1.00 — met, and 5 steps slower than pure IBVS’s 18 because the pose term adds translation the image term alone would not command.

§L.6.6 — Break-it sequence

1. Sign error. Flip v_c’s sign. Predicted: divergence. Actual: divergence — error grows from 488.1 to inf within 60 steps, visibility collapses to 0.07. Predicted correctly.
2. Wrong depth. Scale the depth estimate by k, holding the measured pixels exact. Predicted: Ẑ = 10Z converges 10× slower. Actual: Ẑ = 10Z throws the cube out of frame on frame 1 and never recovers — the gain is 10λ = 3.0, far past the stability boundary. The prediction was backwards. The measured behaviour is a pure gain rescale: Ẑ = 0.1Z needs 185 steps (gain 0.03), Ẑ = 0.5Z needs 34, Ẑ = 2Z needs 17 and still converges, Ẑ = 10Z diverges. Underestimate → slow; overestimate → fast, then unstable.
3. Wrong extrinsic. Apply the twist in a frame rotated by Rz(π/2) — the eye-in-hand / eye-to-hand confusion. Predicted: wrong axis, non-convergence. Actual: divergence, as predicted, but the interesting part is the magnitude: the error grows only to 542 over 60 steps at Rz(π/2) and the cube stays visible 85% of the time, so a short run looks like a slow servo rather than a broken one. At Rz(π) it reaches inf with visibility 0.18. Predicted correctly, with a caveat worth recording: the π/2 case is a quiet failure and a 10-step run would not distinguish it from a small gain.
4. Target loss. Set λ = 3.0. Predicted: target exits FOV within 5 steps. Actual: exits on frame 1, error inf, visibility 0.02. Predicted correctly — but note the threshold is λ ≥ 1.5, not 3.0; λ = 1.0 converges in 6 steps and is the last stable gain.

§L.6.7 — Acceptance criteria

§L.6.8 — What good looks like

1. IBVS’s exponential convergence demonstrated with ||e|| decreasing monotonically — and the canary printed beside it: L L̂⁺’s eigenvalues at Ẑ = kZ run 1 .. k, so the depth break-it’s direction is predicted by the spectrum before the run.
2. PBVS’s exponential convergence demonstrated in pose, and its gain boundary located: λ = 1.0 converges in 3 steps, λ = 1.5 already loses a corner.
3. The 2.5D hybrid’s trajectory shown to be shorter than IBVS and more visible than PBVS — and reported honestly when it is slower (23 vs 18 steps here).
4. The four break-its predicted with symptoms (divergence, slow, wrong axis, out of view) before running, and the prediction scored: 3/4, with #2 recorded as a refuted prediction rather than quietly amended.
5. The visibility constraint quantified (target pixels stay in bounds), not just assumed — visibility is the fraction of frames with all 8 corners inside [0,W]×[0,H], not merely in front of the camera.

§L.6.9 — Runnable lab code

# lab_2_1_3.py — IBVS, PBVS, and the 2.5D hybrid
import torch
import numpy as np

torch.manual_seed(42)
dtype = torch.float64
fx = fy = 500.0
cx, cy = 320.0, 240.0
W, H = 640, 480
LAM = 0.3          # per-step gain; one step = one unit of time

def cube_corners(side=0.05):
    s = side / 2
    return torch.tensor([[-s,-s,-s],[s,-s,-s],[s,s,-s],[-s,s,-s],
                         [-s,-s,s],[s,-s,s],[s,s,s],[-s,s,s]], dtype=dtype)

def pose_from(p, R):
    """T_wc: CAMERA-to-world. The camera looks down its own -z."""
    T = torch.eye(4, dtype=dtype)
    T[:3,:3] = R
    T[:3,3] = torch.tensor(p, dtype=dtype)
    return T

def rot_z(q):
    c, s = np.cos(q), np.sin(q)
    return torch.tensor([[c,-s,0],[s,c,0],[0,0,1]], dtype=dtype)

def to_camera(T_wc, X_w):
    """World points -> camera frame. X_c = R^T (X_w - p)."""
    return (X_w - T_wc[:3,3]) @ T_wc[:3,:3]

def project(X_cam):
    Z = X_cam[:,2]
    valid = Z > 1e-6
    Zs = Z.clamp(min=1e-6)
    u = fx * X_cam[:,0] / Zs + cx
    v = fy * X_cam[:,1] / Zs + cy
    return torch.stack([u,v], dim=-1), valid

def in_image(s):
    return (s[:,0] > 0) & (s[:,0] < W) & (s[:,1] > 0) & (s[:,1] < H)

def interaction(X_cam, depth_scale=1.0):
    """Pixel-coordinate interaction matrix L_u (2x6 per point).
    depth_scale multiplies the DEPTH ESTIMATE only: the pixels are measured,
    so u, v stay exact while every 1/Z becomes 1/(depth_scale*Z)."""
    rows = []
    for X, Y, Z in X_cam.tolist():
        u, v = fx*X/Z, fy*Y/Z
        Zh = Z * depth_scale
        rows.append([[-fx/Zh, 0.0,   u/Zh,  u*v/fx, -(fx*fx + u*u)/fx,  v],
                     [0.0,   -fy/Zh, v/Zh, (fy*fy + v*v)/fy, -u*v/fy, -u]])
    return torch.tensor(rows, dtype=dtype).reshape(-1, 6)

def exp6(xi):
    """se(3) exponential, xi = [v; w] in the camera (body) frame."""
    v, w = xi[:3], xi[3:]
    th = w.norm().item()
    Kx = torch.tensor([[0,-w[2],w[1]],[w[2],0,-w[0]],[-w[1],w[0],0]], dtype=dtype)
    T = torch.eye(4, dtype=dtype)
    if th < 1e-9:
        T[:3,:3] += Kx
        T[:3,3] = v
        return T
    a = np.sin(th)/th
    b = (1-np.cos(th))/th**2
    c = (th-np.sin(th))/th**3
    T[:3,:3] = torch.eye(3, dtype=dtype) + a*Kx + b*(Kx@Kx)
    T[:3,3] = (torch.eye(3, dtype=dtype) + b*Kx + c*(Kx@Kx)) @ v
    return T

X_obj = cube_corners(0.05)
T_star = pose_from([0.15, 0.08, -0.5], rot_z(np.pi/4))
s_star, _ = project(to_camera(T_star, X_obj))
T0 = pose_from([0.0, 0.0, -0.9], torch.eye(3, dtype=dtype))

def ibvs_step(T, lam=LAM, wrong_sign=False, depth_scale=1.0):
    X_cam = to_camera(T, X_obj)
    s, vis = project(X_cam)
    m = vis & in_image(s)
    if int(m.sum()) < 4:
        return T, float("inf"), int(m.sum())
    e = (s[m] - s_star[m]).reshape(-1)
    L = interaction(X_cam[m], depth_scale)
    xi = (-lam if not wrong_sign else lam) * (torch.linalg.pinv(L) @ e)
    return T @ exp6(xi), e.norm().item(), int(m.sum())

def pbvs_step(T, lam=LAM, T_goal=None):
    goal = T_star if T_goal is None else T_goal
    T_ce = torch.linalg.inv(T) @ goal
    R_e, t_e = T_ce[:3,:3], T_ce[:3,3]
    th = np.arccos(np.clip((R_e.trace().item()-1)/2, -1.0, 1.0))
    if th < 1e-9:
        u = torch.zeros(3, dtype=dtype)
    else:
        u = torch.tensor([R_e[2,1]-R_e[1,2], R_e[0,2]-R_e[2,0], R_e[1,0]-R_e[0,1]],
                         dtype=dtype) / (2*np.sin(th))
    e = torch.cat([t_e, th*u])
    S = torch.tensor([[0,-t_e[2],t_e[1]],[t_e[2],0,-t_e[0]],[-t_e[1],t_e[0],0]], dtype=dtype)
    L = torch.zeros(6, 6, dtype=dtype)
    L[:3,:3] = -torch.eye(3, dtype=dtype)
    L[:3,3:] = S
    L[3:,3:] = -torch.eye(3, dtype=dtype)
    xi = -lam * (torch.linalg.pinv(L) @ e)
    return T @ exp6(xi), e.norm().item()

def hybrid_step(T, lam_ibvs=LAM, lam_pbvs=0.1):
    X_cam = to_camera(T, X_obj)
    s, vis = project(X_cam)
    m = vis & in_image(s)
    if int(m.sum()) < 4:
        return T, float("inf"), int(m.sum())
    e = (s[m] - s_star[m]).reshape(-1)
    xi = -lam_ibvs * (torch.linalg.pinv(interaction(X_cam[m])) @ e)
    t_e = (torch.linalg.inv(T) @ T_star)[:3,3]
    xi[:3] += -lam_pbvs * t_e
    return T @ exp6(xi), e.norm().item(), int(m.sum())

def run(stepfn, n=200, tol=1.0, **kw):
    T = T0.clone()
    errs, full_vis, first_loss = [], [], None
    for k in range(n):
        X_cam = to_camera(T, X_obj)
        s, vis = project(X_cam)
        if not bool(((vis & in_image(s)).sum() == 8)):
            if first_loss is None:
                first_loss = k
        full_vis.append(bool(((vis & in_image(s)).sum() == 8)))
        out = stepfn(T, **kw)
        T, e = out[0], out[1]
        errs.append(e)
        if e < tol:
            break
    return {"errs": errs, "steps": len(errs), "final": errs[-1],
            "vis": float(np.mean(full_vis)), "first_loss": first_loss}

# --- canary: the depth-error direction, in closed form ---
X_cam0 = to_camera(T0, X_obj)
L_true = interaction(X_cam0)
print("[canary] eigenvalues of L @ pinv(L_hat) for Z_hat = k Z:")
for k in (0.1, 0.25, 1.0, 2.0, 4.0, 10.0):
    M = L_true @ torch.linalg.pinv(interaction(X_cam0, k))
    ev = torch.linalg.eigvals(M).real
    ev = ev[ev.abs() > 1e-8]
    _sm, _vm = project(X_cam0)
    _mm = _vm & in_image(_sm)
    _em = (_sm[_mm] - s_star[_mm]).reshape(-1)
    _xi = -LAM * (torch.linalg.pinv(interaction(X_cam0[_mm], k)) @ _em)
    _ratio = float((_em + interaction(X_cam0[_mm]) @ _xi).norm() / _em.norm())
    print(f"  k={k:5.2f}: eig {ev.min().item():.3f} .. {ev.max().item():.3f} | e1/e0 = {_ratio:.3f}")

# --- canary: measured first-step contraction vs the linear prediction ---
print("[canary] first-step contraction e1/e0 (linear model predicts |1-lam|):")
_s, _v = project(X_cam0)
_m = _v & in_image(_s)
_e0 = (_s[_m] - s_star[_m]).reshape(-1)
for lam in (0.3, 0.5, 0.7, 1.0):
    xi = -lam * (torch.linalg.pinv(interaction(X_cam0[_m])) @ _e0)
    _s1, _ = project(to_camera(T0 @ exp6(xi), X_obj))
    _e1 = (_s1[_m] - s_star[_m]).reshape(-1)
    print(f"  lam={lam}: linear {abs(1-lam):.3f} | measured {float(_e1.norm()/_e0.norm()):.3f}")

print("\n--- hypotheses ---")
r = run(ibvs_step)
print(f"H1 IBVS lam=0.3:  steps={r['steps']} final={r['final']:.4f} conv={r['final']<1.0} vis={r['vis']:.2f}")
for lam in (1.0, 1.5, 2.0):
    r = run(ibvs_step, lam=lam)
    print(f"H2 IBVS lam={lam}: steps={r['steps']} final={r['final']:.3e} vis={r['vis']:.2f} first_loss={r['first_loss']}")
r = run(pbvs_step, tol=0.01)
print(f"H3 PBVS lam=0.3:  steps={r['steps']} final={r['final']:.4f} conv={r['final']<0.01} vis={r['vis']:.2f}")
T_pi = pose_from([0.15, 0.08, -0.5], rot_z(np.pi))
r = run(pbvs_step, tol=0.01, T_goal=T_pi)
print(f"H4 PBVS R*=Rz(pi): steps={r['steps']} final={r['final']:.4f} vis={r['vis']:.2f}")
r = run(hybrid_step)
print(f"H5 hybrid:        steps={r['steps']} final={r['final']:.4f} vis={r['vis']:.2f}")

print("\n--- break-its ---")
r = run(ibvs_step, n=60, wrong_sign=True)
print(f"B1 wrong sign:    {r['errs'][0]:.1f} -> {r['final']:.3e} grew={r['final']>r['errs'][0]} vis={r['vis']:.2f}")
for k in (10.0, 2.0, 0.5, 0.1):
    r = run(ibvs_step, n=300, depth_scale=k)
    print(f"B2 Z_hat={k}xZ:    steps={r['steps']} final={r['final']:.3f} conv={r['final']<1.0} vis={r['vis']:.2f}")
r = run(ibvs_step, n=60, lam=3.0)
print(f"B4 lam=3.0:       steps={r['steps']} final={r['final']:.3e} vis={r['vis']:.2f} first_loss={r['first_loss']}")

§L.7 — The cheat sheet

§L.8 — Exercises with solutions

E 2.1.3.1 — The interaction matrix (numerical). Point at (0.3, 0.1, 0.8). Compute L_x.

S 2.1.3.1. x = 0.375, y = 0.125. L_x = [[-1.25, 0, 0.469, 0.047, -1.141, 0.125], [0, -1.25, 0.156, 1.016, -0.047, -0.375]]. The lesson: the -1/Z term dominates; rotation columns are O(1).

E 2.1.3.2 — The convergence rate (numerical). λ = 0.4. How many steps to reach 1% of initial error?

S 2.1.3.2. 0.6^k = 0.01 → k = ln(0.01)/ln(0.6) ≈ 9.0. The lesson: rate 0.4 gives about 9 steps to 1% error.

E 2.1.3.3 — The depth error (diagnostic). L̂_s uses Ẑ = 2Z. What is the convergence rate?

S 2.1.3.3. L_s L̂_s⁺ ≈ 0.5 I on translation. Convergence rate ≈ λ/2. The lesson: depth overestimate halves the effective rate.

E 2.1.3.4 — The PBVS pose error (numerical). T = (I, (0.4, 0.1, 0.6)), T* = (I, (0.5, 0.2, 0.5)). Compute e.

S 2.1.3.4. t_err = (-0.1, -0.1, 0.1), θ = 0, e = (-0.1, -0.1, 0.1, 0, 0, 0). The lesson: when R = R*, the error is purely translational.

E 2.1.3.5 — The sign error (diagnostic). The error doubles every step. What is the bug?

S 2.1.3.5. The sign of v_c is flipped. ė = +λ L_s L̂_s⁺ e causes exponential growth. The lesson: check whether error decreases; if not, sign is wrong.

E 2.1.3.6 — The local minimum (diagnostic). IBVS converges but the pose is 180° rotated from desired. What is the cause?

S 2.1.3.6. Local minimum from symmetric configuration. Image error is zero at 180° flip. The lesson: add a symmetry-breaking feature (depth, prior, additional point).

E 2.1.3.7 — The FOV violation (diagnostic). PBVS drives the camera through the target. What is the cause?

S 2.1.3.7. PBVS’s optimal trajectory rotates through the target. The lesson: use hybrid, or clip the trajectory to keep visibility.

E 2.1.3.8 — The 2.5D hybrid (constructive). Write the hybrid control’s pseudocode.

S 2.1.3.8.

function hybrid_control(s, s_star, T_cam, T_star, lambda_ibvs=0.3, lambda_pbvs=0.1):
    e_2d = s - s_star
    L_s = build_interaction(s, T_cam)
    v_ibvs = -lambda_ibvs * pinv(L_s) @ e_2d
    t_err = translation_of(inv(T_cam) @ T_star)
    v_ibvs[:3] += -lambda_pbvs * t_err      # pose term adds TRANSLATION only
    return v_ibvs

The lesson: the hybrid combines image feedback (visibility) and pose feedback (optimality). The pose term contributes translation, not a full 6-DOF pose error — the rotation is already observable in the image, and adding it twice over-drives the rotation. Measured on the running case: the translation-only form converges in 23 steps; adding the full pose error reaches the same tolerance in 13, i.e. it is faster, so “adds magnitude” is not by itself an argument against it — but it changes the trajectory, and the visibility it buys or costs must be measured, not assumed.

§L.9 — Exit test

You are ready for Lesson 2.1.4 (Point Clouds and Registration) when you can, without notes:

1. State the visual-servoing problem and the two configurations.
2. State the IBVS control law and the interaction matrix.
3. Derive the L_x’s 1/Z scaling and the rotation independence.
4. State the IBVS’s exponential convergence and the depth sensitivity.
5. State the PBVS control law and the pose interaction matrix.
6. State PBVS’s global stability (linearized) and the FOV risk.
7. State the local minima (symmetric, coplanar) and their diagnosis.
8. State the 2.5D hybrid and its advantage.
9. State the visibility condition and the gain’s constraint.
10. Diagnose the sign error (divergence), wrong depth (overestimate → gain kλ, overshoot then divergence; underestimate → slow), wrong extrinsic (wrong axis), and target loss (FOV).
11. State the calibration robustness of IBVS vs. PBVS.
12. State the choice criterion: small motion → IBVS; large motion with known pose → PBVS; both → hybrid.

If any is not answerable, return to the section: 1 to §L.1, 2 to §L.2, 3 to §L.2, 4 to §L.2.1, 5 to §L.3, 6 to §L.3.1, 7 to §L.2.1, 8 to §L.4, 9 to §L.4, 10 to §L.5, 11 to §L.4, 12 to §L.4.

End of Lesson 2.1.3. The learner has now completed visual servoing: IBVS with its interaction matrix and local minima, PBVS with its global stability and FOV risk, and the 2.5D hybrid that avoids both. The next lesson, 2.1.4 — Point Clouds and Registration, uses the depth data from 2.1.1 to reconstruct 3D structures and align them — the geometric foundation for manipulation and SLAM.