Slide 1: Motion from normal flow
Slide 2: Optical flow difficulties • The aperture problem • Depth discontinuities
Slide 3: Translational Normal Flow u tr un  n Z • In the case of translation each normal flow vector constrains the location of the FOE to a half-plane. Intersection of half-planes provides FOE. •
Slide 4: Egoestimation from normal flow • Idea: choose particular directions: patterns defined on the sign of normal flow along particular orientation fields • positive depth constraint • 2 classes of orientation fields: copoint vectors and coaxis vectors
Slide 5: Optical flow and normal flow
Slide 6: Optical flow and normal flow
Slide 7: Coaxis vectors with respect to axis (A,B,C) P(Af/C, Bf/C
Slide 8: Coaxis vectors with respect to axis (A,B,C)
Slide 9: Coaxis vectors
Slide 10: Translational coaxis vectors
Slide 11: Translational coaxis vectors h passes through FOE and (Af/C, Bf/C), defined by 2 parameters
Slide 12: Rotational coaxis vectors
Slide 13: Rotational coaxis vectors
Slide 14: Rotational coaxis vectors g passes through AOR and (Af/C, Bf/C), defined by 1 parameter
Slide 15: Combine translation and rotation Positive + positive Negative + negative Positive + negative  positive negative don’t know (depends on structure)
Slide 16: translational Coaxis pattern rotational combined
Slide 17: -vectors: Translation
Slide 18: -vectors: Rotation u rot  r2   r2      sin      cos  f f  
Slide 19: -vectors: Translation and Rotation U W V W  
Slide 20: Three coaxis vector fields alpha beta gamma
Slide 21: Copoint vectors
Slide 22: copoint vectors O
Slide 23: Copoint vectors defined by point (r,s)
Slide 24: Translational copoint vectors
Slide 25: AOR AOR FOE FOE FOE AOR    AOR : Negative : Positive : Don't know FOE
Slide 26: Translational copoint vectors FOE (r, s) k passes through FOE and (r,s) defined by 1 parameter
Slide 27: Rotational copoint vectors
Slide 28: Rotational copoint vectors AOR (r, s) l passes through AOR and (r,s), is defined by 2 parameters
Slide 29: translational component rotational component FOE AOR (r, s) (r, s) FOE AOR (r, s)
Slide 30: Three coaxis vector fields (a) (b) (c)
Slide 31: a,b,c : positive and negative  vectors c,d,e: Fitting of  patterns g: Separation of (coaxis pattern h: Separation of (x0, y0) copoint pattern
Slide 32: AOR AOR FOE FOE FOE AOR    AOR : Negative : Positive : Don't know FOE
Slide 34: Optical illusion
Slide 36: What is the Problem? • Flow can be accurately estimated in an image patch corresponding to a smooth scene patch, • But erroneous flow estimates are obtained for image patches corresponding to scene patches containing discontinuities Image Flow Scene structure Discontinuities 3D Motion
Slide 37: Depth variability constraint • Errors in motion estimates lead to distortion of the scene estimates. • The distortion is such that the correct motion gives the “smoothest” (least varying) scene structure.
Slide 38: Depth estimation • Scene depth can be estimated from normal flow measurements: 1 un  u  n  u tr  n  u rot  n Z ˆ 1 u n  u rot ω  n  ˆ ˆ u tr (t )  n Z 
Slide 39: Visual Space Distortion ˆ u tr t  n ˆ Z  Z  D, D   u tr  t   u rot δω    n  • Wrong 3D motion gives rise to a rugged (unsmooth) depth function (surface). • The correct 3D motion leads to the “smoothest” estimated depth.
Slide 40: Inverse depth estimates correct motion incorrect motion
Slide 41: The error function • A normal flow measurement: 1 un  u tr  n  u rot  n Z 1 ˆ ˆ un  u tr (t )  n  u rot (ω)  n ˆ, ω For an estimate t ˆ Z • The error function to be minimized: 1  ˆ)  ni )  ˆ    Wi  un i  u rot (ω)  ni  ( )(u tr (t  ˆ Z   i 2
Slide 42: The error function • Estimated normal flow 1 ˆ ˆ ˆ un  u tr (t )  n  u rot (ω)  n ˆ Z • The error function to be minimized: ˆ    Wi  un  un  R i 2 • Global parameters: t , ω ˆˆ ˆ • Local parameter: Z 1  ax  by  c locally planar patches: ˆ Z
Slide 43: Error function evaluation ˆ • Given a translation candidate t , each local depth can be computed as a linear function ˆ. of the rotation ω • We obtain a second order function of the rotation; its minimization provides both the rotation and the value of the error function.
Slide 44: Is derived from image gradients only Brightness consistency: Flow: Planar patch:
Slide 45: Handling depth discontinuities • Given a candidate motion, the scene depth can be estimated and further processed to find depth discontinuities. • Split a region if it corresponds to two depth values separated in space.
Slide 46: The algorithm • Compute spatio-temporal image derivatives and normal flow. • Find the direction of translation that minimizes the depth-variability criterion. – Hierarchical search of the 2D space. – Iterative minimization. – Utilize continuity of the solution in time; usually the motion changes slowly over time.
Slide 47: Divide image into small patches Search in the 2D space of translations For each candidate 3D motion, using normal flow measurements in each patch, compute depth of the scene. For each image patch The Algorithm Depth variation small? NO Distinguish between two cases existence of a discontinuity at the patch Split the patch and repeat the process YES Use the error wrong 3D motion Use the error
Slide 49: 3D reconstruction • Comparison of the original sequence and the re-projection of the 3D reconstruction.
Slide 50: 3D model construction