7 On the Propagation of CurvesSnake Model (1987) [Kass-Witkin-Terzopoulos]Planar parameterized curve C:R-->RxRA cost function defined along that curveThe internal term stands for regularity/smoothness along the curve and has two components (resisting to stretching and bending)The image term guides the active contour towards the desired image properties (strong gradients)The external term can be used to account for user-defined constraints, or prior knowledge on the structure to be recoveredThe lowest potential of such a cost function refers to an equilibrium of these terms

9 Optimizing Active ContoursTaking the Euler-Lagrange equations:That are used to update the position of an initial curve towards the desired image propertiesInitial the curve, using a certain number of control points as well as a set of basic functions,Update the positions of the control points by solving the above equationRe-parameterize the evolving contour, and continue the process until convergence of the process…

10 Pros/Cons of such an approachLow complexityEasy to introduce prior knowledgeCan account for open as well as closed structuresA well established technique, numerous publications it worksUser InteractivityDemetri Terzopoulos is a very good friendConsSelection on the parameter space and the sampling rule affects the final segmentation resultEstimation of the internal geometric properties of the curve in particular higher order derivativesQuite sensitive to the initial conditions,Changes of topology (some efforts were done to address the problem)

13 The Level Set MethodLet us consider in the most general case the following form of curve propagation:Addressing the problem in a higher dimension…The level set method represents the curve in the form of an implicit surface:That is derived from theinitial contour accordingto the following condition:

14 The Level Set Method Construction of the implicit functionAnd taking the derivative with respect to time (using the chain rule)And we are DONE…(1)

15 The Level Set MethodLet us consider the arc-length (c) parameterization of the curve, then taking the directional derivative of in that dire- ction we will observe no change:leading to the conclusion that the is ortho-normal to C wherethe following expression for the normal vectorEmbedding the expression of the normal vector to:the following flow for the implicit function is recovered:(2)

16 Level Set Method (the basic derivation)Where a connection between the curve propagation flow and the flow deforming the implicit function was establishedGiven an initial contour, an implicit function is defined and deformed at each pixel according to the equation (2) where the zero-level set corresponds to the actual position of the curve at a given frameEuclidean distance transforms are used in most of the cases as embedding function

17 Overview of the MethodThe level set flow can be re-written in the following formwhere H is known to be the Hamiltonian. Numerical approximations is then done according to the form of the HamiltonianDetermine the initial implicit function (distance transform)Evolve it locally according to the level set flowRecover the zero-level set iso-surface (curve position)Re-initialize the implicit function and Go to step (1) of the loopComputationally expensiveOpen Questions: re-initialization…and numerical approximations

19 Level Set Method and Internal Curve PropertiesThe normal to the curve/surface can be determined directly from the level set function:The curvature can also be recovered from the implicit function, by taking the second order derivative at the arc lengthWhere we observe no variation since the implicit function has constant “zero” values, and given that as well as one can easily prove that:That can also be extended to higher dimensions

21 From theory to Practice (Narrow Band) [Chop:93, Adalsteinsson-Sethian:95]Central idea: we are interested on the motion of the zero-level set and not for the motion of each iso-phote of the surfaceExtract the latest positionDefine a band within a certain distanceUpdate the level set functionCheck new position with respectthe limits of the bandUpdate the position of the bandregularly, and re-initialize the implicit functionSignificant decrease on the computational complexity, in particular when implemented efficiently and can account for any type of motion flows

23 Handling the Distance FunctionThe distance function has to be frequently re-initialized…Extraction of the curve position & re-initialization:Using the marching cubes one can recover the current position of the curve, set it to zero and then re-initialize the implicit function: the Borgefors approach, the Fast Marching method, explicit estimation of the distance for all image pixels…Preserving the curve position and refinement of the existing function (Susman-smereka-osher:94)Modification on the level set flow such that the distance transform property is preserved (gomes-faugeras:00)Extend the speed of the zero level set to all iso-photes, rather complicated approach with limited added value?

24 From theory to Practice (Fast Marching) [Tsitsiklis:93,Sethian:95]Central idea: “move” the curve one pixel in a progressive manner according to the speed function while preserving the nature of the implicit functionConsider the stationary equationSuch an equation can be recovered for all flows where the speed function has one sign (either positive or negative), propagation takes place at one directionIf T(x,y) is the time when the implicit function reaches (x,y):

25 Fast Marching (continued)Consider the stationary equation in its discrete form:And using the assumptionthat the surface propaga-tes in one direction, the so-lution can be obtained byoutwards propagation fromthe smallest T value…active pixels, the curve has already reached themalive pixels, the curve could reach them at the next stagefar away pixels, the curve cannot reach them at this stage

26 Fast Marching (continued)INITIAL STEPInitialize for the all pixels of the front (active), their first order neighbors alive and the rest far awayFor the first order neighbors,estimate the arrival time according to:While for the rest the crossing time is set to infinityPROPAGATION STEPSelect the pixel with the lowest arrival time from the alive onesChange his label from alive to active and for his first order neighbors:If they are alive, update their T value according toIf they are far away, estimate the arrival time according to:

27 Fast Marching Pros/Cons, Some ResultsFast approach for a level set implementationVery efficient technique for re-setting the embedding function to be distance transformSingle directional flows, great importance on initial placement of the contoursAbsence of curvature related terms or terms that depend on the geometric properties of the curve…Results are courtesy: J. Sethian, R. Malladi, T. Deschamps, L. Cohen

29 Emigration from Fluid Dynamics to Vision(Caselles-Cate-Coll-Dibos:93,Malladi-Sethian-Vemuri:94) have proposed geometric flows to boundary extractionWhere g(;) is a function that accounts for strong image gradientsAnd the other terms are application specific…that either expand or shrink constantly the initial curveDistance transforms have been used as embedding functionsMalladi-Sethian-Vemuri:94

30 Geodesic Active Contours [Caselles-Kimmel-Sapiro:95, Kichenassamy-Kumar-etal95]Connection between level set methods and snake driven optimizationThe geodesic active contour consists of a simplified snake model without second order smoothnessThat can be written in a more general form asWhere the image metric has been replaced with a monotonically decreasing function:

31 Geodesic Active Contours [Caselles-Kimmel-Sapiro:95, Kichenassamy-Kumar-etal95]Leading to the following more general framework…,One can assume that smoothness as well as image terms are equally important and with some “basic math”That seeks a minimal length geodesic curve attracted by the desired image properties…

32 Geodesic Active ContoursThat when minimized leads to the following geometric flow:Data-driven constrained by the curvature forceGradient driven term that adjusts the position of the contour when close to the real 0bject boundaries…By embedding this flow to a level set framework and using a distance transform as implicit function,

33 Geodesic Active Contours…That has an extra term when compared with the flow proposed by Malladi-Sethian-Vemuri.Single directional flow…requires the initial contour to either enclose the object or to be completely inside...Results are courtesy: R. Deriche

34 Gradient Vector Flow Geometric Contours [paragios-mellina-ramesh:01]Initial conditions are an issue at the active contours since they are propagated mainly at one directionRegion terms (later introduced) isa mean to overcome this limitation…an alternative is somehow to extendthe boundary-driven speed function to account for directionality, thus recovering a field (u,v)One can estimate this field close to the object boundaries…whereThe image gradient at the boundaries is tangent to the curveWhile the inward normal normal points towards the object boundaries

35 Gradient Vector Flow Geometric Contours [paragios-mellina-ramesh:01]Let (f) be a continuous edge detector with values close to 1 at the presence of noise and 0 elsewhere…The flow can be determined in areas with important boundary information (Important f)And areas where there changes on f, |Gradient(f)|While elsewhere recovering such a field is not possible and the only way to be done is through diffusionThis can be done through an approximation of image gradient at the edges and diffusion of this information for the rest of the image plane

36 Gradient Vector Flow Geometric ContoursThis flow can be used within a geometric flow towards image segmentation…The direction of the propagation should be the same with the one proposed by the recovered flow, therefore one can penalize the orientation between these two vectors.That is integrated within the classicalGeodesic active contour equation and isimplemented using the level set functionusing the Additive Operator SplittingThe inner product between the curvenormal and the vector field guides the curve propagation

37 Additive Operator Splitting [Weickert:98, Goldenberg-Kimmel:01]Introduced for fast non-linear diffusionApplied to the flow of the geodesic active contourWhere one can consider a signed Euclidean distance function to be the implicit function, leading to:That can be written as:That can be solved in an explicit form:Or a semi-implicit one:

38 Additive Operator Splitting (Weickert:02)Or in a semi-implicit oneThat refers to a triagonal system of equations and can be done using the Thomas algorithm…at O(N) and has to be done once…

41 The Mumford-Shah framework [chan-vese:99, yezzi-tsai-willsky-99]The original Mumford-Shah framework aims at partitioning the image into (multiple) classes according to a minimal lengthcurve and reconstructing the noisy signal in each classLet us consider - a simplified version - the binary case and the fact that the reconstructed signal is piece-wise constantWhere the objective is to reconstructthe image, using the mean values for theinner and the outer regionTractable problem, numerous solutions…

42 The Mumford-Shah framework [chan-vese:99, yezzi-tsai-willsky-99]Taking the derivatives with respect to piece-wise constants, it straightforward to show that their optimal value corresponds to the means within each region:While taking the derivatives with respect and using the stokes theorem, the following flow is recovered for the evolution of the curve:An adaptive (directional/magnitude)-wise balloon forceA smoothness force aims at minimizing the length of the partitionThat can be implemented in a straightforward manner within the level set approach

43 The Mumford-Shah framework – Criticism & ResultsAccount for multiple classes?Quite simplistic model, quite often the means are not a good indicator for the region statisticsAbsence of use on the edges, boundary information

44 Geodesic Active Regions [paragios-deriche:98]Introduce a frame partition paradigm within the level set space that can account for boundary and global region-driven informationKEY ASSUMPTIONSOptimize the position and the geometric form of the curve by measuring information along that curve, and within the regions that compose the image partition defined by the curve:(input image) (boundary) (region)

45 Geodesic Active RegionsWe assume that prior knowledge on the positions of the objects to be recovered is available as well as on the expected intensity properties of the object and the background

46 Geodesic Active RegionsSuch a cost function consists of:The geodesic active contourA region-driven partition module that aims at separating the intensities properties of the two classes (see later analogy with the Mumford-Shah)And can be minimized using a gradient descent method leading to:Which can be implemented using the level set method as follows…

50 Level Set & Geometric FlowsWhile evolving moving interfaces with the level set method is quite attracting, still it has the limitation of being a static approachThe motion equations are derived somehow,The level set is used only as an implementation tool…That is equivalent with saying that the problem has been somehow already solved…since there is not direct connection between the approach and the level set methodology

53 Level Set DictionaryUsing the Dirac function and integrating within the image domain, one can estimate the length of the curve:While integrating the Heaviside Distribution within the image domainSuch observations can be used to define regional partition modules as follows according to some descriptorsThat can be optimized with respect to the level set function (implicitly with respect to a curve position)

54 Level Set OptimizationAnd given that :An adaptive (directional & magnitude wise) flow is recovered for the propagation of an initial surface towards a partition that is optimal according to the regional descriptors…The same idea can be used to introduce contour-driven terms…

55 Level Set Optimizationand optimize them directly on the level set spaceCurve-driven terms:Global region-driven terms:According to some image metrics…defined along the curve and within the regions obtained through the image partition according to the position of the curve, that can be multi-component but is representing only one class

56 Multiphase Motion [zhao-chan-merinman-osher:96]Up to now statistics and image information have been used to partition image into two classes,Often, we need more than object/background separation, and therefore the case of multi-phase motion is to be considered…N objects/curves, represented by N level set functionsHow to deal with occlusions,one image pixel cannot beassigned to more than one curve…How to constrain the solutionsuch that the obtained partitionconsists of all image data

57 Multi-Phase Motion (continued)For each class, boundary, smoothness as well as region components can be consideredSubject to the constraint at each pixel:a hard and local constraint difficult to be imposed that could be replaced with a more convenientThat can be optimized through Lagrange multipliers method…

58 Multiphase Motion & Mumford-Shah [samson-aubert-blanc-feraud:99]Image Segmentation and Signal Reconstruction (direct application of the (zhao-chan-merinman-osher:96) within the Mumford Shah formulation…)Separate the image into regions with consistent intensity propertiesRecover a Gaussian distribution that expresses the intensity properties of each class, or force the intensity properties of each class to follow some predefined image characteristicsThat when optimized leads to a set of equations that deforming simultaneously the initial curves according to:

60 Multi-Phase Motion PROS CONSTaking the level set method to another levelDealing with multiple (multi-component) objects, and multiple tasksIntroducing interactions between shape structures that evolve in parallelCONSComputationally expensiveDifficult to guarantee convergenceNumerically unstable & hard to implementPrior knowledge required on the number of classes and in some cases on their properties…PARTIAL SOLUTION: The multi-phase Chan-Vese model

61 Multi-Phase Motion [vese-chan:02]Introduce classification according to a combination of all level sets at a given pixelLEVEL SET DICTIONARYClass 1:Class 2:Class 3:Class 4:And therefore by taking these products one can define a modified version of the mumford-shah approach to account with four classes while using two level set functions…

63 Multi-Phase Motion with more advanced data-driven termsThe assumption of piece-wise constant is rather weak in particular in medical imaging…Several authors have proposed more advanced statistical formulations that are recovered “on the fly” to determine the statistics of each classThe case of non-parametric approximations of the histogram within each region is a promising direction

65 Knowledge-based Object ExtractionObjective:recover from the image a structureof a particular – known to some extend– geometric formMethodologyConsider a set of training examplesRegister these examples to a common poseConstruct a compact model that expresses thevariability of the training setGiven a new image, recover the area where theunderlying object looks like that one learntAdvantages of doing that on the LS space:Preserve the implicit geometryAccount with multi-component objects…… all wonderful staff you can do with the LS

66 Knowledge-based Segmentation [leventon-faugeras-grimson-etal:00]Concept: Alternate between segmentation& imposing prior knowledgeLearn a Gaussian distribution of theshape to be recovered from a trainingset directly at the space of implicit functionsThe elements of the training set are registeredA principal component analysis is use to recoverthe covariance matrix of probability density function of this setALTERNATEEvolve a let set function according to the geodesic active contourGiven its current form, deform it locally using a MAP criterion so it fits better with the prior distributionUntil convergence…

67 Knowledge-based Segmentation [leventon-faugeras-grimson-etal:00]Limitations:Data driven & prior term are decoupledBuilding density functions on high dimensional spaces is an ill posed problem,Dealing with scale and pose variations (they are not explicitely addressed)

68 Knowledge-based Segmentation [chen-etal:01]Concept level:Use an average model as prior in its implicit functionFor a given curve find the transformation that projects it closer to the zero-level set of the implicit representation of the priorFor a given transformation evolve the curve locally towards better fitting with the prior…Couple prior with the image driven term in a direct form…Issues to be addressed:Model is very simplistic (average shape) – opposite to the leventon’s case where it was too much complicated…Estimation of the projection between the curve and the model space is tricky…not enough support…data term can be improved…

70 Knowledge-based Segmentation [tsai-yezzi-etal:01]At a concept level, prior knowledge is modeled through a Gaussian distribution on the space of distance functions by performing a singular value decomposition on the set of registered training set,The mumford-shah framework determined at space of the model is used to segment objects according to various data-driven termsThe parameters of the projection are recovered at the same time with the segentation result…A more convenient approach than the one of Leventon-etalWhich suffers from not comparing directly the structure that is recovered with the model…

71 Knowledge-based Segmentation [paragios-rousson:02]Prior is imposed by direct comparison between the model and evolving contour modulo a similarity transformation…The model consists of a stochastic level set with two components,A distance map that refers to the average modelAnd a confidence map that dictates the accuracy of the modelObjective: Recover a level set that pixel-wise looks like the prior modulo some transformation

72 Model ConstructionFrom a training set recover the most representative model;If we assume N samples on the training set, then the distribution that expresses at a given point most of these samples is the one recovered through MAPWhere at a given pixel, we recover the mean and the variance that best describes the training set composed of implicit functions at this point, where the mean corresponds to the average valueConstraints on the variance to be locally smooth is a natural assumption

73 Model Construction (continued)The calculus of variations can lead to the estimation of the mean and variance (confidence measure) of the model at et each pixel,However, the resulting model will not be an implicit function in the sense of distance transform (averaging distance transforms doesn’t necessary produce one)One can seek for a solution of the previously defined objective function subject to the constraint the “means” field forms a distance transform using Lagrange multipliers…An alternative is to consider the process in repeated steps where first a solution that fits the data is recovered and then is projected to the space of distance functions…

74 Imposing the (Static) PriorDefine/recover a morphing function “A” that creates correspondence between the model and the priorIn the absence of scale variations, and in the case of global morphing functions one can compare the evolving contour with the model according toThat modulo the morphing function will evolve the contours towards a better fit with the modelOne can prove that scale variations introduce a multiplicative factor and they have to be explicitly taken into account

75 Static Prior (continued)Where the unknowns are the morphing function and the position of the level setCalculus of variations with respect to the position of the interface are straightforward:The second term is a constant inflation term aims at minimizing the area of the contour and eventually the cost function and can be ignored…since it has no physical meaning.

77 Static Prior (continued)One can also optimize the cost function with respect to the unknown parameters of the morphing functionLeading to a nice “self-sufficient” system of motion equations that update the global registration parameters between the evolving curve and the modelHowever, the variability of the model was not considered up to this point and areas with high uncertainties will have the same impact on the process

79 Taking Into Account the Model UncertaintiesMaximizing the joint posterior (segmentation/morphing) is a quite attractive criterion in “inferencing”Where the Bayes rule was considered and given that the probability for a given prior model is fixed and we can assume that all (segmentation/morphing) solutions are equally probable, we getUnder the assumption of independence...within pixels…and then finding the optimal implicit function and its morphing transformations is equivalent with

80 Taking Into Account the Model UncertaintiesThat can be further developed using the Gaussian nature of the model distribution at each image pixelA term that aims at recovering a transformation and a level set that when projected to the model, it is projected to areas with low variance (high confidence)A term that aims at minimizing the actual distance between the level set function and the model and is scaled according to the model confidence…would prefer have a better match between the model and level set in areas where the variability is low,while in areas with important deviation of the training set, this term will be less important

81 Taking the derivatives…Calculus of variations regarding the level set and the morphing function:The level set deformation flow consists of two terms:that is a constant deflation force (when the level set function collapses, eventually the cost function reaches the lowest potential)An adaptive balloon (directional/magnitude-wise) force that inflates/deflates the level set so it fits better with the prior after its projection to the model space…In areas with high variance this term become less significant and data-terms guide the level set to the real object boundaries...

85 Implicit Active Shapes [rousson-paragios:03]The Active Shape Model of Cootes et al. is quite popular to object extraction. Such modeling consists of the following steps:Let us consider a training set of registered surfaces (implicit representations can also be used for registration [4]). Distance maps are computed for each surface:The samples are centered with respect to the average representation :

86 Implicit Active Shapes [rousson-paragios:03]Training set:The principal modes of variation are recovered through Principal Component Analysis (PCA). A new shape can be generated from the (m) retained modes:

88 The priorA level set function that has minimal distance from a linear from the model space…The unknown consist of:The form of the implicit functionThe global transformation between the average mode and the image,The set of linear coefficients that when applied to the set of basis functions provides the optimal match of the current contour with the model spaceAnd are recovered in a straightforward manner using a gradient descent method…

About project

Feedback

To ensure the functioning of the site, we use cookies. We share information about your activities on the site with our partners and Google partners: social networks and companies engaged in advertising and web analytics. For more information, see the Privacy Policy and Google Privacy &amp Terms.
Your consent to our cookies if you continue to use this website.