Transcription
Pattern Recognition in Image Analysis IImageAnalysis:A l i ExtractionE t ti off knowledgekl d fromfimageidata.d t Pattern Recognition: Detection and extraction of patterns from data. Pattern: A subset of data that mayy be described byy some well-defined set of rules.Patterns may constitute the smallest entity in the data that represent knowledge.pixelthis is a patternPRIA01: Introduction, Klaus D. Toennies1
Applications of Pattern Recognition automatic analysis of medical images automatic inspection of parts on an assembly line hhumanspeechh recognitioniti byb computerst classification of seismic signals (e.g., for oil and mineral exploration) selection of tax returns to audit,, stocks to buy,y, ppeoplep to insure identification of people from fingerprints, retinal scans, handwriting, . automatic inspection of printed circuits, printed characters, handwriting recognition automatic analysis of satellite pictures (e.g., weather condition, water reserves,mineral prospects,.) Î Pattern recognition (in general as well as applied to images) is mainly aclassification task.Î Imagesg often constitute the data source for ppattern recognition.gPRIA01: Introduction, Klaus D. Toennies2
Pattern Recognition and Objects Patterns in images represent (attributes of) objects Pattern recognition tasks are– Object detection tasksto find an unknown number of instances of a known kind of an objectin the imageg– Object recognition tasksto recognise a detected object as one of a specific kindPRIA01: Introduction, Klaus D. Toennies3
Detection and Recognition DDetectiont ti techniques:t h isegmentation, objectmatching, searchingtechniques Recognition techntechniques: feature computation/ reduction,classification,l ifi ticlusteringPRIA01: Introduction, Klaus D. Toennies4
Detection and Recognition May be used independently of each other I combinationInbi ti as process chainh i– Detection (and segmentation) separates object detail of objects of some broadg „cars“)„)class ((e.g.– Recognition separates detected objects in sub-classes (e.g. „sports cars“,„limousines“, ) Some problems need an integrated solution (e.g. „separate different car typesdirectly in an image“) I tIntegratedt d solutionl ti isi more „human-lokehl k vision“i i “ butb t muchh more difficultdiffi lt– Potentially every pixel contributes to the solution– Features are not guaranteed to belong to an object to be classifiedPRIA01: Introduction, Klaus D. Toennies5
Classical Solution Pattern recognition as a processing pipeline– (Segmentation)– Feature computation– ClassificationPRIA01: Introduction, Klaus D. Toennies6
Pattern Recognition as a ClassificationTaskFeatures: (f1, f2, ., fn)Objects to be classified are described by features.Features are evaluated to separate objects intodifferent classes.Features: (f1, f2, ., fn)PRIA01: Introduction, Klaus D. Toennies7
Feature DetectionAn important prerequisite for feature detection in images is the extraction of structures thathave common feature values Î Segmentation.Why?Often, features are not computed from single pixels but from pixel sets. Theircomputation is erroneous if feature values change over the tionSegmentation and Pattern Recognition:Knowledge on the features to be evaluatedgreatly enhances the segmentation success.Pattern RecognitionPRIA01: Introduction, Klaus D. Toennies8
ClassificationClassification: grouping patterns (samples) according to their features into differentclasses.How do I decide this? Decide which features are relevant to the problem and decide on a way tocompute them. Decideid on a generall classificationl ifi i techniqueh i(based(b d on theh type off features)f) Train a classifier based on samples of the images to be analysed:-Find out a differentiation between different meanings based on featurecharacteristics.-Find a suitable generalisation of the feature values based on the training set EstimateEitheh error off theh classifierl ifi usingi an independent,i ddrepresentativei test datadset. Apply the classifier to images of the problem domain.PRIA01: Introduction, Klaus D. Toennies9
Classification Techniques Statistical pattern recognition: Assume that the pattern is a sample of from anumber of known distributions and assign it to the one class to which it mostlikely belongs to.Requires a feature vector as input information.Solutions are, e.g., kNN-classifier, backpropagation networks, supportvectorector machines Semantical pattern recognition: Assume that the features of a pattern followsknown laws / rules for constructing the pattern and assign it to the class whoserules are most closely followed.Requires a feature structure to reflect the rulesSolutionsSl tiare grammars (not( t oftenft usedd ini imageianalysis)l i ) andd graphhmatching techniquesPRIA01: Introduction, Klaus D. Toennies10
Is Pattern Recognition Equivalent toImageg Analysis?yTask: Detecting windows and doors inimageimage. Doors are rotated with respect to camera Windows are partially occludedScene is 3-d but features are from 2-dpicture.Feature-based pattern recognition from 3-d scenes: Features are those of the objectj and not that of its projection!p j Pattern recognition may require 3-d surface reconstruction prior to analysis.PRIA01: Introduction, Klaus D. Toennies11
Useful and not so UsefulFeaturesThis should have similarappearances!ppUseful features pertain to objectUseless features are those pertaining toimaging the scene (e.g., highlights .)PRIA01: Introduction, Klaus D. Toennies12
Finding the Features Reconstruct 3-d surfaces ( 3-d computer vision) Induce locations of light sources S th i influencesSynthesisei flfromflightli ht sources, surfacefcurvature,tcamera positioniti Extract object features after accounting for synthesised influencesThis is much too complicated in real-world applications!Solutions: looking at 2-d scenes only for feature-based pattern recognition using reconstructed 3-d geometric features (fitting methods)PRIA01: Introduction, Klaus D. Toennies13
2-d Scenes and Almost 2-d Scenes Medical images such as CT, MRI, etc. satellite images and aerial photos ddrawingsiandd textt t imagesi some images from computer aided manufacturingPRIA01: Introduction, Klaus D. Toennies14
What if it is truly 3-d?What,3 d?These cars belong to the same class whilethis one does notPRIA01: Introduction, Klaus D. Toennies15
The not-so-classical way:Fitting a ModelGenerate modelsGd l off theh classl andd decided id on qualityliof fit.Model:Md l geometrical, 3-d model: fit edges to modeledges. Picture 2-d2 d model (set of 2-d2 d pictures): fit tointerpolation from pictures.Why?WhHuman Vision: There must besome fast classification that maybe later refined.refinedPRIA01: Introduction, Klaus D. Toennies16
3-d Model Fitting Rough classification of an object 3-d model may aid to segmentation in 3-d being pre-requisite to PRtechniques 2 d model may allow for segmentation free classification2-dProblem: scene may contain more than just the object (i.e., segmentation beforesegmentation) and in may be distorted by artefacts / noise.Î Find the “important”important features in an image (saliency of features)PRIA01: Introduction, Klaus D. Toennies17
Fitting 2-d to 2-d Top-Down-Approach Alternative to segmentation-tofeatures-to-objectj Does not requires segmentationSimple road model Requires appropriate model– Capturingpg the essence of membersof an object class– Capturingpg acceptablepvariabilityyPRIA01: Introduction, Klaus D. Toennies18
Top-Down vs. Bottom-Up Bottom-Up Classical Pattern Recognition– GenerationGti off (potentially)( t ti ll ) meaningfuli f l featuresf t– Conceptual distance between features and semantics is large (shape features,texture features)– Requires elaborated classification techniques but feature generation is (largely)domain independent Top-Down Model Fitting– Generation of a meaning model of object‘s appearance in an image– ConceptualCl distancedibbetweenmodeld l andd semanticsi isi smallll– Simple classification in low-dimensional feature space but domain-dependentPRIA01: Introduction, Klaus D. Toennies19
What this course will be about.Segmentation gtechniques,q , deformable templates,p, texture segmentationgReview segmentationFeatures Feature detection, feature representation, feature reductionPattern Recognition Techniques statistical, semantic, and neural network pattern recognition. combined with a project on texture segmentationAim of the course is to give an overview on techniques and applications in patternrecognition as applied to image analysis.PRIA01: Introduction, Klaus D. Toennies20
Background / LiteratureRequirementsBasic knowledge of image processing (what is a filter, a gradient, what issegmentation, etc.)Literature R.O.Duda, P.E.Hart, D.G.Stork, Pattern Classification, 2nd edition, 2001, Wiley andSons. E. Gose,, R. Johnsbaugh,g , S. Jost,, Pattern Recognitiongand Imageg Analysis,y , PrenticeHall, 1996 R.Jain, R.Kasturi, B.G.Schunck, Machine Vision, McGraw-Hill, 1995 E.R.Davies,ER D i MachineM hi VisionVi i – Theory,ThAlAlgorithms,ithPPracticalities,ti liti 3nd3 d edition,ditiAcademic Press, 2002. Powerpoint slides of the course (on the net)PRIA01: Introduction, Klaus D. Toennies21
Is Pattern Recognition Equivalent toIs Pattern Recognition Equivalent to Imaggye Analysis? Task: Detecting windows and doors in image Doors are rotated with respect to camera Windows are partially occluded. Windows are partially occluded Scene is 3-d but features are from 2-d picture. Feature-based pattern recognition from 3-d scenes:
