Transcription
InfillingOpenCV 3.0Uses in Robotics and ARGary BradskiVP Perception and Core Software,Magic LeapDirector: OpenCV Foundation1
OpenCV Thanks!For Key Support Intel– for getting it started and helping fundthe challenge Google– for growing support in Google Summerof Code over the last 5 years Nvidia– Supporting Cuda version with lots ofother help2
Outline: OPENCV 3.0 Intro– Learning OpenCV Version 2.0 coming by Aug– Announcing 50K Vision Challenge OpenCV BackgroundOpenCV 3.0 High LevelOpenCV 3.0 ModulesBrand New in OpenCVOpenCV Examples– Robotics– Augmented Reality3
Learning OpenCV V2.0 Out in Summer 2014!In C :V2.04
OpenCV 50K VisionChallenge5
OpenCV Background6
What is OpenCV Open Source Computer VisionLibrary Routines focused on real time imageprocessing and 2D 3D computervision.– On Linux, Windows, Mac, Androidand iOS– C , C, Java, Matlab and Pythoninterfaces Free for commercial or research usein whole or in part.7
OpenCV License Based on BSD license Free for commercial and researchuse Does not force your code to be open You need not contribute back–But you are very welcome to contributeback!8
RenewedIntel SupportOpenCVFoundationOpenCV HistoryNvidia SupportIntel SupportWillow SupportAlpha Release at CVPR 2000. Windows only.Release2.3.PythonGPU SupportRelease 3.0 RefactoreBeta 1. Linux supportRelease2.0.C OpenCV StartedRelease 1.1 Release 2.4BetaBeta2 Beta3 Beta45 Release 003200220012000209919Main Current Sponsors:Google Summer of Code9
Environments, Platforms Languages:– C , C#, Python, C, Java Platforms:10
OpenCV and HardwareAcceleration OpenCV was a central basis forOpenVX– a hardware abstraction layer– for embedded vision acceleration OpenVX Supporters:11
12
Where is OpenCV Used? Academic and Industry ResearchSecurity systemsGoogle Maps, StreetviewImage/video search and retrievalStructure from motion in moviesMachine vision factory production inspection systemsAutomatic Driver Assistance SystemsSafety monitoring (Dam sites, mines, swimming pools)Robotics – personal, industrial, hobbyCoin production in China2M downloads
PopularityOver 9M downloads!Ramping to 160K downloads/month
OpenCV Corporation Founded this July, 2012Documentation: http://opencv.org (user site)http://docs.opencv.org http://code.opencv.org (developer site) Contribute (via Credit, debit or paypal):– http://tinyurl.com/7eujyo2For corporatesupportAnd/or partnership, contactGarybradski@gmail.comI am looking forentrepreneurial people tostaff up OpenCV: VisionBusiness DevSoftwareHardware15
What’s In OpenCV High level16
Robot supporOpenCV Overview: 2500 algorithmseloper http://code.opencv.org; User: http://opencv.orgImage PyramidsGeneral Image Processing reo, 3DSegmentationFeaturesUtilities andData g Detection, RecognitionMatrix Math17
OpenCV Algorithm ModulesOverviewHighGUI:I/O, InterfaceImageProcessingTransformsFittingOptical pth, PoseObjectrecognitionMachinelearningComputational PhotographyNormals,Planes, 3DFeaturesCORE:Data structures, Matrix math, Exceptionsetc18
Machine Learning Library (MLL)CLASSIFICATION / REGRESSIONFast Approximate NN (FLANN)Extremely Random TreesCARTNaïve BayesMLP (Back propagation)Statistical Boosting, 4 flavorsRandom ForestsSVMFace Detector(Histogram bis ABBCBBCTUNING/VALIDATIONCross validationBootstrappingVariable importanceSampling methodshttp://opencv.org1919
20
21
OpenCV Architecture andDevelopment22
23
24
25
What’s In OpenCV 3.0 Modules26
CORE:OpenCV Modules: CoreData structures,Matrix math,Exceptions etcCore27
OpenCV Modules: HighGUIHighGUI:I/O, Interface28
OpenCV Modules: Image ProcessingImageProcessing29
OpenCV Modules: TransformsTransformsHoughLog-Polar30
OpenCV Modules: FittingFitting2D Rigid ObjectsEllipseDelaunay3DConvex Hull31
OpenCV Modules: Optic Flow, Trackhttp://www.youtube.com/watch?v bWyBGmzfP-g// opencv/samples/c/lkdemo.cOpticalFlowint main( ){Tracking CvCapture* capture ?cvCaptureFromCAM(camera id) :cvCaptureFromFile(path);lkdemo.c, 190 linesif( !capture ) return -1;(needs camera to run)for(;;) {I ( x dx, y dy, t dt ) I ( x, y , t );IplImage* frame cvQueryFrame(capture); I / t I / x ( dx / dt ) I / y ( dy / dt );if(!frame) break;// copy and process imageG X b,cvCalcOpticalFlowPyrLK( ) I x2 , I x I y I cvShowImage( “LkDemo”, result ); , b It x X ( x, y ), G I x I y , I y2 c cvWaitKey(30); // run at 20-30fps speed Iy if(c 0) {// process e.com/watch?v 1osj7kRgswk
OpenCV Modules: SegmentationGrab youtube.com/watch?v OxmDonZjahttp://www.youtube.com/watch?v Ktrjh5-KLKoWatershed33
CalibrationOpenCV Modules: CalibrationHomography3D view of checkerboardhttp://www.youtube.com/watch?v DrXIQfQHFv0Un-distorted image34http://www.youtube.com/watch?v PuWQ
OpenCV Modules: Features, VSLAMFeaturesVSLAMRead two input images:Mat img1 imread(argv[1], CV LOAD IMAGE GRAYSCALE);Detect keypoints in both images:// detecting keypointsFastFeatureDetector detector(15);vector KeyPoint keypoints1;detector.detect(img1, keypoints1);Compute descriptors for each of the keypoints:// computing descriptorsSurfDescriptorExtractor extractor;Mat descriptors1;extractor.compute(img1, keypoints1, descriptors1);Now, find the closest matches between descriptors from the first image to the second:// matching descriptorsBruteForceMatcher L2 float matcher; Change one or both of thesevector DMatch matches;to switch detector and/ormatcher.match(descriptors1, descriptors2, matches);linesdescriptor types3535
OpenCV Modules: Depth, PoseDepth, PoseNormals,Planes, 3DFeaturesLeft – right feature alignment:Some examples of3D stereo depth maps:36
OpenCV Modules: Obj Rec/MLObjectrecognitionMachine w.youtube.com/watch?v RF0VpR4xog37
OpenCV Modules: Comp PhotogComputationalPhotographyImage Stitching(Occipital Corp.)Tilt-shiftTexturalInpainting38
Brand New in OpenCV3.0User Contrib Module Thanks to Google Summer of Code!!– Supporting 15 interns!Accepted pull requests:xtended Python interface 8. New line segment detector9. Haze removal, depth estimationD object recognition and poseKAZE features10.GPU accelerated dense optical floar detection11.DTAM & pose estimationomputational photography 12.PNP pose detectionustom calibration and planar13.VisualARsaliency filtersense optical flow14.Text detection and reading in wild15.TLD tracker39
OpenCV Examples Industrial Perception Magic Leap40
Industrial Perception Sensor driven, Real time planning Applied to distribution41
Magic Leap Augmented Reality done right Lots of computer vision (We’re hiring) Gesture recognition demo42
Gesture was going to be livedemo43
Questions?44Photo: Gary Bradski
Language Modules GPU/CudaAndroidiOSPythonJava45
OpenCV GPU Module: Image processingbuilding ilteringmatchindetectorenginegs High-level algorithms:Stereo matchingFace detectionFeaturematching46
OpenCV GPU ModuleExampleMat frame;VideoCapture capture(camera);cv::HOGDescriptor hog;Mat frame;VideoCapture capture(camera);cv::gpu::HOGDescriptor ure frame;capture frame;GpuMat gpu frame;gpu frame.upload(frame);vector Rect found;hog.detectMultiScale(frame, found,1.4, Size(8, 8), Size(0, 0),1.05, 8);vector Rect found;hog.detectMultiScale(gpu frame,found,1.4, Size(8, 8), Size(0, 0),1.05, 8); Designed verysimilar!47
OpenCV GPU ModulePerformanceTesla C2050 (Fermi) vs. Core i5760 2.8GHz (4 cores, TBB, SSE)– Average speedup for primitives:33 For “good” data (large images arebetter) Without copying to GPUWhat can you get from yourcomputer?– opencv\samples\gpu\perfomance48
OpenCV Android ModuleOpenCV 2.4 for Android:Native Android Camera SupportMultithreadingJava API (soon)Tegra HW Optimizations (soon)Wiki with the latest mlSupport/discussion encv49
OpenCV iOS Module Full support50
OpenCV Python Module Full Python interface Example: Depth image from Kinect:51
OpenCV Java Module52
Book and Foundation53
Learning OpenCV V2.0 Out in Summer 2014!In C :V2.054
OpenCV Foundation Support55
Release 2.2. Android SupportRelease 3.0 Refactored Release 2.3. GPU Support. Modules Willow Support OpenCV Foundation Intel Support Google Summer of Code Nvidia Support Renewed Intel Support Main Current Sponsors: 9
