Transcription
Android and OpenCV TutorialComputer Vision Lab Tutorial26 September 2013Lorenz Meier, Amaël Delaunoy, Kalin KolevInstitute of Visual Computing
Tutorial Content Strengths / Weaknesses ofAndroid Java vs Native Overview of available SDKs Complete example Assignment Project plan coaching Question sessionInstitute of Visual Computing2!
Android Software Development KitsSoftware Development Kit (SDK)Native Development Kit (NDK)Institute of Visual Computing3!
Android Android 4.3 Jelly Bean Targeted compatibilityversion: 4.1 Android SDK:http://developer.android.com/sdk/index.html Android ex.htmlInstitute of Visual Computing4!
Java SDK Easy to setup and get started Typically only mean of accessingthe camera Slow for core vision algorithms Slow for some sensor accessfunctions te of Visual Computing5!
Android NDK (C/C , GNU/Linux API) Still easy enough to get started Offers full OpenCV functionality on the device Important: Have NDKROOT set as environmentvariable in Eclipse tmlexport PATH oid/android-ndk-r9: PATHexport NDKROOT /Users/lomeier/android/android-ndk-r9"chmod u x n/android binary package/android dev intro.html!Institute of Visual Computing6!
Computer Vision Toolkits for AndroidOpenCVQualcomm VuforiaQualcomm FastCVOpenVXInstitute of Visual Computing7!
OpenCV on Android Main referencehttp://opencv.org/platforms/android.html Watch out! API has changed, use onlydocumentation and tutorials from 2012/2013 Start off with the OpenCV Tutorial #3: ntroduction/android binary package/android dev intro.html!Institute of Visual Computing8!
Qualcomm Vuforia SDK Virtual Reality SDK for any Android / iOS //www.youtube.com/watch?v BlF60qYTrPI!!Institute of Visual Computing9!
Qualcomm FastCV SDK Similar API to OpenCV Collection of low-levelcomputer visionprimitives On many functions fasterthan OpenCV w.youtube.com/watch?v zTxpFCpG-GA!!Institute of Visual Computing10!
Preview: OpenVX – the new standard SDK OpenGL equivalent forhardware-acceleratedcomputer vision Co-developed withOpenCV All major industryplayers First draft end 2013Institute of Visual Computing11!
Android SDK Android NDK OpenCVInstitute of Visual Computing12!
Android SDK Installation Download from:http://developer.android.com/sdk/index.html Unzip the directory to alocal directory withoutspaces in the pathInstitute of Visual Computing13!
Android NDK Installation Download dex.html Unzip to same top-levelfolder as the SDKInstitute of Visual Computing14!
OpenCV for Android Installation Download from:http://opencv.org/downloads.html Unzip to same top-levelfolder as Android SDK andNDKInstitute of Visual Computing15!
Set up Eclipse / ADTInstitute of Visual Computing16!
Final directory layout After all SDKs are installed, the directory structureshould look like similar to the screenshot below. Open the Eclipse folder and start the executable.Institute of Visual Computing17!
Start Eclipse / ADTInstitute of Visual Computing18!
Start Android SDK ManagerInstitute of Visual Computing19!
Install the Android Support LibraryInstitute of Visual Computing20!
Install the Android Support LibraryInstitute of Visual Computing21!
Android Project SetupInstitute of Visual Computing22!
Android Project Setup Right-click into the Eclipseproject view Select Import. – Existingprojects into workspace Select OpenCV Folder All tutorials will show upin the project list Right click the Tutorial #2Institute of Visual Computing23!
Import new projectInstitute of Visual Computing24!
Institute of Visual Computing25!
Institute of Visual Computing26!
Institute of Visual Computing27!
Institute of Visual Computing28!
Project Properties Setup – ENV variablesInstitute of Visual Computing29!
Commandline Setup Imported Eclipse projects can contain errors and fail tocompile dependencies Set up your command line environment by adding thisto your /.bashrc or /.bash profile file:export PATH oid/android-ndk-r8b: PATHexport NDKROOT /Users/lomeier/android/android-ndk-r8b"source /.bash profile" Make the ndk-build script executablechmod u x d"Institute of Visual Computing30!
Commandline compilation Change to the folder to the projectcd /samples/tutorial-3-native" Force a ndk clean buildndk-build -B" Build / upload now in EclipseInstitute of Visual Computing31!
Launch ApplicationInstitute of Visual Computing32!
Institute of Visual Computing33!
Put Phone into developer modeSettings - About phoneSettings - Developer optionsTap on build numberEnable USB debuggingFrom: http://howto.cnet.com/8301-11310 -in-android-4.2/!Institute of Visual Computing34!
Reset USBUNPLUG USB.RE-PLUG USB. CONFIRM THE USB CONNECTION.Institute of Visual Computing35!
Select Cell PhoneInstitute of Visual Computing36!
TroubleshootingUnsatisfied link error?Force a native build on the command line.cd /dir/to/OpenCV/samples"cd tutorial-2-mixedprocessing"ndk-build -B"Institute of Visual Computing37!
Run the Sample (jni/jni part.cpp) Standard example code extracts FAST cornersMat& mGr *(Mat*)addrGray;"Mat& mRgb *(Mat*)addrRgba;"vector KeyPoint v;""FastFeatureDetector detector(50);"detector.detect(mGr, v);!for( unsigned int i 0; i v.size(); i )!{"const KeyPoint& kp v[i];!circle(mRgb, Point(kp.pt.x, kp.pt.y), 10, Scalar(255,0,0,255));!}"Institute of Visual Computing38!
Class Assignment (jni/jni part.cpp) Extend sample to extract and match ORB featuresbetween two frames Then color all pixels with ORB features translucent blueMat& mGr *(Mat*)addrGray; //Mat& mRgb *(Mat*)addrRgba; //vector KeyPoint v; // You need""""// and thisThis is the gray scale frame"This is the frame in color"to store the ones from the last frameframe""OrbFeatureDetector detector(50);"detector.detect(mGr, v);!for( unsigned int i 0; i v.size(); i )!{"const KeyPoint& kp v[i];!circle(mRgb, Point(kp.pt.x, kp.pt.y), 10, Scalar(0,0,255,100));!! line( from old point to new match)!}"OrbDescriptorExtractor "BruteForceMatcher cv::HammingLUT "Institute of Visual Computing39!
Class Assignment Hand in until next http://Thursday (October 3, 2013)www.howtogeek.com/121133/how-to-take Screenshot (from device,screenshots-on-androidnot photos of display)devices-since-4.0/ Code from jni part.cpp,inlined into emailInstitute of Visual Computing40!
Camera CalibrationInstitute of Visual Computing41!
Camera calibration Additional tutorial ifrequiredPinhole camera model OpenCV functionsavailable Undistortion and cameramatrix map from imagepixel locations tonormalized cameracoordinates (“rays”)Institute of Visual Computing42!
Camera calibration1.Find and match cornersbool findChessboardCorners(image,patternSize, corners, flags); "double calibrateCamera(objectPoints,imgPoints, imgSize, camMatrix,distCoeffs, rvecs, tvecs, flags,criteria); "Mat initCameraMatrix2D(objPoints,imgPoints, imgSize, aspectRatio); "2. Calibrate camera(undistortion Matrix,distCoeffs, imgSize, alpha,newImgSize, PixROI, cPP); "3. Optimize matrix4. Test result on imagevoid undistort(src, dst, cameraMatrix,distCoeffs, newCameraMatrix); ""Institute of Visual Computing43!
Project Plan and ManagementInstitute of Visual Computing44!
Project Proposal Q&A Papers / Scope Library choice Milestones Block Diagram Gantt ChartInstitute of Visual Computing45!
Diagrams in Report Data flow / block diagram Gantt ChartInstitute of Visual Computing46!
Project Management Use Github issue tracker andmilestones http://github.com Use the Github for Windows /Mac OS app as GUI Simple GIT guide:http://rogerdudler.github.com/git-guide/ GIT SCM book:http://git-scm.com/book/ch1-3.htmlInstitute of Visual Computing47!
Android and OpenCV Tutorial Computer Vision Lab Tutorial 26 September 2013 Lorenz Meier, Amaël Delaunoy, Kalin Kolev . . Android Software Development Kits Software Development Kit (SDK) Native Development Kit (NDK) Inst
