WIXLIB

PCL Tutorial:The Point Cloud Library By ExampleJeff DelmericoVision and Perceptual Machines Lab106 Davis HallUB North Campusjad12@buffalo.eduFebruary 11, 2013Jeff DelmericoFebruary 11, 20131/38

Point CloudsDefinitionA point cloud is a data structure used to represent a collection ofmulti-dimensional points and is commonly used to representthree-dimensional data.In a 3D point cloud, the points usually represent the X, Y, and Zgeometric coordinates of an underlying sampled surface. Whencolor information is present, the point cloud becomes 4D.Jeff DelmericoFebruary 11, 2013Introduction2/38

Where do point clouds come from?IRGB-D camerasIStereo camerasI3D laser scannersITime-of-flight camerasISythetically from software(e.g. Blender)Jeff DelmericoFebruary 11, 2013Introduction3/38

Point Cloud LibraryIIIIPCL is a large scale, open project for 2D/3D image and pointcloud processing (in C , w/ new python bindings).The PCL framework contains numerous state-of-the artalgorithms including filtering, feature estimation, surfacereconstruction, registration, model fitting and segmentation.PCL is cross-platform, and has been successfully compiled anddeployed on Linux, MacOS, Windows, and Android/iOS.Website: pointclouds.orgJeff DelmericoFebruary 11, 2013Introduction4/38

Getting PCLIFirst, download PCL for your system from:http://pointclouds.org/downloads/IIf you want to try the python bindings (currently for only asubset of the full PCL functionality), go here:http://strawlab.github.com/python-pcl/IPCL provides the 3D processing pipeline for ROS, so you canalso get the perception pcl stack and still use PCL standalone.IPCL depends on Boost, Eigen, FLANN, and VTK.Jeff DelmericoFebruary 11, 2013Using PCL5/38

Basic StructuresThe basic data type in PCL is a PointCloud. A PointCloud is atemplated C class which contains the following data fields:I width (int) - secifies the width of the point cloud dataset inthe number of points.IIIheight (int) - Specifies the height of the point cloud datasetin the number of points.IIIthe total number of points in the cloud (equal with thenumber of elements in points) for unorganized datasetsthe width (total number of points in a row) of an organizedpoint cloud datasetset to 1 for unorganized point cloudsthe height (total number of rows) of an organized point clouddatasetpoints (std::vectorhPointTi) - Contains the data arraywhere all the points of type PointT are stored.Jeff DelmericoFebruary 11, 2013Using PCL6/38

Basic StructuresIis dense (bool) - Specifies if all the data in points is finite(true), or whether the XYZ values of certain points mightcontain Inf/NaN values (false).Isensor origin (Eigen::Vector4f) - Specifies the sensoracquisition pose (origin/translation). This member is usuallyoptional, and not used by the majority of the algorithms inPCL.Isensor orientation (Eigen::Quaternionf) - Specifies thesensor acquisition pose (orientation). This member is usuallyoptional, and not used by the majority of the algorithms inPCL.Jeff DelmericoFebruary 11, 2013Using PCL7/38

Point TypesIPointXYZ - float x, y, zIPointXYZI - float x, y, z, intensityIPointXYZRGB - float x, y, z, rgbIPointXYZRGBA - float x, y, z, uint32 t rgbaINormal - float normal[3], curvatureIPointNormal - float x, y, z, normal[3], curvatureIHistogram - float histogram[N]IAnd many, many, more. Plus you can define new types to suityour needs.Jeff DelmericoFebruary 11, 2013Using PCL8/38

Building PCL ProjectsPCL relies on CMake as a build tool. CMake just requires thatyou place a file called CMakeLists.txt somewhere on your projectpath.CMakeLists.txtcmake minimum required(VERSION 2.6 FATAL ERROR)project(MY GRAND PROJECT)find package(PCL 1.3 REQUIRED COMPONENTS common io)include directories( PCL INCLUDE DIRS)link directories( PCL LIBRARY DIRS)add definitions( PCL DEFINITIONS)add executable(pcd write test pcd write.cpp)target link libraries(pcd write test PCL COMMON LIBRARIES PCL IO LIBRARIES)Jeff DelmericoFebruary 11, 2013Using PCL9/38

Building PCL ProjectsGenerating the Makefile & Building the Project cd /PATH/TO/MY/GRAND/PROJECTmkdir buildcd buildcmake .makeJeff DelmericoFebruary 11, 2013Using PCL10/38

PCD File FormatA simple file format for storing multi-dimensional point data. Itconsists of a text header (with the fields below), followed by thedata in ASCII (w/ points on separate lines) or binary (a memorycopy of the points vector of the PC).I VERSION - the PCD file version (usually .7)I FIELDS - the name of each dimension/field that a point can have (e.g. FIELDSxyz)I SIZE - the size of each dimension in bytes (e.g. a float is 4)I TYPE - the type of each dimension as a char (I signed, U unsigned, F float)I COUNT - the number of elements in each dimension (e.g. x, y, or z would onlyhave 1, but a histogram would have N)I WIDTH - the width of the point cloudI HEIGHT - the height of the point cloudI VIEWPOINT - an acquisition viewpoint for the points: translation (tx ty tz) quaternion (qw qx qy qz)I POINTS - the total number of points in the cloudI DATA - the data type that the point cloud data is stored in (ascii or binary)Jeff DelmericoFebruary 11, 2013I/O11/38

PCD Example# .PCD v.7 - Point Cloud Data file formatVERSION .7FIELDS x y z rgbSIZE 4 4 4 4TYPE F F F FCOUNT 1 1 1 1WIDTH 213HEIGHT 1VIEWPOINT 0 0 0 1 0 0 0POINTS 213DATA ascii0.93773 0.33763 0 4.2108e 060.90805 0.35641 0 4.2108e 060.81915 0.32 0 4.2108e 060.97192 0.278 0 4.2108e 060.944 0.29474 0 4.2108e 060.98111 0.24247 0 4.2108e 060.93655 0.26143 0 4.2108e 060.91631 0.27442 0 4.2108e 060.81921 0.29315 0 4.2108e 060.90701 0.24109 0 4.2108e 060.83239 0.23398 0 4.2108e 060.99185 0.2116 0 4.2108e 060.89264 0.21174 0 4.2108e 06.Jeff DelmericoFebruary 11, 2013I/O12/38

Writing PCD Fileswrite pcd.cpp#i n c l u d e p c l / i o / p c d i o . h #i n c l u d e p c l / p o i n t t y p e s . h intmain ( i n t a r g c , c h a r a r g v ){p c l : : P o i n t C l o u d p c l : : PointXYZ c l o u d ;// F i l l i n t h e c l o u d d a t acloud . width 50;cloud . height 1;cloud . is dense f a l s e ;cloud . p o i n t s . r e s i z e ( cloud . width cloud . height ) ;f o r ( s i z e t i 0 ; i c l o u d . p o i n t s . s i z e ( ) ; i ){c l o u d . p o i n t s [ i ] . x 1024 r a n d ( ) / (RAND MAX 1 . 0 f ) ;c l o u d . p o i n t s [ i ] . y 1024 r a n d ( ) / (RAND MAX 1 . 0 f ) ;c l o u d . p o i n t s [ i ] . z 1024 r a n d ( ) / (RAND MAX 1 . 0 f ) ;}p c l : : i o : : s a v e P C D F i l e A S C I I ( ” t e s t p c d . pcd ” , c l o u d ) ;return (0);}Jeff DelmericoFebruary 11, 2013I/O13/38

Reading PCD Filesread pcd.cpp#i n c l u d e p c l / i o / p c d i o . h #i n c l u d e p c l / p o i n t t y p e s . h intmain ( i n t a r g c , c h a r a r g v ){p c l : : P o i n t C l o u d p c l : : PointXYZ :: P t r c l o u d ( new p c l : : P o i n t C l o u d p c l : : PointXYZ );// Load t h e f i l ei f ( p c l : : i o : : l o a d P C D F i l e p c l : : PointXYZ ( ” t e s t p c d . pcd ” , c l o u d ) 1){PCL ERROR ( ” C o u l d n ’ t r e a d f i l e t e s t p c d . pcd \n” ) ;r e t u r n ( 1);}// Do some p r o c e s s i n g on t h e c l o u d h e r ereturn (0);}Jeff DelmericoFebruary 11, 2013I/O14/38

Getting Point Clouds from OpenNIopenni grabber.cpp#i n c l u d e p c l / i o / o p e n n i g r a b b e r . h #i n c l u d e p c l / v i s u a l i z a t i o n / c l o u d v i e w e r . h c l a s s SimpleOpenNIViewer{public :S i m p l e O p e n N I V i e w e r ( ) : v i e w e r ( ”PCL OpenNI V i e w e r ” ) {}v o i d c l o u d c b ( c o n s t p c l : : P o i n t C l o u d p c l : : PointXYZRGBA :: C o n s t P t r &c l o u d ){i f ( ! v i e w e r . w a sS t o pp e d ( ) )v i e w e r . showCloud ( c l o u d ) ;}p c l : : v i s u a l i z a t i o n : : CloudViewer viewer ;Jeff DelmericoFebruary 11, 2013I/O15/38

Getting Point Clouds from OpenNIopenni grabber.cppvoid run ( ){p c l : : G r a b b e r i n t e r f a c e new p c l : : OpenNIGrabber ( ) ;b o o s t : : f u n c t i o n v o i d ( c o n s t p c l : : P o i n t C l o u d p c l : : PointXYZRGBA :: C o n s t P t r&) f b o o s t : : b i n d (& S i m p l e O p e n N I V i e w e r : : c l o u d c b , t h i s , 1 ) ;i n t e r f a c e r e g i s t e r C a l l b a c k ( f ) ;i n t e r f a c e s t a r t ( ) ;w h i l e ( ! v i e w e r . w a s St o p pe d ( ) ){boost : : t h i s t h r e a d : : s l e e p ( boost : : posix time : : seconds ( 1 ) ) ;}i n t e r f a c e s t o p ( ) ;}};i n t main ( ){SimpleOpenNIViewer v ;v . run ( ) ;return 0;}Jeff DelmericoFebruary 11, 2013I/O16/38

Normal Estimationcompute normals.cppvoiddownsample ( p c l : : P o i n t C l o u d p c l : : PointXYZRGB :: P t r &p o i n t s , f l o a t l e a f s i z e ,p c l : : P o i n t C l o u d p c l : : PointXYZRGB :: P t r &d o w n s a m p l e d o u t ){p c l : : V o x e l G r i d p c l : : PointXYZRGB v o x g r i d ;vox grid . setLeafSize ( leaf size , leaf size , l e a f s i z e );vox grid . setInputCloud ( points );v o x g r i d . f i l t e r ( d o w n s a m p l e d o u t ) ;}v o i d c o m p u t e s u r f a c e n o r m a l s ( p c l : : P o i n t C l o u d p c l : : PointXYZRGB :: P t r &p o i n t s ,f l o a t n o r m a l r a d i u s , p c l : : P o i n t C l o u d p c l : : Normal :: P t r &n o r m a l s o u t ){p c l : : N o r m a l E s t i m a t i o n p c l : : PointXYZRGB , p c l : : Normal n o r m e s t ;// Use a FLANN b a s e d KdTree t o p e r f o r m n e i g h b o r h o o d s e a r c h e sn o r m e s t . s e t S e a r c h M e t h o d ( p c l : : s e a r c h : : KdTree p c l : : PointXYZRGB :: P t r( new p c l : : s e a r c h : : KdTree p c l : : PointXYZRGB ));// S p e c i f y t h e l o c a l n e i g h b o r h o o d s i z e f o r c o m p u t i n g t h e s u r f a c e n o r m a l snorm est . setRadiusSearch ( normal radius ) ;// S e t t h e i n p u t p o i n t snorm est . setInputCloud ( points ) ;// E s t i m a t e t h e s u r f a c e n o r m a l s and s t o r e t h e r e s u l t i n ” n o r m a l s o u t ”n o r m e s t . compute ( n o r m a l s o u t ) ;}Jeff DelmericoFebruary 11, 20133D Features17/38

compute normals.cppv o i d v i s u a l i z e n o r m a l s ( c o n s t p c l : : P o i n t C l o u d p c l : : PointXYZRGB :: P t r p o i n t s ,c o n s t p c l : : P o i n t C l o u d p c l : : PointXYZRGB :: P t r n o r m a l p o i n t s ,c o n s t p c l : : P o i n t C l o u d p c l : : Normal :: P t r n o r m a l s ){pcl : : v i s u a l i z a t i o n : : PCLVisualizer viz ;v i z . addPointCloud ( points , ” p o i n t s ” ) ;v i z . addPointCloud ( normal points , ” normal points ” ) ;v i z . a d d P o i n t C l o u d N o r m a l s p c l : : PointXYZRGB , p c l : : Normal ( n o r m a l p o i n t s , n o r m a l s , 1 , 0 .viz . spin ();}i n t main ( i n t a r g c , c h a r a r g v ){// Load d a t a from pcd . . .p c l : : P o i n t C l o u d p c l : : PointXYZRGB :: P t r d s ( new p c l : : P o i n t C l o u d p c l : : PointXYZRGB );p c l : : P o i n t C l o u d p c l : : Normal :: P t r n o r m a l s ( new p c l : : P o i n t C l o u d p c l : : Normal );// Downsample t h e c l o u dconst f l o a t v o x e l g r i d l e a f s i z e 0 . 0 1 ;downsample ( c l o u d , v o x e l g r i d l e a f s i z e , d s ) ;// Compute s u r f a c e n o r m a l sconst f l o a t normal radius 0 . 0 3 ;c o m p u t e s u r f a c e n o r m a l s ( ds n o r m a l r a d i u s , normals ) ;// V i s u a l i z e t h e n o r m a l sv i s u a l i z e n o r m a l s ( c l o u d , ds , n o r m a l s ) ;return (0);}Jeff DelmericoFebruary 11, 20133D Features18/38

Computing 3D FeaturessetSearchSurface FalsesetSearchSurface TrueJeff DelmericosetInputCloud Falsecompute on all points,using all pointscompute on all points,using a subsetFebruary 11, 2013setInputCloud Truecompute on a subset,using all pointscompute on a subset,using a subset3D Features19/38