Transcription
Lecture Slides forINTRODUCTION TOMachine LearningETHEM ALPAYDIN The MIT Press, r/ ethem/i2ml
CHAPTER 1:Introduction
Why “Learn” ? Machine learning is programming computers tooptimize a performance criterion using exampledata or past experience.There is no need to “learn” to calculate payrollLearning is used when:Human expertise does not exist (navigating on Mars),Humans are unable to explain their expertise (speechrecognition)Solution changes in time (routing on a computer network)Solution needs to be adapted to particular cases (userbiometrics)3Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
What We Talk About When WeTalk About“Learning” Learning general models from a data of particularexamplesData is cheap and abundant (data warehouses, datamarts); knowledge is expensive and scarce.Example in retail: Customer transactions toconsumer behavior:People who bought “Da Vinci Code” also bought “The FivePeople You Meet in Heaven” (www.amazon.com) Build a model that is a good and usefulapproximation to the data.4Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Data Mining Retail: Market basket analysis, Customerrelationship management (CRM)Finance: Credit scoring, fraud detectionManufacturing: Optimization, troubleshootingMedicine: Medical diagnosisTelecommunications: Quality of serviceoptimizationBioinformatics: Motifs, alignmentWeb mining: Search engines.5Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
What is Machine Learning? Optimize a performance criterion using exampledata or past experience.Role of Statistics: Inference from a sampleRole of Computer science: Efficient algorithms toSolve the optimization problemRepresenting and evaluating the model forinference6Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Applications AssociationSupervised LearningClassificationRegression Unsupervised LearningReinforcement Learning7Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Learning Associations Basket analysis:P (Y X ) probability that somebody who buys X alsobuys Y where X and Y are products/services.Example: P ( chips beer ) 0.78Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Classification Example: CreditscoringDifferentiatingbetween low-riskand high-riskcustomers fromtheir income andsavingsDiscriminant: IF income θ1 AND savings θ2THEN low-risk ELSE high-risk9Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Classification: Applications Aka Pattern recognitionFace recognition: Pose, lighting, occlusion (glasses,beard), make-up, hair styleCharacter recognition: Different handwriting styles.Speech recognition: Temporal dependency.Use of a dictionary or the syntax of the language.Sensor fusion: Combine multiple modalities; eg, visual (lipimage) and acoustic for speech Medical diagnosis: From symptoms to illnesses.10Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Face RecognitionTraining examples of a personTest imagesAT&T Laboratories, Cambridge 11Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Regression Example: Price of aused carx : car attributesy : pricey g (x θ )g ( ) model,θ parametersy wx w012Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Regression Applications Navigating a car: Angle of the steering wheel (CMUNavLab)Kinematics of a robot arm(x,y)α2α1 g1(x,y)α2 g2(x,y)α1 Response surface design13Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Supervised Learning: Uses Prediction of future cases: Use the rule to predictthe output for future inputsKnowledge extraction: The rule is easy tounderstandCompression: The rule is simpler than the data itexplainsOutlier detection: Exceptions that are not coveredby the rule, e.g., fraud14Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Unsupervised Learning Learning “what normally happens”No outputClustering: Grouping similar instancesExample applicationsCustomer segmentation in CRMImage compression: Color quantizationBioinformatics: Learning motifs15Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Reinforcement Learning Learning a policy: A sequence of outputsNo supervised output but delayed rewardCredit assignment problemGame playingRobot in a mazeMultiple agents, partial observability, .16Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Resources: Datasets UCI Repository:http://www.ics.uci.edu/ mlearn/MLRepository.html UCI KDD ation.html Statlib: http://lib.stat.cmu.edu/Delve: http://www.cs.utoronto.ca/ delve/17Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Resources: Journals Journal of Machine Learning Research www.jmlr.orgMachine LearningNeural ComputationNeural NetworksIEEE Transactions on Neural NetworksIEEE Transactions on Pattern Analysis and MachineIntelligenceAnnals of StatisticsJournal of the American Statistical Association.18Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Resources: Conferences International Conference on Machine Learning (ICML)ICML05: http://icml.ais.fraunhofer.de/European Conference on Machine Learning (ECML)ECML05: http://ecmlpkdd05.liacc.up.pt/Neural Information Processing Systems (NIPS)NIPS05: http://nips.cc/Uncertainty in Artificial Intelligence (UAI)UAI05: http://www.cs.toronto.edu/uai2005/Computational Learning Theory (COLT)COLT05: http://learningtheory.org/colt2005/International Joint Conference on Artificial Intelligence (IJCAI)IJCAI05: http://ijcai05.csd.abdn.ac.uk/International Conference on Neural Networks (Europe)ICANN05: http://www.ibspan.waw.pl/ICANN-2005/.19Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0)
Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.0) 6 What is Machine Learning? Optimize a performance criterion using example data or past experience. Role of Statistics: Inference from a sample Role of Computer science: Efficient algorithms to Solve the optimization problem
