模式識別教學大綱

Introduction

Patternrecognitiontechniquesareusedtoautomaticallyclassifyphysicalobjects(handwrittencharacters,tissuesamples)orabstractmultidimensionalpatterns(n

pointsin

d

dimensions)intoknownorpossiblyunknowncategories.Anumberofcommercialpatternrecognitionsystemsareavailableforcharacterrecognition,handwritingrecognition,documentclassification,fingerprintclassification,speechandspeakerrecognition,whitebloodcell(leukocyte)classification,militarytargetrecognition,etc.Mostmachinevisionsystemsemploypatternrecognitiontechniquestoidentifyobjectsforsorting,inspection,andassembly.Thedesignofapatternrecognitionsystemrequiresthefollowingmodules:(i)sensing,(ii)featureextractionandselection,(iii)decisionmakingand(iv)performanceevaluation.Theavailabilityoflowcostandhighresolutionsensors(e.g.,digitalcameras,microphonesandscanners)anddatasharingovertheInternethaveresultedinhugerepositoriesofdigitizeddocuments(text,speech,imageandvideo).Needforefficientarchivingandretrievalofthisdatahasfosteredthedevelopmentofpatternrecognitionalgorithmsinnewapplicationdomains(e.g.,text,imageandvideoretrieval,bioinformatics,andfacerecognition).

Designofapatternrecognitionsystemtypicallyfollowsoneofthefollowingapproaches:(i)templatematching,(ii)statisticalmethods,(iii)syntacticmethodsand(iv)neuralnetworks.Thiscoursewillintroducethefundamentalsofstatisticalpatternrecognitionwithexamplesfromseveralapplicationareas.Techniquesforanalyzingmultidimensionaldataofvarioustypesandscalesalongwithalgorithmsforprojection,dimensionalityreduction,clusteringandclassificationofdatawillbeexplained.Thecoursewillpresentvariousapproachestoexploratorydataanalysisandclassifierdesignsostudentscanmakejudiciouschoiceswhenconfrontedwithrealpatternrecognitionproblems.Itisimportanttoemphasizethatthedesignofacompletepatternrecognitionsystemforaspecificapplicationdomain(e.g.,remotesensing)requiresdomainknowledge,whichisbeyondthescopeofthiscourse.StudentswilluseavailableMATLABsoftwarelibraryandimplementsomealgorithmsusingtheirchoiceofaprogramminglanguage.

Prerequisites

CSE232,MTH314,andSTT441,orequivalentcourses.

TextBook

Duda,HartandStork,PatternClassification,SecondEdition,Wiley,2001.

Youmayfindthe

erratalist

useful.

AnumberofbooksonpatternrecognitionhavebeenputontheAssignedReadingintheEngineeringLibrary.Inaddition,anumberofjournals,includingPatternRecognition,PatternRecognitionLetters,IEEETrans.PatternAnalysis&MachineIntelligence(PAMI),IEEETrans.Geoscience&RemoteSensing,IEEETrans.ImageProcessing,andIEEETrans.Speech,Audio,andLanguageProcessingroutinelypublishpapersonpatternrecognitiontheoryandapplications.

AssignedReading

FollowingbooksareonholdintheEngineeringlibraryforassignedreadingforCSE802.

TheodoridisandKoutroumbas

PatternRecognition

ChristopherBishop

PatternRecognitionandMachineLearning

Fukunaga

IntroductiontoStatisticalPatternRecognition

DevijverandKittler

PatternRecognition:AStatisticalApproach

TouandGonzalez

PatternRecognitionPrinciples

YoungandCalvert

Classification,EstimationandPatternRecognition

Pavlidis

StructuralPatternRecognition

GonzalezandWintz

SyntacticPatternRecognition

Oja

SubspaceMethodsofPatternRecognition

Watanabe

PatternRecognition:HumanandMechanical

JainandDubes

AlgorithmsforClusteringData

(Downloadthebook)

Schalkoff

PatternRecognition:Statistic,StructuralandNeuralApproaches

CourseSchedule

Jan8

IntroductiontoPatternRecognition(Ch1)

StatisticalPatternRecognition:AReview

Lectureslides:

PatternRecognition

HW1

assigned

HW1Solutions

Jan10,15,17

StatisticalDecisionTheory(Ch2)

Jan15:

HW2

assigned;

HW1due

Lectureslides:

Chapter2

NotesonBayesClassification

AnIntroductiontoMatlab

.

Jan22

StatisticalDecisionTheory(Ch2)

Lectureslides:

Neyman-PearsonRule

LinearDiscriminantFunctions

Jan24,29

ParameterEstimation(Ch3)

BayesEstimatorformultivariateGaussiandensitywithunknowncovariancematrices

BayesEstimatorunderquadraticloss

Jan24:

HW3

assigned;

HW2due

Lectureslides:

Chapter3

Jan31

ParameterEstimation(Ch3)

CurseofDimensionality(Ch3)

CoinTossingExample

AProblemofDimensionality:ASimpleExample

Lectureslides:

CurseofDimensionality

Feb5,7

ComponentanalysisandDiscriminants(Ch3)

PrincipleComponentAnalysis(PCA)

Principalcomponentanalysisforfacerecognition.

Lectureslides:

ComponentAnalysis&Discriminants

Feb5:

HW4assigned;

HW3due

Feb12,14,19

NonparametricTechniques(Ch4)

Lectureslides:

NonparametricTechniques

ABranchandBoundAlgorithmforComputingk-NearestNeighbors

Feb19:

HW5assigned;

HW4due

Feb21

DecisionTrees(Ch8)

lectureslides

HierarchicalClassifierDesignUsingMutualInformation

-SethiandSarvarayudu

Feb26

MidTermExam

Feb28

ProjectDiscussion

Mar5,7

SPRINGBREAK

Mar12

ProjectProposalDue(2pages)

LinearDiscriminantfunctions(Ch5)

Lectureslides:

Lineardiscriminantfunctions

Mar14,19

LinearDiscriminantfunctions(Ch5)

SupportVectorMachines

Mar14:

HW6assigned;

HW5due

Mar21,26

NeuralNetworks(Ch6)

Lectureslides

Lectureslides-2

audiofile-1forLectureslides-2

audiofile-2forLectureslides-2

audiofile-3forLectureslides-2

Anoteoncomparingclassifiers

ATutorialonArtificialNeuralNetworks

Performanceevaluationofpatternclassifiersforhandwrittencharacterrecognition

Mar28,Apr2

ErrorRateEstimation,Bagging,Boosting(Ch9)

Mar28:

HW7assigned,

HW6due

Apr4

ClassifierCombination(Ch9)

Lectureslidesonclassifiercombination

CombinationofMultipleClassifiersUsingLocalAccuracyEstimates

byWoods,KegelmeyerandBowyer

Handwritingdigitsrecognitionbycombiningclassifiers

byvanBreukelen,Duin,TaxanddenHartog

Apr9

FeatureSelection

Lectureslidesonfeatureselection

BranchandBoundAlgorithmforFeatureSubsetSelection

byNarendraandFukunaga

FeatureSelection:Evaluation,Application,andSmallSamplePerformance

byJainandZongker

Apr11,16,18

UnsupervisedLearning,Clustering,andMultidimensionalScaling(Ch10)

April11:

HW7due

LectureSlides:Introductiontoclustering

LectureSlides:EMAlgorithm

LectureSlides:Largescaleclustering

TalkonLargeScaleClustering

DataClustering:50YearsBeyondK-means

(Download

PresentationSlides

here)

GraphTheoreticalMethodsforDetectingandDescribingGestaltClusters

byC.Zahn

ANonlinearMappingforDataStructureAnalysis

byJ.Sammon

RepresentationandRecognitionofHandwrittenDigitsUsingDeformableTemplates

byJainandZongker

Apr23

Semi-supervisedlearning

Semi-supervisedlearning

byXiaojinZhu

BoostCluster

byLiu,JinandJain

ConstrainedK-meansClusteringwithBackgroundKnowledge

byWagstaffetal.

Semi-supervisedclusteringbyseeding

byBasuetal.

Apr25

FinalProjectPresentation

FinalProjectReportDue

May1

FINALEXAM,7:45a.m.-9:45a.m.,

3400EB

Grading

Coursegradewillbeassignedbasedonscoresonsixhomeworkassignments,twoexamsandoneproject.Weightsforthesethreecomponentsareasfollows:HW(25%),MIDTERMEXAM(25%),FINALEXAM(25%),PROJECT(25%).Thecumulativescorewillbemappedtothelettergradeasfollows:90%orhigher:4.0;85%to90%:3.5;80%to85%:3.0andsoon.

Boththeexamswillbeclosedbook.MakeupexamswillbegivenONLYifproperlyjustified.

Homeworksolutionsmustbeturnedintheclassonthedatetheyaredue.Latehomeworksolutionswillnotbeaccepted.Homeworksolutionsshouldbeeithertypedorneatlyprinted.

PleaserefertoMSU'spolicyonthe

IntegrityofScholarship.

Allhomeworksolutionsmustreflectyourownwork.Failuretodosowillresultinagradeof0inthecourse.

CourseProject

Thepurposeoftheprojectistoenablethestudentstogetsomehands-onexperienceinthedesign,implementationandevaluationofpatternrecognitionalgorithms.Tofacilitatethecompletionoftheprojectinasemester,itisadvisedthatstudentsworkinteamsoftwo.Youareexpectedtoevaluatedifferentpreprocessing,featureextraction,andclassification(includingbaggingandboosting)approachestoachieveashighaccuracyaspossibleontheselectedclassificationtask.Thetaskfortheprojectisdescribed

here

.

Theprojectreportshouldclearlyexplaintheobjectiveofthestudy,somebackgroundwor