神經(jīng)網(wǎng)絡(luò)與模糊系統(tǒng)-課件

神經(jīng)網(wǎng)絡(luò)與模糊系統(tǒng)學(xué)生：導(dǎo)師：ARCHITECTUREANDEQUILIBRIA

結(jié)構(gòu)和平衡CHAPTER6L函數(shù)與系統(tǒng)的穩(wěn)定性對于一個(gè)系統(tǒng)構(gòu)造一個(gè)Lyapunov方程若，系統(tǒng)穩(wěn)定若，系統(tǒng)漸進(jìn)穩(wěn)定系統(tǒng)穩(wěn)定能構(gòu)造L方程能構(gòu)造系統(tǒng)穩(wěn)定PREFACE李雅普諾夫切比雪夫馬爾科夫圣彼得堡數(shù)學(xué)學(xué)派6.1NeutralNetworkAsStochasticGradientsystem1.synapticconnectiontopologiesFeedforwardFeedback2.howlearningmodifiestheirconnectiontopologiesSupervised訓(xùn)練數(shù)據(jù)label特征模型訓(xùn)練a.訓(xùn)練測試數(shù)據(jù)特征模型labelb.測試Unsupervised2.howlearningmodifiestheirconnectiontopologies訓(xùn)練數(shù)據(jù)特征模型訓(xùn)練a.訓(xùn)練測試數(shù)據(jù)特征模型結(jié)果b.測試K-meansNEURALNETWORKTAXONOMYGRADIENTDESCENTLMSBACKPROPAGATIONREINFORCEMENTLEARNINGRECURRENTBACKPROPAGATIONVECTORQUANTIZATIONSELF-ORGANIZINGMAPSCOMPETITIVELEARNINGCOUNTER-PROPAGATIONBABAM

BROWNIANANNEALINGBOLTZMANNLEARNINGABAMART-2

BAM-COHEN-GROSSBERGMODELHOPFIELDCIRCUITBRAIN-STATE-IN-A-BOXMASKINGFILEDADAPTIVERESONANCEART-1ART-2FeedforwardFeedbackSupervisedUnsupervisedDECODINGEDCODING6.2GlobalEquilibria:convergenceandstabilityThreedynamicalsystemsinneuralnetwork:1）synapticdynamicalsystem2）neuronaldynamicalsystem3）jointneuronal-synapticdynamicalsystemEquilibriumissteadystate(forfixed-pointattractors).Convergenceissynapticequilibrium:Stabilityisneuronalequilibrium:Moregenerallyneuralsignalsreachsteadystateeventhoughtheactivationsstillchange.

Steadystate:GlobalStabilityStochasticGlobalStabilityStability-ConvergencedilemmaNeuronsfluctuatefasterthansynapsesfluctuate.Learningtendstodestroytheneuronalpatternsbeinglearned.Convergenceunderminesstability.6.3Synapticconvergencetocentroids:AVQAlgorithmsCompetitivelearningadaptivelyquantizestheinputpatternspace.Probabilitydensityfunctioncharacterizesthecontinuousdistributionsofpatternsin.CompetitiveAVQStochasticDifferentialEquations:Thedecisionclassespartitionintokclasses:Centroidof:Therandomindicatorfunctions：圖像處理質(zhì)心定位灰度質(zhì)心法灰度質(zhì)心法TheStochasticunsupervisedcompetitivelearninglaw:Equilibrium：AsdiscussedinChapter4:Thelinearstochasticcompetitivelearninglaw:Thelinearsupervisedcompetitivelearninglaw:Thelineardifferentialcompetitivelearninglaw:Inpractice：CompetitiveAVQAlgorithms1.Initializesynapticvectors:2.Forrandomsample,findtheclosestsynapticvector:3.Updatethewinningsynapticvector(s)bytheUCL,SCL,orDCLlearningalgorithm.UnsupervisedCompetitiveLearning(UCL)definesaslowlydecreasingsequenceoflearningcoefficients.Example:SupervisedCompetitiveLearning(SCL)DifferentialCompetitiveLearning(DCL)denotesthetimechangeofthejthneuron’scompetitivesignalinthecompetitionfield:實(shí)際中，只使用該信號差的符號或ThefixedcompetitionmatrixWdefinesasymmetriclateralinhibitionTopologywithin.StochasticEquilibriumandConvergenceCompetitivesynapticvectorconvergetodecision-classcentroids.Thecentroidsmaycorrespondtolocalmaximaofthesampledbutunknownprobabilitydensityfunction.AVQcentroidtheorem:IfacompetitiveAVQsystemconverges,itconvergestothecentroidofthesampleddecisionclass.Proof.Supposethejthneuroninwinsthecompetition.Supposethejthsynapticvectorcodesfordecisionclass.Suppose.Thecompetitivelaw.IngeneraltheAVQcentroidtheoremconcludesthatatequilibrium:Q.E.D6.4AVQConvergenceTheoremCompetitivesynapticvectorsconvergeexponentiallyquicklytopattern-classcentroids.Proof.ConsidertherandomquadraticformL:Note.Thepatternvectorsxdonotchangeintime.Thecompetitivelaw.ChosetheaverageE[L]asLyapunovfunctionforthestochasticcompetitivedynamicalsystem.Assume:sufficientsmoothnesstointerchangethetimederivativeandtheprobabilisticintegral—tobringthetimederivative“inside”theintegral.ThecompetitiveAVQsystemisasymptoticallystable,andingeneralconvergesexponentiallyquicklytoalocallyequilibrium.Suppose.Sincep(x)isanonnegativeweightfunction,theweightedintegralofthelearningdifferencesmustequalzero:Averageequilibriumsynapticvectorarecentroids:.Q.E.D6.5GlobalstabilityoffeedbackneuralnetworksGlobalstabilityisjointlyneuronal-synapticsteadystate.Globalstabilitytheoremsarepowerfulbutlimited.Theirpower:theirdimensionindependence.nonlineargenerality.theirexponentiallyfastconvergencetofixedpoints.Theirlimitation:nottelluswheretheequilibriaoccurinthestatespace.Stability-ConvergenceDilemma1.Asymmetry:NeuronsinandfluctuatefasterthanthesynapsesinM.2.Stability:(patternformation).

3.Learning：4.Undoing:TheABAMtheoremoffersageneralsolutiontostability-convergencedilemma.TheRABAMtheoremextendsthisresulttostochasticneuralprocessinginthepresenceofnoise.

6.6TheABAMTheoremHebbianABAMmodels:CompetitiveABAMmodels:Ifthepositivityassumptionshold,thenthemodelsareasymptoticallystable.Proof.TheproofusestheboundedlyapunovfunctionL:alongtrajectories.ProvesglobalstabilityforthecompetitiveABAMsystem.Thisprovesasymptoticglobalstability.Thesquaredvelocitiesdeceaseexponentiallyquickly.Q.E.DHigher-OrderABAMsAdaptiveResonanceABAMsDifferentialHebbianABAMs關(guān)鍵是找出解決問題的規(guī)律

6.7StructuralStabilityofUnsupervisedLearningIsunsupervisedlearningstructuralstability?StructuralstabilityisinsensitivitytosmallperturbationsStructuralstabilityignoresmanysmallperturbations.Suchperturbationspreservequalitativeproperties.Basinsofattractionsmaintaintheirbasicshape.PatternSpaceManifoldintersectionintheplane(manifold).Intersectionpointsaandbaretransversal.Pointcisnot:ManifoldsBandCneednotintersectifevenslightlyperturbed.Nopointsaretransversalin3-spaceunlessBisasphere(orothersolid).6.8RandomAdaptiveBidirectionalAssociativeMemoriesBrowniandiffusionsperturbRABAMmodels.SupposedenoteBrownian-motion(independentGaussianincrement)processesthatperturbstatechangesintheithneuronin,thejthneuronin,andthesynapse,respectively.ThediffusionRABAMcorrespondstotheadaptivestochasticdynamicalsystem:WecanreplacethesignalHebbdiffusionlawwiththestochasticcompetitivelaw，differentialHebbianordifferentialcompetitivediffusionlaws,ifweimposetighterconstrainstoensureglobalstability.Thesignal-HebbiannoiseRABAMmodel:TheRABAMtheoremensuresstochasticstability.Ineffect,RABAMequilibriaareABAMequilibriathatrandomlyvibrate.Thenoisevariancescontroltherangeofvibration.AverageRABAMbehaviorequalsABAMbehavior.RABAMTheorem.TheRABAMmodelaboveisglobalstable.Ifsignalfunctionsarestrictlyincreasingandamplificationfunctionsandarestrictlypositive,theRABAMmodelisasymptoticallystable.Proof.TheABAMlyapunovfunctionL:FortheRABAMsystem:alongtrajectoriesaccordingasQ.E.D6.9Noise-Satu