機械工程師手冊,儀器儀表,系統(tǒng),控制系統(tǒng),和MEMch19.doc

791CHAPTER19NEURALNETWORKSINFEEDBACKCONTROLSYSTEMSF.L.LewisAutomationandRoboticsResearchInstituteUniversityofTexasatArlingtonFortWorth,TexasShuzhiSamGeDepartmentofElectricalandComputerEngineeringNationalUniversityofSingaporeSingapore1INTRODUCTION7922BACKGROUND7932.1NeuralNetworks7932.2NNControlTopologies7943FEEDBACKLINEARIZATIONDESIGNOFNNTRACKINGCONTROLLERS7953.1MultilayerNNController7963.2Single-LayerNNController7983.3FeedbackLinearizationofNonlinearSystemsUsingNNs7983.4PartitionedNNsandInputPreprocessing7994NNCONTROLFORDISCRETE-TIMESYSTEMS8005MULTILOOPNNFEEDBACKCONTROLSTRUCTURES8005.1BacksteppingNeurocontrollerforElectricallyDrivenRobot8015.2CompensationofFlexibleModesandHigh-FrequencyDynamicsUsingNNs8025.3ForceControlwithNeuralNets8036FEEDFORWARDCONTROLSTRUCTURESFORACTUATORCOMPENSATION8046.1FeedforwardNeurocontrollerforSystemswithUnknownDeadzone8046.2DynamicInversionNeurocontrollerforSystemswithBacklash8057NNOBSERVERSFOROUTPUTFEEDBACKCONTROL8068REINFORCEMENTLEARNINGCONTROLUSINGNNs8078.1NNReinforcementLearningController8088.2AdaptiveReinforcementLearningUsingFuzzyLogicCritic8099OPTIMALCONTROLUSINGNNs8109.1NNH2ControlUsingtheHamiltonJacobiBellmanEquation8119.2NNHControlUsingtheHamiltonJacobiIsaacsEquation81310APPROXIMATEDYNAMICPROGRAMMINGANDADAPTIVECRITICS81511HISTORICALDEVELOPMENT,REFERENCEDWORK,ANDFURTHERSTUDY81711.1NNforFeedbackControl81711.2ApproximateDynamicProgramming819REFERENCES821BIBLIOGRAPHY825Mechanical Engineers Handbook: Instrumentation, Systems, Controls, and MEMS, Volume 2, Third Edition.Edited by Myer KutzCopyright 2006 by John Wiley & Sons, Inc.792NeuralNetworksinFeedbackControlSystems1INTRODUCTIONDynamicalsystemsareubiquitousinnatureandincludenaturallyoccurringsystemssuchasthecellandmorecomplexbiologicalorganisms,theinteractionsofpopulations,andsoon,aswellasman-madesystemssuchasaircraft,satellites,andinteractingglobaleconomies.VonBertalanffy1wereamongthersttoprovideamoderntheoryofsystemsatthebeginningofthecentury.Systemsarecharacterizedashavingoutputsthatcanbemeasured,inputsthatcanbemanipulated,andinternaldynamics.Feedbackcontrolinvolvescomputingsuitablecontrolinputs,basedonthedifferencebetweenobservedanddesiredbehavior,foradynam-icalsystemsuchthattheobservedbehaviorcoincideswithadesiredbehaviorprescribedbytheuser.Allbiologicalsystemsarebasedonfeedbackforsurvival,witheventhesimplestofcellsusingchemicaldiffusionbasedonfeedbacktocreateapotentialdifferenceacrossthemembranetomaintainitshomeostasis,orrequiredequilibriumconditionforsurvival.Volterrawasthersttoshowthatfeedbackisresponsibleforthebalanceoftwopopulationsofshinapond,andDarwinshowedthatfeedbackoverextendedtimeperiodsprovidesthesubtlepressuresthatcausetheevolutionofspecies.Thereisalargeandwell-establishedbodyofdesignandanalysistechniquesforfeed-backcontrolsystemswhichhasbeenresponsibleforsuccessesintheindustrialrevolution,shipandaircraftdesign,andthespaceage.Designapproachesincludeclassicaldesignmethodsforlinearsystems,multivariablecontrol,nonlinearcontrol,optimalcontrol,robustcontrol,Hcontrol,adaptivecontrol,andothers.Manysystemsonedesirestocontrolhaveunknowndynamics,modelingerrors,andvarioussortsofdisturbances,uncertainties,andnoise.This,coupledwiththeincreasingcomplexityoftodaysdynamicalsystems,createsaneedforadvancedcontroldesigntechniquesthatovercomelimitationsontraditionalfeed-backcontroltechniques.Inrecentyears,therehasbeenagreatdealofefforttodesignfeedbackcontrolsystemsthatmimicthefunctionsoflivingbiologicalsystems.Therehasbeengreatinterestrecentlyinuniversalmodel-freecontrollersthatdonotneedamathematicalmodelofthecontrolledplantbutmimicthefunctionsofbiologicalprocessestolearnaboutthesystemstheyarecontrollingonline,sothatperformanceimprovesautomatically.Techniquesincludefuzzylogiccontrol,whichmimicslinguisticandreasoningfunctions,andarticialneuralnetworks(NNs),whicharebasedonbiologicalneuronalstructuresofinterconnectednodes,asshowninFig.1.Bynow,thetheoryandapplicationsofthesenonlinearnetworkstructuresinfeedbackcontrolhavebeenwelldocumented.ItisgenerallyunderstoodthatNNsprovideanelegantextensionofadaptivecontroltechniquestononlinearlyparameterizedlearningsystems.ThischaptershowshowNNsfulllthepromiseofprovidingmodel-freelearningcon-trollersforaclassofnonlinearsystems,inthesensethatastructuralorparameterizedmodelofthesystemdynamicsisnotneeded.Thecontrolstructuresdiscussedaremultiloopcon-trollerswithNNsinsomeoftheloopsandanoutertrackingunity-gainfeedbackloop.Throughout,therearerepeatabledesignalgorithmsandguaranteesofsystemperformance,includingbothsmalltrackingerrorsandboundedNNweights.Itisshownthatasuncertaintyaboutthecontrolledsystemincreasesorasonedesirestoconsiderhumanuserinputsathigherlevelsofabstraction,theNNcontrollersacquiremoreandmorestructure,eventuallyacquiringahierarchicalstructurethatresemblessomeoftheelegantarchitecturesproposedbycomputerscienceengineersusinghigh-leveldesignapproachesbasedoncognitivelin-guistics,reinforcementlearning,psychologicaltheories,adaptivecritics,oroptimaldynamicprogrammingtechniques.ManyresearchershavecontributedtothedevelopmentofarmfoundationforanalysisanddesignofNNsincontrolsystemapplications.SeeSection11onhistoricaldevelopmentandfurtherstudy.2Background793DendritesNucleusMyelinNode of RanvierAxonCell bodyAxon terminalsSynapsesFigure1Nervoussystemcell.(Withpermissionfrom/jgjohnso/index.html.)2BACKGROUND2.1NeuralNetworksThemultilayerNNismodeledbasedonthestructureofbiologicalnervoussystems(seeFig.1)andprovidesanonlinearmappingfromaninputspaceRnintoanoutputspaceRm.Itspropertiesincludefunctionapproximation,learning,generalization,classication,andsoon.Itisknownthatthetwo-layerNNhassufcientgeneralityforclosed-loopcontrolpur-poses.Thetwo-layerNNshowninFig.2consistsoftwolayersofweightsandthresholdsandhasahiddenlayerandanoutputlayer.Theinputfunctionx(t)hasncomponents,thehiddenlayerhasLneurons,andtheoutputlayerhasmneurons.OnemaydescribetheNNmathematicallyasTTyW(Vx)whereVisamatrixofrst-layerweightsandWisamatrixofsecond-layerweights.Thesecond-layerthresholdsareincludedastherstcolumnofthematrixWTbyaugmentingthevectoractivationfunction()by1intherstposition.Similarly,therst-layerthresh-oldsareincludedastherstcolumnofthematrixVTbyaugmentingvectorxby1intherstposition.ThemainpropertyofNNsweareconcernedwithforcontrolandestimationpurposesisthefunctionapproximationproperty.2,3Let(x)beasmoothfunctionfromRnRm.Then,itcanbeshownthatiftheactivationfunctionsaresuitablyselectedandisrestrictedtoacompactsetSRn,thenforsomesufcientlylargenumberLofhidden-layerneurons,thereexistweightsandthresholdssuchthatonehasTT(x)W(Vx)(x)with(x)suitablysmall.Here,(x)iscalledtheneuralnetworkfunctionalapproximationerror.Infact,foranychoiceofapositivenumberN,onecanndaNNoflargeenoughsizeLsuchthat(x)NforallxS.FindingasuitableNNforapproximationinvolvesadjustingtheparametersVandWtoobtainagoodtto(x).Notethattuningoftheweightsincludestuningofthethresholdsaswell.TheneuralnetisnonlinearintheparametersV,whichmakesadjustmentoftheseparametersdifcultandwasinitiallyoneofthemajorhurdlestobeovercomeinclosed-794NeuralNetworksinFeedbackControlSystemsFigure2Two-layerNN.loopfeedbackcontrolapplications.Iftherst-layerweightsVarexed,thentheNNislinearintheadjustableparametersW(LIP).Ithasbeenshownthat,iftherst-layerweightsVaresuitablyxed,thentheapproximationpropertycanbesatisedbyselectingonlytheoutputweightsWforgoodapproximation.Forthistooccur,(VTx)mustprovideabasis.Itisnotalwaysstraightforwardtopickabasis(VTx).Ithasbeenshownthatthecerebellarmodelarticulationcontroller(CMAC),4radialbasisfunction(RBF),5fuzzylogic,6andotherstructuredNNapproachesallowonetochooseabasisbysuitablypartitioningthecompactsetS.However,thiscanbetedious.Ifoneselectstheactivationfunctionssuitably(e.g.,assigmoids),thenitwasshowninRef.7that(VTx)isalmostalwaysabasisifisselectedrandomly.2.2NNControlTopologiesFeedbackcontrolinvolvesthemeasurementofoutputsignalsfromadynamicalsystemorplantandtheuseofthedifferencebetweenthemeasuredvaluesandcertainprescribeddesiredvaluestocomputesysteminputsthatcausethemeasuredvaluestofollow,ortrack,thedesiredvalues.Infeedbackcontroldesignitiscrucialtoguaranteebyrigorousmeansboththetrackingperformanceandtheinternalstabilityorboundednessofallvariables.Failuretodosocancauseseriousproblemsintheclosed-loopsystem,includinginstabilityandunboundednessofsignalsthatcanresultinsystemfailureordestruction.TheuseofNNsincontrolsystemswasrstproposedbyWerbos8andNarendraandParthasarathy.9NNcontrolhashadtwomajorthrusts:approximatedynamicprogramming,3FeedbackLinearizationDesignofNNTrackingControllers795PlantControlu(t)Outputy(t)NN controllerNN systemidentifierEstimatedoutput )(tyIdentificationerrorDesiredoutput)(tydPlantControlu(t)Outputy(t)NN controllerDesiredoutput)(tydTracingerrorPlantControlu(t)Outputy(t)NN NNDesiredoutput)(tydTrackingerror(a)(b)(c)u(t)y(t)NN controllerNN systemidentifieroutput )(tyerroroutput)(tydu(t)y(t)NN controlleroutput)(tyderroru(t)y(t)NN NNoutput)(tyderrorcontroller 1controller 2Figure3NNcontroltopologies:(a)indirectscheme;(b)directscheme;(c)feedback/feedforwardscheme.whichusesNNstoapproximatelysolvetheoptimalcontrolproblem,andNNsinclosed-loopfeedbackcontrol.Manyresearchershavecontributedtothedevelopmentoftheseelds.SeeSection11andtheReferencesandBibliography.SeveralNNfeedbackcontroltopologiesareillustratedinFig.3,10someofwhicharederivedfromstandardtopologiesinadaptivecontrol.11Solidlinesdenotecontrolsignalowloopswhiledashedlinesdenotetuningloops.Therearebasicallytwosortsoffeedbackcontroltopologies:indirectanddirecttechniques.InindirectNNcontroltherearetwofunc-tions;inanidentierblock,theNNistunedtolearnthedynamicsoftheunknownplant,andthecontrollerblockthenusesthisinformationtocontroltheplant.DirectcontrolismoreefcientandinvolvesdirectlytuningtheparametersofanadjustableNNcontroller.ThechallengeinusingNNsforfeedbackcontrolpurposesistoselectasuitablecontrolsystemstructureandthentodemonstrateusingmathematicallyacceptabletechniqueshowtheNNweightscanbetunedsothatclosed-loopstabilityandperformanceareguaranteed.Inthischapter,weshallshowdifferentmethodsofNNcontrollerdesignthatyieldguaranteedperformanceforsystemsofdifferentstructureandcomplexity.Manyresearchershavepar-ticipatedinthedevelopmentofthetheoreticalfoundationforNNsincontrolapplications.SeeSection11.3FEEDBACKLINEARIZATIONDESIGNOFNNTRACKINGCONTROLLERSInthissection,theobjectiveistodesignanNNfeedbackcontrollerthatcausesaroboticsystemtofollow,ortrack,aprescribedtrajectoryorpath.Thedynamicsoftherobotareunknown,andthereareunknowndisturbances.Thedynamicsofann-linkrobotmanipulatormaybeexpressedas12796NeuralNetworksinFeedbackControlSystemsM(q)q嬠V(q,q)(q嬠G(q)嬠F(q)嬠嬠嬠嬠(1)mdwithq(t)嬠Rnthejointvariablevector,M(q)aninertiamatrix,Vmacentripetal/Coriolismatrix,G(q)agravityvector,andF(嬠)representingfrictionterms.Boundedunknowndis-turbancesandmodelingerrorsaredenotedby嬠dandthecontrolinputtorqueis嬠(t).Thesliding-modecontrolapproachofSlotine13,14canbegeneralizedtoNNcontrolsystems.Givenadesiredarmtrajectoryqd(t)嬠Rn,denethetrackingerrore(t)嬠qd(t)嬠q(t)andtheslidingvariableerrorr嬠嬠嬠e,where嬠嬠嬠T嬠0.Asliding-modemanifoldeisdenedbyr(t)嬠0.TheNNtrackingcontrollerisdesignedusingafeedbacklinearizationapproachtoguaranteethatr(t)isforcedintoaneighborhoodofthismanifold.Denethenonlinearrobotfunction(x)嬠M(q)(q嬠嬠e)嬠V(q,q)(q嬠嬠e)嬠G(q)嬠F(q)(2)dmdwiththeknownvectorx(t)ofmeasuredsignalssuitablydenedintermsofe(t),qd(t).TheNNinputvectorxcanbeselected,forinstance,asTTTTTTx嬠eeqqq(3)ddd3.1MultilayerNNControllerANNcontrollermaybedesignedbasedonthefunctionalapproximationpropertiesofNNs,asshowninRef.15.Thus,assumethat(x)isunknownandgivenapproximatelyastheoutputofaNNwithunknownidealweightsW,Vsothat(x)嬠WT嬠(VTx)嬠嬠with嬠anapproximationerror.Thekeyisnowtoapproximate(x)bytheNNfunctionalestimate,withthecurrent(estimated)NNweightsasprovidedbythetuningTT(x)嬠W嬠(Vx)V,Walgorithms.Thisisnonlinearinthetunableparameters.Standardadaptivecontrolap-VproachesonlyallowLIPcontrollers.NowselectthecontrolinputTT嬠嬠W嬠(Vx)嬠Kr嬠v(4)vwithKvasymmetricpositive-denite(PD)gainandv(t)acertainrobustifyingfunctiondetailedinRef.15.ThisNNcontrolstructureisshowninFig.4.TheouterPDtrackingloopguaranteesrobustbehavior.TheinnerloopcontainingtheNNisknownasafeedbacklinearizationloop,16andtheNNeffectivelylearnstheunknowndynamicsonlinetocancelthenonlinearitiesofthesystem.LettheestimatedsigmoidJacobianbe.Notethatthisjacobianis嬠嬠嬠d嬠(z)/dz嬠Tz嬠VxeasilycomputedintermsofthecurrentNNweights.Then,thenextresultisrepresentativeofthesortoftheoremsthatoccurinNNfeedbackcontroldesign.ItshowshowtotuneortraintheNNweightstoobtainguaranteedclosed-loopstability.Theorem(NNWeightTuningforStability)Letthedesiredtrajectoryqd(t)anditsderivativesbebounded.Takethecontrolinputfor(1)as(4).LetNNweighttuningbeprovidedbyTTTTTW嬠F嬠r嬠F嬠嬠Vxr嬠嬠F嬠r嬠WV嬠Gx(嬠嬠Wr)嬠嬠G嬠r嬠V(5)withanyconstantmatricesF嬠FT嬠0,G嬠GT嬠0,andscalartuningparameter嬠嬠0.Initializetheweightestimatesas.Thentheslidingerrorr(t)andNNW嬠0,V嬠randomweightestimatesareuniformlyultimatelybounded.W,V3FeedbackLinearizationDesignofNNTrackingControllers797Robot IRobust v(t)Tracking loopf(x)rNonlinear inner loop=.=.=.Robot system IRobust controlTracking Nonlinear =.=.=.=.=.=.termqdeeeKvqqqqdqdqdFigure4NNrobotcontroller.Aproofofstabilityisalwaysneededincontrolsystemsdesigntoguaranteeperform-ance.Here,thestabilityisprovenusingnonlinearstabilitytheory(e.g.,anextensionofLyapunovstheorem).ALyapunovenergyfunctionisdenedas1T1T11T1LrM(q)rtrWFW)trVFV)222wheretheweightestimationerrorsare,withtrthetraceop-VVV,WWWeratorsothattheFrobeniusnormoftheweighterrorsisused.Intheproof,itisshownthattheLyapunovfunctionderivativeisnegativeoutsideacompactset.Thisguaranteestheboundednessoftheslidingvariableerrorr(t)aswellastheNNweights.Specicboundsonr(t)andtheNNweightsaregiveninRef.15.Thersttermsof(4)areveryclosetothe(continuous-time)backpropagationalgorithm.17ThelasttermscorrespondtoNarendrase-modication18extendedtononlinear-in-the-parametersadaptivecontrol.Robustadaptivetuningmethodsfornonlinear-in-the-parametersNNcontrollershavebeenderivedbasedontheadaptivecontrolapproachesofe-modication,Ioannous-modication,orprojectionmethods.ThesetechniquesarecomparedbyIoannouandSun19forstandardadaptivecontrolsystems.RobustnessandPassivityoftheNNWhenTunedOnlineThoughtheNNinFig.4isstatic,sinceitistunedonline,itbecomesadynamicsystemwithitsowninternalstates(e.g.,theweights).ItcanbeshownthatthetuningalgorithmsgiveninthetheoremmaketheNNstrictlypassiveinacertainnovelstrongsenseknownasstate-strictpassivity,sothattheenergyintheinternalstatesisboundedabovebythepowerdeliveredtothesystem.Thismakestheclosed-loopsystemrobusttoboundedun-knowndisturbances.Thisstrictpassivityaccountsforthefactthatnopersistenceofexcitationconditionisneeded.Standardadaptivecontrolapproachesassumethattheunknownfunction(x)islinearintheunknownparametersandacertainregressionmatrixmustbecomputed.Bycontrast,theNNdesignapproachallowsfornonlinearityintheparameters,andineffecttheNNlearnsitsownbasissetonlinetoapproximatetheunknownfunction(x).Itisnotrequired798NeuralNetworksinFeedbackControlSystemsNonlinear IRobust ur(t)Tracking loopr(t)Nf(x)controlg(x)Xdx(t)e(t)Nonlinear system IRobust control()Tracki)Nonlinear inner loops)control()()()()termKvFeedback lineFigure5FeedbacklinearizationNNcontroller.tondaregressionmatrix.ThisisaconsequenceoftheNNuniversalapproximationprop-erty.3.2Single-LayerNNControllerIftherst-layerweightsVarexedsothat,with毠(x)selectedTTT(x)毠W毠(Vx)毠W毠(x)asabasis,thenonehasthesimpliedtuningalgorithmfortheoutputlayerweightsgivenbyTW毠F毠(x)r毠毠F毠r毠WThen,theNNisLIPandthetuningalgorithmresemblesthoseusedinadaptivecontrol.However,NNdesignstilloffersanadvantageinthattheNNprovidesauniversalbasisforaclassofsystems,whileadaptivecontrolrequiresonetondaregressionmatrix,whichservesasabasisforeachparticularsystem.3.3FeedbackLinearizationofNonlinearSystemsUsingNNsManysystemsofinterestinindustrial,aerospace,andU.S.DepartmentofDefense(DoD)applicationsareintheafneform,withd(t)a