CSET：中國的軍事人工智能挑戰(zhàn)（英文版）

ExecutiveSummaryChinahasmadesignificantinvestments,andapparentprogress,inmilitaryAIoverthepastseveralyears.GivenAIandrelatedemergingtechnologiescouldplaycriticalrolesinfuturewars,theU.S.nationalsecurityestablishmentisworriedaboutfallingbehindChinaindevelopinganddeployingthesetechnologies.1SuchconcernshaveelicitedmuchattentionandalarmintheUnitedStates.Itisdifficult,however,todevelopacomprehensiveunderstandingofwhereeachmilitarystandswhenitcomestomilitaryAIdevelopmentandadoption,letaloneconductadirectcomparisonbetweenthetwo.Moreover,oftenmissingfromexistingassessmentsofChinesemilitaryAIareChineseperspectilackthereof,indevelopingandfieldingAI-relatedtechnologiesandcapabilities.Thisreportseekstofillthatanalyticalgap.ThispaperoutlinesseveralofthekeychallengesChineseisfacingregardingthedevelopmentanddeploymentofAIandrelatedemergingtechnologies.Itdrawson59Chinese-languagejournalarticlespublishedbetween2020and2022byPeople’sLiberatheChinesedefenseindustry,andacademicsworkingonAIandotheremergingtechnologies.Keyfindingsincludethefollowing:scholarsbelievethatAIandotheremergingtechnologiespresentChinawithauniqueopportunitytocatchuptoorsurpasstheUnitedStatesmilitarily.militaryindevelopingandfieldingAIandrelatedemergingtechnologies.3.ChinesedefenseexpertsclaimthatthePLAisfacingbarrierstodevelopinganddeployingAIandrelatedemergingtechnologies.TheobstaclestheexpertshighlightaresimilartosomeofthosewithwhichtheU.S.militaryalsoappearsPLA’sabilitytoguaranteenetworkandcybersecurity,maintaincommunicationsinfuturehigh-intensityconflicts,anddeveloptrustworthyAIsystems.4.TheexpertsalsopointoutissuesthatmaybeespeciallyrelevanttothePLA’sabilitytoeffectivelyuseAI-enabledmilitarysystems.Theseconcernsrevolvearoundmilitarydatacollection,management,andanalysis,aswellasthedevelopmentofhigh-endsensors.TheyalsoappearworriedaboutthePLA’sinsufficientlyrobustmilitarystandardsandtestingandevaluationpractices.Theseproblems,aswellasthosementionedintheaboveparagraph,maycomplicateChina’spathtowardcarryingoutAI-enabledsystemswarfare,akeyoperationalconceptthatthePLAbelieveswouldhelpitwinfuturewars.5.ContrarytosomeU.S.discussionsofChineseexpertswhoseargumentshavebeenanalyzedinthisreportvoicemisgivingsaboutusinginsufficientlytrustworthyAIsystemsinmilitarycontexts.2Theyalsoexpressconcernsabouttherisksofoutbreaksorescalationsofwars,civiliandeaths,andfriendlyforcetargetingbyAI-enabledmilitarysystemsduetoinsufficientlytrustworthyAI.Finally,theynotethatuntrustworthymilitaryAIsystemsmaybelesseffectiveinfuturewars.TableofContents Introduction 4 7ABriefSurveyofChina’sProgressTowardAI-EnabledWarfare 10ChineseExperts’ViewsofAI-EnabledWarfare 12TechnologicalBarrierstoDevelopingandFieldingMilitaryAI 14 29 32Author 33Acknowledgements 33AppendixA 34 35IntroductionChinesedefensedoctrineandwritingsenvisionChina’smilitarymodernizationtobeunfoldinginthreesuccessivebutoftenoverlainformatization(信息化),andintelligentization(智能化).WhenXiJinpingcametopowerin2013,Chinabegantofocusonpreparingforinformatizedwarfare,orusinginformationtechnologytoimprovecommandandcontrol,aidprecisionstrikes,andutilizespace,cyber,andelectromagneticcapabilities.3Inhis2017speechatthe19thtowardinformatizationby2020.4Inthesamespeech,XiexhortedthePLAtospeedtheprocessofmilitaryintelligentization,broadlydefinedastheuseofartificialintelligence(AI)andotheradvancedtechnologiesfordefenseapplications.5TheChinesegovernmentandstrategiccommunityareconfidentthatAIandotheremergingtechnologiescouldprovidethePLAwithanopportunitytocatchupto,orevensurpass,themilitarypoweroftheUnitedStates.6Forinstance,prominentPLAaffiliates,suchasLieutenantGeneralLiuGuozhi,formerlyoftheCentralMilitaryCommission,havearguedthatwhenitcomestomilitarycapabilities,intelligentizationisa“rareopportunity”to“overtake[competitors]onthecurve.”7Officialgovernmentguidancemirrorstheleadership’sgrowingfocusonintelligentizedwarfare.China’s2019DefenseWhitePapernotedthePLA’sincreasingfocusonbolsteringits“intelligence”inwarfare,whilethe14thFive-YearPlandirectedthemilitaryto“accelerateintegrateddevelopmentofmechanization,informatization,andintelligentization.”8Indeed,intelligentizationhasbecomethelodestarforthePLA’sdevelopmentandmodernization.9ThePLAhasnot,however,releasedofficialguidanceonitsconceptsforintelligentizedwarfare,norpreciselyonhowAIandrelatedtechnologiesfactorintotheconcept.Moreover,definitionsoftheterm,aswellasthatofinformatization,remainpurposefullyvague.Atthesametime,Chinesemilitarystrategistsanddefenseacademicshavepublishedthousandsofmediacommentaries,think-tankreports,andjournalarticlesthatcanhelpanalystsgainasenseofhowChinaisthinkingaboutusingAIandrelatedemergingtechnologiesinwarfare.10ThemajorityofthesematerialsarepublishedinChinese.Whiletheyaregenerallyunder-examinedbyEnglish-speakingexperts,severalresearchershaveanalyzedvaryingselectionstogleaninsightsintohowChineseanalyststhinkfuturewarswillbefought.SomereportsfocusonhowChinesedefenseexpertsenvisionhowAIandotheremergingtechnologieswillrevolutionizemilitarycapabilities,whileothersfocusonthesetechnologies’impactsondeterrenceandstrategicstability.11Stillotherresearchinterrogatesmilitary-civilfusionprograms,corporateandacademicinitiativestoboostmilitaryintelligentization,anddoctrinalAIandotheremergingtechnologiessuchasuncrewedsystems,autonomy,aanalysisappearcentraltoChinesemilitaryconceptsforfightingfuturewars.Thereis,however,relativelylimitedresearchfocusedonunderstandinghowChinesestrategistsanddefenseexpertsviewtheirowncountry’sprogresstowarddevelopinganddeployingthesetechnologiesandcapabilities,whichareultimatelymeanttousherinthefinalstagesofChina’smilitarymodernization.Itisthisanalyticalgapthatthereportseekstofill.Throughareviewof130Chinese-languagejournalarticlesaboutAIandthefutureofwarfare,IidentifyseveralareasinwhichChinesedefenfacingdifficultiesindevelopingmilitaryAIandrelatedemergingtechnologies.ThereportalsoprovidesinformationonthetechnologiesthatChineseauthorsbelieveBeijingshoulddeveloptoredressitscurrentshortcomings.ThatthepapersareChinese-languagejournalarticlesissignificant,astheyaremeanttocontributetotheChineseacademicliteratureonmilitaryAIandotheremergingtechnologiesandareaimedatlocaldefenseexperts.Iargue,therefore,thatthesepaperscontaininsightsthatareworthnotingsincetheylikelyreflectaspectsofongoingdebatesinChinaoverthemilitaryrelevanceofAIandotheremergingtechnologies.AreadingofChinesejournalarticlesaboutthemilitaryapplicationsofAIandrelatedemergingtechnologiesconveysthesensethatChinaishopefulaboutthemilitaryutilityofAIandisfocusedonrealizingitspotential.Atthesametime,ChinesedefenseexpertsareconcernedaboutthetrustworthinessofAIsystemsandworrythatBeijingmaybetrailingtheUnitedStatesinthedevelopmentanddeploymentofAIandrelatedemergingtechnologiesinseverevealdoubtsaboutChina’sabilitytodevelopseveraltechnologiesthatwouldallowittocatchupto,orsurpass,theU.S.military.Whiletheanalysisinthisreporthasanumberoflimitations,itnonethelessrevealsimportant,andattimesunexpected,insightsintohowChinesedefenseexpertsviewtheirowncountry’scapabilitiesandprogressonmilitaryAIandrelatedemergingtechnologies.ScopeandMethodologyThisreportisbasedonananalysisofreflections,hypothesmilitaryAIandrelatedemergingtechnologiesinacorpusof130ChinesejournalarticlesauthoredbyPLA-affiliatedandnon-PLA-affiliatedChinesedefenseexperts.ThesepaperswerepublishedbetweenJanuary1,2020,andDecemberwerecapturedinaChinaNationalKnowledgeInfrastructure(C30,2023.*Ofthese130articles,Ifocusmostcloselyonthe59papersthatfeatureddiscussionsofperceivedbarrierstothedevelopmentandfieldingofmilitaryAIandrelatedemergingtechnologies.Twentyofthose59wereauthoredbyaffiliatesofthePLAoritsuniversitiesandacademies;16wercompaniesorresearchorganizationsintheChinesemilitaryindustrialcomplex;12werebyacademicsatnon-PLAaffiliateduniversities;andtheother11hadnoauthoraffiliationinformation(8),werewrittenbyscholarsatcompanieswithoutclearlinkstothePLAortheChinesedefenseindustry(2),orwereauthoredbyresearchersaffiliwithChineseministries(1).Intermsofthejournalsinwhichthe59paperswerepublished,overthree-quartersappearedinpublicationsmanagedbythePLAoritsacademies/universities,organizationsinChina’smilitaryindustrialcomplex,orotherForeachofthe59papers,IcreaoutlineaboutthecurrentstateofAIandrelatedemergingtechnologies;authors’viewsonthedifficultiesofdevelopingand/oroperationalizingAI-enabledmilitarysystems;and,whererelevant,recommendationsthatthescholarsmakeforChinatoenhanceitsabilitytodevelopanddeploymilitaryAIsystemsandrelatedemergingtechnologies.Someoftheauthorsnoteproblemsallmilitariesarefacing,whileotherspointout*SeeAppendixAforafullexplanationofthemethodologyforgeneratingtheCNKIsnapshot.Becauseofthepairedkeywordapproach,thesnapshotcapturedarticlesthatwerenotrelevanttomilitaryAI.Todeterminethefinallist,theauthorreadthetitlesandabstractsofeveryarticlecapturedinthesnapshotanddeterminedwhethereachwasrelevanttomilitaryAI.Thisprocesshelpednarrowdowntoalistof130articlesrelevanttomilitaryAI,eachofwhichtheauthorreadinfull.specificissueswithwhichChinaishavingdifficulties.Inthispaper,IidentifywhetherInsomecases,authorsdidnotidentifycurrentshortcomingsinChina’smilitaryAIcapabilitiesbutproviderecommendationsforfuturedevelopment.GiventhepotentialprofessionalconsequencesassociatedwithopenlycriticizingthePLAanditscapabilities,Itreatedtheserecommendations,aswellasgeneralstatementsaboutexpertsbelievethePLAneedstomakeprogresstobecomeaworld-classfightingThe59papersinthedatasetbroadlyfitintosevencategories:1)DiscussionsofthecurrentstateofAI-enabledmilitarycapabilities,manyofwhichincludeanalysisoflimitationsandrecommendationsfordevelopingspecifictechnologies;2)BroadoverviewsoftheexpectedfuturecapabilitiesofAI-enabledmilitary3)ExaminationsoftheimpactofAIandotheremergingtechnologiesonstrategicstability;4)Surveysofdevelopmentsinspecificmilitaryfieldsduringacertainyear;5)Coverageofhowdifferentmilitaries(mostlytheUnitedStates)haveapproacheddevelopingspecificAI-enabledsystems;6)Discussionsofthemoral,ethical,andlegalaspectsofusingAI-enabledmilitarysystems;and7)TechnicalpapersonspecifictechThemajorityofthepapersIcitebelowarefromthefirstcategory.Insomeinstances,however,Iuseinformationfromarticlesintheothercategories.PriortopresentingtheFirst,thestudyanalyzesperceivedbarrierstotheChinesemilitary’sadoptionofAIandemergingtechnologies(andtheirpotentialuseandutilityinfuturewar),butitisuncleartowhatextenttheseperceptionsmatchreality.NoteveryauthorisaffiliatedwiththePLA,whichlikelylimitstheirknowledgeofandaccesstoinformationaboutsensitivemilitarytechnologies.EvenPLA-affiliatedauthorsmaynotbeawareofthelatestadvancementsinthemilitary’sdevelopmentandadoptionofAIandotheremergingtechnologiesorbeauthorizedtopubliclydiscussthisinformation.Therefore,thispaperdoesnotmakeanyclaimsabouttheinfluenceofthecitedauthors—ortheirviews—withinthePLAorbroaderChinesedefenseestablishment.Second,thisreportavoidslinkingknownChinesetechnologiesorcapabilitieswiththecitedexperts’viewsofextantAI-enabledsystemsandrelatedemergingtechnologies;itsolelyintendstoillustratehowChinesedefenseexpertsperceivebarrierstothePLA’seffortstodevelopanddeploythesesystems,technologies,andcapabilities.BecausetheinformationrelayedinthesearticlesisaimedprimarilyatotherChinesedefenseexperts,itisunlikelytheauthorswouldintentionallymisrepresentormischaracterizecertaintechnologiesorcapabilitieintermsofaccuracyandreliability,andassuch,tThird,thereportfeaturespapersavailableintheCNKIdatabaseuptoDecember31,2022,whichmaynotsufficientlycapturearticlesinnewerormorespecializedjournals.Fourth,thereportfocusesononlyasubsetofperceivedchallengestoAIadoptioninthePLA,includingthoserelatedtotechnology,standards,andtestingandevaluation.Manyotherconcernsarealsoraisedinthearticlesreviewedforthisstudy,includingthosesurroundingoperationalconceptsandworkforceissues,buttheyarenotanalyzedbelow.Finally,astheChinesepoliticalsystemtightens,itmaybecomemoreandmoredifficultforChineseexpertstovoiceviewsatoddswithofficialPartystandpoints.Self-censorshipcouldimpactthecontentofthearticles,whilestatecensorshipcouldremoveentirepublications,therebyskewingthedatasetandtheanalysis.GiventheopacityofboththeChinesegovernmentandmilitary,however,thesearticlesmayprovideinsightsintoChinesestrategicthinkingonseveraltechnologicalissuesthattheABriefSurveyofChina’sProgressTowardAI-EnabledWarfareChineseofficialsbelieveAIcouldusherinanewrevolutioninmilitaryaffairs(RMA)thatcouldfundamentallyalterhowfuturewarsarefought.14SeniorChineseofficialsarguethePLAhasanopportunity—givenChina’sstrongscienceandtechnologybase—todevelopcutting-edge,disruptiveAI-enabledmilitarycapabilities.15Xi’sfocusonmilitarymodernizationhasledthePLAtoaugmentitsdrivetorealizescientificandtechnologicaladvances,or“innovation-drivendevelopment.”16OntopofarenewedfocusonS&Tdevelopments,in2015thePLAembarkedonsignificantreformsaimedatdrivingmilitarymodernizationandimprovingthePLA’sabilitytoconductjoint,multi-domainoperations.17Aspartofthesereforms,thePLAestablishedtheStrategicSupportForce(SSF)tooverseethedevelopmentopsychological,cyber,andspace-relatedmilitarycapabilities.TheSSFwasrecentlyre-18TheScienceandTechnologyCommission,whichistaskedwithdrivingscienceandtechnologyinnovation,wasformerlyunderthepurviewoftheGeneralArmamentDepartmentbutisnowastandaloneCentralMilitaryCommissionentity.19ThePLAhasreformedtheAcademyofMilitaryScienceanddirectedittocombineemergingtechnologieswithnoveloperationalconcepts.20Additionally,theChinesemilitaryisleaningonarangeofdefense-relatedresearchinstitutions,aswellasdefenseindustryplayers,tocreateinnovativetechnologiesandconceptsforfuturewarfare.21Inshort,China’smilitaryreformsandmodernizationareaimedathelpingthePLAdevelopandfieldnoveltechnologies,aswellascreatenewoperationalconcepts,inordertobecomeaworld-classfightingforcebythemiddleofthiscentury.22PreviousresearchhasoutlinedhowthePLAisdevelopingrangeofAI-enabledmilitarysystems.Forexample,CSETChinesemilitaryisacquiringAI-enabledtechnologyforintelligenceanalysis,informationwarfare,logisticsandpredictivemaintenance,andnavigationandtargetrecognition.23ThePLAisalsoreportedlydevelopingmachinelearningforremotesensingandsituationalawareness;AI-enabledcommunicationstechnologiesthatarebetterprotectedfromjamming;neuralnetworkscapableofguidingmissilesandunderwatervehicles;andAI-enableddatamanagementandfusioncapabilities,amongChinesedefenseacademicsandstrategistshaveauthoredmyexploringfutureoperatingconceptsanddetailinghowAIandotheradvancedtechnologiesmayimpactfuturemilitarycapabilities.PreviousanalysishasfoundthatChinesestrategicthinkersbelievethatAIandautonomywillincreasethespeedandimprovetheeffectivenessofmilitaryoperationsandaidindetection,targeting,andstrikes.25Theyseeimprovementsinintelligentmunitions,aerialdrones,andexploitationofintelligence,surveillance,andreconmostimportantdevelopmentsdrivingadvancesinwarfare.AIwill,theyargue,enhancebattlefieldsensing,improvecommandandcontrol(C2),aiddecision-making,andshortentheOODA(observe-orient-decide-act)loop.26ChineseExperts’ViewsofAI-EnabledWarfareManyofthejournalarticlessurveyedforthisreportfeaturesimilarlyexuberantexpectations.Theauthorsgenerallybelievethatadvancementsinalgorithms,communicationstechnologies,sensors,anddataprocessingwillleadtoimprovedinterconnectionsamongweapons,equipment,andsoldiers.AdvancementsinAI,theyclaim,willalsoaugmenttargetingandspeedChina’sdevelopmentofjoint,multi-domainwarfightingcapabilities.Theynotehowthedeploymentofnext-generationcommand,control,communications,computers,intelligence,surveillance,andreconnaissance(C4ISR)systemswillprovidemilitarieswithbettersituationalawareness,battlefieldreconnaissance,long-rangeprecisionstrikes,andmoreefficientOthersarguethatnext-generationtechnologiessuchas5Gwillensure“seamlessnetworkingamongunmannedsystems.”28SomeChinesecommentatorscontendthatunmannedaerialvehicle(UAV)swarmswillimproveadvancedmilitaries’abilitytostrikeadversaries.29Andyetothersassertthatsoldiersandintelligentsystemswill“seamlesslycollaborate,”allowingfortheexecutionofjoint,multi-domainoperations.30Autonomousmachineswill,accordingtosomescholars,vastlyreducethenumberofhumansonfuturebattlefieldsandexpandthedepthofthebattlespace.31Finally,theChineseexpertsspeculatethattechnologicalprogresswillallowforadvancesincyber,psychological,andcognitivewarfare.32TheaboveisonlyasmallselectionofthediversepredictionsthatChinesescholarsmakeabouttheimpactofAIandrelatedemergingtechnologiesonfuturemilitarycapabilities.Theyarerepresentative,however,ofthegeneralviewthatprogressinAIwillallowadvancedmilitariestoimprovetheirC4ISR,dataanalysis,andstrikecapabilities,graduallyremovehumansfromfuturebattlefields,andfieldincreasinglyautonomoussystems.AI-enabledgatheringandprocessingofinformationwill,theyargue,leadtofasterandbetterdecision-makingandmoreprecisetargeting,allowingthePLAtobecomealeadingfightingforce.Theintegrationofemergingtechnologieswithcutting-edgemilitarysystemswill,inthewordsofonescholar,leadtoa“1+1>2”effect,meaningthenext-generationkillchainwillbegreaterthanthesumofitsDespitetheirpredictionsaboutAI-enabledimprovementsinwarfightingcapabilities,manyscholarsalsovoiceconcernsoverthepotentiallydestabilizingeffectsofAI-enabledmilitarysystems.Themajorityofscholarswhosewritingsarereferencedinthisreportbelieveemergingtechnologiescouldnegativelyaffectstrategicstability,leadingtotheincreasedincidenceofpreemptivestrikesacomplicatingescalationcontrol.34Moreover,theyworryabouttheproliferationofAI-enabledequipmenttoterroristsandothernon-stateactors.35Asaconsequence,thescholarssurveyedhererepeatedlynotethatAIisa“double-edgedsword,”asitprovidesopportunitiesformilitaryadvancementwhileconcurrentlyamplifyingrisks.36Despitetheseconcerns,theyappeartofavordevelopingnext-generationmilitarycapabilities,astheirsuccessfuloperationalizationwouldprovidethePLAwithitsbestchanceattriumphingoveradversariesinfuturecombat.Thereis,however,asignificantgapbetweenpredictedfuturecapabilitiesandextanttechnologicalrealities.TheremainderofthisreportfocusesonthetechnologicalbarriersChineseobserversidentifythatmayhinderthePLA’smodernizationefforts,aswellastheirrecommendationsforR&Dofspecifictechnologiesandcapabilitiesthatwill,theyhope,allowChinatoovercomeitscurrentshortcomings.TechnologicalBarrierstoDevelopingandFieldingMilitaryAIandRelatedEmergingTechnologiesU.S.defenseofficialsandindustryleadersWashingtoninthedevelopmentanddeploymentofAI-enabledmilitarycapabilities.37Themajorityofthedefenseexpertswhosearticlesareanalyzedinthisreport,however,believethattheUnitedStatesappearstobeaheadofChinaregardingthemilitaryuseofAI.SeveralpapersoutlinepubliclyreportedU.S.testsofAI-enabledsystemsandarguetheUnitedStatescontinuestooutpaceChinainthedevelopmentofmilitaryAI.NationalDefenseTechnology(國防科技)journal,writtenbyauthorsfromtheLaboratoryforHighTechnology(高技術(shù)實驗室)oftenreferredtoasChina’sMassachusettsInstituteofTechnology(MIT)—argue“comparedwithothermilitarypowersintheworld,militaryintelligenceinChinaisstillinitsinfancy.”38Thisis,perhaps,anoverstatementofChina’srelativetechnologicalshortcomings.ButotherscholarsalsonotetechnicalissueswithAIandotheremergingtechnologiesthathindertheirmilitaryapplicationbothinChinaandglobally.ThefollowingsectionsdelveintoseveralperceivedtechnologicalbarrierstotheuseofAIandotheremergingtechnologiesinwarfare,includingthoserelatedtotrustworthyAI,data,networkandcybersecurity,sensors,communicationstechnologies,standards,andtestingandevaluation.Inthemajorityofthefollowingsections,IalsodiscusstheproposedtechnologiestheexpertsidentifythatcouldaidBeijinginovercomingitsArtificialIntelligenceSystems:Capabilities,Characteristics,andConcernsChinesedefenseexpertsbelieveemergingtechnologieswillincreasethespeedofcombat,battlefieldunpredictability,andthefogofwar.39WhiletheyarguethatAI-enabledmilitarysystemswillplayaroleincreatingtheseconditions,theyalsonotetheneedtouseAItocounteractthesepotentialbattlefieldeventualities.Thethinkers’predictionsaboutthefutureofAI-enabledsystemsandtheirimpactonthebattlefield,however,aretemperedbyanotablelackoftrustinthetechnologyasitexiststoday.TheauthorsvoicedoubtsaboutthecurrentcapabilitiesofAItechnologies,notingthatAIiscurrently“weak,”orbestsuitedtohandlespecifictaskswithclearrequirementsandboundaries.40AIstillstrugglestounderstandabstractconcepts,autonomouslyreason,andmakedecisions.41Furthermore,itisunabletoreasonbeyondmathematicalcalculations,renderingsystemsthatemployAIunabletomakevaluejudgmentsordifferentiatebetweencombatantsandnon-combatantsonthebattlefield.42ThenarrowapplicabilityofAI-enabledsystemslimitstheirutility,aswellasthebroaderdevelopmentof“intelligentwarfare,”atleastintheshortterm.43ChineseobserversalsoappeartoharbordeepconccausehumansnottotrustAI-generateddecisionrecommendations.44SomeauthorsnotethatlackofexplainabilityposeshiddendangersforAI’sapplication.45Otherexpertsarticulateconcernsaboutensuringsystems’predictaimportantly,controllability,aslosingcontrolofAIsystemscouldincreasetheriskofescalationandbroaderinstability.46TheauthorsalsoexpressconcernsabouttrustinAIthatshouldbefamiliartothoseintooperateeffectively,giventheuncertaintiesofcontemporaryorfuturebattlefields,ortoberesilienttoshiftsintheoperationalenvironment.47Moreover,anoverrelianceon suchAIsystemscouldleadtoinadvertentescalationorautomationbias.48Giventhe“slowandimmaturedevelopment”ofAItechnology,theexpertsarguethat,asAI-enabledandautonomousmilitarysystemsbecomeincreasinglycomplicated,theywillcontinuetobeunreliable,fragile,andunstable,leadingtoserioussafetyrisks,increasedchancesofmiscalculation,anduncertainefficacy.49DuetotheincreasingcomplexityofAIsystems,someauthorsworrythatconfigurationerrorswillleadtosecurityrisks.50OtherauthorslamentthatAIsystems’Achilles’heelistheirvulnerabilitytospecializedattacksontheirsoftwareorhardware.51GivenAI’srelianceonhigh-qualitytrainingdata,expertsnotethatneuralnetworkstrainedoninsufficientdatastruggletogeneralizetheirapplicationsandcontinuetobevulnerabletodisruptionssuchasadversarialsamples,orinputsdesigneThescholarsalsocontendthatcurrentAI-enabledmilitarysystemsarepronetoalgorithmicbias,whichcouldleadtomisjudgments,falsealarms,andothererroneousoutputsthatcouldthreatenstrategicstability.53Giventhesechallenges,aswellasthoserelatedtoguaranteeingsafety,atleastoneexpertexplicitlynotestheimportanceofexercisingcautioninapplyingmilitaryAI.54Table1:ConcernsAbouttheTrustworthinessofCurrentAI-EnabledSystems1.ExplainabilityisoneofthemainbottlenecksrestrictingthedevelopmentofAItechnology.562.Machinelearning-drivenAIalgorithmshaveweakexplainability.573.The“black-boxproblem”ofmachinelearninglimitsexplainability.584.Autonomousweaponssystemsdependonfuzzylogic,neuralnetworks,anddeeplearning,allofwhichreduceexplainability.595.AIalgorithmshavepoorexplainabilityandposerisksshouldtheybeusedinnuclearcommandanwarningsystems.606.Thelackofexplainabilitynegativelyimpactstrustworthiness.617.ExplainabilitychallengescreatehiddendangersforAIapplication.628.Currentalgorithmexplainabilityisinsufficientand,alongwithtrustworthiness,willbecomethefocusoffutureAIdevelopment.631.DeployingmilitaryAIrequiresrestrictions,duetoconcernsaboutthetechnology’scontrolla