數(shù)字游戲中人工智能的未來研究途徑：一份探索性報(bào)告

FutureResearchAvenuesforArtificialIntelligenceinDigitalGaming:AnExploratoryReport

MarkusDablander*

arXiv:2412.14085v1[cs.LG]18Dec2024

CommissionedbyBeamFoundationtasIndependentContractArtificialIntelligenceResearcher

beam\

Abstract

Videogamesareanaturalandsynergisticapplicationdomainforartificialintelli-gence(AI)systems,offeringboththepotentialtoenhanceplayerexperienceandimmersion,aswellasprovidingvaluablebenchmarksandvirtualenvironmentstoadvanceAItechnologiesingeneral.Thisreportpresentsahigh-leveloverviewoffivepromisingresearchpathwaysforapplyingstate-of-the-artAImethods,partic-ularlydeeplearning,todigitalgamingwithinthecontextofthecurrentresearchlandscape.Theobjectiveofthisworkistooutlineacurated,non-exhaustivelistofencouragingresearchdirectionsattheintersectionofAIandvideogamesthatmayservetoinspiremorerigorousandcomprehensiveresearcheffortsinthefuture.Wediscuss(i)investigatinglargelanguagemodelsascoreenginesforgameagentmodelling,(ii)usingneuralcellularautomataforproceduralgamecontentgeneration,(iii)acceleratingcomputationallyexpensivein-gamesimulationsviadeepsurrogatemodelling,(iv)leveragingself-supervisedlearningtoobtainusefulvideogamestateembeddings,and(v)traininggenerativemodelsofinteractiveworldsusingunlabelledvideodata.Wealsobrieflyaddresscurrenttechnicalchallengesassociatedwiththeintegrationofadvanceddeeplearningsystemsintovideogamedevelopment,andindicatekeyareaswherefurtherprogressislikelytobebeneficial.

*

仁GoogleScholar

|

囹LinkedIn

|

曰GitHub

t谷Homepage

|

X

2

1Introduction

Inthelastdecade,theriseofadvancedneuralnetworkarchitectureshasledtoaseriesofdramatic

breakthroughsinthefieldsofmachinelearningandartificialintelligence(AI).TheGPU-acceleratedtrainingoflarge,carefullydesigneddeeplearningmodelshasenabledresearcherstotacklepreviouslyintractablechallengesindiverseareassuchascomputervision[

1,

2,

3,

4,

5

],naturallanguageprocessing[

6,

7,

8,

9

],artificialcontentgeneration[

10,

11,

12,

13

],andcomputationalchemistry[

14,

15,

16,

17,

18]

.Oneexceptionallypromisingandnaturalapplicationareaformoderndeeplearning,whichwillbeexploredinthisreport,isdigitalgaming.

ThefocusofAIresearchongamesalreadyhasalongandimportanthistory.Inparticular,thestudyofclassicalboardgamessuchasChess,Checkers,andGohasbeenformativeandinstrumentalfortheAIfieldasawhole[

19,

20]

.Thehighlystructurednatureofmanygamesallowsfortheemergenceofgreatcomplexityandstrategicdepthfromsimplerulesthatcaneasilybeexpressedinacomputationalframework;consequently,gameshavelongbeenconsideredidealtestinggroundsforthereasoningandplanningcapabilitiesofAIagents.Asignificantmilestonewasreachedin2016,whenthefirstAIsystemachievedsuperhumanperformanceinthegameofGo[

21

],which,atthattime,representedthelastmajor,popularboardgameinwhichhumanexpertsstilloutperformedcomputers.

Particularlysincethen,digitalgameshaveincreasinglybeenrecognisedasoneofthenextgreatfrontiersofAIresearch.Inrecentyears,considerableprogresshasbeenmadetowardsdevelopingAIagentscapableofmasteringreal-timestrategyvideogames,suchasStarCraftII[

22

],andmultiplayeronlinebattlearenavideogames,suchasDota2[

23

],bothofwhichposeafargreaterchallengetoAIsystemsthanclassicalboardgames.Simultaneously,theconstructionofgeneralAImodelsthatcanlearntoplaymultiple,qualitativelydistinctarcadevideogameshasemergedasanactivefieldofresearch[

24,

25,

26

],andworkinthisareamayserveasasteppingstonetowardsthedevelopmentofmoregeneralAIsystemsinotherdomains.

Importantly,itisnotmerelythecasethatvideogameshavethepotentialtoenrichcontemporaryAIresearch;theconverseistrueaswell.TherelationshipbetweenAIresearchanddigitalgamingismutualandsynergistic[

19,

20,

27

],withvideogamesprovidingvaluablebenchmarks,test-bedsandvirtualenvironmentsfornovelAIsystems,whilenovelAIsystems,inturn,provideawealthofopportunitiesforvideogamedeveloperstoenhancetheircreativeproducts.PartlyduetotherapidprogressofrecentAItechnologies,inparticulardeeplearning,manyoftheseopportunitiesarestillunderutilisedandhaveyettobeexplored.

Thisreportaimstogiveaconcise,preliminaryoverviewofaselectionoffivepotentialresearchavenuesfortheapplicationofstate-of-the-artAItechniquestodigitalgaming.WhileouremphasiswillmainlybeonresearchdirectionswherecontemporaryAImethodscanenhancedigitalgaming,thereciprocalconnectionbetweenAIandvideogamesmakesitconceivablethatinvestigatingthesetopicscouldalsodrivenewinsightsandadvancementsinAIitself.Theobjectiveofthisexploratoryreportisnottoprovideacomprehensivesetofmatureresearchproposals,ortopresentnoveloriginalresearchfindings.Instead,thefocusisonofferingaspeculativecollectionofhigh-levelideasthatmayservetoinspiremorerigorousandfocusedresearcheffortsinthefuture.Theselectedareasarenotinanywayexhaustive,butratherrepresentacuratedandnecessarilysubjectivecollectionofideasdeemedparticularlyintriguingduringourexaminationofthecurrentresearchlandscape.

ThefoundationalbookfromYannakakisandTogelius

[20

],whichservedasoneofthemostvaluablereferencesforthiswork,outlinesthreecoreapplicationsofAItovideogaming:

?AIforgameplayingandagentmodelling,whichincludessimulatingtheroleofahumanplayer[

22,

23

],orcontrollingothergameagentsinthebroadestsense,suchasnon-playercharacters(NPC)[

28,

29

],orhiddenagentsgoverningaspectsofthegameenvironment[

30]

.

?AIforproceduralcontentgeneration

[31

],whichincludesthealgorithmiccreationofgamelevels,music,textures,art,dialogues,items,characters,oranyotherdigitalcontent.

?AIforplayermodelling

[32

],whichincludesthemodellingofhumanplayercharacteristics,suchasplayertype,predictedin-gamebehaviour,oremotionalstate,basedonmeasuredgameplayandplayerdata.

Allofthesethreeareasarereflectedintheresearchavenuesdiscussedbelow,withagreateremphasis

onthefirsttwo.

3

2LargeLanguageModelsforGameAgentModelling

Largelanguagemodels(LLMs)suchasOpenAI’sGPT-4[

33

],Google’sLlama3[

34

],andAn-thropic’sClaude3[

35

]haverecentlyrisentoenormousprominenceduetotheiradvancedcapa-bilitiestomaintainrealisticconversationalarcsandgenerateflexiblesolutionstoawiderangeoflanguage-relatedtasks.Currentstate-of-the-artLLMsarebasedalmostexclusivelyonvariantsofthetransformerarchitecture,introducedintheseminalworkofVaswanietal.

[7]

in2017.Transformernetworksrelyontheconceptofself-attention,adeeplearningmechanismdesignedtoeffectivelycapturelong-rangedependenciesandcontextualinformationinsequentialdata.ThecoretrainingprocessformanyLLMsisself-supervisedandautoregressive,meaningthattheLLMistrainedtogeneratetextbyprobabilisticallypredictingthenextword(orsubwordtoken)inatextbasedontheprecedingwords.LLMsregularlycontainbillionsoftrainableparametersandarefrequentlytrainedonvastcorporaofunlabelledtextualdatacollectedfrompubliclyavailablesourcessuchasbooksandwebsites[

36]

.

Atthemoment,LLMsareattractingsignificantattentionwithinthevideogameAIresearchcommu-nityfortheirpotentialapplicabilitytoadiversearrayofgaming-relatedtasks[

37,

38]

.Forexample,LLMshaverecentlybeenexploredforthealgorithmiccreationofnewvideogamelevelsinSuperMarioBros

[39

],theautonomousplayingofMinecraftthroughthegenerationofcodeforasuitablegameAPI[

40

],thesystematicextractionofplayersentimentfromwrittengamereviews[

41

],andtheautomaticgenerationofdynamicaudiocommentaryforLeagueofLegendsgameplay[

42]

.CoveringallpromisingusecasesofLLMsindigitalgamingwouldbebeyondthescopeofthisexploratoryreport.However,webrieflyhighlightonepossibleresearchdirectionweconsidertobeparticularlyinteresting,namelytheuseofLLMsforgameagentmodelling.

GameagentmodellingincludesthedevelopmentandcontrolofNPCssuchasteammates,enemies,sidekickcompanions,merchants,bystanders,andothervirtualcharactersinthebroadestsense.PerhapsoneofthemostevidentandfruitfulapplicationsofLLMsinthiscontextwouldbetoequipNPCagentswiththeabilitytohavenaturalandunscriptedconversationswitheachotherandwithhumanplayers.Firstinvestigationsinthisareahavealreadybegun[

43,

44,

45

];furtheradvancementsinintegratingLLMsasNPCdialoguesystemsmaybeabletomarkedlyenhancetherealismofvirtualcharacters,leadingtosubstantiallydeeperandmoreimmersivevideogameexperiences.

However,theoverallpotentialofLLMsforagentmodellingmayexceedthealreadyappealingareaofdynamicdialoguegeneration.In2024,Huetal.

[46]

gaveaconceptualdescriptionofanentirecognitivearchitectureforgeneralgameagentsthatembedsanLLMasthecorethinkingcomponentwithinanetworkofothersubmodulescoveringperception,memory,role-playing,action,andlearning.DrawingcloselyfromtheworkofHuetal.,onemightenvisionanLLM-basedcognitivearchitecturebroadlyworkingasfollows:theperceptionmoduletranslatescurrentgamestatesintotextualdescriptions;thethinkingmodule,poweredbyanLLM,receivesoutputsfromtheperceptionmoduleandrelevanttext-basedmemoriesretrievedfromthememorymoduletooutputtextualactionplans;theseplansaretranslatedbytheactionmoduleintoexecutablelow-levelin-gameactions;theLLM-basedthinkingprocessisadditionallybiasedwithcharacterinformationbytherole-playingmodule;andcontinuouslyupdatedwithtechniquessuchasreinforcementlearningorsupervisedfinetuningbythelearningmodule.Onemayalsoconsiderintroducingaseparategoalmodulethatmanagestheobjectivesoftheagentinatext-basedmannerandinteractswiththeothermodules.

Whileeachoftheabovemodulescouldeasilywarrantitsownextensiveresearchprogramme,firstsuccessfulattemptstodesigngameagentsviatheintegrationofLLMsintobroadercognitivearchitectureshavealreadybeenmade.Mostnotably,Parketal.

[44]

createdaninteractiveartificialsocietyconsistingofavirtual2Dvillagewith25distinctLLM-basedgameagentswithdifferentpersonalitiesandprofessions.Eachagentmaintainsatext-basedmemorystreamthatcontainsacomprehensivelistoftheagent’sperceptions,alongwithgeneratedactionplansandsynthesisedhigher-orderreflections.AnLLMinteractswiththeagent’smemorystreamandcurrentperceptionstogeneratenewreflectionsandadaptactionplans.Thisapproachleadstoanimpressivelycomplexandconvincingsetofself-organisingemergentsocialbehaviours:agentsleadnaturaldialogues,coordinateactions,spreadinformation,anddynamicallyupdatesocialrelationshipmemories.Asimple,schematicoverviewofanLLM-basedcognitivearchitecture,heavilyinspiredbytheworksofParketal.

[44

]andHuetal.

[46

],isdepictedinFigure

1.

Furtherefforts,suchasthosebyParketal.,tointegrateLLMsintoabroadernetworkofcognitivemodulescouldnotonlycontributetothedevelopmentofmoreimmersivevideogamecharactersand

4

text

gameenvironmentfeatures

perceptionmodule

gamestate

executableactionsequences

actionmod

ule

thinkingmodule

(LLM)

text

text

memorymodule

text

Figure1:Simple,high-leveloverviewofaconceivableLLM-basedcognitivearchitectureforavideogameagent,stronglyinfluencedbytheworksofParketal.

[44]

andHuetal.

[46]

.Theperceptionmoduletranslatesgameenvironmentfeatures(pixels,statisticalfeatures,vectorialembeddings,etc.)extractedfromthegamestateintotextualdescriptions.Thememorymodulestorespasttextualperceptions,aswellasothermemoryitemsthatareeitherpredetermined(fixedcharacterinformation,basicgoals,etc.)orgeneratedbythethinkingmodule(novelknowledge,reflections,goals,proceduralskills,etc.).Thethinkingmodule,basedonalargelanguagemodel(LLM),processescurrenttextualperceptionsandrelevanttextualmemoryitemsretrievedfromthememorymodule,andoutputstextualactionplansandnewmemoryitems.Thetextualactionplansareconvertedbytheactionmoduleintolow-levelsequencesofin-gamebehavioursthatareexecutedtochangethegamestate.

morehuman-likevirtualagentsforplaytesting,butalsoadvanceresearchonthedisputedquestionofhowsuitableLLMstrulyareascoreenginesforartificialgeneralintelligence[

47,

48]

.

3NeuralCellularAutomataforProceduralContentGeneration

Cellularautomata(CA)[

49,

50

]areafamilyofextensivelyinvestigatedanddiversemathematicalmodelsrepresentedbygridsofcells,whosestatesevolveindiscretetime.Ateachtimepointt,eachcellhasastaterepresentedbyanumber(oravectorofnumbers),anditsstateattimet+1isdeterminedbyitsownstateandthestatesofitsneighboringcellsattimet,accordingtoalocaltransitionfunctionthatdefineshowthestatesevolve.

AsimpleandiconicexampleofCAthatmanyreadersmaybefamiliarwithisgivenbyConway’sGameofLife

[51

],whichtakesplaceonaninfinite2Dorthogonalgridofsquarecells,eachofwhichcanonlybeinoneoftwopossiblestates,deadoralive.Givensomeinitialconfiguration,cellstatesstarttoevolvebasedonasimpletransitionfunctionthatonlytakesintoaccounthowmanydeadoraliveneighboursacellhasatagiventime.Inspiteofitsextremesimplicity,Conway’sGameofLifeexhibitsanimpressivesetofcomplexself-organisingbehaviours

.1

CAhavealreadybeenusedinvideogameswithconsiderablesuccess,forinstancetogrowinfinitecavelevelsforthegameCaveCrawler

[52

],automaticallygenerateplayablemazesformazerunninggames[

53

],modelgranularmedialikesandorsoil[

54

],orsimulateerosioninvirtualenvironments[

55]

.CAarehighlycomputationallyefficientmodelsthatcanbeusedtogenerateintricatevirtualcontent.Atthesametime,CAareconceptuallysimple,intuitivetounderstandandeasytoimplement.However,theconstructive,emergentnatureofCAalsomakesthemdifficulttocontrol[

20]

.Ingeneral,givenalocaltransitionfunction,itisverydifficulttopredictwhichpatternwillariseovertimefromaspecificinitialgridstate;evenextremelysimilarinitialstatesmayquicklydivergeinachaoticmanner,leadingtoentirelydifferentoutcomes[

56]

.Similarly,identifyingalocaltransitionfunctionthatovertimemapsagiveninitialstatetoadesiredpatternisanontrivialtechnicalproblem.ThesepropertieslimittheutilityofCAasproceduralcontentgeneratorsforvideogamesbymakingitchallengingtoimposeessentialconstraintsongeneratedcontent.Suchconstraintsmay

1VideoillustrationofConway’sGameofLife

5

targetpattern

lst

glecell

si

te

binitiaa

ork)

lcaltransitinfnctingrdienttrainableneurlnetw

Figure2:Conceptualdiagramofhowaneuralcellularautomaton(NCA),oncetrained,coulditerativelygeneratethetargetimageofatree(imagenotgeneratedbyactualNCA,usedforillustrativepurposesonly).AnNCAisacellularautomatonwhoselocaltransitionfunctionisparametrisedbyaneuralnetwork.Mordvintsevetal.

[59]

showedhowanNCAcanbetrainedwithgradient-basedmethodstoorganicallygrowanarbitrary,predefinedtargetpatternfromasingleinitialcell.TheNCAcanalsolearntoautomaticallyconvergebacktoitsintendedtargetpatternwhendisturbedinamannerthatresemblesself-regeneration.

includeguaranteedsolvabilityforagamelevel,oraparticularshape,connectivityandaestheticstyleforagameobject.

Recently,neuralcellularautomata(NCA)

[57,

58

]havebeenincreasinglyinvestigatedasasignifi-cantwaytoaddresssomeoftheseshortcomingsandallowforsubstantiallygreatercontroloverthedynamicalprocessesgoverningCA.AnNCAisaCAwhoselocaltransitionfunctionisparametrisedbyatrainableneuralnetwork.OneofthekeycontributionsinthefieldofNCAwasmadebyMordv-intsevetal.

[59]

in2020,whodemonstratedhowanNCAparametrisedbyaconvolutionalneuralnetworkcanbeeffectivelytrainedinadifferentiableend-to-endmannerviagradient-basedmethodstoiterativelygenerateanypredefinedtargetimagefromasinglecell(seeFigure

2

foranillustrationofthisidea).TheymoreovershowedhowNCAcanbetrainedtoexhibitself-regeneration,or,inthelanguageofdynamicalsystemstheory,howthetargetimagecanbeturnedintoanattractor.Aself-regeneratingNCAautomaticallyconvergesbacktoitsintendedtargetpatternwhenperturbed

.2

TheseminalworkofMordvintsevetal.

[59]

hasimplicationsstretchingintodiverseareas,includ-ingmorphogenesis,embryonicdevelopment,regenerativemedicine,self-organisation,andswarmrobotics.Inaddition,firstattemptshavealreadybeenmadetoapplyNCAinvideogamere-search[

60,

61,

62,

63,

64,

65

]:Earleetal.

[60]

successfullytrainedNCAtogeneratelevelsfor2Dtile-basedgameswhiletakingintoaccountvalidityanddiversityconstraints;Sudhakaranetal.

[63]

usedNCAinthevirtualworldofMinecraftforthetargetedmorphogeneticgrowthofcomplex3Dobjectssuchascastlesandtrees;andPajouheshgaretal.

[64]

employedNCAforthevirtualsynthesisofdesiredtextureson3Dmeshes.

TheseearlystudieshighlightthepotentialofNCAasanoveldeep-learning-basedtoolforproceduralcontentgenerationinvirtualenvironments[

66]

.However,manypromisingresearchdirectionsremaintobeinvestigated.AdditionalworkcouldfurtherexplorethecapabilitiesofNCAtobetrainedviacustomlossfunctionsdesignedtopromotespecificdesignconstraintsforvideogamecontentgeneration.FuturestudiescouldalsomoredeeplyinvestigateNCAforthecreationofrealistictexturesfordigitalobjectsinacomputationallyefficientmanner,orforaccuratelysimulatingorganicandregenerativeprocesses,suchasthegrowthofnaturalecosystems,agingcharacters,materialdegradation,orwoundhealing.Beyondcontentgeneration,itmayalsobeinterestingtoinvestigate

NCAasefficientlytrainableswarmintelligencemodelstoinduceemergentbehavioursingroupsof

locallyconnectedNPCs.

2Interactiveanimationsofself-regeneratingNCAbyMordvintsevetal.[]

59

6

4DeepSurrogateModellingtoAccelerateComputationallyExpensive

In-GameSimulations

useftosimulatedataset

Rn

Intheirseminalstudy,Gilmeretal.

[14

]notonlyintroducedmessage-passingasaunifyingframeworkforgraphneuralnetworkarchitectures;inadditiontothissalientcontribution,theyalsoshowedthatagraphneuralnetworkcanlearntoefficientlypredictquantum-chemicalpropertiesofsmallmoleculesusingtheQM9dataset[

67

]fortraining.TheQM9datasetconsistsofaround134kchemicalcompounds;eachcompoundcomeswithasetofnumericallabelsthatrepresentapproximationsofrelevantquantum-chemicalproperties.Foreachcompound,eachnumericallabelistheresultofaquantum-mechanicalsimulationbasedonwhatisknownasdensityfunctionaltheory

[68]

.Densityfunctionaltheorysimulations,whilehighlyusefulforelucidatingtheelectronicstructureofamolecule,areassociatedwithprohibitivecomputationalcosts;forexample,generatingallthelabelsintheQM9datasetforasinglemoleculewithnineheavyatomsonasinglecoreofaXeonE5-2660processorwith2.2GHzandcommonlyusedsoftwaremaytakearoundonehour[

14]

.Incontrast,thegraphneuralnetworkfromGilmeretal.,whichwastrainedontheQM9datasetinasupervisedmanner,canestimatetheoutcomeofdensityfunctionaltheorysimulationsforanovelmoleculeinafractionofasecond.Thiscorrespondstoaspeed-upoffiveordersofmagnitude,makingitcomputationallyfeasibletorapidlypredictquantum-chemicalpropertiesforlargemolecularlibraries.

comnputationallycostlyf(slowtoevaluate

unctionf

)

deepnetwork(fasttoevalulate)

training

usetotraino

a{(r1,f(acr1)),…,(arm,f(arm))}SXRn

Figure3:Illustrationoftheelementaryideabehinddeepsurrogatemodelling:AcomputationallyexpensivefunctionfisrepeatedlyevaluatedtogenerateatrainingdatasetD,whichisthenusedtotrainadeepnetworkΦθ.Aftertraining,Φθactsasacomputationallyfastapproximationoff.

TheworkofGilmeretal.representsaprimeexampleofwhatcanbereferredtoasdeepsurrogatemodelling

[69,

70,

71,

72]

.Ahigh-levelillustrationofthekeyideabehinddeepsurrogatemodellingisgiveninFigure

3.

Inoneofitsmostelementaryforms,deepsurrogatemodellingisatechniqueusedtospeeduptheevaluationofacomputationallyexpensivefunction

f:X→Rn

thatisofinterestforapracticalapplication.Forinstance,fcouldrepresentanexpensivenumericalcomputersimulation.InourearlierexamplefromGilmeretal.,thedomainXwouldbeasetofmoleculargraphs,andfwouldrepresentadensityfunctionaltheorysimulationthatmapsmoleculargraphstonumericalquantum-chemicalpropertiesexpressedasvectorsinRn.Initially,a(sometimesconsiderable)computationaleffortismadetocreateadataset

D={(x1,f(x1)),...,(xm,f(xm))}?X×Rn,

whichisthenusedtotraintheparametersθofasuitabledeeplearningarchitectureΦθ:X→Rn

7

inasupervisedmanner.Aftertraining,thedeepnetworkΦθcanbeusedasasurrogateforf,approximatingthevalueoff(兒)withΦθ(兒)fornovel兒outsidethetrainingsetD.Furthermore,Φθcanalsobeoptimisedinsteadoffwhenlookingformaximisersorminimisersoff.WhileΦθmaybelessaccuratethantheoriginalsimulationfunctionf,itcanbeordersofmagnitudefastertoevaluate.

Deepsurrogatemodellingisparticularlyusefulinsituationsrequiringcomputationallyexpensiveandrepetitivesimulations[

70]

.Videogamesregularlyinvolveaplethoraofsuchsimulations,spanningdiverseareaslikegameplaybalancing,difficultytuning,fluidandparticledynamics,Newtonianmechanics,pathfinding,environmentalsystems,realisticlighting,soundpropagation,gamestateprediction,andproceduralcontentgeneration.Assuch,digitalgamingmaybewell-suitedfortheapplicationofdeepsurrogatemodelstoaccelerategameplay,reduceloadingtimesandoptimisethegamedevelopmentprocess.Despitetheseencouragingpossibilities,thenumberofstudiesexploringdeepsurrogatemodelsforvideogamesappearstoberelativelylimited[

73,

74,

75,

76,

77,

78,

79]

.However,earlyworkinthisfieldhasalreadyshownsomesuccess.Forexample,Bhattetal.

[74]

trainedadeepsurrogatemodelonsimulateddatatopredictthebehaviourofagameagentinnovelenvironments,applyingthemodeltoacceleratethealgorithmicgenerationofnewenvironmentsthatleadtodiverseagentbehaviours.Overall,theutilityofdeepsurrogatemodellingfordigitalgamingmaystillbeunderexplored,offeringnotableopportunitiesforfutureresearch.

5Self-SupervisedVideoGameStateRepresentationLearning

BeingabletorepresenttheabstractstateofavideogameintermsofameaningfulnumericalvectorisakeyelementinalargevarietyofmodernAIapplicationsfordigitalgaming[

20]

.Inthiscontext,ahigh-qualityvectorialrepresentationtechniqueshouldbeabletocondensetheessentialfeaturesofavideogamestateintoaninformativeembeddingthatcanbeeffectivelyusedfordownstreamAItasks.Suchtasksmay,forexample,includeusingagamestateembeddingasamodelofperceptionforanautonomousgameagent[

26,

80

],predictingtheemotionalstateofaplayerfromgameplayvideostreams[

81

],predictingfuturegamestatesfromcurrentones[

82

],dynamicallyadaptinggamemusicdependingonthestateofagame[

83

],oralgorithmicallytranslatinggamestatesintonaturallanguagedescriptions[

44]

.

Apowerfuldeeplearningparadigmforvectorialdatarepresentationthathasemergedinrecentyearsisself-supervisedlearning

[84

],whichoffersacollectionofstrategiestolearnrich,general-purposeembeddingssolelyfromtheinternalstructureofunlabelledinputdata.Whilesuperviseddeeplearningisbasedontheextractionoftask-specificfeaturesfromlabelleddatasets,self-supervisedlearningdoesnotrelyondataannotationbyhumansubjects,andinsteadallowsonetofindflexiblefeaturerepresentationsinatask-agnosticmanner.Labelleddataisfrequentlyscarceandhardtoobtain;self-supervisedlearningmethodsdonotsufferfromcomparablelimits,astheycantakeadvantageofvastcorporaofunlabelleddatasets,suchascuratedlibrariesofimagesandtextextractedfromtheinternet.Representationslearnedviaself-supervisedtrainingcanbeemployedinavarietyofways,includingclustering[

85

]andanomalydetection[

86]

.Importantly,theycanalsobefine-tunedondownstreamsupervisedtasks[

87

],anapproachthatregularlyleadstosubstantialboostsinperformancecomparedtopurelysupervisedtechniques.

Whileself-supervisedlearninghasbecomeasignificantareaofresearchindomainslikenaturallanguageprocessingandcomputervision[

88

],comparableworkindigitalgamingisstillrelativelysparse.Aliteraturesearchforstudiesthatuseconceptsfromself-supervisedlearningindigitalgamingrevealedonly12instances[

80,

89,

90,

91,

92,

93,

94,

95,

96,

97,