人工智能項目失敗的根本原因及其成功的方法

ResearchReport

JAMESRYSEFF,BRANDONDEBRUHL,SYDNEJ.NEWBERRY

TheRootCausesofFailure

forArtificialIntelligence

ProjectsandHowThey

CanSucceed

AvoidingtheAnti-PatternsofAI

rtificialintelligence(AI)iswidelyrecognizedastechnologywiththepotentialtohavea

transformativeeffectonorganizations.1AlthoughAIwasoncereservedforadvancedtech-

nologycompanieswiththeabilitytohiretoptalentandspendmillionsofdollars,alltypes

A

oforganizationsareadoptingAItoday.Private-sectorinvestmentinAIincreased18-foldfrom2013to2022,2andonesurveyfoundthat58percentofmidsizecorporations3haddeployedatleastoneAImodeltoproduction.4Similarly,theU.S.DepartmentofDefense(DoD)isspending$1.8billioneachyearonmilitaryapplicationsforAI,andDoDleadershaveidentifiedAIasoneofthemostcrucialtechnologiestothefutureofwarfare.5

AIisalreadymakingimpactsacrossawidevarietyofindustries.Pharmaceuticalcompaniesareusingittoacceleratethepaceandsuccessrateofdrugdevelopment.6Retailers,suchasWalmart,aredeployingAIforpredictiveanalyticssothattheyknowwhentorestockinventoryandhowtooptimizetheirend-to-endsupplychains.7Finally,inthedefenserealm,AIispilotingfighterjets,8detecting

enemysubmarines,9andimprovingcommanders’awarenessofthebattlefield.10Theseexamplesdem-onstratetherelevanceofAItoorganizationsinavarietyofindustriesandforavarietyofusecases.

However,despitethepromiseandhypearoundAI,manyorganizationsarestrugglingto

deliverworkingAIapplications.Onesurveyfoundthatonly14percentoforganizationsrespondedthattheywerefullyreadytoadoptAI,eventhough84percentofbusinessleadersreportedthat

theybelievethatAIwillhaveasignificantimpactontheirbusiness.11Managersanddirectorsfindthemselvesunderenormouspressuretodosomething—anything—withAItodemonstratetotheirsuperiorsthattheyarekeepingupwiththerapidadvanceoftechnology.12Buttoomanymanagershavelittleunderstandingofhowtotranslatethisdesireintoaction.Bysomeestimates,morethan80percentofAIprojectsfail.13Thisistwicethealready-highrateoffailureincorporateinformationtechnology(IT)projectsthatdonotinvolveAI.14

SUMMARY

2

Background

Althoughleaderswidelyrecognizetheimportanceofartificialintelligence(AI),successfullyimplementingAI

projectsremainsaseriouschallenge.aAccordingtoonesurvey,84percentofbusinessleadersrespondedthattheybelievethatAIwillhaveasignificantimpactontheirbusiness,and97percentofbusinessleadersreportedthattheurgencytodeployAI-poweredtechnologieshasincreased.bDespitethis,thesamesurveyfoundthat

only14percentoforganizationsrespondedthattheywerefullyreadytointegrateAIintotheirbusinesses.

Bysomeestimates,morethan80percentofAIprojectsfail—twicetherateoffailureforinformationtechnol-

ogyprojectsthatdonotinvolveAI.cThus,understandinghowtotranslateAI’senormouspotentialintoconcreteresultsremainsanurgentchallenge.Inthisreport,wedocumentlessonslearnedfromthosewhohavealreadyappliedAI/MLsothatU.S.DepartmentofDefenseleadershipandotherscanavoidthesefailuresormitigate

risksintheirplanning.

Approach

ToinvestigatewhyAIprojectsfail,weinterviewed65experienceddatascientistsandengineers.ParticipantshadatleastfiveyearsofexperiencebuildingAI/MLmodelsinindustryoracademia.Weselectedparticipantsacrossavarietyofcompanysizesandindustriestoensurethatthesefindingswouldbebroadlyrepresentative.Theoutputoftheseinterviewsissummarizedinthisanalysis.

Takeaways

OurinterviewshighlightedfiveleadingrootcausesofthefailureofAIprojects.First,industrystakeholdersoftenmisunderstand—ormiscommunicate—whatproblemneedstobesolvedusingAI.Toooften,trainedAImodelsaredeployedthathavebeenoptimizedforthewrongmetricsordonotfitintotheoverallbusinessworkflowandcontext.Second,manyAIprojectsfailbecausetheorganizationlacksthenecessarydatatoadequatelytrain

aneffectiveAImodel.Third,insomecases,AIprojectsfailbecausetheorganizationfocusesmoreonusingthelatestandgreatesttechnologythanonsolvingrealproblemsforitsintendedusers.Fourth,organizationsmightnothaveadequateinfrastructuretomanagetheirdataanddeploycompletedAImodels,whichincreasesthe

likelihoodofprojectfailure.Finally,insomecases,AIprojectsfailbecausethetechnologyisappliedtoprob-lemsthataretoodifficultforAItosolve.AIisnotamagicwandthatcanmakeanychallengingproblemdisap-pear;insomecases,eventhemostadvancedAImodelscannotautomateawayadifficulttask.

IndustryRecommendations

Toovercometheseissues,leadersshouldconsiderthesefiveprinciplesforsuccessinAIprojects:

?Ensurethattechnicalstaffunderstandtheprojectpurposeanddomaincontext:Misunderstandingsand

miscommunicationsabouttheintentandpurposeoftheprojectarethemostcommonreasonsforAIproj-ectfailure.EnsuringeffectiveinteractionsbetweenthetechnologistsandthebusinessexpertscanbethedifferencebetweensuccessandfailureforanAIproject.

?Chooseenduringproblems:AIprojectsrequiretimeandpatiencetocomplete.BeforetheybeginanyAIproject,leadersshouldbepreparedtocommiteachproductteamtosolvingaspecificproblemforat

leastayear.IfanAIprojectisnotworthsuchalong-termcommitment,itmostlikelyisnotworthcommit-tingtoatall.

?Focusontheproblem,notthetechnology:Successfulprojectsarelaser-focusedontheproblemtobesolved,notthetechnologyusedtosolveit.ChasingthelatestandgreatestadvancesinAIfortheirownsakeisoneofthemostfrequentpathwaystofailure.

3

?Investininfrastructure:Up-frontinvestmentsininfrastructuretosupportdatagovernanceandmodel

deploymentcansubstantiallyreducethetimerequiredtocompleteAIprojectsandcanincreasethevolumeofhigh-qualitydataavailabletotraineffectiveAImodels.

?UnderstandAI’slimitations:DespiteallthehypearoundAIasatechnology,AIstillhastechnicallimitationsthatcannotalwaysbeovercome.WhenconsideringapotentialAIproject,leadersneedtoincludetechnicalexpertstoassesstheproject’sfeasibility.

AcademiaRecommendations

Toovercometheissuesdescribedbyouracademicinterviewees,leadersshouldconsiderthesetworecommendations:

?Overcomedata-collectionbarriersthroughpartnershipswithgovernment:Partnershipsbetween

academiaandgovernmentagenciescouldgiveresearchersaccesstodataoftheprovenanceneededforacademicresearch.ThefederalgovernmentshouldexpanditsinvestmentinsuchprogramsasD(theU.S.government’sopendatasite)andseektoincreasethenumberofdatasetsavailableforresearch.

?Expanddoctoralprogramsindatascienceforpractitioners:Neweracademicsoftenfeelpressuretofocusonresearchthatleadstocareersuccessasopposedtoresearchthathasthemostpotentialtosolveimportantproblems.Computerscienceanddatascienceprogramleadersshouldlearnfromdisciplines,

suchasinternationalrelations,inwhichpractitionerdoctoralprogramsoftenexistsidebysideateventhetop-rankeduniversitiestoprovidepathwaysforthemost-advancedresearcherstoapplytheirfindingstourgentproblems.

aForthisproject,wefocusedonthemachinelearning(ML)branchofAIbecausethatisthetechnologyunderpinningmostbusinessapplicationsofAItoday.ThisincludesAImodelstrainedusingsupervisedlearning,unsupervisedlearning,or

reinforcementlearningapproachesandlargelanguagemodels(LLMs).ProjectsthatsimplyusedpretrainedLLMs(some-timesknownaspromptengineering)werenotincludedinthescopeofthiswork.

bCiscoAIReadinessIndex.

cKahn,“WantYourCompany’sAIProjecttoSucceed?”

Thepurposeofthisexploratoryanalysisistohelpleadersandmanagerswithinalltypesoforga-nizationswhoarestrugglingtounderstandhow

toexecuteAIprojectsintheirorganizationavoid

someofthemostcommonreasonsforAIproject

failures.Todoso,weinterviewed65experiencedAIengineersandresearchersacrossavarietyofcom-paniesandindustries,aswellasacademia.From

theseinterviews,weidentifiedthemostfrequentlyreportedanti-patternsofAI—commonresponsestorecurringproblemsthataretypicallyineffectiveorevencounterproductive.15Wehopetohelporga-nizationsavoidmakingthesecommonmistakes

andtoprovideleadersandmanagersendeavoringtounderstandAIwithpracticaladvicetohelpthemgetstarted.

AIprojectshavetwocomponents:thetechnologyasaplatform(i.e.,thedevelopment,use,anddeploy-mentofAItocompletesomesetofbusinesstasks)andtheorganizationoftheproject(i.e.,theprocess,struc-

ture,andplaceintheoverallorganization).ThesetwoelementsenableorganizationsandAItoolstowork

togethertosolvepressingbusinessproblems.16

IT-typeprojectscanfailformanyreasonsnot

relatedtothetechnologyitself.Forexample,projectscanfailbecauseofprocessfailures(i.e.,flawsinthewaytheprojectisexecuted),interactionfailures(i.e.,problemswithhowhumansinteractwiththetech-nology),orexpectationfailures(i.e.,amisalignmentintheanticipatedvalueoftheproject).17Breakdownsinanycomponentcouldresultinaprojectfailure,

whichresultsinincreasedcostsforthesponsoring

enterprise.ThereisalargebodyofliteratureonhowITprojectsfail.However,AIseemstohavedifferentprojectcharacteristics,suchascostlylaborandcapi-talrequirementsandhighalgorithmcomplexity,thatmakethemunlikeatraditionalinformationsystem.18

Thehigh-profilenatureofAImayincreasethedesireforstakeholderstobetterunderstandwhatdrivestheriskofITprojectsrelatedtoAI.

4

Mostpriorworkonthistopichastakenoneoftwoforms.Insomecases,anindividualdatascien-tistormanagerdiscussestheirpersonalexperiencesandbeliefsaboutwhatcausesAIprojectstofail.19Inothercases,consultingfirmsconductawidespreadsurveyofITleaderstodiscusstheirexperiences

withAI.20Forexample,McKinseyhasconducted

anannualsurveyaboutAIforseveralyears.21Addi-tionally,onestudyconductedasystematicliteraturereviewandinterviewswithsixexpertstoexplorethefactorsthatmightcausegeneralAIprojectstofail.22

Ourstudydiffersfromthispriorworkinseveralways.First,wefocusontheperspectiveoftheindi-

vidualsbuildingAIapplicationsasopposedtothe

businessleadersoftheorganization.Abottom-up

approachallowsustodiscusswhyAIprojectsfail

fromthepointofviewofthepeoplewhointimatelyunderstandthespecificsofthetechnology.Second,weconductedsemistructuredinterviewsasopposedtorelyingonmultiple-choiceorshort-answersurveyquestions.Althoughtheburdenofconducting

interviewsmeansthatthesamplesizeofthisstudyissmallercomparedwiththoseofmultiple-choice

surveystudies,thisapproachallowedustoexploretheissuesraisedingreaternuanceanddepth.Finally,weconductedsubstantiallymoresemistructured

interviewswithexpertscomparedwithpriorauthorswhotookthisapproach.

Methods

Togatherdataforthisreport,weconductedsemi-

structuredinterviewswithexperiencedAIpractitio-nersinbothindustryandacademia.Duringthese

interviews,wedefinedthefailureofanAIprojectasaprojectthatwasperceivedtobeafailurebytheorga-

nization.Weincludedbothtechnicalfailuresand

businessfailureswithinthisdefinition.Eachinter-

vieweewasaskedtodiscussthetypesoffailuresthattheyperceivedtobethemostfrequentorimpactful

andwhattheybelievedtherootcausesofthesefail-ureswere.Wethenidentifiedcommonrootcauses

basedontheinterviewresponses.Theinterviews

wereconductedbetweenAugustandDecember2023.

Theapproachtakeninthisreporthasstrengthsandweaknesses.Conductinginterviewswithopen-

endedquestionsofexperienceddatascientistsandMLengineersallowedustodiscoverwhatthese

professionalsbelievearethegreatestproblemsandchallengeswhenattemptingtoexecuteAIprojects.However,becausethemajorityofourinterviewees

werenonmanagerialengineersinsteadofbusinessexecutives,theresultsmaydisproportionatelyreflecttheperspectiveofindividualswhodonotholdlead-ershippositions.Thus,theresultsmaybeskewed

towardidentifyingleadershipfailures.

IndustryParticipants

WeidentifiedpotentialindustryparticipantsusingtheLinkedInRecruitertoolandLinkedInInMail

messages.Potentialparticipantshadatleastfive

yearsofAI/MLexperienceinindustryandjobtitlesthatindicatedthattheywereeitheranindividual

contributororamanagerinthedatascienceorMLengineeringtechnicaldisciplines.23Weselected

participantstorepresentavarietyofexperiences

andbackgrounds.Inparticular,weselectedpar-

ticipantsfromdifferentcompanysizes(start-ups,

largecompanies,andmedium-sizedcompanies)andindustries(technology,healthcare,finance,retail,consulting,andothers).Industryparticipantswereoffereda$100honorariumforagreeingtotakepartina45-minuteinterview.

Atotalof379potentialindustrycandidateswereidentifiedandcontacted.Ofthese,50individuals

ultimatelyparticipatedinaninterview,represent-ingmorethan50uniqueorganizations.24Fourteenindividualssentamessagedecliningtoparticipateinthestudy;theseindividualswereremovedfromthecandidatepoolandhadnofurthercontactfromthestudyteam.25Table1illustratesthepercentagesofpotentialcandidateswhoeitherparticipatedordeclinedtoparticipateinthestudy.

Industryinterviewsusedaconsistentbatteryofquestions,whichisprovidedinAppendixA.Allinterviewswereconductedwithapromiseofanonymitytoensurethatparticipantsfeltfreetospeakcandidlyabouttheirexperiences.

5

AcademiaParticipants

Weconducted15interviewsofacademicsdrawn

fromconveniencesamplesduringconferencesandfromindividualsknowntotheresearchteam.Theseinterviewsrangedacrossschooltypes(e.g.,engi-

neeringprogramsandbusinessschools)anddegreelevels(e.g.,tenure-trackresearcher,non–tenure-trackresearcher,graduatestudent,andundergraduate

orresearchassistant).Theseinterviewsusedacon-sistentbatteryofquestions,whichispresentedin

AppendixB.Ourinterviewswereconductedwith

thepromiseofanonymitytoallownon–tenure-trackacademicresearchersandnonresearcherengineerswhosupporttheresearcheffortstohaveanopportu-nitytospeakwithoutattribution.Table2illustratestheacademiccandidateresponserates.

FindingsfromIndustryInterviews

Acrossalloftheinterviewsconductedwithexperi-encedAIpractitionersfromindustry,fivedominantrootcausesemergeddescribingwhyAIprojects

fail.Overall,intervieweesexpressedthatthemostcommonrootcauseoffailurewasthebusiness

leadershipoftheorganizationmisunderstanding

howtosettheprojectonapathwaytosuccess.Ourintervieweesalsonotedthatthesetypesoffailureshadthemostimpactontheultimateoutcomeoftheprojectcomparedwiththeotherrootcausesoffail-uretheydiscussed.

Theothernotablerootcauseoffailureidentifiedbyintervieweeswaslimitationsinthequalityand

utilityofdataavailabletotraintheAImodels.Thesetworootcauseswerecitedspontaneouslybymorethanone-halfoftheintervieweesastheprimaryrea-sonsthatAIprojectsfailedorunderperformed.

Inadditiontothemostfrequentfailurepatternscited,threeotherrootcauseswerenotedbyamean-ingfulnumberofinterviewees.26First,someinter-vieweesnotedthelackofinvestmentininfrastruc-

turetoempowertheteam.Second,someintervieweesdiscussedthedifferencebetweenthetop-downfail-urescausedbyleadershipandthebottom-upfailurescausedbyindividualcontributorsonthedatascienceteam.Finally,someintervieweesdiscussedproject

TABLE1

IndustryCandidateResponseRates

Candidate

Indicators

Pool

Accepted

Declined

Numberofcandidates

379

50

14

Percentage

100

13.2

3.7

TABLE2

AcademicCandidateResponseRates

Candidate

Indicators

Pool

Accepted

Declined

Numberofcandidates

37

15

22

Percentage

100

40.5

59.5

failurescausedbyfundamentallimitationsinwhatAIcanactuallyachieve.Whilethesefailurepatternswerecitedlessfrequentlythanthetwodominantrootcauses,theyeachwerecitedbyaone-quartertoone-thirdoftheinterviewparticipants.

Leadership-DrivenFailures

Morethananyothertypeofissue,ourintervieweesnotedthatfailuresdrivenbythedecisionsandexpec-tationsoftheorganization’sbusinessleadershipwerefarandawaythemostfrequentcausesofprojectfail-ure.Eighty-fourpercentofourintervieweescitedoneormoreoftheserootcausesastheprimaryreason

thatAIprojectswouldfail.Theseleadership-drivenfailurestookseveralforms.

OptimizingfortheWrongBusinessProblem

First,alltoooften,leadershipinstructsthedatasci-enceteamtosolvethewrongproblemwithAI.This

resultsinthedatascienceteamworkinghardfor

monthstodeliveratrainedAImodelthatmakes

littleimpactonthebusinessororganization.In

manycases,thisisduetoacommunicationbreak-downbetweenthedatascienceteamandtheleadersoftheorganization.

Fewbusinessleadershaveabackgroundindatascience;consequently,theobjectivestheysetneedtobetranslatedbythetechnicalstaffintogoalsthatcan

6

beachievedbyatrainedAImodel.Infailedprojects,eitherthebusinessleadershipdoesnotmakethem-selvesavailabletodiscusswhetherthechoicesmade

bythetechnicalteamalignwiththeirintent,ortheydonotrealizethatthemetricsmeasuringthesuccessoftheAImodeldonottrulyrepresentthemetricsofsuccessforitsintendedpurpose.Forexample,busi-nessleadersmaysaythattheyneedanMLalgorithmthattellsthemthepricetosetforaproduct—but

whattheyactuallyneedisthepricethatgivesthemthegreatestprofitmargininsteadofthepricethat

sellsthemostitems.Thedatascienceteamlacksthisbusinesscontextandthereforemightmakethewrongassumptions.Thesekindsoferrorsoftenbecome

obviousonlyafterthedatascienceteamdeliversacompletedAImodelandattemptstointegrateitintoday-to-daybusinessoperations.

UsingArtificialIntelligencetoSolveSimpleProblems

Inothercases,businessleadersdemandthatthetech-nicalteamapplyMLtoaproblemthatdoesnottrulyrequireit.Noteveryproblemiscomplexenough

torequireanMLsolution:Asoneinterviewee

explained,histeamswouldsometimesbeinstructedtoapplyAItechniquestodatasetswithahandfulofdominantcharacteristicsorpatternsthatcouldhavequicklybeencapturedbyafewsimpleif-thenrules.Thismismatchcanhappenfordifferentreasons.Insomecases,leadersunderstandAIonlyasabuzz-

wordanddonotrealizethatsimplerandcheaper

solutionsareavailable.Inothercases,seniorleaderswhoarefarremovedfromtheimplementationdetailsdemandtheuseofAIbecausetheyareconfident

thattheirbusinessareamusthavecomplexproblems

Manyleadersarenot

preparedforthetime

andcostofacquiring,cleaning,andexploringtheirorganization’sdata.

thatdemandcomplexsolutions.Regardlessofthecause,whilethesetypesofprojectsmightsucceedinanarrowsense,theyfailineffectbecausetheywerenevernecessaryinthefirstplace.

OverconfidenceinArtificialIntelligence

Additionally,manyseniorleadershaveinflated

expectationsofwhatAIcanbeexpectedtoachieve.Therapidadvancementsandimpressiveachieve-

mentsofAImodelshavegeneratedawaveofhype

aboutthetechnology.PitchesfromsalespeopleandpresentationsbyAIresearchersaddtotheperceptionthatAIcaneasilyachievealmostanything.Inreality,optimizinganAImodelforanorganization’suse

casecanbemoredifficultthanthesepresentationsmakeitappear.AImodelsdevelopedbyacademicresearchersmightnotworkeffectivelyforallofthepeculiaritiesofanorganization’sbusiness.Many

businessleadersalsodonotrealizethatAIalgo-

rithmsareinherentlyprobabilistic:EveryAImodelincorporatessomedegreeofrandomnessanduncer-tainty.Businessleaderswhoexpectrepeatabilityandcertaintycanbedisappointedwhenthemodelfailstoliveuptotheirexpectations,leadingthemtolosefaithintheAIproductandinthedatascienceteam.

UnderestimatingtheTimeCommitmentNeeded

Finally,manyinterviewees(14of50)reportedfindingthatseniorleadersoftenunderestimatedtheamount

oftimethatitwouldtaketotrainanAImodelthat

waseffectiveatsolvingtheirbusinessproblems.

Evenwhenanoff-the-shelfAImodelisavailable,ithasnotbeentrainedonanorganization’sdataandthusitmaynotbeimmediatelyeffectiveinsolvingthespecificbusinessproblems.Manyleadersarenotpreparedforthetimeandcostofacquiring,clean-ing,andexploringtheirorganization’sdata.They

expectAIprojectstotakeweeksinsteadofmonths

tocomplete,andtheywonderwhythedatascienceteamcannotquicklyreplicatethefantasticachieve-mentstheyhearabouteveryday.Evenworse,in

someorganizations,seniorleadersrapidlyswitch

theirprioritieseveryfewweeksormonths.Inthesecases,projectsthatareinprogresscanbediscardedbeforetheyhavetheopportunitytodemonstratereal

7

results,orcompletedprojectscanbeignoredbecausetheynolongeraddresswhatleadershipviewsasthemostimportantprioritiesofthecompany.Evenwhentheprojectissuccessful,leadersmaydirecttheteamtomoveonprematurely.Asoneintervieweeputit,

“Often,modelsaredeliveredas50percentofwhattheycouldhavebeen.”27

Bottom-Up–DrivenFailures

Incontrasttothetop-downfailurepatternsdriven

bytheorganization’sbusinessleadership,manyinter-viewees(16of50)notedadifferenttypeoffailure

patterndrivenbythedatascientistsontheteam.

Technicalstaffoftenenjoypushingtheboundariesofthepossibleandlearningnewtoolsandtechniques.Consequently,theyoftenlookforopportunitiesto

tryoutnewlydevelopedmodelsorframeworksevenwhenolder,more-establishedtoolsmightbeabetterfitforthebusinessusecase.Individualengineersanddatascientistsalsohaveastrongincentivetobuild

uptheirexperienceusingthelatesttechnological

advancementsbecausetheseskillsarehighlydesiredinthehiringmarket.AIprojectsoftenfailwhentheyfocusonthetechnologybeingemployedinsteadoffocusingonsolvingrealproblemsfortheirintendedendusers.Whileitisimportantforanorganizationtoexperimentwithnewtechnologiesandprovideitstechnicalstaffwithopportunitiestoimprovetheir

skillsets,thisshouldbeaconsciouschoicebalancedagainsttheotherobjectivesoftheorganization.

Data-DrivenFailures

Afterleadership-drivenfailures,intervieweesidenti-fieddata-drivenfailuresasthesecondmostcommonreasonthatAIprojectsendinfailure.Thesedifficul-tiesmanifestedinanumberofways.

Manyinterviewees(30of50)discussedpersistent

issueswithdataquality.Oneintervieweenoted,80percentofAIisthedirtyworkofdataengi-neering.Youneedgoodpeopledoingthedirtywork—otherwisetheirmistakespoisonthe

algorithms.Thechallengeis,howdowecon-vincegoodpeopletodoboringwork?28

TooFewDataEngineers

Thelackofprestigeassociatedwithdataengineer-

ingactsasanadditionalbarrier:Oneinterviewee

referredtodataengineersas“theplumbersofdata

science.”29Dataengineersdothehardworkof

designingandmaintainingtheinfrastructurethat

ingests,cleans,andtransformsdataintoaformat

suitablefordatascientiststotrainmodelson.Despitethis,oftenthedatascientiststrainingtheAImodelsareseenasdoing“therealAIwork,”whiledata

engineeringislookeddownonasamenialtask.30

Thegoalformanydataengineersistogrowtheir

skillsandtransitionintotheroleofdatascientist;

consequently,someorganizationsfacehighturnoverratesinthedataengineeringgroup.Evenworse,

theseindividualstakealloftheirknowledgeabout

theorganization’sdataandinfrastructurewhentheyleave.Inorganizationsthatlackeffectivedocumen-tation,thelossofadataengineermightmeanthat

nooneknowswhichdatasetsarereliableorhowthe

meaningofadatasetmighthaveshiftedovertime.

PainstakinglyrediscoveringthatknowledgeincreasesthecostandtimerequiredtocompleteanAIproject,whichincreasesthelikelihoodthatleadershipwill

loseinterestandabandonit.

LackofSuitableData

Additionally,insomecases,organizationslacktherightkindofdatatotrainAImodels.ThisfailurepatternisparticularlycommonwhenthebusinessisapplyingAIforthefirsttimeortoanewdomain.Intervieweesnotedthatbusinessleadersoften

wouldbesurprisedtolearnthattheirorganizationlackedsufficientdatatotrainAIalgorithms.Asoneintervieweeputit,“Theythinktheyhavegreatdatabecausetheygetweeklysalesreports,buttheydon’trealizethedatatheyhavecurrentlymaynotmeetitsnewpurpose.”31Inmanycases,legacydatasetswereintendedtopreservedataforcomplianceor

loggingpurposes.Unfortunately,structuringdataforanalysiscanbequitedifferent:Itoftenrequiresconsiderablecontextaboutwhythingshappened

asopposedtosimplywhathappened.Forexample,ane-commercewebsitemighthaveloggedwhat

linksusersclickon—butnotafulllistofwhatitemsappearedonthescreenwhentheuserselectedone

8

orwhatsearchqueryledtheusertoseethatiteminthefirstplace.Thismaymeanthatdifferentfieldsneedtobepreserved,ordifferentlevelsofgranular-ityandqualitymaybenecessary.Thus,evenifanorganizationhasalargequantityofhistoricaldata,thatdatamaynotbesufficienttotrainaneffectiveAIalgorith