ba01公平機(jī)器學(xué)習(xí)延遲影響

DelayedImpactofFairMachineLearning

LydiaT.Liu?

SarahDean?

MoritzHardt?

EstherRolf? MaxSimchowitz?

April10,2018

Abstract

Fairnessinmachinelearninghaspredominantlybeenstudiedinstaticclassificationsettingswithoutconcernforhowdecisionschangetheunderlyingpopulationovertime.Conventionalwisdomsuggeststhatfairnesscriteriapromotethelong-termwell-beingofthosegroupstheyaimtoprotect.

Westudyhowstaticfairnesscriteriainteractwithtemporalindicatorsofwell-being,suchaslong-termimprovement,stagnation,anddeclineinavariableofinterest.Wedemonstratethateveninaone-stepfeedbackmodel,commonfairnesscriteriaingeneraldonotpromoteimprovementovertime,andmayinfactcauseharmincaseswhereanunconstrainedobjectivewouldnot.Wecompletelycharacterizethedelayedimpactofthreestandardcriteria,contrastingtheregimesinwhichtheseexhibitqualitativelydifferentbehavior.Inaddition,wefindthatanaturalformofmeasurementerrorbroadenstheregimeinwhichfairnesscriteriaperformfavorably.

Ourresultshighlighttheimportanceofmeasurementandtemporalmodelingintheevalua-tionoffairnesscriteria,suggestingarangeofnewchallengesandtrade-offs.

1

Introduction

Machinelearningcommonlyconsidersstaticobjectivesdefinedonasnapshotofthepopulationatoneinstantintime;consequentialdecisions,incontrast,reshapethepopulationovertime.Lendingpractices,forexample,canshiftthedistributionofdebtandwealthinthepopulation.Jobadvertisementsallocateopportunity.Schooladmissionsshapethelevelofeducationinacommunity.

Existingscholarshiponfairnessinautomateddecision-makingcriticizesunconstrainedmachinelearningforitspotentialtoharmhistoricallyunderrepresentedordisadvantagedgroupsinthepopulation[ExecutiveOfficeofthePresident,2016,BarocasandSelbst,2016].Consequently,avarietyoffairnesscriteriahavebeenproposedasconstraintsonstandardlearningobjectives.Eventhough,ineachcase,theseconstraintsareclearlyintendedtoprotectthedisadvantagedgroupbyanappealtointuition,arigorousargumenttothateffectisoftenlacking.

Inthiswork,weformallyexamineunderwhatcircumstancesfairnesscriteriadoindeedpromotethelong-termwell-beingofdisadvantagedgroupsmeasuredintermsofatemporalvariableofinterest.Goingbeyondthestandardclassificationsetting,weintroduceaone-stepfeedbackmodelofdecision-makingthatexposeshowdecisionschangetheunderlyingpopulationovertime.

?DepartmentofElectricalEngineeringandComputerSciences,UniversityofCalifornia,Berkeley

1

arXiv:1803.04383v2[cs.LG]7Apr2018

Ourrunningexampleisahypotheticallendingscenario.Therearetwogroupsinthepopulationwithfeaturesdescribedbyasummarystatistic,suchasacreditscore,whosedistributiondiffersbetweenthetwogroups.Thebankcanchoosethresholdsforeachgroupatwhichloansareoffered.Whilegroup-dependentthresholdsmayfacelegalchallenges[RossandYinger,2006],theyaregenerallyinevitableforsomeofthecriteriaweexamine.Theimpactofalendingdecisionhasmultiplefacets.Adefaulteventnotonlydiminishesprofitforthebank,italsoworsensthefinancialsituationoftheborrowerasreflectedinasubsequentdeclineincreditscore.Asuccessfullendingoutcomeleadstoprofitforthebankandalsotoanincreaseincreditscorefortheborrower.

Whenthinkingofoneofthetwogroupsasdisadvantaged,itmakessensetoaskwhatlendingpolicies(choicesofthresholds)leadtoanexpectedimprovementinthescoredistributionwithinthatgroup.Anunconstrainedbankwouldmaximizeprofit,choosingthresholdsthatmeetabreak-evenpointabovewhichitisprofitabletogiveoutloans.Onefrequentlyproposedfairnesscriterion,sometimescalleddemographicparity,requiresthebanktolendtobothgroupsatanequalrate.Subjecttothisrequirementthebankwouldcontinuetomaximizeprofittotheextentpossible.Anothercriterion,originallycalledequalityofopportunity,equalizesthetruepositiveratesbetweenthetwogroups,thusrequiringthebanktolendinbothgroupsatanequalrateamongindividualswhorepaytheirloan.Othercriteriaarenatural,butforclaritywerestrictourattentiontothesethree.

Dothesefairnesscriteriabenefitthedisadvantagedgroup?Whendotheyshowaclearadvantageoverunconstrainedclassification?Underwhatcircumstancesdoesprofitmaximizationworkintheinterestoftheindividual?Theseareimportantquestionsthatwebegintoaddressinthiswork.

1.1

Contributions

Weintroduceaone-stepfeedbackmodelthatallowsustoquantifythelong-termimpactofclassi-ficationondifferentgroupsinthepopulation.WerepresenteachofthetwogroupsAandBbyascoredistributionπAandπB,respectively.ThesupportofthesedistributionsisafinitesetXcor-respondingtothepossiblevaluesthatthescorecanassume.Wethinkofthescoreashighlightingonevariableofinterestinaspecificdomainsuchthathigherscorevaluescorrespondtoahigherprobabilityofapositiveoutcome.AninstitutionchoosesselectionpoliciesτA,τB:X→[0,1]thatassigntoeachvalueinXanumberrepresentingtherateofselectionforthatvalue.Inourexample,thesepoliciesspecifythelendingrateatagivencreditscorewithinagivengroup.Theinstitutionwillalwaysmaximizetheirutility(definedformallylater)subjecttoeither(a)noconstraint,or(b)equalityofselectionrates,or(c)equalityoftruepositiverates.

Weassumetheavailabilityofafunction?:X→Rsuchthat?(x)providestheexpected

changeinscoreforaselectedindividualatscorex.Thecentralquantitywestudyistheexpecteddifferenceinthemeanscoreingroupj∈{A,B}thatresultsfromaninstitutionspolicy,?μjdefinedformallyinEquation(2).Whenmodelingtheproblem,theexpectedmeandifferencecanalsoabsorbexternalfactorssuchas“reversiontothemean”solongastheyaremean-preserving.Qualitatively,wedistinguishbetweenlong-termimprovement(?μj>0),stagnation(?μj=0),anddecline(?μj<0).Ourfindingscanbesummarizedasfollows:

1.Bothfairnesscriteria(equalselectionrates,equaltruepositiverates)canleadtoallpossibleoutcomes(improvement,stagnation,anddecline)innaturalparameterregimes.WeprovideacompletecharacterizationofwheneachcriterionleadstoeachoutcomeinSection3.

2

Thereareaclassofsettingswhereequalselectionratescausedecline,whereasequaltruepositiveratesdonot(Corollary3.5),

Underamildassumption,theinstitution’soptimalunconstrainedselectionpolicycanneverleadtodecline(Proposition3.1).

Weintroducethenotionofanoutcomecurve(Figure1)whichsuccinctlydescribesthedif-ferentregimesinwhichonecriterionispreferableovertheothers.

WeperformexperimentsonFICOcreditscoredatafrom2003andshowthatundervariousmodelsofbankutilityandscorechange,theoutcomesofapplyingfairnesscriteriaareinlinewithourtheoreticalpredictions.

Wediscusshowcertaintypesofmeasurementerror(e.g.,thebankunderestimatingtherepay-mentabilityofthedisadvantagedgroup)affectourcomparison.Wefindthatmeasurementerrornarrowstheregimeinwhichfairnesscriteriacausedecline,suggestingthatmeasurementshouldbeafactorwhenmotivatingthesecriteria.

Weconsideralternativestohardfairnessconstraints.

Weevaluatetheoptimizationproblemwherefairnesscriterionisaregularizationtermintheobjective.Qualitatively,thisleadstothesamefindings.

Wediscussthepossibilityofoptimizingforgroupscoreimprovement?μjdirectlysubjecttoinstitutionutilityconstraints.Theresultingsolutionprovidesaninterestingpossiblealternativetoexistingfairnesscriteria.

2.

3.

4.

5.

Wefocusontheimpactofaselectionpolicyoverasingleepoch.Themotivationisthatthedesignerofasystemusuallyhasanunderstandingofthetimehorizonafterwhichthesystemisevaluatedandpossiblyredesigned.Formally,nothingpreventsusfromrepeatedlyapplyingourmodelandtracingchangesovermultipleepochs.Inreality,however,itisplausiblethatovergreatertimeperiods,economicbackgroundvariablesmightdominatetheeffectofselection.

Reflectingonourfindings,wearguethatcarefultemporalmodelingisnecessaryinordertoaccuratelyevaluatetheimpactofdifferentfairnesscriteriaonthepopulation.Moreover,anunder-standingofmeasurementerrorisimportantinassessingtheadvantagesoffairnesscriteriarelativetounconstrainedselection.Finally,thenuancesofourcharacterizationunderlinehowintuitionmaybeapoorguideinjudgingthelong-termimpactoffairnessconstraints.

1.2

Relatedwork

RecentworkbyHuandChen[2018]considersamodelforlong-termoutcomesandfairnessinthelabormarket.Theyproposeimposingthedemographicparityconstraintinatemporarylabormarketinordertoprovablyachieveanequitablelong-termequilibriuminthepermanentlabormarket,reminiscentofeconomicargumentsforaffirmativeaction[FosterandVohra,1992].Theequilibriumanalysisofthelabormarketdynamicsmodelallowsforspecificconclusionsrelatingfairnesscriteriatolongtermoutcomes.Ourgeneralframeworkiscomplementarytothistypeofdomainspecificapproach.

Fusteretal.[2017]considertheproblemoffairnessincreditmarketsfromadifferentperspective.Theirgoalistostudytheeffectofmachinelearningoninterestratesindifferentgroupsatanequilibrium,underastaticmodelwithoutfeedback.

3

Ensignetal.[2017]considerfeedbackloopsinpredictivepolicing,wherethepolicemoreheavilymonitorhighcrimeneighborhoods,thusfurtherincreasingthemeasurednumberofcrimesinthoseneighborhoods.Whiletheworkaddressesanimportanttemporalphenomenonusingthetheoryofurns,itisratherdifferentfromourone-stepfeedbackmodelbothconceptuallyandtechnically.

Demographicparityanditsrelatedformulationshavebeenconsideredinnumerouspapers[e.g.Caldersetal.,2009,Zafaretal.,2017].Hardtetal.[2016]introducedtheequalityofopportunityconstraintthatweconsideranddemonstratedlimitationsofabroadclassofcriteria.Kleinbergetal.[2017]andChouldechova[2016]pointoutthetensionbetween“calibrationbygroup”andequaltrue/falsepositiverates.Thesetrade-offscarryovertosomeextenttothecasewhereweonlyequalizetruepositiverates[Pleissetal.,2017].

Agrowingliteratureonfairnessinthe“bandits”settingoflearning[seeJosephetal.,2016,etsequelae]dealswithonlinedecisionmakingthatoughtnottobeconfusedwithourone-stepfeedbacksetting.Finally,therehasbeenmuchworkinthesocialsciencesonanalyzingtheeffectofaffirmativeaction[seee.g.,Keithetal.,1985,Kalevetal.,2006].

1.3

Discussion

Inthispaper,weadvocateforaviewtowardlong-termoutcomesinthediscussionof“fair”machinelearning.Wearguethatwithoutacarefulmodelofdelayedoutcomes,wecannotforeseetheimpactafairnesscriterionwouldhaveifenforcedasaconstraintonaclassificationsystem.However,ifsuchanaccurateoutcomemodelisavailable,weshowthattherearemoredirectwaystooptimizeforpositiveoutcomesthanviaexistingfairnesscriteria.Weoutlinesuchanoutcome-basedsolutioninSection4.3.Specifically,inthecreditsetting,theoutcome-basedsolutioncorrespondstogivingoutmoreloanstotheprotectedgroupinawaythatreducesprofitforthebankcomparedtounconstrainedprofitmaximization,butavoidsloaningtothosewhoareunlikelytobenefit,resultinginamaximallyimprovedgroupaveragecreditscore.Theextenttowhichsuchasolutioncouldformthebasisofsuccessfulregulationdependsontheaccuracyoftheavailableoutcomemodel.

Thisraisesthequestionifourmodelofoutcomesisrichenoughtofaithfullycapturerealisticphenomena.Byfocusingontheimpactthatselectionhasonindividualsatagivenscore,wemodeltheeffectsforthosenotselectedaszero-mean.Forexample,notgettingaloaninourmodelhasnonegativeeffectonthecreditscoreofanindividual.1Thisdoesnotmeanthatwrongfulrejection(i.e.,afalsenegative)hasnovisiblemanifestationinourmodel.Ifaclassifierhasahigherfalsenegativerateinonegroupthaninanother,weexpecttheclassifiertoincreasethedisparitybetweenthetwogroups(undernaturalassumptions).Inotherwords,inouroutcome-basedmodel,theharmofdeniedopportunitymanifestsasgrowingdisparitybetweenthegroups.Thecostofafalsenegativecouldalsobeincorporateddirectlyintotheoutcome-basedmodelbyasimplemodification(seeFootnote2).Thismaybefittinginsomeapplicationswheretheimmediateimpactofafalsenegativetotheindividualisnotzero-mean,butsignificantlyreducestheirfuturesuccessprobability.

Inessence,theformalismweproposerequiresustounderstandthetwo-variablecausalmecha-nismthattranslatesdecisionstooutcomes.Thiscanbeseenasrelaxingtherequirementscomparedwithrecentworkonavoidingdiscriminationthroughcausalreasoningthatoftenrequiredstrongerassumptions[Kusneretal.,2017,NabiandShpitser,2017,Kilbertusetal.,2017].Inparticular,theseworksrequiredknowledgeofhowsensitiveattributes(suchasgender,race,orproxiesthereof)

1Inreality,adeniedcreditinquirymaylowerone’screditscore,buttheeffectissmallcomparedtoadefaultevent.

4

causallyrelatetovariousothervariablesinthedata.Ourmodelavoidsthedelicatemodelingstepinvolvingthesensitiveattribute,andinsteadfocusesonanarguablymoretangibleeconomicmech-anism.Nonetheless,dependingontheapplication,suchanunderstandingmightnecessitategreaterdomainknowledgeandadditionalresearchintothespecificsoftheapplication.Thisisconsistentwithmuchscholarshipthatpointstothecontext-sensitivenatureoffairnessinmachinelearning.

2 ProblemSetting

WeconsidertwogroupsAandB,whichcompriseagAandgB=1?gAfractionofthetotalpopulation,andaninstitutionwhichmakesabinarydecisionforeachindividualineachgroup,

calledselection.IndividualsineachgroupareassignedscoresinX:=[C],andthescoresfor

C?1

groupj∈{A,B}aredistributedaccordingπj∈Simplex .Theinstitutionselectsapolicyτ:=(τA,τB)∈[0,1]2C,whereτj(x)correspondstotheprobabilitytheinstitutionselectsanindividualingroupjwithscorex.Oneshouldthinkofascoreasanabstractquantitywhichsummarizeshowwellanindividualissuitedtobeingselected;examplesareprovidedattheendofthissection.

Weassumethattheinstitutionisutility-maximizing,butmayimposecertainconstraintstoensurethatthepolicyτisfair,inasensedescribedinSection2.2.Weassumethatthereexistsafunctionu:C→R,suchthattheinstitution’sexpectedutilityforapolicyτisgivenby

U(τ)=P gP

j∈{A,B}j x∈X

τj(x)πj(x)u(x).

(1)

Noveltothiswork,wefocusontheeffectoftheselectionpolicyτonthegroupsAandB.Wequantifytheseoutcomesintermsofanaverageeffectthatapolicyτjhasongroupj.Formally,forafunction?(x):X→R,wedefinetheaveragechangeofthemeanscoreμjforgroupj

P

x∈X

(2)

?μj(τ):=

πj(x)τj(x)?(x).

Weremarkthatmanyofourresultsalsogothroughif?μj(τ)simplyreferstoanabstractchangeinwell-being,notnecessarilyachangeinthemeanscore.Furthermore,itispossibletomodifythedefinitionof?μj(τ)suchthatitdirectlyconsidersoutcomesofthosewhoarenotselected.2Lastly,weassumethatthesuccessofanindividualisindependentoftheirgroupgiventhescore;thatis,thescoresummarizesallrelevantinformationaboutthesuccessevent,sothereexistsafunctionρ:X→[0,1]suchthatindividualsofscorexsucceedwithprobabilityρ(x).

Wenowintroducethespecificdomainofcreditscoresasarunningexampleintherestofthepaper,afterwhichwepresenttwomoreexamplesshowingthegeneralapplicabilityofourformulationtomanydomains.

Example2.1(Creditscores).Inthesettingofloans,scoresx∈[C]representcreditscores,andthebankservesastheinstitution.Thebankchoosestograntorrefuseloanstoindividualsaccordingtoapolicyτ.Bothbankandpersonalutilitiesaregivenasfunctionsofloanrepayment,and

2Ifweconsiderfunctions?p(x):X→Rand?(nx):X→Rtorepresenttheaverageeffectofselectionand

non-selectionrespectively,then?μj(τ):=

x∈Xπj(x)(τj(x)?p(x)+(1?τj(x))?n(x)).Thismodelcorresponds

toreplacing?(x)intheoriginaloutcomedefinitionwith?p(x)??n(x),andaddingaoffset x∈Xπj(x)?n(x).

Undertheassumptionthat?p(x)??n(x)increasesinx,thismodelgivesrisetooutcomescurvesresemblingthoseinFigure1uptoverticaltranslation.Allpresentedresultsholdunchangedunderthefurtherassumptionthat

?μ(βMaxUtil)≥0.

5

thereforedependonthesuccessprobabilitiesρ(x),representingtheprobabilitythatanyindividualwithcreditscorexcanrepayaloanwithinafixedtimeframe.Theexpectedutilitytothebankisgivenbytheexpectedreturnfromaloan,whichcanbemodeledasanaffinefunctionofρ(x):

u(x)=u+ρ(x)+u?(1?ρ(x)),whereu+denotestheprofitwhenloansarerepaidandu?thelosswhentheyaredefaultedon.Individualoutcomesofbeinggrantedaloanarebasedonwhether

ornotanindividualrepaystheloan,andasimplemodelfor?(x)mayalsobeaffineinρ(x):

?(x)=c+ρ(x)+c?(1?ρ(x)),modifiedaccordinglyatboundarystates.Theconstantc+

thegainincreditscoreifloansarerepaidandc?isthescorepenaltyincaseofdefault.

Example2.2(Advertising).Asecondillustrativeexampleisgivenbythecaseofadvertisingagenciesmakingdecisionsaboutwhichgroupstotarget.Anindividualwithproductinterestscorexrespondspositivelytoanadwithprobabilityρ(x).Theadagencyexperiencesutilityu(x)relatedtoclick-throughrates,whichincreaseswithρ(x).Individualswhoseetheadbutareuninterestedmayreactnegatively(becominglessinterestedintheproduct),and?(x)encodestheinterestchange.Iftheproductisapositivegoodlikeeducationoremploymentopportunities,interestcancorrespondtowell-being.Thustheadvertisingagency’sincentivestoonlyshowadstoindividualswithextremelyhighinterestmayleavebehindgroupswhoseinterestisloweronaverage.Arelatedhistoricalexampleoccurredinadvertisementsforcomputersinthe1980s,wheremaleconsumersweretargetedoverfemaleconsumers,arguablycontributingtothecurrentgendergapincomputing.

Example2.3(CollegeAdmissions).Thescenarioofcollegeadmissionsorscholarshipallotmentscanalsobeconsideredwithinourframework.Collegesmayselectcertainapplicantsforacceptanceaccordingtoascorex,whichcouldbethoughtencodea“collegepreparedness”measure.Thestu-dentswhoareadmittedmight“succeed”(thiscouldbeinterpretedasgraduating,graduatingwithhonors,findingajobplacement,etc.)withsomeprobabilityρ(x)dependingontheirpreparedness.Thecollegemightexperienceautilityu(x)correspondingtoalumnidonations,orpositiveratingwhenastudentsucceeds;theymightalsoshowadropinratingoralossofinvestedscholarshipmoneywhenastudentisunsuccessful.Thestudent’ssuccessincollegewillaffecttheirlatersuccess,whichcouldbemodeledgenerallyby?(x).Inthisscenario,itischallengingtoensurethatasinglesummarystatisticxcapturesenoughinformationaboutastudent;itmaybemoreappropriatetoconsiderxasavectoraswellasmorecomplexformsofρ(x).

Whileavarietyofapplicationsaremodeledfaithfullywithinourframework,therearelimitationstotheaccuracywithwhichreal-lifephenomenoncanbemeasuredbystrictlybinarydecisionsandsuccessprobabilities.Suchbinaryrulesarenecessaryforthedefinitionandexecutionofexistingfairnesscriteria,(seeSec.2.2)andaswewillsee,evenmodelingthesefacetsofdecisionmakingasbinaryallowsforcomplexandinterestingbehavior.

denotes

2.1

TheOutcomeCurve

Wenowintroduceimportantoutcomeregimes,statedintermsofthechangeinaveragegroupscore.Apolicy(τA,τB)issaidtocauseactiveharmtogroupjif?μj(τj)<0,stagnationif

?μj(τj)=0,andimprovementif?μj(τj)>0.Underourmodel,MaxUtilpoliciescanbechosen

inastandardfashionwhichappliesthesamethresholdτMaxUtilforbothgroups,andisagnostictothedistributionsπAandπB.Hence,ifwedefine

?μMaxUtil:=?μj(τMaxUtil)

(3)

j

6

OUTCOMECURVE

RelativeImprovementRelativeHarm

ActiveHarm

0

1

SelectionRate

(b)

SelectionRate

0

0

1

1

*

SelectionRate

(a)

(c)

Figure1:

Theabovefigureshowstheoutcomecurve.Thehorizontalaxisrepresentstheselection

rateforthepopulation;theverticalaxisrepresentsthemeanchangeinscore.(a)depictsthefull

spectrumofoutcomeregimes,andcolorsindicateregionsofactiveharm,relativeharm,andnoharm.In(b):agroupthathasmuchpotentialforgain,in(c):agroupthathasnopotentialforgain.

?μMaxUtil,

andrelativeim-

wesaythatapolicycausesrelativeharmtogroupjif

?μ(τj)

<

j

j

provementif?μj(τj)>?μMaxUtil.Inparticular,wefocusontheseoutcomesforadisadvantaged

j

group,andconsiderwhetherimposingafairnessconstraintimprovestheiroutcomesrelativetothe

MaxUtilstrategy.Fromthispointforward,wetakeAtobedisadvantagedorprotePctedgroup.

Figure1displaystheimportantoutcomeregimesintermsofselectionratesβj:= x∈Xπj(x)τj(x).

Thissuccinctcharacterizationispossiblewhenconsideringdecisionrulesbasedon(possiblyran-domized)scorethresholding,inwhichallindividualswithscoresaboveathresholdareselected.InSection5,wejustifytherestrictiontosuchthresholdpoliciesbyshowingitpreservesoptimality.InSection5.1,weshowthattheoutcomecurveisconcave,thusimplyingthatittakestheshapedepictedinFigure1.Toexplicitlyconnectselectionratestodecisionpolicies,wedefinetheratefunctionrπ(τj)whichreturnstheproportionofgroupjselectedbythepolicy.Weshowthatthisfunctionisinvertibleforasuitableclassofthresholdpolicies,andinfacttheoutcomecurveis

?1

preciselythegraphofthemapfromselectionratetooutcomeβ7→?μ(r (β)).Next,wedefine

A

A

thevaluesofβthatmarkboundariesoftheoutcomeregions.

Definition2.1(Selectionratesofinterest).GiventheprotectedgroupA,thefollowingselectionratesareofinterestindistinguishingbetweenqualitativelydifferentclassesofoutcomes(Figure1).WedefineβMaxUtilastheselectionrateforAunderMaxUtil;β0astheharmthreshold,such

that?μA(r?1(β0))=0;β?astheselectionratesuchthat?μismaximized;βastheoutcome-

πA

A

complementoftheMaxUtilselectionrate,?μr?1(β))=?μ(r?1(βMaxUtil))withβ>

βMaxUtil.

AAπ

A

πA

7

2.2

DecisionRulesandFairnessCriteria

Wewillconsiderpoliciesthatmaximizetheinstitution’stotalexpectedutility,potentiallysubjecttoaconstraint:τ∈C∈[0,1]2Cwhichenforcessomenotionof“fairness”.Formally,theinstitutionselectsτ?∈argmaxU(τ)s.t.τ∈C.Weconsiderthethreefollowingconstraints:

Definition2.2(Fairnesscriteria).Themaximumutility(MaxUtil)policycorrespondstothenull-constraintC=[0,1]2C,sothattheinstitutionisfreetofocussolelyonutility.Thedemographicparity(DemParity)policyresPultsinequalselecPtionratesbetweenbothgroups.Formally,the

constraintisC= (τA,τB):

x∈XπA(x)τA=

x∈XπB(x)τB

.Theequalopportunity(EqOpt)

policyresultsPinequaltruepositiverates(TPR)betweenbothgroup,whereTPRisdefinedas

x∈Xπj(x)ρ(x)τ(x).

TPRj(τ):=

EqOptensuresthattheconditionalprobabilityofselectiongiven

πj(x)ρ(x)

x∈X

thattheindividualwillbesuccessfulisindependentofthepopulation,formallyenforcedbytheconstraintC={(τA,τB):TPRA(τA)=TPRB(τB)}.

Justastheexpectedoutcome?μcanbeexpressedintermsofselectionrateforthreshold

policies,socanthetotalutilityU.Intheunconstrainedcause,UvariesindependentlyovertheselectionratesforgroupAandB;however,inthepresenceoffairnessconstraintstheselectionrate

foronegroupdeterminestheallowableselectionratefortheother.TheselectionratesmustbeequalforDemParity,butforEqOptwecandefineatransferfunction,G(A→B),whichforeveryloanrateβingroupAgivestheloanrateingroupBthathasthesametruepositiverate.Therefore,whenconsideringthresholdpolicies,decisionrulesamounttomaximizingfunctionsofsingleparameters.ThisideaisexpressedinFigure2,andunderpinstheresultstofollow.

3

Results

Inordertoclearlycharacterizetheoutcomeofapplyingfairnessconstraints,wemakethefollowingassumption.

Assumption1(Institutionutilities).Theinstitution’sindividualutilityfunctionismorestringent

thantheexpectedscorechanges,u(x)>0=??(x)>0.(Forthelinearformpresentedin

Example2.1,u?< isnecessaryandsufficient.)

c?

u+ c+

Thissimplifyingassumptionquantifiestheintuitivenotionthatinstitutionstakeagreaterriskbyacceptingthantheindividualdoesbyapplying.Forexample,inthecreditsetting,abanklosestheamountloanedinthecaseofadefault,butmakesonlyinterestincaseofapayback.Using

Assumption1,wecanrestrictthepositionofMaxUtilontheoutcomecurveinthefollowingsense.

0≤?μMaxUtil≤

Proposition3.1(MaxUtildoesnotcauseactiveharm).UnderAssumption1,

?μ?.

WedirectthereadertoAppendixCfortheproofoftheaboveproposition,andallsubsequentresultspresentedinthissection.TheresultsarecorollariestotheoremspresentedinSection6.

3.1

ProspectsandPitfallsofFairnessCriteria

Webeginbycharacterizinggeneralsettingsunderwhichfairnesscriteriaacttoimproveoutcomes

overunconstrainedMaxUtilstrategies.Forthisresult,wewillassumethatgroupAisdisadvantaged

8

1

0

MUDPEO

1

0

SelectionRate

Figure2:Bothoutcomes?μandinstitutionutilitiesUcanbeplottedasafunctionofselectionrateforonegroup.Themaximaoftheutilitycurvesdeterminetheselectionratesresultingfrom

variousdecisionrules.

inthesensethattheMaxUtilacceptancerateforBislargecomparedtorelevantacceptanceratesforA.

Corollary3.2(FairnessCriteriacancauseRela