Algorithms, Incentives, and Democracy - Elizabeth Maggie Penn - 6/2/2023

all right so it's it's four o'clock so we will get started um it's a it's a great pleasure uh for me to introduce Maggie Penn today um to give the this year's thanks McKelvey Memorial lecture um Maggie you know got her ba at UC Berkeley um in 1999 and then she came here and got her Masters and PhD and got the PHD in 2003. I had

the honor of being honored committee she's been on the faculty Carnegie Mellon Harvard washu the University of Chicago and since 2018 she's been on the faculty at Emory now some of you were here when uh Jeff Banks and Dick mcelvey were here some of you know Jeff and dick they were really exceptional Scholars and they really made some important and Lasting contributions in in many academic

fields um but you know more importantly to me there were great colleagues there were great mentors um great Scholars and just great people they mentored and guided me for example and I'll forever be grateful to both of them for that and they've mentored and guided countless others Jonathan Katz who couldn't be here today he and I over tasked with trying to find this year's lecturer and

when we thought about who we should invite this year the the first person and the last person that we thought of the only person that we thought of was Maggie um like Jeff and and dick she's really making some important contributions in lots of fields and it's very early in her career and so you know they're going to be lots of contributions um coming she's well

known as a great colleague and a great mentor especially to many many students who I've talked to and she's also just a fantastic person she was truly a fun graduate student to have here at Caltech and has been fun as a colleague to see her move through the profession and and build into the important scholar that she's become so please join me in welcoming Maggie Maggie's

going to talk about algorithms incentives and democracy [Applause] thanks Mike and thank you for the honor of being able to present in this seminar so both Jeff and Richard were my teachers uh the first one of the first papers I published which was part of my second year paper at Caltech was on the bank's set and Richard was my my dissertation advisor and there's probably no

one that has had a bigger influence on me sort of intellectually and just in terms of the philosophy he had uh than Richard John and I dedicated our first book to him so he was a very important person to me and I'm so happy to be here to be able to to present my work to you in in their honor uh so this is recent work

with John Patty who's in the audience also a Caltech PhD and in this work we're looking at classification algorithms so specifically binary classification algorithms so these are algorithms that are increasingly ubiquitous in our lives and they're being used to make decisions that can affect people at a very profound level so some examples are screening loan applications so the binary aspect of this algorithm is the applicant

credit worthy or not or informing parole and bail decisions is this defendant at high risk of recidivism okay so there are many different applications of these types of algorithms they're being used all the time so classification algorithms don't just sort people into categories such as credit worthiness or non-creditworthiness they also change people's behavior and oftentimes these algorithms are being designed explicitly to do that so for

example we can think of a fraud prediction algorithm and the goal of this algorithm may also to be to deter fraud so to change people's fraud Behavior or a crime prediction algorithm that affects police deployment in an effort to reduce crime so sometimes this can be done deliberately fetches into these examples but oftentimes it can also be the case or I don't know often times but

it can also be the case that these algorithms change Behavior inadvertently so we can think of an algorithm that evaluates college readiness and incidentally it may also promote college readiness in a population of students so this kind of Behavioral feedback is at the heart of formal modeling in the social sciences it's something that we've been doing in economics and political science for for 50 years but

it's just recently making its way into the machine learning literature over the last decade or so so what got us started on this project was thinking about the city of Ferguson after the Michael Brown shooting so we were thinking about cities that issue fines on their citizens in a predatory sort of way in order to raise revenue for the city so as an example think of

Two Cities and I'll return to this this sort of thought experiment later in the talk so one city is issuing tickets to maximize operating Revenue the other is issuing them to maximize Public Safety and it might be the case that the residents of these cities are identical in terms of their underlying propensity to drive safely so these cities might look the same in terms of Street

infrastructure or commute times fast cars all the things that might might drive unsafe for safe driving but in equilibrium because of the way that people are being classified and because of the incentives that the the ticketing algorithm is giving people driving ticketing and Public Safety might look very different in these two cities because people respond to how they're being classified by this algorithm the algorithm wants

them to respond so for some of these reasons the use of classification many other reasons too the use of classification data has been facing a great deal of scrutiny and has been the object of democratic reform in some cities and these regulations oftentimes focus on changing the stakes of certain algorithmic classifications so an exam an example is the prohibition of credit scores in some cities to

determine housing eligibility so by prohibiting the use of this classification data it reduces the stakes of the creditworthiness classification right or we could think about the elimination of cash bail as reducing the stakes of a high release risk score so what we do in this project is is uh this is a formal a formal Model A game theoretic model we use a very simple binary classification

algorithm to consider a few questions that are kind of like fundamentally social science kinds of questions so the first is how do the objectives of the designer of an algorithm so these two cities for example how do their objectives affect the distribution of behavior in a population in equilibrium um the second and then the second part of today's talk is what happens when the rewards and

punishments to classification are democratized by the people being classified so what if the stakes of the algorithm are the product of democratic reform what do people want and how does that affect what an algorithm designer can accomplish and then finally and I think I'll hopefully I'll have time to get to this at the end of today's talk in a companion paper what we what what we're

sort of setting this problem up to do is to think about how to think about algorithmic fairness in settings where algorithms and behavior are independent so to move away from statistical Notions of algorithmic fairness and think more about welfare Notions of the consequences of these algorithms okay so I'll give a brief literature review because this is a short talk but uh there's uh a large and

very cool literature on algorithmic fairness uh part of this is sort of an impossibility result that tells us that we can't simultaneously satisfy different statistical fairness Notions and in the machine learning learning literature what were the the closest to is the literature on strategic classification and so this is a literature that uh tries to learn a classifier to maximize accuracy with the knowing that the data

are endogenous to the classifier so people might be trying to manipulate how they're classified they might be lying about some data or they might be changing their behavior in some way that the algorithm is trying to to determine okay and and then finally probably closest to us is a literature on the fairness of equilibria in algorithm design so a paper Fair prediction with endogenous Behavior has

a result that we also generate in our model that I'll talk about a little later and just a plug for a paper by me and John that I really like that came out recently that's also related to this project is this paper on Banning the box so removing felony status from job applications and so what the what the role is of that in a moral hazard

context and the fair prediction paper is in a policing context so we think that the model today can kind of Encompass a lot of these different types of contexts okay so our contributions in this project relative to some of this related literature is that we we characterize what optimal algorithms look like for a general designer so we can think about a predatory designer or an accuracy

motivated designer or a compliance maximizing designer okay and so this framework enables us to really see how the preferences of the designer shape societal outcomes we micro found individual incentives in this paper which some of those other papers do too but what this framework does is it lets us think about the relationship between statistical and Welfare based Notions of of the fairness of a classifier so

think about let's just think about things like Envy freeness and then finally we endogenize the rewards and penalties and really think about the relationship between algorithms and the types of incentives that people have in order to be classified one way or another okay so I'll get to the model this is a very simple model so we have a unit mass of people and a binary set

of behaviors that people can engage in just zero or one and this Behavior represents something that each person makes a choice about whether to do this and is going to be potentially rewarded on the basis of so they're going to be classified on the basis of this behavior that they undertake and then there's an algorithm designer who designs a classification algorithm this designer is going to

observe a noisy signal of the behavior that each person undertook and classify each though each person as a zero or a one based on the signal that they see and if the person is classified as a one that will represent that person receiving a reward and the reward could potentially be negative so it could be the case that you receive a penalty so the way that

we're thinking of an algorithm is as consisting of two parts so the first is something that's exogenous in this model and we call this just the testing Precision how noisy is the data that the designer has about each person what each person has done so this Precision is fee and it's the probability that the signal that the designer sees equals the behavior that the person engaged

in and we always assume that this is greater than a half but it might be it might be one so it might be the case that the designer can perfectly see what each person did and I'll talk about that a little later so that's exogenous to the model and endogenous to the model what the designer is doing is is choosing a classifier and this classifier has

two parts Delta one Delta naught and Delta sub signal is the probability that the decision of the designer his classification Choice equals the signal conditional on the designer having observed that signal okay so Delta naught is the probability a signal of zero is translated into a zero Delta one is the probability of one is translated into a one so just to make sure everybody is like

on the same page because we're going to use a lot of these Deltas throughout the talk uh 1 1 would mean that everybody is classified according to their signal and 0 0 would mean that everybody has classified the opposite of their signal and Delta one Delta naught of 0 1 means that everyone is classified as a zero so if I see a one I classify them

as a zero if I see a zero I classify them as a zero okay so those are just some examples but these are probabilities so people receive a reward if they're classified as a one and zero otherwise and all this reward represents is just their benefit just the difference between what they get if they're classified as a one versus a zero so the fact that it's

R zero it could be two R and R or whatever it's just the the difference between being classified as a one and a zero and the designer doesn't pay for this reward and the designer doesn't profit from it so we'll talk more about this in a bit people choose a costly behavior and we're going to call this compliance okay so they choose to comply or not

and in order to comply each person pays a cost gamma that's their own cost and we're going to assume that those costs are distribu distributed according to a log concave PDF that has full support so this full support assumption is just there so that we don't have boundary conditions on the problem but what it's going to mean is that there's always a positive proportion of compliers

and non-compliers regardless of any incentives people could have because if I have a negative cost it means that I actually want to comply and some people are going to have costs you know arbitrarily negative so there's no way to induce them to not comply and some people are going to have costs that are arbitrarily positive and no one can induce them to to comply so the

timing of the decisions people privately observe their costs the designer publicly commits to a classifier people make their compliance decisions they're classified according to this algorithm and then payoffs happen which I haven't described yet for you and I'll say please stop me at any point in the talk if you have a question about the model or yeah Rod when the designers yeah everybody knows everybody knows

what this classifier is so this could be like I'm going through the same traffic light there's a traffic camera on it I know that that's a totally accurate traffic camera and so I'm going to get classified as a zero if I speed or something yes it's a big assumption and everybody in the town so in this case we are assuming that that's not the way they

take advantage they take advantage otherwise okay so the designers payoffs uh basically can be described by this confusion Matrix so we have sort of four outcomes for any given person they engaged they complied they were assigned to one they complied they were assigned to zero and so on and the designer cares about the fraction of people that fall into each cell of the of this confusion

Matrix so A1 is the designer's payoff for a person that falls into cell one being a one be not as his payoff for these two positives and so on and so I'll give you some examples of specific payoffs that could be described in this way that are sort of natural okay so people's incentives of whether to comply or not people are going to comply if they're

expected payoff of complying is greater than their expected path of not complying so if they comply they pay this negative gamma cost but they have some probability they're assigned a one and if they don't there's some probability they're also assigned a one okay depending on the noise of the algorithm and so compliance is going to occur when people's costs are sufficiently small it's just a simple

expression so conditional on a classifier we're going to have an equilibrium fraction of people that are going to comply in this world okay and that equilibrium fraction is the the CDF evaluated you know with an argument of the right side of this inequality it's just the fraction of people whose costs are less than R times that expression okay and so some examples of these costs could

be like college readiness everybody has a private college readiness cost to becoming College ready it could be how rich my family is the quality of my schooling and so on so we have our equilibrium compliance term which we're calling Pi just the fraction of people for a given classifier that choose to comply and this road term which is R times rho is the argument of the

CDF we're terming that the expected responsiveness of the algorithm to the signal and what Ro tells us is the chance that if I see a signal of one I assign that person a one okay so when row is positive the classifier is incentivizing people to comply because it if I comply I'm more likely to get a one than if I don't comply and when row is

negative the classifier is disincentivizing people to comply because if I comply I'm more likely to be assigned a zero then I am if I don't comply okay and finally when this row term is zero classification is totally independent of this signal and we're going to call this a null classifier so when this term is zero it means that Delta 1 plus Delta naught equals one which

means that the probability I'm assigned a 1 if I send a signal of one is the probability I'm assigned a 1 if I send a signal of zero okay so there's no incentive for anybody to try to comply in order to to experience a higher chance of classification some way or another and so for any null classifier the equilibrium fraction of compliers is just F of

zero where f is our CDF of the cost distribution okay so what is this algorithm doing it's determining the distribution of outcomes in this confusion Matrix so it's incentivizing Behavior it's telling us How likely we are that a person is going to be in each of these rows but it's also classifying the behavior and optimally it's trying to do that in a way that is most

advantageous for the designer so this is the designer's problem and I just wrote it out this way to make the point that what makes the problem interesting is the fact that the classifier affects Behavior the fact that we have this Pi term in the designers expected payoff function because if Behavior were not affected by the classifier then the designer's payoff would be linear in the classifier

and the designer would always just want to choose something on the corner a zero zero or a zero one or something like that okay and so and so the the designer is choosing an algorithm choosing a Delta one Delta naught to optimally generate behavior and bend signals of that behavior into the cells of the Matrix that are the most advantageous okay so here's some examples of

cells of the Matrix that of designer payoffs that are kind of natural so this would be a designer that's maximizing accuracy so all this designer wants to do is is get the decision right is to accurately classify Behavior with this decision okay so we could think about this as being like an epistemic judgment a legal ruling of some sort this type of designer only cares about

trying to induce compliance so he doesn't care how people are classified but he cares about the number of people that choose the beta equals one activity and so this would be a designer that's interested in maximizing Public Safety deterring fraud okay this designer only cares about Behavior this could be a predatory designer so we lived in Chicago there were predatory Towers who were really interested at

kind of like what you were describing interested in getting people to do the wrong thing and like tow your car so this could be a designer that just wants people to fall into this lower cell of not complying and getting ticketed and then finally this is kind of a natural setting for a moral hazard type of problem so we could think about an employment um situation

or college admissions we have our designer they want to admit the qualified student their worst off if they admit an unqualified student and they kind of get something in between if they don't admit the student so the point of these examples is that we think that the framework can kind of capture a pretty wide variety of different types of designer preferences and capture a lot of

different problems that we think are interesting okay so I'm going to give you some partial characterizations of what optimal classifiers look like so the first is that if we drive rewards to Infinity if these rewards are sufficiently large the designer can Channel every person into any cell of the confusion Matrix so if the designer loves these false negatives loves getting people to work but not paying

them for it with a sufficiently high reward the designer can get a hundred percent of the population to fall into that cell and so this is why we're not letting the designer choose the rewards it's not an interesting problem he can always do his best by driving that reward up any optimal classifier is going to require one of those two Delta terms to be either zero

or one so we're always on the edge of our classifier space but when the other classifier is in between zero or one what that represents is an effort by the designer to stimulate behavior in the population by optimally classifying because if the designer were always classifying people optimally conditional on this signal he would always classify uh without he would always choose a classifier that's in this

set always choose something on the corner but when he chooses something in you know that's probabilistic he is committing sorry committing to optimally misclassify in order to drive Behavior that's beneficial for himself it's going to look like a violation of sequential rationality but it's not because he's committing to it before that at the start of the game if the designer wants to maximize or minimize compliance

we have a very simple classifier so the designer is always going to choose either one one or zero zero in a setting where the designer just cares about incentivizing Behavior okay finally in the general case it's always going to be the case and we have to for our results we assume a partial ordering on the cells of the Matrix that I'll talk about on the next

slide but it's always going to be the case that the problem is strictly quasi-concave in one of these deltas and quasi-convex in the other okay so what it means is that it's always going to be the case that one of these is zero or one and the other is unique and interior or vice versa and to know which World we're in all we need to do

is know the sign of the reward so just from a technical perspective this simplifies our problem a ton because it turns our problem into a one-dimensional optimization problem and so we use this a lot and that ordering on the cells of the Matrix that we need in order to get that is that A1 is bigger than a naught weekly and B1 is greater than or equal

to B naught and there's other orders we could do too but this is the natural one that we use throughout the paper so we always prefer true positives to false negatives weekly and true negatives to false positives and we think this makes sense and all of the vignettes that I presented the predatory the maximum accuracy all satisfy this condition okay so I'm going to walk you

through a few examples of optimal classifiers and then we're going to endogenize the the penalties to classification so hopefully people can see a little bit of what I've put up here so this is an example of optimal ticketing in Two Cities so suppose we have a reward of two for this example and suppose that our signal is totally accurate so we have like a traffic camera

it's completely precise the designer can totally see who's who's speeding and let's suppose that individual costs to safe driving are just distributed normal uh with a mean of zero zero one okay so we have a designer that's in an accuracy motivated City an accuracy motivated designer that has a 1-1 on the uh off diagonals and then we have a designer that's a little more Revenue motivated

so this designer gets a one for True negatives and like a 0.55 for True positives okay optimal ticketing in our city that's accuracy motivated is going to be one one obviously we have a totally accurate signal so the designer is always going to follow that signal and is going to create a totally accurate decision 100 of the time and that classifier is also going to maximize

it happens to maximize compliance so we have 98 of people are complying and driving safely but in the revenue City the optimal classifier is 1.8 and so what that means is that the designer doesn't ticket any safe drivers but also doesn't take it 20 of the Speeders okay and so obviously this shouldn't be a surprising example to anybody but it's just sort of illustrating our model

and what what we're producing so this makes speeding more desirable but because we're not ticketing all the Speeders and it increases speeding and so it increases tickets for the designer and we lower compliance so now 94 of the people drive safely the second example I wanted to present was something that was kind of thought provoking to us given the literature on algorithmic fairness and this is

an illustration of the fact that maximizing accuracy is not kind of a neutral goal even though we often think of it as a neutral goal okay so let's think about an accuracy motivated designer now let's let the reward be pretty high 10. and the signal accuracy is kind of noisy so the designer only can see what people actually did correctly 75 of the time okay and

again we have these normal costs if the classifier just followed the signal we would have almost a hundred percent compliance okay so everybody basically would choose to comply but the designer is only going to accurately classify three quarters of them right because if he follows the signal he's only as accurate as the signal is so 25 of the People are classified wrong okay but if the

designer decided to classify everybody as a one since everybody is a one compliance would disappear and we would drop down to only being 50 accurate okay so the optimal classifier in this case um if we're interested in accuracy is 1.37 so what this means is that a hundred percent of the ones are classified as ones and 63 percent of the zeros are classified as ones so

the design was classifying a lot of people as ones so this incentivizes less compliance than following the signal so now we the only only 97 percent of people choose beta equal one instead of a hundred but it's correctly classifying ninety percent of the population so it's doing a pretty good job of classifying by injecting that noise into the classifier now if this same designer faced a

different population of people now suppose this person is facing a population of people that's distributed uh with a higher higher mean okay so the mean is normal the distribution is normal one one our optimal classifier in this case is 0.92 so anybody sending a signal of one is classified as a zero and 92 percent of the zeros are classified as zeros so this optimal classifier is

going to disincentivize compliance so normally if we just had a null Cloud if people weren't being classified 16 of these people would have a negative cost and would naturally want to comply but under this under this classifier only eight percent are complying and so this is this is disincentivizing compliance but it's just as accurate as the other as the other classifier it's correctly classifying 90 of

the population okay so we have our two populations and the picture on the left is what people would naturally want to do and the picture on the right is what the classifier induces them to do in its pursuit of accuracy and so in our in our low-cost City the complier is inducing people to comply and our high cost City the comp the classifier is inducing people

to not comply and we think that this is an important kind of philosophical point about accuracy so we often think of accuracy motivations as Fair motivations neutral motivations and the algorithmic fairness literature focuses largely on error rates across group of classification but here both of our groups are being classified almost identical percentages of the time okay but the algorithm is incentivizing totally different behavior for people

and if compliance is a social good this algorithm in its pursuit of accuracy is hurting people and so what the end of this talk is we'll be on if there's if there's time for me to get to it is this a companion paper that John and I are working on which makes the point that the effective classification on equilibrium Behavior should also be an object of

our fairness considerations and not just the errors and classification so moving on to the second half of the talk thinking about what can we do about algorithms that are potentially uh manipulating behavior in a way that's painful to people so when the stakes of classification are high enough the designer can induce almost a hundred percent of people to be to comply or not comply and oftentimes

the designer is going to want to induce that kind of identical behavior in people because it makes his classification problem easier but obviously some level of aggregate compliance is beneficial to a society and a lot of different types of environments driving you know so on and so the question we're asking now is if the stakes of classification are the product of democratic reform what does optimal

classification and what does optimal compliance look like once we endogenize that reward that people face okay so previously our designer was committing to a classifier people were choosing their behavior and signals were sent okay and then classification happened payoffs are received and so our reward was exogenous now we're going to change the first step in this problem so people are going to vote on a reward

in response to the designer's choice of classifier the designer can choose the classifier in response to that reward what we're looking for is a fixed point in the reward classifier space so the timing doesn't matter what we're looking for is an optimal classifier that induces the median voter to choose an optimal reward and that reward induces that same classifier from the designer so we're looking for

a stable kind of classifier reward pair so we have to put a a restriction on what these rewards can look like because if people were like totally unconstrained and rewards then people would want rewards of infinity and everyone would comply and be able to get potentially so what we're going to assume is a budget balance condition on these rewards sort of similar to what we might

see in a predatory city where the fees that people are paying is financing the city so voters are going to receive a reward if they're classified as a one and that reward is going to be financed by a tax that's borne by each voter of the size of the reward times the average the expected number of people that are rewarded so what this means is that

if people are classified as a one they get R times 1 minus the expected number of people that are rewarded if they're classified as a zero they pay are times that expected number okay so this gives us budget balance all the people that are paying a penalty that penalty is distributed to the people that are classified as a one we're going to add one more thing

to this model that is not necessary we just thought it was interesting so we're going to assume that voters also share preferences over aggregate Behavior okay and so in every person's payoff function is an externality a term that reflects aggregate compliance so people each receive for some parameter T that we're assuming is weekly positive doesn't could be zero they're receiving T times the aggregate compliance okay

so people might might want safe driving in their City and so then we want to find what each voter's optimal reward is and the only thing I'm going to mention about this optimization problem is that the voters payoff functions are Define piecewise because they're going every voter is going to face a reward cut off below which they will not comply and above which they will comply

okay so that's why the payoff function is defined in these two chunks for each person okay so if we have a classifier that's not null then we show that conditional on the behavior a person takes voters payoffs are single peaked in rewards and maximized at an interior reward so there is a well-defined solution to this problem if we have a null classifier then payoffs are flat

in the reward everybody is being classified exactly with the same probability each way and so everybody's receiving the same payoff and so by budget balance everybody's receiving zero so what these voters are essentially voting on is whether they optimally prefer to be a complier with one ideal reward or non-complier with a different ideal reward that I will Define for you in a sec okay so what

do these optimal rewards look like the optimal rewards for people are of this form that I put up here and you'll see that the optimal rewards are defined implicitly and what they do is they set the virtual value of the sort of expected voter equal to T and so what this means is that when a voter is choosing their optimal reward they are making the same

calculation as a profit maximizing firm that's choosing a price okay so it's a very similar it's an identical kind of uh cost benefit analysis that voters are making so I want a higher price I want a higher reward but the bigger I make that reward the more people are going to engage in this behavior and so the more I'm going to have to pay them off

okay so like the higher the price the firm sets the fewer people buy the good right and that's I did we weren't expecting this actually we set the problem up but but that is the problem that voters are facing so for prospective compliers the optimal reward is this term up there for non-compliers it's are not and the only point to make about these two rewards is

that they're identical for every person okay so there are only two possible ideal points that any person could have and they're identical okay so what that means is that we have half the people have the same optimal level of reward or over half the people um and so we have a conversation winning reward there is a democratic equilibrium to the optimal reward um and the next

thing to note is that optimal Rewards are going to generate a fixed fraction of compliance so if a classifier is not null it could be a predatory person it could be a accuracy maximizing person or a compliance maximizing person for any non-null classifier compliance is independent of the classifier and equals exactly the number of people that the median voter wants to comply this F of K

star term so what this means is that by taking a vote over these rewards the voters can completely neutralize the algorithm and they neutralize the ability of the designer to affect Behavior the only two behaviors that are possible in this model are either the level of compliance that's F of K star which is exactly what the median voter wants or F of zero which we could

get if the designer is just like I'm choosing a null classifier I'm classifying everybody the same way so there's no incentives to classification okay so as I said a second ago there's a conversating condorstay winning reward it's the one that's preferred by the median voter of the cost distribution and if we assume that the distribution of costs are symmetric about mu then we have a pretty

natural cut point in whether we're going to see a high reward or a low reward whether the median is a complier or a non-complier so if the costs of the median voter are less than T where T is that parameter on the externality in people's payoffs to compliance then the median is going to prefer the high reward and to comply and if the cost of the

median voter is below T the median is going to prefer a low reward and to not comply so T is the cut point in costs at which The democratically Chosen reward is sort of determined when we have a symmetric distribution of costs and then finally social welfare on this model if we want to maximize benthamite social welfare the optimal reward is T over that responsiveness term

and this shouldn't be like a shocker to anybody that democratically chosen rewards are always inconsistent with social welfare maximization so the socially optimal reward is lower than what a compliant voter wants without wanting that high reward because he's complying but it's higher than what a non-compliant voter wants so that's socially optimal reward is in between and the reason it's and the reason why it's uh Democratic

rewards aren't social welfare maximizing is that the voters are kind of um preying on each other a little bit right so I'm choosing a reward that enables me to kind of profit off my fellow citizens that get a bad classification okay so now we'll talk about what equilibria looked like in the model uh a little bit so as I had said earlier the way that we're

defining equilibria is as a fixed point so the voter is choosing a reward that maximizes his payoff conditional on the classifier and the classifier is chosen to maximize the designer's payoff conditional on that reward so there can be zero one two or three equilibria to this model but I'll tell you when we're we have we can guarantee some cases in which we can guarantee equilibria um

so if we have multiple equilibria they're always Pareto ranked for the designer and the the median voter and it's pretty easy to calculate which one makes the best off um but there's a reason and and so if we have three equilibria for example or two equilibria so we could we can potentially have a null equilibrium where the voter just chooses a reward of zero and the

designer chooses a null classifier or we can have an equilibrium with a positive reward and a positively responsive algorithm or a negative reward and a negatively responsive algorithm okay so there's sort of a duel to the problem like I can give a high reward to compliers or I can give a negative reward to non-compliers and sometimes we can get both of those types of equilibria but

sometimes we can't and the reason we have equilibrium non-existence in the problem is one reason is easy to get around the space of rewards isn't compact so if we bound the space of rewards we would we would get around this problem but the trickier problem for us is that the designer's best response correspondence isn't convex valued so the shape of the designer's payoff function is a

saddle and he always has two local Maxima okay so when can we get equilibria so this is kind of a messy expression just because the designer's payoff has that four terms in it but we we can always guarantee that there is a null equilibrium if F of zero that's the fraction of people that would choose to comply condition just naturally conditional onward reward of zero is

not in this open interval okay and so what that means is that we need F of zero like an inaccuracy setting F of zero can't be between 1 minus Phi and Phi um but in different types of settings it's different so we need F of zero to not be too Central in certain problems so we can always guarantee a null equilibrium if either A1 equals a

naught or B1 equals B naught or both of those conditions hold so for example if if the designer's compliance maximizing or minimizing we always have a null equilibrium we always have equilibrium existence if only one cell of the confusion Matrix gives the designer a positive payoff we always have or any payoff we always have a we always have equilibrium existence and then finally and what we

think is kind of the most natural setting to think of we're always going to have equilibrium existence if there's any noise in our signal accuracy and if F of 0 is for example sufficiently small and that's what that would say is that most people pay a positive cost to compliance which is kind of a not it's very natural in a lot of these problems to assume

that everybody pays a strictly positive cost to compliance so as long as as long as most people pay us efficiently uh a sufficient a positive cost to compliance we get equilibrium existence Okay so I'm going to give you an example of what equilibria look like um in the model so I'm returning to the optimal ticketing example so let's go back to this designer there's two designers

one is accuracy motivated and one kind of prefers the true negatives to the true positives and there's one more parameter that we need to Define is which is that externality to compliance so for this case we're going to just let it equal one okay so people get an additive term in their payoff function that's the fraction of compliers and so in this particular example both of

these cities give us a unique equilibrium and it's not a null equilibrium okay so in their first city people are going to democratically choose a reward of 1.51 we're going to get 100 accuracy 93 compliance and social welfare in the median's payoffs are in the table in the revenue motivated City all the safe drivers or class don't get ticketed um 83 of the unsafe drivers do

get ticketed and the reward is higher now between the the penalty to being ticketed it's 1.82 and this gives us slightly lower accuracy but the thing to keep in mind about this example there is the thing I want to point out is that compliance welfare and the medians payoff are identical in these two cases so we have a noisier classifier here and higher Stakes to classification

but the median is indifferent between everybody every person in society is indifferent between these two classifiers okay by democratizing the reward so for any non-null classifier the median wants to set a reward to give us exactly 93 compliance that's just the the solution to that optimal reward and in order to get that level of compliance when we have some noise in the classifier we have to

work a little harder to induce people to comply so we have a higher reward there okay so oftentimes in this model the only equilibria are null equilibria so if for example we have a predatory designer and voters want compliance they enjoy the externality of other people complying for example then a null equilibrium is going to exist and it's going to be the only possible equilibrium okay

so when the median and the designer are irrevocably at odds with each other any equilibrium is going to have to set the stakes of classification to zero and so we're thinking of this as like a defund the classifier like stop using it okay and so this is a majoritarian desire to like neutralize the incentive effects of the algorithm okay and then the final point that I

will make about this paper is just it's just a simple point about democracy and efficiency so the median is always trying to induce this fixed level of compliance that's his you know profit maximizing level if we have an accuracy motivated designer for example or I don't know he could have any kind of preferences the designer might be better off at that level of compliance classifying everybody

as a zero for example okay or using a null classifier but if we set the rewards to the problem exogenously if we take the rewards away from society to induce artificially higher compliance than what the median wants we can sometimes improve outcomes kind of across the board so we can make the designer better off the medium better off and aggregate social welfare better off by using

a different reward than what the median wants because we can sometimes induce the designer to to start using that signal information in a way that's beneficial to people okay okay so for the last 10 minutes of the talk I'm going to talk uh are there any questions I feel like I thought this was a very long talk but I'm kind of a going through it pretty

quickly but okay I'll continue like the like welfare or what is c feet uh-huh yeah yeah I'm glad you asked that question like so what's the effective fee on this problem so yeah yeah so so just like the the median is indifferent over the classifier the median is actually also indifferent over fee as long as it's not a hat I mean as long as it is

slightly informative and so what the median is just going to do in that case is um jack up the reward really high because to because the media needs to incentivize people for all that noisiness that people are facing so the median is gonna so if we have a very low accuracy signal the median is just going to increase rewards in order to still get that same

93 compliance that we want so in this case p is one but we could also do this with fee of like 0.51 and we would still have the same compliance welfare and median payoff I know I should have put that on a slide I think it's kind of okay so for this last part of the talk I'll talk a little bit about recent work and John's

been working on this today so this is not as uh as finished as the other stuff but uh we're pretty excited about it so this is um thinking about classification and algorithmic fairness in these settings where Behavior responds to the classifier so the field of algorithmic fairness and I know many of you are working in this field so but I'll describe it to those of you

better job of equalizing some error metric across populations of people and what that error metric is is Up For Debate and obviously there's many different ways of thinking about errors and classification so we could have try to equalize false positive rates or um many different types of rates that we could try to equalize so the question that we've had for a long time with this with

this project and this I'll note that this is a big project for us so this is a book project there's going to be there are already a number of different pieces to it but um one of the first things that we started thinking about in it was how how we characterize algorithmic fairness when the behavior that we're trying to classify is sensitive to the algorithm and

we thought of a classic example in statistical discrimination from the 1970s okay so this is just a rovian statistical discrimination in hiring a classification algorithm is going to give an applicant a job or not attention the basis of whether that person is qualified and it's costly to become qualified for everybody suppose that this algorithm is sexist and it thinks that all women are unqualified okay then

rationally no woman is no every woman knows they're not going to get hired no woman becomes qualified and the algorithm perfectly classifies every woman as unqualified so by most measures of algorithmic fairness this classifier's treatment of women is not problematic this is a perfect classifier it's totally accurate okay and so if we're thinking about accuracy as our measure of fairness this is an example where something

seems wrong and so just to give you some preliminaries in this literature so we're going to think about two different groups of people that only differ in their members cost distributions distributions of becoming qualified or of complying so now we're going to have two cdfs F1 and F2 and these groups might also differ in their signal accuracies so it might be easier for me to see

whether people are qualified in one group than another so they might also differ in fee one and fee two so error rate balance says and this is these are well-known algorithmic fairness properties so error rate balance says conditional on Behavior we want this algorithm to be equally accurate for the two groups okay so conditional on a person being qualified this algorithm is making the same level

of number of Errors for men and women okay when we look at qualified people predictive parity says conditional on the designer's decision the algorithm is equally accurate for these two groups okay so this is like conditional when we look at the people that are hired we want men and women to be equally qualified okay so these these two Notions are conditioning on different things one is

conditioning on the rows of our confusion Matrix and one is conditioning on The Columns of it and a very well-known theorem in this literature tells us that it's impossible to simultaneously satisfy these two Notions of fairness and this was the big debate in the compass recidivism prediction tool it satisfied predictive parity so this is used to to determine um recidivism risk for defendants but it didn't

satisfy error rate balance and they found that more blacks were incorrectly classified as being high risk of recidivism than whites were and more whites were incorrectly classified as being low risk of recidivism okay so in this new project that John and I are looking at we want to think about fairness when the data or a function of the algorithm and we think that these statistical Notions

of fairness are disconnected from welfare based Notions of fairness which are so common in economics and in the social sciences and so we're asking are there connections to be drawn between these these different types of fairness criteria so who benefits or is harmed by the pursuit of fairness and how do these Notions of fairness relate to individual incentives like in our example of a rovian discrimination

and the incentives of women to get qualified for a job so we Define two uh two Notions that are a little different than what's being used in the literature so we're not the first people to think about Envy freeness in this classification context but we are thinking of it in a different way than what we've seen in other literature we're thinking of it as at the

individual level where people know their their types they know their own costs so Envy freeness would say conditional on my costs or any person's cost no one wishes to be in a different group so if my cost to qualification is X I don't wish that I was a member of some other group that had that cost tax okay and equal opportunity which we should probably call

it something else because this is also sort of an overloaded term in this literature but this says conditional on my cost conditional on my type behavioral choices are independent of group okay so for men and women for example who face some cost to becoming qualified for the job the the employer is using an algorithm to hire them that always induces a man and a woman with

the same cost to do the same thing okay so that we don't have that that situation in that seems so pathological and a preview a taste to come John was proving this this week for binary classification problems and when preferences are group independent what that means is that people have the same payoff functions they don't depend on their group which is obviously a strong assumption but

we can't have a situation where men want to be classified as a one and women want to be classified as a zero so we're assuming people have the same the same payoffs the notion of envy freeness that I just presented is equivalent to equal opportunity and is equivalent to error rate balance so satisfaction of one that implied satisfaction of all of them which provides a defensive

error rate balance it's a statistical notion but there's also a welfare-based defense of it that that we were sort of surprised by but this is not to say the airweight balance is like necessarily the way to go there are also different related defenses of predictive parity that we're also working on and part of which one is a more natural notion of fairness depends on the the

the kind of what we're using this classification data for okay so to conclude the topic uh so the designers of these algorithms have preferences over how people behave oftentimes and how they're classified and when the stakes of classification are exogenous which they probably are in most cases designers can manipulate the population of behaviors in a potentially dramatic way and what we think is interesting is that

manipulation might be intentional or not but in any case we're thinking of it as representing a structural basis of inequality so we have people that are the same but the way they are being judged by someone else is inducing them to become different and potentially unequal um what our democracy points are is that when the stakes of when the stakes of classification are democratic the designer's

ability to manipulate Behavior becomes quite limited and the designer can only face a choice of manipulating the number of compliers between either F of zero if he chooses a null classifier or or the level of compliance that the median wants okay and then this is my final slide so some ongoing and future work um so one thing that I didn't talk about in this talk is

connecting these results In algorithmic fairness with the results on optimal classification that were like the whole first part of the talk so one of the questions that we're interested in is are there certain types of designers certain designer preferences that optimally yield NV free classification and the answer to that is sometimes there are there are certain types of designer preferences that when those designers are making

decisions for different groups they are creating classifications of those groups that yield Envy freeness across groups if we think about this same problem with respect to democratizing the stakes to classification so if we fix our our classifier and let people decide on the rewards can those Democratic rewards be consistent with Envy free classification and the answer to that is in general no they can't because the

optimal reward from the voters perspectives is totally dependent on the distribution of costs so if we have two groups with two distribu different distributions of costs they're going to choose very different levels of penalties and Rewards and then finally just some conceptual issues there are a lot that we're working through in the book that we haven't solved that we're we're still thinking about so what does

it mean to change groups from an Envy free perspective like when I become male do I inherit their do I keep my uh the noisiness of my signal or do I inherit the male signal noisiness so we're assuming you inherit it right now but there are lots of different it's not necessarily clear how we think about these types of fairness criteria um obviously the our results

on Democracy as Michael was saying preferences for noise and transparency are a really important issue and what we've found is that voters are actually given the setup we have sort of totally indifferent over noise even though all of these Notions of algorithmic fairness are like so centered on noise um so that's something that we're working on thinking more about and then finally what is classification used

for so is this classifier meeting out the Rewards or is it used in some other sort of larger uh larger situation that the voter is involved in um and so that's getting at our Notions of predictive parity but it's five o'clock I think it's time for the happy hour so I could take some questions right now but I can also just talk to you right there

at the reception