Date: 27th Feb 2020

We discussed the following paper in the weekly reading group of CNeRG, IIT Kharagpur.

• Liu et al. “Delayed Impact of Fair Machine Learning.” International Conference on Machine Learning. 2018.

This paper studies the impact of applying fair machine learning on the population in money lending scenario. The authors have strongly questioned the long standing assumption on fair machine learning: “Applying fairness constraints on machine learning will improve the situation for the protected groups”. The paper shows that under different conditions, the fairness constraint may even cause harm to the protected group which it intends to protect. The authors have also emphasized on the requirement of domain-specific modeling of delayed impact of fair ML to better understand which fairness constraint(s) serves the best under different conditions.

We used the slides made available by the authors. We also made the following toy examples to make it easier for the audience to understand the concepts.

#### Example-1

Consider the set of credit scores as Χ = {‘low’, ‘high’}. Let ‘low’ and ‘high’ correspond to 100 and 500 scores. The success probability of ‘low’ and ‘high’ applicants are 0 and 1 correspondingly. If someone gets a loan from the bank and repays successfully then her score improves by 75 points and the bank gains 25 utility by earning the interest amount. On the other hand, if the loan is defaulted then the candidate is set to lose 150 points in credit score, and the bank is set to lose 100 points. Let the group-wise distribution of the population over scores are as given in the table below.

group\scores ‘low’ ‘high’ total
black 90 10 100
white 300 100 400

Let’s assume the bank has an upper limit on the number of loans they can issue.

• Bank’s upper limit on loans = 60
Here, an unconstrained utility maximization will select 60 applicants randomly out of 110 (10 blacks + 100 whites) ‘high’ scored ones. Therefore the expected number of selected black applicants will be as given below.
E[#blacks] = 60 * (10/110) = 60/11
As (60/11)<10, all the selected blacks will repay their loan amounts with probability 1.
The expected change in the average score of black group = (60/11)* (75)/(100) ≈ 4.09
Now let’s say, the bank is legally mandated to maintain demographic parity in lending. The bank would end up giving loans in 100:400 or 1:4 ratio, i.e., 12 blacks and 48 whites. Then the bank would give loans to 10 black applicants with ‘high’ scores and 2 blacks with ‘low’ scores.
The expected change in the average score of black group = (10 * 75 - 2 * 150)/100 = 4.5
Here, we can clearly see that demographic parity constraint causes better improvement in black group’s average score in comparision to unconstrained lending practice.
• Bank’s upper limit on loans = 80
Here, an unconstrained utility maximization will select 80 applicants randomly out of 110 (10 blacks + 100 whites) ‘high’ scored ones. Therefore the expected number of selected black applicants will be as given below.
E[#blacks] = 80 * (10/110) = 80/11
As (80/11)<10, all the selected blacks will repay their loans with probability 1.
The expected change in the average score of black group = (80/11)* (75)/(100) ≈ 5.45
Now let’s say, the bank is mandated to maintain demographic parity in lending. It would result in giving loans in 100:400 or 1:4 ratio, i.e., 16 blacks and 64 whites. The bank would give loans to 10 black applicants with ‘high’ scores and 6 blacks with ‘low’ scores.
The expected change in the average score of black group = (10 * 75 - 6 * 150)/100 = -1.5
Here, we can clearly see that demographic parity constraint has caused harm to black group by resulting in a decline in average score while unconstrained lending practice could have improved group’s average score. Thus, demographic parity may cause harm to the protected group by overaccepting.

#### Example-2

Consider the set of credit scores as Χ = {‘low’, ‘high’}. Let ‘low’ and ‘high’ correspond to 100 and 500 scores. The success probability of ‘low’ and ‘high’ applicants are 0 and 1 correspondingly. If someone gets a loan from the bank and repays successfully then her score improves by 75 points and the bank gains 25 utility for getting the interest amount. On the other hand, if the loan is defaulted then the candidate is set to lose 150 points in credit score, and the bank is set to lose 100 points. Let the group-wise distribution of the population over scores are as given in the table below.

group\scores ‘low’ ‘medium’ ‘high’ total
black 90 0 10 100
white 200 100 100 400

Let’s assume the bank has an upper limit on the number of loans they can issue.

• Bank’s upper limit on loans = 80
Here, an unconstrained utility maximization will select 80 applicants randomly out of 110 (10 blacks + 100 whites) ‘high’ scored ones. Therefore the expected number of selected black applicants will be as given below.
E[#blacks] = 80 * (10/110) = 80/11
As (80/11)<10, all the selected blacks will repay their loans with probability 1.
The expected change in the average score of black group = (80/11)* (75)/(100) ≈ 5.45
Now let’s say, the bank is mandated to maintain equal opportunity in lending. Equal opportunity requires the ratio of selection rate to the success rate of all the groups to be equal. Here it would result in selecting 80 from, (i) each of 10 ‘high’ scored blacks with probability (10/160), (ii) each of 100 ‘high’ scored whites with probability (100/160), and (iii) each of 100 ‘medium’ scored whites with probability (50/160).
Now E[#blacks] = 80*(10/160) = 5
The expected change in the average score of black group = (5 * 75 )/100 = 3.75
Here, we can clearly see that equal opportunity constraint has resulted in less improvement than the unconstrained lending practice. Thus, equal opportunity constraint may cause under-acceptance and relative harm to the protected group.