THE PROBLEM
Black and brown students face rampant inequality on a daily basis. They are suspended more often, placed on lower academic tracks, and taught content to which they often cannot culturally relate, compared to white students for whom the education system is designed. These experiences quickly label students in ways that exclude them from the system, leading them down a well-researched school-to-prison pipeline. Discrimination is reflected in a range of “objective” outcome data that illustrate a race gap in attendance and discipline records, grades, and test scores. Edtech companies use such data to train algorithms that promise to personalize learning, identify at-risk students, and save teachers time. Without examining the biases that influence this data, companies using artificial intelligence and machine learning can amplify existing bias, along with their own assumptions, into the products schools use.
WHY SHOULD I CARE?
Black and brown students make up over half of students in American K-12 public schools today, yet edtech companies rarely design products with their unique needs in mind. Unfortunately, algorithms don’t work well on students for whom they were not designed. Technologies in other sectors have run into major racial discrimination challenges, with reports of racial bias in facial recognition for surveillance, in risk assessment for the criminal justice system, and in smart recruitment systems for corporate hiring. Similarly, technologies designed to predict dropouts, identify behavioral issues, and personalize learning may perpetuate the same outcomes from the past; products that are designed without examining the underlying bias will further encode the racist history of our systems.
Even well-meaning companies can unknowingly introduce racial bias into their products. As schools trust technology to take on a greater share of human decisions in order to free up teacher time, they expose black and brown students to the risk that algorithms will unknowingly limit their opportunities – based on the black and brown students whom the system has already failed.
Although schools must abide by equal opportunity and anti-discrimination laws, there are no existing laws or regulations to hold edtech companies accountable for bias in their products. Companies are not required to disclose to schools or families whether or not they use artificial intelligence or machine learning, nor how the data they collect is used within their algorithms. Digital equity is not just about access to devices and the internet. Schools must champion the needs of their black and brown students and require edtech companies to commit to racial equity, on par with their commitments to data privacy and accessibility.
WHAT SHOULD I DO?
Prior to purchase, schools should require that edtech companies adopt a set of equitable practices and disclose the risks and impact of their algorithms to students and families. Edtech Equity provides a set of questions and evaluation rubric that schools can use to assess a product’s potential impact to racial equity, and we encourage schools to bring students and families into the conversation. Schools and their communities should be seen as partners in the design and development of school products to ensure that teachers, students, and families are comfortable with the risk and impact of the products they use.
Links to further research:
1. Data Snapshot: School Discipline2. The Other Segregation3. Culturally Responsive Teaching4. Race, Disability and the School-to-Prison Pipeline5. Status and Trends in the Education of Racial and Ethnic Groups 20186. Chronic Absenteeism in the Nation's Schools7. Disproportionality in student discipline: Connecting policy to research8. Inequality at school9. The Black-White Test Score Gap: Why It Persists and What Can Be Done10. Racial/Ethnic Enrollment in Public Schools11. Population validity for Educational Data Mining models: A case study in affect detection12. Face Recognition Vendor Test (FRVT)
13. Machine Bias14. Complaint and Request for Investigation, Injunction, and Other Relief 
