The Robots Are Coming: AI Replaces Line Judges at U.S. Open, With Global Implications for Jobs

This post was originally published on CFR's Net Politics blog.

Sports have long been a substitute for warfare in terms of global competition. In fact, the Olympic Games were established in ancient Greece, in part, as an antidote to war and pushed Greeks to work together and achieve a more peaceful state. The original games even paid homage to fallen soldiers that were lost as a result of war. Sports bring together players from all over the world to participate in healthy international competition, with fans cheering for their favorite teams and players–sometimes along nationalist, ethnic pride, or other lines.

Having had the good fortune of attending the U.S. Open men’s semi-final match between Frances Tiafoe and Carlos Alcaraz this September–my first time back to the Open since before the pandemic–I witnessed a new development. There were far fewer people on the court–no line judges! Instead, the U.S. Open (and other international tennis tournaments, like the Australian Open) are now using Hawk-Eye live, which uses optical tracking and camera collaboration technology to replace calls that line judges would make otherwise. Yes, the robots are coming, this time to an international sporting competition near you!

Indeed, over two hundred people were replaced by the Hawk-Eye Live technology. As NPR notes, “But some of their voices live on: the recorded calls heard during play include the voices of line judges who went into a studio inside Arthur Ashe Stadium and, essentially, recorded their swan songs.” As Sean Cary, who oversees officiating for the United States Tennis Association (USTA)–which runs the U.S. Open–points out, "The thing that I think is really cool is that we've been able to program the system to know that if the ball is way out, it's going to be a softer outcall. But if it's a really close one like it would be with a live line umpire, they generally yell at the top of their voices to make sure everyone hears it." He also contends that replacing linesmen with the Hawk-Eye provides “a much cleaner court for our broadcast partners and our sponsorship partners." 

But even as technology brings benefits, including speed and accuracy in certain circumstances, it raises certain challenges. And the trend toward automation was only expedited by the COVID-19 pandemic as a means to reduce human contact. Jobs that require more physical presence–such as operating machinery and service industry work–have a more physical closeness, frequency of interactions, and exposure to strangers have made the need for automation stronger in the wake of the pandemic.

The move toward automation raises at least three concerns. One of the concerns is job displacements when entire professions or particular aspects of employment are automated. As I’ve written previously, the pandemic (and, relatedly, automation) accelerated job loss for marginalized groups, even as these historically disadvantaged groups were forced to continue to work in-person, given their overrepresentation in “essential” (frontline) work. These racial and gender justice paradoxes should prompt policymakers to be alert to the distributional justice implications of automation. Workers will need to be reskilled for the future of work in growth sectors, such as technology or the green economy. Women globally are particularly affected by automation–in some countries more than others–as women and girls are often pushed into jobs which are vulnerable to automation and are underrepresented in science, technology, engineering, and math (STEM) jobs and education.

A second concern is that while technological innovation, such as the Hawk-Eye system, can lead to greater accuracy in say, line calls, it also carries the image of being error-free, such that its determinations are rarely questioned. Indeed, few tennis players challenged the calls made by the Hawk-Eye. Yet, as University of Virginia Law Professor Danielle Citron, warns:

The cognitive system’s engineering literature has found that human beings view automated systems as error-resistant. As a result, operators of government decision systems are less likely to search for information that would contradict a computer-generated solution. Studies show that human beings rely on automated decisions even when they suspect system malfunction. The impulse to follow a computer’s recommendation flows from human “automation bias”—[which] effectively turns a computer program’s suggested answer into a trusted final decision.

Technology often embeds human error, subjectivity, and even bias. For example, Amazon got rid of AI hiring technology after it found that the system discriminated against women because it relied on data from the past five to ten years, a time when women were underrepresented in the technology industry. Amazon’s AI system, in effect, learned to discriminate against women–teaching itself that male candidates were preferable. The system penalized resumes that included the word “women’s,” such as “women’s chess club captain” and “women’s college.”

A final and related concern is that humans are in the loop–whether upstream, in the design of AI, algorithmic governance, robots, and other tools, or downstream, in the curation and moderation of social media and other aspects of the internet. Human bias, subjectivity, and error are potentially imported into supposedly automated decision-making at several points. As we shift from man plus machine to machine plus man in many workplaces, we should be attentive to what is gained and what is lost. The humans working behind the scene—and “behind the screen” as Sarah Roberts elegantly reminds us—are often different workers than the workers who had the jobs that have now been automated.

The emergence of the Hawk-Eye is a development that all sports fans should keep our eye on, even as it keeps an eye on us. Sports are just one arena where new technologies are being used in what has traditionally been a role filled by a person. The workplace and many sectors of our economy are ever-changing, with technological innovation part and parcel of seismic changes in our relationship to work. Technology will continue to change the character of work in the United States, and the government must be prepared to address the problems posed by this technological development.