REVIEW 2 minor 84 references
Even clearly defined rules require a complex translation process to be operationalized by technology, as shown by MLB's seven-year effort to automate ball and strike calls.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-06-30 18:59 UTC pith:GOIG6ITK
load-bearing objection The MLB ABS case shows that even rulebook-clear enforcement still needs years of sociotechnical translation work because ground truth mixes written rules with prior practice and stakeholder balancing.
Inside Baseball: The Automated Ball-Strike System as an Object Lesson in Technological Rule Enforcement
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Although the strike zone is an area clearly defined in the rulebook, it took MLB seven years to figure out how to automate calling balls and strikes with ABS, showing how even seemingly straightforward rules require a complex translation process to operationalize via technological systems. The distance between rule and implementation exists because the ground truth of the strike zone has historically been a hybrid of the rulebook definition and umpires' enforcement decisions, and because ABS must fit into an ecosystem that balances multiple stakeholder values.
What carries the argument
The seven-year design process of the Automated Ball-Strike System (ABS), which operationalizes distance to the strike zone by negotiating historical umpire practices and stakeholder interests rather than applying the rulebook definition alone.
Load-bearing premise
The ground truth of the strike zone can be treated as solely the rulebook definition instead of a long-standing hybrid with umpires' enforcement decisions.
What would settle it
A direct test comparing ABS call accuracy and acceptance when it strictly follows the rulebook definition versus when it incorporates historical umpire strike zone data would show whether the hybrid approach is required for practical use.
If this is right
- Evaluation of automated rule systems must account for how they are experienced in practice rather than centering only on fidelity to the written rule.
- Technological enforcement systems need to balance multiple stakeholder values within existing ecosystems instead of assuming a neutral mapping from rule to output.
- In-depth social science approaches are needed to assess implementation beyond conventional accuracy metrics.
- Similar translation distances are likely to appear in other domains where rules meet automated enforcement.
Where Pith is reading between the lines
- The same hybrid-ground-truth issue could arise in automated traffic enforcement or regulatory compliance tools where historical human discretion has shaped effective rules.
- Designers of future systems might need to include historical practice data as a core input rather than starting from formal rules alone.
- This perspective could shift focus in system evaluation from pure technical accuracy to acceptance by affected parties over time.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an in-depth case study of Major League Baseball's seven-year experimentation with the Automated Ball-Strike System (ABS). It claims that even though the strike zone is clearly defined in the rulebook, automating ball-strike calls required extensive design work because the ground truth has always been a hybrid of the rulebook definition and umpires' enforcement practices, and because implementation must balance multiple stakeholder values within the existing ecosystem. Drawing on Science and Technology Studies (STS) frameworks, the authors argue that this reveals a 'distance' between clear rules and technological operationalization, challenging conventional evaluation paradigms that center only on rule-to-implementation distance and instead calling for social science approaches to assess how such systems are experienced in practice.
Significance. If the historical and contextual analysis holds, the paper supplies a concrete, non-circular object lesson for FAccT on the sociotechnical complexities of automated rule enforcement. It credits the use of an external seven-year case study (with traced design decisions) to illustrate why in-depth interpretive methods are needed beyond technical metrics, contributing to conversations on fairness and accountability in sociotechnical systems.
minor comments (2)
- [Abstract] Abstract: The claim that 'it took MLB seven years to figure out how to automate' would benefit from a brief parenthetical example of one key design decision (e.g., sensor placement or zone calibration) to preview the empirical substance for readers.
- The manuscript would be strengthened by explicitly naming the specific STS analytic frameworks invoked (e.g., which concepts from actor-network theory or boundary objects) when they are first applied to the ABS case.
Simulated Author's Rebuttal
We thank the referee for their supportive review, accurate summary of the manuscript's contributions, and recommendation for minor revision. The assessment of significance aligns well with our intent to provide a concrete STS case study for FAccT audiences.
Circularity Check
No significant circularity identified
full rationale
The paper is a qualitative STS case study of MLB's ABS development, tracing historical and ecosystem factors in rule operationalization. Its central claims rest on external historical records, rulebook text, and stakeholder analysis rather than any derivation chain, equations, fitted parameters, or self-citations that reduce assertions to inputs by construction. No load-bearing steps match the enumerated circularity patterns; the argument is self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The ground truth of the strike zone is contested because the rule in practice has always reflected a hybrid between the rulebook definition and umpires' enforcement decisions
read the original abstract
Clearly-defined rules are often assumed to be straightforward to automate and evaluate. We challenge this assumption through an in-depth study of Major League Baseball's (MLB) seven-year experimentation with the Automated Ball-Strike System (ABS). ABS is envisioned to call balls and strikes accurately: a seemingly straightforward use of technology to objectively determine the distance between a pitch and the strike zone. Although the strike zone is an area clearly defined in the rulebook, it took MLB seven years to figure out how to automate calling balls and strikes with ABS, showing how even seemingly straightforward rules require a complex translation process to operationalize via technological systems. In this paper, we trace the design decisions that led to the current implementation of ABS. Our case study reveals that "distance" exists even between a clear rule and its technological implementation. Using analytic frameworks from Science and Technology Studies (STS), we show that such distance exists because (1) historically, the "ground truth" of the strike zone is contested: the rule in practice has always reflected a hybrid between the rulebook definition and umpires' enforcement decisions; and (2) the use of ABS is embedded in an existing eco-system, where the implementation of a technological enforcement system needs to balance multiple stakeholder values. This perspective challenges conventional evaluation paradigms that center on the distance between a formalized rule and its technological implementation, and instead calls for evaluating how such systems are experienced in practice. Addressing this question requires in-depth social science approaches, contributing to ongoing conversations in FAccT about the implementation and evaluation of sociotechnical systems.
Figures
Reference graph
Works this paper leans on
-
[1]
MLB on track to expand robot umps to all Triple-A ballparks.AP News(Jan
2023. MLB on track to expand robot umps to all Triple-A ballparks.AP News(Jan. 2023). https://apnews.com/article/mlb-sports-rob- manfred-c4c8746bc3be2e932f109fc91cff6f81
2023
-
[2]
Ali Alkhatib and Michael Bernstein. 2019. Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. doi:10.1145/3290605.3300760
-
[3]
Louise Amoore. 2006. Biometric borders: Governing mobilities in the war on terror.Political Geography25, 3 (March 2006), 336–351. doi:10.1016/j.polgeo.2006.02.001
-
[4]
Louise Amoore. 2024. The deep border.Political Geography109 (March 2024), 102547. doi:10.1016/j.polgeo.2021.102547
-
[5]
L. A. Times Archives. 1985. On Close Calls.Los Angeles Times(Oct. 1985). https://www.latimes.com/archives/la-xpm-1985-10-22-me- 12377-story.html Section: Sports
1985
-
[6]
Aparna Balagopalan, Haoran Zhang, Kimia Hamidieh, Thomas Hartvigsen, Frank Rudzicz, and Marzyeh Ghassemi. 2022. The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22). Association for Computing Machinery, New York, NY, USA, 11...
-
[7]
2025.Official baseball rules, 2025(2025 edition ed.)
Major League Baseball. 2025.Official baseball rules, 2025(2025 edition ed.). Triumph Books, Chicago, IL. OCLC: 1520628966
2025
-
[8]
2019.Race after technology: abolitionist tools for the new Jim code
Ruha Benjamin. 2019.Race after technology: abolitionist tools for the new Jim code. Polity Press, Cambridge, England
2019
-
[9]
Ronald Blum. 2021. Automated strike zone coming to minors but a while from MLB.AP News(March 2021). https://apnews.com/ article/technology-mlb-baseball-rob-manfred-minor-league-baseball-fd64782047ad44f447e272375c712da5
2021
-
[10]
Ronald Blum. 2023. Strike zone at Triple-A to get slightly bigger starting Tuesday.AP News(Sept. 2023). https://apnews.com/article/robot- umpires-minor-league-strike-zone-2541180f8aaac1c79659a4e681e0a271
2023
-
[11]
Ronald Blum. 2023. What is a strike in baseball? Robots, rule book and umpires view it differently.AP News(July 2023). https: //apnews.com/article/mlb-robot-umpires-strike-zone-40ec7285ae4d1ccaf2621adcb8d72b02 Section: Sports
2023
-
[12]
Ronald Blum. 2025. Robot umpires are getting their first MLB test during spring training.AP News(Feb. 2025). https://apnews.com/ article/mlb-robot-umpires-abs-9034454b5a795262bf97446ff38d0361
2025
-
[13]
Tim Booth. 2023. MLB catchers wary of looming robo umps amid rules changes.AP News(Feb. 2023). https://apnews.com/article/mlb- sports-minnesota-twins-rob-manfred-jose-trevino-063f72484e37a99e6ff428aaaf156d2d
2023
-
[14]
Mark Bovens and Stavros Zouridis. 2002. From Street-Level to System-Level Bureaucracies: How Information and Communication Technology is Transforming Administrative Discretion and Constitutional Control.Public Administration Review62, 2 (2002), 174–184. doi:10.1111/0033-3352.00168 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/0033-3352.00168
-
[15]
Bowker and Susan Leigh Star
Geoffrey C. Bowker and Susan Leigh Star. 2000.Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA, USA
2000
-
[16]
David Brandt. 2019. MLB’s top prospects deal with good, bad of ’robot’ umpires.AP News(Oct. 2019). https://apnews.com/mlbs-top- prospects-deal-with-good-bad-of-robot-umpires-2972799c8928483c92156c182909625b
2019
-
[17]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology.Qualitative Research in Psychology3, 2 (Jan. 2006), 77–101. doi:10.1191/1478088706qp063oa _eprint: https://doi.org/10.1191/1478088706qp063oa
-
[18]
Ben Cafardo. 2011. Happy 10th birthday, K Zone.ESPN Front Row(July 2011). https://www.espnfrontrow.com/2011/07/happy-10th- birthday-k-zone/
2011
-
[19]
Anthony Castrovince. 2025. ABS Challenge System coming to MLB full time in ’26.MLB.com(Sept. 2025). https://www.mlb.com/news/abs- challenge-system-mlb-2026
2025
-
[20]
Anthony Castrovince. 2025. MLB testing automated ball-strike challenge system during spring games.MLB.com(Feb. 2025). https: //www.mlb.com/news/automated-ball-strike-calls-mlb-spring-games
2025
-
[21]
Mori Chan. 2008. Explanation of the strike zone in baseball. Wikimedia Commons. https://commons.wikimedia.org/w/index.php? curid=5591282 Licensed under CC BY 2.0
2008
-
[22]
2017.Bad Call: Technology’s Attack on Referees and Umpires and How to Fix It
Harry Collins, Robert Evans, and Christopher Higgins. 2017.Bad Call: Technology’s Attack on Referees and Umpires and How to Fix It. MIT Press, Cambridge, MA, USA
2017
-
[23]
J. J. Cooper. 2025. By Ditching MLB Rule Book, ABS Strike Zone Has Found Its Footing. https://www.baseballamerica.com/stories/by- ditching-mlb-rule-book-abs-strike-zone-has-found-its-footing/
2025
-
[24]
Creswell and Vicki L
John W. Creswell and Vicki L. Plano Clark. 2011.Designing and Conducting Mixed Methods Research. SAGE Publications, Inc, Los Angeles
2011
-
[25]
Katie Dean. 2003. Umpires to Tech: You’re Out!Wired(June 2003). https://www.wired.com/2003/06/umpires-to-tech-youre-out/ Section: tags
2003
-
[26]
Detlef. 2026. Evolution of ESPN’s K-Zone Technology [2001-2022].YouTube(April 2026). https://www.youtube.com/watch?v= AnHcyxU9PYA Inside Baseball: ABS as an Object Lesson in Technological Rule Enforcement FAccT ’26, June 25–28, 2026, Montreal, QC, Canada
2026
-
[27]
Sam Dykstra. 2020. Toolshed: Farms prep for FSL robot umps. https://www.milb.com/news/toolshed-farm-systems-prepare-for-florida- state-league-robot-umpires-313089276
2020
-
[28]
2021.Criminal Futures: Predictive Policing and Everyday Police Work
Simon Egbert and Matthias Leese. 2021.Criminal Futures: Predictive Policing and Everyday Police Work. Routledge, London ; New York
2021
-
[29]
Mary Flanagan, Daniel C. Howe, and Helen Nissenbaum. 2008. Embodying Values in Technology: Theory and Practice. InInformation Technology and Moral Philosophy, Jeroen van den Hoven and John Weckert (Eds.). Cambridge University Press, Cambridge, 322–353. doi:10.1017/CBO9780511498725.017
-
[30]
2014.Values at Play in Digital Games
Mary Flanagan and Helen Nissenbaum. 2014.Values at Play in Digital Games. MIT Press, Cambridge, MA, USA
2014
-
[31]
Society for American Baseball Research. 2025. SABR Analytics Conference. https://sabr.org/analytics
2025
-
[32]
Charles Fried. 2012. Balls and Strikes.Emory Law Journal61, 4 (Jan. 2012), 641. https://scholarlycommons.law.emory.edu/elj/vol61/iss4/1
2012
-
[33]
Batya Friedman and David G. Hendry. 2019.Value Sensitive Design: Shaping Technology with Moral Imagination. The MIT Press. doi:10.7551/mitpress/7585.001.0001
-
[34]
Kyle Glaser. 2021. Did Pitch Clocks, Robo Umps And Other 2021 MiLB Rules Changes Work? https://www.baseballamerica.com/stories/ did-pitch-clocks-robo-umps-and-other-2021-minor-league-baseball-rules-changes-work/
2021
-
[35]
Jimmy Golen. 2023. Robo umps reach Triple-A, but MLB rollout still uncertain. https://apnews.com/article/robo-umps-abs-triplea- ccc901dc69c6101fb6a793e5fe867a77
2023
-
[36]
Ben Green. 2022. The flaws of policies requiring human oversight of government algorithms.Computer Law & Security Review45 (July 2022), 105681. doi:10.1016/j.clsr.2022.105681
-
[37]
2019.Ethnography: Principles in practice
Martyn Hammersley and Paul Atkinson. 2019.Ethnography: Principles in practice. Routledge
2019
-
[38]
2019.Strike Four: The Evolution of Baseball
Richard Hershberger and John Thorn. 2019.Strike Four: The Evolution of Baseball. Rowman & Littlefield Publishers, Lanham, Maryland
2019
-
[39]
David J. Hunter. 2018. New metrics for evaluating home plate umpire consistency and accuracy.Journal of Quantitative Analysis in Sports14, 4 (Dec. 2018), 159–172. doi:10.1515/jqas-2018-0061
-
[40]
Abigail Z. Jacobs and Hanna Wallach. 2021. Measurement and Fairness. InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21). Association for Computing Machinery, New York, NY, USA, 375–385. doi:10.1145/3442188. 3445901
-
[41]
Ben Jedlovec. 2020. Introducing Statcast 2020: Hawk-Eye and Google Cloud. https://technology.mlblogs.com/introducing-statcast-2020- hawk-eye-and-google-cloud-a5f5c20321b8
2020
-
[42]
Meg Leta Jones. 2017. The right to a human in the loop: Political constructions of computer automation and personhood.Social Studies of Science47, 2 (April 2017), 216–239. doi:10.1177/0306312717699716
-
[43]
Meg Leta Jones and Karen Levy. 2018. Sporting Chances: Robot Referees and the Automation of Enforcement. doi:10.2139/ssrn.3293076
-
[44]
Waki Kamino, Andrea W. Wen-Yi, Dhruv Agarwal, Sil Hamilton, Eun Jeong Kang, Jieun Kim, Keigo Kusumegi, Pegah Moradi, Daniel Mwesigwa, Yan Tao, I-Ting Tsai, Ethan Yang, Shengqi Zhu, Shu-Jung Han, Chi-Jung Lee, Michael Joseph Sack, Tianhong Catherine Yu, Weslie Khoo, Andy Elliot Ricci, Yoyo Tsung-Yu Hou, Boyoung Kim, Selma šabanović, David J. Crandall, Kare...
-
[45]
InProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’25)
Million Eyes on the ’Robot Umps’: The Case for Studying Sports in HRI Through Baseball. InProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’25). IEEE Press, Melbourne, Australia, 1383–1388
2025
- [46]
-
[47]
Ian R. Kerr. 2010. Digital Locks and the Automation of Virtue. https://papers.ssrn.com/abstract=2115655
2010
-
[48]
Barbara Kiviat, Sara Sternberg Greene, and Hesu Yoon. 2025. Exceptions in the Algorithmic Age: Evidence from the Case of Tenant Screening.Amer. J. Sociology(Oct. 2025). doi:10.1086/739108
-
[49]
Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores.arXiv preprint arXiv:1609.05807(2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[50]
John Koepke and David Robinson. 2018. Danger Ahead: Risk Assessment and the Future of Bail Reform.Washington Law Review93, 4 (Dec. 2018), 1725. https://digitalcommons.law.uw.edu/wlr/vol93/iss4/4
2018
-
[51]
2025.Fewer Rules, Better People: The Case for Discretion
Barry Lam. 2025.Fewer Rules, Better People: The Case for Discretion. W. W. Norton & Company
2025
-
[52]
2009.Code: And other laws of cyberspace
Lawrence Lessig. 2009.Code: And other laws of cyberspace. ReadHowYouWant. com
2009
-
[53]
2022.Data Driven: Truckers, Technology, and the New Workplace Surveillance
Karen Levy. 2022.Data Driven: Truckers, Technology, and the New Workplace Surveillance. Princeton University Press. Publisher: Princeton University Press
2022
-
[54]
Ben Lindbergh and Rob Arthur. 2021. MLB Just Tried a Bunch of Experimental Rules in the Minors. How Well Did They Work?The Ringer(Oct. 2021). https://www.theringer.com/2021/10/21/mlb/experimental-rules-atlantic-league-robo-umps
2021
-
[55]
Rob Maaddi. 2019. ’Robot umpires’ debut in independent Atlantic League. https://apnews.com/article/93f7e8be7f404ff2a6eadf2e9295c9d3
2019
-
[56]
Emanuel Maiberg. 2026. ’Students Are Being Treated Like Guinea Pigs:’ Inside an AI-Powered Private School.404 Media(Feb. 2026). https://www.404media.co/students-are-being-treated-like-guinea-pigs-inside-an-ai-powered-private-school/
2026
-
[57]
Jonathan Mayo and William Boor. 2019. Major League Baseball tests robot umpires Arizona Fall League. (Sept. 2019). https: //www.mlb.com/news/major-league-baseball-tests-robot-umpires-arizona-fall-league FAccT ’26, June 25–28, 2026, Montreal, QC, Canada Andrea Wen-Yi Wang, Waki Kamino, David Mimno, Karen Levy, and Malte F. Jung
2019
-
[58]
Theodore McKee. 2007. Judges as Umpires.Hofstra Law Review35, 4 (Jan. 2007). https://scholarlycommons.law.hofstra.edu/hlr/vol35/ iss4/3
2007
-
[59]
MLB. [n. d.]. Pitch-tracking Era | Glossary. https://www.mlb.com/glossary/miscellaneous/pitch-tracking-era
-
[60]
2025.MLB Umpire Camps
MLB Advanced Media, LP. 2025.MLB Umpire Camps. https://www.mlb.com/official-information/umpires/camps
2025
-
[61]
Christina M Mulligan. 2008. Perfect Enforcement Of Law: When To Limit And When To Use Technology. 4 (2008)
2008
-
[62]
Nick Paumgarten. 2003. No Flag on the Play.The New Yorker(Jan. 2003). https://www.newyorker.com/magazine/2003/01/20/no-flag- on-the-play Section: dept. of super slo mo
2003
- [63]
-
[64]
Trevor J. Pinch and Wiebe E. Bijker. 1984. The Social Construction of Facts and Artefacts: or How the Sociology of Science and the Sociology of Technology might Benefit Each Other.Social Studies of Science14, 3 (Aug. 1984), 399–441. doi:10.1177/030631284014003004
-
[65]
Robert Lee Poe and Soumia Zohra El Mestari. 2024. The Conflict Between Algorithmic Fairness and Non-Discrimination: An Analysis of Fair Automated Hiring. InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’24). Association for Computing Machinery, New York, NY, USA, 1907–1916. doi:10.1145/3630106.3659015
-
[66]
Roscoe Pound. 1910. Law in books and law in action.American Law Review44 (1910), 12
1910
-
[67]
Michael Rich. 2012. Should We Make Crime Impossible?SSRN Electronic Journal(2012). doi:10.2139/ssrn.2029201
- [68]
-
[69]
Richard Sandomir. 2001. TV SPORTS; You Have Entered The Strike Zone.The New York Times(July 2001). https://www.nytimes.com/ 2001/07/19/sports/tv-sports-you-have-entered-the-strike-zone.html
2001
-
[70]
Richard Sandomir. 2016. An Electronic Umpire? Baseball Tried It (In the 1950s!).The New York Times(April 2016). https://www.nytimes. com/2016/05/01/sports/baseball/more-machine-than-man-an-electric-umpires-call-of-the-future.html
2016
-
[71]
1991.Playing by the Rules: A Philosophical Examination of Rule-Based Decision-Making in Law and in Life
Frederick Schauer. 1991.Playing by the Rules: A Philosophical Examination of Rule-Based Decision-Making in Law and in Life. Clarendon Press. Google-Books-ID: CNXhBQAAQBAJ
1991
-
[72]
Michael S. Schmidt. 2025. A Q&A With Rob Manfred, M.L.B. ’s Commissioner, on the Future of Baseball.The New York Times(April 2025). https://www.nytimes.com/2025/04/06/briefing/rob-manfred-interview-baseball.html
2025
-
[73]
Selbst, Danah Boyd, Sorelle A
Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2018. Fairness and Abstraction in Sociotechnical Systems. https://papers.ssrn.com/abstract=3265913
2018
-
[74]
Esther Shein. 2024. AI Judging in Sports – Communications of the ACM. https://cacm.acm.org/news/ai-judging-in-sports/
2024
- [75]
-
[76]
2025.Southern Umpires Camp
Southern and International Baseball Umpire Training Camps. 2025.Southern Umpires Camp. https://baseballumpirecamps.com/page/ southern
2025
-
[77]
Hansa Srinivasan, Candice Schumann, Aradhana Sinha, David Madras, Gbolahan Oluwafemi Olanubi, Alex Beutel, Susanna Ricco, and Jilin Chen. 2024. Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’24). As...
- [78]
- [79]
-
[80]
Nikki Stevens and Os Keyes. 2021. Seeing infrastructure: race, facial recognition and the politics of data.Cultural Studies35, 4-5 (Sept. 2021), 833–853. doi:10.1080/09502386.2021.1895252 _eprint: https://doi.org/10.1080/09502386.2021.1895252
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.