AR-MLB
An AR Umpire Helmet that augments officiating and fan experience in the MLB

Team

Jason Wang

Role

User Research
Concept Development
Industrial Design
Interface Design
User-Center Design

Tools

Figma
After Effects
Solidworks
Keyshot
Adobe Aero

Overview

Even the best Umpires make mistakes due to human error. These bad calls could lead to financial and emotional losses for organizations, as game attendance has been plummeting due to fan dissatisfaction with the game quality.

AR-MLB is a smart helmet that uses Augmented Reality to support Major League Baseball Umpires in making calls. Furthermore, the technology creates an opportunity for fans to experience the game through the Umpire’s point of view, engaging the younger generation in America’s favorite pastime.

Design objective

To design and use of Augmented Reality to improve Umpires' ability to make accurate calls.

OutCome: Industrial design

OutCome: concept AR prototype

By using the computer to generate the visual strike zones and pitch trail. Umpire can use that information of the pitch feedback to evaluate and result in making accurate pitch calls.

OutCome: ar interface

But most importantly, the generated strike zone is to help umpires. Using cameras and radar systems on the home plate and around the field, The system will give basic Pitch Feedback. Green pitch trail for definite strikes. A Red trail for a definite ball. As for tight corner pitches, I designed it to be an interesting touch, where it will be an Orange pitch trail for the Umpires to make their OWN evaluation and decision. In the end, it all comes down to being consistent.

choice of client

Why the Major League Baseball?

Analytics has shown that the slow pace of play is deteriorating the future of MLB. The frequent replays, time-outs, and inaccurate calls from the officials have made watching baseball uninspiring compared to other sports. MLB is not innovating to keep up with the world of sports broadcasting. The traditional TV broadcasts, controversial calls, and slow-paced gameplay have made baseball difficult for younger generations to enjoy.

research

Accuracy Matters

Accuracy matters in competitive sports but officiating is never close to being perfect. Even in the highest stage, Umpires still miss calls due to the human nature of inconsistency, which may cost a lot of financial losses and even championships to teams.

Is it time for Robot Umpires?

Currently, Minor League Baseball is experimenting with robot umpires to increase the accuracy of calls. It is close for them to bring it to the Major League. However, I believe that instead of having a robot judge calls, the game should use technology to aid humans rather than make decisions for them. To move forward, America's Pastime needs to embrace the future with new technology but also respect tradition and the human element of the game.

Improving Attendances and Broadcast Rating

The second part of this project focus is to improving the fan experience. As one of the oldest sports in the USA, the Major League is not making innovative changes with the average age of attendance of 57 years old. It has been a huge decrease in young audiences in both broadcast & attendance.

Primary Research

Interviews

Interviewed 2 Amateur Umpires, 1 Amateur Catcher, and 5 MLB fans, these are the top insights I have discovered

Pain Points

Umpire's Pain Points

developing CONSISTENCY
blind-spots
not respected
uncomfortable gears

Catcher's Pain Points

dealing w/ biased call
fight w/ the ump
forced to new strategies vs. inconsistent umps.

Fan's Painpoints

bored w/ broadcast style
frustrated about bad calls
gameplay too slow

secondary Research

Umpires' Performance

In 2018, MLB Umpires made 34,294 incorrect ball and strike calls, which is 14 errors a game. Although there is improvement in the last few decades, fact is Umpires can't call balls and strikes any better than this.

missed calls by year

Umpires' Blind Spots and Biased Judgements

Umpires from 2008 through 2018 also exhibited a pronounced and persistent blind spot with a number of incorrect calls at the top of the strike zone. Remarkably, pitches thrown in the top right and left part of the strike zone were called incorrectly almost 27 percent of the time. And while there were improvements in umpiring, the incorrect calls around the bottom right strike zone in 2018 were still a mind-boggling 18.25 percent. Data results confirm that strike-zone blind spots penalized certain pitchers more than others. This time, however, batters benefited from such flubbed calls, as strike zones shrank, forcing pitchers to throw fewer pitches up in the zone. High strikes are typically harder to hit than low strikes for most batters.

bad call rate at STRIKE ZONES

Biased Call after 2 strikes

Attendance slipping

It's no secret that Major League Baseball is losing fans, with MLB reporting a loss of 11 million in-person paying fans between 2007 and 2019. The demographics of the fan base are older and less diverse, with a median fan age of 57 and clocking in at 60% white, according to North Jersey↗

average ticket sales per game

What we could design for

Opportunities

industrial design: moodboard

Redesigning Umpire's Helmet

By incorporating Augmented Reality into the helmet Umpires wear in-game, they could use the technology to augment their ability to make accurate calls.

The design aesthetic comes from mashing up a catcher-style helmet with AR glasses, and having styling keys from gaming, motorcycles, and sci-fi, and bringing that to a traditional American game, which leverages the brand to the next future generation.

styling moodboard

adjectives: ar-mlb is...

Smart

Reliable

Immersive

Sporty

industrial design: sketching

Redesigning Umpire's Helmet

By incorporating Augmented Reality into the helmet Umpires wear in-game, they could use the technology to augment their ability to make accurate calls.

The design aesthetic comes from mashing up a catcher-style helmet with AR glasses, and having styling keys from gaming, motorcycles, and sci-fi, and bringing that to a traditional American game, which leverages the brand to the next future generation.

final direction

industrial design: prototyping

Studying how an AR goggles could fits in a helmet

One key aspect of the design process was studying the space usage and ergonomics of the helmet. To do this, I used a ski mask as a stand-in for the augmented reality goggles and placed it within an umpire mask to see how it fit and functioned within the overall design. This allowed me to get a sense of the spatial constraints and make any necessary adjustments to ensure a comfortable and functional fit.

Additionally, I experimented with different ways in which the goggles could be removed at will, as umpires often need to take off the mask to get a better view of the base runners. This involved testing various attachment and detachment mechanisms to find the most effective and efficient solution

prototypes: detachment mechanisms

configuration

Integrating Technology into the Design

In this section of the design process, I focused on researching and integrating key technologies into the helmet concept. This involved identifying the specific features and capabilities that were most important to include, as well as considering the placement of these technologies within the overall design.

Through this research, I was able to identify the key technologies that would be most impactful and relevant to the project and then worked to integrate them into the design in a way that was both functional and aesthetically pleasing. This included determining the best location for components such as the microphone and augmented reality glasses, as well as considering how these technologies could be seamlessly integrated into the overall form and structure of the helmet."

technology configuration

Open Masked Mode

To see active plays without removing the entire helmet
(E.g. Runner sliding home plays)

final design

ar hud interface

Designing the digital experience

In this section of the design process, I turned my attention to the digital experience of the helmet, specifically focusing on the heads-up display (HUD) interface. This involved designing the layout and functionality of the interface, as well as considering how it would be used by the umpire in real-time during a game. I began by sketching out a range of ideas and experimenting with different layouts and design elements, looking for a solution that would be intuitive and easy to use.

As part of this process, I also designed a digital strike zone feature, which would allow the umpire to see a real-time representation of the strike zone on the HUD. This would provide them with additional information and context as they made calls on the field, helping to ensure more accurate and consistent results.

concept hud experience

video prototype

ar hud interface

Augmented Strikezone

The digital strike zone feature is designed to provide umpires with a real-time representation of the strike zone as they make calls on the field. As the pitch is thrown, the umpire will see a light-colored pitch tray pass through the virtual strike zone on the heads-up display (HUD). This strike zone is unique to the batter, as it is adjusted according to their height and other relevant factors. In addition to showing the strike zone itself, the pitch sequence is also displayed on the HUD to help the umpire build a consistent standard for their calls.

Flagging pitches through trails and zone

The digital strike zone feature on the helmet's heads-up display (HUD) helps umpires make more accurate and consistent calls for balls and strikes. Pitches inside the zone are indicated by a green pitch tray, while pitches outside the zone are indicated by a red pitch tray. This visual representation, along with the labeled pitch sequence on the HUD, allows umpires to make faster and more accurate decisions and develop a consistent standard for their calls."

Additionally, the labeled pitch sequence displayed on the HUD can help umpires develop a more consistent standard for their calls, as it provides them with context and information about the pitches that have been thrown in the current at-bat.

Corner Zone: When Pitches become controversial

One key aspect of the design was the extended zone outside the edges of the traditional strike zone, which I refer to as the "corner zone." For these pitches, the pitch trail becomes white, and the umpire is given the freedom to subjectively determine whether to call it a strike or a ball. This was an important aspect of the design, as it allows the umpire to retain some discretion and judgment in their calls, rather than relying solely on the technology to make the decision.

When the Umpire need to host a review

One of the key benefits of the helmet is its ability to allow umpires to quickly and easily review close or controversial plays. Umpires can review close plays and be more informed to make accurate decisions, particularly on difficult "bang-bang" plays, and this allows them to do so more quickly than with traditional replay methods. This can help to reduce delays and improve the overall flow of the game.

fan experience

Fan experience in virtual reality

In addition to improving the performance of umpires on the field, the augmented reality helmet project could also enhance the experience of young baseball fans and make MLB VR more complete. Currently, MLB VR offerings tend to focus on stats and data, rather than providing a more immersive and interactive experience.

MLB VR IN OCULUS: FOCUS on STats rather than immersiveness

Immersive Viewing Experience

How the AR-MLB Brings Fans Closer to the Action

By incorporating the helmet's camera, which could broadcast the umpire's point of view (POV) to fans, viewers could get a new and unique perspective on the game that feels like they are on the field with the players.

For example, the camera could share detailed footage of a home run swing or a jaw-dropping curveball from the umpire's POV, engaging the audience's adrenaline and providing a more exciting and immersive viewing experience. By providing this new perspective, the helmet could help to make baseball more fun and exciting for a wider audience.

MLB VR after AR-MLB: inviting new perspectives for broadcast

MLB VR after AR-MLB: broadcasting ump's pov in game

reflection

How AR-MLB challenges my product design (UX) skills

Overall, I have learned a lot during my senior thesis project as an industrial designer. I have gained valuable experience in physical design, including aesthetics and styling, mechanism, and mechanical design. In addition to these traditional areas of focus, I have also had the opportunity to delve into AR interface design and UX design, as I worked to create a virtual experience for the umpire that was intuitive and easy to use. This project has sparked my interest in UX design and has made me want to shift my focus to studying this area more in-depth. If given more time and resources, I would love the opportunity to prototype this project with real engineers and see how it performs in the field. Overall, this project has been a valuable learning experience and has opened up new areas of interest for me as a designer.