Tech Notes
Many of us are familiar with the subtitles and captions that help us follow along when music swells over dialogue in movies and TV shows. But what if you couldn’t see the myriad of stunning visuals that bring your favorite performances to life? Instead of words written out on a screen, you would need a voice to guide you. Insert audio descriptions here. This often overlooked form of accessibility is essential for the 2.2 billion blind and low vision (BLV) folks worldwide that rely on it to experience content in meaningful ways, both personally and professionally.
Historically, these descriptions have been a one-size-fits-all sort of deal. A narrator speaks during the natural pauses in dialogue, describing visual elements for a general audience using pre-scripted audio. For blind and low vision users, the result is a passive experience that often falls short of capturing the full richness of the content.
Enter the I4AD project. Led by Khang Dang, a Ph.D. candidate at the New Jersey Institute of Technology, this initiative brings an innovative and user-centered approach to audio descriptions (AD) that leaves me wondering how we ever managed before sliced bread.
So, what is I4AD, and how is it carving out a new and refreshing approach to the old, static format? According to Khang, who presented his work at the XR Access Symposium on July 27, 2025, I4AD is “an Immersive, Interactive, Intelligent, and Individualized Audio Description framework co-designed with blind and low-vision users and audio description professionals.” While many accessibility efforts fall short by excluding disabled users during the research, design, and development phases, Khang’s approach included them from the start. He gathered input through 102 focused surveys and 58 in-depth interviews. Six audio description professionals were also involved in the co-designing process. Together, they developed what Khang describes as “not just as a system” but “a collection of diverse perspectives from blind and low vision people on how to make virtual reality musical performances accessible to them.” When applied to the world of extended reality (XR), which encompasses both virtual and augmented reality, their combined input becomes part of the experience itself, transforming it into a customizable and immersive art installation of sound.
Khang’s love for musicals was part of the driving force behind I4AD’s creation, and during the pandemic, there was a noticeable uptick in virtual reality concerts. This further fueled Khang’s inspiration to reimagine how blind and low vision audiences might more fully experience moments that “rely on visual spectacle.” He explains, “I’ve always loved musicals on the screen […] but not everyone gets to enjoy musicals in the same way. […] Take Miss Saigon, for example. Audiences are often stunned by the iconic helicopter scene. Or Back to the Future: The Musical, where the DeLorean car doesn’t just travel through time, it actually lifts off the stage and flies over the audience.”
For mediums like virtual reality (VR) and musical performances that are largely visual by nature, it’s hard to conceive of a way to bridge the gap for those with visual impairments. Despite the growing popularity of VR experiences, only 25 percent of the blind and low vision participants surveyed had ever tried virtual reality. Even more telling, 68 percent had never watched a musical on screen, and 34 percent had never attended a live performance. That lack of exposure was not due to disinterest. In fact, while just 28 percent had even heard of VR musical performances, 96 percent said they would love to attend if given the opportunity. As Khang puts it, “The excitement was there. The access wasn’t.”
Participants described several key barriers, and Khang utilized personas to cohere these diverse perspectives into focus, crafting quotes that offer insight into a range of blind and low vision user experiences. Many respondents pointed to an inability to personalize the experience and a lack of spatial information.
“A sense of orientation is really important when you have vision loss.”
Others noted that sound often overlaps in these performances, which leaves little to no room for audio descriptions to exist at all.
“It’s difficult to add audio description in between lyrics or music.”
To build on that, participants expressed varying needs when it came to the amount of information presented. Some preferred detailed, richly descriptive narration. Others found that short, concise explanations conveyed just enough to follow along without overwhelming the experience or causing aural fatigue.
Khang’s approach to gathering information raises the standard for inclusive user research. While not overtly stated, his process makes a compelling case for including disabled users from the very beginning, and that commitment shines through in the solutions he set forth. At this year’s XR Access Symposium, Khang presented a live demonstration of his research in action, offering solutions built with BLV users in mind:
- Spatial AD: The source of audio descriptions is attached to moving objects and performers, shifting in space to provide spatial awareness and a form of 3D mapping. So wherever the dancers move, the audio description follows.
- View-Dependent AD: Descriptions are dependent on the direction the user is facing, giving them only the information that’s right in front of them.
- Explorative AD with AI-driven Q&A: Users have the ability to pause a performance and ask questions. The AI listens for voice commands and analyzes scene snapshots. Users can ask for details about what might be happening around them, or for a description of what something looks like, on demand.
- Individualized Results Based on Personal Preferences: The amount of detail, pronunciation, voice, and emotional tone of AI responses can all be customized. Users can even specify the kind of visual information they care about most.
Khang is currently putting these solutions to the test through paid user studies at the New Jersey Institute of Technology in Newark. His goal is not just to improve audio descriptions for VR musical performances; he is working to reimagine what participation looks like for blind and low vision individuals. By enhancing spatial awareness and tailoring the experience to each user, Khang aims to create immersive VR spaces where blind and low-vision participants can engage just as fully as sighted ones.
Khang’s work is not only about access. It is about artistry, interaction, and agency. He is one of many who are contributing to a quiet revolution. Hopefully, it will take center stage and begin to close the gap, not just in access but in experience.
About the Author
Kristen Reed is a Teach Access Student Ambassador and Student Academy alum, currently interning with AFB’s Talent Lab. A late-diagnosed Autist and psychology student at Western Governors University with a foundation in digital design and web development best practices, Kristen aspires to embed accessibility into the early phases of design and development through inclusive user research. They’re passionate about advocacy and building digital spaces that work for everyone.
About AFB Talent Lab
The AFB Talent Lab aims to meet the accessibility needs of the tech industry – and millions of people living with disabilities – through a unique combination of hands-on training, mentorship, and consulting services, created and developed by our own digital inclusion experts. To learn more about our internship and apprenticeship programs or our client services, please visit our website at www.afb.org/talentlab.
