[280/366] video game audio description

[cosmolinguist]

This article shows every sign of not being written by a visually impaired or even disabled person ("Audio description, as a concept, has been a fixture in movies and TV shows for decades" ("decades"?)..."those who are either blind or visually impaired ("or"?)...keeps calling things "annotated" rather than described..."It is strange to consider that for decades, publishers scarcely paid mind to the reality that there are many avid gamers who do not see very well, or are hard of hearing, or have difficulty manipulating a joystick") but other than that, I think it's really interesting.

Much as I love audio description, it never even occurred to me to expect it in video games (except from diffrentcolours, bless him, who tries to keep me from feeling left out by narrating what he's up to). It seems like a huge endeavor, because it is:

Games are dynamic. Players wander around open worlds at their own pace; they pick flowers, smelt ore, and discover Korok seeds. No two combat encounters are exactly the same. It's an onerous job to fill in the details of a two-hour movie or a season of television for visually impaired folks, but it's at least easy to storyboard and conceptualize. Doing the same for say, a 40 hour Assassin's Creed campaign plus sidequests and collectables would require reinventing the wheel entirely.

The idea that an audio describer could read and react to everything Ellie does as she skulks around Seattle? That would require an inordinate amount of time, effort, and problem-solving. Descriptive Video Works is a company that's used to getting a call from a client at the last minute. If they were to truly imbue a new game with a harmonious A.I. narrator for the blind, that would require their service to be a core priority for the developer -- long before they started generating enough assets for a trailer.

I don't know what's going on with that last sentence there -- stop saying "the blind" you weirdo, and also why is it an AI narrator all of a sudden? -- but it makes a realy good point: accoriding to this article a couple of new games have audio-described trailers and that's just like any TV/movie job for audio describers. The article even says DVW were called in at the last minute to do one of them. Audio description for movies and TV, and trailers, can be and is done as an afterthought tacked onto (what's considered) a finished audiovisual product (by a prevailing culture that doesn't care if it's accessible). That just will not work if a whole game is going to be described. The accessibility is going to have to be much more integrated.

I'm really curious to see how that turns out, which will take a few years at least. It intrigues me partly because I was surprised at how...emotional?...I got just glancing at all the accessibility options for The Last of Us 2. The game isn't my genre at all -- the one thing 2020 hasn't lacked is emotionally harrowing content! -- but in a way that was good because I could just be overwhelmed on one axis at a time, like I could think well, I don't care about this game so I can just care about these accessibility options. And I cared a lot: there were so many options that I actually wonder if part of my emotions were just that kind of anxiety I get when my brain is overwhelmed (this is a super-fun and super-useful thing it does, thanks brain).

But some of it was genuine emotion too. At the time I wrote "I've always thought of video games as Not My Thing but is that really an inherent fact about me or them or my relation to them? Or is it just that everything since Tetris has been too inaccessible? It's weird to think that a whole entertainment media I'd written off decades ago might not always be off-limits and thus uninteresting for me." Because "I don't like video games" has been sufficient for me to write off all video games since 8-bit that I had to stop and consider whether that was really true or whether I'd just never had the chance to find out.

And getting to have a reaction to something that no longer has to be mediated by a barrier, something there previously had been no point getting personally invested in because of that barrier...even just speculating that the barrier might be removed can be a hell of an emotional experience, as many disabled people could tell you.

Comments

[steorra]

I've wondered sometimes what audio-primary computer games might be like. Not just ones where visual-primary games are made accessible with an audio component, but where the audio component was primary and any visuals were supplementary.

If you know anything about this or have any thoughts on it, I'd be interested.

[cosmolinguist]

I'm not a great person to ask since I don't know anything about video games. :) I think I've at least heard of indie games that do that, but I can't imagine a big-budget one like those talked about in this article (Assassin's Creed, etc.) doing something like that, it'd presumably be seen as a "waste" of the graphics possibilities.

[barakta]

Ugh to "onerous" in terms of games, that kind of work is not that huge and I bet any money by audio describing it, they'll learn about things they like, or want to change or different perspectives.

I know when I've attempted audio description it's been a really useful 'thought exercise' as well as an active accessibility thing I believe in doing so everyone can access it... It's made me think "what are the important details" "what doesn't matter" "Should I put little Easter Eggs in like my friend Doug does in his ALT text?"... What am I describing? Why? Am I objective? Do I want to be objective (if it's just me and my mates?).

Fuck off ableist sightie twats with their onerousing and the blinding. Fuck off.

I need to write about some wonderful VI people I know (yourself included) and why your blindnesses are also important parts of you that matter and teach me stuff and add value to our world and...

[cosmolinguist]

Ugh I hadn't even really noticed "onerous" there, yeah you can see how annoyed I already was by that point. :) It obviously was written by someone who writes about video games rather than someone who writes about disability. Which I think is interesting in a "remind me how people think of us outside the happy little bubble I mostly live in these days" way, even as it is aggravating.

And it's really not onerous to describe movies and TV, which is actually what the writer is talking about there! It's a straightforward process that many people are skilled at now, it's even something people seem to do as a hobby/volunteer thing these days (at least in a Facebook group I'm in!). Cakling it onerous isn't just incorrect: it also reinforces the perception of AD as a nuisance and something it's fine to do without.

I'd be really interested to see what you'd write about the VI people you know and what you've gained from our perspective and everything. It has been interesting for me to watch this happen on Mastodon (which I use instead of twitter these days), it has a really strong culture of describing its images and from this people are learning, as you say, what's important vs. irrelevant and how fun Easter eggs can be. But because the image descriptions are (mostly) available to everyone, not just accessible via screenreader, it's also been interesting to see them learn that they as sighted people can benefit from this too. Sometimes the description helps explain a joke/meme/gif/reference they didn't get just from the visual, or when there's extra information there that benefits them. I think it really is good for everybody to think about the whats and whys and objectivity and stuff when they're sharing things.

[silveradept]

It seems like good audio description of a game would have to be tailored to the game type. I've seen things like the Games Done Quick sight-less plays of Ocarina of Time and games like Punch-Out!!! for Wii, but those seem to be about recognizing specific unique audio cues and executing a series of predetermined responses to those cues.

In addition to good sound design that gives someone an idea of what is going on, it seems like there would need to be specific modes that help a person line up what they want to do through audio cues, or to keep them moving toward a selected way point, and all of the text, action, and interface would need to be described so that someone can navigate the whole thing. I'm imagining a "descriptive mode" where a button push pauses the action and a description is pieced together about the important elements on the screen from recorded fragments and then given to the player with a question on the end of what the player wants to focus on or respond to, with the game then assisting the player by lining them up to interact with the thing they've chosen to focus on.

Which, as you say, would be something integrated into the game from the beginning, rather than something that is bolted on to the end.

[cosmolinguist]

Yeah it sort of sounds like going back to some old text-adventure tropses and norms and skills (in both writing and playing the games). I can see that working.

[silveradept]

I am amused at the thought of someone having to narrate "You are likely to be eaten by a grue."

For some games, a descriptive mode would be easier than others. I'm trying to think of how to do it in a high action game like Assassin's Creed where characters would be expected to perform daring feats regularly. The Eagle Vision concept that simplifies the world into friends, foes, and targets could help with that, but there's a lot of difficult platforming sections that rely on sight to achieve their cinematic qualities and their tension, so I'm not actually sure what to do about that, outside of a waypoint guide that vibrates to indicate when you're on line and some clever audio design so that a character says "the target is looking at me suspiciously. I need to get out of their line of sight." with the narrative guide promoting that. And, of course, added time for any quick time events to resolve because they are being described rather than seen.

[cosmolinguist]

Yeah, I think diffrentcolours is right where he says below that a described game would have to be much slower because of both bandwidth and latency. I think the culture shift to allow that and do it well might prove more difficult than the technical aspect. I'd really hope that the developers would be consulting with (ideally, sometimes even be if possible) visually impaired gamers, because I think what does and doesn't work for people who benefit from audio description may well surprise people who don't.

[diffrentcolours]

I guess there are two different things here. One is narrating dynamic action carried out by the player, for an observer - which sounds difficult but doable. I can noodle about how it's a matter of prioritising use of a single channel for audio-description by competing events, something like a Real-Time Operating System. It sounds like quite an interesting challenge.

The other is providing enough information via audio description to a player to allow that player to make decisions and take actions. This is a much harder problem because of both the bandwidth (it's slower to describe an image than to display it) and latency (the lead time between starting to describe a situation, and the player being able to make an informed decision on their next action). I don't think that'd be impossible, particularly not using the kind of dynamic "hint" based control systems of most games, but anything like a modern game would have to be drastically slowed to compensate. But if that's what it takes to make the game playable for more people, it's probably doable.

[cosmolinguist]

I hadn't even thought about audio description for an observer (probably because I'm so pathologically worried about AD bothering sighted people, but I'd think it'd legitimately be a distracion for a sighted gamer). Which is a shame because now I think it sounds really interesting and I want to exist so I can try it. :)

Yeah, an audio described game would have to happen a lot slower than modern games do otherwise. Which is pretty normal for blind people anyway -- I remember telling the MOSI staff whne I was doing a tour for some blind people there to allow us a bunch more time than they were expecting iut'd take, and then understand that it'd taken even longer than that -- so we're used to it. And I'm not the only kind of partially-sighted person who'd be able to glean a lot more informaiton even from the visuals if it was slowed down a lot, because that gives my slowprocessing brain time to catch up and that always helps me make more sense of things.