Multimodality : A Dissection

Snow Fall is a multimodal project put together by the New York Times.  The article itself is about an avalanche in the Washington Cascades that occurred in February of 2012.  16 people were involved, 13 survived.

What separates this article from most is the presentation.  Unlike standard news stories that include block text occasionally sprinkled with a photo or two, this article was written using HTML5 technologies including audio and video.  But it goes beyond merely adding a few pictures and videos.  Animated images are used as headers for each page, highlighting the main themes of each part of the article.  Videos are placed alongside of the text, adding additional insight to key sections of the story.  Images within the article are actually slideshows.  Clicking on them creates an overlay over the article, allowing the reader to click through a larger version of each image.  Captions run across the bottom of each image adding additional information on what the reader is viewing.

In some sections of the article, images and videos are used to explain concepts.  For instance, the article talks about new safety equipment such as air bags :

Similar advances in safety gear, such as easy-to-use digital beacons and air bags, have helped make the backcountry feel less dangerous. Beacons help rescuers find people buried under the snow, while air bags deploy a large balloon meant to help keep the skier closer to the surface of an avalanche. A leading American manufacturer of safety gear is named, appropriately, Backcountry Access.

That explanation, however, doesn’t really convey what these air bags are.  What kind of a balloon?  Are we talking about a typical balloon we see at birthday parties?  Maybe they mean a weather balloon.  How does the balloon deploy?  Is the skier encased inside of it?  Does it surround them?

All of these questions can be answered in the text, but it may take a lengthy explanation.  Instead, the writers inserted a video, clearly showing the reader what is meant.  In fact, it not only shows what is meant by “air bag,” but also shows a typical deployment of the air bag.  This simple video, displayed off to the side, provides an immense amount of insight without having to read pages of explanation.  That video can be accessed at the following URL : http://graphics8.nytimes.com/packages/flash/multimedia/bundles/projects/2012/AvalancheDeploy/airbag_final.webm

Other elements used throughout the article add other insights.  Images and audio of the victims let the reader see and hear that the victims are real flesh and blood as opposed to just a written name in an article.  Images of the mountain, both static and animated, grant the reader a view of the terrain the victims were navigating.  Additional interactive pieces change as the reader scrolls through the text, offering a visual representation of the commentary within the article.

While the text itself contains the majority of the information for the piece, it uses related visual elements for additional explanation and immersion.  Links within the text are used to indicate related visual elements as well as activate those elements when the links are clicked.  This brings the textual and visual elements together to reinforce concepts and build a more complete picture of the overall story.

As a whole, this is a pretty amazing article.  The combination of technologies used helps to immerse the user in the story.  It’s both an informative story as well as a work of art.  I imagine we’ll see more articles like this moving forward, though I don’t believe these will be created as a normal course of daily reporting.  The immense amount of effort required for each article is prohibitive.