Tuesday, March 29, 2011

Sneak Preview: 3D Video Support in the Silverlight Media Framework

I'm excited to announce that the upcoming 2.5 release of the Silverlight Media Framework (also known as the Media Platform Player Framework) will include support for playback of stereoscopic 3D video content. This powerful new functionality will allow developers to easily integrate 3D video content into their player experiences, and will allow dynamically changing, enabling and disabling 3D mode based on an end-user's preferences.

Content providers can now provide a single encode of a stream, which the player can then dynamically display in a range of different 3D modes or standard 2D mode based on the user's preferences (and hardware/3D glasses availability). This dynamism provides a powerful tool for online broadcast of 3D content in that it does not leave any users out in the cold, unable to consume the content. You can now reach your entire user base, with 3D as an optional feature, with only a single video source!
Here's a sneak preview of what will be included, as well as some background on how it all works:

What's Included
  • Anaglyph Stereoscopic 3D video plugin, packaged with the SMF 2.5 source code
    • Support for Anaglyph 3D, Greyscale 3D, Left eye only modes
  • NVidia 3DVision Active Shutter Stereoscopic 3D plugin available for separate download
  • Support for 3rd party Stereoscopic 3D plugins
    • Tools to build your own Stereoscopic 3d implementation
  • Simple implementation of 3D support for consuming applications:
    • A new "S3DProperties" property has been added to the PlaylistItem class
    • Simply set up the appropriate S3D Properties, add a reference to the S3D plugin you'd like to use, and 3D will work automatically!
  • Stereoscopic 3D supported for:
    • Progressive Download (WMV, MP4)
    • Smooth Streaming
  • Four new sample applications will be included in the Samples project to demonstrate how to use the Anaglyph 3D plugin:
    • Simple Anaglyph 3D example
    • Multi-mode Anaglyph 3D example (toggling 3D on and off, changing modes)
    • 3D support via HTML playlists
    • 3D support via JavaScript playlists
  • Full documentation will be available on http://smf.codeplex.com when SMF 2.5 is released
Building Your Own S3D SMF-based Video Player
The documentation to be released with SMF 2.5 will include full instructions on how to include 3D video content in your player, complete with sample applications. For a peek at how easy the setup will be, here's a quick how-to video: http://ecn.channel9.msdn.com/o9/content/smf/howto/v2/videos/smf3d.wmv
The code generated during this video will also be available on Codeplex on the release date.

Writing your own S3D Plugin
The SMF 2.5 documentation will also include details on how to build a Stereoscopic 3D plugin of your own that will integrate with the new I3DPlugin interface.

How Does Stereoscopic 3D Work? 
Three of the most widely used 3D video display technologies are passive complementary color anaglyph (just referred to as "anaglyph" in this post), passive polarized, and active liquid crystal shutter, each with pros and cons. Let's take a quick look at all three, and discuss how they work in the Silverlight Media Framework:

The anaglyph approach to 3D uses two complementary colors as a filter for each eye. Most often, red and cyan are the colors used, and these are the colors currently supported by the SMF's Anaglyph S3D plugin. An anaglyph image (below) is displayed such that the left eye will view only red colors in the spectrum, and the right eye will view only blue and green colors (which combine to make cyan) to create the 3D effect.

The SMF's anaglyph plugin creates an anaglyph image by taking a base video source with the left eye and right eye images side by side or top and bottom, and uses Silverlight pixel shaders to apply color filters to the images and superimpose them together to form a single image. We based the red-cyan anaglyph formula on Peter Wimmer's anaglyph forumula – we used the ITU BT.709 color matrix coefficients instead of the ITU BT.601 coefficients used by Peter.
  • No special hardware is required to display the image
  • The glasses are very cheap (paper glasses can be purchased for less than 50 cents online)
  • The technology has very wide reach, and makes it the most common 3D display technology for images on the internet.
The major disadvantages to anaglyph display are around the quality of the image being viewed:
  • The left and right images are displayed superimposed on top of each other.
  • Due to the color filters, color resolution is lost. One technique to counter the color loss is to display a grayscale or monochrome anaglyph image: a colorless image suffers no color loss. The anaglyph plugin in the SMF supports a grayscale option – the pixel shaders have a modified algorithm that will turn a video stream with color images into a monochrome image before applying the red-cyan color filter.


You probably experienced polarized 3D the last time you saw a 3D movie in a theater. Linear polarization (as used by IMAX), and circular polarization (as used by RealD) work in a similar manner: the left and right eye images are superimposed through polarizing filters. In linear polarization, the polarized lenses in the glasses are oriented 90 degrees offset from each other (one vertically, and one horizontally), and in circular polarization one of the lenses is mounted in reverse (the advantage being that you can tilt your head and still maintain left/right separation).

The Silverlight Media Framework does not currently have support for polarized 3D, since most users don't have interlaced polarized displays (LCD computer monitors such as this one). However, 3rd party developers are free to create a 3D plugin for interlaced polarized displays for the SMF. The approach would involve using pixel shaders to convert the frame compatible images into interlaced images.

  • The color loss with anaglyph is eliminated using polarization, and the technology works well on the "silver screen".
  • This achieves a superior image quality vs. anaglyph, though there is still resolution loss.
  • The glasses are more expensive than anaglyph glasses, and special projection hardware is required to make this work.
  • This limits the versatility and reach of this technology, particularly with online video!
  • Polarization only works with silver screen projection (such as in a theater) or interlaced polarized digital displays. Silver screens are inconvenient for home use and interlaced polarized displays are very rare.
Active Shutter

The anaglyph and polarized approaches are considered "passive" because to the glasses themselves don't have any active pieces. In contrast, Active LCD shutter technology uses powered glasses that filter each eye differently at different times. NVidia's 3D Vision solution uses a 120Hz monitor to present the left and right eye images on alternating frames, while still maintaining a full 60Hz signal per eye. The glasses shut out the image to the left and right eye every 1/120th of a second, syncing with the image on the monitor via an IR transmitter.
NVidia worked with Vertigo to create an active shutter 3D Vision plugin for the Silverlight Media Framework, which will be available on NVidia's site.
  • Full resolution per eye, since the images are not superimposed.
    • This allows full 1080p HD images to be viewed without any resolution loss, creating great looking video.
  • No restrictions on viewing angle
  • Works on any PC with a recent (~5 years old or younger) NVidia GPU and 120Hz monitor and an NVIDIA 3D Vision IR emitter
  • Powered glasses are more expensive than polarized or anaglyph
Personally, I have to say that the NVidia Active Shutter technology is the most stunning 3D experience I've viewed so far – 3D video looks so much better in full resolution HD!

Sample 3D Video Content
Finally, Microsoft and NVidia have made the following sample video content available for developers:

Have fun
3D is exploding onto consumer devices like the Nintendo 3DS, and I expect to see a big jump in 3D video online over the upcoming years. I'm very much looking forward to seeing the applications consumers of the Silverlight Media Framework create using this new addition to the SMF!

Monday, March 7, 2011

Social Marketing of Premium Online Media Experiences

Here at Vertigo, we often come across "Social Integration" as a feature request for an application, particularly in our premium online video experiences. All too often, we see apps on the web that treat social integration as a checkbox on a feature list: Did I provide a link out to Facebook and Twitter? Check!

What these basic social implementations often lack is a broader understanding of the client's overall social media and social marketing strategy. Companies with a clearly defined social marketing strategy understand that there's more to social media than just allowing your users to link to your application.
We've been fortunate to work with many clients who understand that the true power of social media lies in building tools that empower your fans to participate in the creation of your content, creating a two-way conversation that doesn't exist in the world of traditional broadcast media. They also understand that this level of participation and engagement has direct and measurable marketing benefits and impact on the monetization of their content.

Conan's take
This article in Fortune describes how Conan O'Brien discovered the power of social media in reviving his career. His show on TBS is now considered to be at the forefront of online social-savvy: clips from the show are released on YouTube to generate interest (a move that most players in the broadcast world oppose, taking licensed shows off of YouTube), and each show warrants a new Twitter hash tag. The TV audience and the live audience at the taping participate in a real-time conversation on Twitter and actually get to participate and drive the content:
"Between his bits, O'Brien would come backstage and ask, "How's the tweets? How's the audience?" By reading the hashtag stream, Bleyaert recalls, O'Brien and his team could see, for example, that "some guy in the fifth row was using Twitter to try and pick up a 'girl in the white hat, three rows in front of the stage,' " and O'Brien would instantly incorporate that into his next bit."
Conan's advertisers and guests promoting their books/shows/etc. also reap the benefits of this social engagement – when you make a guest appearance on the show, you gain access not just to a TV audience, but to a massive online social audience as well.

Bringing participation to premium online video experiences
Vertigo has recently had the opportunity to build premium online media experiences that take a similar approach:
  • Focus on engagement
  • Give your fans a megaphone to tell the world what they think
  • Allow 2-way participation in the content being viewed
  • Provide tools that make the content more relevant to specific users
  • Allow rapid creation of clips to provide shareable content while it's still relevant
Some examples:

NBC's Sunday Night Football Extra 2010 and NHL Extra 2011:
NBC Sports has done a great job driving forward a modern social media strategy via their online companions to sports broadcasts. Here are some of the features we worked to build with NBC to drive greater engagement from their fan base:
  1. Allow users to have a (moderated) conversation with each other via Twitter within the application
  2. Allow users to participate in an ongoing conversation throughout the game by asking questions of Mike Florio from Pro Football Talk
  3. Provide a way for fans to wear their colors online by voting for their team in a Twitter Battle, driven by Twitter hash tags.
  4. NBC posts highlight clips of key plays within a minute or two of their happening live (via the Silverlight Rough Cut Editor). This allows fans to very quickly share out that bone-crunching tackle while the content is still hot and relevant.
  5. Built a preview window that allows fans to share specific plays of their choosing with their friends. This empowers users to find the content they care about and drive what they discuss online, not restricting them to editorially-selected clip content. This is content that is relevant to each specific user, and provides a tool for users to create their own take on the content, allowing another avenue for participation in the event.
Microsoft's PDC 2010:
The 2010 PDC event moved a large physical conference almost completely online. Microsoft recognized from the outset that to enable the level of networking and personal connections that happen at a physical conference, an aggressive social media strategy would need to be put in place. To help Microsoft achieve that goal, we built the following features in the PDC 2010 experience:
  1. Added a Twitter hash tag per session to allow users to discuss the content live in real-time
  2. Allowed real-time Q&A and Polling to allow the online audience to interact with the presenter and affect the content being presented.
  3. Quickly created clips and on-demand content with the Rough Cut Editor.
  4. Allowed bookmarking of specific locations in each session to allow users to share and discuss content relevant to them.
More engagement = more value
At the end of the day, what is the intrinsic value of all this additional participation and engagement? The answer is that it creates far more involved users. Those users will watch more video, and generate more revenue. They will also become evangelists of your content, drawing in a much larger overall audience (and again, more revenue). Moreover, those users will transcend the concept of a user, and become fans. Those fans will become a built-in marketing base (and marketing mechanism via the megaphone you gave them) for any future projects that follow similar participatory models. Finally, just like Conan's guests gain access to a huge social following by appearing on his show, the content and advertisers in your video experience will gain exposure to your social audience, which has a great deal of inherent value. An example here is NBC's Mike Florio, who now has a daily online broadcast, Pro Football Talk Live, that complements his appearance on the weekly Sunday Night Football online broadcasts.

You're not in control
One of the hesitations I often hear expressed about opening up your content to your users and fan base is the loss of control. What if my fans want the content to go in a different direction than I do? In my opinion, you're better off at least knowing that this is what your fans want, and having a 2-way channel available to discuss it with them. If you want to carve a bold vision and move in a direction you think your fans don't want now, but will like eventually, you have the tools to engage with them and let them know your thinking. And otherwise, maybe your fans have a good idea you haven't thought of: Betty White may not come to mind as a Saturday Night Live host, but perhaps listening to your online fans will net you a big success, as NBC did with the Betty White Facebook campaign.

Another concern is that if you open up the discussion to the world, people who don't like you can tarnish your image. While there are tools that can help mitigate blatantly offensive content appearing on your site, you'll probably have to accept a bit of risk that negative feedback will be made public. My opinion here is "that's OK". You're never going to please everybody, and the value of having a public discussion about your content most likely outweighs any negativity out in the ether. Furthermore, this kind of feedback can serve as a type of analytics system. If your fan base passionately speaks out about a buggy feature or a content decision they dislike, at least you know it! If you're committed to the path of building a great experience your users will love, you should have faith that your efforts will pay off and the love will drown out any negative feelings in the public sphere.