Thursday, November 4, 2010

PDC 2010: Using Silverlight and Azure to Give Global Reach to a Local Conference

When Microsoft announced that this year's Professional Developers Conference would be limited to 1,000 attendees and held on the Microsoft Campus in Redmond, some were surprised at the departure from the typical conference site at the LA Convention Center and the smaller than usual limit on the number of attendees. In fact, the 2010 PDC turned out to be the largest PDC that Microsoft has ever done. By making use of Silverlight and IIS Smooth Streaming technology, Microsoft was able to extend the traditional reach of the PDC to a much larger global audience than had ever previously been possible.

Vertigo is proud to have participated in this groundbreaking event by building an interactive Silverlight Media Framework-based video player and PDC conference application that allowed the widest level of participation ever for a PDC. The online consumption numbers that Steve Ballmer mentioned in his PDC Summary are pretty staggering, considering that only 1,000 people physically attended the conference:
"In addition to more than 30,000 developers at 250 PDC events worldwide, another 100,000 developers viewed the event online using Silverlight, with 10% of the online audience taking advantage of simultaneous translation into Japanese, Spanish, French and Chinese. This is incredible reach."

Beyond this amazing breadth of reach is the depth of participation that the online PDC experience enabled. By integrating real-time Q&A, polls, and Twitter conversations, users watching remotely don't just watch remotely – they actually get to participate in the sessions as if they were right there in the room. Additionally, international users enjoyed simultaneous translation into 5 different languages to allow a level of understanding of the content that may not have been possible outside the online experience.

Let's take a look at some of the features of the application:

Live HD Streaming of 2 Video Feeds per Session with DVR Capability
The PDC player shows 2 synced up video feeds simultaneously: the "slide deck/code view" feed, which shows what the presenter is actually doing on the presentation computer, and the feed showing the actual presenter in a smaller video rectangle. These feeds can be swapped so that the more interesting feed can be seen in the larger video window at any time. Users enter live presentations at the live point, but can DVR back to any point in the presentation they wish via the timeline. Users can also use the DVR controls to jump back 15 seconds if they missed something, or to rewind or fast forward while watching the video content.

Live Multi-Language Audio
In order to extend the reach of the PDC internationally, Microsoft Studios organized a phalanx of translators in their facility during the conference to provide live audio translation in 4 languages (Chinese, Japanese, French, and Spanish) for each of the 4 sessions – that's 16 different translations happening simultaneously and live. The player allows users to switch seamlessly to an alternate audio track to listen to these translations. This is a huge feature, as it allowed international viewers to understand the content being discussed in a way they may never have been able to except in this unique online experience.

Live Closed Captioning

The complement to live audio translation, live presentation of Closed Captions for each of the 4 simultaneous sessions (again provided by the folks at MS Studios) provided a way for users with hearing disabilities or those users who did not have the ability to play audio in their environment to consume the content.

Sharing/Deep Linking
Each of the PDC's sessions can be shared out via Twitter. In addition to this, a specific location within a video can be deep-linked to using the bookmark feature to allow users to direct their colleagues to specific segments of a video covering a topic of interest. For example, here's a deep link to Eric Schmidt discussing the PDC online experience on Channel 9 Live (this is a great discussion of what it took for all the partners involved to make the PDC online event come together). The Channel 9 feed is a 7 and a half hour video, but the bookmark/deep linking allows me to single out the topic within the Channel 9 show that I'm interested in.

Multiple Content Delivery Networks (CDNs)Because the PDC's online video content is distributed internationally, the team set up a system to identify which CDN would provide the best video delivery experience to the user and have the player select that network as the video delivery endpoint. For instance, users in China were directed to stream video cached by ChinaCache. Users in the United States currently pull video content from Windows Azure's CDN, and live feeds were delivered by Akamai.

Real-Time Polling and Q&A
A key goal of the PDC online experience was to allow users to engage with the live sessions and participate in the same way that a user in the same physical room as the session would. To this end, the player allowed users to ask questions and receive back answers, and to participate in polls initiated by the presenter of the session. These were driven by services running on Windows Azure to ensure scalability for a large number of users.

The final piece of the puzzle in bringing an online conference into social parity with the experience of physically attending the conference is to allow attendees to engage with other attendees, and to foster discussions and conversations among the audience about the material. Each session at the PDC was given an identifying hash tag, and users could engage in a conversation with each other about the material in real-time as it was being presented via Twitter from within the PDC application. An advantage to the online approach is that these discussions can happen in parallel with the session itself without causing a distraction or interruption to the session (and anything a user might miss while conversing can simply be re-watched by DVR-ing backwards).

Ratings and Metrics

Another key goal of the PDC online experience was to track metrics data about both the health of the video ecosystem (number of video failures, number of video starts, etc.), and the value of each session's content to the users. At the conclusion of each session, users are asked to rate and comment on the session. Here's the cool part: that feedback is then consumed in real-time by the PDC application and used to guide other users to what the community considers to be the most important content. More on this in the next section:

The PDC Now page allowed for a central hub and landing page for the application that guides users to the most valuable content. Live content was always featured front-and-center, but the deep value of this page lies in its display of the ratings and metrics data gathered from all of the active players on the internet. As you can see above, users can browse content based on data such as the most popular or most watched videos, or view sessions based on which were the most highly rated by viewers. The player also displays data about the number of users currently viewing each session (see the yellow arrow above).


The guide section (of which both a Silverlight and HTML version were built) provided additional ways for users to explore content – by the PDC schedule, by session, or by speaker. While browsing for content on the Guide or PDC Now pages, the active video simply minimizes to the bottom-right corner and continues to play with audio. This allows users to explore content without losing track of the information being presented in their active session.

Skip Into

Each live session begins with a few minutes of video slate while the encoders are fired up but the session has not yet started. However, we don't want users to have to sit though that slate once the session becomes Video on Demand content. We also don't want to spend time editing out the slate, so a solution was devised to allow the player to use a "Skip Into" time to begin the video at the exact time the session actually starts.

Download MaterialsUsers can access all the relevant Powerpoint decks as well as downloadable WMV copies of the session videos in the Download Materials section.

News FeedThe RSS-based news feed allows users to keep in touch with the latest news from the conference.

Windows Phone 7 App

Vertigo also built a Windows Phone 7 app that allows smooth streaming video content to be viewed using the Silverlight Media Framework for Windows Phone. See Ben Riga's blog post for more details.

Dream Team
Pulling off an event as large and broad-reaching as the PDC online experience is a huge effort involving many partners. For a look at all of the partners involved in this effort and their roles:
  • iStreamPlanet
    Encoding and delivery of the live and on-demand video.
  • Built and operated the PDC online player application, schedules, and Windows Phone 7 app.
  • Creation and operation of the Content Management System that allowed all of the complex data about the event to be managed in real-time.
  • Inlet
    Manufacturer of the Spinnaker encoders used for the live video content.
  • Akamai
    Content Delivery for live streams.
  • Natively delivering content into China via their CDN.
  • Delivering all on-demand video via the Azure CDN solution.
  • MS Studios
    Provided closed captioning, audio translation, and origination of the video streams in Redmond.
  • MS DPE Team
    Organized all partners and deliverables and oversaw the PDC online event operation.
  • Provided support for smooth streaming technology.
  • Provided support for Silverlight technology used by the player.
  • Evertz
    Manufacturer of the Closed Captioning encoders.
I want to send a big thank you both to the talented designers and developers at Vertigo who helped the PDC application become a reality, as well as all of our partners on this project who worked very hard to make the streaming live video experience itself a reality.

Full Screen
Let's end it with a screenshot of what the player looks like in full screen, where the secondary video transitions to a Picture-in-Picture mode:

Wednesday, September 22, 2010

Sunday Night Football Extra 2010: Back and Better than Ever

I'm proud to announce the launch of Vertigo's latest live HD video experience: NBC's Sunday Night Football Extra 2010! You can check out the live action every Sunday Night this season.
We first worked with NBC last year, building the Sports Emmy®-nominated SNF Extra 2009 application. We've since partnered with NBC to build the 2010 Vancouver Winter Olympics video experience, as well as this year's US Open (golf) and Wimbledon live HD video players. Vertigo's team for this project has returned to Sunday Night Football again this fall with the ambition to design the most engaging live sports experience ever created for the web.

A great way to get a feel for the application in only a couple of minutes is to watch this video, which provides a quick high-level run down of all the features in the application. For some more depth, let's crack open the application, take a look at some of the goals we set out with, and discuss how we solved some tricky design challenges to create an industry-leading live video experience for the web!

Great looking, High Definition video
Using Microsoft's IIS Smooth Streaming technology and working with our partners at iStreamPlanet and Akamai, the video you see in this player will once again serve as the standard for clean, smooth, multi-bitrate, adaptive HD video on the web. The core video player is built using the open source Silverlight Media Framework, which Vertigo developed in cooperation with Microsoft.

Picture-in-Picture (PIP)

One of the major goals for this project was to make the entire application more engaging to users. What better place to start driving engagement than in the actual video being consumed? The PIP provides the following:
  • The PIP control allows viewers to see the action from an alternate camera angle while watching the simulcast feed in the main window.
  • The menu on the PIP provides a streamlined way to switch to another camera angle, or to swap the main video into the PIP and start watching an alternate angle in the main window.
  • Great viewing experience: This year, we decided to make the PIP a first-class video player in its own right.
  • Full Screen Scaling: To provide a great experience on televisions and large monitors, the PIP will scale up in size when users move into full screen mode. This keeps the PIP relevant, even from across the room.
  • Draggable: Don't like where the PIP is positioned? Drag it anywhere you'd like!
  • Collapsible: If you'd like a totally clean video experience, you can use the Minus button to collapse the PIP and remove it from your field of view.
The Ultimate Timeline
One reason that Sunday Night Football is such a fun event to work on is all of the interesting data we have to work with. One of the biggest challenges we faced was how to visualize all of that data without creating a noisy or distracting user interface. We also had to envision how to allow users to socially interact with all of this data, and how to create data of their own that might be interesting to their own social circles. Here's how we addressed these challenges:
  • Positional Thumbnail Preview: When mousing over the timeline, the player displays a preview image of what's happening in the video at the point you're hovering. This allows users to quickly browse an entire game and look for interesting action.
  • Play by Play Data: We use team-color-coded markers on the timeline to indicate when key plays occur. When mousing over one of the key play timeline markers, data from the NFL displays inside the timeline preview image indicating the details of the play that happened.
  • Deep Linking to Plays: Users can select any point on the timeline to share a deep link into that location within the video with their social network. This feature allows a user to brag to her friends about a big sack on a 3rd down on Facebook, or to tweet the location of a close-up shot of a player who attended her high school a few years ago on Twitter.

Flexible Video Modes

We have so much great functionality this year that we faced a major design challenge in deciding where to put it all! Job #1 of a good online video player is to provide big, beautiful video with minimal distractions and visual noise, but our audience of football fans also loves statistics, social interaction, highlight clips and updates from the sidelines. To have our cake and eat it too, we decided to create 2 video modes: "Big Video", and "Engage".
The application starts in Big Video Mode, where you get a large space to watch the action with optimal video size and quality.

Big Video Mode:

When you want to dive in to all of the additional content and fun features offered by the player, clicking the reveal button transitions to a (still very respectably sized) video window, exposing all the additional functionality. This approach allows interaction with the statistics and social features while still watching the video.

Engage Mode:

Multi-speed DVR Controls
To give our users full control over the action, we provide instant replay, slow motion, and 4x/8x fast forward and rewind DVR controls. Direct every play, see that last catch over and over, and even decide on that last penalty call for yourself!

Twitter Battle

When it comes down to it, football fans tune in to see a battle on the gridiron every Sunday. To let fans be a part of that battle, we included the Twitter Battle as a leading social feature.
  • Users are encouraged to Vote/Tweet for their favorite team.
  • "Tug of war" results are shown within the app
  • The Tweets contain links back to the player to garner larger audiences and add fuel the battle

Twitter Integration, Ask Mike Florio

Some of the most popular features in the 2009 SNF application were the interactive Chat with Mike Florio that allowed fans to see their questions answered in real time, and Andrea Kremer's tweets from the sidelines. However, those features were scattered across the application. To improve on the concept of interactive Chat with Mike Florio and tweeting with Andrea Kremer, we unified and aggregated all of the Twitter streams including: Q&A from Mike Florio, Andrea's tweets from the sideline, and even stand-out tweets from the fans in the Twitter Battle!

Highlight Clips

While SNF Extra 2010 gives users greater control than ever over their game experience, sometimes you just want a nice big thumbnail to take you to that last touchdown. NBC's editors can also use the Silverlight Rough Cut Editor to create drive cut-downs and other aggregated highlight content on the fly.


For those fans that love to keep up to date on all the details, the Statistics section provides in-game updates every 1 minute.

Team Effort 
I'd like to thank the entire team at Vertigo for all of your dedication, enthusiasm, and creativity in creating a beautiful, innovative piece of software. I'd also like to thank our partners at NBC Sports, iStreamPlanet, Microsoft, Akamai, Ascender, Inlet, and TweetRiver for your commitment and expertise that make the Sunday Night Football experience possible!

Monday, April 12, 2010

Vertigo @ NAB

Vertigo is in attendance at NAB, where we're talking about our recent work delivering High Definition Media content over the web. You can swing by the Microsoft booth to talk with myself or Mike Moser, where we'll be showing off some cool demos. On Tuesday, we'll run a quick tutorial every hour on the hour demonstrating how to build a SMF 2.0-based video player in just a few minutes.

Also – be sure to check out our video players running in Inlet and Omneon's booths demonstrating live ad insertion.

Thursday, March 18, 2010

Video from Vertigo's events at MIX 10

I just got back into the office after spending a week in Las Vegas for the annual Microsoft MIX Conference. Vertigo had a huge presence at MIX this year, where the media work we've focused on for the last year was on display. I personally had a great time, and I want to thank everyone who worked hard to make this conference a success!
Here's a summary of the videos on the MIX site that relate to Vertigo's work.

Vertigo was once again featured the main MIX keynote. A ~5 minute video about our Olympics player was shown about 6 minutes into it! You can see the keynote here: (jump to 6:30 into the video)
Scott Stanfield, our CEO, later went onstage and did a live demo of an app we built for Netflix for the Windows Phone 7.
Vertigo also built the live Smooth Streaming player that allowed users to watch the keynote live.

Smooth Streaming:
Scott did a great session with my colleague Mike Hanley, where they demoed doing a “Do It Yourself” Camera to Screen smooth streaming demo – pretty entertaining if you’re into video, smooth streaming, or even cinematography – you can watch it here:

Scott and I presented a session focusing on the work we did for the Olympics, how things worked behind the scenes, and why it matters. You can watch it here:
Jason Suess (the Olympics PM at Microsoft) did a presentation focusing on the possibilities that the new open-source Rough Cut Editor provides, and how we used it during the Olympics:

Silverlight Media Framework:
Eric Schmidt ran a presentation on the Silverlight Media Framework that Vertigo has developed and released on CodePlex:

Friday, January 15, 2010

Olympic Error Handling and Fiddler as a Remote Proxy

As we're nearing the opening ceremonies for the 2010 Olympic Winter Games, our team is busy testing failure and recovery scenarios. We want to ensure that even if something goes wrong in our video delivery stream, users will have as seamless an experience as possible, and will have their video quickly and automatically restored. In the event that a catastrophic failure occurs in the video delivery stream, we make several attempts to automatically re-connect to resolve the problem:

Failing that, we provide a means to manually retry the reconnection attempt, or to simply browse to other video content.

In order to test the different scenarios that can lead to these kinds of errors, we often use Fiddler as a debugging tool. We often use the Autoresponder feature to return "502 Unavailable" responses to requests for valid, working video chunks in order to trigger these error scenarios for testing purposes. The other day, I encountered a situation where I needed to trigger an error in full-screen mode. As you may know, in Silverlight full screen mode, clicking on another application outside the full screen Silverlight app will exit full screen mode. I needed to trigger a video failure via Fiddler in full screen mode, but could not use Fiddler without exiting full screen and invalidating my test.

The solution to this problem is to use Fiddler on another computer as a remote proxy. This allowed me to do anything I wanted on my main computer, controlling the responses to the network traffic on another machine. Thanks to Olga Lepekhina and David Woods for pointing me to this solution.

To make it work, open up Fiddler on another computer on your network. Open up Tools -> Fiddler Options, and open the Connections tab. Check the box for "Allow remote computers to connect", and restart Fiddler:

On your main computer, go to Internet Explorer's Internet Options and click the LAN settings button on the Connection tab. Check the "Use a proxy server for your LAN" box. You can then enter the IP address of the other machine (the one running Fiddler), along with port 8888. (You may need to click the Advanced button to enter these settings).

This works in Firefox too, via a similar procedure: