After working with it for just a couple of projects, I cannot see ever going back to a WMV source file. They just seem so... small. In contrast, I've been able to deliver multiple concurrent full browser 720p videos without killing the systems on which they are running. The videos start immediately, are more scalable, and just look better - this provides a much better user experience than the old "buffering" animation while you're waiting for content.

The basic process of getting adaptive streaming up and running wasn't too bad, once we figured out a hosting solution. We've had an account at Limelight for quite some time, and although they were telling us they supported adaptive streaming, when push came to shove, it turns out they really don't. It was always "just around the corner".

We did a little digging and found a local hosting provider here in Portland that fully supports adaptive streaming and is a Microsoft Gold Certified partner. We had been using them for a while for various hosting needs, and they have really stellar support - if you're looking for an inexpensive adaptive streaming solution, check out Opus Interactive. They know what they're doing and can get it set up fast!

Our early smoke tests worked out pretty well. We used some various example code we found online to access some existing smooth streaming content. The process was pretty straightforward:

1) Add a reference to the SmoothStreaming.dll in your application

2) Add

using Microsoft.Expression.Encoder.AdaptiveStreaming;

to your usings.

3) Create a new adaptive streaming source:

    AdaptiveStreamingSource MainSrc =
        new AdaptiveStreamingSource(MainVideo, new Uri
            ("http://video3.smoothhd.com/ondemand/Got_Imagination_(California).ism/Manifest"));

4) Assign the AdaptiveStreamingSource to your media element:

    MainVideo.SetSource(MainSrc);

Note that in step 3, we're pointing to the base file that Encoder 2 produced, and adding "/manifest" to the URI string. There is no file called "manifest" - you're telling the server which file to use as the manifest. The file being used as the manifest is an XML file that describes the available video and audio streams.

Fire up your application and you're good to go. We were able to read test streams from smoothhd.com and also had success in placing the Big Buck Bunny demo on our own hosting provider and pointing to that.

Life is good, until....

In creating some test streams for an application we were building, we starting running into trouble. Strangely, the code had worked fine for an application we just built, but broke on the very next application we worked on. The application was tossing out errors about reading the manifest file.

We did some digging and the root cause of the problem is that with Encoder 3, the format of the manifest file has changed. We had outsourced the encoding of the videos for the earlier project, and they had obviously utilized Encoder 2, which is why the code worked the first time through.

The change in the manfiest file format is no problem at all if you are using the Silverlight template from Encoder, because the libraries were updated there. However, if you have written a custom solution, you're going to need to update your code. This is both a good thing and a bad thing. It takes a little more code to make the streams go, but there appears to be a lot more stuff you can control.

Thankfully, it turns out not to be too difficult (in hindsight) to utilize the Encoder 3 adaptive streaming output in your own code, and the new libraries are also backwards compatible with the Encoder 2 manifest file format, so any existing streams you have encoded will not need to be re-encoded.

Here's how to use adaptive streaming with files encoded with Encoder 3:

1) Start by navigating to your Encoder 3 folder with Windows Explorer. On my systems, the path is:
C:\Program Files\Microsoft Expression\Encoder 3\Templates\en

2) The template you want is called "Graphing". In that folder, you will find a Silverlight application (XAP) file called "SmoothStreaming.xap". Change the extension to .zip, and open the archive up. You want to extract both the "PlugInMssCtrl.dll", and "SmoothStreaming.dll" libraries for use in your own projects.

3) In your application, right-click the project References, and pick "Add Reference". Locate and add both libraries.

4) Add

using Microsoft.Expression.Encoder.AdaptiveStreaming;

to your usings.

5) Next (you can code these however you'd like, this way just suited my needs), create variables for your video URI and AdaptiveStreamingSource:

        private string uriString = "http://video3.smoothhd.com/ondemand/Got_Imagination_(California).ism/Manifest";
        private AdaptiveStreamingSource MainSrc = new AdaptiveStreamingSource();

6) Now here's where it changes a bit. The AdaptiveStreamingSource has a MediaElement property, which needs to be assigned to your MediaElement:

MainSrc.MediaElement = MainVideo;

This is flipped from how it worked before. In the earlier example, the video source was set to the AdaptiveStreamingSource. Now, the AdaptiveStreamingSource has a MediaElement associated with it.

7) The AdaptiveStreamingSource also needs the location of the manifest file:

MainSrc.ManifestUrl = new Uri(uriString);

8) Finally, you can set up an event listener for "MainSrc.MediaElement.Loaded", and play the video once it has been loaded with the following code:

MainSrc.StartPlayback();

That should get the video going for you. I've run into a few quirks using the adaptive streams. The most troublesome was a "MediaEnded" event on a particular video that was raising 10 seconds or more before my video ended. This made it impossible for me to reset the state of the application when the video I was playing finished on its own. Still, that is a relatively minor issue that is easy to work around.

If you're interested in an example that uses this code, I've placed one online that you can download.

It's been so amazing to see how far the technology has advanced in such a short period of time, and to be able to be involved in that has been exciting. Looking forward to seeing where it continues to lead us.

In his blog, Tim covered the type of system he was using. I decided to go with something bigger, thinking it would simulate an interactive kiosk-type experience. That led me to the HP TouchSmart IQ526. This is the all-in-one machine with a big 22" touch screen.

Of course, it comes with Vista pre-installed. Windows 7 is needed to work with multi-touch, so my first task was to download and install Win7, and any necessary patches. I also grabbed software I would need for development, such as the Express Edition of Visual Studio. I figured I would stub out the programs in Visual Studio on my main laptop, and then tweak the touch functionality on the HP. I also grabbed Firefox and a couple of other tools.

It took me a while to get all the software installed, but as it was installing, I wrote up a test program on my main system. Once I got the system software updated, I tested the touch functionality. Windows 7 comes with an updated version of Paint that is multi-touch enabled. My son absolutely loved it:

Nick using Win7 Paint

After letting him do his thing for a bit, I finally got my turn on the system. Excitedly, I opened up Firefox and opened my application. Touched the screen, but nothing happened. Maybe I missed? Tried again, but no luck. Tried Tim's test program - same result. Nothing. After consulting with Tim a bit, there wasn't a real definitive answer as to why the system wasn't registering the touches. I didn't want to get stuck with an expensive 22" paperweight, so I made the decision to revert the system and return it, and instead picked up a system from the same line Tim was using. I ended up with a TouchSmart TX2-1270us.

As you would expect, the new system had Vista installed out of the box. Two things really hit me. 1) shoot, I really do have to reinstall all that stuff again, and 2) wow, that screen is TINY. At any rate, I was not deterred, so I started the installs over again. For this system, there were some updated (albeit beta) touch drivers from N-trig, the folks behind the touch technology on this system.

The installations finished up, and once again, I was at the moment of truth. I just *knew* it was going to work this time, so I excitedly launched Firefox and opened up my demo project. Touched the screen again, and.... nothing. Tried Tim's demo project, and.... nothing. In the middle of further email-based consultation with Tim, a thought occurred to me as to what the problem may be. I closed Firefox, and opened IE instead. Touched the screen, and there was the touch indicator. As it turns out, the public version of Windows 7 (build 7100) has a glitch that prevents the touchpoints from working in Firefox.

Now, I know what you're thinking, and while it didn't hit me immediately, I did wake up in the middle of the night in a cold sweat wondering if I had tried IE on the big 22" HP. I really don't know, but I'm not inclined to try and find out. For the record, the MSDN release of Windows 7 corrects this problem, but alas, I don't have an MSDN sub, so I am on build 7100 for the time being.

At any rate, the small laptop is very light, very portable, and very usable. It won't serve the purpose if we wanted to do an interactive touch kiosk in a retail space, for example, but for working through Silverlight 3's multi-touch functionality, it does the job.

I have written a few applications that use multi-touch, and the next few blog posts will describe the applications and how they work. Nothing fancy, mind you, but enough to dip into it and get a feel for how things work.

One thing that is immediately obvious is that touch is such a natural way of computing - you don't have to explain to a 4-year old how to use it. They just "get" it immediately. It also becomes habitual quickly - I sometimes find myself poking the screen on my Dell and then wondering why the computer just sits there staring back at me. And then I realize it's not a touch screen and I get mad because now I have fingerprints to clean off the screen.

I really think there is an opportunity here to deliver some really cool content over the web via Silverlight 3... now it's up to us as developers to find useful and interesting ways to implement touch.

I realize it's been fairly quiet on my blog for a while. I'm happy to tell you that it's because I was working through an update to my book, which is now finished. Foundation Silverlight 3 Animation has shipped to the printer, and should be available near the end of August. Here's a peek at the cover:

Hoping to get back to posting tutorials and tips soon!

Why wait? Silverlight 3 saved me a lot of trouble - the bitmap effects make perfect drop shadows with nice, soft edges, no imperfectly-shaped paths or duplicate text fields. I also get perspective transforms, so now I can build out the panels along a flat plane, and rotate the entire panel into 3D. One more new feature I used was the ImageCompleted event. No more image "popping" when a storyboard starts before an image is fully loaded.

The default view of the application is a gallery with 16 images in it:

Gallery Wall Main View

If no action is taken, the gallery will cycle through the images. If you select a thumbnail on the left, the image is displayed on the panel on the right for 20 seconds before the slideshow is resumed. If you would like a closer look at an image while the slideshow is running, you can click the image on the right and the panel will animate to an alternate view as shown following:

Gallery Wall Alternate View

If you have Silverlight 3 installed, the completed app can be seen here. Since the code is available at the end of this post, I figured I'd hit on a few points from the application, and let you dig up the rest if you're so inclined.

Reflections
My first approach with the application was to insert the images as object instances, so I could use just one object. However, since I had the element binding available to me, I decided to go the long way and actually create each image element to take some of the new features for a spin. This was done by creating a Canvas, and then placing an Image and Rectangle element inside. The Image element is used to hold the thumbnail, and the Rectangle element is used for a stroke highlight when the mouse is over the selected thumbnail. Notice that the image has a DropShadow effect on it.

 
<Canvas x:Name="canvas" Grid.Row="0" Grid.Column="0" Opacity="0">
	<Image x:Name="ImageTile_00" Source="images/msh640x480.jpg"
                        Opacity="1" Width="120" Height="93" Margin="5,5,0,0"
                        Stretch="Fill" Cursor="Hand">
		<Image.Effect>
			<DropShadowEffect Opacity="0.6" ShadowDepth="3"/>
		</Image.Effect>
	</Image>
	<Rectangle x:Name="StrokeHighlight_00" Width="120"
              Height="93" Stroke="#FFFFBA00" Opacity="0"
              Margin="5,5,0,0" StrokeThickness="2"/>
</Canvas>
 

Each of the thumbnail images is duplicated in the reflections container, and bound to its counterpart using the element to element binding of Silverlight 3. The code for the reflections container that matches the image container shown above looks like this:

 
<Canvas x:Name="Reflection" Grid.Row="0" Grid.Column="0"
    Opacity="{Binding ElementName=canvas, Mode=TwoWay, Path=Opacity}">
	<Image x:Name="ReflectionTile_00"
              Source="{Binding ElementName=ImageTile_00, Mode=TwoWay, Path=Source}"
              Opacity="{Binding ElementName=ImageTile_00, Mode=TwoWay, Path=Opacity}"
              Width="120" Height="93" Margin="5,5,0,0" Stretch="Fill" Cursor="Hand">
		<Image.Effect>
			<DropShadowEffect Opacity="0.6" ShadowDepth="3"/>
		</Image.Effect>
	</Image>
	<Rectangle x:Name="ReflectionStroke_00" Width="120" Height="93" Stroke="#FFFFBA00"
               Opacity="{Binding ElementName=StrokeHighlight_00, Mode=TwoWay, Path=Opacity}"
               Margin="5,5,0,0" StrokeThickness="2"/>
</Canvas>
 

Notice that I've bound multiple properties - for the Image, both Source and Opacity are bound. This means if I change a thumbnail image, the reflection image updates automatically. If I animate the Opacity property of the thumbnail image, the reflection image opacity also animates. The text caption on the right side panel was done the same way.

ImageOpened Event
There are two sets of storyboards in the project. One that handles the slideshow functionality, and one that handles the selected image functionality. Both work in essentially the same manner. Fade the image being displayed, update the image to the next one to display, and then show the image. I handled the last step in an ImageOpened event handler to ensure that the target image was available before begining the "show" storyboard.

For the change that occurs when the main image fades out, that event listener looks like this:

 
SelectedImage.ImageOpened += new EventHandler<RoutedEventArgs>(SelectedImage_ImageOpened);
 

My event handler handles both the case of a new image being loaded for the slideshow, or a new image being loaded by user selection. I flip a flag called StopSlideshow when a thumbnail is selected, and then test against that to shift to the appropriate storyboard set. The handler looks like this:

 
private void SelectedImage_ImageOpened(object sender, RoutedEventArgs e)
{
    if (!StopSlideshow)
    {
        ShowMainImage.Begin();
    }
    else
    {
        ShowForSelected.Begin();
    }
}
 

All of the images for the application are configured via an XML file called imagefile.xml, which is stored in the ClientBin folder. The format is fairly self-explanatory, but looks like this:

<image thumb="Images/adamsSunrise640x480_thumb.jpg"
url="Images/adamsSunrise640x480.jpg"
caption="Sunrise over Mt. Adams (from Mt. St. Helens)"/>

There were two things I did not code into this version of the application, which I will likely go back and add later. The first is the "spotlight" effect seen in the original application, which was done with an opacity mask. The reason why I left that off is that it doesn't work right in the current rev of Silverlight 3. I get an unmasked edge along the left of the panel. I haven't decided yet if I like the cleaner, brighter look of the updated app more.

The second is a simple test to check and see if the selected image is the same as the one being displayed. If it is, no action should be taken, but as it stands, the app still accepts input, and cycles out of and then in to the same image, which is not an ideal experience, but it's getting late....

Now, as promised, the code for this project can be found here.

You can download it here.

I had a conversation with Jesse Liberty at MIX09 this past week, and we talked about different techniques for animating various objects. Out of that conversation, I wanted to explore how to go about emulating a 3D object with the new Perspective Tools in Blend/Silverlight 3.

I spent some time putting together an application that simulates a 3D playing card rotation. If you have Silverlight 3 installed, you can see that application here. The project files are available at the end of this post.

The basic functionality for the project involves two images - a "card front" image, which in this case was the Jack of Clubs, and a "card back" image, which is a standard blue backed deck. The two images are aligned right on top of one another. The 3D perspective manipulations are enabled and bound to sliders as Corey describes in the article linked above. I added a bit of code in the code-behind file to "watch" for certain ranges and collapse the visibility of the "card back" image when the "card front" should be showing. There are two tricks involved with this.

First, the card face image is loaded and by default is scaled to -1X. This reverses the image. This is because the perspective tools actually flip around the image. If it was not scaled to -1X, the card front would actually show up backwards when it's rotated and made visible.

The second is the drop shadow bitmap effect I added. This one was a bit confusing - in Joe Stegman's session on the new features added to Silverlight 3, he made the point several times that bitmap effects were the last step before an object is written to the screen. If this were the case, the effects would not be affected by the perspective transforms, but they appear to be as the drop shadow flips along with the transform. At any rate, this was easy enough to solve. I grouped my card object into another canvas, and moved the effect off of the canvas containing the card images to the new container.

Let's take a look at some code. Following is the "CardEffectContainer" Canvas object, which is where the actual drop shadow effect is applied to the object. Inside of that is the "Jack_Clubs" Canvas object, which contains the Canvas.Projection used to make the card appear as though it is rotating. You can also see the "CardFront" and "CardBack" images. As I mentioned above, notice that the CardFront image is scaled -1X.

 
<Canvas x:Name="CardEffectContainer" Height="252" Width="185" Canvas.Left="300" Canvas.Top="110">
	<Canvas.Effect>
		<DropShadowEffect Opacity="0.5" BlurRadius="8"/>
	</Canvas.Effect>
	<Canvas x:Name="Jack_Clubs" Height="252" Width="185" IsHitTestVisible="False">
		<Canvas.Projection>
			<PlaneProjection x:Name="CardRotation" RotationX="0" RotationY="0" RotationZ="0"/>
		</Canvas.Projection>
		<Image x:Name="CardFront" Height="252" Width="185"
                             Source="cardFront.png" RenderTransformOrigin="0.5,0.5">
			<Image.RenderTransform>
				<TransformGroup>
					<ScaleTransform ScaleX="-1"/>
					<SkewTransform/>
					<RotateTransform/>
					<TranslateTransform/>
				</TransformGroup>
			</Image.RenderTransform>
		</Image>
		<Image x:Name="CardBack" Height="252" Width="185" Canvas.Left="0" Source="cardBack.png"/>
	</Canvas>
</Canvas>
 

After that, I have 3 Canvas object, each containing a Slider and associated TextBlock for display. Following is the listing for the SliderY control. Notice that the Value is using Binding to tie the Slider Control's Value to RotationY property in the "CardRotation" Canvas' PlaneProjection shown in the above listing. This makes it so I don't need to write any code in order to allow the Slider to affect the rotation value.

 
<Canvas x:Name="YRotationStuff" Height="37" Width="200" Canvas.Left="20" Canvas.Top="491">
	<Slider x:Name="SliderY" Width="200"
              Value="{Binding ElementName=CardRotation, Mode=TwoWay, Path=RotationY}"
              Minimum="0" Maximum="360" Canvas.Top="19"/>
	<TextBlock x:Name="SliderYValue" Text="Y Rotation: 0.00" TextWrapping="Wrap" Foreground="#FFFFFFFF"/>
</Canvas>
 

I'm not going to show the X and Z rotation sliders here in this post, but they look just like the Y Slider code shown above. In my code behind, I needed to watch the rotations in order to hide the card back at the appropriate times. This was done by setting up two event listeners as shown below.

 
SliderY.ValueChanged += new RoutedPropertyChangedEventHandler(Slider_ValueChanged);
SliderX.ValueChanged += new RoutedPropertyChangedEventHandler(Slider_ValueChanged);
 

Notice that both listeners call the same event handler - Slider_ValueChanged - which is shown following. This event handler starts out by updating the text visible to the user. I orignally had this property bound but wanted to do some formatting on the text, so pushed it to the code behind.

After that, you'll see a series of "if" statements that test ranges of values to determine the rotational state of the card. For example, the first statement checks to see if the card has been rotated along the X axis (tilting the card forward or backwards) more than 90 degrees, or less than 270 degrees.

That condition would result in the card front being visible (to visualize this, hold a playing card straight out in front of you and rotate it towards you - when you pass 90 degrees, you can see the card front, when you pass 270 degrees, you're viewing the card back again).

Since we're dealing with multiple axes here, we also need to check the Y rotation value. If the card has been rotated along the Y axis less than 90 degrees, or more than 270 degrees, then we know that the X and Y values are in a range where the card front should be visible, and we collapse the CardBack image from view. Easy, right?

You can see the other set of tests used to determine the card front visibility in the listing.

 
private void Slider_ValueChanged(object sender, RoutedPropertyChangedEventArgs<double> e)
{
    SliderXValue.Text = "X Rotation: " + String.Format("{0:0.00}", SliderX.Value);
    SliderYValue.Text = "Y Rotation: " + String.Format("{0:0.00}", SliderY.Value);
 
    if (SliderX.Value > 90 && SliderX.Value < 270 && (SliderY.Value < 90 || SliderY.Value > 270))
    {
        CardBack.Visibility = Visibility.Collapsed;
    }
 
    else if (SliderX.Value > 90 && SliderX.Value < 270 && (SliderY.Value > 90 && SliderY.Value < 270))
    {
        CardBack.Visibility = Visibility.Visible;
    }
 
    else if ((SliderX.Value < 90 || SliderX.Value > 270) && (SliderY.Value < 90 || SliderY.Value > 270))
    {
        CardBack.Visibility = Visibility.Visible;
    }
 
    else if ((SliderX.Value < 90 || SliderX.Value > 270) && (SliderY.Value > 90 || SliderY.Value < 270))
    {
        CardBack.Visibility = Visibility.Collapsed;
    }
}
 

That handles X and Y rotations, but what about Z? The Z-axis runs directly into your screen and would spin the card like a propeller (assuming the transform origin were in the center as it is here). Since the visibility of the card front is handled when the X and Y sliders are manipulated, all we need to do is update the feedback we're giving the user when the Z-slider is manipulated (remember that the value is bound through the XAML, so no code is necessary to update the actual Z rotation).

This is done with a simple event listener:

 
SliderZ.ValueChanged += new RoutedPropertyChangedEventHandler(SliderZ_ValueChanged);
 

and its associated handler, shown here:

 
private void SliderZ_ValueChanged(object sender, RoutedPropertyChangedEventArgs e)
{
    SliderZValue.Text = "Z Rotation: " + String.Format("{0:0.00}", SliderZ.Value);
}
 

From that, you get a fairly convincing 3D object rotation that was created from just a couple of images. Of course, a playing card is flat, so it's easy to manipulate in this manner. If you have all of the Silverlight 3 bits installed and would like the grab the project file, it's here.

The DeepZoom app contains a collection of posters that were put together by various researchers to describe their projects. There's some pretty interesting stuff in there. One of the ones I find especially interesting is Headlight, which is the 3rd poster down in the right hand column.

I was fortunate enough to be able to take Headlight for a spin, and it's a pretty slick add-on to Composer. The projects are very similar to Composer - you import images and lay them out, but you also have the chance to add tooltips, menus, and links to the items in the collection before doing the export. There is even an option to enable some metrics for items in the collection. The team working on the Headlight project doesn't have a public-facing site just yet, but keep this one on your radar - it might change the way you view ads.

The crux of the comments were to use a tool like ComponentOne XapOptimizer, which can be found here: http://labs.componentone.com/XapOptimizer/

I tested it on a couple of projects, and it seems to give me about the equivalent reduction in archive size as unzipping and rezipping the XAP package. It appears to be another great tool to add to your arsenal, although you may be able to squeeze out some additional space by optimizing the graphics in your application prior to utilizing XapOptimizer. Definitely worth checking out.