Next Project - Port SimpleViewer to Silverlight

Having run out of ideas on how further to exploit Deep Zoom, I am going to now focus my energy on porting the closed-source flash based SimpleViewer to Silverlight. Again, my objective is to learn Silverlight, so I will try to push some boundaries (if required). As part of this exercise, I will try to stick to the original goal of SimpleViewer, that is keep the download as small as possible. The current version (1.8.5) of SimpleViewer is only 17k and that is what I am aiming for. This, of course, means no using the extended controls, but only what is available in the core Silverlight 2 SDK!

So, why SimpleViewer? I actually use it in my personal blog and have always wanted more control over it. I also want to know what it takes to create the same experience in Silverlight. I am expecting this to be a trivial exercise, but let's see!

My first roadblock!

I wanted to keep everything as dynamic as possible. This means no explicitly specifying the height and width of any control. I was happy to see that the image control provides this capability and is able to work well within the constraints of MaxWidth and MaxHeight. This was all wonderful and perfect for my needs but now I wanted to draw a simple border around the image. Pretty easy...all I need is the dimensions of the image (control), but this is where it got a little complicated. The image control does give access to the dimensions but a little too late. I won't go into the details here but I did post it on the Silverlight.net forum is anyone's interested. By the way this is a great forum and one of the best places to get answers to all your Silverlight questions or issues.

What I discovered is that the image loaded event is not a reliable place to extract the dimensions from the image control. I tried a kludgey workaround by creating a timer and accessing the dimensions from the image control a little later - around 100 milli seconds - but this is definitely not reliable. But at least I had a solution!

While I was playing with it some more, I discovered the SizeChanged event. This also doesn't help if the image dimensions are extracted from the image control itself, but I discovered that the event itself has the size, and that is when I had my solution!

There might be some issues with this so please leave a comment if you find something. I am assuming this is a bug with the image loaded event so, hopefully, it will be fixed in the next release of Silverlight. But, for now, it looks like I can make some "not so kludgey" progress!

MyImage.SizeChanged += (sender, e) => {
    if (e.PreviousSize.Width == 0 && e.PreviousSize.Height == 0 && e.NewSize != e.PreviousSize) {
        // e.NewSize contains the dimension of the image
        // MyImage.Width and MyImage.ActualWidth is unreliable so use e.NewSize instead
        DrawBorderAroundImage(e.NewSize);
    }
};

Deep Zoom - Album Creator

No, this one is not by me!

Marauderz has developed a Deep Zoom Album Creator which uses some code from my blog so I thought I'd mention it here. I am glad to see my code being leveraged in some useful tools, hopefully more of which we shall be seeing soon.

How about some designers getting involved and skinning this baby?

Deep Zoom - SlideShow

I was wondering how best to showcase the functionality from my last post and the idea of SlideShow popped up in my head. Also, I was bored and wanted to play around with Deep Zoom some more...

Update Oct 14, 2008: Removed the example from this post to reduce load time of this page. Please use the example from this post, DeepZoom sample ported to Silverlight 2 Beta 2, to try the functionality.

Click on the Start Slide button to activate the slide show. Click on the button again to stop the slide show.

The code to enable slide show is very simple. Most of the work is done by the function described in my last post.

Xaml

<Button Grid.Column="2" x:Name="buttonSlideshow" Width="70" Content="Start Slide" IsEnabled="False"/>

Code

int _slideshowImageIndex = 0;
System.Windows.Threading.DispatcherTimer _myDispatcherTimer;

msi.Loaded += delegate(object sender, RoutedEventArgs e)
{
    _myDispatcherTimer = new System.Windows.Threading.DispatcherTimer();
    _myDispatcherTimer.Tick += new EventHandler(Each_Tick);
    buttonSlideshow.IsEnabled = true;
};

buttonSlideshow.Click += delegate(object sender, RoutedEventArgs e)
{
    if ((string)buttonSlideshow.Content == "Start Slide") {
        buttonSlideshow.Content = "Stop Slide";
        StartSlideShow();
    }
    else {
        buttonSlideshow.Content = "Start Slide";
        StopSlideShow();
    }
};

public void StartSlideShow()
{
    // Set the timer interval to 200 ms so that the tick event is called immediately
    _myDispatcherTimer.Interval = new TimeSpan(0, 0, 0, 0, 200);
    _myDispatcherTimer.Start();
}

public void StopSlideShow()
{
    _myDispatcherTimer.Stop();
}

public void Each_Tick(object o, EventArgs sender)
{
    // Reset the timer to 5 sec so that each slide appears after 5 seconds
    _myDispatcherTimer.Interval = new TimeSpan(0, 0, 0, 0, 5000);
    ZoomFullAndCenterImage(_slideshowImageIndex);
    if (++_slideshowImageIndex == msi.SubImages.Count)
        _slideshowImageIndex = 0;
}

Deep Zoom - Zooming and centering image to fill the Deep Zoom screen

One of the readers (see comments here) wanted to zoom and center an image from the collection in such a way that it would fill the screen. Here is the code snippet that will fulfill this requirement. I haven't applied this to an example yet, but I will, if there is enough interest.

public void ZoomFullAndCenterImage(int subImageIndex)
{
    Rect subImageRect = GetSubImageRect(subImageIndex);
    Rect imageRectElement = msiLogicalToElementRect(GetSubImageRect(subImageIndex));
    
    // Calculate the zoom factor such that the image will fill up the entire screen
    double zoomFactor = (msi.Width / imageRectElement.Width) < (msi.Height / imageRectElement.Height) ? 
                                        msi.Width / imageRectElement.Width : msi.Height / imageRectElement.Height;
    
    // Center the image
    DisplaySubImageCentered(subImageIndex);
    // Use the mid point of the image to zoom from
    msi.ZoomAboutLogicalPoint(zoomFactor, (subImageRect.Left+subImageRect.Right)/2, (subImageRect.Top+subImageRect.Bottom)/2);
}

Dissecting Hard Rock Memorabilia...- Part 9 - Source code

Due to the large number of requests for the source code and my inability to respond to every request, I decided to post the link to the source code here.

I would appreciate a quick comment to this post if you do decide to download the source code. This way I get an idea of how much interest there is for the code.

Disclaimer: I used the project created by Scott Hanselman, and parts of the code was borrowed from the Expression Team and Soul Solutions blog. I used Notepad++ as my editor since I do not have access to Visual Studio. As a result of this and the fact that I have very limited time to dedicate to this activity, the code is as dirty as it gets!

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 8

For completion sake, I decided to post the animation and randomize code here. The code to do the actual animation is lifted as is from the Expression Team blog but I did make a few subtle changes -

1. Fixed the memory leak issue by getting rid of the Storyboard after it has achieved its purpose
2. Used a single storyboard to animate multiple images. I don't know if this is better than multiple storyboards (one for each image) so it would be nice to hear some opinions

Code to animate the image

private void AnimateImage(MultiScaleSubImage currentImage, Point futurePosition)
{
    // Create Keyframe
    SplinePointKeyFrame endKeyframe = new SplinePointKeyFrame();
    endKeyframe.Value = futurePosition;
    endKeyframe.KeyTime = KeyTime.FromTimeSpan(TimeSpan.FromSeconds(1));

    KeySpline ks = new KeySpline();
    ks.ControlPoint1 = new Point(0, 1);
    ks.ControlPoint2 = new Point(1, 1);
    endKeyframe.KeySpline = ks;

    // Create Animation
    PointAnimationUsingKeyFrames moveAnimation = new PointAnimationUsingKeyFrames();
    moveAnimation.KeyFrames.Add(endKeyframe);

    Storyboard.SetTarget(moveAnimation, currentImage);
    Storyboard.SetTargetProperty(moveAnimation, "ViewportOrigin");

    // Add the animation to the Storyboard
    _moveStoryboard.Children.Add(moveAnimation);
}

Code to invoke the animation - I decided against animating the hidden images (like Hard Rock) since my goal is to work with large amount of images and I didn't want to incur the additional performance overhead

private Storyboard _moveStoryboard;

public void InitStoryboard()
{
    _moveStoryboard = new Storyboard();
    msi.Resources.Add(_moveStoryboard);

    _moveStoryboard.Completed += (sender, e) => msi.Resources.Remove((Storyboard)sender);
}
       
public void ArrangeImages()
{
    InitStoryboard();
   
    ...
   
    // Then display the displayable images to scale
    for (int i=0; i<_imagesToShow.Count; i++) {
        double X = rows[subImages[i].RowNum].CalcX(subImages[i].ColNum);
        double Y = margin;
        for (int j=0; j<subImages[i].RowNum; j++)
            Y += spaceBetweenImages + rows[j].Height;
       
        _imagesToShow[i].ViewportWidth = containerAspectRatio / subImages[i].Width;
        AnimateImage(_imagesToShow[i], new Point(-(X / subImages[i].Width), -(Y / subImages[i].Width)));
        _imagesToShow[i].Opacity = 1.0;
    }
   
    // Play Storyboard
    _moveStoryboard.Begin();
}

Code to Randomize the images. The code for the RandomizeListOfImages is lifted from the Expression Team blog

buttonRandomize.Click += (sender, e) => RandomizeAndArrange();

private void RandomizeAndArrange() 
{
    _imagesToShow = RandomizedListOfImages();
    ArrangeImages();
}

private List<MultiScaleSubImage> RandomizedListOfImages()
{
    List<MultiScaleSubImage> imageList = new List<MultiScaleSubImage>();
    Random ranNum = new Random();

    // Store List of Images
    _imagesToShow.ForEach(subImage => imageList.Add(subImage));

    // Randomize Image List
    int numImages = imageList.Count;
    for(int i=0; i<numImages; i++) {
        MultiScaleSubImage tempImage = imageList[i];
        imageList.RemoveAt(i);

        int ranNumSelect = ranNum.Next(imageList.Count);

        imageList.Insert(ranNumSelect, tempImage);
    }
    return imageList;
}

Deep Zoom - Known issues

I spent a few days on the silverlight forums to see if there were any known issues with the current Deep Zoom beta release. The below list of my findings is mainly to help me keep track of the issues so that I don't spend a good amount of my time trying to fix what cannot be fixed (or can it?)!

1. Source property change is not handled well! There seems to be a workaround for non collection images but in general it seems to be broken - http://silverlight.net/forums/p/11709/44231.aspx#44231
2. Cross domain not working! Hosting the application and the deep zoom images in different domains does not work - http://silverlight.net/forums/p/13152/43883.aspx#43883
3. MultiScaleImage loses the image resolution after ScaleTransform! I noticed this when getting in/out of full screen mode as well - http://silverlight.net/forums/p/13014/42316.aspx#42316
4. MultiScaleImage only supports absolute paths! See my previous post for an example of how Vertigo worked around this (same domain only) - http://silverlight.net/forums/p/12182/40164.aspx#40164
5. Transparent images are not supported in collection mode? http://forums.expression.microsoft.com/en-US/thread/7f5aace6-9ab1-4503-8b34-f0378b731e98

For those of you who might be thinking of playing around with the bin file format you might want to wait a little while. The final release version of Silverlight 2 Deep Zoom is expected to support the xml file format instead of the bin file format - http://silverlight.net/forums/p/12784/41724.aspx#41724

Deep Zoom - Highlighting the clicked SubImage

Having played with the Hard Rock demo extensively, I noticed that it is a little difficult to see which image is associated with the displayed metadata. In general, it is the centered image but it could get confusing sometimes. This could be alleviated if there is a highlight or a border around the image.

It's not that hard actually...

1. Changes to the XAML

<Grid x:Name="LayoutRoot">
 <Grid.RowDefinitions>
  <RowDefinition Height="*"/>
  <RowDefinition Height="5"/>
  <RowDefinition Height="30"/>
  <RowDefinition Height="30"/>
 </Grid.RowDefinitions>
 
 <Border x:Name="msiBorder" Grid.Row="0" BorderThickness="1" BorderBrush="Black">
  <MultiScaleImage x:Name="msi" UseSprings="false" Margin="2,2,2,2"/>
 </Border>
 <Canvas Grid.Row="0" Margin="3,3,2,2">
  <Border x:Name="imageBorder" BorderThickness="2" BorderBrush="Black" Opacity="0" />
 </Canvas>

 ...

2. The code to show and hide the highlight on a image

public void ShowImageHighlight(int nSubImageIndex)
{    // Hightlight the image by drawing a border around it

    int borderWidth = 2;        // Setting the default border width to 2
    if (msi.ViewportWidth < 1)    // This adjusts for zoomed images where a border of 2 is insufficient
        borderWidth = (int) (borderWidth / msi.ViewportWidth);
        
    Rect borderRect = ExpandRect(msiLogicalToElementRect(GetSubImageRect(nSubImageIndex)), borderWidth);
    
    imageBorder.Width = borderRect.Width;
    imageBorder.Height = borderRect.Height;
    imageBorder.SetValue(Canvas.LeftProperty, borderRect.X);
    imageBorder.SetValue(Canvas.TopProperty, borderRect.Y);
    imageBorder.BorderThickness = new Thickness(borderWidth);
    imageBorder.Opacity = 1.0;
}

public void HideImageHighlight()
{
    imageBorder.Opacity = 0.0;
}

Rect msiLogicalToElementRect(Rect rect)
{
    return new Rect(msi.LogicalToElementPoint(new Point(rect.Left, rect.Top)), 
                    msi.LogicalToElementPoint(new Point(rect.Right, rect.Bottom)));
}

Rect ExpandRect(Rect rect, int expandBy)
{
    return new Rect(rect.Left - expandBy, rect.Top - expandBy, rect.Width + expandBy*2, rect.Height + expandBy*2);
}

3. The hooks to invoke the above code

int _clickedImageIndex = -1;

msi.MotionFinished += (sender, e) => { if (_clickedImageIndex >= 0) ShowImageHighlight(_clickedImageIndex); };

msi.MouseLeftButtonDown += (sender, e) => { _clickedImageIndex = -1; };
new MouseWheelHelper(this).Moved += (sender, e) => { _clickedImageIndex = -1; };

msi.MouseLeftButtonUp += delegate(object sender, MouseButtonEventArgs e)
{
    if (mouseButtonPressed && !dragInProgress)
    {
        Point p = this.msi.ElementToLogicalPoint(e.GetPosition(this.msi));
        int subImageIndex = SubImageHitTest(p);
        if (subImageIndex >= 0) {
            DisplaySubImageCentered(subImageIndex);
            _clickedImageIndex = subImageIndex;
        }

        bool shiftDown = (Keyboard.Modifiers & ModifierKeys.Shift) == ModifierKeys.Shift;
        if (shiftDown)  Zoom(0.5, e.GetPosition(this.msi));
        else            Zoom(2.0, e.GetPosition(this.msi));
    }
    mouseButtonPressed = false;
    dragInProgress = false;
};

I updated the previous example Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 7 with this code so you can see how it works!

Vertigo BigPicture (aka Hard Rock?) sample code

Vertigo has published their BigPicture code on codeplex. As of this time, only the mouse handler code is available but hopefully this will change. The mouse handler code is clean and well documented though so I am going to use it on my project.

One of the known issues with MultiScaleImages is that it doesn't properly handle relative URI in the Source property. Vertigo has found a neat workaround by converting a relative Uri to an absolute Uri using the Uri.TryCreate method

// Directly lifted from http://www.codeplex.com/BigPicture
Uri collectionUri;

if (Uri.TryCreate(App.Current.Host.Source, "/Collection/items.bin", out collectionUri))
    image.Source = collectionUri;

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 7

One of the readers asked me if I could post code to center the clicked image on the screen. So here goes...

Update Oct 14, 2008: Removed the example from this post to reduce load time of this page. Please use the example from this post, DeepZoom sample ported to Silverlight 2 Beta 2, to try the functionality.

The example above centers and zooms the clicked image.

Here is the code

msi.MouseLeftButtonUp += delegate(object sender, MouseButtonEventArgs e)
{
    if (mouseButtonPressed && !dragInProgress)
    {
        Point p = this.msi.ElementToLogicalPoint(e.GetPosition(this.msi));
        int subImageIndex = SubImageHitTest(p);
        if (subImageIndex >= 0)
            DisplaySubImageCentered(subImageIndex);

        bool shiftDown = (Keyboard.Modifiers & ModifierKeys.Shift) == ModifierKeys.Shift;
        if (shiftDown)  Zoom(0.5, e.GetPosition(this.msi));
        else            Zoom(2.0, e.GetPosition(this.msi));
    }
    mouseButtonPressed = false;
    dragInProgress = false;
};

void DisplaySubImageCentered(int indexSubImage)
{
    if (indexSubImage < 0 || indexSubImage >= msi.SubImages.Count)
        return;
    
    Rect subImageRect = GetSubImageRect(indexSubImage);
    double msiAspectRatio = msi.ActualWidth / msi.ActualHeight;

    Point newOrigin = new Point(subImageRect.X - (msi.ViewportWidth / 2) + (subImageRect.Width / 2), 
                                subImageRect.Y - ((msi.ViewportWidth / msiAspectRatio) / 2) + (subImageRect.Height / 2));

    msi.ViewportOrigin = newOrigin;
}

int SubImageHitTest(Point p)
{
    for (int i=0; i<msi.SubImages.Count; i++) {
        Rect subImageRect = GetSubImageRect(i);
        if (subImageRect.Contains(p))
            return i;
    }

    return -1;
}

Rect GetSubImageRect(int indexSubImage)
{
    if (indexSubImage < 0 || indexSubImage >= msi.SubImages.Count)
        return Rect.Empty;
    
    MultiScaleSubImage subImage = msi.SubImages[indexSubImage];
    
    double scaleBy = 1 / subImage.ViewportWidth;
    return new Rect(-subImage.ViewportOrigin.X * scaleBy, -subImage.ViewportOrigin.Y * scaleBy, 
                                    1 * scaleBy, (1 / subImage.AspectRatio) * scaleBy);
}

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 6

The filter example is finally here!

Update Oct 14, 2008: Removed the example from this post to reduce load time of this page. Please use the example from this post, DeepZoom sample ported to Silverlight 2 Beta 2, to try the functionality.

Click on the Filter button to filter the images based on the tag entered in the text box to the left of the filter button. The following tags will yield some result: lilies, tulip, rose, hibiscus, iris, blue, white, yellow, red, peach, pink, purple. The flower tag will return all the images. The current limitation is that only one tag can be used at a time.

Behind the scenes...

This was actually pretty easy. I used the method suggested by George (see the first comment in this post ) instead of storing a pointer in the Tag property of the MultiScaleSubImage control.

This is what I did to accomplish basic filter functionality:
1. Modified the Deep Zoom generated SparseImageSceneGraph.xml file to include some tag information

?xml version="1.0"?>
<SceneGraph version="1">
  <SceneNode>
    <FileName>tigerlilies_cr.jpg</FileName>
    <ZOrder>1</ZOrder>
    <Tags>
        <Tag>lilies</Tag>
        <Tag>flower</Tag>
    </Tags>
  </SceneNode>
  <SceneNode>
    <FileName>tulip_bluebase-1_cr.jpg</FileName>
    <ZOrder>5</ZOrder>
    <Tags>
        <Tag>tulip</Tag>
        <Tag>flower</Tag>
        <Tag>blue</Tag>
    </Tags>
  </SceneNode>
  <SceneNode>
    <FileName>2redandwhitelilies_cr.jpg</FileName>
    <ZOrder>8</ZOrder>
    <Tags>
        <Tag>lilies</Tag>
        <Tag>flower</Tag>
        <Tag>red</Tag>
        <Tag>white</Tag>
    </Tags>
  </SceneNode>
  ...
</SceneGraph>

2. Loaded the SparseImageSceneGraph.xml asynchronously into the control. I could have simplified this by loading the metadata file synchronously or by embedding it in the xap package itself but I wanted to learn the asynchronous download feature provided by Silverlight

using System.Net;
using System.Linq;
using System.Xml.Linq;

XElement _xmlImageMetadata;        // to store the metadata associated with the images

msi.Loaded += delegate(object sender, RoutedEventArgs e)
{
    // Now set the source property. 
    // This ensures that the html page with the silverlight control is loaded completely before loading the multiscaleimage.
    // This is to enhance the overall user experience.
    msi.Source = new Uri("http://thepintospatronus.com/deepzoom/test/items.bin");
    
    // Download the metadata associated with the above uri
    WebClient _downloader = new WebClient();
    _downloader.DownloadStringCompleted += new DownloadStringCompletedEventHandler(imageMetadata_DownloadStringCompleted);
    _downloader.DownloadStringAsync(new Uri("http://thepintospatronus.com/deepzoom/test/SparseImageSceneGraph.xml"));            
};

void imageMetadata_DownloadStringCompleted(Object sender, DownloadStringCompletedEventArgs e)
{
    if (e.Error == null) {
        
        // Convert the string xml representation to a valid XML document
        _xmlImageMetadata = XElement.Parse(e.Result);
        
        // The filter button is disabled by default. Enable it when the metadata is available
        buttonFilter.IsEnabled = true;
    }
}

3. Used Linq to filter the dataset

buttonFilter.Click += delegate(object sender, RoutedEventArgs e)
{
    IEnumerable<int> ZOrders = 
        from sceneNode in _xmlImageMetadata.Elements("SceneNode")
        where 
            (from tag in sceneNode.Elements("Tags").Elements("Tag")
            where
                ((string)tag.Value).ToUpper() == txtFilter.Text.ToUpper()
            select tag)
            .Any()
        select ((int)sceneNode.Element("ZOrder")-1);

    _imagesToShow.Clear();
    _imagesToHide.Clear();
    for (int i=0; i<msi.SubImages.Count; i++) {
        if (ZOrders.Contains(i))
            _imagesToShow.Add(msi.SubImages[i]);
        else
            _imagesToHide.Add(msi.SubImages[i]);
    }
    ArrangeImages();
};

TODO - Better animation.

Comments are welcome!