SimpleViewer-SL ported to Silverlight 2 and uploaded to CodePlex

I finally got around to uploading the SimpleViewer-SL source code to CodePlex. Minor changes were required to get the code to compile on Silverlight 2.

The SimpleViewer-SL project on CodePlex can be found here: http://www.codeplex.com/simpleviewerSL. Please feel free to report all bugs/issues/enhancements on this site.

DeepZoom sample ported to Silverlight 2

The source code can be found here

For those wanting to download the legacy code (for whatever reason!) --

• Silverlight 2 Release Candidate


• Silverlight 2 Beta 2


• Silverlight 2 Beta 1

Note that the disclaimer from this post still holds!

All inline samples have been upgraded to work with Silverlight 2. If you are still using Beta 2 you will need to upgrade to view the samples!

Deep Zoom - Rendering in 2 different layouts based on zoom factor

Berend, the author of Generate Silverlight 2 DeepZoom image collection from multi-page tiff, wanted to know if it was possible to render the contents of the deep zoom control in different layouts based on the current zoom factor. Initially, he wanted to show the images in a single row layout with each image taking the entire height of the control. As the user zooms out, after the image reaches a certain factor - say half the height, he wanted to switch to a thumbnails mode where all the images are displayed in a multi row and column format (much like how it is now).

It was pretty easy to do and here is the result of my experimentation:

Try zooming out so that the first image height is less than half the height of the control. After the motion is completed, it should switch to the thumbnails mode. I have currently based this off the height of the first image. The layout will switch depending on whether the height of the first image is greater than or less than half the height of the control.

The code:

enum DisplayMode {
    Full = 1,
    Thumbnails = 2
}

DisplayMode _displayMode = DisplayMode.Full;

msi.MotionFinished += delegate(object sender, RoutedEventArgs e)
{
    // This is required to ensure that ArrangeImages is called only once
    if (msi.UseSprings == false) {
        ShowAllImages();
        msi.UseSprings = true;
    }
    else {
        DisplayMode _origDisplayMode = _displayMode;
        
        Rect imageRectElement = msiLogicalToElementRect(GetSubImageRect(0));
        _displayMode = (msi.ActualHeight / imageRectElement.Height < 2) ? DisplayMode.Full : DisplayMode.Thumbnails;

        if (_displayMode != _origDisplayMode) {
            Reset();
        }
    }
};

public void Reset()
{
    _lastMousePos = new Point(0, 0);
    msi.ViewportOrigin = new Point (0, 0);
    msi.ViewportWidth = 1.0;

    // Show all the images
    _imagesToShow.Clear();
    _imagesToHide.Clear();
    for (int i=0; i<msi.SubImages.Count; i++) {
        _imagesToShow.Add(msi.SubImages[i]);
    }
    ArrangeImages();
}

public void ArrangeImages()
{
    InitStoryboard();
    
    double containerAspectRatio = this.msi.ActualWidth / this.msi.ActualHeight;
    double spaceBetweenImages = 0.005;

    List<SubImage> subImages = new List<SubImage>();
    _imagesToShow.ForEach(subImage => subImages.Add(new SubImage(subImage)));
    
    // Capture the total width of all images
    double totalImagesWidth = 0.0;
    subImages.ForEach(subImage => totalImagesWidth += subImage.Width);

    // Calculate the total number of rows required to display all the images
    int numRows = (_displayMode == DisplayMode.Thumbnails) ? (int)Math.Sqrt((totalImagesWidth / containerAspectRatio)+1) : 1;

    // Assign images to each row
    List<Row> rows = new List<Row>(numRows);
    for (int i=0; i<numRows; i++)
        rows.Add(new Row(spaceBetweenImages));

    double widthPerRow = totalImagesWidth / numRows;
    double imagesWidth = 0;
    // Separate the images into rows. The total width of all images in a row should not exceed widthPerRow
    for (int i=0, j=0; i<numRows; i++, imagesWidth=0) {
        while (imagesWidth < widthPerRow && j < subImages.Count) {
            rows[i].AddImage(subImages[j]);
            subImages[j].RowNum = i;
            imagesWidth += subImages[j++].Width;
        }
    }

    // At this point in time the subimage height is 1 
    // If we assume that the total height is also 1 we need to scale the subimages to fit within a total height of 1
    // If the total height is 1, the total width is aspectRatio. Hence (aspectRatio)/(total width of all images in a row) is the scaling factor.
    // Added later: take into account spacing between images
    rows.ForEach(Row => Row.Scale(containerAspectRatio));

    // Calculate the total height, with space between images, of the images across all rows
    // Also adjust the colNum for each image
    double totalImagesHeight = (numRows - 1) * spaceBetweenImages;
    rows.ForEach(Row => totalImagesHeight += Row.Height);

    // The totalImagesHeight should not exceed 1. 
    // if it does, we need to scale all images by a factor of (1 / totalImagesHeight)
    if (((_displayMode == DisplayMode.Thumbnails && totalImagesHeight > 1)) || _displayMode == DisplayMode.Full) {
        subImages.ForEach(subImage => subImage.Scale(1 / (totalImagesHeight+spaceBetweenImages)));
        totalImagesHeight = (numRows - 1) * spaceBetweenImages;
        rows.ForEach(Row => totalImagesHeight += Row.Height);
        Debug.Assert(totalImagesHeight <= 1);
    }
    
    // Calculate the top and bottom margin
    double margin = (1 - totalImagesHeight) / 2;

    // First hide all the images that should not be displayed
    _imagesToHide.ForEach(subImage => subImage.Opacity = 0.0);

    // Then display the displayable images to scale
    for (int i=0; i<_imagesToShow.Count; i++) {
        double X = rows[subImages[i].RowNum].CalcX(subImages[i].ColNum);
        double Y = margin;
        for (int j=0; j<subImages[i].RowNum; j++)
            Y += spaceBetweenImages + rows[j].Height;
        
        _imagesToShow[i].ViewportWidth = containerAspectRatio / subImages[i].Width;
        AnimateImage(_imagesToShow[i], new Point(-(X / subImages[i].Width), -(Y / subImages[i].Width)));    // for animation, use this statement instead of the next one
        _imagesToShow[i].Opacity = 1.0;
    }
    // Play Storyboard
    _moveStoryboard.Begin();
}

I'm honored!

I just noticed that my blog is referenced in the MSDN Silverlight Beta 2 documentation, specifically http://msdn.microsoft.com/en-us/library/system.windows.controls.multiscaleimage(VS.95).aspx (search for Filter Example). This is definitely an honor!

Does this mean that I can now ask Microsoft for a free copy of Visual Studio 2008? Just kidding :) I am currently working on a Silverlight project so I have access to VS2008. I still use Notepad++ on my laptop though, since it is underpowered for VS.

I finally got around to uploading the source code for the DeepZoom sample. You will find the link to the code in this post - DeepZoom sample ported to Silverlight 2 Beta 2

Deep Zoom Composer source code

Not really...but pretty close! Check out Berend's article on CodeProject -
Generate Silverlight 2 DeepZoom image collection from multi-page tiff

I haven't tried it myself so I don't know if it works, but it definitely opens up a lot of possibilities - Dynamic gallery generation; Adding images to collection without regenerating all the tiles...

If only Microsoft would open source the DeepZoom Composer code, or at least the image/tile generation part!

Great work, Berend Engelbrecht!

SimpleViewer-SL ported to Silverlight 2 Beta 2

SimpleViewer-SL now works on Beta 2 as well. I updated the inline samples in the previous posts so folks with Beta 2 should be able to view all the sample galleries that I created using SimpleViewer-SL.

If you recollect from my earlier post, I did not use the ScrollViewer control since I wanted to keep the .xap package as small as possible. This is no longer the case since ScrollViewer is now part of the Silverlight framework itself. I still haven't updated the code to take advantage of this but might do so at a later date.

DeepZoom sample ported to Silverlight 2 Beta 2

I have been planning to do this for some time now but finally managed to squeeze some time for this activity. I am going to leave the previous posts as-is so that the folks with Beta 1 can look at the previous sample.

It was a relatively straightforward exercise but one thing I noticed is that msi.Width/Height doesn't work properly in Beta 2, but luckily msi.ActualWidth/ActualHeight works! I also used the new tagging format (Metadata.xml) that the latest DeepZoom Composer generates.

I will post the source code in the next post.
The source code has been uploaded and can be found here. Note that the disclaimer from this post still holds!

Scrolling, without using ScrollViewer!

The main objective when building SimpleViewer-SL was to keep the executable as small as possible and hence I didn't have the option of using ScrollViewer. I still needed scrolling functionality though and hence had to take some alternate approach. This post is a result of my findings. There is possibly no reason for anyone to take this approach, unless there is a compelling reason to not use ScrollViewer, but it might prove useful!

So how did I go about implementing scrolling? The trick is to contain the visible area and use TranslateTransform.

Let's look at the Xaml first

<UserControl x:Class="SimpleViewer_SL.Thumbnails"
 xmlns="http://schemas.microsoft.com/client/2007"
 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
<Grid x:Name="LayoutRoot" HorizontalAlignment="Center" VerticalAlignment="Center">
<Grid x:Name="LayoutThumbnailsViewport" HorizontalAlignment="Left">
 <Grid x:Name="LayoutThumbnails" HorizontalAlignment="Left">
  <Grid.RenderTransform>
   <TranslateTransform x:Name="GridTranslate" />
  </Grid.RenderTransform>
  <Grid.Resources>
     <Storyboard x:Name="GridScrollStoryboard">
     <DoubleAnimation x:Name="GridScrollAnimation" Storyboard.TargetName="GridTranslate" Storyboard.TargetProperty="X" Duration="00:00:00.5"/>
     </Storyboard>
  </Grid.Resources>
 </Grid>
</Grid>
<Canvas x:Name="RightNav" Height="25" Width="30" Background="Transparent" HorizontalAlignment="Right">
 <Path Height="25" Width="30" Stretch="Fill"
  Data="F1 M 0,50L 130,50L 80,0L 130,0L 200,70L 130,140L 80,140L 130,90L 0,90L 0,50 Z">
 </Path>
</Canvas>
<Canvas x:Name="LeftNav" Height="25" Width="30" Background="Transparent" HorizontalAlignment="Left">
 <Path Height="25" Width="30" Stretch="Fill"
  Data="F1 M 200,90L 70,90L 120,140L 70,140L 0,70L 70,0L 120,0L 70,50L 200,50L 200,90 Z">
 </Path>
</Canvas>
</Grid>
</UserControl>

What we are trying to do here is display the thumbnails in a grid, but, as you can see from the above Xaml, I have had to create 3 grids to implement scrolling.

• LayoutThumbnails - This is the grid which will host the thumbnails
• LayoutThumbnailsViewport - This is the grid which will host the translated thumbnails
• LayoutRoot - This is the grid which will host the clipped thumbnails
  

• GridTranslate - This exposes the translate transform of LayoutThumbnails
• GridScrollStoryboard - This is the storyboard that animates GridTranslate to simulate (horizontal) scrolling

Now for some code

public partial class Thumbnails : UserControl
{
 Page _parent = null;
 int _borderWidth = 2;
 int _selectedThumbnailIndex = -1;
 int _rowCount = 0;
 int _colCount = 0;
 int _totalColCount = 0;
 int _numPages = 0;
 int _currPage = 0;
 Size _thumbSize = new Size { Width = 65, Height = 65 };
 int _borderPlusPaddingWidth = 0;

 public double ControlWidth { get { return (_thumbSize.Width + (_borderPlusPaddingWidth * 2)) * _colCount; }}

 public Thumbnails(Page parent)
 {
     _parent = parent;
     _rowCount = _parent.Settings.ThumbnailRows;
     _colCount = _parent.Settings.ThumbnailColumns;
     _totalColCount = (int)(Math.Ceiling((double)_parent.Settings.ImagesSettings.Count / _rowCount));
     _numPages = (int)(Math.Ceiling((double)_totalColCount / _colCount));
     _borderPlusPaddingWidth = _parent.Settings.ThumbPadding + _borderWidth;

     InitializeComponent();
  
     LayoutRoot.Width = ControlWidth; // ****1****
     LayoutThumbnailsViewport.Width = ControlWidth * _numPages; // ****2****
  
     CreateLayout();
     DrawThumbnails();
 }

 int CreateLayout() {
     // Thumbnails 
     LayoutRoot.RowDefinitions.Add(new RowDefinition());
     LayoutThumbnailsViewport.SetValue(Grid.RowProperty, 0);
  
     if (_numPages > 0) {
         // Navigation Arrows
         LayoutRoot.RowDefinitions.Add(new RowDefinition { Height = new GridLength(30) });
         RightNav.SetValue(Grid.RowProperty, 1);
         LeftNav.SetValue(Grid.RowProperty, 1);
     }
 }

 void DrawThumbnails()
 {
     for (int row = 0; row < _rowCount; row++) {
         LayoutThumbnails.RowDefinitions.Add(new RowDefinition { Height = new GridLength(_thumbSize.Height + (_borderPlusPaddingWidth * 2)) });
     }
  
     for (int col = 0; col < _colCount * _numPages; col++) {
         LayoutThumbnails.ColumnDefinitions.Add(new ColumnDefinition { Width = new GridLength(_thumbSize.Width + (_borderPlusPaddingWidth * 2)) });
     }
  
     int imageIndex = 0;
     int startCol = 0;
     for (int page = 0; page < _numPages; page++) {
         for (int row = 0; row < _rowCount; row++) {
             for (int col = 0; col < _colCount && imageIndex < _parent.Settings.ImagesSettings.Count; col++) {
                 CreateImage(row, col + startCol, imageIndex);
                 imageIndex++;
             }
         }
         startCol += _colCount;
     }
  
     LeftNav.MouseLeftButtonUp += (sender, e) => {
         ScrollLeft();
     };
  
     RightNav.MouseLeftButtonUp += (sender, e) => {
         ScrollRight();
     };
 }

 public void CreateImage(int row, int col, int imageIndex)
 {
     Image thumb = new Image();
     thumb.Source = new BitmapImage(new Uri(_parent.Settings.ThumbPath + _parent.Settings.ImagesSettings[imageIndex].Name, UriKind.Relative));

     Border border = new Border {
                         Width = _thumbSize.Width + (_borderWidth*2),
                         Height = _thumbSize.Height + (_borderWidth*2),
                         BorderThickness = new Thickness(_borderWidth),
                         BorderBrush = new SolidColorBrush(_parent.Settings.FrameColor),
                         Margin = new Thickness(_parent.Settings.ThumbPadding),
                         Opacity = 0.6
                                 };
     border.SetValue(Grid.RowProperty, row);
     border.SetValue(Grid.ColumnProperty, col);
     border.SetValue(NameProperty, "Image" + imageIndex.ToString());
     border.Tag = imageIndex.ToString();            // Tag can only handle Strings as of SL2 B1 release
  
     border.Child = thumb;
     LayoutThumbnails.Children.Add(border);
  
     border.MouseLeftButtonUp += (sender, e) => {
         Rect LayoutRootRect = new Rect(0, 0, LayoutRoot.RenderSize.Width, LayoutRoot.RenderSize.Height);
         if (LayoutRootRect.Contains(e.GetPosition(LayoutRoot))) { // ****3****
             Border currentBorder = (Border)sender;
             SelectedThumbnailIndex = Int32.Parse((string)currentBorder.Tag);
         }
     };
 }

 bool CanScrollRight    { get { return ((_currPage + 1) < _numPages); }}
 bool CanScrollLeft    { get { return (_currPage > 0); }}

 void ScrollRight()
 {
     if (CanScrollRight) {
         _currPage++;
         GridScrollAnimation.To = -(_thumbSize.Width + (_borderPlusPaddingWidth * 2)) * _colCount * _currPage;
         GridScrollStoryboard.Begin();
     }
 }

 void ScrollLeft()
 {
     if (CanScrollLeft) {
         _currPage--;
         GridScrollAnimation.To = -(_thumbSize.Width + (_borderPlusPaddingWidth * 2)) * _colCount * _currPage;
         GridScrollStoryboard.Begin();
     }
 }
}

Most of the code is self explanatory but I would like to elaborate on a few key points which is highlighted in the code:

1. The width of LayoutThumbnailsViewport should be equal to or greater than the width of the entire grid. If the width is not explicitly set or is less than the width of the entire grid, translatetransform will clip the contents that lie outside the area of the grid
2. The width of LayoutRoot should be set to the viewing area. This should be less than the width of LayoutThumbnailsViewport if scrolling is desired
3. One undesired effect of creating this layout is that the underlying grid (LayoutThumbnails) and hence the thumbnails receive mouse events even for the contents that is outside the visible area (LayoutRoot). To circumvent this I had to put this check - if (LayoutRootRect.Contains(e.GetPosition(LayoutRoot))) - in the border.MouseLeftButtonUp event handler. (Note that border is synonymous with thumbnail)

SimpleViewer-SL Beta 1

Update 5/8/08: Fixed blank screen bug when navigating between thumbnails, and improved browser resizing support...

The Silverlight version of the flash based photo album viewer - SimpleViewer, is here! I decided to call it SimpleViewer-SL for now.

Here are a few examples that were created using SimpleViewer-SL (the keyboard arrow keys are supported as well)

1. Thumbnails on the left, 3x3 grid


2. Thumbnails on the right, 5x4 grid


3. Thumbnails on bottom, 2x4 grid


4. Thumbnails on the left, with background image

SimpleViewer-SL is highly customizable. It is compatible with SimpleViewer and hence all options exposed by SimpleViewer are applicable to SimpleViewer-SL as well. The options are listed here. The only XML option that is not supported at this time is enableRightClickOpen. Note that the text fields like title and Caption should be plain text (no HTML) for now since I haven't implemented the HTML text control as yet. As for the HTML options, the only supported option is xmlDataPath. The HTML option should be passed to the SimpleViewer-SL Silverlight object using the param tag - <param name="initParams" value="xmlDataPath=gallery1.xml" />

The album can be created using any of the methods described here . After creating the albums, a few modifications to the generated file(s) is required to make it work on SimpleViewer-SL.

The xap can be found here.

The directory structure for the samples is as follows:

 gallery1/
   SimpleViewer-SL.xap
   gallery1.html
   gallery1.xml
   gallery2.html
   gallery2.xml
   gallery3.html
   gallery3.xml
   gallery4.html
   gallery4.xml
   thumbs/
     65x65 thumbnail images
   images/
     the corresponding big images

Between the above links, the sample html and xml files, it should be fairly straightforward for anyone to figure out how to use SimpleViewer-SL. Overall it is fairly feature complete but there is always room for improvement. It is a little unstable, in that sometime navigating between thumbnails results in a blank screen, but I think it is more to do with Silverlight being a Beta release (looks like it was more to do with my code!).

This exercise took me a little over a week of part time work so I have to say that I am pretty impressed with Silverlight. Most of my time was spent in figuring out a way to work around Silverlight 2 Beta 1 bugs. I suspect that I can simplify the code a bit after the final release of Siverlight 2. Nevertheless, the current size of SimpleViewer-SL is a mere 17kb, the same as SimpleViewer, with lots of opportunity for improvement.

For those looking for the code, I am planning to codeplex it when I get some time. Feel free to leave comments to pressure me into doing it sooner rather than later!

Deep Zoom Composer Updated!

A nice update to Deep Zoom Composer can be found here - http://blogs.msdn.com/expression/archive/2008/05/03/an-update-to-deep-zoom-composer.aspx

The new update includes: improved exporting (includes mousewheel, pan, zoom, and keyboard functionality), better design experience, collection export bug fix, and greater access to help.

Nice!

Next Project - Port SimpleViewer to Silverlight

Having run out of ideas on how further to exploit Deep Zoom, I am going to now focus my energy on porting the closed-source flash based SimpleViewer to Silverlight. Again, my objective is to learn Silverlight, so I will try to push some boundaries (if required). As part of this exercise, I will try to stick to the original goal of SimpleViewer, that is keep the download as small as possible. The current version (1.8.5) of SimpleViewer is only 17k and that is what I am aiming for. This, of course, means no using the extended controls, but only what is available in the core Silverlight 2 SDK!

So, why SimpleViewer? I actually use it in my personal blog and have always wanted more control over it. I also want to know what it takes to create the same experience in Silverlight. I am expecting this to be a trivial exercise, but let's see!

My first roadblock!

I wanted to keep everything as dynamic as possible. This means no explicitly specifying the height and width of any control. I was happy to see that the image control provides this capability and is able to work well within the constraints of MaxWidth and MaxHeight. This was all wonderful and perfect for my needs but now I wanted to draw a simple border around the image. Pretty easy...all I need is the dimensions of the image (control), but this is where it got a little complicated. The image control does give access to the dimensions but a little too late. I won't go into the details here but I did post it on the Silverlight.net forum is anyone's interested. By the way this is a great forum and one of the best places to get answers to all your Silverlight questions or issues.

What I discovered is that the image loaded event is not a reliable place to extract the dimensions from the image control. I tried a kludgey workaround by creating a timer and accessing the dimensions from the image control a little later - around 100 milli seconds - but this is definitely not reliable. But at least I had a solution!

While I was playing with it some more, I discovered the SizeChanged event. This also doesn't help if the image dimensions are extracted from the image control itself, but I discovered that the event itself has the size, and that is when I had my solution!

There might be some issues with this so please leave a comment if you find something. I am assuming this is a bug with the image loaded event so, hopefully, it will be fixed in the next release of Silverlight. But, for now, it looks like I can make some "not so kludgey" progress!

MyImage.SizeChanged += (sender, e) => {
    if (e.PreviousSize.Width == 0 && e.PreviousSize.Height == 0 && e.NewSize != e.PreviousSize) {
        // e.NewSize contains the dimension of the image
        // MyImage.Width and MyImage.ActualWidth is unreliable so use e.NewSize instead
        DrawBorderAroundImage(e.NewSize);
    }
};

Deep Zoom - Album Creator

No, this one is not by me!

Marauderz has developed a Deep Zoom Album Creator which uses some code from my blog so I thought I'd mention it here. I am glad to see my code being leveraged in some useful tools, hopefully more of which we shall be seeing soon.

How about some designers getting involved and skinning this baby?

Deep Zoom - SlideShow

I was wondering how best to showcase the functionality from my last post and the idea of SlideShow popped up in my head. Also, I was bored and wanted to play around with Deep Zoom some more...

Update Oct 14, 2008: Removed the example from this post to reduce load time of this page. Please use the example from this post, DeepZoom sample ported to Silverlight 2 Beta 2, to try the functionality.

Click on the Start Slide button to activate the slide show. Click on the button again to stop the slide show.

The code to enable slide show is very simple. Most of the work is done by the function described in my last post.

Xaml

<Button Grid.Column="2" x:Name="buttonSlideshow" Width="70" Content="Start Slide" IsEnabled="False"/>

Code

int _slideshowImageIndex = 0;
System.Windows.Threading.DispatcherTimer _myDispatcherTimer;

msi.Loaded += delegate(object sender, RoutedEventArgs e)
{
    _myDispatcherTimer = new System.Windows.Threading.DispatcherTimer();
    _myDispatcherTimer.Tick += new EventHandler(Each_Tick);
    buttonSlideshow.IsEnabled = true;
};

buttonSlideshow.Click += delegate(object sender, RoutedEventArgs e)
{
    if ((string)buttonSlideshow.Content == "Start Slide") {
        buttonSlideshow.Content = "Stop Slide";
        StartSlideShow();
    }
    else {
        buttonSlideshow.Content = "Start Slide";
        StopSlideShow();
    }
};

public void StartSlideShow()
{
    // Set the timer interval to 200 ms so that the tick event is called immediately
    _myDispatcherTimer.Interval = new TimeSpan(0, 0, 0, 0, 200);
    _myDispatcherTimer.Start();
}

public void StopSlideShow()
{
    _myDispatcherTimer.Stop();
}

public void Each_Tick(object o, EventArgs sender)
{
    // Reset the timer to 5 sec so that each slide appears after 5 seconds
    _myDispatcherTimer.Interval = new TimeSpan(0, 0, 0, 0, 5000);
    ZoomFullAndCenterImage(_slideshowImageIndex);
    if (++_slideshowImageIndex == msi.SubImages.Count)
        _slideshowImageIndex = 0;
}

Deep Zoom - Zooming and centering image to fill the Deep Zoom screen

One of the readers (see comments here) wanted to zoom and center an image from the collection in such a way that it would fill the screen. Here is the code snippet that will fulfill this requirement. I haven't applied this to an example yet, but I will, if there is enough interest.

public void ZoomFullAndCenterImage(int subImageIndex)
{
    Rect subImageRect = GetSubImageRect(subImageIndex);
    Rect imageRectElement = msiLogicalToElementRect(GetSubImageRect(subImageIndex));
    
    // Calculate the zoom factor such that the image will fill up the entire screen
    double zoomFactor = (msi.Width / imageRectElement.Width) < (msi.Height / imageRectElement.Height) ? 
                                        msi.Width / imageRectElement.Width : msi.Height / imageRectElement.Height;
    
    // Center the image
    DisplaySubImageCentered(subImageIndex);
    // Use the mid point of the image to zoom from
    msi.ZoomAboutLogicalPoint(zoomFactor, (subImageRect.Left+subImageRect.Right)/2, (subImageRect.Top+subImageRect.Bottom)/2);
}

Dissecting Hard Rock Memorabilia...- Part 9 - Source code

Due to the large number of requests for the source code and my inability to respond to every request, I decided to post the link to the source code here.

I would appreciate a quick comment to this post if you do decide to download the source code. This way I get an idea of how much interest there is for the code.

Disclaimer: I used the project created by Scott Hanselman, and parts of the code was borrowed from the Expression Team and Soul Solutions blog. I used Notepad++ as my editor since I do not have access to Visual Studio. As a result of this and the fact that I have very limited time to dedicate to this activity, the code is as dirty as it gets!

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 8

For completion sake, I decided to post the animation and randomize code here. The code to do the actual animation is lifted as is from the Expression Team blog but I did make a few subtle changes -

1. Fixed the memory leak issue by getting rid of the Storyboard after it has achieved its purpose
2. Used a single storyboard to animate multiple images. I don't know if this is better than multiple storyboards (one for each image) so it would be nice to hear some opinions

Code to animate the image

private void AnimateImage(MultiScaleSubImage currentImage, Point futurePosition)
{
    // Create Keyframe
    SplinePointKeyFrame endKeyframe = new SplinePointKeyFrame();
    endKeyframe.Value = futurePosition;
    endKeyframe.KeyTime = KeyTime.FromTimeSpan(TimeSpan.FromSeconds(1));

    KeySpline ks = new KeySpline();
    ks.ControlPoint1 = new Point(0, 1);
    ks.ControlPoint2 = new Point(1, 1);
    endKeyframe.KeySpline = ks;

    // Create Animation
    PointAnimationUsingKeyFrames moveAnimation = new PointAnimationUsingKeyFrames();
    moveAnimation.KeyFrames.Add(endKeyframe);

    Storyboard.SetTarget(moveAnimation, currentImage);
    Storyboard.SetTargetProperty(moveAnimation, "ViewportOrigin");

    // Add the animation to the Storyboard
    _moveStoryboard.Children.Add(moveAnimation);
}

Code to invoke the animation - I decided against animating the hidden images (like Hard Rock) since my goal is to work with large amount of images and I didn't want to incur the additional performance overhead

private Storyboard _moveStoryboard;

public void InitStoryboard()
{
    _moveStoryboard = new Storyboard();
    msi.Resources.Add(_moveStoryboard);

    _moveStoryboard.Completed += (sender, e) => msi.Resources.Remove((Storyboard)sender);
}
       
public void ArrangeImages()
{
    InitStoryboard();
   
    ...
   
    // Then display the displayable images to scale
    for (int i=0; i<_imagesToShow.Count; i++) {
        double X = rows[subImages[i].RowNum].CalcX(subImages[i].ColNum);
        double Y = margin;
        for (int j=0; j<subImages[i].RowNum; j++)
            Y += spaceBetweenImages + rows[j].Height;
       
        _imagesToShow[i].ViewportWidth = containerAspectRatio / subImages[i].Width;
        AnimateImage(_imagesToShow[i], new Point(-(X / subImages[i].Width), -(Y / subImages[i].Width)));
        _imagesToShow[i].Opacity = 1.0;
    }
   
    // Play Storyboard
    _moveStoryboard.Begin();
}

Code to Randomize the images. The code for the RandomizeListOfImages is lifted from the Expression Team blog

buttonRandomize.Click += (sender, e) => RandomizeAndArrange();

private void RandomizeAndArrange() 
{
    _imagesToShow = RandomizedListOfImages();
    ArrangeImages();
}

private List<MultiScaleSubImage> RandomizedListOfImages()
{
    List<MultiScaleSubImage> imageList = new List<MultiScaleSubImage>();
    Random ranNum = new Random();

    // Store List of Images
    _imagesToShow.ForEach(subImage => imageList.Add(subImage));

    // Randomize Image List
    int numImages = imageList.Count;
    for(int i=0; i<numImages; i++) {
        MultiScaleSubImage tempImage = imageList[i];
        imageList.RemoveAt(i);

        int ranNumSelect = ranNum.Next(imageList.Count);

        imageList.Insert(ranNumSelect, tempImage);
    }
    return imageList;
}

Deep Zoom - Known issues

I spent a few days on the silverlight forums to see if there were any known issues with the current Deep Zoom beta release. The below list of my findings is mainly to help me keep track of the issues so that I don't spend a good amount of my time trying to fix what cannot be fixed (or can it?)!

1. Source property change is not handled well! There seems to be a workaround for non collection images but in general it seems to be broken - http://silverlight.net/forums/p/11709/44231.aspx#44231
2. Cross domain not working! Hosting the application and the deep zoom images in different domains does not work - http://silverlight.net/forums/p/13152/43883.aspx#43883
3. MultiScaleImage loses the image resolution after ScaleTransform! I noticed this when getting in/out of full screen mode as well - http://silverlight.net/forums/p/13014/42316.aspx#42316
4. MultiScaleImage only supports absolute paths! See my previous post for an example of how Vertigo worked around this (same domain only) - http://silverlight.net/forums/p/12182/40164.aspx#40164
5. Transparent images are not supported in collection mode? http://forums.expression.microsoft.com/en-US/thread/7f5aace6-9ab1-4503-8b34-f0378b731e98

For those of you who might be thinking of playing around with the bin file format you might want to wait a little while. The final release version of Silverlight 2 Deep Zoom is expected to support the xml file format instead of the bin file format - http://silverlight.net/forums/p/12784/41724.aspx#41724

Deep Zoom - Highlighting the clicked SubImage

Having played with the Hard Rock demo extensively, I noticed that it is a little difficult to see which image is associated with the displayed metadata. In general, it is the centered image but it could get confusing sometimes. This could be alleviated if there is a highlight or a border around the image.

It's not that hard actually...

1. Changes to the XAML

<Grid x:Name="LayoutRoot">
 <Grid.RowDefinitions>
  <RowDefinition Height="*"/>
  <RowDefinition Height="5"/>
  <RowDefinition Height="30"/>
  <RowDefinition Height="30"/>
 </Grid.RowDefinitions>
 
 <Border x:Name="msiBorder" Grid.Row="0" BorderThickness="1" BorderBrush="Black">
  <MultiScaleImage x:Name="msi" UseSprings="false" Margin="2,2,2,2"/>
 </Border>
 <Canvas Grid.Row="0" Margin="3,3,2,2">
  <Border x:Name="imageBorder" BorderThickness="2" BorderBrush="Black" Opacity="0" />
 </Canvas>

 ...

2. The code to show and hide the highlight on a image

public void ShowImageHighlight(int nSubImageIndex)
{    // Hightlight the image by drawing a border around it

    int borderWidth = 2;        // Setting the default border width to 2
    if (msi.ViewportWidth < 1)    // This adjusts for zoomed images where a border of 2 is insufficient
        borderWidth = (int) (borderWidth / msi.ViewportWidth);
        
    Rect borderRect = ExpandRect(msiLogicalToElementRect(GetSubImageRect(nSubImageIndex)), borderWidth);
    
    imageBorder.Width = borderRect.Width;
    imageBorder.Height = borderRect.Height;
    imageBorder.SetValue(Canvas.LeftProperty, borderRect.X);
    imageBorder.SetValue(Canvas.TopProperty, borderRect.Y);
    imageBorder.BorderThickness = new Thickness(borderWidth);
    imageBorder.Opacity = 1.0;
}

public void HideImageHighlight()
{
    imageBorder.Opacity = 0.0;
}

Rect msiLogicalToElementRect(Rect rect)
{
    return new Rect(msi.LogicalToElementPoint(new Point(rect.Left, rect.Top)), 
                    msi.LogicalToElementPoint(new Point(rect.Right, rect.Bottom)));
}

Rect ExpandRect(Rect rect, int expandBy)
{
    return new Rect(rect.Left - expandBy, rect.Top - expandBy, rect.Width + expandBy*2, rect.Height + expandBy*2);
}

3. The hooks to invoke the above code

int _clickedImageIndex = -1;

msi.MotionFinished += (sender, e) => { if (_clickedImageIndex >= 0) ShowImageHighlight(_clickedImageIndex); };

msi.MouseLeftButtonDown += (sender, e) => { _clickedImageIndex = -1; };
new MouseWheelHelper(this).Moved += (sender, e) => { _clickedImageIndex = -1; };

msi.MouseLeftButtonUp += delegate(object sender, MouseButtonEventArgs e)
{
    if (mouseButtonPressed && !dragInProgress)
    {
        Point p = this.msi.ElementToLogicalPoint(e.GetPosition(this.msi));
        int subImageIndex = SubImageHitTest(p);
        if (subImageIndex >= 0) {
            DisplaySubImageCentered(subImageIndex);
            _clickedImageIndex = subImageIndex;
        }

        bool shiftDown = (Keyboard.Modifiers & ModifierKeys.Shift) == ModifierKeys.Shift;
        if (shiftDown)  Zoom(0.5, e.GetPosition(this.msi));
        else            Zoom(2.0, e.GetPosition(this.msi));
    }
    mouseButtonPressed = false;
    dragInProgress = false;
};

I updated the previous example Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 7 with this code so you can see how it works!

Vertigo BigPicture (aka Hard Rock?) sample code

Vertigo has published their BigPicture code on codeplex. As of this time, only the mouse handler code is available but hopefully this will change. The mouse handler code is clean and well documented though so I am going to use it on my project.

One of the known issues with MultiScaleImages is that it doesn't properly handle relative URI in the Source property. Vertigo has found a neat workaround by converting a relative Uri to an absolute Uri using the Uri.TryCreate method

// Directly lifted from http://www.codeplex.com/BigPicture
Uri collectionUri;

if (Uri.TryCreate(App.Current.Host.Source, "/Collection/items.bin", out collectionUri))
    image.Source = collectionUri;

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 7

One of the readers asked me if I could post code to center the clicked image on the screen. So here goes...

Update Oct 14, 2008: Removed the example from this post to reduce load time of this page. Please use the example from this post, DeepZoom sample ported to Silverlight 2 Beta 2, to try the functionality.

The example above centers and zooms the clicked image.

Here is the code

msi.MouseLeftButtonUp += delegate(object sender, MouseButtonEventArgs e)
{
    if (mouseButtonPressed && !dragInProgress)
    {
        Point p = this.msi.ElementToLogicalPoint(e.GetPosition(this.msi));
        int subImageIndex = SubImageHitTest(p);
        if (subImageIndex >= 0)
            DisplaySubImageCentered(subImageIndex);

        bool shiftDown = (Keyboard.Modifiers & ModifierKeys.Shift) == ModifierKeys.Shift;
        if (shiftDown)  Zoom(0.5, e.GetPosition(this.msi));
        else            Zoom(2.0, e.GetPosition(this.msi));
    }
    mouseButtonPressed = false;
    dragInProgress = false;
};

void DisplaySubImageCentered(int indexSubImage)
{
    if (indexSubImage < 0 || indexSubImage >= msi.SubImages.Count)
        return;
    
    Rect subImageRect = GetSubImageRect(indexSubImage);
    double msiAspectRatio = msi.ActualWidth / msi.ActualHeight;

    Point newOrigin = new Point(subImageRect.X - (msi.ViewportWidth / 2) + (subImageRect.Width / 2), 
                                subImageRect.Y - ((msi.ViewportWidth / msiAspectRatio) / 2) + (subImageRect.Height / 2));

    msi.ViewportOrigin = newOrigin;
}

int SubImageHitTest(Point p)
{
    for (int i=0; i<msi.SubImages.Count; i++) {
        Rect subImageRect = GetSubImageRect(i);
        if (subImageRect.Contains(p))
            return i;
    }

    return -1;
}

Rect GetSubImageRect(int indexSubImage)
{
    if (indexSubImage < 0 || indexSubImage >= msi.SubImages.Count)
        return Rect.Empty;
    
    MultiScaleSubImage subImage = msi.SubImages[indexSubImage];
    
    double scaleBy = 1 / subImage.ViewportWidth;
    return new Rect(-subImage.ViewportOrigin.X * scaleBy, -subImage.ViewportOrigin.Y * scaleBy, 
                                    1 * scaleBy, (1 / subImage.AspectRatio) * scaleBy);
}

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 6

The filter example is finally here!

Update Oct 14, 2008: Removed the example from this post to reduce load time of this page. Please use the example from this post, DeepZoom sample ported to Silverlight 2 Beta 2, to try the functionality.

Click on the Filter button to filter the images based on the tag entered in the text box to the left of the filter button. The following tags will yield some result: lilies, tulip, rose, hibiscus, iris, blue, white, yellow, red, peach, pink, purple. The flower tag will return all the images. The current limitation is that only one tag can be used at a time.

Behind the scenes...

This was actually pretty easy. I used the method suggested by George (see the first comment in this post ) instead of storing a pointer in the Tag property of the MultiScaleSubImage control.

This is what I did to accomplish basic filter functionality:
1. Modified the Deep Zoom generated SparseImageSceneGraph.xml file to include some tag information

?xml version="1.0"?>
<SceneGraph version="1">
  <SceneNode>
    <FileName>tigerlilies_cr.jpg</FileName>
    <ZOrder>1</ZOrder>
    <Tags>
        <Tag>lilies</Tag>
        <Tag>flower</Tag>
    </Tags>
  </SceneNode>
  <SceneNode>
    <FileName>tulip_bluebase-1_cr.jpg</FileName>
    <ZOrder>5</ZOrder>
    <Tags>
        <Tag>tulip</Tag>
        <Tag>flower</Tag>
        <Tag>blue</Tag>
    </Tags>
  </SceneNode>
  <SceneNode>
    <FileName>2redandwhitelilies_cr.jpg</FileName>
    <ZOrder>8</ZOrder>
    <Tags>
        <Tag>lilies</Tag>
        <Tag>flower</Tag>
        <Tag>red</Tag>
        <Tag>white</Tag>
    </Tags>
  </SceneNode>
  ...
</SceneGraph>

2. Loaded the SparseImageSceneGraph.xml asynchronously into the control. I could have simplified this by loading the metadata file synchronously or by embedding it in the xap package itself but I wanted to learn the asynchronous download feature provided by Silverlight

using System.Net;
using System.Linq;
using System.Xml.Linq;

XElement _xmlImageMetadata;        // to store the metadata associated with the images

msi.Loaded += delegate(object sender, RoutedEventArgs e)
{
    // Now set the source property. 
    // This ensures that the html page with the silverlight control is loaded completely before loading the multiscaleimage.
    // This is to enhance the overall user experience.
    msi.Source = new Uri("http://thepintospatronus.com/deepzoom/test/items.bin");
    
    // Download the metadata associated with the above uri
    WebClient _downloader = new WebClient();
    _downloader.DownloadStringCompleted += new DownloadStringCompletedEventHandler(imageMetadata_DownloadStringCompleted);
    _downloader.DownloadStringAsync(new Uri("http://thepintospatronus.com/deepzoom/test/SparseImageSceneGraph.xml"));            
};

void imageMetadata_DownloadStringCompleted(Object sender, DownloadStringCompletedEventArgs e)
{
    if (e.Error == null) {
        
        // Convert the string xml representation to a valid XML document
        _xmlImageMetadata = XElement.Parse(e.Result);
        
        // The filter button is disabled by default. Enable it when the metadata is available
        buttonFilter.IsEnabled = true;
    }
}

3. Used Linq to filter the dataset

buttonFilter.Click += delegate(object sender, RoutedEventArgs e)
{
    IEnumerable<int> ZOrders = 
        from sceneNode in _xmlImageMetadata.Elements("SceneNode")
        where 
            (from tag in sceneNode.Elements("Tags").Elements("Tag")
            where
                ((string)tag.Value).ToUpper() == txtFilter.Text.ToUpper()
            select tag)
            .Any()
        select ((int)sceneNode.Element("ZOrder")-1);

    _imagesToShow.Clear();
    _imagesToHide.Clear();
    for (int i=0; i<msi.SubImages.Count; i++) {
        if (ZOrders.Contains(i))
            _imagesToShow.Add(msi.SubImages[i]);
        else
            _imagesToHide.Add(msi.SubImages[i]);
    }
    ArrangeImages();
};

TODO - Better animation.

Comments are welcome!

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 5

Trying to find out which image is clicked? Maybe this code snippet will help...

msi.MouseLeftButtonUp += delegate(object sender, MouseButtonEventArgs e)
{
    Point p = this.msi.ElementToLogicalPoint(e.GetPosition(this.msi));
    int subImageIndex = SubImageHitTest(p);
    if (subImageIndex >= 0)
        // look up a custom corresponding data structure or xml and return the metadata associated with this image
}

int SubImageHitTest(Point p)
{
    for (int i=0; i<msi.SubImages.Count; i++) {
        MultiScaleSubImage subImage = msi.SubImages[i];
        
        double scaleBy = 1 / subImage.ViewportWidth;
        Rect subImageRect = new Rect(-subImage.ViewportOrigin.X * scaleBy, -subImage.ViewportOrigin.Y * scaleBy, 
                                        1 * scaleBy, (1 / subImage.AspectRatio) * scaleBy);
        if (subImageRect.Contains(p))
            return i;
    }

    return -1;
}

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 4

Update Oct 14, 2008: Removed the example from this post to reduce load time of this page. Please use the example from this post, DeepZoom sample ported to Silverlight 2 Beta 2, to try the functionality.

Click on the Randomize button to randomly arrange the images.

Click on the Full Screen button to get into full screen mode. I am not sure if there is bug in Silverlight or I am doing something wrong but it seems like the MultiScaleImage height and width is not reset after it gets out of the full screen mode! Also, the keyboard seems to be disabled in full screen mode (except for the ESC key)!

Click on the Set Source button to set the source for the MultiScaleImage control to the one specified in the text box to the left of the button. This should be a fully qualified URI and should point to a valid items.bin file (http://.../items.bin). I could not get this to work across domains even after placing a crossdomain.xml file in the root folder of my alternate web host but maybe I was doing something wrong. It seems to work if the file is in the same host but for some reason it is not arranging the images. You can try this url and see what I mean - http://thepintospatronus.com/deepzoom/test2/items.bin

What I learned from this exercise so far...

• How to use some of the Siverlight 2 controls, position them on the screen so that they scale, and apply borders to them!
• How to enable full screen mode. The msdn help section refers to some Javascript code to achieve this but I wanted to do it in managed code. Finally figured it out...

  buttonFullScreen.Click += (sender, e) => Application.Current.Host.Content.IsFullScreen = !Application.Current.Host.Content.IsFullScreen;
     
  Application.Current.Host.Content.FullScreenChanged (sender, e) => // Will be called when the control renders in full screen;
  Application.Current.Host.Content.Resized (sender, e) => // Will be called when the control gets out of full screen;

• How to host this puppy! Initially I tried silverlight.live.com but then realized that I couldn't upload the deep zoom images up there. So I used my host to upload the images but then realized that the control located at silverlight.live.com couldn't access the images from my host - I even played around with the crossdomain.xml with no success. I then decided to host everything on my host (bluehost.com) since they did support the .xap mime type without any additional tweaking

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 3

In Part 2 we saw the code for arranging the images in a nice clean layout. This post will discuss when to call the ArrangeImages() method.

When the page that contains the Silverlight control is first loaded, the MultiScaleImageControl will load the images and render it on the screen. The issue here is that the ArrangeImages method hasn't had a chance to do its magic yet! Hence we have to intercept the MultiScaleImage control somehow before it renders the images. This can be achieved by the following 2 steps:

1. Set UseSprings to false in the Xaml

<MultiScaleImage x:Name="msi" UseSprings="false"/>

2. Intercept the MultiScaleImage.Motionfinished Event. Call ArrangeImages in this handler

msi.MotionFinished += delegate(object sender, RoutedEventArgs e)
{
  // This is required to ensure that ArrangeImages is called only once
  if(msi.UseSprings == false) {
      ArrangeImages();
      msi.UseSprings = true;
  }
};

In addition, the ArrangeImages method can be called on demand - as in a button click or after the dataset is filtered.

Here is a sample image of what the output looks like after ArrangeImages is called on 17 arbitrarily cropped images

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 2

In Part 1 of this series, I discussed the logic to arrange the images in a grid - where the height of all images in a row is the same but can change across rows. The objective is to maintain the aspect ratio of the image and to not stretch it. This post provides some sample code snippets that implement the algorithm.

Some considerations

1. The code caters to spacing between images even thought this was not discussed in the logic
2. The _imagesToShow and _imagesToHide collections should be populated with MutliScaleSubImages prior to calling ArrangeImages
3. There is 1 condition that is not addressed: if the height of all rows exceed the height of the container - it can be ignored for now if clipping is acceptable
4. This code is based off the sample code published by Scott Hanselman and hence the use of the variable msi (which is an instance of MultiScaleImage)
5. I am sure this code and/or logic can be improved upon so please feel free to leave your comments

List<MultiScaleSubImage> _imagesToShow = new List<MultiScaleSubImage>();
List<MultiScaleSubImage> _imagesToHide = new List<MultiScaleSubImage>();

public void ArrangeImages()
{
   double containerAspectRatio = msi.Width / msi.Height;
   double spaceBetweenImages = 0.005;

   List<SubImage> subImages = new List<SubImage>();
   _imagesToShow.ForEach(subImage => subImages.Add(new SubImage(subImage)));
  
   // Capture the total width of all images
   double totalImagesWidth = 0.0;
   subImages.ForEach(subImage => totalImagesWidth += subImage.Width);

   // Calculate the total number of rows required to display all the images
   int numRows = (int)Math.Sqrt((totalImagesWidth / containerAspectRatio)+1);
  
   // Create the rows
   List<Row> rows = new List<Row>(numRows);
   for (int i=0; i<numRows; i++)
       rows.Add(new Row(spaceBetweenImages));

   double widthPerRow = totalImagesWidth / numRows;
   double imagesWidth = 0;
   // Separate the images into rows. The total width of all images in a row should not exceed widthPerRow
   for (int i=0, j=0; i<numRows; i++, imagesWidth=0) {
       while (imagesWidth < widthPerRow && j < subImages.Count) {
           rows[i].AddImage(subImages[j]);
           subImages[j].RowNum = i;
           imagesWidth += subImages[j++].Width;
       }
   }

   // At this point in time the subimage height is 1 
   // If we assume that the total height is also 1 we need to scale the subimages to fit within a total height of 1
   // If the total height is 1, the total width is aspectRatio. Hence (aspectRatio)/(total width of all images in a row) is the scaling factor.
   // Added later: take into account spacing between images
   rows.ForEach(Row => Row.Scale(containerAspectRatio));

   // Calculate the total height, with space between images, of the images across all rows
   // Also adjust the colNum for each image
   double totalImagesHeight = -spaceBetweenImages(numRows - 1) * spaceBetweenImages;
   rows.ForEach(Row => totalImagesHeight += Row.Height + spaceBetweenImages);
   Debug.Assert(totalImagesHeight <= 1.0); // TODO: Handle this condition

   // The totalImagesHeight should not exceed 1. 
   // if it does, we need to scale all images by a factor of (1 / totalImagesHeight)
   // This takes care of consideration #3 in the text above
   if (totalImagesHeight > 1) {
        subImages.ForEach(subImage => subImage.Scale(1 / (totalImagesHeight+spaceBetweenImages)));
        totalImagesHeight = (numRows - 1) * spaceBetweenImages;
        rows.ForEach(Row => totalImagesHeight += Row.Height);
        Debug.Assert(totalImagesHeight <= 1);
   }

   // Calculate the top and bottom margin
   double margin = (1 - totalImagesHeight) / 2;

   // First hide all the images that should not be displayed
   _imagesToHide.ForEach(subImage => subImage.Opacity = 0.0);

   // Then display the displayable images to scale
   for (int i=0; i<_imagesToShow.Count; i++) {
       _imagesToShow[i].Opacity = 1.0;    // in case this was hidden earlier

       double X = rows[subImages[i].RowNum].CalcX(subImages[i].ColNum);
       double Y = margin + ((subImages[i].RowNum == 0) ? 0 : rows[subImages[i].RowNum-1].Height + spaceBetweenImages);
       for (int j=0; j<subImages[i].RowNum; j++)
            Y += spaceBetweenImages + rows[j].Height;
      
       _imagesToShow[i].ViewportWidth = containerAspectRatio/ subImages[i].Width;
//      AnimateImage(_imagesToShow[i], new Point(-(X / subImages[i].Width), -(Y / subImages[i].Width)));    // for animation, use this statement instead of the next one
       _imagesToShow[i].ViewportOrigin = new Point(-(X / subImages[i].Width), -(Y / subImages[i].Width));
   }
}

The helper classes:

public class SubImage
{
   private Size _imageSize = new Size(0.0, 1.0);    //normalized height is 1 for all images
   private int _colNum = 0;
   private int _rowNum = 0;
  
   public SubImage(MultiScaleSubImage image)
   {
       // Capture the normalized size of each image (fixed height of 1)
       // This normalization is required since we want the height of all images to be the same but the widths can differ
       _imageSize.Width = image.AspectRatio;
   }

   public int ColNum
   {
       get { return _colNum; }
       set { _colNum = value; }
   }

   public int RowNum
   {
       get { return _rowNum; }
       set { _rowNum = value; }
   }
  
   public double Width  { get { return _imageSize.Width; } }
   public double Height { get { return _imageSize.Height; } }

   public void Scale(double scaleBy)
   {
       _imageSize.Width *= scaleBy;
       _imageSize.Height *= scaleBy;
   }
}

public class Row
{
   List<SubImage> _subImages = new List<SubImage>();
   double _spaceBetweenImages = 0;
  
   public Row(double spaceBetweenImage)
   {
       _spaceBetweenImages = spaceBetweenImage;
   }
  
   public double TotalWidth
   {
       get
       {
           double totalWidth = 0;
           _subImages.ForEach(image => totalWidth += image.Width);
           return totalWidth;
       }
   }

   public int ImageCount {get{return _subImages.Count;}}
   public double Height {get{return (ImageCount <= 0) ? 0 : _subImages[0].Height;}}
   public double TotalSpaceBetweenImages {get{return (ImageCount <= 0) ? 0 : (ImageCount-1) * _spaceBetweenImages;}}

   public void AddImage(SubImage subImage)
   {
       _subImages.Add(subImage);
       subImage.ColNum = _subImages.Count - 1;
   }
  
   public void Scale(double canvasWidth)
   {
       double scaleBy = (canvasWidth - TotalSpaceBetweenImages) / TotalWidth;
       foreach (SubImage subImage in _subImages)
           subImage.Scale(scaleBy);
   }
  
   public double CalcX(int colNum)
   {
       Debug.Assert(colNum < _subImages.Count);

       double X = 0;
       for (int i=0; i<colNum; i++) {
           X += _subImages[i].Width + _spaceBetweenImages;
       }
       return X;
   }
}

Deep Zoom layout.bin file decoded

This is a follow-up to my previous post (Deep Zoom items.bin file decoded)...
The reason for decoding the layout.bin is that the files items.bin and layout.bin are dependent on each other. For the adventurous few who may now decide to play around with the items.bin file, it may be necessary to modify the layout.bin file as well!

Bytes (Hex)

00-04  ItemCount

Item information (Repeat for each item)
05-08  Item Id
09-0C  ViewportOrigin.X
0D-10  ViewportOrigin.Y
11-14  ViewportWidth

Pretty straightforward! I played around with these files and was able to add and delete items easily.

Deep Zoom items.bin file decoded

Thanks to George Bell (check out the comment section on this Expression Team blog), I became aware of a undocumented command line tool, sparseimagetool.exe, that is bundled with the Deep Zoom composer. This is a nifty tool which can generate a collection given a simple xml file. This helps when one needs to process a large number of images. The xml file should be created along the lines of the SparseImageSceneGraph.xml that is output by the Deep Zoom composer.

Usage: SparseImageTool CreateCollection

While I was playing with this tool, I noticed a 'Use XML' format option. I tried it and it generated a items.xml file instead of a items.bin file. While this xml file cannot be used as input to the MultiScaleImage I thought it would be interesting to see if I could decode the items.bin file from this xml file. I don't know what I am going to achieve by decoding this file but it does bring up interesting possibilities; one possibility could be - to implement filtering by modifying the items.bin file itself instead of playing around with the Opacity of each MultiScaleSubImage. This might seem overkill but is worth investigating for a large collection!

This is what my sample items.xml file looks like:

<?xml version="1.0" encoding="UTF-8"?>
<Collection Version="2" UseStringsFile="0" MinLevel="0" MaxLevel="8" PageSizeLog2="8" PageFormat="jpg" PageQuality="1" NextItemId="6" ItemCount="6">
<Thumbnails>
   <Thumbnail Id="0" MinLevel="0" MaxLevel="8" SizeX="0.6953125" SizeY="0.521484375"/>
   <Thumbnail Id="1" MinLevel="0" MaxLevel="8" SizeX="0.6953125" SizeY="0.521484375"/>
   <Thumbnail Id="2" MinLevel="0" MaxLevel="8" SizeX="0.6953125" SizeY="0.521484375"/>
   <Thumbnail Id="3" MinLevel="0" MaxLevel="8" SizeX="0.6953125" SizeY="0.521484375"/>
   <Thumbnail Id="4" MinLevel="0" MaxLevel="8" SizeX="0.6953125" SizeY="0.521484375"/>
   <Thumbnail Id="5" MinLevel="0" MaxLevel="8" SizeX="0.6953125" SizeY="0.521484375"/>
</Thumbnails>
<Items>
   <Item Id="0" Init="test2_images\DSCF0067.sdi" Relative="1" Type="ImagePixelSource" Thumb="0"/>
   <Item Id="1" Init="test2_images\DSCF0078.sdi" Relative="1" Type="ImagePixelSource" Thumb="1"/>
   <Item Id="2" Init="test2_images\DSCF0082.sdi" Relative="1" Type="ImagePixelSource" Thumb="2"/>
   <Item Id="3" Init="test2_images\DSCF0087.sdi" Relative="1" Type="ImagePixelSource" Thumb="3"/>
   <Item Id="4" Init="test2_images\DSCF0090.sdi" Relative="1" Type="ImagePixelSource" Thumb="4"/>
   <Item Id="5" Init="test2_images\DSCF0061.sdi" Relative="1" Type="ImagePixelSource" Thumb="5"/>
</Items>
</Collection>

I used the above XML file to decode the corresponding items.bin file

Bytes (Hex)

00     Version            (2)
01-04  UseStringsFile     (0)
05-08  MinLevel           (0)
09-0C  MaxLevel           (8)
0D-10  PageSizeLog2       (8)
11-14  ?? - PageQuality?  (1)
15-1C  ?? - PageFormat?   (jpg)
1D-20  NextItemId         (6)
21-24  ItemCount          (6)

Thumbnail information for 1 thumbnail (Repeat for each thumbnail)
25-28  Thumbnail Id       (0)
29-2C  MinLevel           (0)
2D-30  MaxLevel           (8)
31-34  SizeX              (0.6953125)
35-38  SizeY              (0.521484375)

Item information (Repeat for each item)
9D-A0  Item Id            (0)
A0-A4  Init Length        (24)
A4-BC  Init               (test2_images\DSCF0067.sdi)
BD     Relative           (1)
BE-C1  Type Length        (16)
C2-D1  Type               (ImagePixelSource)
D2-D5  Thumb              (0)

Dissecting Hard Rock Memorabilia and Silverlight Deep Zoom - Part 1

I have been following some of the Silverlight presentations on Mix 2008 and was totally blown away with the Hard Rock Memorabilia demo. Vertigo had created a truly compelling site! I was hoping that they would put some of their source code online, but since this did not happen, I decided to play around with Silverlight and Deep Zoom myself and attempt to create a similar experience.

There were some parts of the puzzle that needed to be addressed before a similar experience could be created:

1. Mouse and keyboard handlers, including Mouse Wheel support, for panning and zooming the Deep Zoom image
2. Figuring out how to manipulate the Deep Zoom images as a collection instead of a single large image
3. Arranging the images in a grid, where the height of all images in a row is the same but can change across rows. This gives a clean look to the images
4. Animating the images so that they appear to slide into position
5. Adding extra information to each image so that they can be filtered on
   

Part 1 is easily solved since a lot of people have already provided solutions for the mouse and keyboard handlers. Some of the contributors who caught my attention: Scott Hanselman, Soul Solutions Blog, and the Expression Team at Microsoft.

I figured out Part 2 myself but noticed that the Expression Team already posted a solution for this. The DeepZoom composer provides a way to export the images as a collection. The images can then be accessed using the MultiScaleImage.SubImages collection.

The Expression Team has provided a partial solution to Part 3. They arranged the images in a grid but did not arrange it such that the height of all images in a row remain the same. I will attempt to provide an algortihm for this in this post and maybe provide some code snippet in the next post. I do not have access to Visual Studio 2008 at this time and hence am using notepad++ to write code. I am actually working off Hanselman's source code so I will wait until I create my own project before posting the entire source code. In the meantime, I will try to provide some code snippets...

Once again, the Expression Team has provided a very good sample for animating images, which addresses Part 4. This is one area I was struggling with since I lack animation skills. I am learning more everyday but for now I am happy to borrow this part of the code from the Expression Team!

I haven't reached Part 5 yet but I suspect that the Name/Tag property on the MultiScaleSubImage can be used to store additional information regarding the image. At the very least, a pointer of some sort can be stored in the Tag property which can then be used to retrieve more information on the image.

Now for the algorithm for Part 3 (arrange images in a grid, where the height of all images in a row is the same but can change across rows)

1. Infer the MultiScaleImage aspect ratio - MultiScaleImage.Width / MultiScaleImage.Height
   
2. Capture the normalized width of each image - What I mean by this is that the height of each image should remain the same. Assuming that we want the height to be a fixed '1', the width would equal the aspect ratio of the image (not the MultiScaleImage aspect ratio but the aspect ratio of the SubImage itself)
   
3. Capture the total normalized width of all images - This is the sum of the normalized widths of each image
   
4. Calculate the number of rows required to display the images - For this we need to normalize the heights of all images within the MultiScaleImage container. This can be achieved by dividing the total normalized width of images by the MultiScaleImage aspect ratio: Number of rows = Sqrt of [total image width (3.) / MultiScaleImage aspect ratio (1.)]
5. Capture the width per row - total images width (3.) / Number of rows (4.)
   
6. For each row, capture all the images that will fit in that row. This is based on the width of each image (2.) and the max width per row (5.)
7. At this point in time, each row will contain a set of images which may or may not take up the width allocated to the entire row. Scale the rows by a factor of MultiScaleImage aspect ratio (1.) / total normalized width of images in a row. This will ensure that images in a row will fill up the width of the entire row. This will also normalize the height of all rows such that the sum of height of all rows will not exceed '1' (the normalized height of the MultiScaleImage)
   
8. After step 8, all the rows should contain images that fill up the width of the entire row but the row height between rows may differ. By this we ensure that the images will fill the width of the container (MultiScaleImage) but may not fill up the height. Hence we need to calculate the top and bottom margin for display purposes
   
9. Total Margin = 1 - total normalized height of rows. Top and bottom margin = Total margin / 2
10. Un-normalize the images by scaling them back to the actual container size and display them - SubImage.ViewportWidth = MultiScaleImage aspect ratio / SubImage.width, SubImage.ViewportWidth = Point(X / SubImage.width, Y / SubImage.width) where X and Y are calculated based on the row/col value and the prior images width

That should do it! I will try to post some code snippet in subsequent posts.

A few tips (I discovered along the way...)

• SubImage can be hidden/shown by controlling the Opacity property. This is useful for implementing filter functionality
• Instead of setting the MultiScaleImage Source property in the Xaml, a better place to set it is in the MultiScaleImage.Loaded event. This way the entire html page with the Silverlight control will be loaded and then the MultiScaleImage control will try to load the images. This enhances the overall user experience