Windows Vista

1 Comment

One of my post popular blog posts deals with problems I was having with the stability of Windows Vista. I had major problems hibernation and sleep modes on my Dell PC. My PC never reliably went to sleep or hibernation and crashed either during going to sleep or when waking up. A lot of other people encountered similar problems. This blog post has 66 comments so far.

No driver update or Windows update solved my problem until October 9, 2007.

Around that time Microsoft released a set of reliability updates that are going to be part of SP1. There are supposedly also going to show up in Windows Update, but I manually downloaded and installed them ahead of time. Note that I am not running the beta of SP1.

If you encounter similar problems with Windows Vista, I highly recommend that you install these updates as well (if you haven't got them through auto update yet):

KB941649 

This update resolves some compatibility and reliability issues in Windows Vista. By applying this update, you can achieve better reliability and hardware compatibility in various scenarios.

KB941600

This update resolves some reliability issues in the USB core components on the Windows Vista operating system.

I regularly sync my Windows Mobile phone with Outlook 2007 using ActiveSync/Vista Sync Center. I don't know what causes this, but every so often I end up with duplicate contacts and calendar items. This happens especially with recurring items like birthdays. If I am not quick enough to remove the duplicates, I end up with 2, 4 or even 8 of them!

In the past I have used the Duplicates Remover for Outlook from MAPILab for this. I used the trial edition that works for 30-days. It is limited to removing 10 items at a time, so I had to run it multiple times. I didn't use it often enough to shell out the $24/€19 to buy it.

Today I tried version 2.5.2 of this Outlook add-in. It has a wizard-like interface that is easy to use. I got my contact list and calendar cleaned up again.

[Full disclosure: By writing this review I am eligible for a free license for this Outlook add-in. This does not effect my opinion about this tool. Obviously, if I wouldn't find it useful, I wouldn't want to have a license.]

1 Comment

After a long day and night of coding, I released version 0.8.0.0 of my Flickr Metadata Synchr tool on CodePlex this morning. I finally solved the long-standing problem I was having with the Windows Imaging Component (WIC) to update metadata. So this is the first fully-functional release of my application.

Functionality of FlickrMetadataSynchr v0.8.0.0

This is what the app does:

  • It allows you to select a set of your photos on Flickr and a folder on your hard drive with images.
  • It reads the metadata for both Flickr images and the local images. The metadata that is read is:
    • Title
    • Description
    • Author
    • Tags
    • Geo-info (GPS coordinates)
    • Date and time taken
    • Last update date and time
  • It matches images on Flickr with local images based on the date and time taken.
  • It determines on a per picture basis in what direction the metadata should by synced, i.e., which side should be updated, if any. Currently the most recently updated side wins. I am getting help from Timo Proescholdt for a better algorithm that will allow for a merge of metadata, i.e., a two-way synch.
  • It updates the metadata on Flickr and in the local images.

This is a screenshot of the app:

 

Previous posts on this tool

Workaround for WIC problems

During my holiday in France in July I received e-mail from Robert A. Wlodarczyk who works at Microsoft. He pinged me to say that he had released new sample code to update metadata using WIC. Yesterday, I tried to incorporate similar code into my application and ran into the same type of problems as before.

Because his sample was working, I wasn't ready to give up again. I finally tracked the problem down to a threading issue. WIC is throwing strange InvalidOperationException and InvalidFormatException exceptions with messages like "Cannot write to the stream" when it is called from a background thread. My app is multi-threaded so that the UI doesn't hang when it is busy syncing.

After I got confirmation from Robert that WIC indeed suffers from a threading issue, I solved the problem with a work around. I now marshal the call to the code that uses WIC to update metadata to the UI thread using the WPF Dispatcher object. This causes the app to become non-responsive for small amounts of time during the update of local metadata. But that is better than a non-fully-functional app.

Fully functional, give it a try

So all is well that ends well. After finally getting WIC to work, I could do away with the C++ code that was causing me headaches 😉 And my app now works on Windows XP again. You just need to have the .NET Framework 3.0 installed.

If you have images on Flickr and you have been busy tagging them, give my app a spin! You can always find the latest release on CodePlex. The source code is also available under a GPL license on CodePlex.

Installing is easy. You just need to unzip the ZIP-file, which contains three files, to a folder. Start the FlickrMetadataSynchr.exe file and you are done. The app remembers the last settings.

If you find any issues, please report them using the Issue Tracker for my app on CodePlex.

The Future

Even though the app is now able to sync metadata in both Flickr images and local images, there is always room for improvement. Here are my ideas, some of which are based of suggestions by people on CodePlex:

  • Improve the synchronization to also allow two-way synchronization for a picture pair. I.e., one side doesn't have to win. For example, if the Flickr image has just the title set and the local image the description, the metadata should be merged.
  • Add UI to see the match that is made by the tool and how it proposes to sync the metadata.
  • Allow you to exclude images if the match isn't good.
  • Allow you to overrule the sync proposal and sync the metadata in a different direction (on a per property basis) .
  • Add UI to store multiple mappings between Flickr sets and local folders. Currently the app only remembers the last folder and Flickr set that was used.
  • Add click-once deployment. That way the app can automatically check for new versions and update itself.

If you have any other ideas please post them at the discussions page for my app on CodePlex.

1 Comment

PDC07 has been postponed indefinitely. Dennis already warned me just before MIX07 that Microsoft wouldn't have anything to announce at the PDC after the MIX conference.


No new date for a PDC has been given, yet Microsoft already announced the date for MIX08! It will be held March 5-7, 2008 in Las Vegas.


So when will the next PDC be? The official statement by Microsoft is that it will be when they are ready to announce the "next wave of platform technologies".  The .NET technology is still forbidden from entering large parts of the Windows Operating System.


Will Microsoft attempt to pull off a synchronized wave of platform technologies  again? Synchronizing the release of



  • Visual Studio .NET 2005, .NET 2.0 and SQL Server 2005

  • .NET 3.0 and Windows Vista

proved to be mighty difficult for Microsoft to pull off. They only managed to release Windows Vista by cutting the dependencies between the OS and .NET. .NET 3.0 is only bundled with Windows Vista. Vista doesn't make use of it.


Vista even makes it very difficult to pull some things of using .NET. See my metadata API difficulty requiring me to use C++ and the fact that is is very difficult to create a decent Sidebar gadget using WPF.


The synchronization of .NET 3.5 and the ADO.NET Entity Framework failed. ADO.NET vNext has been delayed and will be released after the LINQ wave.


However, there is hope glimmering on the horizon. The new CoreCLR used by Silverlight 1.1 is able to co-exist with other CLR versions in one process. Silverlight will be a great technology for the Vista Sidebar. ASP.NET Ajax was released out-of-band. Microsoft has said that the next revision of the full CLR (remember .NET 3.5 still uses CLR 2.0) will also have this feature.

1 Comment

I just experienced my first Windows Vista crash that didn't even display a Blue Screen Of Death (BSOD). My computer just suddenly rebooted, complaining about an improper system shutdown.

So what did I do?

I was trying to mount a VHD file (Virtual Hard Disk) with the vhdmount.exe tool from Virtual Server 2005 R2 SP1 RC. Apparently that made Windows Vista very unhappy.

So why did I want to mount a VHD file?

Windows Vista comes with the CompletePC Backup tool that allows you to backup an entire harddisk to a VHD file on an external hard drive. CompletePC Backup only allows you to restore the entire image in the case of hardware failure or system corruption. This makes it less then useful (as in completely useless) when you want to restore individual files. vhdmount.exe to the rescue! Or so you would think.

Is mounting a VHD file easy on Windows Vista?

No. This blog entry by the VirtualPC Guy would make you think that it is just a matter of installing the vhdmount.exe tool, importing some registry settings once, and just doubleclicking a VHD file. That doesn't actually work on Windows Vista with UAC turned on. The tool requires elevated priviliges and the Windows Explorer process doesn't have them. The tool doesn't initiate a UAC prompt to elevate. So you have to run a command line (cmd.exe) with administrative priviliges and run the vhdmount.exe tool from there.

Why did I want to restore some files?

I haven't actually lost data yet. I just wanted to make sure that I can restore my data in the unfortunate case that disaster chooses to strike in the future. The recent CodePlex disaster made me look into my own backup strategies again.

Now I know, that my backup strategy doesn't work ;(

Why didn't my backup strategy work?

My external USB-harddrive is formatted using the EXT2 filesystem (read here why) and I think the Ext2IFS driver doesn't play too well with vhdmount.exe. I'll have to investigate further, but the prospect of having to suffer through multiple crashes in the process is not very alluring.

The situation


I am currently working on version 0.6.0.0 of my Flickr Metadata Synchr tool. The goals for my open source project on CodePlex are described on the Flickr Metadata Synchr wiki page and you can always find the latest status there.


At the moment the latest public release is version 0.5.5.0. The feature set for v0.5.5.0 is roughly:



  • Allow you to select a Flickr photoset and a local directory with images.
  • Load metadata from both local and Flickr images into internal metadata structures.
  • Compare these metadata structures and synchronize them.
  • Update metadata on Flickr after the synchronization.

One of the features planned for v0.6.0.0 is updating the XMP and IPTC metadata in locally stored images. I was planning on doing this through the Windows Imaging Component (WIC) which is part of the .NET Framework 3.0. WIC is also available as a separate download for Windows XP and Windows Server 2003.


Windows Presentation Foundation provides a nice managed API for reading and writing metadata through WIC. It provides the SetQuery and GetQuery methods on the BitmapMetadata class. I was already using the GetQuery method, which works fine. However, I hit a snag when I wanted to use the SetQuery method to update metadata.


Plan A


There is a way to do this through the InPlaceBitmapMetadataWriter class. It just touches the metadata structures in the image file and doesn't have to read or write the entire stream with pixel information. This will give you excellent performance and so you don't run the risk of having to reencode the pixel stream or loosing metadata. The sad thing is that it almost never works. The image file often does not have enough room in its metadata structures to allow metadata fields to be filled or updated. When you try to save the updated metadata, the InPlaceBitmapMetadataWriter fails. That is probably why the save method is called TrySave. By the way, the code sample on that MSDN Page is dead wrong. If you call TrySave before updating metadata, it always succeeds. Probably because there is nothing to save yet. You have to call it after you update the metadata, and then it returns false ;( Which means your metadata was not updated successfully.


Plan B


So I tried plan B: Creating a new image file by writing out a copy of the original image, but now with updated metadata. This means you have to grab the original BitmapFrame from the JpegBitmapDecoder. Clone it, update its metadata and write it out again using the JpegBitmapEncoder.


This is where I hit a major problem. The Save() method on the JpegBitmapEncoder almost always fails with an InvalidOperationException with the error message "Cannot write to the stream". When the encoder is able to write out the image, the JPEG turns out to be reencoded with a different quality than the original. This is noticeable through a significant change in size of the file. This happens even though I specified the BitmapCreateOptions.PreservePixelFormat option when opening the image with the decoder. Googling (or Windows Live Searching if you will) for a solution didn't yield anything useful.


Plan C


I had to come up with a Plan C. The Windows Vista Shell is obviously able to update metadata in images without affecting the JPEG quality and without creating a copy of the image file. This led me to an MSDN article titled "Photo Metadata Policy". This is the introduction:



Metadata (file properties) for photo files can be stored using multiple metadata schemas, in different data formats and in different locations within a file. In Windows Vista™, the Microsoft® Windows® Shell provides a built-in property handler for photo files formats, such as JPEG, TIFF, and PNG, to simplify metadata retrieval.


When a piece of metadata is present in different underlying schemas, the built-in property handler determines which value to return . For instance, the Author property may be stored in the following locations in a TIFF file:


  • The Creator tag in the XMP Dublin Core schema:
    /ifd/xmp/purl.org/dc/elements/1.1/dc:creator

  • The Artist tag in the EXIF schema:
    /ifd/{ushort=315}

  • The Artist tag in the EXIF schema embedded in an XMP block:
    /ifd/xmp/ns.adobe.com/tiff/1.0/tiff:artist

On read, the property handler determines the value that takes precedence over the others that exist in the file and returns it. On write, the property handler makes sure it leaves each schema in a resolved and consistent state with the others. This may mean either updating or removing the tag in question in a given schema.


This would also help to solve another piece of the metadata puzzle: what to do with the several different options of putting metadata in image files (XMP versus IPTC, multiple possible XMP places, etc.). After updating an image, I want the metadata in the different blocks to be consistent. WIC doesn't help with this. You have to sort it out yourself. The Windows Vista Shell does help with metadata reconciliation.


So all seems to be well. Just use the Shell API to update the metadata. I would love to be able to do this from C#. Yet that doesn't seem to be possible or it is extraordinarily difficult. The "file property" handling is implemented in propsys.dll through a COM based API. But you can't add a reference to this COM library in a C# project. It doesn't have a type library ;( The only option I can find is to use C++ and use the propsys.h and propsys.idl files that are distributed in the Windows SDK. This is horrible. I guess I have to dust off my C++ skills to be able to call a brand-new Windows Vista API. WTF?!


The "Longhorn" promise for managed code


Do you remember the promises Microsoft made back in 2003 for the new Windows Client OS codenamed "Longhorn"? I sure do, since I visited the PDC03 conference where this was all announced. Microsoft promised us a brave new world where all Windows APIs could be accessed easily from managed code. Three and a bit years later we have a new Windows Client OS called Vista that doesn't live up to this promise. Microsoft has implemented new APIs that seem to be inaccessible from managed code other than through C++.


Now I can understand why part of the promise was lost during the infamous "Longhorn Reset" at Microsoft. Microsoft's ambition to completely wrap all existing Win32 APIs in WinFX was too big. But why Microsoft would be creating new APIs without managed code in mind is beyond me...


I found some C++ code on the blog of Ben Karas that is indeed able to update metadata  I hate having to add this C++ code to my project. It would require people to have Visual C++ and the Windows SDK (especially the Windows Vista header and library (*.h, *.idl, *.lib) files) installed to be able to build my code in Visual Studio.


A plan D might be to manually create C# wrappers for the COM interfaces of propsys.dll. This article describes how to do this for COM interfaces in general.

1 Comment

[Update 2007-02-24: My Flickr Metadata Synchr project is now live on CodePlex. I will keep the Wiki page on CodePlex updated and will not further update this blog post. If you are interested in the progress, you can subscribe to the Flickr Metadata Synchr project's RSS feed.]

I blogged previously on my digital workflow for photographs here and here. In that last post I described how I violated the golden rule of metadata. Now, I am thinking of writing an application that will fix this for me. I already thought of a name: the Flickr Metadata Synchr. Once I get a skeleton version of the app up-and-running, you will be able to view this project on CodePlex. If I fail miserably, the project will never show up 😉

Here is what I am planning of building:

Name of the application

Flickr Metadata Synchr

Purpose

To synchronize relevant metadata added to images stored on Flickr with the original versions of those images stored locally on your hard drive.

Relevant metadata on Flickr is:

  • Title
  • Description
  • Tags
  • GPS location info

The synchronization will be one-way in the first release. The embedded metadata in EXIF, IPTC and XMP sections in locally stored images can be updated with the metadata of matching pictures on Flickr.

Functionality

  • Ability to authenticate the user with Flickr, i.e., obtain a user token to let the app have read permissions for the user on Flickr.
  • Ability to select a folder with pictures on a local hard disk or network drive. Optionally include subfolders. These pictures will be called the local set.
  • Ability to select a set of pictures on Flickr. This set should belong to the authenticated user. Or the ability to select all pictures of the user tagged with a certain keyword. These pictures will be called the Flickr set.
  • Ability to let the app match pictures between the local and the Flickr set. Match will be done on “date captured”.
  • Ability for the user to cancel the matching process which might be time-consuming.
  • Ability to show the user the match that was made by the application to give the user the opportunity to review for any matching errors.
  • Ability to show the relevant metadata for pictures in both the local and the Flickr set.
  • Ability for the user to deselect matched image pairs that he or she does not want to process.
  • Ability to update the matched local images with metadata from Flickr.
  • Ability for the user to cancel the updating process which might be time-consuming.

Possible future functionality

  • Cache metadata retrieved from Flickr between app restarts to improve performance. Ask the user if it is okay to not update the data. If the user has made significant updates on Flickr, caching will cause more harm than good.
  • Ability to sync Flickr metadata the other way around. Read metadata from local images and update the metadata of matching pictures on Flickr.

Dependencies

Development tools

1 Comment

 


I am still looking for a good solution for my digital workflow for pictures on Windows Vista. I want to be able to easily tag my pictures locally on my hard-drive and on Flickr and keep the metadata in sync.


Let me describe the state of affairs.


Metadata in images


With metadata I mean the info I add manually to a picture, like the title, author, description, tags and GPS info. A digital camera already adds lots of other data to your pictures like aperture, camera model, if the flash fired or not, etc. This info is usually stored in an EXIF section in the JPEG file. For other metadata there are several options: EXIF, IPTC of XMP. IPTC is an older standard. XMP is more flexible (e.g., it allows Unicode) and modern.


The Golden Rule


This page states the golden rule of metadata:



Store the metadata in your images


Adding metadata to pictures


While in Redmond two weeks ago, I uploaded a subset of the pictures that I took to Flickr. I added the metadata through the site. Flickr does not embed this metadata in the file. If you download the original image file, the title, description and tags are gone. So I basically violated the Golden Rule.


A better option is to add the metadata before you upload your images to Flickr. Locally I can embed the metadata as IPTC or XMP in the JPEGs using a variety of tools. This way your metadata flows with your image to Flickr. If you download it again, the metadata is stil there.


The easiest option to add metadata to your images is to use the Windows Photo Gallery built into Windows Vista. This stores metadata in an XMP section in the image file.


Unfortunately, Flickr only imports metadata from EXIF and IPTC and not from the XMP section. So all the images that you tag in Vista show up untagged on Flickr.


Vista Flickr Uploader


My colleague Matthijs (who still won't reboot his blog, not even to market his own app 😉 has written a Vista Flickr Uploader tool to solve this problem.


His Vista Flickr Uploader app is written in C# and uses .NET 3.0, most notably Windows Presentation Foundation and the Windows Imaging Component. He open sourced it through CodePlex. This tool solves one way of the problem. When uploading your pictures, it extracts XMP info from your pictures and adds it to Flickr as metadata.


Microsoft Photo Info


As an alternative to tagging through Windows Vista Photo Gallery, you might want to try out the Microsoft Photo Info tool. This is a free download from Microsoft that works on both XP and Vista. It integrates into the Windows Explorer. The great thing about this tool is that it can read and write both IPTC and XMP info. So your metadata is recognized by Flickr when you upload your images.


Embedding metadata from Flickr


So Vista Flickr Uploader or Microsoft Photo Info solve one way of the problem: getting metadata from your images onto Flickr. The other problem still remains: getting metadata from Flickr into your images. I found one tool (.NET 2.0 based) that is able to download images from Flickr and embed the Flickr metadata as IPTC. It was quite unstable and wasn't really able to find my images so I won't link to it.


Remaining problem


Another problem that this download tool caused was inconsistency between the IPTC and XMP info in the image files. Imagine the following scenario:



  • Tag your images with XMP info in Vista.

  • Upload them with the Vista Flickr Uploader.

  • Flickr now shows the metadata from the XMP.

  • Change the title, description or tags on the Flickr site.

  • Download the image from Flickr with a tool that embeds the Flickr metadata into IPTC.

You end up with an image file with both IPTC and XMP info. The XMP info is out-of-date, yet it takes precedence in Windows Vista ;(


So I am looking for a tool that downloads pictures from the Flickr site that embeds the Flickr metadata as both IPTC and XMP info or just as XMP. I wasn't able to find such a tool, so I will probably have to write my own app to do this.


[Update 2007-08-31: I have now written such a tool: FlickrMetadataSynchr.]


ExifTool


A possible alternative might be to use a really powerful metadata tool and library written in Perl that allows you to extract and embed metadata in almost all known formats. Phil Harvey's ExifTool can even be used to read ID3 info from MP3 files. I could use it to wipe XMP info from images so the IPTC info is used again by Vista.


The problem with this command line tool is that it works really low-level with the potential of damaging the metadata beyond repair. I.e., other programs refuse to load your images because they can't make sense of the metadata.

1 Comment

The global launch of Windows Vista yesterday brought a large number of updates with it. Yesterday I installed most of these updates (*). ATI released a new set of Catalyst drivers for my Radeon video card. And Microsoft made Vista drivers available for my new Windows LifeCam.

After these updates I noticed that my computer successfully goes into and comes out of hibernation. Previously I experienced severe problems with sleep and hibernation. These are gone now.

There is one major issue still bugging me. When Vista comes out of sleep or hibernation it cannot connect to the Internet. The wireless connection with my WiFi access point becomes available again, i.e., it shows up as connected. But Windows Vista says it has no Internet connectivity and indeed I cannot ping my SpeedTouch 716 ADSL modem. Running "Diagnose and repair" does not help. The only option to get network connectivity back is to disconnect from my wireless network and manually connect again.

All in all, I am making progress and I do not regret the switch from XP to Vista like Dennis does. I am also happily using Vista at work.

*) Out of the available updates, I installed only the Dutch language pack and not all other exotic languages that became available 😉 But where is the Dutch text-to-speech version?! And will there be any other English language voices other than Microsoft Anna?!

1 Comment

[Update 2007-10-18: Check out my blog post on two new Windows updates that solved this problem for me]

Just when I thought that I had tackled my hibernate problems in Windows Vista, I encounter serious problems with the sleep option.

When my computer goes to sleep it crashes and reboots. When my computer was idle this happened every hour, because the Balanced power plan puts the computer to sleep after one hour of inactivity. So I changed my power plan to High Performance so the computer never goes to sleep automatically ;(

Yesterday I reinstalled Windows Vista, mainly to rearrange my hard disks, but the problem remains. Albeit without a reboot after the crash.

There are many others that encounter crashes as well according to the post Windows Vista's Hideous Wakeup Support.

My hardware configuration is:

Dell Dimension 8300 with Pentium 4 3.0 GHz, 1 GB RAM, ATI Radeon 9800 PRO, Soundblaster Audigy 2 and Hauppauge WinTV PVR PCI II.

Windows Vista has a new reliability monitor that recorded the crash last night:

Does anyone else, with or without a similar hardware configuration, experience such problems?