As you will probably know, Visual Studio 2010 and .NET Framework 4 will RTM on April 12, 2010 and will be available for download on MSDN Subscriptions Downloads the same day.
The Windows Azure team is committed to making .NET Framework 4 available in Windows Azure within 90 days of the RTM date.
A lesser known fact is that the latest available Windows Azure build already has a .NET 4 version installed, namely the RC bits. Although this version cannot be used to run applications on (because .NET 4 is not yet exposed in the Windows Azure dev tools), you can use this build to test if the presence of .NET 4 has impact on existing .NET 3.5 apps running on Windows Azure.
Read the official announcement here.