Enterprise Library

1 Comment

The Microsoft Enterprise Library has always been one of the most popular things to come out of the patterns & practices team. Yesterday p&p reached a major milestone by releasing version 5.0 of EntLib.

The improvements are too numerous to sum up here, but let me mention one: this release has full .NET 3.5 SP1 and .NET 4 compatibility and works great from both Visual Studio 2008 SP1 and Visual Studio 2010 RTM.

Full details can be found in Grigori Melnik’s blog post on this release. Or you can go straight to the download page or the documentation.

1 Comment

As announced on Tom Hollander's blog, the Microsoft Enterprise Library 3.0 will include a new application block: the Policy Injection Application Block.


Enterprise Development Reference Architecture (EDRA)


Edward and me noticed a striking similarity with an earlier effort by Microsoft Patterns and Practices. For some other people who have been following the P&P guidance for some years now, this similarity didn't go unnoticed as well. For example in the post: Can anyone say Shadowfax?


"Shadowfax" was the codename for the Enterprise Development Reference Architecture (EDRA) released by Microsoft in 2004.


One of the important goals for EDRA was the separation of business logic from cross cutting concerns in enterprise applications. This was implemented by providing a pipeline of pre- and posthandlers that could be inserted declaratively using configuration in XML format. Messages would pass through this pipeline before reaching the business logic. The response would go back through the pipeline as well. EDRA handlers could inspect and even alter the messages flowing through the pipeline.


One of the other important goals for EDRA was the ability to physically separate the service interface from the service implementation. As in distributing these layers across different tiers and across security boundaries.


Our business unit at LogicaCMG followed and evaluated this effort in 2004 and even used it in some projects. We especially liked the fact that this was a ready-made framework for "policy injection". We made some extensions for cross cutting concerns not originally included in EDRA. Previously, we worked on a home-made framework that was based on .NET remoting extensibility to configure cross-cutting concerns. Although architecturally sound, it was far from complete because it still had a long way to go to fulfill our vision.


EDRA also included an early prototype of the Guidance Automation Toolkit to help framework users with building an EDRA based service. This was known as the "Microsoft IPE Wizard Framework". Check out this blog post from Daniel Cazzulino to see the EDRA wizard framework in action.


Eventually we reluctantly decided to drop EDRA. Some of the reasons for this were:



  • The EDRA wizards were hard to extend.
  • You had to mess with a big XML file to configure handlers.
  • No proper .NET 2.0 support.
  • No WS-* support.
  • No support for calling out into other services from the service implementation.
  • No big adoption in the worldwide .NET community. Not a lot of publicly available handlers.
  • No clear path towards Windows Communication Foundation. The internal messaging structure was not based on SOAP.

Especially the last reason was probably the biggest reason why Microsoft decided to stop the P&P effort on EDRA. The 1.1 release was announced to be the final release.


The Microsoft Enterprise Library has become far more successful with respect to adoption world-wide than EDRA. There are several EntLib extensions freely available. I've contributed my RollingFileTraceListener to the greater EntLib community. According to the feedback I got, this extension has been really useful for several people and companies. Microsoft has realized the lack of this functionality in the Enterprise Library and EntLib 3.0 will include a similar rolling file trace listener out-of-the-box.


EDRA is still being used by some companies. One of the largest implementations is the Commonwealth Bank of Australia CommSee Solution that is based on EDRA. Microsoft is still using this as a reference case. For instance, there was a presentation on CommSee at LEAP2007 in Redmond.


I disliked the idea that EDRA pushed you into the direction of distributing the service interface and service implementation across different tiers. Both layers had to be realized in managed code using EDRA, so the service interface could also just call the service implementation in process. Remember the first law of distributed computing: "Don't distribute!" (unless you have to).


Of course, it was great that you could distribute service interface and service implementation. But not all applications need this.


So in my opinion it is better to have different frameworks that cleanly support these orthogonal concepts:



  • Separating business logic from cross cutting concerns.
  • Separating service interface from service implementation across physical tiers.

Proseware


By the way, a sample application that was commissioned by Microsoft to provide guidance on how to build distributed systems never saw the public light of day. It was the Proseware application designed and build by newtelligence's Clemens Vasters. I never got a clear answer from Richard Turner, the responsible program manager at Microsoft, for why it would not be released. But I think it was because of internal Microsoft politics: Proseware was too close to the release of WCF and might be perceived as conflicting guidance (too close turned out to be two years!).


Proseware included a great idea to improve the reliability and scalability of web services. By using one-way messaging for both requests and response. That way you can use queuing inside your service to handle heavy loads and to automatically retry failed attempts at processing messages (for example when a database is temporarily unavailable) without bothering the service clients with having to retry. To achieve this, a web service interface would be a very thin facade around an MSMQ transactional message queue. The web service would only return a fault when the message could not be placed onto the queue (highly unlikely). The message would re-enter the queue if the service implementation failed at processing it so it could be processed again. A response from the service implementation would be send using a one-way message to the recipient specified in the WS-Addressing ReplyTo header in the original request. Note that this recipient does not have to be the original caller! It could well be a different service and the response message is really a new request. Check out this blog entry for more details on Proseware.


Eventually, we will all leave the world of developing and calling synchronous services in distributed systems, but that may take a while 😉 Anyway, sorry for this digression, enter the Policy Injection Application Block.


Policy Injection Application Block


The new Policy Injection Application Block (PIAB) wisely focuses on just the separation of business logic from cross cutting concerns. Windows Communication Foundation is the way to go for distributing your system, i.e., for building connected systems.


The PIAB shares the idea of a pipeline of pre- and post handlers processing messages. It uses the MarshalByRefObject and TransparentProxy infrastructure that was originally designed for .NET remoting to transparently insert policies when they are enabled. The client just thinks it is calling the business logic object directly. Take a look at the pictures in Tom's blog entry to get a better idea of how this works and read Edward Jezierski's blog post.


You might also be interested in the comments posted to Tom's announcement. Several people are concerned about the performance impact that inserting policies will have on your code. This is caused by the technical implementation that Microsoft has chosen to insert policies (or should they be called aspects? ;). As I haven't looked into detail at the new block, I do not have a firm opinion on this matter yet.


The transparent aspect of the policy injection does conflict with the wisdom put into WCF. WCF follows one of the important tenets of service orientation: "Make boundaries explicit". Don't fool the client into thinking they are just performing a local method call, because the performance and reliability characteristics are entirely different. WCF achieves this explicitness by using DataContracts and ServiceContracts. It does not expose everything by default, you have to opt-in. It also makes you more aware that you should not use chatty interfaces across service boundaries.


One of the comments to Tom's blog post states that the overhead of just having a PIAB policy injection could mean that the mere act of calling a method is 50 times slower than a direct method call. If this is the case, you should be well aware and design your objects accordingly: don't implement chatty interfaces!


The future will tell if the PIAB will be more successful than EDRA at enabling the separation of business logic from cross cutting concerns in .NET enterprise applications.

1 Comment

The December 2006 CTP of the Web Service Software Factory (WSSF) from Microsoft P&P ships with a 2.0 version of the Microsoft Enterprise Library. The Data Accesss factory inside Service Factory generates code which uses the Data Access Application Block from Enterprise Library.

This interface of the Data Access block hasn't changed significantly in the January 2007 CTP of version 3.0 of the Enterprise Library. I wondered if I could use this version of EntLib together with Service Factory.

It turns out that this is really easy. The Service Factory uses a registry key to locate the EntLib binaries. This key is HKEY_LOCAL_MACHINESOFTWAREMicrosoftpatterns and practicesService FactoryEntlibBinaryPath.

It has a default value of C:Program FilesMicrosoft Service FactoryEnterprise Library Binaries. After you have installed EntLib 3.0, change the value to the location of the new binaries. For the January 2007 CTP the default location is C:Program FilesMicrosoft Enterprise Library 3.0 - January 2007 CTPBin.

If you create repository classes in your DataAccess project using the "Create data repository classes from business entities" recipe, the project will reference and use the 3.0 binaries.

1 Comment

Rolling File Trace Listener

Note: Archived version of the article about the 0.5.2.0 version. This is no longer the latest version.

Erwyn van der Meer, LogicaCMG
Based on Rolling File Sink by Hisham Baz, Avanade

Download v0.5.2.0 source code + documentation. The source code and documentation are provided "AS-IS". Feel free to change and/or extend them. Please contact me or leave a comment if you find an issue.

Last updated: April 18, 2006

Overview

This article describes the RollingFileTraceListener extension of the
Logging block of version 2.0 of the Microsoft Enterprise Library.  This
is a custom trace listener that can be plugged into the Enterprise
Library Logging block like the standard trace listeners that are
included. The FlatFileTraceListener shipped with EntLib 2.0 may not be
adequate for enterprise systems since log files by default will
continue to grow unchecked.  This new trace listener provides support for rolling over log files based on age of the log file or file size.

Usage

The ZIP file contains a Visual Studio 2005 solution and projects
with C# sources that are fully documented with XML comments. Extract
the ZIP file, open the EnterpriseLibraryExtensions.sln
solution in Visual Studio 2005 and build the solution. This will create
the assemblies for the RollingFileTraceListener and a test application.

In Windows the maximum path + filename length for a file is 260
characters. Please ensure you extract the ZIP file into a directory
with a path that is less than 82 characters in length. Otherwise you
will get a compilation error because during compilation Visual Studio
will create an intermediate file with a really long name for which the
path exceeds the limit of 260 characters.

In order to use the new trace listener from the Enterprise Library
Configuration tool, you must copy the assemblies into the same
directory as the configuration tool. There are two files that need to be copied - LogicaCMG.EnterpriseLibraryExtensions.Logging.dll and LogicaCMG.EnterpriseLibraryExtensions.Logging.Configuration.Design.dll.  When these files are in the same directory, they will be registered during the startup of the tool. This
will allow you to add a new Rolling File Trace Listener from the
right-click context menu on the Trace Listeners collection node.

Screenshot of how you can add a RollingFileTraceListener to an application configuration file using the Enterprise Library Configuration Console.

Configuration

The trace listener creates a new log file when the current file exceeds defined thresholds.  The
RollingFileTraceListener provides file age and file size thresholds
that can be configured using the Enterprise Library Configuration tool
like any other trace listener.

Screenshot of the configuration properties that can be set for a RollingFileTraceListener.

 The RollingFileTraceListener has the following properties

  • Filename – filename with optional relative or full path
  • Header – string (optional)
  • Footer – string (optional)
  • MaximumLogFilesBeforePurge – integer
  • Timestamp Format – string, e.g., yyyy-MM-dd [HH:mm:ss]  (optional)
  • Age Threshold - integer
  • Age Unit – None, Minutes, Hours, Days, Weeks, Months
  • Size Threshold – integer
  • Size Unit – None, KB, MB, GB

Design

The RollingFileTraceListener uses a database-style log roll-over process.  All new log entries are written to the file defined in the FileName property.  Before a log entry is written, the trace listener checks to see if the file has grown too big or the age limit has expired.  If
the current log file exceeds the size or age threshold, then the log
file is renamed with the current timestamp or an incremental counter.  The new file is created with the same name defined in FileName. So the current log file always has the same name.

Age Threshold

The age threshold allows you to limit the growth of your log files according to age.  The threshold is compared by evaluating the current date/time against the log file creation date/time.  For
example, if the log file was created on 1/1/2006 at 3PM and the age
threshold was set to 2 days, a new log file will be created for the
first log entry written after 1/3/2006 at 3PM.

There are two properties that control the age threshold behavior – AgeThreshold and AgeUnit.  Select the appropriate unit size first – None, Minutes, Hours, Days, Weeks or Months - by setting the AgeUnit property.  Set the AgeThreshold property to 0 or the AgeUnit to None to disable the age threshold check.

Size Threshold

In addition to age threshold, you can limit log file growth based on file size

There are two properties that control the age threshold behavior – SizeThreshold and SizeUnit.  Select the appropriate unit size first – None, Kilobytes, Megabytes or Gigabytes - by setting the SizeUnit property.  Set the SizeThreshold property to 0 or the SizeUnit to None to disable the file size threshold check. 

Combination Thresholds

Both the age and size thresholds can be applied at the same time.  For
example, logs can be configured to roll-over every week or earlier if
the file grows over 10MB. To accomplish this, set the AgeUnit to Weeks, AgeThreshold to 1, SizeUnit to Megabytes and SizeThreshold to 10.

Filename Formatting

When log file exceeds the defined thresholds, it is given a new unique name.

The TimestampFormat property can be used to define a date-time format string used for generating new file names.  This
allows overgrown or overaged log files to be renamed and stamped with
the current timestamp. The timestamp is appended to the end of the new
filename, before the extension.  For example, if TimestampFormat is set to yyyy-MM-dd and FileName is set to app.log, then when rolling over on February 15, 2006 the current log file is renamed to app2006-02-15.log.

If the new filename already exists, then an incremental counter is added until a unique combination is found.  If the TimestampFormat property is left blank, the counter will always be appended to the end of the FileName.  The counter is formatted to 7 digits (e.g., app[0000123].log) and counting begins from 1.

Purging

Log files can be automatically deleted after a certain maximum
number of log files are present by setting the
MaximumLogFilesBeforePurge property.  Set this property to 0 to disable automatic purging.  Purging
occurs before the current log file is renamed. The log files are
deleted in reverse order of their last modification time.

Change log

Version 0.5.2.0 – Fixed a bug that occurred when the
application configuration file is changed while an application using
the RollingFileTraceListener is running. In that case a GUID was
prepended to the filename of the log file, because the log file was
still locked by another RollingFileTraceListener instance.

Version 0.5.1.0 – Fixed a bug that prevented the
formatter name from being saved correctly when using the Enterprise
Library Configuration tool.

Version 0.5.0.0 – First public release.

Known problems

A GUID might be prepended to the filename of the log file

A RollingFileTraceListener instance "owns" the log file it is
writing to and locks it for exclusive write access when it writes the
first log entry. It keeps the file locked until the instance is
disposed. If another RollingFileTraceListener  instance
is created that points to the same file, before the first instance is
disposed, the second instance cannot open this file for writing and
will write to a new file with a GUID prepended to its name.

The RollingFileTraceListener indirectly derives from
System.Diagnostics.TextWriterTraceListener. This class changes the
filename to include a GUID when the file with the specified filename
cannot be written to. This is because RollingFileTraceListener
indirectly calls the EnsureWriter method on its base class
TextWriterTraceListener. .NET Reflector shows this code for
System.Diagnostics.TextWriterTraceListener.EnsureWriter() in System.dll
(slightly rewritten to improve clarity):

    try
    {
        this.writer = new StreamWriter(fileNameWithPath, true, encoding1, 0x1000);
        break;
    }
    catch (IOException)
    {
        Guid guid1 = Guid.NewGuid();
        fileName = guid1.ToString() + fileName;
        fileNameWithPath = Path.Combine(folderPath, fileName );
    }

Terms and conditions for the "Enterprise Library 2.0 Extensions including the Rolling File Trace Listener"

Copyright (c) 2006 Erwyn van der Meer

Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated documentation files
(the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to permit
persons to whom the Software is furnished to do so, subject to the following
conditions:

The above copyright notice and this permission notice
shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT
WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR
THE USE OR OTHER DEALINGS IN THE SOFTWARE.

1 Comment

Rolling File Trace Listener


Erwyn van der Meer, LogicaCMG
Based on Rolling File Sink by Hisham Baz, Avanade

Download v0.5.2.0 source code + documentation. The source code and documentation are provided "AS-IS". Feel free to change and/or extend them. Please contact me or leave a comment if you find an issue.


Last updated: April 18, 2006


Overview


This article describes the RollingFileTraceListener extension of the Logging block of version 2.0 of the Microsoft Enterprise Library.  This is a custom trace listener that can be plugged into the Enterprise Library Logging block like the standard trace listeners that are included. The FlatFileTraceListener shipped with EntLib 2.0 may not be adequate for enterprise systems since log files by default will continue to grow unchecked.  This new trace listener provides support for rolling over log files based on age of the log file or file size.


Usage


The ZIP file contains a Visual Studio 2005 solution and projects with C# sources that are fully documented with XML comments. Extract the ZIP file, open the EnterpriseLibraryExtensions.sln solution in Visual Studio 2005 and build the solution. This will create the assemblies for the RollingFileTraceListener and a test application.


In Windows the maximum path + filename length for a file is 260 characters. Please ensure you extract the ZIP file into a directory with a path that is less than 82 characters in length. Otherwise you will get a compilation error because during compilation Visual Studio will create an intermediate file with a really long name for which the path exceeds the limit of 260 characters.


In order to use the new trace listener from the Enterprise Library Configuration tool, you must copy the assemblies into the same directory as the configuration tool. There are two files that need to be copied - LogicaCMG.EnterpriseLibraryExtensions.Logging.dll and LogicaCMG.EnterpriseLibraryExtensions.Logging.Configuration.Design.dll.  When these files are in the same directory, they will be registered during the startup of the tool. This will allow you to add a new Rolling File Trace Listener from the right-click context menu on the Trace Listeners collection node.


Screenshot of how you can add a RollingFileTraceListener to an application configuration file using the Enterprise Library Configuration Console.


Configuration


The trace listener creates a new log file when the current file exceeds defined thresholds.  The RollingFileTraceListener provides file age and file size thresholds that can be configured using the Enterprise Library Configuration tool like any other trace listener.


Screenshot of the configuration properties that can be set for a RollingFileTraceListener.


 The RollingFileTraceListener has the following properties



  • Filename – filename with optional relative or full path
  • Header – string (optional)
  • Footer – string (optional)
  • MaximumLogFilesBeforePurge – integer
  • Timestamp Format – string, e.g., yyyy-MM-dd [HH:mm:ss]  (optional)
  • Age Threshold - integer
  • Age Unit – None, Minutes, Hours, Days, Weeks, Months
  • Size Threshold – integer
  • Size Unit – None, KB, MB, GB

Design


The RollingFileTraceListener uses a database-style log roll-over process.  All new log entries are written to the file defined in the FileName property.  Before a log entry is written, the trace listener checks to see if the file has grown too big or the age limit has expired.  If the current log file exceeds the size or age threshold, then the log file is renamed with the current timestamp or an incremental counter.  The new file is created with the same name defined in FileName. So the current log file always has the same name.


Age Threshold


The age threshold allows you to limit the growth of your log files according to age.  The threshold is compared by evaluating the current date/time against the log file creation date/time.  For example, if the log file was created on 1/1/2006 at 3PM and the age threshold was set to 2 days, a new log file will be created for the first log entry written after 1/3/2006 at 3PM.


There are two properties that control the age threshold behavior – AgeThreshold and AgeUnit.  Select the appropriate unit size first – None, Minutes, Hours, Days, Weeks or Months - by setting the AgeUnit property.  Set the AgeThreshold property to 0 or the AgeUnit to None to disable the age threshold check.


Size Threshold


In addition to age threshold, you can limit log file growth based on file size


There are two properties that control the age threshold behavior – SizeThreshold and SizeUnit.  Select the appropriate unit size first – None, Kilobytes, Megabytes or Gigabytes - by setting the SizeUnit property.  Set the SizeThreshold property to 0 or the SizeUnit to None to disable the file size threshold check. 


Combination Thresholds


Both the age and size thresholds can be applied at the same time.  For example, logs can be configured to roll-over every week or earlier if the file grows over 10MB. To accomplish this, set the AgeUnit to Weeks, AgeThreshold to 1, SizeUnit to Megabytes and SizeThreshold to 10.


Filename Formatting


When log file exceeds the defined thresholds, it is given a new unique name.


The TimestampFormat property can be used to define a date-time format string used for generating new file names.  This allows overgrown or overaged log files to be renamed and stamped with the current timestamp. The timestamp is appended to the end of the new filename, before the extension.  For example, if TimestampFormat is set to yyyy-MM-dd and FileName is set to app.log, then when rolling over on February 15, 2006 the current log file is renamed to app2006-02-15.log.


If the new filename already exists, then an incremental counter is added until a unique combination is found.  If the TimestampFormat property is left blank, the counter will always be appended to the end of the FileName.  The counter is formatted to 7 digits (e.g., app[0000123].log) and counting begins from 1.


Purging


Log files can be automatically deleted after a certain maximum number of log files are present by setting the MaximumLogFilesBeforePurge property.  Set this property to 0 to disable automatic purging.  Purging occurs before the current log file is renamed. The log files are deleted in reverse order of their last modification time.



Change log


Version 0.5.2.0 – Fixed a bug that occurred when the application configuration file is changed while an application using the RollingFileTraceListener is running. In that case a GUID was prepended to the filename of the log file, because the log file was still locked by another RollingFileTraceListener instance.


Version 0.5.1.0 – Fixed a bug that prevented the formatter name from being saved correctly when using the Enterprise Library Configuration tool.


Version 0.5.0.0 – First public release.



Known problems


A GUID might be prepended to the filename of the log file


A RollingFileTraceListener instance "owns" the log file it is writing to and locks it for exclusive write access when it writes the first log entry. It keeps the file locked until the instance is disposed. If another RollingFileTraceListener  instance is created that points to the same file, before the first instance is disposed, the second instance cannot open this file for writing and will write to a new file with a GUID prepended to its name.


The RollingFileTraceListener indirectly derives from System.Diagnostics.TextWriterTraceListener. This class changes the filename to include a GUID when the file with the specified filename cannot be written to. This is because RollingFileTraceListener indirectly calls the EnsureWriter method on its base class TextWriterTraceListener. .NET Reflector shows this code for System.Diagnostics.TextWriterTraceListener.EnsureWriter() in System.dll (slightly rewritten to improve clarity):


    try
    {
        this.writer = new StreamWriter(fileNameWithPath, true, encoding1, 0x1000);
        break;
    }
    catch (IOException)
    {
        Guid guid1 = Guid.NewGuid();
        fileName = guid1.ToString() + fileName;
        fileNameWithPath = Path.Combine(folderPath, fileName );
    }