Team Foundation Server: Server-side Validation & Interception

Some time ago I presented a simple way to version TFS Web Services in order to intercept and perform server side validations.

In this post I will introduce another way of doing so.

Check out the  (poor documented) ITeamFoundationRequestFilter interface in MSDN. This interface is part of an extensibility method called TFS Filters.

Using this interface as a regular TFS plug-in (meaning you deploy it in the plugins directory as you do with subscribers) you can inspect method executions, requests, etc.

It is very useful when you need to create an audit log, measure performance (recording execution time or calls), diagnose connection problems, etc. In fact, when you connect to TFS from a version of Visual Studio  lower than 2010, an implementation of this filter (check the Microsoft.TeamFoundation.ApplicationTier.PlugIns.Core.UserAgentCheckingFilter) throws an exception indicating that you need a patch for doing so.

The latter is the functionality I am personally more interested in: validations.

The interface defines the following methods in order to handle a request life cycle (in chronological order):

1. BeginRequest: called when a new TFS request (ASP.NET) is about to be executed. At this phase, you do not know which operation is about to be executed (at least not directly).
2. RequestReady: called when security and message validations has already happened (only at Web Service level). At this phase, you do not know which operation is about to be executed (at least not directly).
3. EnterMethod: called when a logical TFS method is about to be called. At this phase, you now know which operation TFS is about to execute (and you even have its parameters).
4. LeaveMethod: called when a logical TFS method has already been executed.
5. EndRequest: called when ASP.NET request is about to end.

Sample debugging session (method information available when EnterMethod is executed):

Turns out that you can only abort the execution of the current request in the BeginRequest and RequestReady methods:

1. The only way of doing so, is throwing an exception that inherits from Microsoft.TeamFoundation.Framework.Server.RequestFilterException.
2. At this phase (BeginRequest or RequestReady), you do not know yet which Method is being called by the client (at least not directly).
3. Even if you throw exceptions from the EnterMethod method, you wouldn’t abort the current execution (there are some nice try-catch internal code preventing the error from aborting the execution). The exception will only be logged.

Note: As in my previous post, the next code sample is only meant to be an experiment in any case :).

By doing some ASP.NET HttRequest manipulation thought, you can implement a validation filter in the RequestReady or BeginRequest methods.

1. Get the current ASP.NET HttpContext.Request
2. Read the entire InputStream and then restore its original state (assuming it is not a forward only stream).
3. Deserialize the SOAP Envelope and reading the parameters from there.

So… as I already made it clear, consider other alternatives before using this as production code.

TFS Filter Sample

public class TeamFoundationRequestFilterSample : ITeamFoundationRequestFilter
{
    public void BeginRequest(TeamFoundationRequestContext requestContext)
    {
        // Can abort here
        TfsPackageValidator.ValidatePackage();
    }

    public void RequestReady(TeamFoundationRequestContext requestContext)
    {
        // Can abort here
    }

    public void EndRequest(TeamFoundationRequestContext requestContext)
    {
    }

    public void EnterMethod(TeamFoundationRequestContext requestContext)
    {
        // Cannot abort here
    }

    public void LeaveMethod(TeamFoundationRequestContext requestContext)
    {
        // Cannot abort here
    }
}

The above sample code validates inputs in the BeginRequest method (you can always move it to the RequestReady). The TfsPackageValidator class inspects the current ASP.NET request and deserializes the input parameters. If it finds an Update operation over a Work Item (by looking at the SOAP Message Body), it continues with the validation process. What we are looking for here, is the Package element with the update information about the work item.

And here is my TfsPackageValidator class:

TfsPackageValidator class

static class TfsPackageValidator
{
    public static void ValidatePackage()
    {
        var soapXml = ReadHttpContextInputStream();

        var packageElement = ReadUpdatePackageFromSoapEnvelope(soapXml);

        if (packageElement != null)
            ValidatePackage(packageElement);
    }

    static void ValidatePackage(System.Xml.Linq.XElement package)
    {
        var updateWorkItemElement =
            package.Descendants("UpdateWorkItem").FirstOrDefault();

        if (updateWorkItemElement != null)
        {
            var priorityColumnElement =
                updateWorkItemElement.Descendants("Column").Where(
                    c => (string)c.Attribute("Column") == "Microsoft.VSTS.Common.Priority").FirstOrDefault();

            if (priorityColumnElement != null)
            {
                int priority = 0;
                var priorityText = priorityColumnElement.Descendants("Value").FirstOrDefault().Value;
                if (!string.IsNullOrEmpty(priorityText) && int.TryParse(priorityText, out priority))
                {
                    if (priority > 2)
                        throw new TeamFoundationRequestFilterException("Priorities grater than 2 are not allowed.");
                }
            }
        }
    }

    static XElement ReadUpdatePackageFromSoapEnvelope(string soapXml)
    {
        var soapDocument = XDocument.Parse(soapXml);

        var updateName =
            XName.Get(
                "Update",
                "http://schemas.microsoft.com/TeamFoundation/2005/06/WorkItemTracking/ClientServices/03");

        var updateElement =
            soapDocument.Descendants(updateName).FirstOrDefault();

        return updateElement;
    }

    static string ReadHttpContextInputStream()
    {
        var httpContext = System.Web.HttpContext.Current;

        string soapXml = null;
        using (var memoryStream = new MemoryStream())
        {
            byte[] buffer = new byte[1024 * 4];
            int count = 0;
            while ((count = httpContext.Request.InputStream.Read(buffer, 0, buffer.Length)) > 0)
                memoryStream.Write(buffer, 0, count);
            memoryStream.Seek(0, SeekOrigin.Begin);
            httpContext.Request.InputStream.Seek(0, SeekOrigin.Begin);

            soapXml = Encoding.UTF8.GetString(memoryStream.GetBuffer());
        }

        return soapXml;
    }
}

Try running this code when saving a Work Item with a priority field (by editing a single Work Item. If you edit the work item in a query view, the message passed to the server would be slightly different). You should see something similar to the following message in the client application:

The exception message is displayed in the dialog box (as a Technical information for administrator).

Download the sample code.

Webcast MSDN: Migración y construcción de aplicaciones para Windows Azure con VM Role y el modo de administración

I will participating in a new Screencast tomorrow about new Azure features requested by the community. Admin mode and VM Role.

Click here to join.

Asynchrony in C# 5: Dataflow Async Logger Sample

Check out this (very simple) code examples for TPL Dataflow.

Suppose you are developing an Async Logger to register application events to different sinks or log writers.

The logger architecture would be as follow:

Note how blocks can be composed to achieved desired behavior. The BufferBlock<T> is the pool of log entries to be process whereas linked ActionBlock<TInput> represent the log writers or sinks.

The previous composition would allows only one ActionBlock to consume entries at a time.

Implementation code would be something similar to (add reference to System.Threading.Tasks.Dataflow.dll in %User Documents%\Microsoft Visual Studio Async CTP\Documentation):

TPL Dataflow Logger

var bufferBlock = new BufferBlock<Tuple<LogLevel, string>>();

ActionBlock<Tuple<LogLevel, string>> infoLogger =
    new ActionBlock<Tuple<LogLevel, string>>(
        e => Console.WriteLine("Info: {0}", e.Item2));

ActionBlock<Tuple<LogLevel, string>> errorLogger =
    new ActionBlock<Tuple<LogLevel, string>>(
        e => Console.WriteLine("Error: {0}", e.Item2));

bufferBlock.LinkTo(infoLogger, e => (e.Item1 & LogLevel.Info) != LogLevel.None);
bufferBlock.LinkTo(errorLogger, e => (e.Item1 & LogLevel.Error) != LogLevel.None);

bufferBlock.Post(new Tuple<LogLevel, string>(LogLevel.Info, "info message"));
bufferBlock.Post(new Tuple<LogLevel, string>(LogLevel.Error, "error message"));

Note the filter applied to each link (in this case, the Logging Level selects the writer used). We can specify message filters using Predicate functions on each link.

Now, the previous sample is useless for a Logger since Logging Level is not exclusive (thus, several writers could be used to process a single message).

Let´s use a Broadcast<T> buffer instead of a BufferBlock<T>.

Broadcast Logger

var bufferBlock = new BroadcastBlock<Tuple<LogLevel, string>>(
    e => new Tuple<LogLevel, string>(e.Item1, e.Item2));

ActionBlock<Tuple<LogLevel, string>> infoLogger =
    new ActionBlock<Tuple<LogLevel, string>>(
        e => Console.WriteLine("Info: {0}", e.Item2));

ActionBlock<Tuple<LogLevel, string>> errorLogger =
    new ActionBlock<Tuple<LogLevel, string>>(
        e => Console.WriteLine("Error: {0}", e.Item2));

ActionBlock<Tuple<LogLevel, string>> allLogger =
    new ActionBlock<Tuple<LogLevel, string>>(
    e => Console.WriteLine("All: {0}", e.Item2));

bufferBlock.LinkTo(infoLogger, e => (e.Item1 & LogLevel.Info) != LogLevel.None);
bufferBlock.LinkTo(errorLogger, e => (e.Item1 & LogLevel.Error) != LogLevel.None);
bufferBlock.LinkTo(allLogger, e => (e.Item1 & LogLevel.All) != LogLevel.None);

bufferBlock.Post(new Tuple<LogLevel, string>(LogLevel.Info, "info message"));
bufferBlock.Post(new Tuple<LogLevel, string>(LogLevel.Error, "error message"));

As this block copies the message to all its outputs, we need to define the copy function in the block constructor. In this case we create a new Tuple, but you can always use the Identity function if passing the same reference to every output.

Try both scenarios and compare the results.

Asynchrony in C# 5 (Part II)

This article is a continuation of the series of asynchronous features included in the new Async CTP preview for next versions of C# and VB. Check out Part I for more information.

So, let’s continue with TPL Dataflow:

1. Asynchronous functions
2. TPL Dataflow
3. Task based asynchronous Pattern

Part II: TPL Dataflow

Definition (by quote of Async CTP doc): “TPL Dataflow (TDF) is a new .NET library for building concurrent applications. It promotes actor/agent-oriented designs through primitives for in-process message passing, dataflow, and pipelining. TDF builds upon the APIs and scheduling infrastructure provided by the Task Parallel Library (TPL) in .NET 4, and integrates with the language support for asynchrony provided by C#, Visual Basic, and F#.”

This means: data manipulation processed asynchronously.

“TPL Dataflow is focused on providing building blocks for message passing and parallelizing CPU- and I/O-intensive applications”.

Data manipulation is another hot area when designing asynchronous and parallel applications: how do you sync data access in a parallel environment? how do you avoid concurrency issues? how do you notify when data is available? how do you control how much data is waiting to be consumed? etc. 

Dataflow Blocks

TDF provides data and action processing blocks. Imagine having preconfigured data processing pipelines to choose from, depending on the type of behavior you want. The most basic block is the BufferBlock<T>, which provides an storage for some kind of data (instances of <T>).

So, let’s review data processing blocks available. Blocks a categorized into three groups:

1. Buffering Blocks
2. Executor Blocks
3. Joining Blocks

Think of them as electronic circuitry components :)..

1. BufferBlock<T>: it is a FIFO (First in First Out) queue. You can Post data to it and then Receive it synchronously or asynchronously. It synchronizes data consumption for only one receiver at a time (you can have many receivers but only one will actually process it).

2. BroadcastBlock<T>: same FIFO queue for messages (instances of <T>) but link the receiving event to all consumers (it makes the data available for consumption to N number of consumers). The developer can provide a function to make a copy of the data if necessary.

3. WriteOnceBlock<T>: it stores only one value and once it’s been set, it can never be replaced or overwritten again (immutable after being set). As with BroadcastBlock<T>, all consumers can obtain a copy of the value.

4. ActionBlock<TInput>: this executor block allows us to define an operation to be executed when posting data to the queue. Thus, we must pass in a delegate/lambda when creating the block. Posting data will result in an execution of the delegate for each data in the queue.

You could also specify how many parallel executions to allow (degree of parallelism).

5. TransformBlock<TInput, TOutput>: this is an executor block designed to transform each input, that is way it defines an output parameter. It ensures messages are processed and delivered in order.

6. TransformManyBlock<TInput, TOutput>: similar to TransformBlock but produces one or more outputs from each input.

7. BatchBlock<T>: combines N single items into one batch item (it buffers and batches inputs).

8. JoinBlock<T1, T2, …>: it generates tuples from all inputs (it aggregates inputs). Inputs could be of any type you want (T1, T2, etc.).

9. BatchJoinBlock<T1, T2, …>: aggregates tuples of collections. It generates collections for each type of input and then creates a tuple to contain each collection (Tuple<IList<T1>, IList<T2>>).

Next time I will show some examples of usage for each TDF block.

* Images taken from Microsoft’s Async CTP documentation.

Microsoft Team Foundation Server 2010 Service Pack 1

Last week Microsoft has released the first Service Pack for Team Foundation Server. Several issues have been fixed and included in this patch.

Check out the list of fixes here.

Cool stuff has been shipped with this new released, such as the expected Project Service Integration.

PS: note that these annoying bugs has been fixed:

1. Team Explorer: When you use a Visual Studio 2005 or a Visual Studio 2008 client, you encounter a red "X" on the reporting node of the team explorer.
2. Source Control: You receive the error "System.IO.IOException: Unable to read data from the transport connection: The connection was closed." when you try to download a source.

Async CTP (C# 5): How to make WCF work with Async CTP

If you have recently downloaded the new Async CTP you will notice that WCF uses Async Pattern and Event based Async Pattern in order to expose asynchronous operations.

In order to make your service compatible with the new Async/Await Pattern try using an extension method similar to the following:

WCF Async/Await Method

public static class ServiceExtensions
{
    public static Task<DateTime> GetDateTimeTaskAsync(this Service1Client client)
    {
        return Task.Factory.FromAsync<DateTime>(
            client.BeginGetDateTime(null, null),
            ar => client.EndGetDateTime(ar));
    }
}

The previous code snippet adds an extension method to the GetDateTime method of the Service1Client WCF proxy.

Then used it like this (remember to add the extension method’s namespace into scope in order to use it):

Code Snippet

var client = new Service1Client();
var dt = await client.GetDateTimeTaskAsync();

Replace the proxy’s type and operation name for the one you want to await.

Asynchrony in C# 5 (Part I)

I’ve been playing around with the new Async CTP preview available for download from Microsoft. It’s amazing how language trends are influencing the evolution of Microsoft’s developing platform. Much effort is being done at language level today than previous versions of .NET.

In these post series I’ll review some major features contained in this release:

1. Asynchronous functions
2. TPL Dataflow
3. Task based asynchronous Pattern

Part I: Asynchronous Functions

This is a mean of expressing asynchronous operations. This kind of functions must return void or Task/Task<> (functions returning void let us implement Fire & Forget asynchronous operations). The two new keywords introduced are async and await.

• async: marks a function as asynchronous, indicating that some part of its execution may take place some time later (after the method call has returned). Thus, all async functions must include some kind of asynchronous operations. This keyword on its own does not make a function asynchronous thought, its nature depends on its implementation.
• await: allows us to define operations inside a function that will be awaited for continuation (more on this later).

Async function sample:

Async/Await Sample

async void ShowDateTimeAsync()
{
    while (true)
    {
        var client = new ServiceReference1.Service1Client();
        var dt = await client.GetDateTimeTaskAsync();
        Console.WriteLine("Current DateTime is: {0}", dt);
        await TaskEx.Delay(1000);
    }
}

The previous sample is a typical usage scenario for these new features. Suppose we query some external Web Service to get data (in this case the current DateTime) and we do so at regular intervals in order to refresh user’s UI. Note the async and await functions working together.

The ShowDateTimeAsync method indicate its asynchronous nature to the caller using the keyword async (that it may complete after returning control to its caller). The await keyword indicates the flow control of the method will continue executing asynchronously after client.GetDateTimeTaskAsync returns. The latter is the most important thing to understand about the behavior of this method and how this actually works.

The flow control of the method will be reconstructed after any asynchronous operation completes (specified with the keyword await). This reconstruction of flow control is the real magic behind the scene and it is done by C#/VB compilers. Note how we didn’t use any of the regular existing async patterns and we’ve defined the method very much like a synchronous one.

Now, compare the following code snippet  in contrast to the previuous async/await:

Traditional UI Async

void ComplicatedShowDateTime()
{
    var client = new ServiceReference1.Service1Client();
    client.GetDateTimeCompleted += (s, e) =>
    {
        Console.WriteLine("Current DateTime is: {0}", e.Result);
        client.GetDateTimeAsync();
    };
    client.GetDateTimeAsync();
}

The previous implementation is somehow similar to the first shown, but more complicated. Note how the while loop is implemented as a chained callback to the same method (client.GetDateTimeAsync) inside the event handler (please, do not do this in your own application, this is just an example). 

How it works? Using an state workflow (or jump table actually), the compiler expands our code and create the necessary steps to execute it, resuming pending operations after any asynchronous one.

The intention of the new Async/Await pattern is to let us think and code as we normally do when designing and algorithm. It also allows us to preserve the logical flow control of the program (without using any tricky coding patterns to accomplish this). The compiler will then create the necessary workflow to execute operations as the happen in time.

Webcast MSDN: El Futuro de C# y Visual Basic

I will be presenting an screencast tomorrow about next versions of C# and VB.NET. We will review some exciting new features.

Register here.