Failed import of TFS 2008 to TFS 2010 Beta 2

We're running 2 Team Foundation Servers (TFS), one is our main legacy server (TFS 2008) and one is the new shiny, exciting TFS 2010 Beta 2 with it's go live license. We set up the 2010 one to play with but have become so impressed with it we (OK, I) decided to migrate the contents of the 2008 server into it.

There is a capability that comes with TFS 2010 called TfsConfig that has a lovely option "import" that should take the contents of the 2008 databases and bring them into the 2010 databases. Type "TfsConfig help import" for the precise syntax.

I tried this and it failed. During step 8 of 198.

Fine, I thought, I'll just wait for the RTM version and we'll continue to use 2008 and 2010 in parallel in the meantime. But the failed import had broken by 2008 instance. The TfsVersionControl database in the 2008 SQL Server had gone. After a lot of faffing, I worked out that it was still attached but had been renamed to Tfs_ProjectX where ProjectX was the name of the new Team Project Collection in 2010.

I detached it and reattached with the correct name. All was looking good, but the TFS 2008 was still completely dead. Turns out that there is a Guid hidden in each of the 2008 databases that tells them which TFS Instance they belong to. Unfortunately the TFS 2010 import attempt had changed the TfsVersionControl databases InstanceId. This meant I was seeing this kind of error in the App Layer event log

TF30046: The instance information does not match.

Solution to this is to reset the Instance Id across all of the various databases in the 2008 server. To do this, issue the following command:

“%TFSInstallDir%\Tools\InstanceInfo.exe" stamp /setup /install /rollback /d TFSWorkItemTracking,TFSBuild,TFSVersionControl,TFSIntegration,TfsWarehouse /s <>

Replacing %TFSInstallDir% with your C:\Program Files\.... where ever you have InstanceInfo.exe, part of TFS install. This command was taken from this link. The /rollback needs to be removed once you are happy that this works. Once the non-rolled back command has been issued, then all was well and TFS 2008 was working again.

Caution : Static Events!

I have spent quite a lot of time recently tracking down memory leaks in our C++/CLI & C# application. This has meant using a wide variety of tools and generally thinking far too hard!

The tools we've been using include:

* SOS - Son of Strike debugger, used inside WinDBG
* SOSEX - SOS extensions
* IBM's Purify & Quantify
* GlowCode
* Excel
* VS Debugger....

The list goes on and on.

Special note of thanks to GlowCode who actually provide a tool that can track unmanaged memory leaks in a mixed mode application. In the good old days before .Net, I used Rational (as it was then) Purify to track leaked memory in my C++ applications. It was fairly straight forward to use and between that and its sister product Quantify, I have had many a productive session plugging memory leaks. Since .Net came along, Purify is now all but useless, only tracking managed memory and not showing me where I am leaking unmanaged memory in a mixed application. I've been looking for a tool to do such a job for years and recently came across a link on StackOverflow that pointed to this post which mentioned GlowCode. Having tried it, I love it. It tells me where I am leaking memory in both managed and native heaps in the same application!

One of the biggest problems we've had is static events on a static event broker object holding on to large object graphs causing our application to run out of memory. The Event Broker is meant to act as a central point where classes can subscribe to (attach delegates to) events that other parts of the system can raise in blissful ignorance of each other. Everyone knows about the EventBroker but not about the source or sinks of the events. This has worked well for us as is used extensively throughout our application.

What we hadn't realised (or at least, what we hadn't thought about too hard) is when an object subscribes to a static event, unless it explicitly unsubscribes from it, it will never be garbage collected. This might sound obvious, but most events that are subscribed to are not unsubscribed from explcitly, for example button handlers. When everything goes out of scope and is no longer 'rooted', the GC will collect the whole lot in one go. Unfortunately, this does not happen for static events because the static delegate is a root for the object that has subscribed to it. This means that when the GC trawls through all the heap objects in the thread during the collection, it will find a root for every subscribed object graph, so will never collect them.

The best tool for finding such leaks (OK, not technically leaks in the C/C++ sense, but pure badness all the same) was the SOS debugger and WinDBG. SOS has the extremely helpful command "!dumpheap -type fullyQualifiedTypeName" that will tell you how many objects of the specified type you currently have in memory, where they are (their address) and then using "!gcroot address", will tell you what is causing the GC root to them. SOSEX also has the !refs command which does something very similar but in a prettier fashion.

This lack of collection meant our application was eventually producing OutOfMemoryExceptions! We then discovered the MemoryFailPoint class in the BCL (another StackOverflow article) and this has allowed us to degrade gracefully, rather than simply corrupt out heap (as an OOM exception is likely to do) and force the application to close.

The idea behind the MemoryFailPoint is that before you allocate a chunk on memory, you ask whether there is a large enough piece of contiguous memory available on the managed heaps. If there is, then fine, your code carries on as normal, if there isn't then the MemoryFailPoint throws an InsufficientMemoryException exception. The key point is that an OOM will leave your application in an unstable state, whereas an InsufficientMemoryException won't. The trick is being able to estimate how much contiguous memory (in MB) you'll need. You want to do it at a fairly course level, as the performance of the code is decreased, but the up side is that the code protected by a MemoryFailPoint should not thrown an OOM exception.

Gallio & NCover

There are a lot of posts around on the internet regarding how to gather coverage data from unit tests running in Gallio. I have spent all evening trying to get them working and have found that some of the command lines that appear to work for everyone else simply don't work for me, specifically they may run the tests but provide no coverage data or, more commonly, simply don't run the tests at all.

But i have cracked it now, so in order that I don't forget, I'm going to record it here!

To run the command line "Gallio.Echo.exe" to gather coverage data using NCover 1.5.8 simply use:

Gallio.Echo.exe /r:NCover TestAssembly.dll

It really is that simple! You may/should be able to do it using NCover.Console.exe and passing in a few parameters, but this is by far the simplest and most reliable way of doing it! You should be able to use the

/runner_property:NCoverArguments='TestAssembly.dll'

to tell NCover which assembly to provide coverage data for, as this command line will include coverage data for your tests and some of the Gallio code as well as the code under test, but I haven't got that bit working yet!

InternalsVisibleTo, C++/CLI and as_friend

We came across an interesting problem today that had an unexpected answer. The problem we had is we have an assembly we're trying to unit test. The classes we're interested in testing are internal to the assembly but implement public interfaces. We don't want to make them public just for the purposes of testing as this could lead to "accidental" use elsewhere in the code, breaking the encapsulation we're after.

We know that there is an attribute [InternalsVisibleTo] that we can apply to the assembly under test giving the test assembly permission to see the internal classes. We have done this with C# test assemblies and all is fine, but it wasn't working for our C++/CLI test assembly.

All of our assemblies are signed, so after an hour checking that all the public key tokens matched, we were stumped. Then after much Googling, we came across the "as_friend" option on the #using directive. This is the magic flag you need in the C++/CLI test assembly to allow it access to the internals of the assembly under test. There is no explaination (that I have found) in the MSDN documentation why the C++/CLI test assembly needs this and a C# one doesn't!

Now, in order to use this option you can't use the standard C++/CLI References mechanism in the project properties dialog as it ain't there. So you need to remove the reference to the assembly under test from there and use

#using "AssemblyUnderTest.dll" as_friend

in the stdafx.h file of the test assembly. This should then fix the issue.

Extension Methods in .Net 2.0

I was recently reading C# in Depth by Jon Skeet and also playing with Rhino Mocks by Oren/Ayende and was struck by the power and simplicity of both Extension Methods and also Lambda expressions. These are both C# 3 compile time features which I thought would be very useful to me in my C# 2 application which sounds like a shame. Until that is I came across this blog by Daniel Moth who describes how to use Extension Methods in C# 2.0. [Actually, as Jon Skeet has pointed out, I mean using C#3 in a .Net 2.0 assembly, see his comment for a link that explains the difference.]

Once you realise that both of these features are purely compiler trickery then all you need to do is figure out how to get them to work in .Net 2 assemblies. Daniel describes how to get Extension Methods working, by defining an Attribute that is usually available to the C# compiler when building a .Net 3.0 assembly (ExtensionAttribute). The attribute needs to be defined in the System.Runtime.CompilerServices namespace rather than one of your own.

This tricks the VS2008 C# compiler into thinking that it is compiling to .Net 3 rather than .Net 2, at least as far as Extension Methods are concerned, so now it is happy for you to add the this keyword in the static methods parameter list.

Next came lambda methods, this is also compiler tricks and simply compiles to .Net 2.0 delegate code underneath. Again you need to get the compiler to think that some things not normally available to it in C# 2 are indeed available. In this case, we need a more extensive Action<> delegate. .Net 2 provies Action that allows for an action on a single class with no return value. For what I wanted, Lambdas, we needed the .Net 3 Action class. This allows you to call delegates with return types.
This class can be defined in any namespace.

My reason for wanting these capabilities was to simplify some WinForm validation code we're writing. Imagine we have a ComboBox and want to perform validation everytime the SelectedItem changes. We want to turn the background colour to be Red if there is no selected item, and to green is something in the list has been selected. If we have 3 such combos on a form, we could capture the SelectedItemChanged event for each and perform the set of checks to see if there is a selected item and then colour the backgrounds.

Or we could use an Extension Method. I now have a ComboBoxExtension class which has the following method signature

public static void InitialiseValidation(this ComboBox combo, Color defaultColor, Action rule)

This is an Extension Method on ComboBox (first parameter), defines the default, not selected colour (second parameter) and an arbitary validation rule which returns bool and acts on a ComboBox (third parameter). This means I can write the following code in the WinForm code:

someCombo.InitialiseValidation(MColourFactory.InvalidColour, c => c.SelectedItem != null);

This simply means that when the SelectedItem property on someCombo is null, it will colour the combo the InvalidColour from our colour factory (red in this case).

There are also Extension Methods to add additional rules beyond this first one as well as explicit IsValid methods that use the same rule set.

All of this runs quite happily in a C#, .Net 2.0 assembly using the VS2008 C# compiler to do it, if you're using VS2005 I suspect you're out of luck!

Debugging Gallio tests in Visual Studio

We use the very useful Gallio unit testing framework for both our C# and C++/CLI unit tests. We then use Resharper to execute the C# tests within Visual Studio [remember to reinstall Gallio after you have installed Resharper].

But ReSharper does not work with C++/CLI so it is more difficult to run the tests within Visual Studio. Obviously the test projects produce XXXTests.dll assemblies which cannot be run directly by selecting the projects to be the "Start Up" project and pressing F5. It is possible to set the debugging properties on the project to use Gallio.Echo.exe to be the program launched by VS when F5 is pressed, passing it the test assembly name as the command line argument. This now allows you to press F5 and see all of your tests run. There are two problems with this, firstly the Command window in which they run closes at the end of the run, making it more difficult to see what passed and failed. Secondly, even though you have asked it to run in the debugger, it won't stop on any breakpoints you have set!

The way to overcome the first issue, if it is important to you, is to use the logging command line arguments and get it to log to file what has happened.

The way to overcome the second issue, which is the main reason we want to run our tests in VS to begin with, it to add the following command line argument:

/r:IsolatedAppDomain

I have no idea what this does (other than run the test in it's own app domain?), but the upshot of it is that the breakpoints become enabled and you can now debug your tests in VS. I'd have thought having a command line option "/r:EnableDebugging" or similar would be more obvious, even if all it did was set the AppDomain flag internally?

This also works well for C# projects where you do not have Resharper or TestDriven.Net.

Example command line:

"C:\Program Files\Gallio\bin\Gallio.Echo.exe" SomeUnitTests.dll /r:IsolatedAppDomain

GPS at Airports

During a recent visit to the States I flew into Washington Dullas airport. On the way home, it would appear that my Garmin 305 GPS watch became turned on in my bag, as you can see from this image, it seemed to have a fun trip around the airport before being loaded onto the aircraft!