Colin's Corner

Failed import of TFS 2008 to TFS 2010 Beta 2

2009-12-08T20:41:00.004Z

We're running 2 Team Foundation Servers (TFS), one is our main legacy server (TFS 2008) and one is the new shiny, exciting TFS 2010 Beta 2 with it's go live license. We set up the 2010 one to play with but have become so impressed with it we (OK, I) decided to migrate the contents of the 2008 server into it.

There is a capability that comes with TFS 2010 called TfsConfig that has a lovely option "import" that should take the contents of the 2008 databases and bring them into the 2010 databases. Type "TfsConfig help import" for the precise syntax.

I tried this and it failed. During step 8 of 198.

Fine, I thought, I'll just wait for the RTM version and we'll continue to use 2008 and 2010 in parallel in the meantime. But the failed import had broken by 2008 instance. The TfsVersionControl database in the 2008 SQL Server had gone. After a lot of faffing, I worked out that it was still attached but had been renamed to Tfs_ProjectX where ProjectX was the name of the new Team Project Collection in 2010.

I detached it and reattached with the correct name. All was looking good, but the TFS 2008 was still completely dead. Turns out that there is a Guid hidden in each of the 2008 databases that tells them which TFS Instance they belong to. Unfortunately the TFS 2010 import attempt had changed the TfsVersionControl databases InstanceId. This meant I was seeing this kind of error in the App Layer event log

TF30046: The instance information does not match.

Solution to this is to reset the Instance Id across all of the various databases in the 2008 server. To do this, issue the following command:

“%TFSInstallDir%\Tools\InstanceInfo.exe" stamp /setup /install /rollback /d TFSWorkItemTracking,TFSBuild,TFSVersionControl,TFSIntegration,TfsWarehouse /s <>

Replacing %TFSInstallDir% with your C:\Program Files\.... where ever you have InstanceInfo.exe, part of TFS install. This command was taken from this link. The /rollback needs to be removed once you are happy that this works. Once the non-rolled back command has been issued, then all was well and TFS 2008 was working again.

Caution : Static Events!

2009-08-10T23:00:00.015+01:00

I have spent quite a lot of time recently tracking down memory leaks in our C++/CLI & C# application. This has meant using a wide variety of tools and generally thinking far too hard!

The tools we've been using include:

* SOS - Son of Strike debugger, used inside WinDBG
* SOSEX - SOS extensions
* IBM's Purify & Quantify
* GlowCode
* Excel
* VS Debugger....

The list goes on and on.

Special note of thanks to GlowCode who actually provide a tool that can track unmanaged memory leaks in a mixed mode application. In the good old days before .Net, I used Rational (as it was then) Purify to track leaked memory in my C++ applications. It was fairly straight forward to use and between that and its sister product Quantify, I have had many a productive session plugging memory leaks. Since .Net came along, Purify is now all but useless, only tracking managed memory and not showing me where I am leaking unmanaged memory in a mixed application. I've been looking for a tool to do such a job for years and recently came across a link on StackOverflow that pointed to this post which mentioned GlowCode. Having tried it, I love it. It tells me where I am leaking memory in both managed and native heaps in the same application!

One of the biggest problems we've had is static events on a static event broker object holding on to large object graphs causing our application to run out of memory. The Event Broker is meant to act as a central point where classes can subscribe to (attach delegates to) events that other parts of the system can raise in blissful ignorance of each other. Everyone knows about the EventBroker but not about the source or sinks of the events. This has worked well for us as is used extensively throughout our application.

What we hadn't realised (or at least, what we hadn't thought about too hard) is when an object subscribes to a static event, unless it explicitly unsubscribes from it, it will never be garbage collected. This might sound obvious, but most events that are subscribed to are not unsubscribed from explcitly, for example button handlers. When everything goes out of scope and is no longer 'rooted', the GC will collect the whole lot in one go. Unfortunately, this does not happen for static events because the static delegate is a root for the object that has subscribed to it. This means that when the GC trawls through all the heap objects in the thread during the collection, it will find a root for every subscribed object graph, so will never collect them.

The best tool for finding such leaks (OK, not technically leaks in the C/C++ sense, but pure badness all the same) was the SOS debugger and WinDBG. SOS has the extremely helpful command "!dumpheap -type fullyQualifiedTypeName" that will tell you how many objects of the specified type you currently have in memory, where they are (their address) and then using "!gcroot address", will tell you what is causing the GC root to them. SOSEX also has the !refs command which does something very similar but in a prettier fashion.

This lack of collection meant our application was eventually producing OutOfMemoryExceptions! We then discovered the MemoryFailPoint class in the BCL (another StackOverflow article) and this has allowed us to degrade gracefully, rather than simply corrupt out heap (as an OOM exception is likely to do) and force the application to close.

The idea behind the MemoryFailPoint is that before you allocate a chunk on memory, you ask whether there is a large enough piece of contiguous memory available on the managed heaps. If there is, then fine, your code carries on as normal, if there isn't then the MemoryFailPoint throws an InsufficientMemoryException exception. The key point is that an OOM will leave your application in an unstable state, whereas an InsufficientMemoryException won't. The trick is being able to estimate how much contiguous memory (in MB) you'll need. You want to do it at a fairly course level, as the performance of the code is decreased, but the up side is that the code protected by a MemoryFailPoint should not thrown an OOM exception.

Gallio & NCover

2009-06-22T23:47:00.002+01:00

There are a lot of posts around on the internet regarding how to gather coverage data from unit tests running in Gallio. I have spent all evening trying to get them working and have found that some of the command lines that appear to work for everyone else simply don't work for me, specifically they may run the tests but provide no coverage data or, more commonly, simply don't run the tests at all.

But i have cracked it now, so in order that I don't forget, I'm going to record it here!

To run the command line "Gallio.Echo.exe" to gather coverage data using NCover 1.5.8 simply use:


Gallio.Echo.exe /r:NCover TestAssembly.dll

It really is that simple! You may/should be able to do it using NCover.Console.exe and passing in a few parameters, but this is by far the simplest and most reliable way of doing it! You should be able to use the


/runner_property:NCoverArguments='TestAssembly.dll'

to tell NCover which assembly to provide coverage data for, as this command line will include coverage data for your tests and some of the Gallio code as well as the code under test, but I haven't got that bit working yet!

InternalsVisibleTo, C++/CLI and as_friend

2009-03-03T20:31:00.008Z

We came across an interesting problem today that had an unexpected answer. The problem we had is we have an assembly we're trying to unit test. The classes we're interested in testing are internal to the assembly but implement public interfaces. We don't want to make them public just for the purposes of testing as this could lead to "accidental" use elsewhere in the code, breaking the encapsulation we're after.

We know that there is an attribute [InternalsVisibleTo] that we can apply to the assembly under test giving the test assembly permission to see the internal classes. We have done this with C# test assemblies and all is fine, but it wasn't working for our C++/CLI test assembly.

All of our assemblies are signed, so after an hour checking that all the public key tokens matched, we were stumped. Then after much Googling, we came across the "as_friend" option on the #using directive. This is the magic flag you need in the C++/CLI test assembly to allow it access to the internals of the assembly under test. There is no explaination (that I have found) in the MSDN documentation why the C++/CLI test assembly needs this and a C# one doesn't!

Now, in order to use this option you can't use the standard C++/CLI References mechanism in the project properties dialog as it ain't there. So you need to remove the reference to the assembly under test from there and use


#using "AssemblyUnderTest.dll" as_friend

in the stdafx.h file of the test assembly. This should then fix the issue.

Extension Methods in .Net 2.0

2009-01-25T21:24:00.006Z

I was recently reading C# in Depth by Jon Skeet and also playing with Rhino Mocks by Oren/Ayende and was struck by the power and simplicity of both Extension Methods and also Lambda expressions. These are both C# 3 compile time features which I thought would be very useful to me in my C# 2 application which sounds like a shame. Until that is I came across this blog by Daniel Moth who describes how to use Extension Methods in C# 2.0. [Actually, as Jon Skeet has pointed out, I mean using C#3 in a .Net 2.0 assembly, see his comment for a link that explains the difference.]

Once you realise that both of these features are purely compiler trickery then all you need to do is figure out how to get them to work in .Net 2 assemblies. Daniel describes how to get Extension Methods working, by defining an Attribute that is usually available to the C# compiler when building a .Net 3.0 assembly (ExtensionAttribute). The attribute needs to be defined in the System.Runtime.CompilerServices namespace rather than one of your own.

This tricks the VS2008 C# compiler into thinking that it is compiling to .Net 3 rather than .Net 2, at least as far as Extension Methods are concerned, so now it is happy for you to add the this keyword in the static methods parameter list.

Next came lambda methods, this is also compiler tricks and simply compiles to .Net 2.0 delegate code underneath. Again you need to get the compiler to think that some things not normally available to it in C# 2 are indeed available. In this case, we need a more extensive Action<> delegate. .Net 2 provies Action that allows for an action on a single class with no return value. For what I wanted, Lambdas, we needed the .Net 3 Action class. This allows you to call delegates with return types.
This class can be defined in any namespace.

My reason for wanting these capabilities was to simplify some WinForm validation code we're writing. Imagine we have a ComboBox and want to perform validation everytime the SelectedItem changes. We want to turn the background colour to be Red if there is no selected item, and to green is something in the list has been selected. If we have 3 such combos on a form, we could capture the SelectedItemChanged event for each and perform the set of checks to see if there is a selected item and then colour the backgrounds.

Or we could use an Extension Method. I now have a ComboBoxExtension class which has the following method signature


public static void InitialiseValidation(this ComboBox combo, Color defaultColor, Action rule)

This is an Extension Method on ComboBox (first parameter), defines the default, not selected colour (second parameter) and an arbitary validation rule which returns bool and acts on a ComboBox (third parameter). This means I can write the following code in the WinForm code:


someCombo.InitialiseValidation(MColourFactory.InvalidColour, c => c.SelectedItem != null);

This simply means that when the SelectedItem property on someCombo is null, it will colour the combo the InvalidColour from our colour factory (red in this case).

There are also Extension Methods to add additional rules beyond this first one as well as explicit IsValid methods that use the same rule set.

All of this runs quite happily in a C#, .Net 2.0 assembly using the VS2008 C# compiler to do it, if you're using VS2005 I suspect you're out of luck!

Debugging Gallio tests in Visual Studio

2009-01-13T14:47:00.002Z

We use the very useful Gallio unit testing framework for both our C# and C++/CLI unit tests. We then use Resharper to execute the C# tests within Visual Studio [remember to reinstall Gallio after you have installed Resharper].

But ReSharper does not work with C++/CLI so it is more difficult to run the tests within Visual Studio. Obviously the test projects produce XXXTests.dll assemblies which cannot be run directly by selecting the projects to be the "Start Up" project and pressing F5. It is possible to set the debugging properties on the project to use Gallio.Echo.exe to be the program launched by VS when F5 is pressed, passing it the test assembly name as the command line argument. This now allows you to press F5 and see all of your tests run. There are two problems with this, firstly the Command window in which they run closes at the end of the run, making it more difficult to see what passed and failed. Secondly, even though you have asked it to run in the debugger, it won't stop on any breakpoints you have set!

The way to overcome the first issue, if it is important to you, is to use the logging command line arguments and get it to log to file what has happened.

The way to overcome the second issue, which is the main reason we want to run our tests in VS to begin with, it to add the following command line argument:

/r:IsolatedAppDomain

I have no idea what this does (other than run the test in it's own app domain?), but the upshot of it is that the breakpoints become enabled and you can now debug your tests in VS. I'd have thought having a command line option "/r:EnableDebugging" or similar would be more obvious, even if all it did was set the AppDomain flag internally?

This also works well for C# projects where you do not have Resharper or TestDriven.Net.

Example command line:

"C:\Program Files\Gallio\bin\Gallio.Echo.exe" SomeUnitTests.dll /r:IsolatedAppDomain

GPS at Airports

2009-01-11T20:50:00.004Z

During a recent visit to the States I flew into Washington Dullas airport. On the way home, it would appear that my Garmin 305 GPS watch became turned on in my bag, as you can see from this image, it seemed to have a fun trip around the airport before being loaded onto the aircraft!

Spending time on the train

2008-10-21T19:48:00.011+01:00

I have been spending a lot of time on the train this week commuting into London for a database training course. During the 2 hours of commuting I have been trying to make productive use of my time by reading on the way in and listening to podcasts on the way home (when my brain is too fried to read!).

One of the .Net Rocks podcasts (episode 232) suggests blogging about some of the technical books you've read and trying to explain why they were of value to you so it might encourage others to read them too. So here goes:

Currently reading (i.e. on the train) :
C# in Depth by Jon Skeet. Bought it after hearing his DNR appearance (episode 383) and was intrigued to find a book that explains the detailed how and why of C#, not just the simple get stuff working level material. He really concentrates on the details of the language and how it has evolved through its three major versions.

I have only read the first couple of chapters (lots to absorb and think about as I read), but have already learnt some interesting things about the language such as:

More about nullable types and that there is a null-coalescing operator ?? used to detect nullness and return a default value instead

INullable i = null;
int j = i ?? 10;
i = 5;
int k = i;
At the end of this, j = 10 and k = 5.

I have also learnt a lot about generics. I knew enough to be dangerous when using the System.Collections.Generic namespace before, but now I feel I could write a genric class with mulitple where statements.
I have also learnt (much to my surprise) that C# does not "pass everything by reference" - I'll let you read the book to see why!

This book fills the whole on my shelf that "Effective C#" would fill if I owned a copy. The Scott Meyers [More] Effective C++ books were two of the most important books I read when I was learning C++ in the days before Google. I am hoping this book will fill in some of the "hard-core" details of C# in the same kind of vain. I spend so much of my life with the C# compiler, it helps to know what it is doing on my behalf (and what it could do if only I knew how to ask!)

I have also recently read (last year or so) :-

Agile Software Development - Alaistair Cockburn - the book that managed to convince me that there is a reason why Agile software development seems to work. It explains some of the theory as to why people on your team behave the way they do and what can be done to influence them.

Patterns of Enterprise Application Architecture - Martin Fowler - An extremely useful and interesting book describing a set of commonly used patterns when building larger scale enterprise applications. The big brother to the gang of four patterns which look at design in the small.

xUnit Test Patterns - Gerard Meszaros - a very thought provoking book on unit testing and its associated patterns. First third is some interesting theory and then the second part is a set of patterns for unit testing. Very useful book to have around in a team trying hard to adopt TTD.

Definitive NHibernate Resources

2008-10-04T22:37:00.004+01:00

At last I have found the definitive source of all things NHibernate, www.nhforge.org. Previously I had been using www.hibernate.org/343.html which seems to be a little out of date (latest version info is wrong as an example), but the new NHibernate community forge site seems to be bang up to date and contain a lot of very interesting community generated content.

Windowless ActiveX controls are not supported

2008-09-16T17:41:00.005+01:00

I recently started to see this intriguing exception message from a WinForm application that we write. After much googling, I came across a number of other blog entries suggesting a wide range of issues, none of which helped much. This is code that has worked for years, but had suddenly ceased to do so!

In the end it turns out that the underlying ActiveX dll had become unregistered on the system, so it was failing the COM initialisation process. An interestingly obscure error message for this condition, but a quick regsvr32 call later and it all worked again!

Certified ScrumMaster

2008-09-16T13:58:00.003+01:00

I have just completed my Certified ScrumMaster training, provided by Danube, in London. An excellent course taught actually using Scrum as the delivery technique, a very efficient way of learning, i.e. by doing!

Now looking forward to rolling Scrum out to my team at work and seeing how we improve quality and productivity at the same time!

CS1504 & MSBuild

2008-09-10T15:22:00.003+01:00

I have just spent half a day trying to reason why VS2008 is happy to build a solution in a particular configuration, but MSBuild 2008 is not prepared to build exactly the same solution in the same configuration!

First a bit of background, we use a AssemblyInfoCommon.cs file to contain all of the common bits of the assembly information, such as copyright, dates, build numbers etc. This is included into each of our C# projects as a link. It then appears in the VS Solution Explorer and all is well.

When I was getting this strange error code of CS1504 and complaining of not being able to find the common file in the project folder, I was a little bemused because the file doesn't live there, but in the parent folder. After much faffing, I discovered that the project file had been hand edited (always a warning sign) and the link to ..\AssemblyInfoCommon.cs was actually ../AssemblyInfoCommon.cs. The incorrect /, rather than \, was the culprit, looks like MSBuild cannot handle it and bails, where as VS2008 just ignores it, so in this case it didn't appear in the solution explorer!

String to Enumeration

2008-08-08T13:40:00.004+01:00

This brief post concerns converting the string form of an enumeration back into the enum class version in C++/CLI.

There are plenty of sites around the internet that tell you how to convert the string form on an enumeration value back into the enum value in C#, but I couldn't find one in C++/CLI. The C# is as follows:

status.State = (State)Enum.Parse(GetType(State), "State String", bool ignoreCase);

The C++/CLI equivalent is

status->State = (States)Enum::Parse(States::typeid, gcnew String(cacheItem->state));

As you can see, cacheItem->state is a CString, not a System::String, so we need to quickly create a System::String for it.

Logging ADODB Recordsets

2008-08-07T19:43:00.018+01:00

The first real post on my blog is the solution to a problem I have had recently. We use ADO (not ADO.Net) recordsets to communicate with our database (Access, yes sorry I know).

These recordsets are generated from the database tables, populated in memory by the software and then we use

recordset->UpdateBatch(ADODB::adAffectAll);

to push the changes back to Access.

We have been having problems with this and getting the very helpful "External component has thrown an exception" error.

Given that the VS debugger can't debug into the COM recordsets, it is not simple to find out what the recordset contains. The solution we settled on is to dump the contents of the recordset to the Output window using the System::Diagnostics::Trace class.

Compared to a similar operation using an ADO dataset, it is rather tedious to write out the contents of any recordset, rather than coding for a specific one.

The following code extract shows how we achived it. This code first checks to see if there are any rows in the recordset, then gets the FieldsPtr for the recordset and iterates through each field, getting the name of each.

Once this has been done, we reset the cursor back to the first row and get each field in the row. The values in the "cells" of the recordset are the good old OleVariants,so you need to determine which of the various variant type the field is, then convert it to something that .Net understands.

Finally, I have used the "|" character to separate the fields and "#" to end each line, just to be sure!

aRecordset->MoveFirst();
    FieldPtr pField;
 
 
    FieldsPtr pFields = aRecordset->GetFields();
    long numberOfColumns = pFields->GetCount();
    System::Diagnostics::Trace::WriteLine("Number of columns in set: " + numberOfColumns + ";");
    System::Diagnostics::Trace::WriteLine("Number of records in set: " + aRecordset->GetRecordCount() + ";");
 
    if (aRecordset->GetRecordCount() > 0)
    {
        for(int i = 0 ; i < numberOfColumns; i++)
        {
            variant_t index;
            index.intVal = i;
            index.vt = VT_I4;
            pFields->get_Item(index, &pField);
            BSTR bstr = pField->GetName();
            System::Diagnostics::Trace::Write(gcnew String(bstr));
            System::Diagnostics::Trace::Write("|");
        }
        System::Diagnostics::Trace::WriteLine("");
 
        aRecordset->MoveFirst();
 
        while(!aRecordset->adoEOF)
        {
 
            for(int count = 1; count < numberOfColumns; count++)
            {
                variant_t index;
                index.intVal = count;
                index.vt = VT_I4;
                pFields->get_Item(index, &pField);
                variant_t var = pField->GetValue();
 
                if (var.vt == VT_R8)
                {
                    System::Diagnostics::Trace::Write(var.dblVal);
                }
                else if (var.vt == VT_R4)
                {
                    System::Diagnostics::Trace::Write(var.fltVal);
                }
                else if (var.vt == VT_I4)
                {
                    System::Diagnostics::Trace::Write(var.lVal);
                }
                else if (var.vt == VT_BSTR)
                {
                    System::Diagnostics::Trace::Write(gcnew String(var.bstrVal));
                }
                else if (var.vt == VT_DATE)
                {
                    COleDateTime DateInfo(var);
                    System::Diagnostics::Trace::Write(gcnew String(DateInfo.Format()));
                }
                else if (var.vt == VT_NULL)
                {
                    System::Diagnostics::Trace::Write("NULL");
                }
                else if (var.vt == VT_BOOL)
                {
                    if (var.boolVal == true)
                    {
                        System::Diagnostics::Trace::Write("true");
                    }
                    else
                    {
                        System::Diagnostics::Trace::Write("false");
                    }
                }
                else
                {
                    System::Diagnostics::Trace::Write("UKNOWN VALUE TYPE");
                }
 
                System::Diagnostics::Trace::Write("|");
            }
            System::Diagnostics::Trace::WriteLine("#");
 
            aRecordset->MoveNext();
        }
    }

I'd like to thank Phil for writing the first version of this code :)

Welcome!

2008-08-07T19:36:00.006+01:00

Welcome to Colin's Corner, my new blog now that I have decided to join the 21st Century!

I am professional software developer working mainly with VC++, C++/CLI and C# for .Net 2.0, and just starting to use Spring and NHibernate. Our main product is a older MFC application we have dragged into the .Net age by extensive use of C++/CLI. This has led me to develop a reasonable understanding of some of the more knarly problems involved in working with "mixed" code on Windows.

This blog is (hopefully) going to be some of the more difficult/interesting tit-bits of knowledge I pick up through my development activities.

Enjoy and I hope you find it useful!

Colin