Testing IL Generation

Marc Gravell points out some interesting problems when generating IL manually. I was lucky that when working on ProxyFoo I did not run into any major headaches with this, but I do have some future ideas (that involve loading from disk) where based on Marc's info this could be a problem.

Background

Early on in development I would occasionaly save a generated assembly and run PEVerify on it. As the project got further along I would simply rely on tests to determine if the generated code worked as expected, but I knew eventually I would want something more robust.

Code generation in ProxyFoo always takes place in a dynamic assembly managed by the ProxyModule class. In order to create an easy to use API all of the static methods will create and use a common default module. The library provides a way for this default module to be cleared which I take advantage of for test seperation. I wanted to use PEVerify as part of every test too.

PEVerify in Testing

To make it easy to reuse, I created ProxyFooTestsBase as a base TextFixture for tests with generated code.

The SetUp configures the default ProxyModule with a unique assembly name for the test and with flags that allow it to be saved.

[SetUp]
public virtual void SetUp()  
{
    string assemblyName = "ZPFD." + TestContext.CurrentContext.Test.FullName.Replace("ProxyFoo.Tests.", "");
    foreach (var c in Path.GetInvalidFileNameChars())
        assemblyName = assemblyName.Replace(c, '_');
    var assemblyPath = assemblyName + ".dll";
    if (File.Exists(assemblyPath))
        File.Delete(assemblyPath);
    ProxyFooPolicies.ProxyModuleFactory = () => new ProxyModule(assemblyName, AssemblyBuilderAccess.RunAndSave);
    ProxyFooPolicies.ClearProxyModule();
}

I prefixed the file with 'Z' for the simple reason of having them be listed last when reviewing build output. The next to last line overrides the default factory method to use the custom name and allow for saving; the last line clears any static content related to the ProxyModule. I had never used TestContext before this, and was glad to find it.

[TearDown]
public virtual void TearDown()  
{
    if (!ProxyModule.Default.IsAssemblyCreated)
        return;
    ProxyModule.Default.Save();
    if (TestContext.CurrentContext.Result.Status==TestStatus.Passed)
    {
        var assemblyPath = ProxyModule.Default.AssemblyName + ".dll";
        Assert.That(PeVerifyAssembly(".", assemblyPath), Is.EqualTo(0));
    }
    ProxyFooPolicies.ProxyModuleFactory = ProxyFooPolicies.DefaultProxyModuleFactory;
    ProxyFooPolicies.ClearProxyModule();
}

TearDown does the real work. If the test passed it saves the assembly, runs PEVerify, and asserts that no issues were found. This is also convenient when debugging issues with generated code, because I know have a unique assembly as part of the output I can load in Reflector (or other dissasembler) to review the code. If necessary the passing test condition can be commented out, but I haven't needed this very often.

When I first added this to my existing tests, I did to my surprise find several instances where everything ran fine but failed verification. In my case I found failures for not calling object's ctor from a generated class and a couple of cases where castclass was needed.

Conclusion

My original concern was for compatiblity. What the CLR lets slide now might not be allowed next time, and after reading Marc's article it was good to see there was even more reason to be this strict. There's nothing earth shattering here so I'm sure this pattern could be adopted for testing generated code in other situations. Sigil also looks really interesting, but in the case of ProxyFoo I wanted to avoid dependencies if possible.