random .NET and web development musings

Here are some of the techniques I use to optimise my builds. Not all of them will be appropriate to you, and not all of them conceptually work together.
Regardless, these techniques can considerably reduce your build time.

Parallelism

Get MsBuild using all your cores:

/m /p:BuildInParallel=true

you can set

/m:x

to tell it to use x cores if you like.

Reduce Projects

Say you have a project structure like this:

src/MyProj.Core
src/MyProj.Persistence
src/MyProj.Web
src/MyProj.Api

where Core and Persistence are class libraries and Web and Api are web applications.

Do you really need Core and Persistence to be separate assemblies? Do you ever use one independant of the other? Are you really building a highly modular, reusable solution?
There is a huge overhead in firing up the compilation process for each assembly. Keep this to a minimum with as few assemblies as possible.

You might also have the following test projects:

src/MyProj.Core.Tests
src/MyProj.Persistence.Tests
src/MyProj.Web.Tests
src/MyProj.Api.Tests

Why? You can most likely reduce these to a single assembly and use namespaces to divide the various assembly tests.

Inter-Project Dependencies

The biggest waste of time in my previous build mechansim has been redundant building.

Say you have the following dependency tree:

Web
 L- Core
 L- Persistence
     L- Core
	 
Api
 L- Core
 L- Persistence
     L- Core

If you call MsBuild twice, once for Web and once for Api, you will needlessly build Core and Persistence twice.

There are two ways to avoid this, one simple and one complicated.

Complicated – Manage the dependencies yourself

For me, this is a no-go really. Its more effort than its worth for simple projects, and its too unmaintainable for large projects. Essentially it involves building each project directly with msbuild without the ResolveReferences target, then xcopying the artefacts around to each project and fiddling with reference paths. It gets very messy, very fast.

Simple – Build a single project

Option one: Just build your test assembly.

Continuing the same example from above, your dependency graph would look like this:

Tests
 L- Web
     L- Core
     L- Persistence
         L- Core
 L- Api
     L- Core
     L- Persistence
         L- Core

You can then use something like the following msbuild command:

msbuild src/MyProj.Tests/MyProj.Tests.csproj /t:ReBuild;ResolveReferences;PrepareResources;_CopyWebApplication /v:m /p:OutputPath=../../build /m /p:BuildInParallel=true

Note the _CopyWebApplication target, this “publishes” the web apps.

This will result in the following file system structure being created:

build/
build/_PublishedWebsites/
build/_PublishedWebsites/MyProj.Web
build/_PublishedWebsites/MyProj.Api

All your assemblies will be in build/, as well as a normal “published” version of each site under _PublishedWebsites.

You can then call your test runner on these :)

Option two: Build a custom single project

Perhaps you don’t have a single test project to build, or you only want to build a subset of all your projects. In this case, you can make a custom project file, and just build that!



	<ItemGroup>
		<ProjectReference Include="src\MyProj.Web\MyProj.Web.csproj" />
		<ProjectReference Include="src\MyProj.Api\MyProj.Api.csproj" />
	</ItemGroup>
	<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
	<Import Project="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets" />

This way, each project is only built once, with the artefacts reused for each referencing project :)

One thing I’ve found is sometimes, a compiler error is thrown: CSC : fatal error CS2008: No inputs specified. I’ve got some projects that do this, and some that don’t and I’ve not been able to identify the difference that causes it.

Regardless, the solution is to include a .cs file (such as AssemblyInfo.cs) in the above project. This does result in an otherwise unwanted assembly being produced, but you can just ignore it. I’ll update this post if/when I find out more.

ILMerge or worse, aspnet_merge

Update: The below doesn’t work, but it will. Working on a patch for ILRepack that will fix this. Stay tuned.

Do you precompile your views with aspnet_compiler? If you do, you probably want to combine the multitude of App_Web_xxxxxx.dll assemblies that get created to reduce your app’s startup time and reduce its memory footprint. If you use aspnet_compiler that comes with the Windows SDK, you’re gonna have a bad time. Use ILRepack instead. Its like ILMerge, but written against Mono.Cecil – so it’s uber fast.

Say you have your artefacts in build/MyProj.Web, run this:

ilrepack /targetplatform:v4 /parallel /wildcards /log /verbose /lib:build/MyProj.Web/bin /out:build/MyProj.Web/bin/MyProj.Web.Views.dll build/MYProj.Web/bin/App_Web_*.dll

You can even go one step further and merge the assemblies into your web assembly for a single DLL:

ilrepack /targetplatform:v4 /parallel /wildcards /log /verbose /lib:build/MyProj.Web/bin /out:build/MyProj.Web/bin/MyProj.Web.Views.dll build/MyProj.Web/bin/MyProj.Web.dll build/MYProj.Web/bin/App_Web_*.dll

YUI Compressor

Using the Java YUI Compressor? STOP! Use the YUI Compressor MSBuild task instead, you will reduce the time this takes by several orders of magnitude. The Java compressor only accepts one file at a time, which causes Java to be fired up for every file you want to compress, this is slow.

Conclusion

There you have it, lots of ways you can make your slow build process run like lightning!

When implementing custom exceptions in .NET, especially if you’re writing a library that others are going to use, always implement the deserialization constructor:

MyException (SerializationInfo info, StreamingContext context) : base(info, context)

Not doing so will cause others to hate you when your exceptions cause further SerializationExceptions.

just do it.

I wanted a simple way to manage canonical links in ASP.NET MVC.

It would have been nice to work these out automatically from the route tables, but for the project I am working on, its far too convoluted to be done easily, I’d rather maintain a fixed definition for each action.

Step up a custom `ActionFilterAttribute`:

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public sealed class CanonicalUrl : ActionFilterAttribute
{
	readonly string url;

	public CanonicalUrl(string url)
	{
		this.url = url;
	}

	public override void OnResultExecuting(ResultExecutingContext filterContext)
	{
		var uri = filterContext.RequestContext.HttpContext.Request.Url;
		var fullyQualifiedUrl = (uri.Scheme + "://" + uri.Host + this.url).ToLowerInvariant();

		foreach (var key in filterContext.RouteData.Values.Keys)
			fullyQualifiedUrl = fullyQualifiedUrl.Replace("{" + key.ToLowerInvariant() + "}",
															filterContext.RouteData.Values[key].ToString());

		filterContext.Controller.ViewData["CanonicalUrl"] = fullyQualifiedUrl;
		filterContext.HttpContext.Response.Headers.Add("link", "<" + fullyQualifiedUrl + ">; rel=\"canonical\"");

		base.OnResultExecuting(filterContext);
	}
}

You can then decorate your actions like this:

[CanonicalUrl("/show/{name}")]
[AcceptVerbs(HttpVerbs.Get)]
public ActionResult Show(string name)

Make sure you stick the link element in your page’s head:

<% if(ViewData["CanonicalUrl"] != null){ %>
	<link rel="canonical" href="<%: ViewData["CanonicalUrl"] %>" />
<%} %>

I’ve used two approaches to “embedding” the ascx views, one thats quite simple, and one thats a bit more complicated and needs quite a lot of explanation as to why I’m doing it that way.

The easiest way to do it is to embed the ascx file, and access it with a Custom VirtualPathProvider.

This approach however suffers several drawbacks.

The first problem comes at dev time. You have to rebuild every time you change the ascx. This might not be a problem for you, but in large projects it can add several seconds delay to every change you make, which for someone whos trying to be productive, can be very annoying.

The easiest solution to this is to modify the custom VirtualPathProvider above so that it is aware whether it is in dev or production mode. I do this with the debug attribute in the web config:

public bool AreInDevMode
{
	get
	{
		var section = ConfigurationManager.GetSection("system.web/compilation") as CompilationSection;
		return section.Debug;
	}
}

Then, when you’re in dev mode, check the file system before checking for the embedded resource. This will perhaps involve some path hackery, but generally I’m ok with this because the path conventions in my projects are consistent.

The second problem with this method is that the view is essentially embedded as a string, the text that makes up the view ascx. This means the view has to be compiled at runtime. This has two issues:

a) its slow, especially when you have many views
b) you dont get build time compilation error checking.

So how do we solve this?

To compile the views, you need the ASP.NET Precompilation tool.

This will create a separate assembly for each compiled view. This is messy, so we can merge them together using the ASP.NET Merge tool.

Getting better, but you still have two assemblies, your main project code and the one with the views in. ILMerge to the rescue. However, before we do that I want to sort out the compiled view types.

If you open a precompiled view dll in reflector, you’ll find its rather messy and things aren’t named very nicely. For example, say your project has the following views:

/Plugins/PluginA/View-A.ascx
/Plugins/PluginB/View.ascx

You’ll end up with the following types:

plugins_plugina_view_a_ascx
plugins_pluginb_view_ascx

So, what I wan’t to do is put them back into the correct namespace. Unfortunately, because certain characters that are allowed in file paths are not allowed in namespaces, these chars get converted to underscores and you therefore can’t work out the namespace you need from the compiled type name alone. (e.g. you dont know if “path_to_some_view_ascx” was originally “path/to/some/view.ascx” or “path/to-some.view.ascx“)

Therefore I iterate over the file system and look for any .ascx, .aspx and .master files, turn their paths into their corresponding type names. I then have a lookup table of the type name and its original path.

THEN I can use Mono.Cecil to alter the type names.

so for the file:

/Plugins/PluginA/View.ascx

I calculate its type name:

plugins_plugina_view_ascx

and the name(space) I want it to have:

Plugins.PluginA.View

Do this for all the views and then I can ILMerge my two assemblies into one, nicely organised final package 😀

So how do you then get MVC to render the view?

First you need to extend WebFormsViewEngine, and override:

protected override IView CreatePartialView(ControllerContext controllerContext, string partialPath) {
    return new WebFormView(partialPath, null);
}

protected override IView CreateView(ControllerContext controllerContext, string viewPath, string masterPath) {
    return new WebFormView(viewPath, masterPath);
}

So that it returns a custom View, which extends WebFormView and overrides Render:

public virtual void Render(ViewContext viewContext, TextWriter writer)

so that instead of creating the view using the BuildManager, it simply instantiates the type you worked out above.

If you followed any of that, well done!

In this series of posts I’m going to describe the techniques I use for supporting a plugin architecture in ASP.NET MVC.

What do I mean by plugin? Well, without going into the specific details of the system I’ve developed this for (it isnt important), a “plugin” is simply a bit of UI functionality I want to encapsulate into a package that I can reuse. These plugins typically include some codebehind, a view (ascx), some javascript and some css.

A typical plugin will look like this:

/plugins/exampleplugin/exampleplugin.cs
/plugins/exampleplugin/exampleplugin.css
/plugins/exampleplugin/exampleplugin.js
/plugins/exampleplugin/exampleplugin.ascx

When using a plugin, I want to just drop a dll in the bin (or reference it, whatever). I don’t want to have to manually add a script tag for the js, or a link for the css, or put the ascx in some views folder. I want everything to be included in a single file, and for everything to just work ™.

To achieve this I use several techniques:

  • Embed the css and javascript into the assembly as resources
  • Precompile the view into the assembly
  • Use a custom virtual path provider to make debug easier, and find the files at runtime.

These posts won’t go in to exact detail about how everything works and is wired together in my system (again, it isnt important and would only obscure the details). This isn’t a tutorial on how to build a plugin framework (that may be a future series of posts), but hopefully it will give you a few tips and tricks to try things out yourself.
There’s lots of bits I’ll gloss over, like optimisation :)

In this post I’ll cover embedding the css and js.

Embedding JS and CSS

Set the Build Action of the files (css, js) to “Embedded Resource” to embed them.

To get the files out of the assembly as string, you’ll need some code that looks like this:

static string GetResource(string key)
{
    var stream = this.GetType().Assembly.GetManifestResourceStream(key);
    using (var streamReader = new StreamReader(stream))
        return streamReader.ReadToEnd();
}

One thing to note is the naming rules that apply when you embed a file. Namespace sections that begin with a number get the number prefixed with an underscores, slashes get converted to dots and dashes to underscores. For example:

/assets/css/960-grid.css
becomes
.assets.css._960_grid.css

You’ll probably want some function that looks like this:

static string GetKey(string name)
{
    // folders beginning with a number have an _ prepended.
    name = Regex.Replace(name, @"/([0-9])", "/_$1");
    // turn / to . and - to _
    return name.Replace('/', '.').Replace('-', '_');
}

OK so that’s how we can get the css and js data out of the assembly, but then what to do with it?

Well, you’re probably going to have multiple plugins on a page, so you will have multiple css and js to include and you’ll want to bunch all this together. If you have some underlying plugin awareness mechanism, you could just pull the files you need:

var scripts = new StringBuilder();
var styles = new StringBuilder();
foreach (var plugin in page.Plugins)
{
    var key = plugin.GetType().FullName;
    key = GetKey(key);
    scripts.Append(GetResource(key + ".js"));
    styles.Append(GetResource(key + ".css"));
}

OR you could just get all the files in some “namespace”:

var scripts = new StringBuilder();
var styles = new StringBuilder();
var names = this.GetType().Assembly.GetManifestResourceNames();
foreach (var name in names)
{
    if(!name.StartsWith(".Plugins"))
        continue;
    
    if(name.EndsWith(".js"))
        scripts.Append(GetResource(name));
    else if(name.EndsWith(".css"))
        styles.Append(GetResource(name));
}

And there you go, you can then render these two strings into your page in a script and style tag :)

Some other ideas / Variations

If you don’t want to render the css and js directly into the page, you could use a script tag with a src pointing at a fake file. The fake file could be mapped to an IHttpHandler in the Web.config, or use an MVC route to a controller action.

For example, route: /assets/scripts/plugins.js to some code that does something similar to the above.

Want to see the most unhelpful code ever written?

Check out System.Transactions.TransactionManager.ValidateTimeout:

internal static TimeSpan ValidateTimeout(TimeSpan transactionTimeout)
{
    if (transactionTimeout < TimeSpan.Zero)
    {
        throw new ArgumentOutOfRangeException("transactionTimeout");
    }
    if (!(MaximumTimeout != TimeSpan.Zero) || ((transactionTimeout <= MaximumTimeout) && !(transactionTimeout == TimeSpan.Zero)))
    {
        return transactionTimeout;
    }
    return MaximumTimeout;
}

So thats saying, if you provide a timeout greater than the machine.config value, ignore it and use the machine.config one. Silently. No complaining, no warning.

To increase the machine.config timeout you need to add the following to machine.config (or adjust existing values if they are present):'

<system.transactions>
  <machineSettings maxTimeout="02:00:00"/>
  <defaultSettings timeout="02:00:00"/> 
 </system.transactions>

Pretty self explanatory when you know how:

Updated this code to use Mono.Cecil 0.9.5.0

var assembly = AssemblyDefinition.ReadAssembly(@"path\to\assembly");

var someType = assembly.MainModule.Types.FirstOrDefault(v => v.Name == "SomeTypeName");

someType.Namespace = "I.Live.Somewhere.New";
someType.Name = "NewName";

assembly.Write(@"path\to\output");

If you do:

Response.StatusCode = 503;

from an ASP.NET request, you will most likely get a response that looks like this:

HTTP/1.1 503 Service Unavailable
Cache-Control: private
Content-Type: text/html
X-Powered-By: ASP.NET
Date: Mon, 27 Sep 2010 17:23:20 GMT
Content-Length: 27

The service is unavailable.

I.E. “The service is unavailable” is the only content returned, regardless of what your response contained. This is because http.sys in IIS is overriding your response with default custom error behaviour. Great. Fix it like this:

Response.StatusCode = 503;
Response.TrySkipIisCustomErrors = true;

You can then return whatever content you like 😀