random .NET and web development musings

Here are some of the techniques I use to optimise my builds. Not all of them will be appropriate to you, and not all of them conceptually work together.
Regardless, these techniques can considerably reduce your build time.

Parallelism

Get MsBuild using all your cores:

/m /p:BuildInParallel=true

you can set

/m:x

to tell it to use x cores if you like.

Reduce Projects

Say you have a project structure like this:

src/MyProj.Core
src/MyProj.Persistence
src/MyProj.Web
src/MyProj.Api

where Core and Persistence are class libraries and Web and Api are web applications.

Do you really need Core and Persistence to be separate assemblies? Do you ever use one independant of the other? Are you really building a highly modular, reusable solution?
There is a huge overhead in firing up the compilation process for each assembly. Keep this to a minimum with as few assemblies as possible.

You might also have the following test projects:

src/MyProj.Core.Tests
src/MyProj.Persistence.Tests
src/MyProj.Web.Tests
src/MyProj.Api.Tests

Why? You can most likely reduce these to a single assembly and use namespaces to divide the various assembly tests.

Inter-Project Dependencies

The biggest waste of time in my previous build mechansim has been redundant building.

Say you have the following dependency tree:

Web
 L- Core
 L- Persistence
     L- Core
	 
Api
 L- Core
 L- Persistence
     L- Core

If you call MsBuild twice, once for Web and once for Api, you will needlessly build Core and Persistence twice.

There are two ways to avoid this, one simple and one complicated.

Complicated – Manage the dependencies yourself

For me, this is a no-go really. Its more effort than its worth for simple projects, and its too unmaintainable for large projects. Essentially it involves building each project directly with msbuild without the ResolveReferences target, then xcopying the artefacts around to each project and fiddling with reference paths. It gets very messy, very fast.

Simple – Build a single project

Option one: Just build your test assembly.

Continuing the same example from above, your dependency graph would look like this:

Tests
 L- Web
     L- Core
     L- Persistence
         L- Core
 L- Api
     L- Core
     L- Persistence
         L- Core

You can then use something like the following msbuild command:

msbuild src/MyProj.Tests/MyProj.Tests.csproj /t:ReBuild;ResolveReferences;PrepareResources;_CopyWebApplication /v:m /p:OutputPath=../../build /m /p:BuildInParallel=true

Note the _CopyWebApplication target, this “publishes” the web apps.

This will result in the following file system structure being created:

build/
build/_PublishedWebsites/
build/_PublishedWebsites/MyProj.Web
build/_PublishedWebsites/MyProj.Api

All your assemblies will be in build/, as well as a normal “published” version of each site under _PublishedWebsites.

You can then call your test runner on these :)

Option two: Build a custom single project

Perhaps you don’t have a single test project to build, or you only want to build a subset of all your projects. In this case, you can make a custom project file, and just build that!



	<ItemGroup>
		<ProjectReference Include="src\MyProj.Web\MyProj.Web.csproj" />
		<ProjectReference Include="src\MyProj.Api\MyProj.Api.csproj" />
	</ItemGroup>
	<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
	<Import Project="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets" />

This way, each project is only built once, with the artefacts reused for each referencing project :)

One thing I’ve found is sometimes, a compiler error is thrown: CSC : fatal error CS2008: No inputs specified. I’ve got some projects that do this, and some that don’t and I’ve not been able to identify the difference that causes it.

Regardless, the solution is to include a .cs file (such as AssemblyInfo.cs) in the above project. This does result in an otherwise unwanted assembly being produced, but you can just ignore it. I’ll update this post if/when I find out more.

ILMerge or worse, aspnet_merge

Update: The below doesn’t work, but it will. Working on a patch for ILRepack that will fix this. Stay tuned.

Do you precompile your views with aspnet_compiler? If you do, you probably want to combine the multitude of App_Web_xxxxxx.dll assemblies that get created to reduce your app’s startup time and reduce its memory footprint. If you use aspnet_compiler that comes with the Windows SDK, you’re gonna have a bad time. Use ILRepack instead. Its like ILMerge, but written against Mono.Cecil – so it’s uber fast.

Say you have your artefacts in build/MyProj.Web, run this:

ilrepack /targetplatform:v4 /parallel /wildcards /log /verbose /lib:build/MyProj.Web/bin /out:build/MyProj.Web/bin/MyProj.Web.Views.dll build/MYProj.Web/bin/App_Web_*.dll

You can even go one step further and merge the assemblies into your web assembly for a single DLL:

ilrepack /targetplatform:v4 /parallel /wildcards /log /verbose /lib:build/MyProj.Web/bin /out:build/MyProj.Web/bin/MyProj.Web.Views.dll build/MyProj.Web/bin/MyProj.Web.dll build/MYProj.Web/bin/App_Web_*.dll

YUI Compressor

Using the Java YUI Compressor? STOP! Use the YUI Compressor MSBuild task instead, you will reduce the time this takes by several orders of magnitude. The Java compressor only accepts one file at a time, which causes Java to be fired up for every file you want to compress, this is slow.

Conclusion

There you have it, lots of ways you can make your slow build process run like lightning!

NO COMMENTS
Post a comment