.NET 5 Adventure: crossgen(2)

How to invoke and use crossgen “manually” after the build pipeline has been run?

Our main Windows app Royal TS has been around for almost 20 years now and it all started with WinForms using the .NET Framework 1.1 back in the day. Since last summer, I spent quite some time porting Royal TS to .NET 5. Doesn’t sound that exciting but let’s just say, there are a lot of obstacles on the way…

One thing I always did with Royal TS (during the installation), I pre-compiled the app using ngen (native image generator) to get better (startup-)performance. This was really necessary because without the ngen, the app was really noticeable slower.

On the up-side: running the same app using .NET 5 without any pre-compilation was roughly the same speed, definitely not slower, as running it with .NET 4.7 with ngen. I think that says a lot about the performance improvements of .NET 5!

ngen is a thing of the past. Now it’s called crossgen and the idea is to do it right after the compilation (before you install it on the client machine). There are quite a lot of downsides to this approach:

• You can only use crossgen for one architecture. This means you have to decide if you want to deploy an optimized version for x86, x64 and maybe later for ARM64 or something like that. If you run your app on a system you did not generate the native image for, it will fallback to JIT execution.
• Your deployment increases in size drastically. If you crossgen everything, you basically ship the IL code (for JIT) AND the generated native image in the same assembly. Most assemblies end up almost twice as big. If you rely on 3rd party libraries like we do (DevExpress), you quickly end up with 500-700 MB deployment size!
• The tooling around crossgen is not quite as good and the documentation is “thin”. ngen was there, it just worked. If you can run crossgen using the msbuild pipeline, it mostly works but if you can’t use the msbuild pipeline, it gets complicated.

There’s one upside though: you do generate the image before you deploy it to your client’s machine. Pre-compilation (depending on the CPU of the client machine) can take up to several minutes.

The Easy Way

To use crossgen in the msbuild pipeline, simply put <PublishReadyToRun>True</PublishReadyToRun> in a property group in your csproj file:

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
    <PublishReadyToRun>True</PublishReadyToRun>
    <RuntimeIdentifier>win-x64</RuntimeIdentifier>
    <SelfContained>True</SelfContained>
  </PropertyGroup>

This is actually well documented and there’s lots of information out there so I will not go further into the details here.

Why not doing it the easy way?

… you might ask. Well, in our case, we do some post processing after the build and this has to happen before the ReadyToRun native image generation kicks in. Obfuscation, for example has to be done before you generate the native images.

Hacking the Pipeline

Again, there’s no guidance or documentation available for any of this and I failed to get this working reliably, even though my friend Martin Ulrich helped me out with the following example.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
    <PublishReadyToRun>True</PublishReadyToRun>
    <RuntimeIdentifier>win-x64</RuntimeIdentifier>
    <SelfContained>False</SelfContained>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
  </ItemGroup>

  <Target Name="CalculateObfuscationInputs" DependsOnTargets="_ComputeAssembliesToPostprocessOnPublish">
    <PropertyGroup>
      <ObfuscationDir>$(IntermediateOutputPath)obfuscation</ObfuscationDir>
    </PropertyGroup>
    <ItemGroup>
      <AssembliesToObfuscate Include="@(ResolvedFileToPublish->WithMetadataValue('PostprocessAssembly', 'true'))" />
      <AssembliesToObfuscateTemporaryLocation Include="@(AssembliesToObfuscate->'$(ObfuscationDir)%(Filename)%(Extension)')" />
      <_PdbsToObfuscateInput Include="@(AssembliesToObfuscate->'%(RelativeDir)%(Filename).pdb')" />
      <PdbsToObfuscate Include="@(_PdbsToObfuscateInput)" RelativePath="%(_PdbsToObfuscateInput.Identity)" Condition="Exists(%(_PdbsToObfuscateInput.Identity))" />
      <PdbsToObfuscateTemporaryLocation Include="@(PdbsToObfuscate->'$(ObfuscationDir)%(Filename)%(Extension)')" />
    </ItemGroup>
    <MakeDir Directories="$(ObfuscationDir)" />
  </Target>

  <Target Name="PrepareForObfuscation" Inputs="@(AssembliesToObfuscate);@(PdbsToObfuscate)" Outputs="@(AssembliesToObfuscateTemporaryLocation);@(PdbsToObfuscateTemporaryLocation)">
    <Copy SourceFiles="@(AssembliesToObfuscate);@(PdbsToObfuscate)" DestinationFiles="@(AssembliesToObfuscateTemporaryLocation);@(PdbsToObfuscateTemporaryLocation)" SkipUnchangedFiles="True" />
  </Target>

  <Target Name="ObfuscateAssembly" AfterTargets="ComputeResolvedFilesToPublishList" DependsOnTargets="CalculateObfuscationInputs;PrepareForObfuscation">
    <Exec Command="echo this should do something" />
    <ItemGroup>
      <ResolvedFileToPublish Remove="@(AssembliesToObfuscate);@(PdbsToObfuscate)" />
      <ResolvedFileToPublish Include="@(AssembliesToObfuscateTemporaryLocation);@(PdbsToObfuscateTemporaryLocation)" />
      <ResolvedFileToPublishAfterObfuscation Include="@(ResolvedFileToPublish)" />
    </ItemGroup>
  </Target>

</Project>

The above sample should work with simple (not self-contained) apps but dealing with all the file collections is really hard if you have a very complex application with a plugin system where plugins (from separate projects) also need to be obfuscated and processed. It’s really hard to debug, makes the build process much more complex, it is error prone and you are “hooking” into the build process by using a, sort of, private target “_ComputeAssembliesToPostprocessOnPublish” which may or may not change in future updates.

Invoking crossgen2 afterwards

My solution was to use crossgen2 (the CLI tool) myself and build my own little “ngen”.

I tried to use crossgen which you can find in your .nuget folder under:

packages\microsoft.netcore.app.runtime.win-x64\5.0.1\tools\crossgen.exe 

I was able to generate the native images ([assembly].ni.dll files) but then – again nothing really documented anywhere – I had no idea how I could actually combine those .ni.dll files into the assemblies. The available docs state:

You should include the native images in your app, either by replacing the original MSIL assemblies with the native images or by putting the native images next to the MSIL assemblies. When the native images are present, the CoreCLR runtime will automatically use it instead of the original MSIL assemblies.

Well, we tried that and it didn’t work. The .ni.dll files were never used and we also didn’t see any performance gains.

Next, I tried crossgen2 which is still kind of “experimental” and is really hard to find and use. As of the time of this writing, you can find it in the nuget package microsoft.netcore.app.crossgen2.win-x64

You can’t really reference that package, so you need to download the nupkg file from nuget, rename it to .zip and extract the package to get to the tool. Luckily the tool has a –help screen, so you can quickly find out how to use it.

crossgen2 doesn’t create .ni.dll files like crossgen, it creates the ready-to-use assembly with the IL and the native image combined.

Quite a ride!

Big disclaimer though:
All of the above is working for our solution. As stated several times, documentation in this area is missing and a lot of stuff I found out by searching through github issues or just lucky guessing. Maybe it is helpful to someone but chances are that this will all change in future updates and that I’m still doing things wrong (or not quite right). Should you have more information, a better solution or any correction to contribute, let me know in the comments.