Packaging a .NET Core app with the Desktop Bridge

The Windows Desktop Bridge is a way to package up Desktop applications for submission to the Microsoft Store or sideloading from anywhere. It’s one of the ways of creating an MSIX package, and there’s a lot more information about the format in the docs. The short version is this: think about it like the modern ClickOnce. It’s a package format that supports automatic updating while users the peace of mind that it won’t put bits all over their system or pollute the registry.

Earlier today, Microsoft announced the first previews of .NET Core 3 and Visual Studio 2019. These previews have support for creating Desktop GUI apps with .NET Core using WPF and Windows Forms. It’s possible to migrate your existing app from the .NET Framework to .NET Core 3. I’ll blog about that in a later post, but it can be pretty straight-forward for many apps. One app that has already made the switch is NuGet Package Explorer; it’s open-source on GitHub and may serve as a reference.

Once you have an application targeting .NET Core 3, some of your next questions may be, “how do I get this to my users?” “.NET Core 3 is brand new, my users won’t have that!” “My IT department won’t roll out .NET Core 3 for a year!”

Sound familiar? One of the really cool things (to me) in .NET Core is that it supports completely self-contained applications. That is to say it has no external dependencies. Nothing needs to be installed on the machine, not even .NET Core itself. You can xcopy the publish output from the project and give it to someone to run. This unlocks a huge opportunity as you, the developer, can use the framework and runtime versions you want, without worrying about interfering with other apps on the machine, or even if the runtime exists on the box.

With the ability to have a completely self-contained app, we can take advantage of the Desktop Bridge to package our app for users to install. As of today, the templates don’t support this scenario out-of-the-box, but with a few tweaks, we can make it work. Read on for the details.

Getting started

You’ll need Visual Studio 2017 15.9, or better yet, the Visual Studio 2019 preview, just released today. In the setup, make sure to select the UWP workload to install the packaging project tools. Grab the .NET Core 3 preview and create your first WPF .NET Core app with it.

Details

The official docs show how to add a Packaging project to your solution, so we’ll pick-up after that article ends. Start with that first. In the future, once the tooling catches up, that’s all you’ll need. For now, as a temporary workaround, the rest of this post describes how to make it work.

Application Project

Let’s start with the main application project, the .csproj or .vbproj file. Add <RuntimeIdentifiers>win-x86</RuntimeIdentifiers> to the first <PropertyGroup>. This ensures that NuGet restore pulls in the runtime-specific resources and puts them in the project.assets.json file. Next, put in the following Target:

Packaging Project

Next up, we need to add a few things to the packaging project (.wapproj). In the <PropertyGroup> that has the DefaultLanguage and EntryPointProjectUniqueName, add another property: <DebuggerType>CoreClr</DebuggerType>. This tells Visual Studio to use the .NET Core debugger. Note: after setting this property, you may have to unload/reload the project for VS to use this setting, if you get a weird debug error after changing this property, restart VS, load the solution and it should be fine.

Next, look for the <ProjectReference ... element. If it’s not there, right click the Application node and add the application reference to your main project. Add the following attributes: SkipGetTargetFrameworkProperties="true" Properties="RuntimeIdentifier=win-x86;SelfContained=true". The full ItemGroup should look something like this:

In that snippet, change NetCoreDesktopBridgeApp\NetCoreDesktopBridgeApp.exe to match your main project’s name and executable.

VCRedist workaround

Bonus section: as a point-in-time issue, you’ll need to declare a package dependency on the VCRedist in your Package.appxmanifest file. Add the following in the <Dependencies> element: <PackageDependency Publisher="CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US" Name="Microsoft.VCLibs.140.00.UWPDesktop" MinVersion="14.0.26905.0" />. When your users install the app, Windows will automatically pull that dependency from the store.

Build & Debug

With the above pieces in place, you can set the packaging project as the startup project and debug as you normally would. It’ll build the app and deploy it to a local application. You can see the output within your packaging project’s bin\AnyCPU\<configuration>\AppX directory. It should have more files than your main application as it’ll have the self-contained .NET Core runtime in it.

Note: I’ve sometimes found that debugging the packaging project doesn’t cause a rebuild if I’ve changed certain project files. A rebuild of the main app project has fixed that for me and then I’m debugging what I expect.

Deployment

There are two main options for deploying the package:

Sideloading with an AppInstaller file. This is the replacement to ClickOnce.

The Microsoft Store. The package can be submitted to the Store for distribution.

Side-loading

Since Windows 10 1803, sideloaded applications can receive automatic updates using an .appinstaller file. This makes AppInstaller a replacement to ClickOnce for most scenarios. The documentation describes how to create this file during publish, so that you can put it on a UNC path, file share, or HTTPS location.

If you sideload, you’ll need to use a code signing certificate that’s trusted by your users. For an enterprise, that can be a certificate from an internal certificate authority, for the public, it needs to be from a public authority. DigiCert has a great offer for code signing certs, $74/yr for regular and $104/yr for EV at this special link. Disclaimer: DigiCert provides me with free certificates as a Microsoft MVP. I have had nothing but great experiences with them though. Once you have the certificate, you’ll need to update your Package.appxmanifest to use it. Automatic code signing is beyond the scope of this article, but please see my code signing service project for something you can deploy in your organization to handle this.

Microsoft Store

The Microsoft Store is a great way to get your app to your users. It handles the code signing, distribution, and updating. More info on how to submit to the store is here and here.

Using the Extras, most of that is eliminated. Instead, what you need is the following:

A project for your reference assemblies. This project specifies the TargetFrameworks you wish to produce. Note: this project no longer has any special naming or directory conventions. Place it anywhere and call it anything.

A pointer (ReferenceAssemblyProjectReference) from your main project to the reference assembly project.

Both projects need to be using the Extras. Add a global.json to specify the Extras version (must be 1.6.30-preview or later):

{
"msbuild-sdks": {
"MSBuild.Sdk.Extras": "1.6.30-preview"
}
}

And at the top of your project files, change Sdk="Microsoft.NET.Sdk" to Sdk="MSBuild.Sdk.Extras"

In your reference assembly project, use a wildcard to include the source files you need, something like: <Compile Include="..\..\System.Interactive\**\*.cs" Exclude="..\..\System.Interactive\obj\**" />.

In your main project, point to your reference assembly by adding an ItemGroup with an ReferenceAssemblyProjectReference item like this:

Notes

The tooling will pass AssemblyName, AssemblyVersion, FileVersion, InformationalVersion, GenerateDocumentationFile, NeutralLanguage, and strong naming properties into the reference assembly based on the main project, so you don’t need to set them twice.

The REFERENCE_ASSEMBLY symbol is defined for reference assemblies, so you can do ifdef‘s on that.

Update July 9: Read the follow-up post for an easier way to implement.

Create and Pack Reference Assemblies

Reference Assemblies, what are they, why do I need that? Reference Assemblies are a special kind of assembly that’s passed to the compiler as a reference. They do not contain any implementation and are not valid for normal assembly loading (you’ll get an exception if you try outside of a reflection-only load context).

Why do you need a reference assembly?

There’s two main reasons you’d use a reference assembly:

Bait and switch assemblies. If your assembly can only have platform-specific implementations (think of a GPS implementation library), and you want portable code to reference it, you can define your common surface area in a reference assembly and provide implementations for each platform you support.

Selectively altering the public surface area due to moving types between assemblies. I recently hit this with System.Interactive (Ix). Ix provides extension methods under the System.Linq namespace. Two of those methods, TakeLast, and SkipLast were added to .NET Core 2.0’s Enumerable type. This meant that if you referenced Ix in a .NET Core 2.0 project, you could not use either of those as an extension method. If you tried, you’d get an error:

error CS0121: The call is ambiguous between the following methods or properties: 'System.Linq.EnumerableEx.SkipLast<TSource>(System.Collections.Generic.IEnumerable<TSource>, int)' and 'System.Linq.Enumerable.SkipLast<TSource>(Sy stem.Collections.Generic.IEnumerable<TSource>, int)'.

The only way out of this is to explicitly call the method like EnumerableEx.SkipLast(...). Not a great experience. However, we cannot simply remove those overloads from the .NET Core version since:

It’s not in .NET Standard or .NET Framework

If you use TakeLast from a .NET Standard library, then are running on .NET Core, you’d get a MissingMethodException.

The method needs to be in the runtime version, but we need to hide it from the compiler. Fortunately, we can do this with a reference assembly. We can exclude the duplicate methods from the reference on platforms where it’s built-in, so those get resolved to the built-in Enumerable type, and for other platforms, they get the implementation from EnumerableEx.

Creating reference assemblies

I’m going to explore how I solved this for Ix, but the same concepts apply for the first scenario. I’m assuming you have a multi-targeted project containing your code. For Ix, it’s here.

It’s easiest to think of a reference assembly as a different project, with the same name, as your main project. I put mine in a refs directory, which enables some conventions that I’ll come back to shortly.

The key to these projects is that the directory/project name match, so it creates the same assembly identity. If you’re doing any custom versioning, be sure it applies to these as well.

The TargetFrameworks should be for what you want as reference assemblies. These do not have to match that you have an implementation for. For scenario #1 above, you’ll likely only have a single netstandard2.0 target. For scenario #2, Ix, given that the surface area has to be reduced on specific platforms, it has more.

There is a Directory.Build.props file that provides common properties and an extra set of targets these reference assembly projects need. (Ignore the bit with NETStandardMaximumVersion, that’s me cheating a bit for the future 😉)

In that props, it defines REF_ASSM as an extra symbol, and sets ProduceReferenceAssembly to true so the compiler generates a reference assembly.

The other key thing in there is a target we’ll need to gather the reference assemblies from the main project during packing.

With these, you can use something like #ifdef !(REF_ASSM && NETCOREAPP2.0) in your code to exclude certain methods from the reference assembly on specific platforms. Or, for the “bait and switch” scenario, you may choose to throw an NotImplementedException in some methods (don’t worry, the reference assembly strips out all implementation, but it still has to compile).

You should be able to build these reference assemblies, and in the output directory, you’ll see a ref subdirectory (in \bin\$(Configuration)\$(TargetFramework)\ref). If you open the assembly in a decompiler, you should see an assembly level: attribute [assembly: ReferenceAssembly]. If you inspect the methods, you’ll notice they’re all empty.

Packing the reference assembly

In order to use the reference assembly, and NuGet/MBuild do its magic, it must be packaged correctly. This means the reference assembly has to go into the ref/TFM directory. The library continues to go into lib/TFM, as usual. The goal is to create a package with a structure similar to this:

The contents of the ref folder may not exactly match the lib, and that’s okay. NuGet evaluates each independently for the intended purpose. For finding the assembly to pass as a reference to the compiler, it looks for the “best” target in ref. For runtime, it only looks in lib. That means it’s possible you’ll get a restore error if you try to use the package in an application without a supporting lib.

Out-of-the-box, dotnet pack gives us the lib portion. Adding a Directory.Build.targets above your main libraries gives us a place to inject some code into the NuGet pack pipeline:

This target gets called during the NuGet pack pipeline and calls into the reference assembly project using a convention: $(MSBuildThisFileDirectory)refs\$(MSBuildProjectName)\$(MSBuildProjectName).csproj. It looks for a matching project in a refs directory. If it finds it, it obtains the TargetFrameworks it has and then gets the reference assembly for each one. It calls the _GetReferenceAssemblies that we had in the Directory.Build.props in the refs directory (thus applying it to all reference assembly projects).

Building

This will all build and pack normally using dotnet pack, with one caveat. Because there’s no ProjectReference between the main project and the reference assembly projects, we need to build the reference assembly projects first. You can do that with dotnet build. Then, call dotnet pack on your regular project and it’ll put it all together.

OSS Build and Release with VSTS

Over the past few weeks I have been moving the build system for the OSS projects I maintain over to use VSTS. Up until recently I was using AppVeyor for builds, as they have provided a generous free offering for OSS for years. A huge thank you goes out to them for their past and ongoing support for OSS. So why move to VSTS? There’s three reasons for me:

Support for public projects. This is key since there’s no point in using their builds if users can’t see the results.

Release Management. The existing build systems like AppVeyor, Jenkins, TeamCity, and Travis, can all build a project. Sure, they have different strengths and weaknesses, and some offer free OSS builds as well, but none of them really has a Release Management story. That is, they can build artifacts….but then what? How do the bits get where you want, like NuGet, MyGet, a Store, etc. This is where release management fits in as a central part of CI/CD. More on this later.

Windows, Linux, and Mac build host support in one system. It’s possible to run a single build on all three at the same time (fan out/in), like how VS Code does. No other host can do this easily today using a hosted build pool. I should note that using a hosted build pool is critical for security if you want to build pull requests from public forks. You don’t want to be running arbitrary code on a private build agent. Hosted agent VMs are destroyed after each use, making them far safer.

Life without Release Management

Many projects strive to achieve a continuous deployment pipeline without using Release Management (RM from now on). This is often achieved by using the build script or configuration to do some deployment steps given certain conditions, like if building a certain branch like master. In some ways, the GitFlow branching strategy encourages this, making it easy to decide that builds from the develop branch are pre-release and should go to a dev environment, while builds from master are production and should thus be deployed to a production environment. To me, this is conflating the real purpose of branches, which should be isolation of code churn, from deployment target. I believe any artifact should be able to be deployed to any environment, releasing is a different process than build and should have no bearing on which branch it comes from. For the vast majority of projects, I believe that a GitHub Flow or Release Flow (video) is a better, simpler, option.

Without RM, a pipeline to deploy a library might look something like this:

Builds on the develop branch get deployed to MyGet by the build system. To me, it doesn’t much matter if it’s in the build script directly or if it’s build server configuration (like the deployment option in AppVeyor).

To create a stable release, code is merged to master and then tagged. Often, tags are the mechanism that determines if there should be a release — effectively, tag a commit, that triggers a build which gets released to NuGet.org.

In this model, it’s usually a different build that gets deployed to release than to dev. I believe that mixing build and release like this ultimately leads to less flexibility and more coupling. The source system has to know about the deployment targets. If you need to change the deployment target, or add another one, you have to commit to the source and rebuild.

Following the single responsibility principle, a build should produce artifacts, that’s it. Deployment is something else, don’t conflate the two concepts. Repeat after me: Build is just build. I think projects have tended to mix the two, in part because there wasn’t a good, free, RM tool. It was easy and pragmatic to do both from the build tool. That changes now with VSTS public projects.

CD Nirvana with Release Management

VSTS has a full featured RM tool (deep dive on docs here) that is part of the platform. It is explicitly designed around the concept of artifacts, environments, and releases. In short, a build that contains artifacts can trigger a release. A release defines one or more environments with specific steps that should execute for each. Environments can be chained after another one, enabling a deployment promotion flow. There are many ways to gate each environment, automated and manual. A configuration I use frequently is to have two environments: MyGet and NuGet (dev and prod, respectively). The NuGet environment has an approval step so that releases don’t automatically flow from dev to prod; rather, I can decide to release to production at any time. Any build is a potential release.

Release steps can do anything and there are many existing tasks built-in for common things (like NuGet push, Azure blob copy, Azure App Services, and Docker) and a rich marketplace for things that aren’t (like creating a GitHub release, tagging the commit, and uploading artifacts to the GitHub release). In addition, you can run any custom script.

I think it’s easier to show by example, and that’s what follows in the next sections.

Versioning

Having an automatic version baked into your build artifacts is a crucial element. I use Andrew Arnott‘s Nerdbank.GitVersioning package to handle that for me. I set the Major.Minor in a version.json file and it increments the Patch based on the Git commit height since the last minor change. Add a prerelease tag to the version, if desired. You can control where the git height goes if you don’t want it in the patch (like 1.2.0-build.{height}). The default is the patch, and I think it’s completely okay to have a release like 1.2.42 if there were 42 commits since the version bump. I believe too much time is wasted on “clean” versions; it’s just a number :). Nerdbank.GitVersioning can also can set the build number in the agent, which is really handy knowing what version was just built.

Structuring your branches without overkill

There are many theories around how to structure your branches in Git. I tend to go with simplicity, aiming for a protected master branch with topic/feature branches for work. In my view, the sole reason for branches should be around code churn and isolation.

When it comes to delivery, there are two main schools of thought: releases and continuous. Releases are the traditional way shipping software. A group of features is batched together and shipped out once someone decides “it’s ready.” Continuous Delivery (CD) takes the thought out of releases: every build gets deployed. Note that doesn’t mean every build gets deployed to all environments, but every build is treated as if it could be.

I bring this up because I choose different tagging/branching strategies based on whether I’m doing releases or full CD.

If you’re doing continuous delivery, I would suggest using a single master branch with a stable version in it. Every build triggers a release, at least to a CI environment. At some point, could be every release, a set schedule, etc, that build gets promoted to the production environment. The key here is that it’s a promotion process; builds are fixed and flow through the environments.

If you’re doing release-based delivery, I would suggest using the master with a prerelease version tagged in it (like 1.2-preview). When you’re ready to stabilize your release, cut a rel/1.2 branch for it. In that branch, remove the prerelease tag and continue your stabilization process. Fixes should target master via a PR and then cherry-picked to the release branch if applicable. The release branch never merges back to master in this model.

In my view, using rel/* is perfect for stabilization of a release, enabling master to proceed to the next release. I’ll come back to my earlier point about branches: they should be for isolation of code churn, not environments. A rel branch isn’t always necessary; I’d only create it if there is parallel development happening.

Examples

I have two examples that illustrate how I implement the strategy above.

First, a library author creating a package that gets deployed to two feeds: a CI feed and a stable feed. The tree uses a preview prerelease tag in master and branches underneath rel for a stable release build. My example shows a .NET library with MyGet and NuGet, but the concepts apply to anything.

Second, I have an application that does continuous deployment to an automatically updating CI feed and controlled releases to the Microsoft Store, Chocolatey, NuGet, and GitHub. All releases move forward in master, and will hotfix under rel only if necessary.

Basic Library

Build

For this first scenario, I’ll talk about Rx.NET. It has a a build definition defined in yaml that has these essential parts (non-relevant parts omitted for brevity):

I’m not going to dive too deep in the YAML itself; instead I’ll call your attention to the documentation and examples. As of now, there may be more up-to-date docs in their GitHub repo.

In this case, I have a build script (build-new.ps1) that calls msbuild and does all of the work. In order to ensure the right things are in the path, I call out to the VsDevCmd.bat first. xUnit, as of 2.4.0 beta 2, has direct support for publishing test results to VSTS if you supply the VSTS_ACCESS_TOKEN variable. Other frameworks are supported by the VSTest task. After running the main build script, I use a task to publish the binaries (NuGet packages) that were generated ty the build.

Another approach is to use the tasks for all of this directly, similar to this. We all have preferences between using scripts like PowerShell, Cake, PSake, etc, and the tasks. Doesn’t matter what you pick, use what works for you.

Release

The previous section was about build. The end result is a versioned set of artifacts that can be used as input to a release process. Rx.NET has a release definition here:

One tip on release naming that’s easily overlooked: it can be customized. I like to put the build number in it so I can associate a release with a version, and I concat it with the instance number (since it’s possible to have multiple releases for a particular version). In the definition options, I use the string Release v$(Build.BuildNumber).$(rev:r). That uses the build number from the primary artifact as the name.

The release defines two environments, MyGet and NuGet with an auto release trigger for builds on the master or rel/* branches. RM lets you put branch filters at any point, so you can enforce that releases only come from a specified branch, if desired. In this case, I tell it to create a release after builds from those branches. Then, in the MyGet environment, I’ve configured it to deploy to that environment automatically upon a release creation. That gets me Build -> Release -> MyGet in a CD pipeline. I do want to control releases to NuGet in two ways: 1) I want to ensure they are in MyGet, and 2) I want to manually approve it. I don’t want every build to go to NuGet. I have configured the NuGet environment to do just that, as well as only allowing the latest release to be deployed (I’m not looking to deploy older releases after-the-fact).

The MyGet environment has one step: a NuGet push to a configured endpoint. The NuGet environment has two steps: create a GitHub release (which will tag the commit for me), and a NuGet push. Releases don’t have to be complicated to benefit from using an RM flow. My release process is simple: when it’s time for a release, I take the selected build and push the “approve” button on the NuGet environment. There’s many other ways to gate releases to environments and you can do almost anything by calling out to an Azure Function as a gate.

It’s a bit hard to see how the release pipelines are configured on the site, so here are some screenshots showing the configuration:

Deployment Trigger:

MyGet Environment:

NuGet Environment:

Environment Triggers for NuGet

The actual release process to NuGet goes like this:

If I want to release a prerelease package, I can just press the approve button. It’ll do the rest.

If I want to release a stable package, I create a branch called rel/4.0 (for example) and make one edit to the version.json to remove the prerelease tag. That branch will never merge back to master. I can do as much stabilization in that branch as needed, and when I’m ready, I can approve that release to the NuGet environment. If there are hotfix releases I need to make, I will always make the changes to master (via a PR), then cherry-pick to the rel branch. This ensures that the next release always contains all of the fixes.

A Desktop Application

NuGet Package Explorer (NPE) is a WPF desktop application that is released to the Microsoft Store, Chocolatey, and GitHub as a zip. It also has a CI feed that auto-updates by using AppInstaller. NPE is delivered via a full CD process. There aren’t any prerelease versions; every build is a potential release and goes through an environment promotion pipeline.

Build

As a Desktop Bridge application, it contains a manifest file that must be updated with a version. Likewise, the Chocolatey package must be versioned as well. While there may be better options, I’m currently using a PowerShell script at build to replace a fixed version with one generated from Nerdbank.GitVersioning. I also update a build badge for use as an deployment artifact later.

Release

The release definition for NPE is more complicated than the previous example because it contains more environments: CI, Prod - Store, Prod - Chocolatey, Prod - NuGet, and Prod - GitHub. Most of the time releases go out to all production environments, but if there’s a fix that’s applicable to a specific environment, it only goes out to that one. All fixes go to CI first.

The triggers for the release and environments are the same as the previous example, so I won’t repeat the pictures. The steps vary per environment, performing the steps needed to take the artifacts and copy to Azure Blob, upload to the Microsoft Store, push to NuGet or Chocolatey, or create a GitHub release with artifacts, as the case may be.

Conclusion

For me, Release Management is a huge differentiator and fits into my way of thinking very well. I like the separation of responsibilities between build and release it provides. Now that public projects are available in VSTS, contributors to the projects can get build feedback and the community can check-in on the deployment status.

I’d love to hear feedback or suggestions and getting me on Twitter is usually the fastest way.

I am thrilled to announce that I received, and accepted, an invitation to join the Microsoft Regional Director program. I’m humbled and honored to be among the ranks of people who I’ve looked up to for most of my professional career. A very huge Thank You to those who nominated and supported me for this program.

If you’re not familiar with what a Regional Director is, the website explains pretty well:

The Regional Director Program provides Microsoft leaders with the customer insights and real-world voices it needs to continue empowering developers and IT professionals with the world’s most innovative and impactful tools, services, and solutions.

Established in 1993, the program consists of 150 of the world’s top technology visionaries chosen specifically for their proven cross-platform expertise, community leadership, and commitment to business results. You will typically find Regional Directors keynoting at top industry events, leading community groups and local initiatives, running technology-focused companies, or consulting on and implementing the latest breakthrough within a multinational corporation.

It is coming up on four years since I was first awarded Windows Developer MVP in July, 2014, and two years since Microsoft awarded me a second category of Visual Studio & Development Technologies. The journey has been incredible, getting to meet so many amazing people along the way.

Registration-free COM with Azure App Services

TL;DR: If you ever got this error on Azure App Services, read-on: The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detail.

Background

This is admittedly an esoteric topic, but useful to know in case you happen to hit it. Azure App Services are wonderful; they make it really easy to host a website or API in Azure, with the platform handling the “hard parts” of HTTPS certificate management, scaling, load balancing, etc. A few months ago, while working on my code signing service, I hit a strange issue when running on Azure that didn’t reproduce locally. MakeAppx.exe, a tool that expands and creates AppX packages, was failing. The code signing service bundles a few utility exe’s that are executed for certain operations. In this case, I needed to extract Appx/Appxbundle files, modify the contents and repack them.

When I started investigating, I went to the Kudu console and tried executing makeappx.exe, I got the following error: The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detail.. No combinations of parameters made a difference, it wasn’t even getting that far.

The cause

After looking at potential differences in that application vs every other application that worked, I noticed that makeappx.exe uses an embedded manifest that declares registration-free COM dependencies. This is because makeappx.exe includes local copies of some files may be newer than the ones included in the operating system. If I replaced the built-in manifest with a default one, I could launch the application, so that was definitely the issue. The caveat is two-fold: 1. I was modifying a Windows SDK binary, and I didn’t want to do that, 2. I’d be tied to a particular version of that tool that matched whatever app services was running on.

Investigating

Given that I was now in uncharted territory, I reached out to the App Service team and they graciously helped. In particular, I need to thank David Ebbo and Petr Podhorsky, without whom I’d still be stuck.

What the investigation found: due to the way App Service maps in the “D” drive (the default one your site runs from). As Petr explains:

Registration-free COM side-by-side is handled by a different process on the box, CSRSS, which runs under a different account and because the path is transferred between processes unchanged, it tries to look at d:\home path, which is a special path and makes sense only for your site process, but nothing else on the box (if there are multiple sites running on the box, they all have their own d:\home and don’t see each other’s content). So it does not find your manifest.

Even if the previous problem was solved, the CSRSS (client server runtime subsystem) process does not have access to the site content anyway.

The solution

Fortunately, there is a workaround: use the HOME_EXPANDED environment variable. That points to the real path of the “C drive” equivalent location, something like C:\DWASFiles\Sites\#1mysite1\home. Don’t worry about the exact location, just use the variable.

Invoking makeappx.exe from the HOME_EXPANDED location works because CSRSS sees the files that the manifest points to and it loads properly. You can see how I applied the workaround here.

Conclusion

While it’s likely rare to have a program using registration-free COM invoked on App Services, it’s not entirely impossible. You might have a web job or function that needs to invoke some program, or you might need to invoke programs for some website functionality like I do. If you do hit this, I hope this workaround unblocks you too.

Continuous Deployment of Cloud Services with VSTS

In my last blog post, I showed how you can use ASP.NET Core with an Azure Cloud Service Web Role. The next step is to enable CI/CD for it, since you really shouldn’t be using “Publish” within Visual Studio for deployment.

As part of this, I wanted to configure the Cloud Service settings per environment in VSTS and not have any configuration checked-in to source control. Cloud Services’ configuration mechanism makes this a bit challenging due to the way it stores configuration, but with a few extra steps, it’s possible to make it work.

What you’ll need

To follow along, you’ll need the following:

Cloud Service the code can live in GitHub, VSTS, or many other locations. VSTS can build from any of them.

Azure Key Vault we’ll use Azure Key Vault to store the secrets. Creating a Key Vault is easy and the standard tier will work.

VSTS this guide is using Visual Studio Team Services, so you’ll need an account there. Those are free for up to five users and any number of users with MSDN licenses.

What we’re going to do

The gist here is that we’ll create a build definition that publishes the output of the Cloud Service project as an artifact. Then, we’ll create a release management process that takes the output of the build and deploys it to the cloud service in Azure. To handle the configuration, we’ll tokenize the checked-in configuration, then use a release management task to read configuration values stored in Key Vault and replace the matching tokenized values before the Azure deployment.

Moving the configuration into Key Vault

Create a new Key Vault to hold your configuration. You should have one Key Vault per environment that you intend to release to, since the secret names will directly translate to variables within VSTS. For each setting you need, create a secret with name like CustomSetting-Setting1 or CustomSetting-Setting2 and set their values. Next, in your ServiceConfiguration.Cloud.cscfg, set the values to be __CustomSetting-Setting1__ and __CustomSetting-Setting2__. The __ is the token start/end, and the value identifies which VSTS variable should be used to replace it.

One tip: If you have Password Encryption certificates or SSL endpoints configured, the .cscfg will have the certificates’ SHA-1 thumbprint’s encoded in them. If you want to configure this per environment, then replace those with token values. The configuration checker will enforce that it looks like a thumbprint, so use values like:

ABCDEF01234567ABCDEF01234567ABCDEF012345

BACDEF01234567ABCDEF01234567ABCDEF012345

Those sentinel values will be replaced with tokens during the build process and those tokens can be replaced with variable values.

If you are using sentinel certificate values, add a PowerShell Task. Configure the PowerShell task by selecting “Inline Script”, expand Advanced and set the working folder to the publish directory (like $(Build.ArtifactStagingDirectory)\app.publish) and use the following script:

This replaces the fake SHA-1 thumbprints with tokens that release management will use. Be sure to define variables in release management that match the names you use.

Finally, set the Publish Artifact step to:

Setting

Value

Display name

Publish Artifact: Cloud Service

Path to Publish

$(Build.ArtifactStagingDirectory)\app.publish

Artifact Name

TheCloudService

Artifact Type

Server

Go to the Variables tab and add two variables:

Name

Value

BuildConfiguration

Release

BuildPlatform

Any CPU

Hit Save & Queue to save the definition and start a new build. It should complete successfully. If you go to the build artifacts folder, you should see TheCloudService with the .cspkg file in it.

Deploying the build to Azure

This release process depends on one external extension that handles the tokenization, the Release Management Utility Tasks. Install it from the marketplace into your VSTS account before starting this section.

In VSTS, switch to the Releases tab and create a new release definition using the “Azure Cloud Service Deployment” template.

Give the environment a name, like “Cloud Service – Prod”.

Click the “Add artifact” box and select your build definition. Should look something like this: If you want continuous deployment, click the “lightning bolt” icon and enable the CD trigger.

Click on the Tasks tab and specify an Azure subscription, storage account, service name and location. If you need to link your existing Azure subscription, click the “Manage” link. If you need a new storage account to hold the deployment artifacts, you can create that in the portal as well, just make sure to create a “Classic” storage account.

Go to the Variables tab and select “Variable groups”, then “Manage variable groups.” Add a new variable group, give it a name like “AspNetCloudService Production Configuration”, select your subscription (click Manage to link one), and select the Key Vault we created earlier to hold the config. Press the Authorize button if prompted. Finally, click Add to select which secrets from Key Vault should be added to this variable group. It’s important to note that it does not copy the values at this point. The secret’s values are always read on use, so they’re always current. Save the variable group and return back to the Release Management definition. At this point, you can select “Link variable group” and link the one we just created.

Add a Tokenize with XPath/Regular Expressions task before the Azure Deployment task.

In the Tokenizer task, browse to the ServiceConfiguration.Cloud.cscfg file, something like $(System.DefaultWorkingDirectory)/AspNetCoreCloudService-CI/TheCloudService/ServiceConfiguration.Cloud.cscfg depending on what you call your artifacts.

Ensure that the Azure Deployment task is last, and you should be all set.

Create a new release and it should deploy successfully. If you view your cloud service configuration on Azure Portal, you should see the real values, not the __Tokenized__ values.

Summary

That’s it, you now have an ASP.NET Core Cloud Service deployed to Azure with CI/CD through VSTS. If you want to add additional environments, simply add an additional key vault and linked variable group for each environment, clone the existing environment configuration in the Release Management editor and set the appropriate environmental values. Variable groups are defined at the release definition level, so for multiple-environments you can use a suffix in your variable names and then update the PowerShell script in step 7 to append that per environment (__MyVariable-Prod__), etc.

Using ASP.NET Core with Azure Cloud Services

Overview

Cloud Services may be the old-timer of Azure’s offerings, but there are still some cases where it is useful. For example, today, it is the only available PaaS way to run a Windows Server 2016 workload in Azure. Sure, you can run a Windows Container with Azure Container Services, but that’s not really PaaS to me. You still have to be fully aware of Kubernetes, DC/OS, or Swarm, and, as with any container, you are responsible for patching the underlying OS image with security updates.

In developing my Code Signing Service, I stumbled upon a hard dependency on Server 2016. The API I needed to Authenticode sign a file using Azure Key Vault’s signing methods only exists in that version of Windows. That meant that using Azure App Services was out, as it uses Server 2012 (based on the version numbers from its command line). That left Cloud Service Web Roles as the sole remaining option if I wanted PaaS. I could have also used a B-Series VM, that’s perfect for this type of workload, but I really don’t want to maintain a VM.

If you have tried to use ASP.NET Core with a Cloud Service Web Role, you’ll probably have come away disappointed as Visual Studio doesn’t let you do this…. until now. Never one to accept no for an answer, I found a way to make this work, and with a few workarounds, you can too.

The solution presented here handles deployment of an MVC & API application that along with config settings and deployment of the ASP.NET Core Windows Hosting Module. VS Cloud Service tooling works for making changes to config and publishing to cloud services (though please use CI/CD in VSTS!)

TheCloudService The Cloud Service project. Contains the configuration files and service definition.

TheWebRole ASP.NET 4.6 project that contains the Web Role startup scripts and “references” the TheWebsite site. This is where the tricks are.

At a high level, the Cloud Service “sees” TheWebRole as the configured website. The cloud service doesn’t know anything about ASP.NET Core. The trick is to get the ASP.NET Core site published and running “in” an ASP.NET site.

Doing this yourself

The Projects

In a new solution, create a new ASP.NET Core 2 project. Doesn’t really matter what template you use. For the descriptions here, I’ll call it TheWebsite. Build and run the site, it should debug and run normally in IISExpress.

Next, create a new Cloud Service (File -> Add -> New Project -> Cloud -> Azure Cloud Service). I’ll call the cloud service TheCloudService, and on the next dialog, add a single Web Site. I called mine TheWebRole.

Finally, on the ASP.NET Template selection, choose “Empty” and continue.

Right now, we have an ASP.NET Core Website and an Azure Cloud Service with a single ASP.NET 4.6 WebRole. Next up is to clear out almost everything from TheWebRole since it won’t actually contain any ASP.NET Code. Delete the packages.config and Web.config files.

Save the project, then select “Unload” from the project’s context menu. Right-click again and select “Edit TheWebRole.csproj”. We need to delete the packages brought in by NuGet along with the imported props and target. There are three areas to delete as noted in the screen shots: Props at the top, all Reference elements with a HintPath pointing to ..\packages\ and the Target at the bottom. At this point, your project file should look similar to this here. You can also view the complete diff.

Magic

Now comes the special sauce — we need a way to have TheWebRole build TheWebsite and include TheWebsite‘s publish output as Content. Doing this ensures that TheCloudService Package contains the correct folder layout. Add the following snippet to the bottom of TheWebRole‘s project file to call Publish on our website before the main build step.

Save the csproj file, then right-click the TheWebRole and click Reload. You can test that the cloud service package is created correctly by right-clicking TheCloudService and selecting Package. After choosing a build configuration and hitting “Package,” the project should build and the output directory pop up.

The .cspkg is really a zip file, so extract it and you’ll see the guts of cloud service packages. Look for the .cssx file and extract that (again, just a zip file)

Inside there, open the approot folder and that is the root of your website. If the previous steps were done correctly, you should see something like the following

You should see TheWebsite.dll, TheWebsite.PrecompiledViews.dll, wwwroot, and the rest of your files from TheWebsite.

Congratulations, you’ve now created a cloud service that packages up and deploys an ASP.NET Core website! This alone won’t let the site run though since the Cloud Service images don’t include the Windows Hosting Module.

Installing .NET Core 2 onto the Web Role

Installing additional components onto a Web Role typically involves a startup script, and .NET Core 2 is no different. There is one complication though: the installer downloads files into the TEMP folder, and Cloud Services has a 100MB hard limit on that folder. We need to specify an alternate folder to use as TEMP with a higher quota (this is what Jaques and Catherine figured out).

In TheCloudService, expand Roles, right click TheWebRole and hit properties. Go to Local Storage and add a new location called CustomTempPath with a 500MB limit (or whatever else your app might need).

Next, we need the startup script. Go to TheWebRole, add a new folder called Startup and add the following files to it. Ensure that the Build Action is set to Content and that Copy to Output Directory is set to Copy if newer. Finally, we need to configure the cloud service to invoke our startup task. Open the ServiceDefinition.csdef file and add the following xml in the WebRole node to define the startup task:

Now we finally have a cloud service that can be deployed, install .NET Core, and run the website. The first time you publish, it will take a few minutes for the role instance to become available since it has to install the hosting module and restart IIS.

Note: I leave creating a cloud service instance in the Azure Portal as an exercise to the reader

Configuration

There are many ways of getting configuration into an ASP.NET Core application. If you know you’ll only be running in Cloud Services, you may consider taking a direct dependency on the Cloud Services libraries and using the RoleEnvironment types to get populate your configuration. Alternatively, you can likely write a configuration provider that funnels in the RoleEnvironment configuration into the ASP.NET Core configuration system.

In my original case, I didn’t want my ASP.NET Core website to have any awareness of Cloud Services, so I came up with another way—in the startup script, I copy the values from the RoleEnvironment into environment variables that the default configuration settings pick up. The key here to making this transparent is knowing that the double-underscore, __, translates into the : when read from an environment variable. This means you can define a setting like CustomSetting__Setting1, and then you can access it with Configuration["CustomSetting:Setting1"], or similar mechanisms.

To bridge this gap, we can add this to the startup script (complete script):

Considerations

Data Protection API Unlike Azure App Services, Cloud Services doesn’t have any default synchronization for the keys. You’ll need a solution for this. If anyone comes up with a reusable solution, I’ll happily mention it here. More info on configuring DPAPI is here.

Local Debugging Due to the way local debugging of cloud services works (it directly uses TheWebRole as a startup project in IIS Express), directly debugging the cloud service does not work with the current patterns. Instead, you can set TheWebsite as a startup project and debug that normally. The underlying issue is that TheWebRole includes TheWebsite as Content and does not copy the published files to TheWebRole‘s directory. It may be possible to achieve this, though you’d likely want additional .gitignore rules to prevent those files from being committed. In my case, I did not want my service to have any direct dependency on Cloud Services, so this wasn’t an issue—I simply needed a Server 2016 web host.

CI / CD with VSTS

It is possible to automate build/deploy of these cloud service web role projects using VSTS. My next blog post will show how to set that up.

Use all TFM’s with SDK-style projects in Visual Studio for Mac

TL;DR

You can now use SDK-style projects, with all supported TFM’s, in Visual Studio for Mac. See getting started for details.

Issue

While Visual Studio for Mac supports the SDK-style projects, there have been a couple of issues blocking use of TFM’s other than net, netstandard, and netcoreapp.

Those TFM’s are hard-coded and an SDK-style project containing any other target frameworks is blocked.

Xamarin on the Mac has a multi-valued MSBuildExtensionsPath property. That means that it can search for targets in different locations. Unfortunately the logic this works with is limited to the <Import /> element, so if you set properties, as is required to use LanguageTargets, it won’t work. Fortunately, with some brainstorming with Ankit Jain and Mikayla Hutchinson, we found a solution.

Building these projects

These projects will build in the IDE (VSfM, VS, etc) or the command line. If you use the command line, you must use msbuild, not dotnet build. Keep in mind that with msbuild, you must explicitly call restore first, so your build steps will look like this:

msbuild /t:restore
msbuild /p:Configuration=Release

Notes

For the beta, since it’s a SemVer2 package, you must be using the NuGet v3 feed. If your VSfM prefs have https://www.nuget.org/api/v2/, you need to update that to be https://api.nuget.org/v3/index.json.

Announcing Reactive Extensions for .NET 4.0 Preview 1!

I am happy to announce that the first preview of Rx.NET 4.0 is now available. This release addresses a number of issues and contains several enhancements.

The biggest enhancement is consolidating the existing packages into one main package, System.ReactiveNuGet. This will prevent most of the pain around binding redirects that were caused by #205. If you are using Rx 4.0 and need to use libraries built against Rx 3.x, then you need to also install the compatibility package System.Reactive.Compatibility. That package contains facades with type forwarders to the new assembly so types are unified correctly. You only need this compatibility package if you are consuming a library built against 3.x. You do not need it otherwise.

If you’re interested in the background behind the version numbers, I suggest reading the thread as it contains the gory details. While the idea was technically sound, it did mean that binding redirects were required for all .NET Framework uses. We heard the feedback loud and clear that this was really painful and took steps to fix it in 4.0.

The fix was to consolidate the previous set of packages into a single System.Reactive package. With the single package, binding redirects are no longer required and the platforms will get the correct Rx package version.

Please try it out and let us know if you encounter any issues at our repo. The full release notes are there too.