Tips for making development teams work better

Azure

There’s a little-known but excellent feature of Azure web apps that makes it easy to access the debugging and development extensions of your Azure website. If you navigate directly to https://your_site.scm.azurewebsites.net/ (replace “your_site” with the real name of your web site) you will find Microsoft’s Kudu service – the engine behind git deployments for Azure web sites.

There’s tons of useful information you can see using this tool, such as process information, environment variables, server variables and a lot more.

More importantly, it’s also a portal to easily install and access site extensions, such as Visual Studio Online (which you can use to edit files in place on your web site).

For example, to access Visual Studio Online, select the ‘Site Extensions’ menu. There you will find listed all the available site extensions. The first should be VSO. You will need to enable it first, which requires a website restart, but once done, you can access VSO anytime using the following URL format:

Like this:

Introduction

One of the benefits of using Azure resource groups is being able to quickly create identical environments using resource group templates and powershell scripts, so when I was asked to quickly whip up a new test environment, I was prepared, and feeling pretty good about having automated all the things in advance.

Using the template I had created earlier, I had a shiny new environment within a few minutes. Then I thought.. but what about the data?

The Problem

The solution we are developing uses a Sql server row level security policy to provide access restrictions, and one of the drawbacks of this method is that it’s not easy to extract data using traditional data extraction tools, as every login only sees their own data (as it should be). We could have dropped the policy, created a backup (bacpac) file, and then imported to the new empty database. However, this would require a little downtime to prevent users seeing each other’s data, and anyway, that’s not really an elegant solution.

What I wanted to achieve was:

Make a copy of an existing database, including all data, users, and schema information.

Keep the same SID (security identifiers) on the new server, so that the row level security wasn’t affected.

Avoid modifying the original database in any way.

Create the new database in a different resource group.

Provide a solution that could be automated.

The Solution

The first step was to create a new database, and populate it with the data from the existing one. The easiest way is to use the portal to create the new database, and specify a backup source, which should list your existing database. Once you click ‘Create’, Azure will do the hard work to import all the data, users and schema information for you.

Step 1. Select source backup.

Step 2.
The next bit is to recreate the logins using the same SID’s on the new server. This is important as row level security uses the SID to determine which user owns which data. To help in doing this, I wrote a small Sql script that needs to be run on the source database. It prints a Sql statement that creates the logins, and maps the logins to the users in the new database.

Copy the output of the previous step, log into the new server as admin, connect to the new database, and run the generated script. The script will create logins on the new server, and then link the login to the user. They key ingredient is to create the login with the correct SID. For this scenario, the password is not important, but if you need to also get the password from the old server, you will need to modify the script to create a login with a given password hash.

Step 4.
Publish the application.
With the hard work done, the last step is to just publish the code. To do this, you can download the publish settings of your application from the portal, import them into Visual Studio, and publish.

Final Thoughts

This solution at this stage was a bit of a quick hack – the next obvious step is to wrap this all into a nice package for easy re-usability. With this process, we are now able to take a snapshot of the current state of our solution, and replicate the entire environment, including data and security, in a little over 30 minutes. While this solution was implemented and tested on Azure, the same process should work in SQL Server 2016.

First up, we were pleasantly surprised that Visual Studio Pro edition with MSDN now gives you $70 in monthly credits. At the same time, the pricing of VM’s was decreased. What this mean is that I can hopefully keep a small VM up and running for more than half the month without running out of credit.

Oh! And Visual Studio 2015 has been released! Aside from a few small issues, existing projects in VS 2013 should work just fine in VS 2015.

A separate, much more serious issue was found in .NET 4.6, which can result in variables having incorrect values due to an optimization bug in the new .NET compiler (RyuJit). A detailed synopsis of the issue is documented here:

Like this:

Introduction

Now that we have a web app in azure, and have connected it to Visual Studio Online in the earlier posts (part 1 and part 2), it’s time to finish up this little project by checking out how the build and deployment pipeline works.

Continuous Integration

Log into the Visual Studio Online account linked to you Azure subscription, and go to to our project, which should be available from the dashboard. Once you click on the project name, go to the ‘BUILD’ tab.

You can see the build definition created for you when the web app was linked to Visual Studio Online. You can also go to the ‘CODE’ tab to confirm that all your project files have been committed to Visual Studio Online’s Git repository.

Once that’s confirmed, we need to test the integration with our development environment. For that, let’s go back to Visual Studio 2013, open the project, and make a change. I’ve added an extra value to the Values Controller class.

Now we need to commit our changes to the local repository, and sync to Visual Studio Online.

If everything worked correctly, we should shortly see our changes on the production site, and our source code in Visual Studio Online. Let’s log back into Visual Studio Online, and check to see what has occurred after committing our changes.

First, in the ‘CODE’ tab, we can see that our changes were synced to the remote repository – the “value3” change is visible in the code explorer view.

Next, let’s check the ‘BUILD’ tab. Sure enough, there’s a new build there created a minute ago – it has a green tick, so the code was successfully compiled on the hosted build server.

Great, just a couple more things – let’s check the build logs, by right-clicking the build, and selecting ‘Open’.

And finally, we need to open our web site, go to the values API, and confirm that the changes are successfully deployed to production.

Conclusion

This series of articles has barely scratched the surface of the capabilities in Visual Studio Online. The next step is to schedule automated tests, set up deployment slots, and customize the build process.

However, I hope this brief look at Visual Studio Online’s features is enough to encourage you to look further.

Share this:

Like this:

Introduction

Last week, I wrote about some of the reasons why you might want to use Visual Studio Online’s hosted build controller to build and deploy your Azure web apps. For small teams, or individuals, you can get all the benefits of a robust build server and continuous integration for your project, without the hassle of maintenance and administration. This week, I’ll build a simple MVC app, and configure it for source control and continuous integration using Visual Studio Online.

This guide is aimed at people who are somewhat familiar with Visual Studio and the basics of source control, but have not used Visual Studio Online or Team Foundation Server before.

Prerequisites

You will need Visual Studio 2013 (any version), as well as an Azure subscription to complete this walkthrough.

Creating the project

I’m going to switch things around from the usual scenario for Visual Studio Online source control, where you are forced to start with a blank project and open it in Visual Studio – if you need to know how to do that, there are plenty of excellent tutorials, like this one at: http://blogs.msdn.com/

In many scenarios you will have an existing solution that you want to add to a new Visual Studio Online repository, and this is quite simple, but not well documented.

We will use the Web API template, and configure it to be hosted in on Azure, like so.

Once you click OK, you will be prompted to create the Azure Web App. The app name needs to be unique, and you will need to specify a resource group, or create a new one. For the region, choose one that is near to you, and you can leave the database option as ‘No Database’. After clicking ‘OK’ the web app will be created and ready to use.

Once finished, you should be able to run the application with Ctrl-F5, using the local IIS Express server.

Configuring the web app for Visual Studio Online

Now that we have an existing application, let’s create the Visual Studio Online account that will allow us to build the solution online.

First, log into the Azure Management Portal (https://manage.windowsazure.com/), and create a new Visual Studio Online Subscription if yo don’t already have one. Go to “App Services\Visual Studio Online\Quick Create” to do this.

Select the URL (needs to be unique, e.g. <yourname>.visualstudio.com), the Azure Active Directory service and the Region closest to you, and click ‘Create Account’.

Linking the project to Visual Studio Online

For a brand new solution you can just click the ‘Open in Visual Studio’ link, and you’re ready to go. But there’s one extra step to link an existing solution to Visual Studio Online.

Basically, what I will be doing is using a local Git repository for the solution’s source control, and then pushing the master branch up to Visual Studio Online.

Go back into Visual Studio, right click the solution, and choose ‘Add to Source Control’. When prompted, choose Git as the source control provider.

Next, you will see the Team Explorer – choose to see the ‘Changes’ screen, type a commit message, and click ‘Commit’ to commit the project files to local source control. At this point, your code is being tracked for changes, and the history is stored on your local drive, so let’s go one better and push those changes up to Visual Studio Online.

Click the icon in Team Explorer that looks like a power plug, and either select an empty team project, or choose to create a new team project. If doing the latter, you will be asked to supply a project name – type VSOWebDeploy and click Create.

In a short while, the new project should be available for use. At this point, you will see a hint in the Team Explorer that you need to clone your repository to get started. This is pretty much what we need, so let’s click that link and clone our repository.

Here, it can get a bit messy, so you need to make sure you do things in the right order, or you’ll get error messages about merge ref not found, which basically means it can’t find anything to merge with.

You can resolve this in a number of ways, and if you are used to command line git, you can just force a push to the remote repository.

A simper way, though, is to go to ‘Unsynced Commits’, enter the remote repository URL, and click ‘Publish’. Your remote repository URL is in the form of https://{Your Account Name}.visualstudio.com/DefaultCollection/_git/{Your Team Project Name>} You can also find the URL by visiting your Visual Studio Online account, and going to the ‘Code’ link, if you have not yet synced to Visual Studio.

Linking the Web App to Visual Studio Online

Next, go to the web apps list, and find VSOWebDeploy in the list of Web Apps. On the dashboard, you will see a link to ‘Set up deployment from source control’.

You will be prompted to choose location of your source code – choose Visual Studio Online, to connect the project to a Visual Studio Online account.

After this, you will be asked to confirm the link, and that’s pretty much it – once linked, you can start using Git and Visual Studio Online builds publish your app directly from Azure, using the Hosted Build Controller.

When working on the code, make sure you remember to commit your changes regularly to the remote repository. By signing in to the Visual Studio Online account, you can explore the code, see a history of the commits, and perform various administrative tasks.

In the next post, I will finish up by setting up an automated build and continuous integration.

Share this:

Like this:

The ‘Edit in Visual Studio’ feature in the Azure Portal allows you to edit files on the live site, without needing to deploy or publish your app. It also contains some really useful tools for when you need to troubleshoot or configure your web app, and only have access to a web browser.

Now you can go the DASHBOARD link of the Web App, and find the ‘Edit in Visual Studio Online’ option in the ‘Quick Glance’ section on the right.

Apart from being able to edit files directly on the live site, you can access the following tools:

Git – for pushing and pulling to a local repository, or VSO

Output Console – for reading debug messages as the site is running

Console – a basic shell with some command line tools

The list of command line tools available in the console can be retrieved by typing “help”. Notable ones include the Node Package Manager (npm), Git, Powershell (ps) and msbuild, along with the usual file and directory manipulation tools.

Like this:

Introduction

So we recently started a new project at work, hosted on Azure. As part of the initial investigation, I stared poking around in the Azure Portal to see what new toys we might be able to play with.

One of the items under review is our source control, build and deployment tool-chain, which, frankly, is a bit outdated. Since Microsoft recently released an updated Build module for Visual Studio Online, I took the time for some further investigation.

With that in mind, let’s have a look at some of the new features, and how it can be leveraged to improve the build and deployment process.

This week, I’m looking at the Hosted Pool Build Controller

Hosted Build Controller

In the past, we have deployed and maintained our own VM’s for builds and deployment. The advantage of this is that all our source code lives on the internal network, and we are in full control of the entire build process.

Visual Studio Online can provide all that for you in a user friendly portal, where you can define your build process, set up continuous integration or a build schedule, and then receive alerts, or just review logs as needed.

The combination of Hosted Builds, Unit Testing, Web App Deployment Slots and Continuous Integration presents a powerful argument to move your builds into the cloud, but it does come with some risks and downsides.

When would you choose to use Visual Studio Online Hosted Builds?

You are already using VSO for source control (TFS or GIT)

Your application is fully hosted on Azure

Maintaining a separate build server is difficult or too time consuming

Downsides to Hosted Builds:

You are storing your source code in the cloud, which poses a security risk

Limited amount of customised build steps are available

Support for languages, compilers and source control systems is limited

No administrator access to the build controller

In summary, VSO Hosted Builds are a great way to get going quickly, provided you are willing to stay ‘in the box’ with regard to compilers, build and deployment tools. There are also limitations on the size of the application (10Gb as of writing), and the list of installed applications.

The list of features is really great, and growing, so it’s well worth checking it out.