Follow Us

Lab Management Evolution

Lab Management was introduced in TFS 2010 to help development teams easily deploy and test their applications on virtual machines in their routine ALM workflows. However, you have not seen us add significant new features to Lab Management in subsequent releases of TFS. Many of you asked us about the evolution of Lab Management. In this article, we will describe how we are thinking about this area, and what you can expect over the next few months.

When we started building Lab Management features for TFS 2010, our focus was on integrating with one platform – Windows Hyper-V managed through System Center Virtual Machine Manager (SCVMM). A significant portion of our investment at that time was focused on filling the gaps that existed in SCVMM. Here are some examples:

There was no concept of multi-tenancy in SCVMM at that time. The notion of a private cloud was just emerging. To integrate with TFS, we had to fill this gap by inventing a number of concepts for mapping different SCVMM host groups and library servers to team project collections and team projects. Contrast that with the present day, where there are a number of mature private and public cloud platforms, all of which have support for multi-tenancy in the form of accounts, subscriptions, and RBAC. The older model of integrating TFS with these cloud platforms does not make sense any more.

Very few resource types were supported by SCVMM at that time – virtual machines being the prominent one. In fact, there was not even a concept of a ‘group of virtual machines’ in SCVMM. It was not possible to clone virtual machines without causing network conflicts. To support deployment of multi-tier applications from TFS, we had to fill that gap by introducing the concept of a “SCVMM environment”. And, we invested quite a bit in isolating one environment from another so that cloning could be done. Contrast that with today’s situation, where all the mature cloud platforms including SCVMM support a variety of resource types and have ways of grouping resources. For instance, Azure supports websites, storage, virtual machines, resource groups, and so on. Virtual networking support in these platforms can accomplish what we tried to do with network isolation in lab management, and much more. It does not make sense for us to get into resource grouping and networking in TFS anymore.

If you think of a standard lab manager persona, he or she would be interested in setting policies on resource consumption (e.g., no developer should get more than 3 medium sized VMs), and on tracking that consumption (e.g., number of unused VMs). We have not focused on such lab management scenarios in the past either. We expect platforms and platform partners to provide such functionality, as they cater more and more to the needs of Dev/Test users. In particular, the Azure Dev/Test Labs effort is one such example. In the foreseeable future, we do not see a need to invest in such areas within TFS/VSO.

It is not just the cloud platforms that have evolved. The push for agility and DevOps also resulted in a number of light-weight portable deployment technologies. Container technologies such as Docker and wrappers such as Vagrant help developers easily provision new machines (or containers) on demand. These technologies are easy to use, easy to script, and fast to update. Moreover, one can deploy the same container or wrapper to staging and production environments. For teams adopting these technologies, lab management needs – in terms of easily provisioning new machines on demand – are mostly met.

Now that we have established what we do not want to do in the future, let us take a look at some of the requirements that have only become stronger now:

Variety of platforms: The variety of platforms on which applications are being developed has grown. You have been asking us to build ALM tools that can help you easily integrate with private cloud platforms such as System Center, Azure Stack, and VMware, or public cloud platforms such as Azure and AWS. These platforms support a variety of resource types, and have made great strides in terms of infrastructure programmability.

Automated ALM workflows: With the increasing need for faster and reliable deployments, the need for seamlessly provisioning fresh resources, deploying new builds, and testing them on an automated basis has increased. Some of you want to do this every time a new build is available, others want to do this on a nightly basis, and the remaining on a more frequent cadence than they are at currently.

Manual ALM workflows: The need for development teams to setup their self-service, bug-bash, demo, or feedback environments has only increased in the DevOps world. Infact, with the DevOps mindset, the manner in which newer builds are deployed to any of these environments should be identical to the way they are deployed eventually to staging and production environments.

Within VSO and TFS, our services around Build, Test, and Release Management have been evolving rapidly as well. You have given us feedback that it is confusing to have environments in both lab management and in release management, especially when they do not integrate. You have also told us that we should simplify automated and manual ALM workflows significantly and that we should enable your teams to grow up from one use case to another. Moving forward, we will expose lab management features as part of the web interface for TFS, instead of in Microsoft Test Manager (MTM). In the rest of this article, we will show you how you can accomplish the automated and manual ALM workflows in this new world. Most of the features that we describe below will be coming over the next few months to you.

Azure integration

You can dynamically provision websites, cloud services, or resource groups in Azure. To do this, you will be able to add your Azure subscription to your VSO account, and start using that subscription as part of tasks in Build and Release Management flows. The resources can be used to deploy your application-under-test or to spin up a test agent and run functional tests.

We will support multiple forms of authentication from your VSO account to Azure – Azure certificates, user credentials, and service principals. With Azure resource group task, you can leverage the full power of Azure’s infrastructure-as-code. Start with hundreds of starter templates that are being defined by the community for Azure, and customize them to build the infrastructure that you need for your apps. Or else, you can use the Azure powershell task to script the resource provisioning commands.

Service endpoint extensibility

Azure is just one of the service endpoints that you can connect to from VSO. We have been working on a model, where customers and partners can contribute extensions that define new endpoint types for a different platform or deployment service, and that can run provisioning or deployment tasks against that service. We have started building some of these common extensions ourselves – Chef is one such example.

Integration with various platform and deployment technologies

Using the extension model, we plan to support various integrations out-of-the-box. Two integrations that we would like to call out as being in our plans are VMware and Docker – VMware for being a consistent ask from you, and Docker for supporting modern DevOps practices.

Machine groups

If you do not find your favorite platform integrated into VSO as an extension, you can still use “standard” machine groups as a way of describing a group of addressable machines. These machine groups can then be used as part of various deployment and testing tasks in Build and Release workflows. Just think of machine groups as the “catch all platform” that is always available to you within VSO/TFS. This concept used to be called “Standard environments” in Lab Management.

.Net and Windows application deployment

We will enable you to deploy .Net applications (IIS, SQL server, Windows services) easily to machine groups defined in VSO/TFS, to pre-provisioned resource groups managed by Azure or other integrated platforms, or to dynamically provisioned resources on various platforms as part of the same automation. Support for DACPAC deployments, IIS web application configuration, or running remote Powershell scripts on the target servers will be provided. If you have already decided on third party deployment systems such as Octopus Deploy, we will have integrations with those as well through the extension model.

Java and Linux application deployment

We will enable you to deploy Tomcat applications easily to resources on various platforms. If you have written shell scripts to deploy your apps to Linux, you can use our xplat agent to run those scripts. You can also deploy your Linux applications using Maven scripts or Ant scripts. All of these tasks are readily available out-of-the-box in your Build and Release workflows. If you have already decided on deployment systems such as Chef or Docker, we will have integrations with those as well.

Environments

Using a release definition you can model a formal pipeline of test, staging, and production environments. Each of these environments in Release Management is a logical entity that describes how resources should be provisioned, how new builds should be deployed, and how tests should be run against the resources of that environment.

If you want less structured and self-service environments for your manual workflows, then we will soon enable you to create release definitions without a formal pipeline. Simply group all the related self-service environments of your team members into a release definition, and you are now setup to use the same steps to deploy the next build to your own environment as to the production environment.

Traceability

Want to know which release is deployed currently on your personal environment, or the status of testing of a release on various environments? You can easily track all the activity on environments or on releases.

Summary

To summarize, we are moving away from building platform-style features and focusing more on integrating those platforms into ALM workflows. Lab management is morphing into being an integral part of Build and Release Management – in the form of service endpoints, provisioning tasks, deployment tasks, and testing tasks. We are increasing our investment in creating new service endpoints and in adding provisioning, deployment, and testing tasks in order to help you easily consume platform features and to make it easy for you to add your own integrations.

FAQ

When can I expect to see these features in VSO and TFS?

Some of these will be in VSO later this year, and the rest during 2016. In particular, we are working to complete Azure integration, service endpoint extensibility, and Windows/.Net application deployments before the end of this year. You can expect to see the same features in TFS early next year. The extensibility will help us and our partners to deliver newer integrations throughout 2016.

I am a current Lab Management user in TFS 201*. What does this mean to me immediately and in the long run?

As you might have noticed, we are continuing to ship Lab Center as part of Microsoft Test Manager (MTM) 2015 and you can still run Build-Deploy-Test workflows in Visual Studio 2015 using XAML build templates. We will continue to ship these features as part of 2015 updates with fixes to any major issues that are reported by you. And, we will continue to support the older versions of these products as well as the 2015 version as per the EULA terms.

But, you can safely assume that we will not invest on new features in MTM Lab Center, nor in enhancing the XAML build templates. We will also not bring in any of the above features such as integrations with newer platforms, integration with newer versions of SCVMM, or integration with the new Build and Release Management services into MTM Lab Center. Similarly, we will not integrate the current Lab environments from MTM into the new Build and Release Management services. We will probably not ship another major version of MTM Lab Center.

Once we have integration with SCVMM (or Azure Stack) in the new Build and Release Management services, we will strongly recommend that you start using those features in the TFS or VSO web interface for your automated and manual ALM workflows as explained in this article. We will separately provide further migration guidance to help you re-create equivalent assets in the new services.

Share This Post

Comments

I have a question about the statement in the post,
"Moving forward, we will completely fold-in the lab management features into Build and Release Management, and would rather not talk about them as a separate set of features in TFS/VSO."
Why don't you want to talk about them as a separate set of features? Is it because there aren't any new features? Speaking of feature lists….is there a list of feature improvments to lab management for TFS 2015?
Also can you please make the timeline be more concrete…"later" seams so vague…
"Some of these will be in VSO later this year, and the rest later"
Also we've spent countless hours, and 10s of thousands of dollars on consultants to get our TFS instance setup with lab management capabilities…so it's really disappointing to hear that "we will not invest on new features in MTM Lab Center" also "We will probably not ship another major version of MTM Lab Center."
If you're not going to invest in MTM or Lab Management, then when the new updates come out, can you please add an option to "uninstall MTM" in the Visual Studio setup, if MTM won't be useful any more stop taking up space on my SSD.

@Peter
"Moving forward, we will …" I have improved this line in the blog post to state the intent more clearly. We are investing in the lab management features in the web interface of TFS and VSO. These features will appear (a) under the service endpoints tab, where you manage connections to different cloud providers and deployment services (b) under the Build hub, where you create build definitions that are equivalent to the XAML lab default template in the current version, and (c) under the Release hub, where you create environments with automated or semi-automated steps to deploy new builds and to run tests. On the whole, if we compare these features with what was available in the current lab management, it is a lot more. For instance, these new features allow you to provision new resources dynamically, provide integrations with additional cloud platforms, allow you to deploy latest builds (which was not possible in MTM), and provide better reporting on such environments. Similar to the rest of the services, we are just de-emphasizing our efforts in rich client.
"….is there a list of feature improvments to lab management for TFS 2015?" There are no new features in the Lab Center of MTM in 2015. The new features will be added to the web interface as described in this article.
"…later seams so vague…". In VSO, the features listed in the FAQ will be available before December 2015. I improved the sentence to be a bit more clearer. When we have more concrete dates, we will provide updates.
"…If you're not going to invest…" To clarify, we are not adding any new features only to MTM. We hope that you will find value in all the new features we are adding to the web interface. We will look at your suggestion to have an option to not install MTM as part of VS.

Thank you for the update. I'd like to ask if you plan to have similar functionalities like today the MTM has:
– Environment Viewer – a simple UI to manage multiple VMs easily and quickly switch between them, manage snapshots, mark in-use and check state for testing. Plus ability to navigate to it with one link, single sign-on.
– MTM Rich Test Client able to test on an environment, collect data and create a snapshot of environment plus link both the snapshot to WIT and environment details.

This post read like a giant "F" you to your customers who've spent years setting up, configuring, and deploying lab management.
This just sucks….you've left us as a .net shop up a crick…time to go look at other viable solutions.

+1 on having the option to not install MTM on my machine if it's no longer going to be updated for new versions of VS. "We will look at your suggestion to have an option to not install MTM as part of VS."

@kovacm, MTM will continue to support the current set of features. Another place where you can view all the VMs, and their current state, and manage their snapshots will be either in Azure portal (if you are using Azure), in Azure stack portal (if you are using new versions of SCVMM), or in VMware portal. These are web-based experiences for each of these and are far easier and richer now than what MTM can offer. Our focus will be on integrating what Azure, VMware, and other platforms already have into Build, Test, and Release flows in TFS. So, you can definitely expect to see links in build and Release summaries to navigate to machines that were deployed as part of those workflows, or to go into the corresponding management portals. We will continue to have data collection capabilities on the machines where you are running tests. Ability to take a snapshot of the environment is going to be present as a task in the Build and Release workflows.
@Kyle, @Gouri, sorry that you feel that way. What we have laid out here is far better than what we did for lab management in the past few years. There was literally no investment in that space from us. We will definitely continue supporting the current tools for a long time. Consider the web interface and the new work that we outlined to be additive on top of that. Please feel free to drop me a note at vijayma_at_microsoft_dot_com so that I can help you understand this direction better and why it is not a take back on lab management features that you are used to.

Vijay,
As a future user of Lab Management, I am very excited to see the future forthcoming changes that your team is planning on introducing. However I do have one question around your latest comment, which says "@kovacm, MTM will continue to support the current set of features. Another place where you can view all the VMs, and their current state, and manage their snapshots will be either in Azure portal (if you are using Azure), in Azure stack portal (if you are using new versions of SCVMM), or in VMware portal. These are web-based experiences for each of these and are far easier and richer now than what MTM can offer. Our focus will be on integrating what Azure, VMware, and other platforms already have into Build, Test, and Release flows in TFS. So, you can definitely expect to see links in build and Release summaries to navigate to machines that were deployed as part of those workflows, or to go into the corresponding management portals. We will continue to have data collection capabilities on the machines where you are running tests. Ability to take a snapshot of the environment is going to be present as a task in the Build and Release workflows"
Above you talk about the new version of SCVMM storing and managing the snapshots in the "Azure Stack Portal". Today we have an On Prem TFS instance and do not use Azure or VSO for business and security reasons. Unfortunately this is non-negotiable at this time for these reasons. Therefore my question is this; will I be able to view and manage my snapshots on a On-Prem capability with the new version of SCVMM, as can be done today, with SCVMM having a library instance that SCVMM accesses or some such capability as opposed to going to Azure with SCVMM?
Thanks Vijay for the very informative and insightful article post into the future of where Lab Management is going. It helps those of us that have not invested in Lab Management …… yet, to be able to plan better into the future for when we do want to implement it.

@Anthony
Azure stack portal or Windows Azure Pack is an on-premises portal hosted in Windows Server and provides the same experiences as those that exist in Azure. It is not tied to Azure and does not require you to have an Azure subscription. I am new to the Azure stack portal as well, and will come back with the answer to your specific question on managing snapshots and library through that portal. If not with that portal, SCVMM does have its own client and the notion of private clouds, which can be used to provide scoped self-service capabilities to users. As we make some more progress, we will come up with some more guidance on which tools to use for self-service with SCVMM.
From VSO and TFS, you will have tasks in Build and RM that will be able to create/start/stop/snapshot VMs managed in SCVMM.

We currently use XaML build definitions to specify test suites of automated test cases to be run in our "smoke test" lab environment. The results are available in MTM so that we can track our manual and automated test coverage together. Reviewing the new vNext build process it looks like the link between the automated tests and the MTM test cases is severed. In the vNext build process you run automated tests by pointing at automated test dll's and using priority and categories to control what is run. It is a shell for running tests from the mstest commandline. As far as I can see there is no way to feed the tests results back up to MTM and test cases.
Can you confirm this? Are there plans for vNext or the new release management process to integrate with MTM and automated test cases in a similar way that I have been doing with my XaML build definition?

@Stephen, yes, there are plans to have a task in Build and RM vNext to integrate with MTM test cases. This is a common ask from customers that have organized their tests in this manner. I am not sure about the timeline, but I don't think it is too far away.

@Stephen
As Vijay says, we are looking to enable this functionality in Build.vNext workflow. It will be great if we can connect so that I understand your scenario and needs more. If interested, please mail me at atinb at Microsoft dot com
Thanks!

@Bruce: Azure Dev/Test Labs is a separate project, and VSO will integrate with that. Azure Dev/Test Lab is a true lab management offering in Azure, which helps with managing a set of virtual machines for your team members. It allows a lab administrator to control the usage of these VMs through policies. The focus in VSO, moving forward, as outlined in this article, is on "consuming" such resources in Build, Deploy, and Test workflows in VSO. The resources themselves can be provisioned in Azure, Azure Dev/Test Labs, SCVMM, VMware, etc.

I have been trying out the Build-Deploy-Test workflow in the new VSO build agent system, but I find it lacking a really nice feature that was in the xaml-based workflow.
Test runs are not being correlated back to test suites and test plans, like they were in the xaml based lab management build. This made it possible to have charting on the dashboard (home page) to view the status of the test plan based on automated Lab builds. In the new workflow I cannot get this to work. I can only ascociate test runs with test configurations, not suites or plans.
Is this feature coming or is it being skipped?