Any opinion on branch per developer approach? This is the first time I have come across it someplace and feel that there’s just no reason too. I’ve been through the branching guidance and even use your article in MSDN magazine from earlier this year as reference. The development teams are offshore and not always experienced developers. I heard they want to use the developer branch as a code review area and then merge in later. Have you seen a branch by developer in practice or useful?

The short answer is if you see no reason (benefit) to adopting a specific branching pattern such as branch per developer, then don’t adopt that pattern. There is a cost to every branching strategy. But there is also a cost to not having a branching strategy that meets your needs.

Scenario One:

A team is comprised on an on-shore team and an off-shore team. During the on-shore team’s work day, they check code into an onshore development branch, which is a full child of Main. At night, they handoff their work to an off-shore team. The off-shore team is working in a separate development branch that is a full-child of the onshore team’s development The hand-off at night is a merge from the onshore branch to the offshore branch (always overwriting the target with the source if there is a conflict). In the morning, the offshore team does a handoff of their work back to the onshore team. Because the onshore team wants to review the changes done overnight by the offshore team, they will do a code review and selective merge from the offshore branch to the onshore branch.

Alternatively some teams uses shelvesets to support code reviews. The developer shelves changes and requests a code review. After a successful code review the developer will be permitted to check these changes into the development branch.

Scenario Two:

Consider this. You have a Scrum project. At the beginning of each Sprint, during the Sprint planning session, the team accepts a certain amount of work for the Sprint. Tasks (User Stories, Product Backlog Items, etc.) are moved from the Product Backlog to the Sprint Backlog. Work begins on the Sprint.

Each Sprint team member takes work from the Sprint backlog and begins work on that item. When done, the team member takes another item from the Sprint backlog. Each member of the Sprint team works in a similar fashion.

What happens if one developer, towards the middle or end of the Sprint takes on work from the Sprint backlog but is unable to complete it in time for the end of Sprint.

We will not, under the rules of Scrum, extend the Sprint until all items are done. We will also not release items (features) that are not done at the end of the Sprint. So what to do?

I presented some possible solutions at a recent Microsoft Engineering conference and again at an internal Technical Readiness conference. Some options:

· Have all developers work (check into) the same branch. At the end of each day each developer checks-in changes to the branch. At the end of the Sprint, separate the completed work from the unfinished work. Do this by cherry picking the changes for completed items as part of a merge from Development to Main.

· Have all developers work (check into) the same branch. Only check-in changes once a feature is complete. This means when a feature is complete there will be one mega check-in of all changes done in the course of implementing the particular feature. Instead of checking changes into the branch on a daily branch, shelve the changes at night (to get them onto the server) and then un-shelve them the following morning.

· Have each developer work on their own (private) branch. These private branches will be full-children of Main. At the end of each day, each developer checks in his/her changes to their private (developer) branch. When a feature is complete, the changes for that feature can be merged to Main and shared with the rest of the team. The developer begins work on the next item from the Sprint backlog. The developer can continue in the same (private) developer branch or start a new one to keep the second feature separate from the first, in the event that the second feature is not done at the end of the Sprint and needs to be held out of the release.

If you do not have a scenario similar to this, then you may not need or want separate developer branches.

]]>https://blogs.msdn.microsoft.com/billheys/2011/08/02/when-would-i-use-a-branch-for-developer-scenario/feed/3VS ALM Rangers Branching Guidance–Shared Code and Dependency Managementhttps://blogs.msdn.microsoft.com/billheys/2011/06/30/vs-alm-rangers-branching-guidanceshared-code-and-dependency-management/
https://blogs.msdn.microsoft.com/billheys/2011/06/30/vs-alm-rangers-branching-guidanceshared-code-and-dependency-management/#respondThu, 30 Jun 2011 08:09:09 +0000https://blogs.msdn.microsoft.com/billheys/2011/06/30/vs-alm-rangers-branching-guidanceshared-code-and-dependency-management/As we begin planning the next release of the branching guidance, a top priority is to address how to manage shared code dependencies (the Dependency Repository / Shared Code issue). The newly release Build Customization Guide (now available for download from Codeplex) begins to address this issue, primarily from a build perspective. For the Branching guidance, I would like to go beyond that offered in the Build Customization Guide. I would like to provide more details for each the recommended scenarios including screen shots from Visual Studio Source Code Explorer to show the proposed folder / branch structure for each scenario.

For the Branching guidance, I am thinking of using Enterprise Library 4.1 AND Enterprise Library 5.0 to show the following:

How to create / structure a Dependency Repository with different versions of assemblies to use with binary references in the Dependent Projects

How to create / structure a Dependency Repository with different versions of source code to use with project references in the Dependent Projects

The four scenarios described in the Build Customization Guide are

A centralized, dedicated Team Project (for example $/Dependencies). This is a good approach for sharing dependencies across multiple team projects.

A dependencies folder in the root of a product’s or project’s Team Project, for example, $/Contoso/Dependencies. This is a good approach for centralizing dependencies and promoting re-use.it will also suit the needs of most teams.

A dependencies folder under each dependent product or project, for example, $/Contoso/ProjectX/Dependencies).

A dependencies folder under a branch in the dependent project’s Team Project, for example,$/Contoso/ProjectX/Main/Dependencies).

I would like to go beyond the guidance in the Build Customization Guide, by providing guidance to help people choose from these four options. For example, what are the benefits and disadvantages of each approach. Why would I choose one over the other?

In addition I want to include, in the Branching guide, examples for each of these four options (or which ever options we choose to recommend) for creating references (one example for binary reference, one example for project reference).

Questions we need to answer:

Do we create references from the Dependent Project directly into the Dependency Repository?

Do we copy DLLs from the repository into a folder in the Dependent Project?

Do we branch from the Repository into the Dependent Projects?

If we branch, where is this target branch located in the Dependent Project?

Is the target branch located outside of the branching structure of the Dependent Project or is it (somehow) nested within the branching structure of the Dependent Project?

What are the issues associated with each approach?

How do I deal with not being able to create nested branches in VS 2010?

Do I allow only one-way merging (from the Repository to the Dependent projects)?

Do I allow bi-directional merging so that the Dependent Project can actually change the source of the shared code and merge these changes back to the repository?

If I don’t branch from the Repository what are my other options (copy? Reference directly?).

If the dependency repository is maintained outside the branching structure of the dependent project what other issues arise (relative folder depth for relative references).

When do I create project references to shared code vs. binary references. What are the pros and cons. Any gotchas?

The four scenarios described above are from the Dependency Management section of the Build Customization Guide. As much as possible, we should ensure that the guidance for sharing code and managing dependencies is consistent between the Branching guide and the Build Customization guide. This does not necessarily mean we have to incorporate all four of the Build guidance scenarios into the Branching guide. We also should not necessarily be confined to these four scenarios.

Some dependency management scenarios I envision discussing in the Branching guide include:

Create a *centralized* dependency repository, in its own Team Project, and show how other consumers (Dependent Team Projects) would create references to either DLLs (binaries) or Source Code (Project References) across Team Project boundaries. What are some of the issues (security is one).

Create a dependency repository *within* a Team Project shared by other consumers within the same Team Project. Have the consumers establish references from projects *within* a branch to either DLLs or Source Code in the repository *outside* the branch. What are some of the issues, such as how to avoid breaking relative references when branching and or building code on a build server.

Create a dependency repository in a folder beneath a branch. This simplifies workspace mapping issues perhaps and would be one way to address how to avoid breaking relative references.

For each of the scenarios we settle on, we should talk about the considerations for choosing one over another. Some of the considerations I can think of include:

The need to support multiple versions of the shared dependencies (DLLs or Source Code) (producers)

Frequent vs. infrequent updating of the dependencies (producers)

Number of consumers (dependent projects)

Team Project Organization (is the repository in its own Team Project – perhaps shared by multiple products, product groups, or teams, or is the repository within a Team Project – perhaps where the same team owns the shared code and the dependent projects, products, or product groups)

Workspace mapping considerations

When to use Project References vs. Binary References

How to propagate changes to the shared code to all of the consumers (Branching from producer to consumer, vs. Copying from producer to consumer, vs Relative References from consumer to producer across branch and / or Team Project boundaries)

]]>https://blogs.msdn.microsoft.com/billheys/2011/06/30/vs-alm-rangers-branching-guidanceshared-code-and-dependency-management/feed/0VS ALM Rangers Branching Guidance – Ideas for the next release.https://blogs.msdn.microsoft.com/billheys/2011/06/30/vs-alm-rangers-branching-guidance-ideas-for-the-next-release/
https://blogs.msdn.microsoft.com/billheys/2011/06/30/vs-alm-rangers-branching-guidance-ideas-for-the-next-release/#commentsThu, 30 Jun 2011 07:19:44 +0000https://blogs.msdn.microsoft.com/billheys/2011/06/30/vs-alm-rangers-branching-guidance-ideas-for-the-next-release/I think most would agree that the Rangers Team has already done a fantastic job with the Rangers Branching Guidance. This is not meant to be a self-serving comment, since the current branching guidance is the result of the hard work of many others in addition to myself. For the past couple of years I have had the opportunity to answer dozens of questions that have been posted to the Branching Guidance discussions forum. I have also worked with a variety of customers, helping them design and plan effective branching strategies. Through these efforts, I have come to believe we can make a number of valuable improvement and additions.

The approach taken in Branching Guidance II (2008) was to propose a series of branch plans of increasing complexity (Basic, Standard, Advanced). This approach has been well received as we have gotten considerable positive feedback. However the primary focus of the branch plans in the current guidance has been to help customers understand different options (from simple to complex) with respect to the branching for release and sustained engineering.

Each of the branch plans in the current guidance has just a single development branch. I recommend thinking about branching for development separately from branching for release and sustained engineering. Nevertheless, how teams develop software is sometimes tightly coupled to or influenced by how teams release software. For example, on a Scrum project, the team needs both a development branch plan and release branch plan that enables Sprint teams to work on multiple features during a Sprint (development), and makes it easy to release only those features that are done at the end of a Sprint.

A branching strategy should make it easy to defer unfinished features to a future Sprint. Even teams that do not practice pure Scrum may have agile or iterative methodologies where completed features are released at the end of each iteration while providing for certain features to started during one iteration but not released until they finished two or three iterations later.

As we contemplate updating the current branching guidance, I propose, taking the existing branching plans (basic, standard, advanced) and describe them as branching plans for release. Then I would propose adding a new set of branching plans for development, such as a basic or single-team development branch plan, feature team development, branching for Sprint development (Scrum), and branching for Iterative development.

As we add more content to the guidance we should view a complete branching strategy as a combination of branching structure + branching process. To that end I propose having much more information in the guidance describing how to manage branches and merges. Some examples:

Why do we recommend daily builds in the Main branch with frequent (daily?) merges from Main to development branches.

Why do we recommend not doing a daily merge from a development branch to the Main branch.

What are typical quality gates a development branch must pass before being allowed to integrate with other branches. I call this state ready to release or ready to share or ready to integrate. I think we need to add content to describe what we mean by ready in this context.

What are the recommended techniques for integrating two or more parallel development branches once they reach a state of ready to integrate? In the current guidance we take a firm stance against adding an integration layer to the development branch plan. In place of an integration layer, we need to better describe the process for integrating two or more parallel feature teams (sibling branches).

Why do we recommend not doing merges (forward integrations) from Main into a Release branch after the release branch is created?

What are the benefits, in a branching strategy, of automated testing, especially Build Verification Tests (BVTs) – and how can these can improve code velocity.

What other techniques for improving code velocity and code quality during development (such as using Static Code Analysis and Style Cop) should be considered as part of branching guidance/

We should also add more guidance on helping teams understand and manage the complexity of merges

Doing more frequent merges from a stable branch (Main) to less stable branches (Development). Doing merges only at specific milestones from less stable branches to more stable branched.

What are merge conflicts and how can they be avoided or made easier to deal with? There are a variety of causes of merge conflicts. Some of these happen at check-in, some happen during a merge. Helping teams gain a fundamental understanding of how merges work and what typical merge conflicts are may help teams improve their velocity.

I would like to add a section on third-party merge tools that may help teams deal more effectively with merge conflicts. There are several tools that do a better job presenting a three-way merge dialog, visualizing merge conflicts and making conflict resolution easier.

To summarize I want to improve the guidance in the area of branching process as well branching structure.

These are just some of the ideas that I would like the VS ALM Rangers team to address in the next release of branching guidance. Some may disagree. Some may have more important priorities. I welcome feedback, but more importantly I welcome contributors to this section of the guidance.

Next Steps:

I am looking to finalize an outline and gain consensus

Drill down and write the content according to the outline

We will be firing up a number of parallel efforts soon, so I am looking for contributors to this specific Epics or initiatives.

I welcome any thoughts people have for moving from *a list of volunteer contributors* to *final draft*

]]>https://blogs.msdn.microsoft.com/billheys/2011/06/30/vs-alm-rangers-branching-guidance-ideas-for-the-next-release/feed/4Renaming a branch with shelvesetshttps://blogs.msdn.microsoft.com/billheys/2011/05/23/renaming-a-branch-with-shelvesets/
https://blogs.msdn.microsoft.com/billheys/2011/05/23/renaming-a-branch-with-shelvesets/#respondMon, 23 May 2011 13:47:01 +0000https://blogs.msdn.microsoft.com/billheys/2011/05/23/renaming-a-branch-with-shelvesets/Caution: Renaming branches in TFS 2010 is a very bad idea unless you follow a very specific set of steps:http://blogs.msdn.com/b/chandrur/archive/2010/06/09/renaming-branches-in-tfs-2010.aspx.

In TFS 2010, Unshelve will follow pending renames but it will not follow renames committed between when the shelveset was created and the current workspace version.

Here are two cases.

Case 1:

In some workspace, an edit is made to $/Proj/Main/foo.txt and is shelved in a shelveset named “changes”.

In your workspace, you have a pending rename on the branch such that $/Proj/Main will become $/Proj/Main2010.

You try to unshelve the shelveset named “changes” into your workspace

In this case, the change on foo.txt will be unshelved in the local folder that corresponds to $/Proj/Main2010, since that rename is currently pending in your workspace.

Case 2:

In some workspace, an edit is made to $/Proj/Main/foo.txt and is shelved in a shelveset named “changes”.

In your workspace, you pend a rename on the branch such that $/Proj/Main will become $/Proj/Main2010.

You check in your changes to rename $/Proj/Main to $/Proj/Main2010.

You try to unshelve the shelveset named “changes”.

In this case, the change on foo.txt will be unshelved in the local folder that corresponds to $/Proj/Main and not the newly renamed branch.

In order to fix this, you can issue a “Get” for your workspace. Get will find conflicts that will force you to select the proper location for this change and allow you to move the unshelved changes under the renamed branch.

3 parallel efforts (two parallel development efforts, working on separate releases of the product and production support)

Production(version 2.0) with the need to issue hotfixes on then current production version

Development Team 1, working on the next minor (Feature) release (version 2.1). New minor releases every 4 weeks

Development Team 2, working on the next major release (version 3.0), planned for production in 1+ years

I am using the Advanced Branching Plan (with Concurrent Hotfix, Service Pack, and Release branches).

Requirements

We need to release hotfixes on the current production version (v2.0), without consulting with project teams which greatly slows things down)

We need to do feature releases in parallel with long running projects.

We do not want to merge these features directly into the development branch for the next major release (Development Team 2) This is seen as a serious project-risk.

We plan to release V2.1 sometime in the next quarter

We plan to release V3.0 sometime in the next two years

Questions from the customer:

How can we release (minor) features on our product when other project teams working on a long-running projects for the next major release see these features as added risk to there project/test plans.

Should I frequently merge RI my project changes into MAIN (even if project runs for 1+ years)

How should I create a new feature branch from MAIN while there are already several RI’s from other running DEV branches. (Branch by what ???) New features of our software always result in heavy discussions with other (isolated) project teams. While they see them as added risk and they always want a complete change-freeze while their projects run. Only hotfixes to production are allowed.

This is because new features will be integrated into their dev-branches because these will be live way before the project will go live.

We are using TFS 2010 and VS 2010

My suggestions:

Use a variation of the standard or advanced branch plan. Both of these provide for sustained engineering (hot fixes) on current release in production (vCurrent) while simultaneously allowing development to proceed on the next release (vNext).

As we point out, the main difference between the standard branch plan and the advanced branch plan is the added branch on the release side in the advanced plan to support Service Pack and Hot Fix sustained engineering to the current release in production.

On the development side, I recommend adopting a pattern such as this:

In the preceding branching plan, development branches are created as children of the Main branch. Although new minor releases of the product are released on a monthly basis, multiple development branches are required since the development effort for some releases must begin before the iteration begins.

For example, in the preceding plan, Dev Team 1 begins work on release v0.1 (which is released at the end of iteration 1. At the same time, Dev Team 2 begins work on release v0.2. Since they need more than four weeks to complete their work on v0.2, they begin development during iteration 1 but release their changes at the end of iteration 2. This branching plan, therefore supports multiple parallel, overlapping development efforts.

Key to success with this plan is to ensure that Main is used to stabilize only one release at a time. During iteration 1, Main is used to stabilize v0.1 (blue). During iteration 2, Main is used to stabilize v0.2 (red). During iteration 3, Main is used to stabilize v0.3 (green). It is important not to comingle changes by merging into Main from parallel development on different releases (major or minor).

Any development branch can (and probably should) accept regular merges from Main (forward integrations – FI) – perhaps on a daily basis. If the development team working on a major release chooses not to accept merges from Main, this is their choice. At some point, presumably they will want to integrate changes from minor releases (for example v2.1) into their ongoing development (v3.0). However, in your case you indicate the development team for version 3.0 (the next major release) would like to proceed as if there is a code freeze in effect. This may or may not be a reasonable constraint. Since, in your scenario, they will not be releasing their code to production for more than one year, it may be easier to accept changes more frequently rather than waiting until they finish development on the major release.

You will note, that in the preceding plan, new features are integrated with the current state of the code in Main by first merging (FI) Main to the feature branch (for example Dev Team1). Once this this integration builds successfully and is tested, it can be merged (RI) back Main. Changes from a development or feature branch should only be merged to Main when they reach a milestone and pass quality gates (ready to release or ready to share with other teams).

There are times, however, where it may be necessary to integrate changes (for example from multiple feature teams working on the same release) without merging them into Main. For this reason, it may be necessary to insert an integration layer between Main and the Feature Teams

In the preceding diagram, Dev Team1 has a single feature team and therefore does not need an integration layer. However, Dev Team2 has two feature teams working in parallel and need to integrate their changes without merging to Main. During iteration 1, Main is used for stabilizing v0.1. An integration branch is added between Main and the Feature Branches for Dev Team2. The feature teams for Dev Team2 can integrate their changes using this integration branch until Main opens up for v0.2 stabilization.

]]>https://blogs.msdn.microsoft.com/billheys/2011/05/23/isolating-and-integrating-parallel-feature-teams-working-on-separate-overlapping-releases/feed/0How do I create bug fixes for EVERY version of a product EVER releasedhttps://blogs.msdn.microsoft.com/billheys/2011/05/05/how-do-i-create-bug-fixes-for-every-version-of-a-product-ever-released/
https://blogs.msdn.microsoft.com/billheys/2011/05/05/how-do-i-create-bug-fixes-for-every-version-of-a-product-ever-released/#respondThu, 05 May 2011 12:23:58 +0000https://blogs.msdn.microsoft.com/billheys/2011/05/05/how-do-i-create-bug-fixes-for-every-version-of-a-product-ever-released/A customer asks the following question:

“We are building factory machine. New machines are usually delivered with the current stable software version. We don’t do regular updates because our customers prefer stability over new features. After reading the Rangers Branching Guide I have some questions. I understand the need for the main and the development branch, we had a similar thing using feature branches for feature development merging the finished feature into trunk/main.

But we must be able to create an independent bug fix for every version of our product we ever delivered. Customers are using our machines in a specific version perfectly optimized for their production process. If they find a bug or need a change they don’t want to get any other bug fix or feature we added in the meantime.

Should we create a label for every released/delivered version of the software. Whenever a customer requests a fix for instance for version v6.5, I create a "Release v6.5.1" branch from main, fix the bug, create a release/label for v6.5.1.0 and reverse integrate the fix into main. If this customer needs another fix I can do the next fix on the same branch (v6.5.1.1). If a different user with the same base version needs a different fix I can create a "Release v6.5.2" branch.

Can I achieve this with one of your strategies. I didn’t find a way, but maybe I’m missing something.”

What do you do when you no longer have version 6.5 in Main? Suppose you have released a new major version, Release 7.0?

One characteristic of labels in TFS to be aware of – they are not immutable. You can probably achieve your goals by labeling your code with each release (6.5, 6.5.1, etc.). But TFS will not prevent you from modifying the *contents* of a label after it is applied to a branch. This may cause problems with auditors who want the contents of release *locked down*. This is one reason we recommend creating a release branch for each release, and locking it down. Since you need to be able to apply bug fixes to any prior release, you would need perhaps two branches for each release. Branch Main -> Servicing and then branch Servicing -> Release each time you do a release.

Lets say you have just released version 6.5.0. You could branch Main to create a Servicing 6.5. Label the code Release 6.5.0, but also branch Servicing 6.5 to create Release 6.5.0 and lock it down.

Now you need to create a bug fix against Release 6.5.0 – apply this bug fix in the Servicing 6.5 branch and release it to your customer. ﻿﻿Label the code in Servicing 6.5 as Release 6.5.1 but also Branch Servicing 6.5.0 (with the hot fix) to Release 6.5.1 and lock it down.

Now if you need to apply a second bug fix to Release 6.5.1 make this fix in Servicing 6.5 (which now has the code for 6.5.1 as latest version). Make the fix in Servicing 6.5. Label the code in Servicing 6.5 as Release 6.5.2 but also branch Servicing 6.5 to create Release 6.5.2 and lock it down.

Each time you get a new bug fix for Release 6.5.x the fix would be made in Servicing 6.5 branch, but you need to decide which sub-version of the code to apply the fix to. It could be either 6.5.0, 6.5.1, 6.5.2. You should have a label you can roll back to if you do not want the bug fix against the latest version.

Later when you release version 7.0.0, you would branch Main to Servicing 7.0 and then branch Servicing 7.0 to Release 7.0.0 and lock it down.

If you are concerned about the overhead of creating a new lock-down branch for major Release and each minor hot-fix you could probably get by with only creating Release branches for the major releases and not each hot fix. Nevertheless, read my latest blog to gain a better understanding of the actual overhead involved in creating a new branch: ﻿﻿﻿﻿﻿http://www.tinyurl.com/tfsversioning.

Normally when you have a one-to-many relationship in a hierarchy (such as a branch hierarchy), you would make the *one* the parent and the *many* children of the parent. You might think you want to invert the relationship between the Release branch and the Servicing branch, since you have many possible bug fixes for a given release. But if you invert the relationship in this way: Branch Main to Release 6.5.0 and then branch Release 6.5.0 to Servicing 6.5.1 and later branch Release 6.5.0 to Servicing 6.5.2. But when you put the servicing branch(es) as a child of the Release branch, you cannot merge the bug fix back to Main without merging it into the Release branch first. You would no longer have a copy of Release 6.5.0 (as released) since you would have merged post-release bug fixes into it in order to get them all the way up to Main.

Recently I got a request from a customer to help them estimate the expected size of a proposed TFS 2010 environment, and to help them estimate a ten-year growth prediction. This customer was focusing narrowly on the overhead required to create a new branch. This customer was asking "how much database space is needed when creating a new branch?” With this information, this customer believed they could “know how many branch they can cut, and how much space they need to add on storage”

Some of their specific questions include:

When creating a new branch (for example branch MAIN -> DEV), what actually happens to the data?

Are all files from the source branch (MAIN) copied to the target branch (DEV)?

Is just metadata added to the target branch (DEV) when it is created (such that no file copies occur)

What is saved when a file is edited in this new branch (DEV) and changes are checked-in to the branch?

Is a new copy of the entire file containing the changes saved in the branch (DEV)?

Is only a delta of the changes made saved when the file is checked in to the branch (DEV)?

What happens to database size when a merge operation (reverse integration from DEV to MAIN) takes place?

If some changes are merged from a source branch (DEV) back to a target branch (MAIN), will the size of the target branch (MAIN) only increase to reflect the delta, or will a new copy of the changed file(s) be saved to the target branch (MAIN)?

Is there guidance for estimating database sizing when creating new branches?

What do we tell customers about planning branches regarding to server capacity (disk space, database size, and so on).

If TFS always saves deltas (diffs) when changes are checked-in (regardless if this is the result of creating a new branch, or editing file and checking in changes to the same-branch), isn’t it pretty time-consuming to get the latest version of a file that has been edited frequently and has lots of changesets.

To answer some of these questions it helps to have an understanding of how TFS 2010 stores changes to files. First I want to thank the TFS Product Group, in particular Justin Pinnix for contributing these technical details and giving me permission to blog on this.

TFS uses a “reverse delta process” to store changes. The best way to understand this process is to start by considering a single file. When a file is added or changed in TFS Source Control, it will be stored in one of three ways:

As a full, uncompressed file

As a zipped file (using Gzip)

As a delta (storing just the diffs between two versions of a file)

When a file is initially added to TFS Source Control, it will be stored using method 1 (full, uncompressed file) or 2 (compressed or zipped), whichever is appropriate based on the type and size of the file.

Next, a user checks out this file, makes changes (edits), and checks-in a new version of the file with these changes. When the check in happens, a new (second) full version of the file is saved to the database using the same compression rules as the first version. If both the current version and previous version of the file are less than a certain size (16MB), the previous version will be considered for “deltafication”, which is done by a background job. This background job computes a delta (between the current and previous version). If size of the resulting delta is smaller than size of the previous version of the file, the delta will replace the previous version of the file.

When you create a new branch (for example from MAIN -> DEV), a shallow copy of each of the files on the source branch (MAIN) is made. Following this, both branches refer to the same copy of each file (the file content), and only metadata is copied to the new branch (DEV). Thus, when a new branch is created, only metadata is copied. Only when the first real change is made in this new branch, is a new deltafication chain is started for changed file(s).

Following the branch operation, a user checks-out a file from the source branch (MAIN), makes changes (edits), and checks-in a new version of this file with these changes, The previous version of this file (which is now used by both the MAIN and DEV branches in this example) will be considered for deltafication.

However, a user checks-out a file from the target branch (DEV), makes changes, and checks-in a new version of the file into this branch (DEV), a new chain is started and no deltafication happens.

Next, if the user once again checks-out this same file from the target branch, makes changes, and checks-in a new version to this branch (DEV), then the previous version of the file in the target branch (DEV) will be considered a candidate for deltafication.

TFS will also compute deltas for binary files. It is difficult to predict database growth based on the sizes of these deltas. The results from deltafication vary greatly depending on what type of file it is, and the nature of the changes.

Another consideration is to understand that adding lines to a file are far cheaper than deleting lines from a file, because these adding lines to a file introduce no new content in a reverse delta scenario. As an example, start with the current version (version 1) of a file with the following content:

LineA LineB LineC

Next, check-out this file, update it to look like this, and check-in a new version (version 2):

LineA LineD LineB LineC

A traditional forward diff algorithm would produce a forward delta (from version 1 to version 2) of:

At line 2, insert the text “LineD”

However, TFS uses reverse deltas (from version 2 to version 1), so the delta would look like:

Delete line 2

The reverse delta is smaller, because it is just a line number and does not have to include any new content (text). The algorithm TFS uses is more complex, but it has the same basic property of adds to a file being cheaper than deletes from a file with reverse deltas.

The TFS deltafication (diff) algorithm makes heavy use of compression, so compressibility of the data makes a big difference. This is the same algorithm used for Windows Update, so Win32 EXEs and DLLs compress quite well with this algorithm.

When storing a full version of a file, TFS will attempt to GZip it if it is less than 2GB. If the file is larger than 2GB, or if the compressed version doesn’t make an improvement, then TFS stores the file uncompressed.

Shelvesets also use the same mechanism for storing their content. However, with shelvesets, no deltafication happens. Each shelveset gets a new copy of its file(s). (except in the case of a merge). When a shelveset is checked-in, a shallow copy occurs and the committed version of a file uses the same content as the shelveset copy of the file referenced. Deltafication will be started on the previous version of the file(s).

This deltafication mechanism applies to the database storage layer. Both the Application Tier (AT) and proxy (cache) servers cache files that go between the clients and the database. This cache saves full versions only, no deltas. If a previous version of a file becomes popular (frequently retrieved from the server), each request will not have to pay the cost of reconstituting it from deltas. Once the cache is populated with a full copy of a specific version of file, it can be retrieved as quickly as the current version is.

The diff algorithm itself is based on a compression algorithm. So, you can think of the deltas as compressed.

Note: The #1 space complaint the TFS team gets for TFS 2010 is not related to version control files at all, but rather is related to the size of test result attachments. Test result attachments are best cleaned up using the Attachment Cleaner Power Tool.

]]>https://blogs.msdn.microsoft.com/billheys/2011/05/05/how-tfs-stores-files-and-calculated-deltas-on-versioned-files/feed/5Learn TFS–The Visual Studio and Team Foundation “Resource Resource”https://blogs.msdn.microsoft.com/billheys/2011/04/09/learn-tfsthe-visual-studio-and-team-foundation-resource-resource/
https://blogs.msdn.microsoft.com/billheys/2011/04/09/learn-tfsthe-visual-studio-and-team-foundation-resource-resource/#respondSat, 09 Apr 2011 07:26:03 +0000https://blogs.msdn.microsoft.com/billheys/2011/04/09/learn-tfsthe-visual-studio-and-team-foundation-resource-resource/Learn TFS is subscription-based Team Foundation Server and Visual Studio ALM learning resource providing videos, articles, how-to guides, and source code to help your team better utilize Microsoft’s ALM toolset. You’ll find content and resources specifically tailored to meet the needs of your developers, testers, project managers, business analysts, and administrators to get the most out of your team and your investment.

One of my favorite features on this site is the daily “VS / TFS Links and Quick Hits” blog. Here you will find links to popular blogs and Web sites with lots of useful information and practical real-world guidance on using Visual Studio 2010 and Team Foundation Server 2010. Every once in a while you might even find one of my blog posts listed here

]]>https://blogs.msdn.microsoft.com/billheys/2011/04/09/learn-tfsthe-visual-studio-and-team-foundation-resource-resource/feed/0Where do I fix a production defect?https://blogs.msdn.microsoft.com/billheys/2011/03/22/where-do-i-fix-a-production-defect/
https://blogs.msdn.microsoft.com/billheys/2011/03/22/where-do-i-fix-a-production-defect/#commentsTue, 22 Mar 2011 13:37:27 +0000https://blogs.msdn.microsoft.com/billheys/2011/03/22/where-do-i-fix-a-production-defect/Question from a customer of the Rangers Branching Guidance (http://tfsbranchingguideiii.codeplex.com )

“My company has a major release every 3 months. In between major releases we have monthly maintenance releases. (Similar to the "Advanced Branching Plan")

Our monthly release usually is limited to high priority bugs based on customer complaints and business needs.

Scenario: Lets say there is a production bug reported by a customer shortly after a major release. It hasn’t been determined if this bug will be included in a hotfix, the next maintenance release, or the next major release. What branch should the developer make their changes?

We currently work bugs in our development branch if we don’t know if the fix will be part of the next maintenance release.

The problem is that this often leads to cherry picking changesets to separate fixed bugs from new development work.

We are thinking about creating a new branch called "Bugs" from the parent "Main". The thought was that this would help isolate sprint development work (new features) from production bug fixes to released code.

Main ———————————————– | | | Development Bugs Release

I am seeing the advantage of having production bug fixes in the Hotfix branch of the Release Vehicle, however I am still a little confused on how to handle multiple service packs.

So for example lets say we had a major release in February with SP1 planned for March and SP2 planned for April. Release branch is locked down and (RI) back to Main. Post February release there are 8 production bugs reported. Developers fix bugs in Hotfix branch.

After bugs have been completed in the Hotfix branch Management determines that: bugs 1, 2, 3 need to be in SP1 (March) bugs 4, 5, 7 need to be in SP2 (April) bugs 7, 8 will go with the next version

Is there a branching strategy that can encompass all this in the Release Vehicle?

My Response:

In the Rangers Branching Guidance, the Release branches have a specific relationship to each other. In each of the plans, the Release branch is locked down (made read-only) to capture each major and / or minor release shipped to a customer. Since the Release branch is read-only it must be at the lowest level in the release branch hierarchy (with Main at the top). If you are doing post-release servicing, then we suggest having a Servicing branch between Main and Release or even a Service Pack branch (as child of Main) and a Hotfix branch (as child of ServicePack) with Release as child of Hotfix (and then made read-only.

Your Bugs branch (as a child of Main corresponds to the HotFix or Servicing branch in the advanced or standard branch plan, respectively. The primary difference is that the Release branch is a child of the hotfix or servicing branch, rather than in a separate branching structure. This allows you to have multiple active release sets (consisting of Service Pack, Hotfix, Release branches) for each of the minor or major releases you need to support in parallel. The hotfix would be applied to the specific release the bug is found in, and then merged (RI) to Main and possibly into vNext development branches.

The branching plans in the Rangers branching guidance do, in fact, separate your sprint work (in a sprint development branch) from the Release and Post release bug fixing. It also allows Main to remain as stable as possible. It also means you won’t need to cherry pick changes (avoid cherry picking if at all possible).

Since the development branches are working on vNext work after vCurrent is released, I discourage you from fixing defects found in vCurrent (post-release) in the vNext development branch. After you release Sprint 1, you should fix defects (post-release) in Sprint 1 on the release side, and fix bugs (pre-release) in Sprint 2 on the development side (vNext).

Some have suggested making the HotFix branch a child of the Release branch. I never make a hotfix branch a child of release (for reasons stated above). To the contrary Release is a child of hotfix. At the time you create the release, the contents of hotfix and release are the same. Release is made read-only and Hotfix is available for making defect fixes against what was released.

The problem with inverting the structure, is you cannot move a hotfix to Main without going through Release, and doing so means you no longer have a copy of the code as released. If you don’t want a locked-down read-only release branch, then skip the hotfix branch altogether, make the hotfixes in the release branch directly (obviously you would not be able to make release read-only in this scenario).

Let’s use the following scenario:

In February, you release v1.0, that has been under development. While it was being developed, v1.0 was vNext. Once it is released, it now becomes vCurrent, and development begins on v2.0 (the new vNext)

So when you release vNext, you create (all at the same time) Service Pack branch for Release 1.0 (child of Main), Hotfix branch for Release 1.0 (child of Service Pack) and Release branch for Release 1.0 (made read-only)

Presumably you have some post-release fixes to production for vCurrent. You fix bugs 1,2, and three in the Hotfix branch. You decide to roll-up these fixes into a Service Pack.. so you Merge (Reverse Integrate) bugs 1,2, and 3 into the Service Pack branch for Release 1.0. After thorough testing you release this Service Pack to customers.

AT THIS POINT (March), you could choose to make a new branch for the Released Service Pack (for example, branch Service Pack 1.0 to Hotfix 1.0.1 and Release 1.0.1 and make Release 1.0.1 read-only.

Now you find more bugs to fix, so you fix these bugs in the hotfix branch (for wither Release 1.0 or Release 1.0.1 depending on whether the bug is in the original release (1.0), or the service pack (release 1.0.1)

Note that you do not need a new branch for the service pack … it continues, once Service pack 1 is shipped in March to be available for stabilizing Service pack 2.

bugs can be fixed in either the Hotfix branch for Release 1.0 or the Hotfix branch for Release 1.0.1 (SP1).

Now (April) you decide to ship SP2 (so you merge any changes in either hotfix branch below the Service Pack branch for Release 1.0.x up to the Service Pack branch (RI). You stabilize the bugs (4,5,6) and ship Service Pack 2.

Same process as in March, you make a new HotFix branch for this new Service Pack, and a new Release branch for Release 1.0.2 (SP2). Make the Release branch Read only.

At this point you have ONE Service Pack branch and three Hotfix branches (1.0, 1.0.1, and 1.0.2), and three Release branches (1.0, 1.0.1, and 1.0.2).

Bugs 7 and 8 get fixed in one of these three hotfix branches. They get merges (RI) to the Service Pack Branch and either stabilized for another SP (SP3) or merged again (RI) to Main where they are part of the next Major/Minor release.

From Main they get merged (FI) to the vNext development branches.

]]>https://blogs.msdn.microsoft.com/billheys/2011/03/22/where-do-i-fix-a-production-defect/feed/1My source and target files are different, but merge tells me there are no changes to mergehttps://blogs.msdn.microsoft.com/billheys/2011/03/16/my-source-and-target-files-are-different-but-merge-tells-me-there-are-no-changes-to-merge/
https://blogs.msdn.microsoft.com/billheys/2011/03/16/my-source-and-target-files-are-different-but-merge-tells-me-there-are-no-changes-to-merge/#respondWed, 16 Mar 2011 08:58:26 +0000https://blogs.msdn.microsoft.com/billheys/2011/03/16/my-source-and-target-files-are-different-but-merge-tells-me-there-are-no-changes-to-merge/When TFS does a merge, it bases the merge on prior merge history, not on the actual contents of the source and target files. There are many reasons why this is necessary, some of which will become apparent by reading this post. For example, a merge may result in a conflict (changes made to both the source and target files). In resolving the conflict you have many choices, including “Keep target version” (sometimes called “Accept Yours”) or “Take Source version” (sometimes called “Accept Theirs”). Sometimes in resolving a merge conflict, you take some changes from the source file and some from the target file. Depending on how you resolve a merge conflict, the source and target files could end up being very different. A subsequent merge will show “No changes to merge” even though content of the source and target files is different.

Is it possible that there was, at one time, a merge with the discard option performed (using the command line TF merge /discard).

When you use the discard switch, TFS will consider unmerged changeset(s) (changesets in the source branch that are not in the target branch) to now be merged, but it will not actually change the content in the target branch’s file.

Another possibility is that a merge conflict occurred on a prior merge, and the user selected “Keep target version” on the merge conflict resolution dialog. This will also update the history to indicate that the changeset was merged from source to target, but the actual change in the source file was not applied to the target file.

If there is a changeset that was previously merged (perhaps incorrectly) and you want to merge it again (for example, I want to merge changeset #123456 from $/Source/File.cs to $/Target/File.cs again), use the /force option with the tf merge command. The /force flag ignores merge history that indicates a particular changeset was previously merged from source to target (perhaps incorrectly), and merges the changeset again. You can do a /force merge for a single changeset or for a range of changesets. For just changeset #123456 the command would be: “tf merge /force $/Source/File.cs;C123456~123456 $/Target/file.cs”

There are some other diagnostic things you might do:

The Command Line TF Merges (note the plural), will show you the changesets from the source file that were previously merged to the target file (and the resulting changeset in the target file)