Soma received a few questions on his blog from folks who received the Release Candidate at the PDC about what works with what.The PDC was tricky because we wanted to have a set of bits that teams from Microsoft could standardize on AND we wanted to hand out the latest and greatest bits to let people try out the Release Candidate.We knew that these two worlds would collide, but decided that getting the latest build into everyone’s hands to find those last show stopper issues outweighed the confusion that might arise.

Jason pulled the questions out of the comments and answered them in Q&A style here. Thanks Jason!

Q. I got the VS RC1 bits at PDC…yea! But am really interested in getting the Biztalk 2006 Beta 1 working with it…which the doc’s say currently work with VS Beta2.

A:BizTalk 2006 Beta1 does in fact require the use of VS Whidbey Beta2.However, the exciting news is that BizTalk 2006 plans to release their Beta2 later this year.The BizTalk 2006 Beta2 will require VS Whidbey RTM!You can also find out more about the compatibilities of the PDC bits here: https://channel9.msdn.com/wiki/default.aspx/Channel9.PDCTheGoods

Q. Will there be updated versions of various .NET Framework 2.0 dependent products like WinFX Runtime Components which work with RC1 or do we have to wait for the final release of .NET Framework 2.0?

A: Each of the various technologies announced at the PDC that leverage the .NET Framework 2.0 have plans to provide updates of their technologies that work with Whidbey RTM.Specifically, BizTalk 2006 and WinFX Runtime Components should have externally available releases (Betas) by the end of the year.

Q. FAQ: How do I know which version of VSWhidbey to use with the technologies from Microsoft that also leverage the .NET Framework v2.0?And why do I have to know this?

A: Many of the technologies that leverage the .NET Framework v2.0 specify in their installers or readme documentation which release of the .NET Framework they require; be it Beta2, August CTP, or the September CTP (aka PDC build).

There have been some community efforts to help developers and other enthusiasts to better understand which builds of the .NET Framework v2.0, SQL Server 2005, VS 2005 all work together.You should check out the Channel9 site that can help detect which version you have installed and suggest combinations of technologies that work together: https://channel9.msdn.com/ctpmadness/Default.aspx.

So, why do you have to know all this?The simple answer is that while the .NET Framework v2.0 can be installed side-by-side with previous major releases of the .NET Framework (v1.1 and v1.0), it cannot be installed side-by-side with other interim releases of v2.0.In other words it’s not side-by-side with itself.

Additionally, the .NET Framework v2.0 has had some great innovation in it over above v1.1 (and v1.0).Some of this innovation has been iterative and included the feedback from you, our customers.These iterations have sometimes resulted in “breaking changes” that prevent applications that were written to work with .NET Framework v2.0 Beta2 to stop functioning on .NET Framework v2.0 RC1/RTM.

We’re putting together documentation that will help developers understand the ‘breaking changes’ between Beta2 and RTM.This will be available at the same time that .NET Framework v2.0 is released…RTM’d!!

As we begin to close down on Ask Mode for the Platform and Design Time teams we enter the final stage of the release, Whidbey RTM Escrow.

DevDiv Whidbey RTM Escrow– thephasebefore the completion of the RTM milestone where the product goes through a period of bake time. No additional work is planned for the Whidbey RTM ENU product during this period of time, “we are done”. All QA teams have completed their final test passes and have signed off on the product, including all Whidbey release criteria. The only work left to complete will be media verification.

Once the Platform teams meet the set of entry criteria required for entering escrow we will complete our final long-haul stress runs and turn our focus to locking down the Design Time teams. When the Design Time teams meet the set of entry criteria for escrow we will turn our focus towards meeting our escrow exit criteria. We will build what we expect to be our final build and QA will begin media verification.

Escrow Entry Criteria

0 recall class bugs left to be fixed for Whidbey RTM

QA teams have completed their Final Test Pass and signed off on the release

Final release criteria met/signed off

Escrow Exit Criteria

Media verification completed

No Bugs marked as Escrow Reset under investigation

What will people be doing during escrow?

Long-Haul Stress – we will continue to run Long Haul Stress to confirm that we have not encountered any new recall class bugs

App Building – we will continue to do App Building to confirm that we have not encountered any new recall class bugs

Media Verification – during the first week of Design Time escrow we will complete media sign-off

DevDiv shiproom will continue to meet daily while we are in real-time bug mode (24-hour triage), where all new escrow bugs filed must be triaged within 24 hours.

Investigating issues and regression testing any changes taken during escrow

Where is the triage bar? What would reset escrow? During escrow only issues deemed “recall class” by shiproom meet the bar and would reset escrow. This would have to be a serious enough issue requiring us to recall the media, such as an MSRC or GDR class security issue.

Would we restart the escrow clock? Yes, depending on where the fix is we will reset escrow based on QA’s expected Whidbey Response Time for verifying and regression testing a given fix. The expectation is that if we take a serious enough escrow resetting fix that we would need a maximum of 2 weeks from the completion of the fixed build to ship RTM.

During Whidbey Beta 1 we built a web application for reporting the division release criteria called DDMetrics. This tool served its purpose for Beta 1 and Beta 2. Now… it is time to take it to the next level and start using an incredibly awesome tool built by the Team Foundation Server team (dogfooding)!

Starting this week we will be using “Team Foundation Server work item tracking” for reporting on our Whidbey RTM release criteria. I just wanted to take the opportunity to say the more time I spend in Team Foundation Server the more excited I get about the awesome features/tools that we are building everyday in the Developer Division. We are very fortunate that we have such a great opportunity to thank the VSTF team by getting some real time usage.

Just some of the TFS Capabilities…

Users of DDMetrics during the previous milestones are going to be thrilled when they see the capabilities in Team Foundation Server… below I will just name a few that I think are very cool:

Customization – we were able to define our own WIT (Work Item Type) to collect only the data necessary, we don’t have the 60+ fields you have in Product Studio just because some info is required for other types of bugs. We only have what is necessary!

Querying – I already created some template queries for people to use that are grouped by AssignedTo, PU’s, or by criteria. In addition to queries based on all criteria that are marked as “Meeting”, “Not Meeting”, “On-Track”, “No Data”, etc. We had some of these capabilities with the old web app, but nothing like we do now. This is a 100 times better…

Excel and Project integration – ok this is cool… you can run a query for work items assigned to you, your PU, or as a criteria owner, open the results in Excel, make your modifications and publish. BAM!!! All your work items have been updated in Team Foundation Server.

Alerts – you want to receive email when work items change? Imagine if you are a single criterion owner and you want to know when we complete a sign-off. Well, now you can.

Much much more… I haven’t even touched on the reporting (SQL Reporting Services or simply through Excel), I will save that for next time or better yet my daily status mails!

The Team Foundation Server team is working hard refreshing their Dogfood servers. Once that completes I will be sending out instructions to the necessary folks for how to access the DDMetrics RTM project.

Most of you are already aware of the Visual Studio 2005 Community Technology Previews (CTP) and are familiar with the philosophy and benefits of the CTPs – you get working bits more frequently, great opportunity for feedback from you to the product teams (using Ladybug), potential of finding issues earlier than RTM, ability for you to verify requested fixes and lots of other good stuff. We are working on striking a balance between releasing CTPs frequently, the amount of time we spend verifying their quality and their effect on the main product schedule.

We have now released the July CTP, which is available to our MSDN subscribers. The SKUs already released in the July CTP includes the VS Pro, VS Std, VSS, VSTO, VS SDK and VS PPE. VSTS and VSTF will be released next week.

So, what really happens behind the scenes before you get the bits? Basically, a group of people work together and start cranking the CTP machine J Just to give you an idea, let me walk through my life as a PM on the release team before the release of Phase 1 of the Visual Studio 2005 July CTP, which was scheduled to release in 3 phases to account for the large number of SKUs:

Phase 1: VS SDK, VS PPE, VS STD due on July 8th

Phase 2: VS Pro, VSS, VSTO due on July 15th

Phase 3: VSTS, VSTF and VS SDK (compliant with the phase 3 bits) due in late July

June 28th– 29th:

Phase 1 work starts a week in advance. I started to scout the Main branch to select a build that would be released as our July CTP. Some people ask “A week early? Can we not just wait until July and then pick a build a day before it is released?” While that is the ideal case, we want to make sure that our customers get a minimum quality of bits. To do this, we need to allow sufficient time (as you will see in the rest of this blog) for install and uninstall testing as well as media verification (For example, virus scans need to be run so that you don’t get viruses as part of the download). To gauge quality, I look at each build to see if it had passed the Integration tests that we run daily on all builds. In this case, Murphy’s law prevailed – the Main lab, which was doing just fine the week before, suddenly had some build issues so I had to wait for those to be resolved and wait for a new build to be generated.

June 29th– July 1st:

– Selected a build as the CTP build since it had passed all the Integration tests.

– Created a save request for the build as new daily builds routinely replace older builds.

– Discussed SQL Server Express dependency with the Yukon team to determine the build of SSE to ship with the July CTP.

July 2nd:

Came into work (thanks to an issue with my smart card not allowing me to VPN into the corporate network) to request actual media to be burned for the CTP. This will ensure that the requests are seen by whoever first walks into the burn lab on Tuesday. Media needs to be burned for purposes of QA who test whether the images on the bits (actually being posted) match the bits of the CTP build on the build drop server.

July 5th:

10:00 AM – So, our detection logic is causing SQL Server Express installation to look like it failed (as the post-install check looks for the version the build was generated with). Decide to push this into Known Issues that accompany the download as the install actually succeeds but only looks like it has failed. Also, its too late now as the media is being burned as we speak.

July 6th:

-Quickly hand over all the media to the test team and create media verification requests for all the SKUs.

-4:00 PM – Test verification finds that the VS Pro un-install fails! After remembering to breathe, I find the Setup devs – Is there a fix? Can we investigate and fix it fast enough to turn around and ship this thing? So, I move the Pro SKU to ship with Drop 2 so that we can get the fix into the build.

– Request MSDN to stage the bits so that the bits will be available when we want to go live as this step usually takes 2-3 days for all the bits to propagate.

– Created a request to post the public symbols on the Internet server so that customers can debug their applications

July 7th:

Create internal CTP site so that folks have one place to view everything about CTPs and their schedule. Provide MSDN with all the documentation they need and the marketing blurb and give them the go-ahead to post the bits.

July 8th:

The bits are live on MSDN! Drop 1 goes out to our customers right on schedule and I periodically check MSDN to ensure that the bits have been replicated. But, what is this – the content looks a little different than what I sent out. Where are the Known issues? I picked up the phone and called the MSDN contact and turns out the content was indeed different. A quick change and a couple of emails later, the content looks good. I then send out a status email to the whole Division informing them of the same. Start to work on Phase 2………

Known issues (July CTP – Drop 1 and 2):

While we haven’t released an official “Known Issues” document and these are also published as Notes in the download location, I wanted to post these here:

1) At the end of the Visual Studio 2005 installation, it might indicate that SQL Server Express is not installed, even though SQL Server Express was installed successfully. Look at Add/Remove Programs to confirm that the SQL Server Express has been installed.

2) Installing on Win2k3 gold will fail until you install MSI 3.1

As you can see, there is still a long way to the realization of our vision where we are releasing builds on a daily basis. We are constantly thinking about ways in which we can streamline these processes and automate as much as of them as we can. But, we are constantly improving on releasing bits “over the wall” to you in a predictable, consistent manner.

On Monday (6/20) the Platform teams (those teams shipping in the .NET Framework Redist) entered DevDiv Tell mode, I see this as a very exciting time of the product cycle!It is one where you can see the light at the end of the tunnel, while at the same time every decision/bug fix gets greater scrutiny as we approach shipping a great product with Whidbey RTM.Tell mode is the first of four stages at the end of the RTM milestone (below I have broken out each stage).Each stage has a higher triage/bug bar than the previous stage as well as increased process and QA coverage in an effort to prevent regressions.

Stage 1: DevDiv Tell Mode:At the division level we will run a daily DevDiv shiproom.Tell mode is going to be 4 – 6 weeks.The focus of tell mode will be in establishing the triage/bug bar, understanding team’s bug trajectory, and moving teams towards a single branch.During shiproom we will be using the triage/bug bar as a guide, which we will continue to refine as we progress through Tell/Ask mode.Some of the data we will discuss at DevDiv shiproom:

Examples of approved/rejected (including tough calls and/or border line bugs) bugs.The focus here centers on the customer scenario and the fix in order to help level set the triage bar for teams across the division.

Have there been any major regressions from previous milestone or from Everett?

Team’s trajectory

RTM Active bugs

Bug fix rate (how many bugs were approved for checkin today)

Incoming/resolved bug rates

Weekly step down goals

Bugs resolved but not yet closed

Stage 2: DevDiv Ask Mode:DevDiv shiproom will continue to meet daily.During Ask mode we will further define and apply an even higher bar for the bug fixes we take into Whidbey RTM. All checkins to Whidbey RTM must be approved by DevDiv shiproom. Stage 3: DevDiv Endgame: This is where the bar is raised to recall class bugs only.Stage 4: DevDiv Escrow: This is a period of bake time, there should not be any Whidbey work or QA planned during this period of time.To be exact, “we are done with the product” and QA has signed off on the release

As we reach Ask mode I will provide more details on how we are progressing and the focus for the other 3 stages.Thanks!-Mike

blog that we’re heading towards a first step in the “Final Security Review” for Whidbey. A couple of weeks ago, our central security team finished a 9 week long Penetration Testing (pen-testing) effort of Whidbey. They had 8 people working just on our product. We began by doing “education” sessions for the security testing team, spending 1-3 hours on each area of the product. Based on these sessions and their expertise and experience, the testing team chose a number of areas to focus on. With a product as large as ours, it was impossible to cover everything. Remember that this is somewhat of a safety-net review. We have done extensive training and invested a lot of time in testing and reviewing our product ourselves.

During those 9 weeks, the test team sent out a report each week with the areas they have covered, and the issues they logged. They defined severities for issues they logged, 1 being “Critical” and 4 being “Low”. Each issue has a description of their concern, and was assigned to the owner team. The agreement was that if they hit multiple critical issues in a component, they stop their testing and require that component to re-confirm their security work. The good news is that this didn’t happen. Not only that, we had no critical issues logged at all!! Now, obviously, this is not a full proof guarantee, but it’s another confirmation that we’ve done the right things.

Overall, they logged 13 “important” issues, 8 “moderate” issues and 9 “low” issues. Right now, the teams are analyzing the issues and forming their recommendations. But again, I’m really happy with this result. On a product the size and complexity as ours, these are really good results.

Our next step is to apply for the SWI Final Security Review signoff. This is being done through a sort of documentation tool. I will use the tool to document that he have met the requirements of

SDL (and a few others that we imposed on ourselves). The documentation involves supplying the team with “proof” like threat models, results of tools run on our product etc. One thing to remember is that throughout the engagement with SWI, we had a member of their team be a virtual member of our security group. This helped us be always in sync with the requirements and make sure we don’t miss any requirements. Right now, I fully expect us to pass with flying colors.

In parallel to this sign off, I’m working on publishing our release criteria for security. For the most part, it is the same as our Beta2 criteria. Since Beta2 allowed customers to “

go-live” with it, we could not afford to hold off any security work for post that release. We do, however, are constantly improving our tools, and from time to time we identify new potential attacks that we want to ensure we are not vulnerable to. So I have a couple of tools (FxCop and PreFast) that we’ve improved that I need to set the bar for. There were also another three requirements that our SWI buddy sent us that we have to factor in.

Another effort that I’m working on separately is getting our division to establish a dedicated security team. Basically, replicating a little SWI of our own. If we had dedicated pen-testers, we could go into a rotation model and ensure coverage for the whole product. We could also have them participate in the early phases of design to provide expert advice on security. I’m getting support for this, now we just need to find the right people.

I think my next blog on this will be when we have FSR sign off. I can tell you all about how we managed the late-coming changes and what lessons we’ve learned.

One of the things we are doing in Whidbey Beta 2 is providing a broad Go-Live License agreement. Some of you may remember we provided an ASP.NET Go-Live License with the Beta 2 release of the .NET Framework 1.0, from an old blog post by Scott Guthrie. We view this as a terrific way to help enable our customers to get Whidbey now. This has already been a long product cycle, and our customers don’t want to wait any longer to get their hands on Whidbey. With the broad Go-Live we are giving customers that opportunity!

SQL Server 2005 Express Edition April CTP (a complete list of products will be included in the Go-Live License).

What does Go-Live mean? It basically breaks down like this…

Internal Deployment, a customer can build an application based on Whidbey Beta 2 and deploy that application in an internal production environment

Web Applications, a customer can create a web application targeting Whidbey Beta 2 for the purposes of hosting that application accessible by a third party

ISV deployment, an ISV can build an application with Whidbey targeting the .NET Fx redist, J# redist, the .NET Compact Fx, or SQL Server 2005 Express Edition (important note, we are not giving ISVs redist rights as part of their applications). In order for a customer to use that ISVs application they will need to agree to the Go-Live License agreement and get the redistributable from Microsoft.

Does Go-Live mean I have redistributable rights? No,as part of the broad Go-Livewe are not providing external redistributable rights in Whidbey Beta 2. Although, an ISV can create an application targeting… say the .NET Framework 2.0 Redistributable…, provide that application to a customer, and inform their customer that they need to go to Microsoft to accept the Go-Live License and download the required redistributable in order to be able to run that application.Will I get product support? No, Microsoft will not be providing support for Whidbey Beta 2. Although, informal community support is available through online forums, such as newsgroups, blogs, etc. A good resource for this information is http://msdn.microsoft.com/community/.Will I get servicing support? No, Microsoft will not be providing any patches, updates or other fixes.When/how do I Sign up for the Go-Live License? The Go-Live agreement is available now: http://lab.msdn.microsoft.com/vs2005/golive/.

Last I blog’d I talked to the Product Metrics/Release Criteria… and now we’ve released Whidbey Beta 2! It has been a long but satisfying road… As you can see from Chris blog below “In the home stretch…” he mentioned stress was going to be a wild card. He was right, which is why Beta 2 took just a little longer than expected.

Over the final few months of Beta 2 we met on a daily basis in shiproom to discuss release criteria, bugs, fixes, stress, builds… With Beta 2 having a “go live” license there was no way we wanted to release Beta 2 before it’s time, before meeting our quality goals. I think that was the right choice and I hope are customers agree!

As we roll from Beta 2 I am in the middle of reviewing our Whidbey RTM release criteria. I will again be holding reviews with all the release criteria owners. The main topics this time around will be about how we did in Beta 2, defining any necessary changes, setting checkpoints, and building a great quality RTM product!

]]>https://blogs.msdn.microsoft.com/release_team/2005/04/18/whidbey-beta-2-product-metricsrelease-criteria-sign-off/feed/0In the home stretch….https://blogs.msdn.microsoft.com/release_team/2005/03/15/in-the-home-stretch/
https://blogs.msdn.microsoft.com/release_team/2005/03/15/in-the-home-stretch/#commentsTue, 15 Mar 2005 13:30:00 +0000https://blogs.msdn.microsoft.com/release_team/2005/03/15/in-the-home-stretch/Wow! It’s hard to believe but we’ve completed 5 weeks of “Ask Mode”. Every day for the past 5 weeks, representatives of each product team in the Division have met daily for 2 to 3 hours reviewing each change that is going into the product. We look first at the customer scenario that is being fixed, we look at why the bug happens (regression, test hole, coding error, etc.), and then we look at the source code changes. It feels like we’ve read every line of source code (Visual C#, Visual Basic .NET, C/C++/MC++, and even a little Assembly for fun!) across the CLR, Framework, and Visual Studio. The teams have done a great job keeping the quality of the Beta 2 tree high and we’ve had excellent build just about every day for the past 5 weeks.

We’ve got about 3 more weeks in March and less than 400 Beta 2 bugs to manage. For a product this large, a few hundred bugs is a very small number if you consider 30+ teams and each investigating (but not necessarily fixing) around 10 apiece. It is all about maintaining stability for Beta 2 right now and driving our Stress programs to hit their criteria. We are moving into “recall class” only bug fixing this week, except for Stress, which we want to continue to take fixes for. Once we do the final build we will need a little time to produce media and verify it across the various SKUs, formats (DVDs, downloads, etc.) and platforms.

Overall, I feel good about our ability to get Beta 2 out by the end of the month, with Stress being the “wild card”. At the same time, my Inbox is full of questions about the InfoWorld article last week on the “slip” of Whidbey Beta 2. Unfortunately, it looks like the Microsoft person who made the comments about Whidbey was misquoted. He stated that Beta 2 was on track for the end of March, with general availability in early April, which is completely accurate. Once you produce the media, it then takes time to duplicate it and get it out to customers, which means the first week or so of April. The Express SKUs and .NET Framework redist will be generally available within a day or so of signing off on Beta 2.

Again, things are looking up, but we’re still not hitting our Stress criteria yet, which we will hold the release for until we do. More updates the closer we get to shipping Beta 2…

]]>https://blogs.msdn.microsoft.com/release_team/2005/03/15/in-the-home-stretch/feed/18Starting Ask Mode!https://blogs.msdn.microsoft.com/release_team/2005/02/07/starting-ask-mode/
https://blogs.msdn.microsoft.com/release_team/2005/02/07/starting-ask-mode/#commentsMon, 07 Feb 2005 17:56:00 +0000https://blogs.msdn.microsoft.com/release_team/2005/02/07/starting-ask-mode/Check off another mini-milestone for Whidbey! Beta 2 Tell Mode is over, and we started “Ask Mode” today. Over the course of the last week, we finished up a round of what we called “aggressive” integrations of the source code branches and are currently in the process of moving to a single Beta 2 branch (we had a tremendous group of dedicated developers in the build lab all weekend). Once all the labs are up into the Beta 2 branch, we begin the stabilization process there and begin fixing bugs in earnest on the live branches.

To recap our Beta 2 schedule (based on Fridays):

ZBB: 11/19/2004

Security Push: 11/15/2004 through 01/14/2005

Tell Mode: 01/17/2005 through 02/04/2005

Ask Mode: 02/07/2005 … until we meet our release criteria, which defines all of the various quality measures for Beta 2.

To be exact, we finished Tell Mode on Friday, but have two teams that need through Wednesday of this week to reduce their bug backlog a little more. So, everyone went into Ask Mode on today except these two teams. We’ll take one last integration of their sources for Beta 2 on Wednesday evening, and then they will be in Ask Mode too.

So starting today, every bug that a team wants to fix for Beta 2 needs to be brought to ship room for approval. The first question asked is “what is the customer scenario?” and if we decide that the benefits sufficiently outweigh the risks, we’ll ask the team to go and check in the fix. Otherwise, they’ll move the bug to the next milestone and fix it in the live trees. We’ll probably batch up a few fixes early this week as we get the Beta 2 branch building. We typically lock the trees from any check-ins so we can stabilize off a non-moving base. Once we open the trees, we’ll take all the approved fixes into Beta 2 and begin building every night. My expectation is that we’ll have solid Beta 2 builds that we can hand off to QA by Thursday or Friday of this week.