If you use a principle-based approach, you can get rid of classes of security issues. SQL injection, cross-site scripting and other flavors of input injection attacks are possible because of some bad practices. Here's a few of the bad practices:

This is a follow up to my post, Manage Energy, Not Time. A few folks have asked me how I figure out energy drains and catalysts.

For me, clarity came when I broke it down into:

Tasks

People

On the task side ...This hit home for me when one of the instructors gave some example scenarios:

You have to analyze a few 1000 rows of data in a spreadsheet

You have to give a last-minute presentation for a few thousand people in an hour

You have a whiteboarding session to design a product

You have to code a 1000 lines to solve an important problem

He asked, "how do you feel?" He said some people will have "energy" for some of these. Others won't. Some people will be excited by the chance to drill into data and cells. He said others will be excited by painting the broader strokes. He then gave more examples, such as, the irony of how you might have the energy to go skiing, but not to go to the movies.

The point he was making was that energy was relative and that you should be aware of what gives you energy or takes it away.

On the people side ...I pay more attention to people now in terms catalysts and drains. With some individuals, I'm impressed at their ability to sap energy. (I can almost hear Gauntlet in the background ..."Your life force is running out ..."). With other individuals, they are clearly catalysts, giving me energy to move mountains.

It's interesting for me now to think of both people and tasks in terms of catalysts and drains. Now I consciously spend more time with catalysts, and less time with drains, and I enjoy the results.

In the software industry, "scenario" usually means one of the following:1. Same as a use case2. Path through a use case3. Instance of a use case

#3 is generally preferred because it provides a testable instance with specific results.

Around Microsoft, we use "scenarios" quite a bit ...1. At customer events, it's common to ask, "What's your scenario". This is another way of asking, "what's your context?" and "what are you trying to accomplish?"2. In specs, scenarios up front set the context for the feature descriptions.3. Marketing teams often use scenarios to encapsulate and communicate key customer pain/problems.4. Testing teams often use scenarios for test cases.

At the end of the day, what I think is important about scenarios is they help keep things grounded, tangible and human. I like them because they can stretch to fit, from fine-grained activities to large-scale, end-to-end outcomes.

One of the most effective approaches I've found for chunking up a project for incremental value is using a Scenario and Feature Matrix.

A Scenario and Feature Matrix organizes scenarios and features into a simple view. The scenarios are your rows. The features are your columns. You list your scenarios in order of "MUST", "SHOULD", and "COULD" (or Pri 1, 2, and 3) .. through vNext. You list your features by cross-cutting and vertical. By cross-cutting, I mean that feature applies to multiple scenarios. By vertical, I mean that feature applies to just one scenario. It helps to think of scenarios in this case as goals customers achieve. It helps to think of the features as chunks of value that support the scenario. The features are a bridge between the customer's scenario and the developer's work. You can make this frame on a whiteboard before baking into slides or docs.

You now have a simple frame where you can see your baseline release, your "cuttable" scenarios, and your dependencies. You can quickly analyze some basic questions:

Do you have a good baseline set of scenarios?

Do you have an incremental story?

Do you have cuttable scenarios?

Can you cut a feature without cutting value for this release?

Because it's visual, it's an easy tool to get the team on board and communicate in terms of value, before getting mired in detail. When you get mired in detail, as you figure out features and dependencies, you can ground yourself back in the scenarios.

From what I've seen over time, most projects can't cut scope without messing up quality, because they weren't designed to. Cutting the leg off your table doesn't help save time or quality, it just makes a bad table. If you didn't have enough time or resources to make four legs should you have started? Should you build the four legs first and get the table standing, before you add that extra widget?

A Scenario and Feature Matrix makes analyzing and communicating these problems simpler because you create a visual strawman. Anytime, you can quickly bring more eyes to the table, it helps. I also like to think of this as "Axiomatic" Project Management at heart because I used simplified axiomatic design principles for the approach. If you're starting a new project, challenge yourself by asking if you can incrementally deliver value and if you can cut chunks of work without ruining your deliverable (or your team), and see if a Scenario and Feature Matrix doesn't help.

I think understanding the cost of NOT doing it is important because it gets you thinking about risk and impact. This sets the stage for an informed business case for security. While your business case mileage may vary, you'll get further with it, than without it.

Manage energy, not time, to get more things done ... This concept really resonates with me. I also like it because it can be counter intuitive or non-obvious.

One way to try and get more things done is to, jam more in your schedule. Yuck! Unfortunately, that's a fairly common practice.

I actually have lots of practices for managing time (outcome-based work breakdown structures, managing outcomes vs. activities, prioritizing outcomes based on usage and value, avoiding over-managing minutia, using outcome-based agendas for meetings, distinguishing getting results vs. building connections in meetings, using time-boxes to deliver incremental results in projects, "zero-mail in the inbox" practice … etc.) While I'm always open to new time management practices, I think I was getting diminishing returns from yet more time management techniques.

So stepping back, here's the situation … I was using a full arsenal of time management techniques, I was known for getting results, and yet I wanted to reach the next level. What happened next was, I noticed a common thread among a few very different trainings and books around leadership and results. Energy was a recurring theme.

Of course, then it made total sense (the beauty of 20/20 hindsight!). We've all had that great hour of brilliance or that unproductive work week. I did a reality check against several past projects. It was easy for me to see the connection of energy and results, when all else was equal. The problem was, I didn't have an arsenal of practices for managing energy. It turns out, I didn't really need to. Simply by knowing what drains me or catalyzes me helped a lot.

Now that I've been aware of this underlying concept for a while, I have learned a few practices along the way. One practice I use is I explicitly ask the team when and how often do they want to deliver customer results (i.e. how often do they want to see the fruits of their effort?). I balance this with capability, customer demand, project constraints and a bunch of other drivers, but the fact that I explicitly try to leverage energy and rhythm, helps crank the energy up a notch (and, as a bonus, results).

I found a way to explore more and churn less on incubation (i.e. R&D) projects. It helps to think of your project experiments and key risks in terms of these three categories and in this order:1. user experience2. technical feasibility3. business value

Sequence matters. If you don't get the user experience right first, who cares if it's technically feasible? Once you get the user experience right, meaning customers get value, the business value will follow.

Here's how I learned this the hard way ...

My project was time-boxed and budget constrained. To keep our stakeholders happy, my strategy was to deliver incremental value. This translated to short ship cycles to test with customers. We used a rhythm of shipping every two weeks. This let us track whether we were trending towards or away from the right solutions.

While this was a relatively short feedback cycle, it wasn't actually efficient. Most of our prototyping was around exploring user experiences, although we didn't know this at the time. We were focused on shipping prioritized customer scenarios and features. Delivering these scenarios and features, mixed exploring user experience, tech feasibility and business value. It's not a bad mix -- it just wasn't the most efficient.

Necessity is the mother of invention. When we weren't "learning' at the pace we expected, we had to find a better way. We moved to rapid prototyping user experience with slideware and walkthroughs. This meant faster feedback and less do-overs than our software prototypes. It also meant, in our software prototypes, we would consciously and explicitly focus on technical feasibility

User experience was the real challenge and the most value. Spending a week to build a software prototype to test technical feasibility and identify engineering risks makes sense. Spending a week to build a software prototype to test user experience, sucks. In other words, what previously took a week or more to build out and test (the user experience), we could now do in a few hours.

In hindsight, it's easy to see that incubation was about user experience, tech feasibility and business value, even though I didn't realize it at the time. It's also easy to see now that the dominant challenge was usually user experience.

The moral of the story isn't that you can use slideware for all your user experience testing. Instead, the lesson I would pass along is be aware of whether you are really testing user experience, tech feasibility or business value. By knowing which category you're exploring, you can then pick the right approach.

When I need to quickly analyze a product and give actionable feeback, I use scenario evaluations. Scenario evaluations are basically an organized set of scenarios and criteria I use to test and evaluate against. It's a pretty generic approach so you can tailor it for your situation. Here's an example of the frame I used to evaluate the usage of Code Analysis (FX Cop) in some security usage scenarios:

Scenario Evaluation MatrixDevelopment life cycle

Scenario: Dev lead integrates FX Cop in build process.

Scenario: Dev lead integrates FX Cop in design process.

Scenario: Developer uses FX Cop in their development process.

Scenario: Tester integrates FX Cop in testing process.

Scenario: Developer integrates FX Cop in deployment process.

Scenario: Dev lead creates a new FX Cop rule to support custom policies.

Scenario: Input data is constrained and validated for type, length, format, and range.

Scenario: Identify output sent to untrusted sources that are not encoded fields

Sensitive Data

Scenario: Check secrets are not hard coded

Scenario: Check plain text secrets are not stored in memory for extended periods of time

Scenario: Check sensitive data is not serialized.

... etc.

In this case, I organized the scenarios by life cycle, app type, and security categories. This makes a pretty simple table. Explicitly listing the scenarios out helps see where the solution fits in and where it does not, as well as identify opportunities. A key aspect for effective scenario evaluation is finding the right matrix of scenarios. For this exercise, some of the scenarios are focused on the user experience of using the tool, while others are focused on how well the tool addresses recommendations. What's not shown here is that I also list personas and priorities next to each scenario, which are also extremely helpful for scoping.

Criteria

What becomes interesting is when I applied criteria to the scenarios above. For example:

Recommended practice compliance

Implementation complexity

Quality of documentation/code

Developer competence

Time to implement

I then walked the scenarios, testing and evaluating against the criteria. This produced a nicely organized set of actionable feedback against how well the solution is working (or not). I think part of today's product development challenge isn't a lack of feedback, but rather a lack of actionable feedback that's organized and prioritized.

The beauty of this approach is that you can use this to evaluate your own solutions as well as others. If you're evaluating somebody else's solution, this actually helps quite a bit because you can avoid making it personal and argue the data.

The other beauty is that you can scale this approach along your product line. Create the frames that organize the tests and "outsource" the execution of the scenario evaluations to people you trust.

I've seen variations of this approach scale down to customer applications and scale up to full-blown platform evaluations for analysts. Personally, I've used it mostly for performance and security evaluations of various technologies and it helps me quickly find holes I might otherwise miss and it helps me communicate what I find.

We faced a lot of user experience design issues early in our R&D project. For example ....

how to filter a large picklist of items

how to optimize views based on type of item

how to integrate social software features (tagging, rating, ...)

Initially, we did a bunch of whiteboard modeling, talk-throughs, and prototyping. The problem was the prototypes weren't efficient. I had a distributed team so it was tough to paint a good picture of the prototype, even when we all agreed to the scenarios and requirements. The other problem was customer reviews were tough because it was easy to rat-hole or get distracted by partial implementations. The worst case was when we would finish a prototype and it would be a do-over.

This radically improved customer verification of the user experience and kept our dev team building out the right experience.

Mocking up in slides is nothing new. The trick was making it efficient and effective:

We prioritized scenarios that were the most risk for user experience.

We created modular slide decks. Each deck focused on exactly one scenario-based task (and scenarios were outcome based). Modular slide decks are easier to build, review and update. Our average deck was around six slides.

Each slide in a deck was a single step in the task from the user's perspective.

Each slide had a visual mock up of what the user would see

To paint some of the bigger stories, we did larger wrapper decks, but only after getting the more fine-grained scenarios right. Our house was made of stone instead of straw. In practice, I see a lot of beautiful end-to-end scenarios decks that are too big, too fragile and too make believe.

For example, here's the slide list for one deck:

scenario - User subscribes to a guidance feed

summary of steps (flat list of the steps)

Step 1. user finds a relevant item

Step 2. user subscribes to view

Step 3. user displays view in RSS reader

What originally took a week to prototype, we could mock up in an hour if not minutes. Do-overs were no longer a problem. In fact, mocking up alternate solutions was a breeze. The beauty was we could keep our release rhythm of every two weeks, while we explored solution paths in the background, with less disruption to the dev team.

The other beauty was we could use the same deck to walkthrough with customers and the dev team. The customers would bang on the user experience. The developers would bang on the technical feasibility. For example, show a catalog to customers and they evaluated the the best way to browse and filter. Sow the same screen to the devs and they would evalute the performance of the catalog. We would also brainstorm the "what-ifs", such as how will the catalog perform when there's a billion items in it ... etc. We got better at teasing out the key risks before we hit them.

Building the software became more an exercise of instantiating the user experience versus leaving too much to be made up on the fly.

To "be the software", it's as simple as letting the user walk through the user experience of performing that task (via the slides), and, as John put it, "you be the software ... you simply state how the software would respond." If slides are too heavy, draw on paper or use a whiteboard. The outcome is the user gets a good sense of what it's like to use your solution, while you get a sense of the user's more specific needs. The interactive approach produces way more benefits than a simple spec review or 1/2-baked prototype.

Yesterday's snowfall in Redmond was interesting for me. During my drive home, it was pretty dark, icy and cold. As I came up Old Redmond Road, I saw an object coming towards me, moving somewhat erratically, that looked too small to be a car.

It wasn't a car at all. It was a cross-country skier making his way down the middle of the road, followed by a trail of cars. I'm not sure at what point the street looked like good skiing and I don't know if he had an exit strategy, but he did seem to be having fun and going the speed limit.

I had my digital camera with me, but I forgot to use it. I was more focused on skating my car down the right side of the road. By the time I got home, I had a bunch of "mental snapshots" of various scenes along the way home, but nothing in hand to share.

That got me thinking of the MyLifeBits project. MyLifeBits is effectively software for "lifelogging" or archiving your life on disk. Although it's a bit extreme for me, there are times where I wish I automatically had more than just the mental snapshots.

When I last met with Rob Caron to walk him through Guidance Explorer, one of the concepts that peaked his interest was test-cases for content. He suggested I blog it, since it's not common practice and could benefit others. I agreed.

If you're an author or a reviewer, this technique may help you. You can create explicit test-cases for the content. Simply put, these are the "tests for success" for a given piece of content. Here's an example of a few test cases for a guideline:

Test Cases for Guidelines

Title

Does the title clearly state the action to take?

Does the title start with an action word (eg. Do something, Avoid something)?

Applies To

Do you list technology and version? (e.g. ASP.NET 2.0)

What to Do

Do you state the action to take?

Do you avoid stating more than the action to take?

Why

Do you provide enough information for the user to make a decision?

Do you state the negative consequences of not following this guideline?

When

Do you state when the guideline is applicable?

Do you state when not to use this guideline?

How

Do you state enough information to take action?

Do you provide explicit steps that are repeatable?

Problem Example

Do you show a real world example of the problem from experience?

If there are variations of the problem, do you show the most common?

If this is an implementation guideline, do you show code?

Solution Example

Does the example show the resulting solution if the problem example is fixed?

If this is a design guideline is the example illustrated with images and text?

If this is an implementation guideline is the example in code?

Additional Resources

Are the links from trusted sites?

Are the links correct in context of the guideline?

Related Items

Are the correct items linked in the context of the guideline?

Additional Tests to Consider When Writing a Guideline

Does the title clearly state the action to take?

Does the title start with an action word (eg. Do something, Avoid something)?

If the item is a MUST, meaning it is prevelant and high impact, is Priority = p1?

If the item is a SHOULD, meaning it has less impact or is only applicable in narrower circumstances, is Priority = p2?

If the item is a COULD, meaning it is nice to know about but isn't highly prevelant or impactful, is Priority = p3?

If this item will have cascading impact on application design, is Type = Design?

If this item should be followed just before deployment, is concerned with configuration details or runtime behavior, is Type = Deployment?

If this item is still in progress or not fully reviewed, is Status = Beta?

Benefits to Authors and ReviewersThe test-cases serve as checkpoints that help both authors and reviewers produce more effective guidance. While you probably implicitly ask many of these questions, making them explicit makes them a repeatable practice for yourself or others. I've found questions to be the best encapsulation of the test because they set the right frame of mind. If you're an author, you can start writing guidance by addressing the questions. If you're a reviewer, you can efficiently check for the most critical pieces of information. How much developer guidance exists that does not answer the why or when? Too much. As I sift through the guidance I've produced over the years, I can't believe how many times I've missed making the why or when explicit.

I'm a fan of the test-driven approach to guidance and here's my top reasons why:

I can tune the guidance across a team. As I see patterns of problems in the quality, I can weed it out by making an explicit test case.

I can tailor test cases based on usage scenarios. For example, in order to use our checklist items for tooling scenarios, our problem and solution examples need to have certain traits. I can burn this into the test cases.

I can bound the information. When is it done and what does "good enough" look like? The test case sets a bar for the information.

I can improve the precision and accuracy of the information. By precision, I mean filter out everything that's not relevant. When it comes to technical information to do my job, I'm a fan of density (lots of useful information per square inch of text). Verbosity is for story time.

Today we published 238 new guidance items in Guidance Explorer. If you use the offline client, it should automatically synchronize to our online store.

We're in the process of performing a guidance sweep. The approach to the sweep is twofold:1. Make existing guidance available in Guidance Explorer.2. Identify user experience issues with the information models and tool design.

Benefits in GEMaking existing guidance available in Guidance Explorer involves re-factoring existing security guidance and performance guidance. The benefits of having the guidance available in Guidance Explorer include:

you can view across topics (for example, you can see across the security and the performance guidance)

you can filter down to exactly the guidance items you need for a given scenario or task

you can build multiple custom views based on how you need to use the guidance

you can build guides on the fly (you can save a view as a Word doc or HTML files for example)

you can tailor the guidance to your scenario (e.g. save an item into your library in GE and edit the guidance to your liking)

you can supplement the guidance for your scenario (because GE is also an authoring environment, you can write your own guidance)

How We Improve Our GuidanceAn underlying strategy in GE was to help support users quickly hunt and gather relevant items rather than try and guess your context and what you need. In other words, it's a tool to help smart people versus a smart tool that might get in your way. This was actually an important decision because we had to pick a problem we knew we could help directly solve and add value.

The feedback from customers on existing guidance was that it was great stuff, but there were 3 key problems:1. it's a copy+paste exercise to grab just the guidance you need2. it's not atomic enough (monoliths over bite-sized chunks)3. many of the items, while they read well, were not actionable enough

That's why we took the following measures on our guidance:

split the guidelines and checklists into individual items (we chunked the guidance into units of action)

we cleaned up our templates for the various guidance types (we gave the chunked items a common look and feel)

made the schema explicitly include answers to "why" and "how", as well as include problem examples and solution examples (we made the chunks more actionable and verifiable)

As we port existing guidance to our updated schemas, we often find guidance items lacking key information such as why or how, or example code.

Guidance Explorer in PracticeWhat's been great so far is that some folks in the field have let me know how they've been using it for customer engagments. Apparently the ability to customize guidance has resonated very well. One consultant in particular has used Guidance Explorer for several engagements to save time and effort. He uses GE as a general purpose rules and guidelines store. He's also tailored guidelines and checklists for different audience levels (executive, development leads, architects, developers, PMs) and for different activities (design reviews, code reviews, and deployment reviews).

A few customers have let me know they are using the UNC share scenario to create guidance libraries for their team development. They told me they like the idea that it is like a simple typed-wiki that you can act on. The fact that they can create views and print out docs from the library has been the main appeal.

The other benefit that more customers are appreciating is the templates for guidelines and checklists. They like the fact that it starts to simplify authoring as well as sharing prescriptive guidance. For anybody who has authored guidelines or checklists, they know that it's challenging to write actionable guidance that can be reused. What we're sharing in Guidance Explorer is the benefit of experience and lessons learned over the years of producing resuable guidance for various audiences.

R&D ProjectAs a reminder and to keep things in perspective, Guidance Explorer is an R&D project. While there are immediately tangible benefits, the real focus is on the learnings around user experience so that patterns & practices can improve it's ability to author and share guidance, and to make progress on helping debottleneck the creation of prescritive guidance for the software industry.

It may seem a bit after the fact, given it is .NET 1.1, but there were a few reasons for this: 1) our focus was more on testing how to codify our library of practices rather than a specific version; 2) we figured adding rules/versions would be easy once we understood the feasibility and work required; 3) our field was still performing code reviews for customers using .NET 1.1 so we could immediately test the impact.

It's important to know the types of rules your tool does or does not cover (policies, requirements, vulnerabilities, and best practices).

It's important to know your various tool options and usage scenarios (e.g. Managed code analysis plugs into check-in policies or part of a build process, custom validators would check deployment at design time, Microsoft Best Practices Analyzer would check deployment at deployment time, Practices Checker would be a manual inspection scenario ... etc.).

It's important to know the ecosystem around your "rules" library (e.g. how do you keep your "rules" library up to date).

I'm continuing to explore various options to manage a library of building codes/practices/rules and then map out which tools can check these items, and where in the life cycle they should be checked. I've been informally referring to this problem as "policy verification through the life cycle."

While this first version is simply a set of links, it's a stepping stone to adding additional functionality. We wanted a way to deliver incremental functionality with a simple interface. The toolbar can be notified when there's new content or an new version of the toolbar itself.

The most interesting learning for me was the trade-offs in user experience in terms of a designing a toolbar:

Who wants yet another toolbar in VS.NET?

Menus are great for dealing with multiple options, but there's something to be said for clicking a button.

Button can be more visible over a menu option, but who wants more buttons?

Use ExcludeSchema Serialization Mode while Exchanging Typed DataSet Over Network

Applies to

.NET 2.0

What to DoSet the Typed DataSet property SchemaSerializationMode to ExcludeSchema while transferring the typed DataSet over network for better performance.

WhyThe serialization of a typed DataSet can be optimized by setting the SchemaSerializationMode property value to ExcludeSchema. When ExcludeSchema is used, the serialized payload does not contain schema information, tables, relations and constraints. This results in a smaller payload for network transfer, which provides better performance.

WhenIf it is required to send the typed DataSet over network, use ExcludeSchema Serialization Mode by setting the the typed DataSet property SchemaSerializationMode to ExcludeSchema.

ExcludeSchema is supported only for a typed DataSet. ExcludeSchema should only be used in cases where the schema information of the underlying typed DataTables, DataRelations and Constraints do not get modified.

HowThe typed DataSet has a new property called SchemaSerializationMode in .NET Framework 2.0. Set the SchemaSerializationMode property of typed DataSet to ExcludeSchema before returning for network transfer as follows:

Problem ExampleA .NET 2.0 Windows Forms based application for Order Management, gets the list of Sales Orders by a Web Service call. The Web Service internally makes a business logic call to get the list of Sales Order in a Typed DataSet. The Web Service returns the Typed DataSet to the smart client application. The code implementation does not exclude schema information, while serializing the DataSet and therefore has a larger size. The larger content can take more time to transfer across network and have a negative impact on performance.

Solution ExampleA .NET 2.0 Windows Forms based application for Order Management, gets the list of Sales Orders by a Web Service call. The Web Service internally makes a business logic call to get the list of Sales Order in a Typed DataSet. The Web Service returns the Typed DataSet to the smart client applicaiton. The code implementation uses SchemaSerializationMode property of the Sales Order Typed DataSet to reduce the size of the serialized content. It gives performance benefit while transferring over network:

Of course, you can also leave off the front and contrast with the behavior you'd like to change_________ ... over static and stale communication approaches.

Notice the ..x "over" ...y approach for the practices/principles above. Sometimes laws/principles/rules come across as common sense or "yeah, I already mostly do that" until you sharply contrast. It's a challenge to both the principle/practice author (am I striking the precise and accurate chord?) and the principle/practice follower (am I really changing behavior?)

WhyThe HandleCollector API helps to optimize Garbage Collector efficiency while working with expensive unmanaged resource handles. The garbage collector cannot track the Memory allocated by Unmanaged Code, it can lead to un-optimized memory management by garbage collector. The HandleCollector can force garbage collection if the threshold number of handles are reached, thereby improves performance of the application.

WhenIf it is required to manage multiple Unmanaged Resource Handles in the application, it is recommended to use HandleCollector API to improve GC efficiency by ensuring the objects are destroyed seamlessly on-time.

HowCreate an instance of HandleCollector by providing three parameters Handle Name (string), Initial Threshold (int) and Maximum Threshold (int). The initial threshold is the the point at which GC can start performing garbage collection. The maximum threshold is the point at which GC must perform garbage collection.

Problem ExampleA .Net 2.0 Windows Forms based application needs to provide lot of GUI features to facilitate Paint functionality. The code internally uses many unmanaged GUI handles to manage various Bitmaps. If the developer misses out to destroy the handle by calling appropriate Dispose method at appropriate time, the object remains in memory till the time garbage collection is performed. Also, if memory allocation is performed within unmanaged resource, the GC may not be even aware of it to force collection. At runtime, it might be required to force garbage collection to optimize GC memory management for unmanaged allocations.

Solution ExampleA .Net 2.0 Windows Forms based application needs to provide lot of GUI features to facilitate Paint functionality. The code internally uses many unmanaged GUI handles to manage various Bitmaps. The application creates an instance of HandleCollector by providing Handle Name, Initial Threshold and Maximum Threshold. When application creates an expensive handle, it increases the total handle count by invoking the Add method. When handle count reaches to the maximum threshold limit, the GC will force garbage collection automatically. Also, if the handle is destroyed, it will automatically reduce the total handle count in the following code:

What to DoUse new .Net 2.0 System.Transactions API for controlling transactions in managed code when working with SQL Server 2005

WhySystem.Transactions API gives flexibility to shift between local database server transactions and distributed database server transactions. The Systems.Transactions API, when used with SQL Server 2005, uses the Promotable Transactions feature through the Lightweight Transactions Manager. It does not create a distributed transaction when not required, resulting in improved performance.

Note If System.Transactions API is used to manage local transactions on SQL Server 2000, the local transaction is automatically promoted to a distributed transaction managed by MSDTC. SQL Server 2000 does not support Promotable Transactions.

WhenIf it is required to control transactions in managed code while working with SQL Server 2005, use Systems.Transactions API for improving performance and flexibility.

This guideline should not be used when working with SQL Server 2000.

HowThe following information is for using "Promotable Transactions".

While using the System.Transaction API is to define Transaction Scope using TransactionScope Class, it defines the boundary for the required transactions.

Within Transaction Scope block use normal ADO.NET code for executing the statements using Connection, Command and Execute methods. If the transaction is successful, invoke TransactionScope.Complete method. If the transaction is unsuccessful, the transaction will be automatically rolled back as it will not execute TransactionScope.Complete in the program flow.

Problem ExampleA web application for Online Shopping, provides a user interface to purchase items. Once the items are purchased, the item entry should be added for billing in a Billing database table in a SQL server 2005 database. At the same time, the stock of the item should be reduced by the number of units sold in an Item Quantity database table. The entire operation needs to be performed in single transaction to maintain data integrity.

The application follows a traditional approach of using SqlTransaction API which enforces only local transactions. If distributed database transactions are required, the code has to be changed and compiled again. This breaks the principle of flexibility and agility in the design. The following code illustrates the problem, which forces local transactions and compromises flexibility to change it to distributed transactions:

Instead, if the new System.Transactions API is used, it supports distributed transactions also.

Solution ExampleA web application for Online Shopping, provides a user interface to purchase items. Once the items are purchased, the item entry should be added for billing in the Billing database table on SQL server 2005. At the same time, the stock of the item should be reduced by the number of units sold in the Item Quantity database table. The entire operation needs to be performed in single transaction to maintain data integrity. The new System.Transactions API is used to provide flexibility without compromise on performance. System.Transactions API has a feature of Promotable Transactions when used with SQL Server 2005. It determines the need of using distributed transactions or local transactions at runtime for improved performance:

Use AddMemoryPressure while consuming Unmanaged Objects through COM Interop

Applies to

.NET 2.0

What to DoUse .NET 2.0 CLR API AddMemoryPressure and RemoveMemoryPressure while consuming the Unmanaged Objects from Managed Code through COM Interop.

WhyThe garbage collector cannot track the Memory allocated by Unmanaged Code, it only tracks the Memory allocated by Managed Code.

If there is a large amount of memory allocation (such as images or video data) in the unmanaged code, the GC will be able to see only the reference of unmanaged objects, but not the size of the memory occupied by the unmanaged object references.

Since GC may be unaware of the large memory allocation within unmanaged code, the GC will not know that a collection should be executed, as it does not realize any "memory pressure" to cause a collection.

The AddMemoryPressure method can be used to inform GC about how much unmanaged memory a managed object will be referencing, when it consumes the Unmanaged Objects from Managed Code. The pro-active indication to GC improves the GC heuristics and collection algorithm, which improves memory management and performance.

WhenIf the Unmanaged Objects invoked from Managed Code allocates a large amount of unmanaged memory at runtime, the GC should be informed about the total memory consumed by the managed and unmanaged code, by invoking AddMemoryPressure method, to improve memory management of GC.

HowApplying memory pressure is the technique using which the GC can be informed about the memory allocation that might be performed by the unmanaged code within a managed code wrapper.

The AddMemoryPressure should be used to inform GC about probable memory allocation by the unmanaged code:

The RemoveMemoryPressure should be used to inform GC to remove memory pressure while destroying the wrapper managed object which was consuming the unmanaged objects:

~Bitmap() { GC.RemoveMemoryPressure(_size); // other clean up code }

Note For every AddMemeoryPressure call, there must be a matching RemoveMemoryPressure call which will remove exactly the same amount of memory pressure as added earlier. Failing to do so can adversely affect the performance of the system in applications that run for long periods of time.

It actually uses unmanaged memory and occupies large amount of system memory. Since the garbage collector cannot track the memory allocated by unmanaged code, the application might run low on memory without triggering garbage collection, degrading the performance of the application.

Use of AddMemoryPressure and RemoveMemoryPressure informs about the memory allocation and deallocation that might be performed by the unmanaged code within a managed code wrapper. The GC will know to perform a collection when there is a memory pressure and will work efficiently.

Step 1. Identify and prioritize the tasks and scenarios.Task-based content is "executable". You can take the actions prescribed in the guidance and produce a result. Scenarios bound the guidance to a context. This helps for relevancy and for evaluating the appropriateness of the recommendations.

Good candidates for tasks and scenarios include:

Problems and pain points

Key engineering decisions

Difficult tasks

Recurring engineering questions

Techniques

In many ways, the value of your prescriptive guidance is a measure of the value of the problem multiplied by how many people share the problem.

These are some practices we learned in the guidance business to write more effective guidelines:

Follow the What To Do, Why and How Pattern

Keep it brief and to the point

Start with Principle Based Recommendations

Provide Context For Recommendations

Make the Guidelines Actionable

Consider Cold vs. Warm Handoffs

Create Thread Killers

Follow the What to Do, Why and How PatternStart with the action to take -- the "what to do". This should be pinned against a context. Context includes which technologies and situations the guidance applies. Be sure to address "why" as well, which exposes the rationale. Rationale is key for the developer audience. It's easy to find many guidelines, missing context or rationale. Some of the worst guidelines leave you wondering what to actually do.

Keep It Brief and to the PointAvoid "blah, blah, blah". Say what needs to be said as succinctly as possible. Ask "why, why, why?" to everything you write - every paragraph and every sentence. Does it add value? Does it help the reader? Is it actionable? Often the answer is no. It's hard to do, but it keeps the content "lean and mean".

Start with Principle-Based RecommendationsA good principle-based recommendation addresses the question: "What are you trying to accomplish?". Expose guidance based on engineering versus implementation or technology of the day. This makes the guidance less volatile and arguably more useful. An example principle-based recommendation would be: Validate Input for Length, Range, Format, and Type. You can then build more specific guidelines for a technology or scenario from this baseline recommendation.

Provide Context for RecommendationsAvoid blanket recommendations. Recommendations should have enough context to be prescriptive. Sometimes this can be as simple as prefixing your guideline with an "if" condition.

Make the Guidelines ActionableBe prescriptive, not descriptive. The guideline should be around actionable vs. just interesting information. Note that considerations are actions provided you tell the reader what to consider, when and why. As a general rule, avoid providing too much background or conceptual information. Point off to primer articles, books etc for background.

Choose Warm vs. Cold HandoffsIf you are sending the reader to another link for a specific piece of information, be explicit. It's as simple as adding "For more information on xyz, see ..." before your link. That's a warm hand off. A cold hand off is simply having a list of links and expecting the reader to follow the links and figure out why you sent them there. The worst is when the links are irrelevant and you simply added links because you could.

Create Thread KillersA "thread killer" is a great piece of information that when quoted or referred to can stop a technical discussion or alias question (a discussion thread) with no further comments. Look at the alias, understand the questions being asked, and tackle the root causes and underlying problems that cause these questions. (Think of the questions as symptoms). Make guidance that nails the problem. A great endorsement is when your "thread killer" is used to address FAQs on discussion aliases and forums.

Where to See This in PracticeThe following are examples of prescriptive guidelines based on these practices:

Use Generics To Eliminate the Cost Of Boxing, Casting and Virtual calls

Applies to

.NET 2.0

What to Do

Use Generics to eliminate cost of boxing, casting and virtual calls

WhyGenerics can be used to improve the performance by avoiding runtime boxing, casting and virtual calls.

List<Object> class gives better performance over ArrayList. An example benchmark of a quick-sort of an array of one million integers may show the generic method is 3 times faster than the non-generic equivalent. This is because boxing of the values is avoided completely. In another example, the quick-sort of an array of one million string references with the generic method was 20 percent faster due to the absence of a need to perform type checking at run time. You results will depend on your scenario. Other benefits of using Generics are compile-time type checking, binary code reuse and clarity of the code.

WhenUse Generics feature for defining a code (class, structure, interface, method, or delegate) which has to be used by different consumers with different types.

Consider replacing a generic code (class, structure, interface, method, or delegate), which does an implicit casting of any type to System.Object and forces the consuming code to cast between Object references to actual data types.

Only if there is a considerable number (>500) to store consider having a special class, otherwise just use the default List<Object> class. List<Object> gives better performance over ArrayList as List<Object> has a better internal implementation for enumeration.

Note The .NET Framework 2.0 provides a suite of generic collection classes in the class library. Your applications can further benefit from generics by defining your own generic code

Problem ExampleAn Order Management Application stores the domain data Item, Price, etc in Cache using ArrayList. ArrayList accepts any type of data and implicitly casts into objects. Numeric data like Order number, customer id, etc are wrapped to object type from primitive types (boxing) while storing in the ArrayList. Consumer code has to explicitly cast the data from Object type to specific data type while retrieving from the ArrayList. Boxing and un-boxing requires lot of operations like memory allocation, memory copy & garbage collection which in turn reduces the performance of the application.

Example code snippet to add items to ArrayList or to get/set items from the ArrayList

Solution ExampleAn Order Management Application stores the domain data Item, Price, etc in Cache. Using Generics feature avoids necessity of run time boxing and casting requirements and makes sure of compile time type checking

Implement the defining code for generic class. Generic class can be implemented only if required, else default List<T> class can be used //Use to allow consumer code to specify the required type

Here's the next .NET Framework 2.0 performance guideline in the series from Prashant Bansode, Bhavin Raichura, Girisha Gadikere and Claudio Caldato.

...

Use Token Handle Resolution API to get the Metadata for Reflection

Applies to

.NET 2.0

What to DoUse the new .NET 2.0 token handle resolution API RuntimeMethodHandle to get the Metadata of members while using Reflection.

WhyThe RuntimeMethodHandle is a small, lightweight structure that defines the identity of a member. RuntimeMethodHandle is a trimmed down version of MemberInfos, which provides the metadata for the methods and data without consuming .NET 2.0 back-end cache.

The .NET Framework also provides GetXxx type of API methods for e.g.. GetMethod, GetProperty, GetEvent to determine the metadata of given Type at runtime. There are two forms of these APIs, the non-plural, which return one MemberInfo (such as GetMethod), and the plural APIs (such as GetMethods).

The .NET framework implements a back-end cache for the MemberInfo metadata to improve the performance of GetXxx API.

The caching policy is implemented irrespective of plural or non-plural GetXxx API call. Such eager caching policy degrades the performance on calls to or non-plural GetXxx API calls.

RuntimeMethodHandle works approximately twice faster than compared to equivalent GetXxx API call, if the MemberInfo is not present in the back-end .NET cache.

WhenIf it is required to get the metadata of a given Type at runtime, use new .NET 2.0 token handle resolution API RuntimeMethodHandle for better performance than traditional GetXxx API calls.

HowThe following code snippet shows how to get the RuntimeMethodHandle:

Problem ExampleA Windows Forms based application needs to dynamically load the plug-in Assemblies and available Types. The application also needs to determine the metadata of a given Type (methods, members etc) at runtime to execute Reflection calls.

The plug-in exposes a Type CustomToolBar, which is derived from Type BaseToolBar. The CustomToolBar Type has 2 methods - PrepareCommand, ExecuteCommand. The BaseToolBar Type has 3 methods - Initialize, ExecuteCommand and CleanUp. To execute the ExecuteCommand method of type CustomToolBar at runtime, it gets the metadata of that method using GetXxx API as shown in the following code snippet.

Since the .NET Framework implements eager caching policy, the call to get the metadata for a single ExecuteCommand method will also get the metadata of all the five methods of CustomToolBar and BaseToolBarTypes.

MethodInfo mi = typeof(CustomToolBar).GetMethod("ExecuteCommand");

The .NET framework implements a back-end cache for the MemberInfo metadata to improve the performance of GetXxx API. The implemented caching policy caches all members by default, irrespective of plural or non-plural API call. Such eager caching policy degrades the performance on calls to or non-plural GetXxx API calls.

Solution ExampleA Windows Forms based application needs to dynamically load the plug-in Assemblies and available Types. The application also needs to determine the metadata of a given Type (methods, members etc) at runtime to execute Reflection calls.

The plug-in exposes a Type CustomToolBar, which is derived from Type BaseToolBar. The CustomToolBar Type has 2 methods - PrepareCommand, ExecuteCommand. The BaseToolBar Type has 3 methods - Initialize, ExecuteCommand and CleanUp. To execute the ExecuteCommand method of type CustomToolBar at runtime, it gets the metadata of that method using RuntimeMethodHandle is used, as shown in the following code snippet. This can improve the performance of the application. :

If the appropriate MemberInfo is already in the back-end .NET cache, the cost of going from a handle to a MemberInfo is about the same as using one of the GetXxx API call. If the MemberInfo is not available in the cache RuntimeMethodHandle is approximately twice faster than the GetXxx API call.

Prashant Bansode, Bhavin Raichura, and Girisha Gadikere teamed up with Claudio Caldato (CLR team) to create some new performance guidelines for .NET Framework 2.0. The guidelines use our new guideline template.

...

Use TryParse Method to Avoid Unnecessary Exceptions

Applies to

NET 2.0

What to DoUse TryParse Method instead of Parse Method for converting string input to a valid .Net data type. For example, use TryParse method before converting a string Input to integer data type.

WhyThe Parse method will throw exception - ArgumentNullexception or FormatException or OverflowException, if the string representation cannot be converted to the respective data type.

Unnecessary Throwing Exceptions and Handling the same such as above has a negative impact on the performance of the application. The TryParse method does not throw an exception if the conversion fails instead it returns false, and hence saves exception handling related performance hit.

WhenIf it is required to convert a string representation of a data type to a valid .Net data type, use TryParse method instead of calling the Parse method to avoid unnecessary exception.

HowThe following code snippet illustrates how to use TryParse method :

Problem ExampleConsider a Windows Forms application for creating an Invoice. The application takes user inputs for multiple items as product name, quantity, price per unit and date of purchase. The user provides these inputs in the text boxes. The user can enter multiple items in an invoice at a given time and then finally submit the data for automatic billing calculation. The application internally needs to convert the string input data to integer (assume for simplicity). If the user enters invalid data in the text box, the system will throw an exception. This has adverse impact on performance of the application.

Solution ExampleConsider a Windows Forms application for creating an Invoice. The application takes user inputs for multiple items as product name, quantity, price per unit and date of purchase. The user provides these inputs in the text boxes. The user can enter multiple items in an invoice at a given time and then finally submit the data for automatic billing calculation. The application internally needs to convert the string input data to integer (assume for simplicity). If the user enters invalid data in the text box, the system will not throw unnecessary exceptions and hence improves the application performance.