BSIMM4 License
This work is licensed under the Creative Commons Attribution-Share Alike 3.0 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.

2012

ii

Executive Summary
The Building Security In Maturity Model (BSIMM) is the result of a multi-year study of real-world software security initiatives. We present the model as built directly out of data observed in fifty-one software security initiatives, from firms including: Adobe, Aon, Bank of America, Box, Capital One, The Depository Trust & Clearing Corporation (DTCC), EMC, F-Secure, Fannie Mae, Fidelity, Goldman Sachs, Google, Intel, Intuit, JPMorgan Chase & Co., Mashery, McKesson, Microsoft, Nokia, Nokia Siemens Networks, QUALCOMM, Rackspace, Salesforce, Sallie Mae, SAP, Scripps Networks, Sony Mobile, Standard Life, SWIFT, Symantec, Telecom Italia, Thomson Reuters, Vanguard, Visa, VMware, Wells Fargo, and Zynga. The BSIMM is a measuring stick for software security. The best way to use the BSIMM is to compare and contrast your own initiative with the data about what other organizations are doing contained in the model. You can then identify goals and objectives of your own and look to the BSIMM to determine which further activities make sense for you. The BSIMM data show that high maturity initiatives are well rounded—carrying out numerous activities in all twelve of the practices described by the model. The model also describes how mature software security initiatives evolve, change, and improve over time.

we report not only on current practices. Our work with the BSIMM model shows that measuring a firm’s software security initiative is both possible and extremely useful.
1
BSIMM4
.
Introduction
S
oftware security began to flourish as a discipline separate from computer and network security in the late 1990s. The BSIMM is not a “how to” guide. Getting security right means being involved in the software development process. but also on the ways in which some initiatives have evolved over a period of years. the BSIMM is a reflection of the software security state of the art. By quantifying the practices of many different organizations. We devote the later portion of the document to a detailed explanation of the 111 activities that now comprise our model and a summary of the raw data we have collected. social. We then explain our model and the method we use for quantifying the state of an initiative. We begin with a brief description of the function and importance of a software security initiative. What kinds of bugs and flaws lead to security problems? How can we identify problems systematically? By the middle of the following decade. carry out. and organizational aspects as well. firms participating in the BSIMM project show measurable improvement in their software security initiatives. Software security encompasses business.T
he Building Security In Maturity Model (BSIMM. and execute the evolution of a software security initiative. Since then. We have updated the description of each activity for BSIMM4 and also added two new activities to the model. and measure initiatives of their own. which comprises ninety-five distinct measurements because some firms use BSIMM to measure each of their business units and some firms have been measured more than once. We use the term Software Security Initiative to refer to all of the activities undertaken for the purpose of building secure software. Our aim is to help the wider software security community plan. Researchers began to put more emphasis on studying the ways a programmer can contribute to or unintentionally undermine the security of a computer system. We present our findings from studying fifty-one initiatives. pronounced “bee simm”) is a study of existing software security initiatives. there was an emerging consensus that creating secure software required more than just smart individuals toiling away. practitioners have come to know that process alone is also insufficient. BSIMM measurements can be used to plan. Instead. structure. Because our study began in 2008. nor is it a onesize-fits-all prescription. we can describe the common ground shared by many as well as the variation that makes each unique. Over time.

thereby allowing us to compare initiatives that use different terms. BSIMM4 is the fourth major version of the BSIMM model. It includes thoroughly updated activity descriptions. exist in different vertical markets or create different work products. we identified a set of common activities. These executives lead an internal group that we call the Software Security Group (SSG). and a longitudinal study. Our Software Security Framework (SSF) and activity descriptions provide a common vocabulary for explaining the salient elements of a software security initiative. Succeeding with the BSIMM without becoming familiar with the literature is unlikely. the practices. data from fifty-one firms. We then created scorecards for each of the nine initiatives that show which activities the initiatives carry out.BSIMM4
The purpose of the BSIMM is to quantify the activities carried out by real software security initiatives. or provide references to the ever-expanding literature. The BSIMM is written with the SSG and SSG leadership in mind. (We present the framework on page 19. We expect readers to be familiar with the software security literature. which we organized according to the Software Security Framework. two new activities. We understand that not all organizations need to achieve the same security goals. operate at different scales. You can become familiar with many concepts by reading both Software Security and The Security Development Lifecycle.
Audience
The BSIMM is meant for use by anyone responsible for creating and executing a software security initiative. In order to validate our work. The BSIMM does not attempt to explain software security basics. Because these initiatives make use of different methodologies and different terminology. and the scorecard we created for their initiative. describe its history. charged with directly executing or facilitating the activities described in the BSIMM.
Method
We built the first version of BSIMM in Fall of 2008 as follows: • • We relied on our own knowledge of software security practices to create the Software Security Framework.
•
2012
2
. We classify our work as a maturity model because improving software security almost always means changing the way an organization works—something that doesn’t happen overnight. We have observed that successful software security initiatives are typically run by a senior executive who reports to the highest levels in an organization. From these interviews.) We conducted a series of nine in-person interviews with executives in charge of software security initiatives. but we believe all organizations can benefit from using the same measuring stick. we asked each participant to review the framework. the BSIMM requires a framework that allows us to describe all of the initiatives in a uniform way.

SAP. Standard Life. We also publish observations about subsets (such as industry verticals) when our sample size for the subset is large enough to guarantee the anonymity of the participants. Capital One.) We used the resulting scores to refine the set of activities and their placement in the framework. On average. Simple observations. F-Secure.77 people (smallest 0. Thomson Reuters. energy (2). As a descriptive model. Sallie Mae. Nokia. telecommunications (3). largest 350. Telecom Italia.48 people (smallest 1.. but we publish aggregate data describing the number of times we have observed each activity (see page 58). insurance (2). Visa. Those companies among the fifty-one who graciously agreed to be identified include: Adobe. Salesforce. We hold the scorecards for individual firms in confidence. Google. Wells Fargo. Microsoft. Symantec. we computed the score for a large firm by scoring some number of major divisions and then combining the scores into one score for the firm. median 1500). security (3). Fidelity. Previous work has either described the experience of a single organization or offered prescriptive guidance based on a combination of personal experience and opinion. largest 100. All fifty-one firms agree that the success of their program hinges on having an internal group devoted to software security—the SSG. we have used the same interview technique to conduct BSIMM assessments for forty-two additional firms (a total of fifty-one firms).” “thou shalt not steal thy neighbors’ bananas. Box. Scripps Networks. One participant has undertaken three such measurements. Rackspace. Intuit. The average number of developers among our targets was 4455 people (smallest 11. Fannie Mae. the goal of the BSIMM is only to observe and report.
Participants
The fifty-one participating organizations are drawn from twelve verticals (with some overlap): financial services (19). The Depository Trust & Clearing Corporation (DTCC). All told. VMware. Sony Mobile. and internet service provider (1). JPMorgan Chase & Co. median 7. McKesson. EMC. Nokia Siemens Networks.286 developers. We like to say that we wandered off into the jungle to see what we could see and discovered that “monkeys eat bananas in X of the Y jungles we visited. cloud (13). simply reported. QUALCOMM. but in the realm of software security it has not previously been applied at this scale. Aon. healthcare (1). media (4). Bank of America. Our “just the facts” approach is hardly novel in science and engineering. independent software vendors (19). Goldman Sachs. technology firms (13).5) with a “satellite” of others (developers. and people in the organization directly engaged in and promoting software security) of 40. We have also conducted a second complete set of interviews with thirteen of the participants in order to study how their initiatives have changed over time. (In four cases. and Zynga. the participants have practiced software security for nearly six years (with some initiatives being brand new at first measurement and the oldest initiative being seventeen years old in September 2012). SSG size on average is 19. Mashery. Intel. largest 30. retail (2). yielding an average percentage of SSG to development of just over 1. the BSIMM describes the work of 974 SSG members working with a satellite of 2039 people to secure the software developed by 218.At this point in the project. median 6).” “do not run while eating a banana. Vanguard. architects.” Notice that the BSIMM does not report “you should only eat yellow bananas.” or any other value judgments. SWIFT.95%.000.
BSIMM4
3
.

and software security is no exception. you can leverage the many years of experience captured in the model.Objectives
We created the BSIMM in order to learn how software security initiatives work and to provide a resource for people looking to create or improve their own software security initiative. comprising twelve practices divided into four domains. Note that no organization carries out all of the activities described in the BSIMM. By using the BSIMM as a guide for your own software security initiative. any software security initiative will have been created with some high-level goals in mind. you can use the BSIMM as a measurement tool to guide your own software security initiative. manage.
2012
4
. See the SSF section below. Activities in each practice are divided into three levels corresponding to maturity. intelligence. Instilling software security into an organization takes careful planning and always involves broad organizational change. In general. Each domain in the Software Security Framework has three practices. and deployment.
Terminology
Nomenclature has always been a problem in computer security. architects. Secure Software Development Lifecycle (SSDL) – Any SDLC with integrated software security checkpoints and activities. Each activity is directly associated with an objective. Security Development Lifecycle (SDL) – A term used by Microsoft to describe their Secure Software Development Lifecycle. See the SSF section below. The BSIMM is appropriate if your business goals for software security include: • • • • Informed risk management decisions Clarity on what is “the right thing to do” for everyone involved in software security Cost reduction through standard. software managers. Satellite – A group of interested and engaged developers. Software Security Framework (SSF) – The basic structure underlying the BSIMM. Activities are divided into three maturity levels in the BSIMM. and evolve software security activities in a coordinated fashion. Software Security Initiative – An organization-wide program to instill. Domain – One of the four major groupings in the Software Security Framework. Also known in the literature as an Enterprise Software Security Program (see chapter 10 of Software Security). SSDL touchpoints. repeatable processes Improved code quality
By clearly noting objectives and by tracking practices with metrics tailored to your own initiative. measure. Here are some of the most important terms we use throughout the document: Activity – Actions carried out or facilitated by the SSG as part of a practice. Software Security Group (SSG) – The internal group charged with carrying out and facilitating software security. We’ve observed that step one of a software security initiative is forming an SSG. The domains are: governance. Practice – One of the twelve categories of BSIMM activities. and testers who have a natural affinity for software security and are organized and leveraged by a software security initiative. See the SSF section below. There are a number of terms we use in the BSIMM that have particular meanings for us. You should tailor the activities that the BSIMM describes to your own organization (carefully considering the objectives we document).

garner resources.
5
BSIMM4
. Sr. Do not attempt to start with network security people and teach them about software. The executives in charge of the software security initiatives we studied have a variety of titles. Director of Application Controls. start with developers and teach them about security. and work directly with hundreds of developers. SDLCs. eleven exist in the CIO’s organization. the best code reviewers sometimes make very poor software architects. initiatives spearheaded by resources from an existing network security group often run into serious trouble when it comes time to interface with development groups. and to perform code review you must actually understand code (not to mention the huge piles of security bugs). SVP of Information Security. one reports to the COO. but software security people are often impossible to find. Make sure you cover architectural capabilities in your SSG as well as you do code. Manager of Product Security. We also observed a fairly wide spread in exactly where the SSG is situated in the firms we studied. A number of the companies we studied did not specify where their SSG fits in the larger organization. Every single one of the fifty-one programs we describe in the BSIMM has an SSG. You also create a place in the organization where software security can take root and begin to thrive. software security can’t only be about finding specific bugs such as the OWASP Top Ten.ly/7dqCn8>. Chief Information Risk Officer. Likewise. Finally. and asking them to perform an Architecture Risk Analysis will only result in blank stares. In particular. and provide political cover for a software security initiative. Carrying out the activities in the BSIMM successfully without an SSG is very unlikely (and has never been observed in the field to date). If you must create software security types from scratch. As you will see below. train. Global Head of Information Security Markets. compilers. SSGs come in a variety of shapes and sizes. However. including: Director of IT Security and Risk Management. Grassroots approaches to software security sparked and led solely by developers and their direct managers have a poor track record in the real world. and CISO. Code review is a very important best practice. and one reports directly to the founder or CEO. fifteen SSGs report to CSOs. the SSG is often asked to mentor. Communications skills. four exist in either the General Counsel’s office or the Office of Compliance and Risk Management. Product Security Manager. bug tracking.Roles
D
etermining who is supposed to carry out the activities described in the BSIMM is an important part of making any software security initiative work. No amount of traditional security knowledge can overcome software cluelessness. SVP of Global Risk Management. see our informIT article You Really Need an SSG <http://bit. and good consulting horse sense are must–haves for at least a portion of the SSG staff.
Executive Leadership
Of primary interest is identifying and empowering a senior executive to manage operations. VP Product and Operations.
The Software Security Group (SSG)
The second most important role in a software security initiative after the senior executive is that of the Software Security Group. so create an SSG before you start working to adopt the BSIMM activities. All good SSGs appear to include both people with deep coding experience and people with architectural chops. you address two management 101 concerns—accountability and empowerment. The best SSG members are software security people. teaching capability. and everything else in the software universe. For more about this issue. nine exist in the CTO’s organization. By identifying a senior executive and putting him or her in charge of software security directly.

especially when it comes to applications they host or attach to as services in the cloud. largest 100.48 people (smallest 1.27% and the smallest was 0. That means two SSG members for every 100 developers when we average the ratios for each participant. and keep them up.5). and those organized according to internal business units. and metrics. median 7. In the statistics reported above.” Administrators must understand the distributed nature of modern systems and begin to practice the principle of least privilege. software security does not end when software is “shipped. and others are very centralized and policy-oriented. SSGs come in three major flavors: those organized according to technical SDLC duties.
2012
6
. defend them.05%. we noted an average ratio of SSG to development of a hair under 2% across the entire group of fifty-one organizations we studied. including Line of Business owners and Product Managers. At the highest level of organization. Software security is a business necessity. although in some cases we do attempt to clarify responsibilities in the goals associated with activity levels within practices. Business requirements should explicitly address security needs. we did observe some commonalities that are worth mentioning. and. custom software.Though no two of the fifty-one firms we examined had exactly the same SSG structure (suggesting that there is no one set way to structure an SSG). and software-as-a-service. and their managers must practice security engineering. As you will see in the Deployment domain of the SSF. Any sizeable business today depends on software to work. we often don’t explicitly point out whether a given activity is to be carried out by the SSG or by developers or by testers. the SSG attempts to empower builders so that they can carry out most of the BSIMM activities themselves with the SSG helping in special cases and providing oversight. SSG size on average among the fifty-one firms was 19. Only by pushing past the standard-issue operations view of security will we begin to make software systems that can stand up under attack. there are several common “subgroups” that are often observed: people dedicated to policy. The largest SSG was 27.
Everybody Else
Our survey participants have engaged everyone involved in the software development lifecycle as a means of addressing software security. You should come up with an approach that makes sense for your organization and takes into account workload and your software lifecycle. groups responsible for training development and delivery. Testers concerned with routine testing and verification should do what they can to keep a weather eye out for security problems. vendor-control groups. The SSG will interact directly with builders when they carry out the activities described in the BSIMM. including those who supply COTS. • Builders. as an organization matures. Generally speaking. are increasingly subjected to SLAs and reviews (such as vBSIMM) that help ensure products are the result of a secure SDLC. penetration testing. Some of the BSIMM activities in the Security Testing practice can be carried out directly by QA. To remind you of the particulars in terms of actual bodies. must understand how early investment in security design and security analysis affects the degree to which users will trust their products. internal “services” groups that (often separately) cover tools. externally-facing marketing and communications groups. Some SSGs are highly distributed across a firm. those organized by operational duties. Vendors. In this version of the BSIMM. Executives and middle management. ensuring that the systems that we build are defensible and not riddled with holes. including developers. If we look across all of the SSGs in our study.
• • • •
•
Participants feel the most important people to enlist for near-term progress in software security are the builders. architects. and middleware development plus shepherding. Operations people must continue to design reasonable networks. incident response groups. strategy.

Putting BSIMM4 to Use
BSIMM4
T
he BSIMM describes 111 activities that any organization can put into practice. initiatives tend to evolve from centralized and specialized in the beginning to decentralized and distributed (with an SSG at the core orchestrating things). and then averaging those scores for a group of firms. and architects) who share a basic interest in software security. The first graph shows data from all fiftyone BSIMM firms. Other more sophisticated analyses are possible. The second graph shows data from the top ten firms (as determined by raw score computed by summing the activities). We call this group the satellite. with one or two members in each product group. Among our population of fifty-one firms. of course. Here are two “spider charts” showing average maturity level in each of the twelve practices over some number of organizations. This suggests that as a software security initiative matures. nine of the ten firms with the highest BSIMM scores have a satellite (90%). Sometimes the satellite is more focused and gets together regularly to compare notes.
7
. The activities are described in terms of the SSF.
BSIMM4 Results
The BSIMM data yield very interesting analytical results. testers. Associated with each activity is an objective. which identifies twelve practices grouped into four domains. but are not directly employed in the SSG. Spider charts are created by noting the highest level of activity in a practice for a given firm (a “high water mark”). Sometimes the satellite is widely distributed. normalization by number of activities.Identifying a Satellite
In addition to the SSG. learn new technologies. Identifying and fostering a strong satellite is important to the success of many software security initiatives (but not all of them). and other schemes. level 3 (outside edge) is considered more mature than level 0 (inside point). Note that in all of these charts. Some BSIMM activities target the satellite explicitly. and twentyfour of the remaining forty-two firms outside of the top ten do (57%). The spider chart has twelve spokes corresponding to the twelve practices. and expand the understanding of software security in an organization. its activities become distributed and institutionalized into the organizational structure. many software security programs have identified a number of individuals (often developers. Of particular interest. and we continue to experiment with weightings by level. resulting in twelve numbers (one for each practice).

The graph below shows the distribution of scores among the population of 51 participating firms. the scores represent a slightly skewed bell curve. we divided the scores into five equal bins. As you can see. we can also take a look at relative maturity and average maturity for one firm against the others.
2012
8
. 93].By computing a raw score for each firm in the study. To make this graph. the range of observed scores is [9. To date.

An expanded version of this chart can be found on page 58. The BSIMM4 Scorecard below shows the number of times each of the 109 activities was observed in the BSIMM4 data.
9
BSIMM4
.We are pleased that the BSIMM study continues to grow (the data set has grown just over 20% since publication of BSIMM3 and is nine and a half times the size it was for the original publication). we began to apply statistical analysis yielding statistically significant results. Note that once we exceeded a sample size of thirty firms.

Below. We include complete descriptions of the new activities in the detailed BSIMM4 description and the BSIMM Skeleton below.4 Automate malicious code detection]. If we observe a candidate activity not yet in the model. Furthermore. we take a closer look at the proposed activity and figure out how it fits with the existing model. we have observed several activities not yet captured in the model. Our criteria for adding an activity to the BSIMM are as follows.New Activities for BSIMM5 Observation
For the first time in the BSIMM project. we found that the spider-graphyielding “high water mark” approach (based on the three levels per practice) is sufficient to get a low-resolution feel for maturity. but we describe the activities in this release.3 Simulate software crisis] and [CR3. In our own work using the BSIMM to assess levels. We also include the activities in the example measurement below so that participating firms know what to expect going forward. One meaningful comparison is to chart your own maturity high water mark against the averages we have published to see how your initiative stacks up. we have plotted data from a (fake) firm against the BSIMM Earth graph. The observations related to these activities will be first reported in the data released with BSIMM5. the candidate activity is tabled as too specialized. especially when working with data from a particular vertical or geography. Using the criteria above.
2012
10
. it is dropped. we determine based on previously captured data and BSIMM mailing list queries how many firms probably carry out that activity.
Measuring Your Firm with BSIMM4
The most important use of the BSIMM is as a measuring stick to determine where your approach currently stands relative to other firms. If the answer is only one firm. the two activities added to the BSIMM4 model are: [CMVM3. if the candidate activity is covered by the existing activities or simply refines or bifurcates an existing activity. and use “activity coverage” to determine their levels and build a scorecard. We have decided to add two new activities to the model going forward. Note which activities you already have in place. If the answer is multiple firms.

allowing the firm to understand the general popularity of an activity among the fifty-one BSIMM participants. The BSIMM Firms column shows the number of observations (currently out of 51) for each activity. you can devise a plan to enhance practices with other activities suggested by the BSIMM. the BSIMM makes it possible to build a long-term plan for a software security initiative and track progress against that plan. Once you have determined where you stand with activities. including eight activities that are the most common in their respective practices (purple boxes). By providing actual measurement data from the field. The scorecard you see here depicts a (fake) firm that performs forty-one BSIMM activities (1’s in the FIRM columns). On the other hand. This can be accomplished by building a scorecard using the data printed on page 58. For the
11
BSIMM4
. the firm does not perform the most commonly observed activities in the other four practices (red boxes) and should take some time to determine whether these are necessary or useful to its overall software security initiative.A direct comparison of all 111 activities is perhaps the most obvious use of the BSIMM.

and ownership behind them. That said. In particular. However. A closer look at the 109 activities reveals more interesting differences. The graph below shows data from the financial services vertical (19 firms) and independent software vendors (19 firms) charted together. the levels that we have identified hold water under statistical scrutiny. The best way to use the BSIMM in planning is to identify goals and objectives of your own and look to the BSIMM to determine which activities make sense. the data show that high maturity initiatives are well rounded and carry out activities in all twelve practices. The breakdown of activities into levels for each practice is meant only as a guide. Choosing among and between practices is not recommended. Level 2 (more difficult and requiring more coordination) slightly less so.
Studying Groups of Firms in BSIMM4
The spider charts we introduce above are also useful for comparing groups of firms from particular industry verticals or geographic locations. and Level 3 (rocket science) are much more rarely observed. browsing through objectives and noting which appeal to your culture is a much cleaner way to proceed. we don’t believe a successful initiative will be able to omit all activity for any of the twelve practices. mandate. there is no inherent reason to adopt all activities in every level for each practice. you can create a strategic plan for your software security initiative moving forward. Adopt those activities that make sense for your organization. you can determine which activities to adopt. This view is informed by the data we have gathered.
2012
12
. Put another way. The levels provide a natural progression through the activities associated with each practice. all initiatives do share common activities. and by ensuring proper balance with respect to domains. By the same measure. By identifying objectives and activities from each practice that would work for you. On average. independent software vendors have equal or greater maturity as compared to financial services firms in six of the twelve practices (one tie)! According to the low-resolution spider graph.record. financial services firms have equal or greater maturity as compared to independent software vendors in seven of the twelve practices (one tie). Though all initiatives look different and are tailored to fit a particular organization. Level 1 activities (straightforward and simple) are commonly observed. and ignore those that don’t. Once you know what your objectives are. these two verticals look essentially the same. Note that most software security initiatives are multi-year efforts with real budget. it is not at all necessary to carry out all activities in a given level before moving on to activities at a higher level in the same practice. In any case. as we describe below. Please note that in our view it is better to put some of the Level 1 activities in each practice into place than to accelerate to Level 3 in any one practice while ignoring other practices.

5 Promote executive awareness of compliance and privacy obligations]. In the Activity columns. The fact that FIs put more emphasis on compliance and policy than ISVs do is not surprising given that FIs are regulated. In addition to observation counts.
There is a great deal of overlap in the most common observations. which explains the sizeable delta in [AM1.3 Schedule periodic penetration tests for application coverage] can be explained with reference to the kinds of applications that FIs build (hosted) versus ISVs (boxed). we have highlighted the most common activity in each practice as observed in the entire BSIMM data pool (51 firms).In the table below you can see the BSIMM Scorecards for the two largest verticals compared side by side.2 Create data classification scheme and inventory]. The delta in [PT2. with a majority of deltas amounting to a count of four or less.2 Require security sign-off for compliance-related risk]. When we began working on the BSIMM. In the compliance and policy practice. The same goes for data classification (ISVs make things to store data and FIs store lots of data).1 Unify regulatory pressures].
13
BSIMM4
. [CP2. We are able to share these data for the first time because the number of firms in each sample is large. we have also highlighted the most commonly observed activities. and [CP2. For example. though there are small visible differences between the average financial services firm and the average independent software vendor when it comes to software security. In the Financial and ISV columns. No such luck. we have highlighted all activities observed in at least 15 out of 19 firms. we expected the data to lead us toward narratives explaining behavior in particular verticals or geographies. These are [CP1. there are three activities that are observed significantly more often in FIs than in ISVs. the commonalities between high-maturity participants in each vertical outweigh these differences by an order of magnitude. however.

where differences between sets are slight. see BSIMM Europe: Measuring software security initiatives worldwide <http://bit. Though individual activities among the twelve practices come and go as shown in the Longitudinal Scorecard below. Software security initiatives mature over time.
BSIMM as a Longitudinal Study
T
hirteen of the fifty-one firms have been measured twice using the BSIMM measurement system.
2012
14
. re-measurement over time shows a clear trend of increased maturity in the population of the thirteen firms re-measured thus far. On average. The raw score went up in eleven of the thirteen firms an average of 17 (a 33% average increase). the time between the two measurements was 20 months.The same sort of thing can be said regarding European + U. firms (thirteen) and US firms (thirty-eight). For an in depth treatment of the European market. in general.ly/TTBfX>.K.

BSIMM4 includes 51 firms. where five of the thirteen firms began executing these activities. four started and four stopped performing [T3. for a total set of 81 distinct measurements.2 Feed results to the defect management and mitigation system].6 Include security resources in onboarding].5 Identify metrics and use them to drive budgets].1 Publish data about software security internally].
15
BSIMM4
. and [CP3. Similarly.3 Drive feedback from SSDL data back to policy].3 Translate compliance constraints to requirements] and [PT1. eleven that had been re-measured. BSIMM2 included 30 firms and 42 distinct measurements (some firms include very large subsidiaries which were independently measured). where seven firms began executing this activity.
BSIMM Over Time
This is the fourth major release of the BSIMM project.2 Engage SSG with architecture]. BSIMM3 included 42 firms.Here are two ways of thinking about the change represented by the longitudinal scorecard. Less obvious from the scorecard is the “churn” among activities.4 Paper all vendor contracts with software security SLAs].5 Establish SSG office hours] and five started and two stopped performing [SM2. [SR2. where six firms began executing this activity. and [CP2. The original study included nine firms and nine distinct measurements. Observations between the four releases are shown side by side below. For example.5 Create SLA boilerplate]. Using the spider diagram.2 Track software bugs found in operations through the fix process]. and [CMVM2. [T3. There are ten activities that four of the firms undertook between measurements. while the sum count of firms remained basically the same for [SM2.1 Create a top N bugs list (real data preferred)]. [SFD1. We see the biggest changes in activities such as [CR1. [T2.4 Require an annual refresher]. five firms started executing this activity while four firms stopped doing this activity. we can plot the high watermarks of the first measurement of the thirteen firms against the second measurement of the thirteen firms. thirteen of which have been re-measured (with one firm measured for a third time). yielding a total set of 95 distinct measurements. [SR1.

2012
16
.

The BSIMM Community also hosts an annual private Conference where up to three representatives from each firm gather together in an off-the-record forum to discuss software security initiatives. working groups.. In 2011. and band together to solve hard problems (e. the first BSIMM Europe Community conference held in Amsterdam included 17 firms with a presence in the European market. The BSIMM website <http://bsimm. 21 of 42 firms participated in the second annual BSIMM Community Conference hosted at Skamania Lodge in Washington State. and mailing list initiated studies are posted.BSIMM Community
T
he fifty-one firms participating in the BSIMM Project make up the BSIMM Community. A moderated private mailing list with 130 members allows SSG leaders participating in the BSIMM to discuss solutions with others who face the same issues. In Spring of 2012.
17
BSIMM4
. discuss strategy with someone who has already addressed an issue. seek out mentors from those are farther along a career path.com> includes a credentialed BSIMM Community section where information from the Conferences.g. forming a BSIMM mobile security working group).

The two most important software security practices are architecture analysis and code review. and creating a standards review board. input validation. and so on). The security testing practice is concerned with pre-release testing including integrating security into standard quality assurance processes. building middleware frameworks for those controls. and tracking/measuring results. creating security standards for technologies in use. and code coverage analysis. identifying software security goals. This domain includes essential software security best practices that are integrated into the SDLC. Note that the BSIMM describes objectives and activities for each practice. application monitoring. and identifying metrics and gates. application of the attack model. To provide some idea of what a practice entails. setting organizational software security policy. Security testing focuses on vulnerabilities in construction. By contrast. Penetration testing focuses on vulnerabilities in final configuration. building standards for major security controls (such as authentication. assigning roles and responsibilities. applying lists of risks and threats. Web application firewalls. determining which COTS to recommend.There are three practices under each domain. Attack models capture information used to think like an attacker: threat modeling. change management. manual analysis. In the governance domain.
2012
20
. and ultimately code signing. the configuration management and vulnerability management practice concerns itself with patching and updating applications. and provides direct feeds to defect management and mitigation. customized profiles for tool use by different roles (for example. Finally. adopting a process for review (such as STRIDE or Architectural Risk Analysis). abuse case development and refinement. The standards and requirements practice involves eliciting explicit security requirements from the organization. developers versus auditors). the penetration testing practice involves more standard outside→in testing of the sort carried out by security specialists. The SSDL Touchpoints domain is probably the most familiar of the four. The security features and design practice is charged with creating usable security patterns for major security controls (meeting the standards defined in the next practice). and incident handling. Architecture analysis encompasses capturing software architecture in concise diagrams. determining budgets. The code review practice includes use of code review tools. developing contractual controls such as Service Level Agreements to help control COTS software risk. version control. and creating and publishing other proactive security guidance. Training has always played a critical role in software security because software developers and architects often start with very little security knowledge. installation and configuration documentation. and technology-specific attack patterns. The software environment practice concerns itself with OS and platform patching. and building an assessment and remediation plan for the organization. Those resources are divided into three practices. the strategy and metrics practice encompasses planning. risk driven white box testing. and auditing against that policy. defect tracking and remediation. The intelligence domain is meant to create organization-wide resources. The practice includes use of black box security tools (including fuzz testing) as a smoke test in QA. in the deployment domain. we include a short explanation of each. data classification. development of tailored rules. The compliance and policy practice is focused on identifying controls for compliance regimens such as PCI DSS and HIPAA.

Standardization Quality Control Quality Control. By understanding these goals. levels. Like domains. Here are the goals associated with each domain in the BSIMM:
Domain
Governance Intelligence SSDL Touchpoints Deployment
Business Goals
Auditability.BSIMM4
W
e present the maturity model as a series of activities associated with each of the twelve practices. At this level. there are three practices in each domain. Our approach is to identify goals for each level of a practice. you can quickly see why adopting some aspects of all four domains makes sense. a goal-based approach includes identifying initiative-level and domain-level goals as well as goals and objectives for practices. Take a look at the BSIMM skeleton if you need help understanding how the basic activities are organized. repeatable processes Improved code quality
Each of the four domains in the SSF has associated goals of its own. ignoring an entire domain would be folly. By understanding these goals. To risk repeating ourselves. Certainly. Accountability. Here are the goals associated with each practice in the BSIMM:
Domain
Governance
Practice
Strategy and Metrics Compliance and Policy Training
Business Goals
Transparency of expectations. Prescriptive guidance for all stakeholders Prescriptive guidance for all stakeholders Quality control Quality control Quality control Quality control Change management Change management
Intelligence
Attack Models Security Features and Design Standards and Requirements
SSDL Touchpoints
Architecture Analysis Code Review Security Testing
Deployment
Penetration Testing Software Environment Configuration Management and Vulnerability Management
21
. Error correction Customized knowledge Reusable designs. Stewardship. each practice has a high-level goal. and activities. Accountability for results Prescriptive guidance for all stakeholders. the top-level goals for the BSIMM writ large are: • • • • Informed risk management decisions Clarity on what is “the right thing to do” for everyone involved in software security Cost reduction through standard. From a top-down perspective. Checks and Balances
In the SSF. you can begin to think about which goals are the most important to your organization and its culture. Change Management
BSIMM4
Transparency. Characterizing and selling an initiative’s business goals is part of making a software security initiative a success. Auditability Knowledgeable workforce. which can be further split into objectives for the practice/level and are in this way associated with activities. you can see at a glance why adopting some aspects of all twelve practices makes sense.

2012
22
.

Socialize the gates. Most organizations pick and choose from a published methodology such as the Microsoft SDL or the Cigital Touchpoints and then tailor the methodology to their needs. with a signature.
BSIMM4
[SM1. Informal risk acceptance alone does not count as security sign off.3]
[SM1.
one
SM Level 1: Attain a common understanding of direction and strategy. or create a collection of papers. author white papers for internal consumption. They also learn what other organizations are doing to attain software security. For example. Ad hoc conversations between the SSG and executives or an SSG where “everyone is an evangelist” does not achieve the desired results. Executives learn about the consequences of inadequate software security and the negative business impact that poor security can have. In some cases. the methodology is published only internally and is controlled by the SSG.GOVERNANCE: Strategy and Metrics (SM)
The overall goals for the Strategy and Metrics practice are transparency of expectations and accountability for results. and other resources on an internal Web site and promote its use. such education arrives courtesy of malicious hackers or public data exposure incidents. This internal marketing function helps keep the organization current on the magnitude of the software security problem and the elements of its solution. actually showing working exploits and their business impact). as the act of accepting risk is more effective when it is formalized (e. The software security process will involve release gates/ checkpoints/milestones at one or more points in the SDLC or. The first two steps toward establishing release gates are: 1) to identify gate locations that are compatible with existing development practices and 2) to begin gathering the input necessary for making a go/no go decision. books. operating. someone in the SSG plays an evangelism role. plan). Goals.. A canonical example of such an evangelist is Michael Howard’s role at Microsoft.. Require security sign-off. executives come to support the software security initiative as a risk management necessity. evolve as necessary. In its most dangerous form. An SSDL process evolves as the organization matures and as the security landscape changes. gather necessary artifacts. or similar) and captured for future reference. Leaders must also ensure that the organization as a whole understands the strategy for achieving these objectives. The SSDL does not need to be publically promoted outside of the firm to count. deploying. form submission. more likely. A common strategic understanding is essential for effective and efficient program execution. the SSG demonstrates a worst-case scenario in a controlled environment with the permission of all involved (e. For example. The risk acceptor signs off on the state of the software prior to release. Bringing in an outside guru is often helpful when seeking to bolster executive attention.g. extend invitations to outside speakers. presentation to the Board can help garner resources for an ongoing software security initiative. Managers must ensure that everyone associated with creating. Preferably.4]
[SM1. In addition. By understanding both the problem and its proper resolution. responsibilities. In many cases. roles. Identify gate locations. but stop short of passing judgment on what constitutes sufficient testing or acceptable test results. the sign-off policy might require the head of the business unit to sign off on critical vulnerabilities that have not been mitigated or SSDL steps that have been skipped.1]
[SM1. responsibilities. The idea of identifying gates first and only enforcing them later is extremely helpful in moving development toward software security without major pain. the SSG can collect security testing results for each project prior to release.g. Importantly at this stage. Create evangelism role to perform internal marketing. and only turn them on once most projects already know how to succeed. and activities are explicitly defined. The organization has a process for security risk acceptance and documents accountability. Evangelists might give talks for internal groups. executive management must set specific objectives for all SSDL stakeholders and ensure that specific individuals are made accountable for meeting those objectives. This gradual approach serves to motivate good behavior without requiring it. Executive management must clarify organizational expectations for the SSDL so that everyone understands the importance of the initiative. Educate executives. and maintaining software understands the written organizational software security objectives.2]
[SM1. The process for addressing software security is broadcast to all participants so that everyone knows the plan. Publish process (roles. the SDLCs.6]
23
. In order to build support for software security throughout the organization. the gates are not enforced.

The SSG might write papers or books. The key here is to tie technical results to business objectives in a clear and obvious fashion in order to justify funding.1]
[SM2. are in turn responsible for ensuring successful performance of SSDL activities. Use an internal tracking application with portfolio view. Publishing information up to executives who then do something about it and drive change in the organization suffices. contractual agreements. but rather with the relevant executives only. and activities become more distributed. Enforce gates with measurements and track exceptions. Managers must explicitly identify individuals responsible for software security risk management accountability. To reduce unacceptable risk. Identify metrics and use them to drive budgets. in progress.7 Identify satellite through training]. Ongoing membership should be based on actual performance. Even recalcitrant project teams must now play along. In other cases. gates are directly associated with controls required by regulations. Software security grows beyond being a risk reduction exercise and becomes a competitive advantage or market differentiator. The SSG publishes data internally on the state of software security within the organization. The information might come as a dashboard with metrics for executives and software development management. The SSG tracks exceptions. A combined inventory and risk posture view is fundamental.
[SM2. The satellite begins as a collection of people scattered across the organization who show an above-average level of security interest or skill. Application owners and the SSG must inform management of the risk associated with each application in the portfolio. SDLC security gates are now enforced: in order to pass a gate. and other business obligations and exceptions are tracked as required by statutory or regulatory drivers. In many cases. In some cases. It incorporates results from activities such as architecture analysis. The SSG and its management choose the metrics that define and measure software security initiative progress. A strong satellite is a good sign of a mature software security initiative. The SSG must advertise its activities externally to create support for its approach and enable ecosystem security. with the philosophy that sunlight is the best disinfectant. a project must either meet an established measure or obtain a waiver. and security testing.two
SM Level 2: Align behavior with strategy and verify adherence. SM Level 3: Practice risk-based portfolio management. Identifying this group is a step towards creating a social network that speeds the adoption of security into software development. Create or grow a satellite.3]
[SM2. A reduction in security defect density could be used to show a decreasing cost of remediation over time. this can cause interesting effects through internal competition. These metrics will drive the initiative’s budget and allocation of resources. The SSG markets the software security initiative outside the firm to build external support. It might have a public blog.) Another way is to ask for volunteers. managers must identify and encourage the growth of a software security satellite (see the Roles section above). It might participate in external conferences or trade shows.1]
[SM3. a complete SSDL methodology can be published and promoted externally. The application records the security activities scheduled. Metrics also allow the SSG to explain its goals and its progress in quantitative terms. As an initiative matures. The SSG uses the tracking application to generate portfolio reports for many of the metrics it uses. The SSG uses a centralized tracking application to chart the progress of every piece of software in its purview. In some cases. A gate could require a project to undergo code review and remediate any critical findings before release. One way to begin is to track the people who stand out during introductory training courses. Depending on the culture. (See [T2.5]
three
[SM3. One such metric could be security defect density. Sharing details externally and inviting critique can bring new perspectives into the firm. In other cases. this information adds a security dimension to the game. gate measures yield key performance indicators that are used to govern the process. code review. Since the concept of security is already tenuous to business people. open book management and publishing data to all stakeholders helps everyone know what’s going on. If the organization’s culture promotes internal competition between groups. and completed. the SSG uses the centralized reporting system to keep track of all of the moving parts. Publish data about software security internally. Run an external marketing program. initial satellite membership is assigned to ensure complete coverage of all development/product groups. making this explicit tie can be very helpful.2]
2012
24
. SSDL managers must ensure quick identification and modification of any SSDL behavior resulting in unacceptable risk. These individuals. publication is not shared with everyone in a firm. these data are published at least among executives. In a more top down approach.2]
[SM2. Sometimes.

Note that outsourcing to hosted environments (e. including vendors. SAS 70. existing business processes run by legal or other risk and compliance groups outside the SSG may also serve as the regulatory focal point. if the organization processes credit card transactions. policy that prescribes and mandates the use of coding guidelines and architecture standards for certain categories of applications does count. The policy provides a unified approach for satisfying the (potentially lengthy) list of security drivers at the governance level. Unify regulatory pressures. or through particular types of PII and the applications that touch them.g. The goal of this activity is to create one set of guidance so that compliance work is completed as efficiently as possible (mostly by removing duplicates). Management-approved prescriptive guidance must be available to all SSDL stakeholders.
[CP1. regulatory. Create policy. the SSG will identify the constraints that the PCI DSS places on the handling of cardholder data. for use in meeting security and compliance objectives. The SSG and application owners must ensure service level agreements address security properties of vendor software deliverables. Identify PII data inventory. For example. Architecture standards and coding guidelines are not examples of software security policy. GLBA. SOX. OCC. All SSDL activities must produce artifacts sufficient to allow auditing for adherence to prescriptive guidance. The SSG takes a lead role in identifying PII obligations stemming from regulation and customer expectations. backed by executives. the SSG acts as a focal point for understanding the constraints such drivers impose on software.1]
[CP1.3]
two
[CP2. this inventory guides privacy planning. firms that create software products that process PII (but don’t necessarily handle PII directly) may provide privacy controls and guidance for their customers. and contractual compliance drivers. The SSG guides the rest of the organization by creating or contributing to software security policy that satisfies regulatory and customer-driven security requirements. The SSG must work with appropriate groups to capture unified compliance requirements in prescriptive guidance and make that knowledge available to SSDL stakeholders. The way software handles personally identifiable information (PII) could well be explicitly regulated. HIPAA or others. In some cases. Likewise.
one
CP Level 1: Document and unify statutory. Risk managers must explicitly take responsibility for software risk. As a result. As an alternative. project teams don’t need to re-learn customer security requirements on their own. the SSG can now create a list of databases that would require customer notification if breached. but even if it is not. On the other hand. Identify PII obligations. project teams can avoid learning the details involved in complying with all applicable regulations. It uses this information to promote best practices related to privacy. Some firms move on to guide exposure by becoming directly involved in standards groups in order to influence the regulatory environment. Policy is what is permitted and denied at the initiative level. the SSG creates a unified approach that removes redundancy from overlapping compliance requirements. A PII inventory can be approached in two ways: through each individual application by noting its PII use. privacy is a hot topic. Executives must overtly promote the SSG and associated software security initiative. Also note.2]
[CP1. The SSG policy documents are sometimes focused around major compliance topics such as the handling of PII or the use of cryptography. A formal approach will map applicable portions of regulations to control statements explaining how the organization complies.. If the business or its customers are subject to regulatory or compliance drivers such as FFIEC.1]
25
BSIMM4
. the cloud) does not relax a majority of PII obligations. When combined with the organization’s PII obligations.GOVERNANCE: Compliance and Policy (CP)
The overall goals for the Compliance and Policy practice are prescriptive guidance for all stakeholders and auditability of SSDL activities. including the need for compliance. PCI DSS. For example. The organization identifies the kinds of PII stored by each of its systems. CP Level 2: Align internal practices with compliance drivers and policy. policy documents relate directly to the SSDL and its use in the firm. In some cases.

and operational issue data drive policy evolution and demands on vendors. auditors.[CP2.) Ultimately policies align themselves with the SSDL data and enhance and improve a firm’s effectiveness. For example.1 Unify regulatory pressures]. and exceptions should be tracked. Executives must ensure that software security policy is periodically updated based on actual data and must demonstrate the organization’s ongoing compliance. and artifacts gathered through the SSDL gives the SSG the ability to demonstrate the organization’s compliance story without a fire drill for every audit. defect. Policies are improved to find defects earlier or prevent them from occurring in the first place. Create regulator eye-candy. or choosing the wrong firm to carry out a penetration test may expose policy weakness. Blind spots are eliminated based on trends in SSDL failures. inadequate architecture analysis.5 Create SLA boilerplate]. Vendor contracts include a service-level agreement (SLA) ensuring that the vendor will not jeopardize the organization’s compliance story and software security initiative. regulators. The SSG has the information regulators want.2]
[CP3.) The SSG tracks the controls. The SSG gains executive buy-in around compliance and privacy activities. For some organizations. ignored security gates. Vendors are required to adhere to the same policies used internally. plan). responsibilities. having an outside expert address the Board works because some executives value outside perspective over internal perspective. the SSG could be forced to take a more active role as referee. The organization can demonstrate compliance with applicable regulations because its practices are aligned with the control statements developed by the SSG. Vendors may also attest to the fact that they are carrying out certain SSDL processes.3]
[CP2. Implement and track controls for compliance. One sure sign of proper executive awareness is adequate allocation of resources to get the job done. If the SDLC is uneven or less reliable. If the organization’s SDLC is predictable and reliable. open source licensing concerns initiate the vendor control process. (See [SM1. and makes sure auditors and regulators are satisfied. Information from the SSDL is routinely fed back into the policy creation process.4]
[CP2. Drive feedback from SSDL data back to policy. CP Level 3: Organizational threat. Vendors must submit evidence that their software security practices pass muster. Promote executive awareness of compliance and privacy obligations. A combination of written policy. The SSG. attack. Paper all vendor contracts with software security SLAs. explaining the direct cost and likely fallout from a data breach could be an effective way to broach the subject. and senior management are satisfied with the same kinds of reports. For example.5]
three
[CP3. Over time. Impose policy on vendors. shepherds problem areas. That can open the door for further software security language in the SLA. a BSIMM score or a vBSIMM score has been used to help ensure that vendors are complying with the firm’s policies. the sign-off policy might require the head of the business unit to sign off on compliance issues that have not been mitigated or SSDL steps related to compliance that have been skipped.
[CP2. The organization has a formal process for compliance risk acceptance and accountability.1 Publish process (roles. A firm doing this properly can explicitly tie satisfying its compliance concerns to its SSDL. controls documentation.1]
[CP3. Sign off should be explicit and captured for future reference. application owners. and legal groups must ensure vendors deliver software that complies with relevant organizational policy. evolve as necessary]. policies should become more practical and easier to carry out.) In some cases. (See [SR2. (See [CP1.2]
Require security sign-off for compliance-related risk. Executives are provided plain-language explanations of the organization’s compliance and privacy obligations and the potential consequences for failing to meet those obligations. recurring vulnerabilities. the SSG might be able to largely sit back and keep score. The risk acceptor signs off on the state of the software prior to release. For other organizations. In some cases. which may be generated directly from various tools. Each new or renewed contract contains a set of provisions requiring the vendor to deliver a product or service compatible with the organization’s security policy. Evidence could include code review results or penetration test results.3]
2012
26
. In some cases.

and project managers could attend the same Introduction to Software Security course.7]
two
[T2. including training and related events. In order to make a strong and lasting change in behavior. developers. The satellite learns about advanced topics or hears from guest speakers.
BSIMM4
[T1.
one
T Level 1: Make customized. The SSG must build interest in software security throughout the organization and provide role-specific training material. The SSG provides awareness training in order to promote a culture of security throughout the organization. Training must include specific information on root causes of errors discovered in process activities and outputs. trainees get information on the tools. all programmers. which is as much about building camaraderie as it is about sharing knowledge or organizational efficiency. by an outside firm. training includes material specific to the company’s history. The SSG must build and enhance a satellite through social activities. This common activity can be enhanced with a tailored approach to an introductory course that addresses a firm’s culture explicitly. For developers. Stories from company history can help steer training in the right direction only if the stories are still relevant. technology stacks. Enhance satellite through training and events. one for Java developers. Likewise. An organization might offer four tracks for engineers: one for architects. A standing conference call meeting does not address this activity. Don’t forget that training will be useful for many different roles in an organization. The SSG strengthens its social network by holding special events for the satellite. One way to do this is to use noteworthy attacks on the company as examples in the training curriculum. The workforce must have role-based knowledge that specifically includes the skills required to adequately perform their SSDL activities. Computer-based training (CBT) is the most obvious choice and can be kept up to date through a subscription model. Offer on-demand individual training. or kinds of bugs that are most relevant to them. that incorporates lessons from actual internal events.6]
[T1. Generic introductory courses covering basic IT security and high level software security concepts do not generate satisfactory results. and others. including computer-based training. Course content is not necessarily tailored for a specific audience. Training might be delivered by members of the SSG. providing awareness training only to developers and not to other roles is also insufficient. product management. For example.1]
[T1. and a fourth for testers. including QA. or through a computer-based training system. Offer role-specific advanced curriculum (tools. they are more likely to understand how the material is relevant to their work and to know when and how to apply what they have learned. one for . Offering pizza and beer doesn’t hurt. Remember that in some cases. CBT courses must be engaging and relevant to achieve their intended purpose. it is also possible to provide training directly through IDEs right at the time it’s needed. by the internal training organization. The training is tailored to the role of trainees. Tool-specific training is also commonly observed in a curriculum. even if they happen only once or twice a year. Software security training goes beyond building awareness and enables trainees to incorporate security practices into their work. executives. Provide awareness training. Create and use material specific to company history. There is no substitute for face-to-face meetings. T Level 2: Create the software security satellite.NET. bug parade). The organization lowers the burden on trainees and reduces the cost of delivering training by offering on-demand training for individuals. When participants can see themselves in the problem. building a new skill (such as code review) may be better suited for instructor-led training.5]
27
. Be wary of training that covers platforms not used by developers (Windows developers don’t care about old Unix problems) or examples of problems only relevant to languages no longer in common use (Java developers don’t need to understand buffer overflows in C).GOVERNANCE: Training (T)
The overall goals for the Training practice are the creation of a knowledgeable workforce and correcting errors in processes. quality assurance engineers. role-based training available on demand.5]
[T1. The SSG and managers must ensure that new hires are exposed to the corporate security culture during onboard activities. technology stacks.

The satellite begins as a collection of people scattered across the organization who show an above-average level of security interest or skill. for those seeking software security guidance. The organization highlights its security culture as a differentiator by hosting external security events.5]
2012
28
. Of course. The refresher keeps the staff up-to-date on security and ensures the organization doesn’t lose focus due to turnover. application owners.4]
[T3. Turnover in engineering organizations is generally high. Everyone involved in making software is required to take an annual software security refresher course. The SSG must be available. Knowledge is its own reward. Provide training for vendors or outsourced workers. Management and the SSG must ensure that all staff members receive appropriate recognition for advancement through the training curriculum. In the best case. but progression through the security curriculum brings other benefits too. The objective is to ensure that new hires enhance the security culture. The process for bringing new hires into the engineering organization includes a module on software security. Training individual contractors is much more natural than training entire outsource firms and is a reasonable way to start. it is important to train everyone who works on your software regardless of their employment status. One way to begin is to track the people who stand out during training courses. it does not take the place of a timely and more complete introductory software security course. The SSG might use half a day to give an update on the security landscape and explain changes to policies and standards. Reward progression through curriculum (certification or HR). Host external software security events. Identify satellite through training. a volunteer army may be easier to lead than one that is drafted. A refresher can be rolled out as part of a firm-wide security day or in concert with an internal security conference.2 Run an external marketing program]. The organization as a whole benefits from putting its security cred on display.3]
[T3.6]
Include security resources in onboarding. but this is enhanced further to cover topics such as secure coding.) In general. Involving a corporate training department and/or HR can make security’s impact on career progression more obvious. Identifying this group is a step towards creating a social network that speeds the adoption of security into software development. with visits to particular product or application groups slated by request. (See [SM2.3 Create or grow a satellite]. the SSG leverages teachable moments and emphasizes the carrot over the stick. Employees benefit from hearing outside perspectives. By acting as an informal resource for people who want to solve security problems. but the SSG should continue to monitor security knowledge in the firm and not cede complete control or oversight. Managers.2]
[T3.[T2. Though a generic onboarding module is useful. as is Intel’s Security Conference. and internal security resources. The SSG offers help to any and all comers during an advertised lab period or regularly scheduled office hours. Establish SSG office hours. and the SSG must provide training to vendors and outsource workers as a method of spreading the security culture. Managers must ensure that all staff members receive this training at least annually. Microsoft’s BlueHat is such an event. Also build morale.) Require an annual refresher. The organization offers security training for vendors and outsource providers. the SSDL. at least periodically.7]
three
[T3. The reward system can be formal and lead to a certification or official mark in the HR system. Developers and testers see a career advantage in learning about security. The generic new hire process covers things like picking a good password and making sure people don’t tail you into the building. Mobile office hours are also a possibility. Office hours might be held one afternoon per week in the office of a senior SSG member. Managers and the SSG must continue to bolster satellite momentum by marketing the security culture externally. or it can be less formal and make use of motivators such as praise letters for the satellite written just before annual review time. T Level 3: Provide recognition for skills and career path progression. Spending time and effort helping suppliers get security right is easier than trying to figure out what they screwed up later on. outsourced workers receive the same training given to employees.
[T2.1]
[T3. (See [SM3.

(See [SR1. Vigilance means never getting too comfortable.6]
29
BSIMM4
. Gather attack intelligence. In many cases. engage with the security researchers who are likely to cause you trouble. For example.4]
[AM1. mailing lists. This list combines input from multiple sources: observed attacks. the SSG collects and publishes stories about attacks against the organization. Build an internal forum to discuss attacks. and reading relevant publications.INTELLIGENCE: Attack Models (AM)
The overall goal for the Attack Models practice is the creation of customized knowledge on attacks relevant to the organization. Depending upon the scheme and the software involved. etc. Many classification schemes are possible—one approach is to focus on PII. In some cases. Build and maintain a top N possible attacks list. The SSG must identify potential attackers and document both the attacks that cause the greatest organizational concern and any important attacks that have already occurred.)
[AM1. Collect and publish attack stories.3]
[AM1. a third-party vendor may be contracted to provide this information. Both successful and unsuccessful attacks can be noteworthy. The outcome of this exercise could be a set of attacker profiles including generic sketches for broad categories of attackers and more detailed descriptions for noteworthy individuals. the satellite. The SSG must communicate attacker information to all interested parties.2 Create a security portal]. and blogs. Simply republishing items from public mailing lists does not achieve the same benefits as active discussion. data may be classified according to protection of intellectual property. Make Sun Tzu proud by knowing your enemy. monitoring attacker forums. the SSG might brainstorm twice per year to create lists of attacks the organization should be prepared to counter “now. Discussing historical information about software attacks has the effect of grounding software security in the reality of a firm. attack model information is used in a list-based approach to architecture analysis.
one
AM Level 1: Create attack and data asset knowledge base. relevance to SOX.5]
[AM1. Customized knowledge must guide decisions about both code and controls. Dissection of attacks and exploits that are relevant to a firm can be particularly helpful. or geographic boundaries. For example. The information comes from attending conferences and workshops. This allows applications to be prioritized by their data classification. especially if they spur discussion of development mitigations.” In some cases. This is particularly useful in training classes in order to counter a generic approach over-focused on top ten lists or irrelevant and outdated platform attacks. The forum serves to communicate the attacker perspective. helping to focus the analysis as in the case of STRIDE. Create a data classification scheme and inventory. Specific and contextual attacker information is almost always more useful than generic information copied from someone else’s list. industry trends. The SSG helps the organization understand attack basics by maintaining a list of attacks most important to the firm. The SSG identifies potential attackers in order to understand their motivations and capabilities. exposure to attack. Regardless of its origin. then derive classifications for applications according to the repositories they use. The organization has an internal forum where the SSG. attack information must be made actionable and useful for software builders and testers. The organization agrees upon a data classification scheme and uses the scheme to inventory its software according to the kinds of data the software handles.” “soon. and others discuss attacks. a subscription to a commercial service provides a reasonable way of gathering basic attack intelligence. The SSG stays ahead of the curve by learning about new types of attacks and vulnerabilities.1]
[AM1. hacker forums.2]
[AM1. In order to maximize the benefit from lessons that do not always come cheap.” and “someday. Hiding information about attacks from people building new systems does nothing to garner positive benefit from a negative happenstance. The SSG could maintain an internal mailing list where subscribers share the latest information on publicly known incidents. this collection helps the organization understand its history. Other approaches to the problem are possible. The business must create a data classification scheme that the SSG uses to inventory and prioritize applications. Over time. The list does not need to be updated with great frequency and the attacks can be sorted in a coarse fashion. Identify potential attackers. it could be easiest to first classify data repositories. impact of disclosure.

Attack patterns must include technology-specific information relevant to the organization. this information can be used to guide the process of building attack patterns and abuse cases.1]
[AM2. Build attack patterns and abuse cases tied to potential attackers. For example. The SSG must provide knowledge and automation to auditors and testers to ensure their activities reflect actual attacks perpetrated against the organization’s software as well as potential attacks. the SSG could catalogue the quirks of all the popular browsers and how they might be exploited. if the organization’s Web software relies on cutting-edge browser capabilities. a story about an attack against poorly managed entitlements could lead to an entitlements attack pattern that drives a new type of testing. The SSG must gather attack intelligence and expand its attack knowledge to include both higher-level attack patterns and lower-level abuse cases. The SSG must conduct attack research on corporate software to get ahead of attacker activity.g..2]
three
[AM3. Create and use automation to do what attackers will do. A science team may include well-known security researchers who publish their findings at conferences like Def Con. Create technology-specific attack patterns.two
AM Level 2: Provide outreach on attackers and relevant attacks. AM Level 3: Research and mitigate new attack patterns. “Ensure data are protected in transit”) and adding “for mobile applications” on the end does not constitute technology-specific attack patterns. The idea here is to push attack capability past what typical commercial tools and offerings encompass and then package that information for others to use. The SSG creates technology-specific attack patterns to capture knowledge about attacks that target particular technologies. These resources do not have to be built from scratch for every application in order to be useful.2]
2012
30
.1]
[AM3. a new attack method identified by the science team could require a new tool. The SSG prepares for security testing and architecture analysis by building attack patterns and abuse cases tied to potential attackers. Tailoring tools to a firm’s particular technology stacks and potential attackers is a really good idea. For example. The SSG will add to the pile based on attack stories. For example. The SSG has a science team that works to identify and defang new classes of attacks before real attackers even know they exist. Attack patterns directly related to the security frontier (currently mobile security and cloud security) may be useful. The SSG arms testers and auditors with automation to do what attackers are going to do. there could be standard sets for applications with similar profiles. Instead. This is not a penetration testing team finding new instances of known types of weaknesses—it is a research group innovating new types of attacks.
[AM2. The SSG packages the new tool and distributes it to testers. If a firm tracks fraud and monetary costs associated with particular attacks. Simply republishing general guidelines (e. Have a science team that develops new attack methods.

In addition to teaching by example. and patterns. accidental noticing is not sufficient.1]
SFD Level 1: Publish security features and architecture. role management. The SSG takes a proactive role in software design by building or providing pointers to secure-by-design middleware frameworks or common libraries. (See [CR3.
one
[SFD1. Create SSG capability to solve difficult design problems. audit/ log. the SSG could modify a popular Web framework such as Struts to make it easy to meet input validation requirements.3]
31
BSIMM4
. When the SSG is involved early in the new project process.) When adopting a middleware framework (or any other widely used software). including OWASP ESAPI.1 Use automated tools with tailored rules].INTELLIGENCE: Security Features and Design (SFD)
The overall goal for the Security Features and Design practice is the creation of customized knowledge on security features. The customized knowledge must drive architecture and component decisions. The SSG must provide secure-by-design frameworks along with additional mature design patterns taken from existing software and technology stacks. he or she could analyze the security implications of existing protocols and identify elements that should be duplicated or avoided. The SSG must provide architects and developers with guidance on security features and participate directly with architecture groups. etc. cryptography. Find and publish mature design patterns from the organization. The SSG fosters centralized design reuse by finding and publishing mature design patterns from and throughout the organization. The SSG can identify an implementation they like and promote it as the accepted solution. this middleware aids architecture analysis and code review because the building blocks make it easier to spot errors. availability. or scalability.2]
two
[SFD2. Rather than have each project team implement all of their own security features (authentication. it contributes to new architecture and solves difficult design problems. should not be considered secure out of the box. Security is a regular part of the organization’s software architecture discussion. key management. Encouraging adoption and use of insecure middleware does not help the software security situation. Engage SSG with architecture. The negative impact security has on other constraints (time to market. In other cases. Eventually the SSG can tailor code review rules specifically for the components it offers. price. For example. frameworks. An ad hoc. One way to keep security from falling out of the discussion is to have an SSG member attend regular architecture meetings. Some problems are best solved only once. Designing for security up front is more efficient than analyzing an existing design for security and then re-factoring when flaws are uncovered. a central architecture or technology team facilitates and enhances this activity.) is minimized. protocols). the SSG provides proactive guidance by building and publishing security features for other groups to use. Generic open source software security architectures. Some design problems will require specific expertise outside of the SSG. The SSG must be available for and capable of solving design problems for others. SFD Level 2: Build and identify security solutions. The architecture group takes responsibility for security the same way they take responsibility for performance. A section of the SSG Web site could promote positive elements identified during architecture analysis so that good ideas are spread. Build and publish security features.1]
[SFD2. enterprise architecture can help the SSG create secure designs that integrate properly into corporate design standards.
[SFD1. In some cases. careful vetting for security before publication is important. Project teams benefit from implementations that come pre-approved by the SSG. Build secure-by-design middleware frameworks and common libraries.2]
[SFD2. This process should be formalized. If an architect from the SSG is involved in the design of a new protocol. and the SSG benefits by not having to repeatedly track down the kinds of subtle errors that often creep into security features.

1Perform security feature review].2]
2012
32
. In particular. Require use of approved security features and frameworks.three
SFD Level 3: Actively reuse approved security features and secure-by-design frameworks. Managers must also require that defined security features and frameworks be used whenever possible. and review teams do not have to contend with finding the same old defects in brand new projects. Managers must ensure there is formal consensus across the organization on secure design choices. (See [AA1. Implementers must take their security features and frameworks from an approved list. A review board or central committee formalizes the process for reaching consensus on design needs and security tradeoffs. Form a review board or central committee to approve and maintain secure design patterns.
[SFD3.1]
[SFD3. Unlike the architecture committee.) Re-use is a major advantage of consistent software architecture. the easier architecture analysis and code review become. The group can also periodically review already-published design standards (especially around cryptography) to ensure that design decisions do not become stale or out of date. this group is specifically focused on providing security guidance. the more a project uses proven components. There are two benefits: developers do not spend time re-inventing existing capabilities.

SR Level 2: Communicate formally-approved standards internally and to vendors. People refer to the site for the latest and greatest on security standards and requirements as well as other resources provided by the SSG.1]
33
BSIMM4
. communication regarding software security expectations is easier.1 Build and publish security features] for one case where the SSG provides a reference implementation of a security standard. The SSG must ensure that all open source software is identified in the organization’s code. Representing these standards as requirements helps with traceability and visibility in the case of audit. secure coding standards. PCI DSS compliance could play a role in the SSDL during the requirements phase. In some cases. and explains in concrete terms (rather than legalese) what the organization expects of the vendor. standards and guidelines can be automated in development environments (e.
[SR1. A healthy relationship with a vendor cannot be guaranteed through contract language alone. Any time a vendor adopts the organization’s security standards. if the organization routinely builds software that processes credit card transactions. and compliance requirements. Managers must ensure that a formal process is used to create standards specific to technology stacks. guidelines can be explicitly linked to code examples to make them more actionable and relevant. A clear set of secure coding standards is a good way to guide both manual and automated code review.1]
[SR1. Software security requires much more than security features. Create secure coding standards. the SSG. the secure coding standards should build upon them. In other cases. Compliance constraints are translated into software requirements for individual projects. When a firm’s SSDL is available publically. The SSG must provide foundational knowledge. (See [SFD1.INTELLIGENCE: Standards and Requirements (SR)
The overall goal for the Standards and Requirements practice is to create prescriptive guidance for all stakeholders..g. An interactive wiki is better than a static portal with guideline documents that only rarely change.4]
two
[SR2. In other cases. Typically this is an internal Web site maintained by the SSG. and product owners must ensure all applicable standards are communicated to third-party vendors and that these standards and other SLAs are reinforced by contractual language approved by legal staff. Managers must ensure that software security information is kept up-to-date and made available to everyone. Communicate standards to vendors. including external parties. If the organization already has coding standards for other purposes. Create a security portal.
one
SR Level 1: Provide easily accessible security standards and requirements. Organizations can enhance these materials with mailing lists and face-to-face meetings. Secure coding standards are necessarily specific to a programming language and can address the use of popular frameworks and libraries. including at the very least: security standards. discusses the vendor’s security practices. but security features are part of the job as well. Translate compliance constraints to requirements. technology standards built for international interoperability reasons can include security guidance.) Standards can be deployed in a variety of ways. Secure coding standards help developers avoid the most obvious bugs and provide ground rules for code review. For example. This is a linchpin in the organization’s compliance strategy—by representing compliance constraints explicitly with requirements. A standard might describe how to perform authentication using J2EE or how to determine the authenticity of a software update. Managers. Likewise.2]
[SR1. demonstrating compliance becomes a manageable task. The SSG meets the organization’s demand for security guidance by creating standards that explain the accepted way to adhere to policy and carry out specific security-centric operations. The organization has a central location for information about software security. as well as beefing up security training with relevant examples. The SSG engages with vendors. it’s a clear win. sharing internal practices and measures (including a BSIMM measurement) can make expectations very clear. The SSG works with vendors to educate them and promote the organization’s security standards.3]
[SR1. Managers and the SSG must document software security choices and convey this material to everyone involved in the SSDL. Create security standards. worked into an IDE).

Create SLA boilerplate. The review board could operate by appointing a champion for any proposed standard. The SSG works with the legal department to create standard SLA boilerplate for use in contracts with vendors and outsource providers. a database.[SR2. this activity is subsumed by a policy constraining the use of open source. Create standards for technology stacks.
[SR2. and a runtime environment for a managed language. an application server. Identify open source. It could also be restricted to open source versions that have been through an SSG security screening process. It is not uncommon to discover old versions of components with known vulnerabilities or multiple versions of the same component. Getting legal to understand security risks can help move an organization to practice decent open source hygiene. Ideally. For the SSG.4 Paper all vendor contracts with software security SLAs]. The organization creates a standards review board to formalize the process used to develop standards and ensure that all stakeholders have a chance to weigh in. Legal often spearheads additional open source controls due to the “viral” license problem associated with GPL code. and made available only through internal repositories. The first step toward managing risk introduced by open source is to identify the open source components in use. The legal department understands that the boilerplate helps prevent compliance or privacy problems. Managers and the SSG must show that any open source code used in the organization is subject to the same risk management processes as code created internally. Enterprise architecture or enterprise risk groups sometimes take on the responsibility of creating and managing standards review boards. The organization uses standard technology stacks. SR Level 3: Require risk management decisions for open source use. further reducing the amount of work required to use the stack safely.1]
2012
34
. this means a reduced workload because the group does not have to explore new technology risks for every new project.) Boilerplate language may call out software security vendor control solutions such as vBSIMM measurements or BSIMM scores. Currently. vendors and outsource providers must meet company software security standards. The onus is on the champion to demonstrate that the standard meets its goals and to get approval and buy-in from the review board. (See [CP2.4]
[SR2. Use of open source could be restricted to pre-defined projects. The organization has control over its exposure to the vulnerabilities that come along with using open source components. had unacceptable vulnerabilities remediated. mobile technology stacks and platforms as well as cloud-based technology stacks are two areas where specific attention to security pays off. A process that relies solely on developers asking for permission does not generate satisfactory results.5]
three
[SR3.3]
[SR2. the organization will create a secure base configuration for each technology stack. At the next level of maturity. Automated tools for finding open source are one way to approach this activity.2]
Create a standards review board. Under the agreement. Control open source risk. A stack might include an operating system. The security frontier is a good place to find traction.

” “Who uses the application?. Have SSG lead review efforts. medium. it is important that some spot checking for validity and accuracy be put in place.1]
[AA1. or low risk. For example. The SSG takes a lead role in performing architecture analysis in order to begin building the organization’s ability to uncover design flaws. Architecture analysis is enough of an art that the SSG needs to be proficient at it before they can turn the job over to the architects and proficiency requires practice. In some cases.” and “Does the application handle PII?” A qualified member of the application team completes the questionnaire. To facilitate the architecture analysis and other processes. The SSG must begin leading architecture analysis efforts. Perform security feature review. The organization learns about the benefits of architecture analysis by seeing real results for a few high-risk. Questions might include. Use a risk questionnaire to rank applications. access control. led by the SSG. The questionnaire is short enough to be completed in a matter of hours. “Which programming languages is the application written in?. At higher levels of maturity. center the analysis process on a review of security features. At higher levels of maturity. use of cryptography. To get started with architecture analysis.
one
AA Level 1: Perform risk-driven AA reviews. it uses consultants to do this work.SSDL TOUCHPOINTS: Architecture Analysis (AA)
The overall goal of the Architecture Analysis practice is quality control.
[AA1. Perform design review for high-risk applications. Security-aware reviewers first identify the security features in an application (authentication. the SSG might carry out the analysis with a minimum of interaction with the project team.3]
[AA1. With a clear design in hand.) then study the design looking for problems that would cause these features to fail at their purpose or otherwise prove insufficient. particularly on high-risk applications. etc. this activity is eclipsed by a more thorough approach to architecture analysis not centered on features. The SSG cannot be successful on its own either—they will likely need help from the architects or implementers in order to understand the design. The reviewers must have some experience performing architecture analysis and understand the architecture being considered. as a way to build internal capability and demonstrate value at the design level. Those performing architecture analysis must ensure the detection and correction of security flaws. though in the long run they do not scale. Because a risk questionnaire can be easy to game. The SSG might use the answers to bucket the application as high. the responsibility for leading review efforts shifts towards software architects. Ad hoc review paradigms that rely heavily on expertise may be used here.4]
35
BSIMM4
. use of the firm’s secure-by-design components can streamline this process. Software architects must enforce adherence to standards and the reuse of approved security features. high-profile applications. If the SSG is not yet equipped to perform an in-depth architecture analysis.2]
[AA1. An over-reliance on self-reporting or automation can render this activity impotent. The organization must provide a lightweight software risk classification. the SSG uses a risk questionnaire to collect basic information about each application so that it can determine a risk classification and prioritization scheme. Do not expect to set a process and use it forever. Approaches to architecture analysis (and threat modeling) evolve over time. a system that was subject to escalation of privilege attacks because of broken access control or a system that stored unsalted password hashes would both be identified in this kind of review.

consistency is very difficult to attain because architecture analysis requires so much experience. Visio templates. Particular attention should be paid to documentation of both the architecture under review and any security flaws uncovered. and whiteboard squiggles are especially useful. Have software architects lead review efforts. Note that even these two methodologies for architecture analysis have evolved greatly over time. This format. In order to build an architecture analysis capability outside of the SSG.
[AA2. The SSG might still contribute to architecture analysis in an advisory capacity or under special circumstances. Microsoft’s STRIDE and Cigital’s ARA are examples of such a process. The SSG defines and documents a process for performing architecture analysis and applies it in the design reviews it conducts. the SSG advertises itself as a resource or mentor for teams who ask for help conducting their own analysis and proactively seek projects to get involved with. makes architecture analysis tractable for people who are not security experts. The SSG will answer architecture analysis questions during office hours and in some cases might assign someone to sit side-by-side with the architect for the duration of the analysis. the SSG plays a more active mentorship role. A standard architecture description can be enhanced to provide an explicit picture of information assets that require protection. Define and use AA process. The process is defined rigorously enough that people outside the SSG can be taught to carry it out. including a means for representing data flow. Software architects throughout the organization lead the architecture analysis process most of the time. Make sure to access up-to-date sources for architecture analysis information as many early publications are outdated and no longer apply. This activity requires a well understood and well documented architecture analysis process. (See [SFD3. Tribal knowledge does not count as a defined process. The SSG must define an architecture analysis process based on a common architecture description language and standard attack models.1 Form a review board or central committee to approve and maintain secure design patterns].2]
2012
36
. Standardize architectural descriptions (including data flow). The architecture analysis process should be applied even when vetted design patterns are in standard use. The organization uses an agreed-upon format for describing architecture. The process includes a standardized approach for thinking about attacks and security properties. Software architects must lead analysis efforts across the organization and must use analysis results to update and create standard architecture patterns that are secure.3]
three
[AA3. Make SSG available as AA resource or mentor. Standardized icons that are consistently used in UML diagrams. Drive analysis results into standard architecture patterns. Failures identified during architecture analysis are fed back to the security design committee so that similar mistakes can be prevented in the future through improved design patterns. combined with an architecture analysis process.two
AA Level 2: Provide outreach on use of documented AA process. In the case of high risk applications or products.1]
[AA3. AA Level 3: Build review and remediation capability within the architecture group.2]
[AA2. Even in that case.1]
[AA2.) Security design patterns can interact in surprising ways that break security. The SSG must facilitate organization-wide use of architecture analysis by making itself available as a resource and mentor.

the review process might be different for different kinds of projects.) Code review information can be incorporated into a CSO-level dashboard that includes feeds from other parts of the security organization. The review for low-risk projects might rely more heavily on automation and the review for high-risk projects might have no upper bound on the amount of time spent by reviewers. a code review gate with a minimum acceptable standard forces projects that do not pass to be fixed and re-evaluated before they ship. (For another way to use the list. The SSG can use the reports to demonstrate progress and drive the training curriculum. The SSG must perform code reviews on high-risk applications whenever it can get involved in the process and must use the knowledge gained to inform the organization of the types of bugs being discovered.1]
[CR1.
[CR1. The list helps focus the organization’s attention on the bugs that matter most. Have SSG perform ad hoc review. The SSG must enforce use of centralized tools reporting to capture knowledge on recurring bugs and push that information into strategy and training. While all projects must undergo code review. In most cases. A firm may use an external service vendor as part of a formal code review process for software security. not constraining themselves to a particular service or tool.4]
[CR1.6 Create and use material specific to company history]. (See [SM2.5 Identify metrics and use them drive budgets]. The bugs found during code review are tracked in a centralized repository. A generic list could be culled from public sources. Make code review mandatory for all projects. The SSG must make itself available to others to raise awareness of and demand for code review. The SSG maintains a list of the most important kinds of bugs that need to be eliminated from the organization’s code.). Management must make code review mandatory for all software projects. The SSG performs an ad hoc code review for high-risk applications in an opportunistic fashion. and actual incidents. security testing. Use automated tools along with manual review. Don’t forget that individual bugs make excellent training examples. Use centralized reporting to close the knowledge loop and drive training. This service should be explicitly connected to a larger SSDL applied during software development and not just “check the security box” on the path to deployment. testing. the SSG might follow up the design review for high-risk applications with a code review. Create a top N bugs list (real data preferred).SSDL TOUCHPOINTS: Code Review (CR)
The overall goal of the Code Review practice is quality control. Incorporate static analysis into the code review process in order to make code review more efficient and more consistent. but a list is much more valuable if it is specific to the organization and built from real data gathered from code review. The SSG must enforce adherence to standards and the reuse of approved security features.” For example. SSG review may involve the use of specific tools and services.5]
[CR1.6]
37
BSIMM4
.) Some firms use multiple tools and real code base data to build top N lists. Code review is a mandatory release gate for all projects under the SSG’s purview. Those performing code review must ensure the detection and correction of security bugs. This repository makes it possible to do summary reporting and trend reporting for the organization. see [T1. The automation does not replace human judgment. etc. Lack of code review or unacceptable results will stop the release train.2]
[CR1. The SSG can periodically update the list and publish a “most wanted” report. the OWASP Top Ten list rarely reflects an organization’s bug priorities. Simply sorting the day’s bug data by number of occurrences does not produce a satisfactory Top N list since these data change so often. Replace ad hoc targeting with a systematic approach at higher maturity levels. For example.
one
CR Level 1: Manual and automated code review with centralized reporting. black box testing. Likewise. but it does bring definition to the review process and security expertise to reviewers who are not security experts. or it may be manual. One potential pitfall with a top N list is the problem of “looking for your keys only under the street light. white box testing. code review information can be fed into a Development-wide project tracking system that rolls up a number of diverse software security feeds (for example: penetration tests.

[This is a new activity that will be reported on in BSIMM5. The SSG must guide developer behavior through coding standards enforcement with automated tools and tool mentors. Although out-of-the-box automation might identify some generic malicious-looking constructs. In some cases. Build a factory. Manual code review for malicious code is a good start. Turn off checks that are not relevant. Combine assessment techniques so that multiple analysis sources feed into one reporting and remediation process. The SSG must combine automated assessment techniques with tailored rules to find problems efficiently. If the SSG is most skilled with the tools. and uses the rules to identify all occurrences of the new bug throughout the entire codebase. A firm with only a handful of software applications will have an easier time with this activity than firms with a very large number of large apps.4]
2012
38
.1]
[CR3. Assign tool mentors. CR Level 3: Build an automated code review factory with tailored rules. a CWE-like approach can help with nomenclature. Centralized use of a tool can be distributed into the development organization over time through the use of tool mentors.2]
CR Level 2: Enforce standards through code review process. Examples of malicious code that could be targeted include: backdoors. the SSG writes rules to find it. The enforced portion of the standard could start out being as simple as a list of banned functions. Code review is objective—it does not devolve into a debate about whether or not bad code is exploitable. guidelines for C++ or Struts) and then enforced during the code review process or directly in the IDE. time bombs. Analysis engines may combine static and dynamic analysis.]
[CR2.two
[CR2. custom rules for static analysis tools used to codify acceptable and unacceptable code patterns in the organization’s codebase will quickly become a necessity. someone from the SSG might work with a development team for the duration of the first review they perform. coding standards are published as developer guidelines specific to technology stacks (for example.5]
three
[CR3. Use automated tools with tailored rules. When a new kind of bug is found. obfuscated program logic.3]
[CR3. The same group that provides tool mentoring will likely spearhead the customization. The SSG must build a capability to find and eradicate specific bugs from the entire codebase. It is possible to entirely eradicate the bug type without waiting for every project to reach the code review portion of its lifecycle. but is insufficient to complete this activity. Combining multiple sources helps drive better informed risk mitigation decisions. Build a capability for eradicating specific bugs from the entire codebase. Mentors are available to show developers how to get the most out of code review tools. Automated code review is used to identify dangerous code written by malicious in-house developers or outsource providers. logic bombs. Automate malicious code detection. The SSG might write scripts to invoke multiple detection techniques automatically and combine the results into a format that can be consumed by a single downstream review and reporting solution. nefarious communication channels. A violation of the organization’s secure coding standards is sufficient grounds for rejecting a piece of code. Enforce coding standards. Tailored rules can be explicitly tied to proper usage of technology stacks in a positive sense and avoidance of errors commonly encountered in a firm’s code base in a negative sense.2]
[CR3. and dynamic code injection. The tricky part of this activity is normalizing vulnerability information from disparate sources that use conflicting terminology. Note that guidelines can be positive (“do it this way”) as well as negative (“do not use this API”). In some cases. Use custom rules to find errors specific to the organization’s coding standards or custom middleware. Alternatively. it could use office hours to help developers establish the right configuration or get started interpreting results. Customize static analysis to improve efficiency and reduce false positives.

Testing begins to incorporate test cases based on abuse cases provided by the SSG. Security tests can be driven from abuse cases identified earlier in the lifecycle or tests derived from creative tweaks of functional tests. Test suites must include functional security testing. The same automation framework houses both. ST Level 2: Integrate the attacker perspective into test plans. Using security results to inform and evolve particular testing patterns can be a powerful mechanism leading to better security testing.4]
three
[ST3.3]
two
[ST2. software security is not security software. and guidelines. but getting started with features is easy.
one
[ST1. Those performing security testing must ensure the detection and correction of security bugs.SSDL TOUCHPOINTS: Security Testing (ST)
The overall goal of the Security Testing practice is quality control performed during the development cycle. and so on are also tested. For example. but come to the SSG for help interpreting the results. Of course. Security tests run alongside functional tests as part of automated regression testing. the testing should be properly integrated into the QA cycle of the SSDL. For the most part. QA must integrate black-box security testing tools into its process. transaction limitations. entitlements. Security mechanisms based on requirements such as account lockout. In some situations. Include security tests in QA automation. The SSG must share its security knowledge and testing results with QA.1]
ST Level 1: Enhance QA beyond functional perspective. The organization uses one or more black box security testing tools as part of the quality assurance process.3]
[ST2. ST Level 3: Deliver risk-based security testing. Abuse and misuse cases based on the attacker’s perspective can also be driven from security policies. The SSG shares results from security reviews with the QA department. For example. Tools such as Rational AppScan or HP WebInspect are relevant for Web applications. QA must progress to include functional edge and boundary condition testing in its test suites. Testers target declarative security mechanisms derived from requirements and security features. For example. Begin to build and apply adversarial security tests (abuse cases). Share security results with QA. a testing team could run the tool. This activity benefits from an engineering-focused QA function that is highly technical. They probe simple edge cases and boundary conditions.” A discussion of boundary value testing leads naturally to the notion of an attacker probing the edges on purpose. The tools are valuable because they encapsulate an attacker’s perspective. When QA understands the value of pushing past standard functional testing using acceptable input. Testers move beyond verifying functionality and take on the attacker’s perspective.1]
[ST2. testers might systematically attempt to replicate incidents from the organization’s history. they begin to move slowly toward “thinking like a bad guy. Regardless of who runs the black box tool. No attacker skills required. Security testing is part of the routine. Ensure QA supports edge/boundary value condition testing. The QA team goes beyond functional testing to perform basic adversarial tests. other groups might collaborate with the SSG to apply the tools. The SSG must ensure this security testing and its depth is guided by knowledge about the codebase and its associated risks.1]
39
. albeit in a generic fashion. QA must build test suites for functional security features and progress to building adversarial tests that simulate the attacker’s perspective. and fuzzing frameworks such as PROTOS and Codenomicon are applicable for most network protocols. Integrate black box security tools into the QA process. The SSG must enforce adherence to standards and the reuse of approved security features. Over time. QA engineers learn the security mindset. attack intelligence. a tester could try to access administrative functionality as an unprivileged user or verify that a user account becomes locked after some number of failed authentication attempts. This turns the corner from testing features to attempting to break the software under test. QA must include security testing in automated regression suites.
BSIMM4
[ST1. security features can be tested in a similar fashion to other software features. What happens when you enter the wrong password over and over? Drive tests with security requirements and security features.

Drive tests with risk analysis results.” then torn transactions will be become a primary target in adversarial testing.
[ST3. Test automation engineers customize a fuzzing framework to the organization’s APIs.[ST3. Don’t let this happen to your tests. but customization goes beyond creating custom protocol descriptions or file format templates.3]
[ST3. Leverage coverage analysis. Testers measure the code coverage of their security tests in order to identify code that isn’t being exercised. if the architecture analysis concludes “the security of the system hinges on the transactions being atomic and not being interrupted partway through. Standard issue black box testing tools achieve exceptionally low coverage. Using standard measurements for coverage such as function coverage. Adversarial tests like these can be developed according to risk profile—high risk flaws first. They could begin from scratch or use an existing fuzzing toolkit.2]
Perform fuzz testing customized to application APIs. Testers use architecture analysis results to direct their work. For example. Test harnesses developed explicitly for particular applications can make good places to integrate fuzz testing.4]
2012
40
. Code coverage drives increased security testing depth. line coverage or multiple condition coverage is fine. leaving a majority of the software under test unexplored. The fuzzing framework has a built-in understanding of the application interfaces it calls into.

Evolving DevOps and integrated team structures do not eliminate the need for formalized defect management systems. Schedule periodic penetration tests for application coverage. the focus of penetration testing moves from “I told you our stuff was broken” to a smoke test and sanity check done before shipping. Give penetration testers everything you have created throughout the SSDL. If your penetration tester doesn’t ask for the code.DEPLOYMENT: Penetration Testing (PT)
The overall goal of the Penetration Testing practice is quality control of software that has moved into deployment. Penetration testers.2]
[PT2. Penetration testers can do deeper analysis and find more interesting problems when they have source code. and development responds using their defect management and release process. The SSG must create an internal penetration testing capability that is periodically applied to all applications. Provide penetration testers with all available information.3]
41
BSIMM4
. external penetration testers demonstrate that the organization’s code needs help. internal penetration testers. with internal or external resources. Penetration testing results are fed back to development through established defect management or mitigation channels. and hand-written scripts. The SSG must share its security knowledge and testing results with all penetration testers. Those performing penetration testing must ensure the detection and correction of security defects.
[PT1. Use penetration testing tools internally. Managers and the SSG must initiate the penetration testing process. Many firms are beginning to emphasize the critical importance of not just identifying but more importantly fixing security problems. whether internal or external. High-profile applications might get a penetration test at least once a year. standard issue network penetration tools that understand the application layer. Tools can include off the shelf products.2]
[PT1. architecture analysis results.3]
two
[PT2. Over time. The tools improve efficiency and repeatability of the testing process. The organization creates an internal penetration testing capability that makes use of tools. PT Level 2: Schedule regular penetration testing by informed. you need a new penetration tester. design documents.
one
PT Level 1: Remediate penetration testing results. Feed results to the defect management and mitigation system.1]
[PT1. The SSG must enforce adherence to standards and the reuse of approved security features. The testing serves as a sanity check and helps ensure yesterday’s software isn’t vulnerable to today’s attacks. Test applications periodically according to an established schedule (which could be tied to the calendar or to the release cycle). External penetration testers bring a new set of eyes to the problem. If security has not been a priority. Many organizations are not willing to address software security until there is unmistakable evidence that the organization is not somehow magically immune to the problem. One important aspect of periodic testing is to make sure that the problems identified in a penetration test are actually fixed and they don’t creep back into the build. Use external penetration testers to find problems. Penetration testers could be brought in to break a high-profile application in order to make the point. and code review results. One way to ensure attention is to add a security flag to the bug tracking and defect management system. The exercise demonstrates the organization’s ability to improve the state of security. Managers and the SSG must ensure that deficiencies discovered are addressed and that everyone is made of aware of progress. This capability can be part of the SSG or part of a specialized and trained team elsewhere in the organization. are equipped with all available information about their target.

Tools improve the efficiency of the penetration testing process without sacrificing the depth of problems the SSG can identify.three
PT Level 3: Carry out deep-dive penetration testing. The SSG must take advantage of organizational knowledge to customize penetration testing tools. These testers are experts and specialists. Creating new types of attacks from threat intelligence and abuse cases prevents checklist-driven approaches that only look for known types of problems. They keep the organization up to speed with the latest version of the attacker’s perspective and they have a track record for breaking the type of software being tested. Managers must ensure that the organization’s penetration testing knowledge keeps pace with advances by attackers. The SSG either creates penetration testing tools or adapts publicly available tools so they can more efficiently and comprehensively attack the organization’s systems. Have the SSG customize penetration testing tools and scripts. Tools that can be tailored are always preferable to generic tools.1]
[PT3.2]
2012
42
. Skilled penetration testers will always break a system. Use external penetration testers to perform deep-dive analysis.
[PT3. The question is whether they demonstrate new kinds of thinking about attacks that can be useful when designing. and hardening new systems. The organization uses external penetration testers to do deep-dive analysis for critical projects and to introduce fresh thinking into the SSG. implementing.

and manipulates. In order to protect intellectual property and make exploit development harder. Intrusion detection and anomaly detection systems at the application level may focus on an application’s interaction with the operating system (through system calls) or with the kinds of data that an application consumes.1]
SE Level 1: Ensure the application environment supports software security.
one
[SE1. Obfuscation techniques could be applied as part of the production build and release process. SE Level 2: Use published installation guides and code signing. The guide should include discussion of COTS components. The SSDL requires the creation of an installation guide to help operators install and configure the software. The operations group ensures required host and network security controls are functioning and proactively monitors software. Doing software security before network security is like putting on your pants before putting on your underwear. The SSG could be responsible for the care and feeding of the system. including application inputs. Publish installation guides. This activity goes beyond host and network monitoring to look for problems that are specific to the software. The organization provides a solid foundation for software by ensuring that host and network security basics are in place. Use code signing. a WAF that is unmonitored makes no noise when an application falls in the woods. The SSG must ensure that application installation and maintenance guides are created for the operations group to use.4]
three
[SE3. The fact that some mobile platforms require application code to be signed does not indicate institutional use of code signing. The organization monitors the input to software it runs in order to spot attacks. a Web application firewall (WAF) can do the job. SE Level 3: Protect client-side code and actively monitor software behavior. originates. In some cases. such as shrink-wrapped applications or thick clients. secure by default is always the best way to go.2]
[SE2. On the other hand. For Web code. and Address Space Layout Randomization (ASLR) can make exploit development more difficult.2]
two
[SE2.DEPLOYMENT: Software Environment (SE)
The overall goal of the Software Environment practice is change management. Of course. It is common for operations security teams to be responsible for duties such as patching operating systems and maintaining firewalls. The organization uses code signing for software published across trust boundaries. The SSG must ensure that all code leaving the organization is protected. the organization erects barriers to reverse engineering.2]
[SE3. Ensure host and network security basics are in place. Use application behavior monitoring and diagnostics. Evolving DevOps and integrated team structures do not eliminate the need for written guides. Use code protection. Employing platform-specific controls such as Data Execution Prevention (DEP). Use application input monitoring. Defanged WAFs that write log files can be useful if somebody reviews the logs periodically. The organization monitors the behavior of production software looking for misbehavior and signs of attack. Code signing is particularly useful for protecting the integrity of software that leaves the organization’s control. such as indications of fraud. Managers must enforce adherence to corporate policy.3]
43
BSIMM2
. the steps are outlined in the installation guide. Responding to attack is not part of this activity. If special steps are required in order to ensure a deployment is secure. Safe Structured Error Handling (SafeSEH). The operations group must monitor software behavior.
[SE1. Those responsible for the software environment must ensure their ability to make authorized changes and to detect unauthorized changes and activity. The SSG must ensure software development processes protect code integrity. installation guides are distributed to customers who buy the software.

Defects identified through operations monitoring are fed back to development and used to change developer behavior. In some cases. When simulations model successful attacks. Experience from operations leads to changes in the SSDL. Defects found during operations are fed back to development. Create or interface with incident response. Often times. Have emergency codebase response. (See [CR3. providing a way to enter incident triage data into an existing bug tracking system (many times making use of a special security flag) seems to work. could begin with the assumption that a critical system or service is already compromised and evaluate the organization’s ability to respond. This requires the ability to reexamine the entire codebase when new kinds of bugs come to light. A regular meeting between the SSG and the incident response team can keep information flowing in both directions. Make sure the loop is closed completely. This works best when root cause analysis pinpoints where in the SDLC an error may have been introduced or slipped by uncaught. The organization has a map of its software deployments. software security initiatives have evolved from incident response teams who began to realize that software vulnerabilities were the bane of their existence. Application owners must enforce adherence to corporate policy. If the data center is burning to the ground. A rapid-response team works in conjunction with the application owners and the SSG to study the code and the attack. An ad hoc approach is not sufficient.3] Build capability for eradicating specific bugs from entire codebase. entered into established defect management systems. The organization fixes all instances of software bugs found during operations and not just the small number of instances that have triggered bug reports. CMVM Level 3: Create a tight loop between operations and development.]
[CMVM1. and push a patch into production. the SSG won’t be among the first responders. The organization can make quick code changes when an application is under attack. The SSG must ensure the SSDL both addresses code deficiencies found in operations and includes enhancements that eliminate associated root causes. Simulate software crisis. Fire drills do not count.1]
CMVM Level 1: Use operations monitoring data to drive developer behavior. operations can reliably identify all of the places where the change needs to be installed. The idea is to close the information loop and make sure security problems get fixed. Enhance the SSDL to prevent software bugs found in operations. The SSG must support incident response. an important question to consider is the time period required to clean things up. The SSG uses operations data to direct evolution in the SSDL and in developer behavior. The SSG and application owners must ensure their ability to track authorized changes to applications and to detect unauthorized changes and activity.DEPLOYMENT: Configuration Management and Vulnerability Management (CMVM)
The overall goal of the Configuration Management and Vulnerability Management practice is change management. Simulations could test for the ability to identify and mitigate specific threats or.3]
2012
44
.1]
[CMVM3.2]
two
[CMVM2. processes in the SSDL can be improved based on operational data. Identify software defects found in operations monitoring and feed them back to development. The SSDL is strengthened to prevent the reintroduction of bugs found during operations.1]
[CMVM2. In many cases. The contents of production logs can be revealing (or can reveal the need for improved logging). Regardless. other applications that share the same components can be fixed as well. The SSG is prepared to respond to an incident. the incident response post mortem could include a “feedback to SSDL” step.2]
[CMVM2. Managers and the SSG must support emergency response to ongoing application attacks. To make this process systematic.2]
[CMVM3. [This is a new activity that will be reported on in BSIMM5. Managers and the SSG must maintain a code inventory. and tracked through the fix process.
one
[CMVM1. The SSG simulates high-impact software security crises to ensure software incident response capabilities minimize damage. The SSG must use operations data to suggest changes in the SSDL and developer behavior. The group either creates its own incident response capability or interfaces with the organization’s existing incident response team. In the best of cases. simulations must focus on security-relevant software failure and not natural disasters or other types of emergency response drills. Sometimes common components shared between multiple projects are noted so that when an error occurs in one application. CMVM Level 2: Ensure that emergency response is available during application attack. in other cases. Develop an operations inventory of applications.) One way to approach this is to create a rule set that generalizes a deployed bug into something that can be scanned for using automated code review. find a resolution.3]
three
[CMVM3. Fix all occurrences of software bugs found in operations. a well-defined process is required. This capability could come in the form of a two-way bridge between the bug finders and the bug fixers. Setting a security flag in the bug tracking system can help facilitate tracking. the emergency response team is the development team itself. If a piece of code needs to be changed. Track software bugs found in operations through the fix process.

The BSIMM Skeleton

T

he BSIMM skeleton provides a way to view the maturity model at a glance and is useful when assessing a software security program. The skeleton includes one page per practice organized by three levels. Each activity is associated with an objective. More complete descriptions of the activities, examples, and term definition can be found in the main document

1] [CMVM1.2] [CMVM2. defect tracking and remediation.3] [CMVM3. Objective
[CMVM1.3]
2012
56
.1] [CMVM3. incident handling.2]
Activity
use ops data to change dev behavior identify software defects found in operations monitoring and feed them back to development
Level
1
know what to do when something bad happens create or interface with incident response
be able to fix apps when they are under direct have emergency codebase response attack use ops data to change dev behavior track software bugs found in operations through the fix process know where the code is develop an operations inventory of applications learn from operational experience fix all occurrences of software bugs found in operations (T: code review) use ops data to change dev behavior enhance the SSDL to prevent software bugs found in operations ensure processes are in place to minimize software simulate software crisis incident impact
2
3
[CMVM3. version control.DEPLOYMENT: CONFIGURATION MANAGEMENT AND VULNERABILITY MANAGEMENT
Patching and updating applications.2] [CMVM2.1] [CMVM2.

learning from experience is also a good strategy.1] [PT1.
57
. We suggest creating a software security initiative strategy and plan by focusing on goals and objectives first and letting the activities select themselves.2]
Twelve Core Activities Everybody Does Activity
establish SSDL gates (but do not enforce) identify gate locations. The twelve core activities are highlighted in yellow.
Objective
[SM1. this section is devoted to describing the set of core activities that we observed in at least thirty-two of the fifty-one organizations we studied and then providing a chart of all 109 activities with a summary score that can be seen as a rough weighting.1] [SE1.Ranking BSIMM4 Activities
C
hoosing which of the 111 BSIMM activities to adopt and in what order can be a challenge.1] [CR1.
Core BSIMM Activities
Of the 109 activities observed in BSIMM4.1] [AM1.4] [CP1. we can say with confidence that these activities are commonly found in highly successful programs. gather necessary artifacts
BSIMM4
promote privacy identify PII obligations promote culture of security throughout the organization provide awareness training stay current on attack/vulnerability environment gather attack intelligence create proactive security guidance around security build and publish security features features meet demand for security features create security standards get started with AA perform security feature review drive efficiency/consistency with automation use automated tools along with manual review execute adversarial tests beyond functional ensure QA supports edge/boundary value condition testing demonstrate that your organization’s code needs help use external penetration testers to find problems too provide a solid host/network foundation for software ensure host and network security basics are in place use ops data to change dev behavior identify software bugs found in operations monitoring and feed them back to development
Activities Observed over Fifty-one Firms
The chart on the next page shows how many of the fifty-one firms we studied have adopted various activities.2]
[CMVM1.5] [SFD1. there are twelve activities that at least thirty-two of the fifty-one firms we studied carry out (63%).2] [T1.4] [ST1. you should consider these twelve activities particularly carefully (not to mention the other 99). Though you can use this as a rough “weighting” of activities by prevalence. one identified in each practice. This suggests that if you are working on an initiative of your own. Of course. Though we can’t directly conclude that these twelve activities are necessary for all software security initiatives. a software security initiative plan is best approached through goals and objectives.1] [SR1. Creating a timeline for rollout is often very useful.1] [AA1. Toward that end.

.6] . . The Training practice had become increasingly problematic statistically from BSIMM2 to BSIMM3. 3]
was removed (level demotion) was removed (level demotion) was removed (level demotion) was removed (level promotion) was removed (level promotion) was removed (a two-level promotion)
The resulting BSIMM4 scorecard can be seen on page 58. To do this. We used the results of an intra-level standard deviation analysis to determine which “outlier” activities to move between levels.2] .4] . 5. [CR2.4] [ T1 .
[ T2 . Here are the five changes we made according to that paradigm: 1. but did not adjust. To accomplish the overhaul.2] [ T2 .5] [T1.1] . 6. and [ST2.
59
BSIMM4
We also carefully considered.2] was removed (level promotion) [SE2. 5.3] became [SE3. even when an activity has simply been moved between levels.
[ T 2 .1] .4] . [ST1. we focused on changes that minimize standard deviation in the average number of observed activities at each level. 4.2] became [ST2.3] became [CR1. [SR2.6] [T1.7] [T2. 2.4] [ T1 .4] . [AM2.2] [ T1 . [SE2.3]
became became became became became became
[T1. we made the following six changes to the Training practice: 1.6] [T2.1] [ T2 . the following activities: [AM1.
[AM2. 4] [ T 1 . 2] [ T 2 . 4. we have chosen to make several adjustments to the model based on the data observed between BSIMM3 and BSIMM4. 3. and in the levels we assigned to various activities in the twelve practices. .4] became [AM1.5]
.
In order to preserve backwards compatibility. 2] [ T 1 . . we completely overhauled one of the practices. . 2.3] . 3.5] . 1] [ T 2 .4] became [CR1. all changes have been made by adding new activity labels to the model.3] was removed (level demotion)
[CR2.6] .Appendix: Adjusting BSIMM3 for BSIMM4
B
ecause the BSIMM is a data driven model.4] was removed (level demotion) [CR2. [CR2. [AA2. [SR1. we took the bull by the horns and completely revisited all of the activities and their levels.7] [T3. 4] [ T 1 .4] was removed (level demotion) [ST1. Our hard and fast rule was to ensure that on average the number of observed activities per level followed a logical progression from common to rare (as outlined in our discussion of levels).3] was removed (level promotion)
For the first time in the BSIMM Project. We made all changes by considering outliers both in the model itself.
. For BSIMM4.