“What is Agile” is a difficult question. It is both a philosophy and a set of software development practices. “Why is Agile important” is a good question for an introductory course. The simple answer: Agile is a good way to make software product more responsive to customer needs – saving effort and money as a side effect. “How hard is it to be Agile” is a deep question. It generally very easy to use agile methods for small software products and with small development teams. For larger products and cross-location teams, it is still possible to apply many agile practices – but there will be more organizational obstacles.

The main tasks of software development are still the same in Agile development – but the flow of activities will look a lot different.

The main ideas of Waterfall: Each of the development activities is a separate “stage” – with a “gate” that they must pass at the end of the stage We can think of team of specialists as working at its own station on a factory assembly line Waterfall can be thought of as an “efficient” process because the work is structured in a way that allows each team to be reassigned to other projects when their part of the project is complete. Unfortunately, this doesn’t always work so well. The communication between teams is usually imperfect and inefficient. It is a lot of work to create “complete” documentation at each stage of the process, and if the requirements or architecture teams have moved on to other projects, the design/development and test teams might not be able to get their questions answered promptly. If there are late requirements and architecture changes, the waterfall process has even more overhead costs – documents need to be updated, re-reviewed, and approved in order to conform to the overall process.

The list of “core ideas” is from this paper -- Noura Abbas, Andrew M. Gravell, and Gary B. Wills, “Historical Roots of Agile Methods: Where Did “Agile Thinking” Come From?” Proceedings of XP 2008, pp. 94-103. In theory, the Waterfall Model could be used in an Adaptive and People-oriented way. In practice, however, Waterfall projects are pretty rigid and bureaucratic. (It is interesting that most manufacturing work has been moving away from the old-fashioned rigid assembly line techniques of the early twentieth century – because management has found that they can get better quality and productivity with cross-trained cross-functional teams. So why are “knowledge workers” (software professionals) are often managed in a more assembly-line style than factory workers?)

Why iteration? It gives the development team a chance to change course several times during the course of development. In this example, with six iterations of 4 weeks each, the team can take a breath and say “we are done with part of the system” six times during a 24-week period – and they should be able to demonstrate that they have implemented some real customer-visible functionality. Then the team can ask some most important questions: “ Are we building the right things?” “ Will the performance be adequate for the customer?” “ Can we find some tools, components, architecture changes, or development process changes that can improve the rest of the development work?”

Planning in Agile development: The most detailed plans are for the current iteration The work for future iterations can be adjusted after talking with the customers No “big design document” – because the future work might change, depending on the customers’ priorities In a good Agile project, the team “gets into a rhythm” – delivering a set of new features every cycle

This is a list of common Agile practices. Most of these practices have been used in a range of projects – from small web-based applications to embedded real-time systems to complex communications systems. You don’t need to use every Agile practice to be an agile project. It is more important to understand the set of agile values and principles – and to use the values and principles to select a reasonable set of practices.

Most of you have seen the Agile Manifesto. It is a brilliant document, and we let it speak for itself here. Let me ask you a couple of things about the Manifesto: What does it say about iteration? What does it say about customer satisfaction with the product? What new things does it tell you to do that you are not already doing? What are the consequences of using processes, tools, documentation, contracts, and plans? You get the idea. The Manifesto is a framework for thinking about what you value; it is not a framework that you can fill in according to your needs. It is nonetheless a firm starting point for Scrum—which itself is a framework that guides you and you can fill in. However, don’t leave your brains behind: you will need your wits just as much as a Scrum practitioner as an Agile fan.

http://www.agilemanifesto.org/principles.html

User stories, scenarios, and use cases are popular methods of writing lightweight requirements – you write stories of how the system will be used from the viewpoint of one of the users. It is a “scenario-based” requirements technique, and it always uses vocabulary that makes sense to the customer. A good requirement / use case / user story is never really “complete” – it is just an outline of what should happen, and it can’t possibly cover all possible errors and branches. But – scenarios are a good thing to focus on during discussions with the customer. A good agile team member will get the customer to talk about some of the alternatives that weren’t listed in the original requirement.

Documentation is not necessarily contrary to Agile values. You should think about keeping the process lightweight -- eliminating documentation that isn’t providing value. Scott Ambler has some interesting thoughts about what kinds of documentation make sense in an agile environment: http://www.agilemodeling.com/essays/agileDocumentation.htm Iterative development changes the way that development planning is done. In Agile methodologies such as Scrum, there is an organized way of doing adaptive planning, and there is a special vocabulary for describing both long-term and short-term plans http://www.mountaingoatsoftware.com/product-backlog http://www.mountaingoatsoftware.com/sprint-backlog Alistair Cockburn has some good ideas about how to use Agile development techniques for situations such as fixed-price, fixed-scope contracts: http://alistair.cockburn.us/Agile+contracts

What is more important – delivering hundreds of features in a product, or delivering just the right set of features to meet your customer’s greatest needs? The answer is not obvious. But here is one consideration: A product with a lot of features can be costly to deploy and costly to maintain. In today’s market, our customers are often looking to reduce their internal training and deployment costs – so it might be better to forget about delivering 92 features in a release… just think about the 16 features that are needed to implement the top 5 use cases. If you develop a product with a small but well-coordinated feature set, you might be able to get it on the market faster – beat the competition. Here are a few good articles on the benefits of keeping your code small: “ Keep It Small” by Jack Ganssle - http://www.ganssle.com/articles/keepsmall.htm “ Less Software” (from the online book Getting Real ) - http://gettingreal.37signals.com/ch10_Less_Software.php “ Small is Beautiful” by John Mashey (slides from an ACM talk in the late 1970s) -- http://www.usenix.org/events/bsdcon/mashey_small/

What does it mean to “Be Agile”? “ Agile is not a practice. It is a quality of the organization and its people to be adaptive, responsive, continually learning and evolving – to be agile , with the goal of competitive business success and rapid delivery of economically valuable products and knowledge.” (Craig Larman and Bas Vodde, Scaling Lean &amp; Agile Development , Chapter 6.) There are a number of Agile practices – you can “do iteration”, “do continuous integration”, “do automated testing”, “do regular customer demos”, and “do regular retrospectives”. These practices will help you be more adaptive and responsive.

A “scrum” is “a meeting with an attitude”. In rugby, the linemen have their arms linked together, and they are working cooperatively to move their opponents back so they can get the ball back to their teammates. In a software scrum, it doesn’t look like the team is doing much, but they really are doing some hard work. They are getting a report on the status of every team member – each person is giving a short summary of the problems they are faced with on their tasks in the current iteration.

Scrum uses timeboxed iterations. This is important. Everyone on the team knows when the iteration will end – and this is useful information when the team members make estimates at the beginning of the iteration.

The Product Backlog is the list of “everything” – in priority order. In a Sprint, the team will work on a very small subset of the Product Backlog. When planning the Sprint, the team will consider the business value they will be able to deliver at the end of the Sprint. The reasons for the Product Backlog: The team can start doing the “most valuable” parts of the system in early iterations – the system features that have the highest business value to the customer. If some new “high-value features” are discovered later, they can be added to the Product Backlog – and they might actually push some other less valuable features out of the release. This is OK – assuming that your customers really want you to build the newly discovered features (instead of just blindly delivering the items in the original release plan). The estimates in the Product Backlog help managers plan the release: they should try not to promise more than the team thinks they can deliver. Even if there are problems delivering everything in the release plan, following the Product Backlog in each iteration will result in the team delivering the maximum customer value possible with the resources available.

There are a number of commercial and open source tools that could be used to manage the Product Backlog. See http://acos.alcatel-lucent.com/wiki/g/be-agile/Tools%20for%20Agile for more information.

The most common technique for doing estimates of the Product Backlog is to use an “artificial” estimation unit: Story Points. This often works better than trying to estimate the number of days of effort. Estimation for a single Sprint – estimating in days or hours makes more sense here.

For more information on how traditional project roles are changed when using Scrum, read Chapter 8 of Mike Cohn’s book Succeeding with Agile -- http://www.infoq.com/articles/cohn-chapter8 The Scrum Master is not the “manager of the team” – the team is actually “self-organizing”. But every team needs help staying organized, and every team needs help with obstacles and impediments.

More information on burndown charts: http://alistair.cockburn.us/Earned-value+and+burn+charts What are the units in the burndown chart? Vertical axis is “person days”, horizontal axis is “days” – one point on the curve for each working day. How to create the curve?? Each day, go to the Task Board. Each task should have an index card, with the remaining estimated effort to complete the task. Just add up the numbers. At the end of the iteration, all of the cards should be “completed” – zero time left to complete. Why could the curve go higher? You might have missed a task in the initial iteration planning, or you might discover as you start a task that it will take longer than the original estimate. If the burndown chart isn’t going down fast enough… the team has to take action to improve things – usually immediately. Slide 19 listed the top three recovery actions: Get more resources Try to reduce scope (negotiate with the Product Owner) Revisit the product’s software architecture

The Daily Scrum Meeting (also known as the “standup meeting”) is a daily ritual. It isn’t really a traditional status meeting. Its main function: to make sure that if someone in the team is stuck for one day, the rest of the team will help get them unstuck immediately.

A popular book on end-of-iteration retrospectives: Agile Retrospectives by Esther Derby and Diana Larsen. The classic book on Retrospectives for software projects: Project Retrospectives by Norm Kerth.

Retrospectives should never be used as part of the “performance management” system for a company – you want the retrospective participants to be free to focus on learning the lessons of the past to make things work better in the future. “Blame the process, not the person.” See the following InfoQ article for more information on the application of the Retrospectives Prime Directive: http://www.infoq.com/articles/retrospective-prime-directive Norm Kerth’s book on Project Retrospectives or Esther Derby and Diana Larsen’s book on Agile Retrospectives .

Big projects – look for information on the “Scrum of Scrums” approach. Managers can “cancel a Sprint” – a bit extreme, but sometimes necessary when the goals of a project change. This “abnormal early termination process” is mentioned in many of the Scrum books. Fixed schedule and fixed contract is never easy, but an experienced Scrum team can do a pretty good job – because they have some practice at doing short term estimation. It is still necessary to add some extra “slack” to the estimates to handle unexpected variation from the estimates. Part-time specialists are almost inevitable – there are specialists in our company who are in demand for their skills, so it may be necessary to adapt Scrum to use them in a part-time capacity. Also, some part-time workers might be needed in the transition to Scrum – such as testers. Multiple-location teams are possible but difficult. You need to think about using several kinds of communication technology to get the effect of face-to-face. Scrum doesn’t directly address architecture issues – but architecture has been discussed by many industry experts. CMMI, ISO9000, and TL9000 certification is possible, even for a team using Scrum. There will be some extra “documentation” that needs to be created and maintained to meet certification requirements.

Continuous integration is one of the most important agile practices for maintaining quality throughout the development cycle. It makes it possible to react early to design and implementation problems. “Automating the build” is something that all Alcatel-Lucent projects should do. We already have many existing practices that help us, such as the use of a source code control system. The biggest obstacle to continuous integration is the amount of time needed to do a “complete build” for large products. This is a common problem on our big systems – but there are some things we can do to make things better for doing “incremental builds”.

User stories are a popular method of writing lightweight requirements – you write stories of how the system will be used from the viewpoint of one of the users. It is a “scenario-based” requirements technique, and it always uses vocabulary that makes sense to the customer. A good user story is never really “complete” – it is just an outline of what should happen, and it can’t possibly cover all possible errors and branches. But – the user story is a good thing to focus on during discussions with the customer. A good agile team member will get the customer to talk about some of the alternatives that weren’t listed in the original user story.

All three of these roles are difficult. Most of the decisions in Scrum are made by the team – it is a “self-managing” team. Decisions about the product requirements need to be made in consultation with the customer… so the Product Owner role is very important. Scrum Master is supposed to help keep everyone on track. If there are “impediments” to the team, the Scrum Master will help find some ways to resolve the blockages. If the team has communications problems, the Scrum Master needs to be ready to facilitate. Example of self-organization http://www.youtube.com/watch?v=T8Doy_7sOoM

Many of us are familiar with the Waterfall Model – it is a “framework” for the software development process

Waterfall Model talks about “development activities through time”

Waterfall Model talks about “teams of people”

As each team finishes, they are assigned to a new product At the end of all of the stages, you have a software product ready to ship Some coordination is required for the handoff from team to team – using “documents” At each stage, the work is passed from one team to another A separate team of specialists for each stage Divide the work into stages Teams Development activities

11.
Agile principles 9. Continuous attention to technical excellence and good design enhances agility. 10. Simplicity--the art of maximizing the amount of work not done--is essential. 11. The best architectures, requirements, and designs emerge from self-organizing teams. 12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. 5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. 7. Working software is the primary measure of progress. 8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. 2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. 3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. 4. Business people and developers must work together daily throughout the project.

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand. (From Norm Kerth’s book on Project Retrospectives See also http://www.retrospectives.com ) Why this rule? The goal of a retrospective is to improve the process , not to assign blame for the problems

As typically used, “Agile” refers to a set of development methodologies and frameworks (software development methodologies) based on and implementing the above principles.

It is a light weight and at the same time very structured development framework – marking the current step in software development models (from waterfall/sequential models – to V-shaped – to incremental – to spiral - to iterative - to “ agile ”). Note, that agile is more than a process model.