A Proposal to Enhance the UML Notation

Requirements and test cases, in particular acceptance tests, are strictly related. Agile methodologies highlight this aspect basing themselves on test driven approaches. The notation of UML use cases may be enhanced in order to allow enhanced UML tools to properly handle links between use cases and tests.

Acceptance tests are part of the design

One of the best practices of the agile paradigm is the “Test Driven Development” or – in the perspective of Dan North1 - the “Behaviour Driven Development”. When I read for the first time the introduction to BDD, I was particularly impressed by the concept of “executable specifications” built upon the text template:

“Executable specifications”, what a wonderful idea! Regardless of the usability of the concept in any context - for example it is very difficult to imagine how to apply it in a literal sense inside a contractual document -, it was the implicit conveyed message that changed the way I see the tests:

the acceptance tests are part of the design!

Why only acceptance tests? What about unit testing, integration testing, system testing? Well,

Unit tests are typically defined to check low level building blocks - such as class methods - that are subject to many implementation decisions to be seriously taken into consideration: I don’t see unit tests as design components, but as aids automatically generated by tools from source code.

System (or subsystem) tests are defined to check readiness to the delivery of high level components, so I regard system tests as equivalent to the acceptance tests.

Integration tests raise the question: my simple answer is that they should be considered acceptance testing. Perhaps the word “should” requires a deeper understanding and it is related in some way to “black box” and “white box” tests. A “black box”2 test focuses on what a system does, while a “white box”3 test on how4. More a test falls within the category of “white box”, the further away from the category of “acceptance”. In other words the more an integration test falls within the category of “black box”, the closer to the category of “acceptance”.

Some agile methodologies, among them XP (eXtreme Programming) and Scrum, suggest using user stories as artifacts for requirements. Some authors, see for example Mike Cohn or Scott Ambler say that the user stories are particularly effective if concise and schematic suggesting the adoption of the following template5:

(T2) As a <role>, I want to <goal/desire>, so that <benefit>

The first impression is that the “so that” clause is optional6, but Chris Matts7 thought that it was so fundamental to propose the following template:

(T3) In order to <receive benefit> as a <role>, I want to <goal/desire>

My point of view is to accept the Chris’s thesis and to regard as essential the “in order to” clause or the equivalent “so that”. As a matter of fact any requirement needs at least an acceptance test and such clauses - in my opinion - shall be used to communicate what is expected by the acceptance tests: the <goal/desire> implies a test whose outcome shall comply with the <benefit>.

In a hypothetical tool that manages both user stories and acceptance tests, each acceptance test should be linked to a specific “so that” clause of a user story and not directly to the user story.

To hit the nail on the head, my vision of the user story template is:

(T4) As a <role>, I want to <goal/desire>, so that <benefit> {and <benefit>}

Each “benefit” – more than the entire “so that” clause - shall have at least a related acceptance test.

UML use cases and acceptance tests

I wonder if the same concepts can be conveyed through UML use cases. In this paper “use case” shall be always be interpreted as “UML use case diagram” or “UML use case artifact”, I avoid to deal with use case descriptions as they have both desirable and undesirable features compared with UML. My aim is to present an idea that fits very well with UML diagrams.

A kind of bijection may be established between a part of a user story and a use case diagram as highlighted by the Figure 1: the “As a <role>” clause corresponds to the “UML actor”, the “I want to <goal/desire>” clause corresponds to the “UML use case”. See it as a structural relationship, a theoretical possibility, not as a good or recommended practice: there are examples that make a mockery of the translation, see Alistair Cockburn8 for a sharp discussion.

Figure 1 : map between user stories and UML use cases

The UML use case artifacts are not strictly able to express the “so that” clause, they can only introduce it as a comment (see Figure 2):

Figure 2 : map between user stories and UML use cases

In this light the UML use case is less expressive than a user story: a hypothetical tool that manages both UML use cases and acceptance tests is forced to connect an acceptance test to a use case, as connecting a primary element (the acceptance test) to an auxiliary element (the comment) is logically wrong and shall be forbidden.

The situation would be quite different if the UML use case artifacts allow to define the acceptance tests as part of the artifact. I figure out that the acceptance tests are for use cases what methods are for classes (see Figure 3).

Figure 3 : Acceptance Tests are for UMLUse Cases what Methods are for Classes

Obviously I’m abusing of the current UML notation to explain my vision.

Simple and complex requirements

The requirements may be divided in two categories: simple and complex. A simple requirement involves only a single piece of business functionality, while a complex requirement involves more than a single business function. A single function does not necessarily map to only a single UML use case, as a main use case decorated with included and extended use cases shall not be intended as a complex requirement. An example will clarify the distinction.

Suppose you have the following stakeholder request:

(SR-1) The system needs to manage a list of products by adding, updating, retrieving items.

It may be rendered by the UML use case diagram in Figure 4 that specifies five simple requirements:

Figure 4 : Product management use cases

As you can notice the included “select product” use case has not been counted as simple requirement.

Suppose you have now the following stakeholder request:

(SR-2) All operations shall be logged for auditing purposes.

The naïve approach (see Figure 5) cannot be satisfactorily: in effect, how can we define acceptance tests?

Figure 5 : naive approach to complex requirements

The question shows one of the reasons why coupling UML use cases with acceptance test cases is strongly desirable: the most important attribute of a requirement is testability, so must be also for UML use cases.

The UML notation allows to define abstract use cases and it allows to define use cases that specialize other use cases and this enables you to draw the remarkable diagram shown in Figure 6.

Figure 6 : abstract use cases

This diagram is exceptionally clear to all object oriented proficient people and easily explained to all people. I would emphasize that the abstract nature of the “logged operation” use case delegates the specialized concrete use cases to supply the required acceptance tests.

Be aware of the big difference between what I called simple requirements and what I called complex requirements: the first ones allow a direct link with tests, while the latter ones do not have such facility and depend on other features.

The problem is how to ensure that all (acceptance) tests are defined.

Tests as UML use case methods

The hypothetical tool introduced earlier could handle automatically the abstract “test logged operation” for example generating the required test for all concrete derived use cases. Be aware that the use case shall anticipate the test case, but it is not its responsibility to detail it: the “use case test method” shall include the name, perhaps the input parameters, perhaps the mode (i.e. for inspection, demonstration, log analysis, etc).

Conclusion

I consider user stories a valuable tool for gathering stakeholder’s requests, but they require further elaboration to detail requirements. Moreover they are not entirely suitable for incorporation in graphical tools: if so they would be more or less equivalent to simple UML use cases. UML use cases offer richer capabilities and, although they are aging, they still retain hidden possibilities: there is room for enhancements. Many modern design tools add to UML diagrams other kind of diagrams in order to support requirements, test cases and some sort of traceability between those elements. I think that a better solution is to make the notation of the UML use cases more powerful as sketched in the previous paragraphs: I see only advantages to encapsulating (acceptance) tests inside the UML use cases, this may also open the door to the introduction of a new UML classifier, the “acceptance test” with the test details. I will end by restating that, in my humble opinion, acceptance tests are part of the design as implicitly or explicitly preached by the agile practices. In a more complete frame, the definition of a system - a high quality definition - conforms to the following equation "SYSTEM DEFINITION" = "REQUIREMENTS" + "ARCHITECTURE" + "ACCEPTANCE TESTS", but its full explanation is matter for another article.

About the Author

Raul Rugiero is a Capgemini Enterprise Software Engineer with more than twenty years of experience in the production and delivery of information systems. He worked in several industries including public administration, telco and - in recent years - aerospace and defense. Raul has held and continued to cover many roles - his favorite is software architect -: nowadays he is dealing mainly with requirements analysis, architecture design, project technical leadership and where appropriate also component development. Not a fan of any particular technology or methodology, but he follows with interest the evolution of many of them.

BDD is not the least mathematically ambiguous form of specification. Also, executable specifications existed way before BDD. BDD brought nothing to the table apart from a platform for discussion which brings in all parties with direct concerns over the software. This closes the distance between the developers and the customers, whcih is the only true value of BDD. The other problems are dealt with better by formal specification languages such as OCL, VDM, Z and Spec#.

Every running system, without exception is a product of two factors. The initial (pre-)conditions and the process. When the process is actioned from the pre-conditions, this causes a set of conditions after the event, to be true (post-conditions). Also, OCL expressions are not stand alone. They are often tied in to diagram elements (such as activity diagrams and meta-models). So it is easy to define:

'As a [role]', 'I want [goal/desire]', 'so that [benefit 1].'

as an OCL expression tied to a use-case diagram. The acceptance test is 'merely' (I do not wish to understate the importance of it) a mechanism for checking that this OCL expression is adhered to be the software under construction.

Additionally, if you take the standpoint that "the acceptance tests are part of the design" then design-by-contract principles are the only way that software is created. I have much less of an issue with this idea, as the test/BDD code is part of the system and is providing a contract for the software system to adhere to (to understand this, step away from the idea of code level interfaces and look at it in a more general sense). So 'code-first' development ceases to exist as in reality, the acceptance tests provide that contract and even when it is split out, it is often split along natural lines which are wrapped with tests before the split takes place, thereby introducing a contract for the split code to adhere to.

So given design-by-contract is the only way to develop software and mechanisms such as Spec# and OCL exist to define software under design-by-contract principles, this makes the whole idea of modifying the UML standard completely redundant. If you absolutely had to define an acceptance test criteria, fire off a spec-by-example in a note tacked on to the OCL expression.

I think you have not captured the essence of the article. "Design by Contract" is a very useful technique to assure the quality of a function, a method or a class implementation. "Design by Contract" has nothing to do with the delivery of a system. When you produce systems for yourself, you are free to not perform any kind of test - if you are confident of the quality of your system -, but if you produce systems for others, others - before accepting your delivery - always perform tests, the so called "acceptance tests", a form of black box tests for the "to be delivered" systems. "Acceptance tests" are normally stated in formal contracts between the parties so they must be defined and performed. "Design by Contract" does not eliminate the necessity to perform tests, it may make us more confidend that tests pass, especially unit tests. One of the aim of the article is to clarify that "acceptance tests" are part of design, that is part of the system definition, because they help to explain better the requirements. The other important aim of the article is to introduce a technique to assure to not forget vital acceptance tests. "Design by Contract" - as OCL expression, as other "coding extension" techniques - will never be able to assure that the code does what requirements ask. A not naive requirement typically involves many classes, many components, often in distributed architectures.

No sure, but that is not what I took from the article and I never stated 'design by contract' removes tests (far from it, it is the tests based on this that ensure symmetry of conformance). However OCL expressions DO prove a system will work and also will work without side effects, which is something Gherkin/SpecFlow cannot guarantee, as the step files themselves are subject to side effects. But that is off topic.

DbC has EVERYTHING to do with the delivery of a system. As you have said, these acceptance tests that are performed by others are effectively a 'contract' to which the software, as a black box, must adhere. Remember, you've stated that the acceptance tests are part of the design. When you create the acceptance tests, you are effectively creating a contract.

Also, the design contract does not have to be part of a white box process (for example, it could be an interface spec with pre and post conditions on the methods which in themselves define the contract that any implementing class must adhere to, even if working using consumer driven contracts. i.e. in a particular context, this is what I want the module to do). OCL contexts in can also be defined at any level, not just code.

Additionally, if I choose to play devils advocate and assume that you need a UML adaptation, what is to stop someone just using a stereotype on a note to denote the test case? Such as «testcase». There are many solutions to this problem, that is assuming (with all due respect) that there is even a problem here to be solved.

Maybe we misunderstood each other. For "formal contracts between the parties" I intend something completely different from software pre/post conditions.

But I'm interested in the discussion about the DbC.

You say "DbC has EVERYTHING to do with the delivery of a system".

In my view "DbC has EVERYTHING to do with the INTRINSIC quality of the system": it may help to write more robust software (I say "may help" and not "helps" because it depends on the reliability of pre/post conditions). I wrote (15 years ago) code in Eiffel language, so I know very well what DbC means in practice.

When I deliver a system my customers are not interested in SOFTWARE LOCAL details, they are interested in verifying "business functionality": for example that an order is stored in the database. To demonstrate it software pre/post conditions are useless, we need to access the database with an external (independent) tool and show the inserted data.

What I want to clarify is that tests required for the verification of a system, are often quite different from the invocation of a function.

But this is the bit I don't understand. Why is it different? So what if it's internal? You can use BDD to do unit tests.

Every system is some other system's subsystem. Even a piece of stand alone software is part of a business process and the user concerns depend on whether that concern is a customer, who may interact with a CSA or a CSA who may interact with the system. This is the 'As a...' in user stories, just like a UML diagram will show the actor, who performs a role.

Let's exemplify this by revisiting a pseudo-gherkin statement.

Given customer 12345 has a basket of 1 itemWhen they choose to delete that itemThen the basket contains zero items

As I mentioned before, Gherking and BDD give us the ability to get closer to the customer's view of how a system should work, but it trades off on many other aspects. UML is not for the customer, it is for IT people. So I could even ask what use would they even get out of it in the same way as BDD? The only aspect would be traceability and even that can be covered through stereotyping.

I am sure you are aware from your work in DbC that the contract defines several things. It defines the interface (statics) that should be exposed and how that interface should behave (dynamics). There is not a system in the world or nature that isn't about statics (interfaces, classes, components, meta-models, actors etc.) and their dynamics (activity diagrams, sequence/collaboration diagrams etc). There is nothing else. Everything else is fluff that can be reduced to one of those two forms. Indeed, the GoF split their patterns it more or less this way. pre and post conditions give you the specifics of what should hold before and after an operation. That is sufficient to test that the dynamics being carried out on an action give you the required result.

Your example of the order being stored in the database is exactly one that a UML model coupled with OCL expressions for pre and post conditions can define (pre-condition: record doesn't exist. post-condition: it does). A customer defining a BDD statement can see this immediately, whilst with a UML diagram they have to learn UML. That is a big ask for some, so they leave it to IT people. but I think it is not unreasonable for software developers to know enough about UML to know how to structure a model and supporting constraints to show the statics and dynamics of the system at work, without having to resort to modifying a standard, which may take a year to ratify, to introduce constructs which can be equally well, if not better dealt with using existing standards. It is just overcomplicating the UML for absolutely no reason. Occam's Razor remember :-)

Agile methodologies and techniques are for me food for thought. BDD ideas seem promising to me, but the overall context may be not suitable: unfortunately we don't live in a perfect world. Let me state the problem in the following way.

In the real world there are projects where:

- requirements are defined by team T1- solution is devised by team T2 - solution is implemented by team T3- acceptance tests are defined by team T4- ...

and perhaps all teams belong to different companies. In big and complex projects things can be even more intricate as *smart* customers never rely on a single team or company. In this context DbC, TDD, BDD and other techniques help to implement components, DbC perhaps helps to integrate components, but none of them will convince the customer to accept the software. Customer may require reports on unit tests and integration tests, but the software will be accepted only if it passes specific tests defined independently by the implementation.

Now let play the role of the team that shall specify this specific independent tests. By the way, in my view the team that defines requirements should also define acceptance tests, but it is another story.

To simplify consider the following scenario:

I have the requirement R1. This requirement R1 shall be satisfied by some software.No code has been yet written. No solution has been yet devised. In any case whatever the solution, whatever the written code, I have to verify that the delivered software satisfies R1. In other words I have to define acceptance tests.

Suppose I define the acceptance tests T1-1, T1-2, ... T1-N for the requirement R1. I can also use OCL expressions to define them: this is not the point. The point is that somewhere shall be stated that T1-1, T1-2, ... T1-N are linked to R1.

I suggested that using UML use cases to define requirements (a possibility, in my view a good choice) we need to link the UML use case with the acceptance tests. This link can be obtained specifying the tests as part of the UML use case (be aware that this does not mean specifying the body of the test inside the UML use case, just the header, that is, an unique identifier of the test). In the article I explained how the flexibility of the UML use cases and their smart usage would allow a tool to perform advanced completeness and consistency checks.

OK, so what you are basically now proposing for the form of projects which you have exemplified is that this be used for traceability and nothing more. OK, this is a different issue.

Now, I 100% agree that UML can fully define requirements. So we agree on that. However, a UML model (especially containing activity and some form of collaboration diagrams) can be used to validate a well defined collaboration against a well defined use-case activity. This is the only element of traceability (use case X implemented by collaboration X. Indeed, if you are diagramming it, it will form part of the same use-case 'document' which also includes the solution. SO labelling is not necessary) and if the solution defined in the latter model, together with OCL expression for the DbC elements within it, satisfies the 'requirement' defined by the activity diagram, then the solution works. This is testing the design before writing any code. For my sins, I work in both BDUF and Agile environments. I still actively use RUP as well as AUP, Scrum and XP and in that case, this idea is also redundant.