Thursday, September 29, 2011

Ensuring that everyone hears, understands, and implements the architects' decisions

A whole technology for doing this was worked out for the System/360 hardware design effort, and it is equally applicable to software projects.

Written Specifications—the Manual

The manual, or written specification, is a necessary tool, though not a sufficient one. The manual is the external specification of the product.

It describes and prescribes every detail of what the user sees. As such, it is the chief product of the architect. It must also refrain from describing what the user does not see. That is the implementer's business, and there his design freedom must be unconstrained.

The style must be precise, full, and accurately detailed. A user will often refer to a single definition, so each one must repeat all the essentials and yet all must agree. This tends to make manuals dull reading, but precision is more important than liveliness.

Formal Definitions

English, or any other human language, is not naturally a precision instrument for such definitions. Therefore the manual writer must strain himself and his language to achieve the precision needed. An attractive alternative is to use a formal notation for such definitions, but it also has its weaknesses so I think we will see future specifications to consist of both a formal definition and a prose definition.

If one has both, one must be the standard, and the other must be a derivative description, clearly labeled as such. Either can be the primary standard.

Direct Incorporation

A lovely technique for disseminating and enforcing definitions is available for the software system architect. It is especially useful for establishing the syntax, if not the semantics, of inter-moduleinterfaces. This technique is to design the declaration of the passed parameters or shared storage, and to require the implementations to include that declaration via a compile-time operation (a macroor a % INCLUDE in PL/I). If, in addition, the whole interface is referenced only by symbolic names, the declaration can be changed by adding or inserting new variables with only recompilation, not alteration, of the using program.

Conferences and Courts

Needless to say, meetings are necessary. The hundreds of man-to-man consultations must be supplemented by larger and more formal gatherings. We found two levels of these to be useful.

The first is a weekly half-day conference of all the architects, plus official representatives of the hardware and software implementers, and the market planners. The chief system architect presides. Anyone can propose problems or changes, but proposals are usually distributed in writing before the meeting. A new problem is usually discussed a while. The emphasis is on creativity, rather than merely decision. The group attempts to invent many solutions to problems, then a few solutions are passed to one or more of the architects for detailing into precisely worded manual change proposals.

Detailed change proposals then come up for decisions. These have been circulated and carefully considered by implementers and users, and the pros and cons are well delineated. If a consensus emerges, well and good. If not, the chief architect decides. Minutes are kept and decisions are formally, promptly, and widely disseminated.

Decisions from the weekly conferences give quick results and allow work to proceed. If anyone is too unhappy, instant appeals to the project manager are possible, but this happens very rarely.The fruitfulness of these meetings springs from several sources:

The same group—architects, users, and implementers—meets weekly for months. No time is needed for bringing people up to date.

The group is bright, resourceful, well versed in the issues, and deeply involved in the outcome. No one has an "advisory" role. Everyone is authorized to make binding commitments.

When problems are raised, solutions are sought both within and outside the obvious boundaries.

The formality of written proposals focuses attention, forces decision, and avoids committee-drafted inconsistencies.

The clear vesting of decision-making power in the chief architect avoids compromise and delay.

As time goes by, some decisions don't wear well. Some minor matters have never been wholeheartedly accepted by one or another of the participants. Other decisions have developed unforeseen problems, and sometimes the weekly meeting didn't agree to reconsider these. So there builds up a backlog of minor appeals, open issues, or disgruntlements. To settle these we held annual supreme court sessions, lasting typically two weeks. (I would hold them every six months if I were doing it again.)

These sessions were held just before major freeze dates for the manual. Those present included not only the architecture group and the programmers' and implementers' architectural representatives, but also the managers of programming, marketing, and implementation efforts. The project manager presided. The agenda typically consisted of about 200 items, mostly minor, which were enumerated in charts placarded around the room. All sides were heard and decisions made. By the miracle of computerized text editing (and lots of fine staff work), each participant found an updated manual, embodying yesterday's decisions, at his seat every morning.These "fall festivals" were useful not only for resolving decisions, but also for getting them accepted. Everyone was heard, everyone participated, everyone understood better the intricate constraints and interrelationships among decisions.

The Telephone Log

As implementation proceeds, countless questions of architectural interpretation arise, no matter how precise the specification. Obviously many such questions require amplifications and clarifications in the text. Others merely reflect misunderstandings. It is essential, however, to encourage the puzzled implementer to telephone the responsible architect and ask his question, rather than to guess and proceed. It is just as vital to recognize that the answers to such questions are ex cathedra architectural pronouncements that must be told to everyone.One useful mechanism is a telephone log kept by the architect. In it he records every question and every answer. Each week the logs of the several architects are concatenated, reproduced, and distributed to the users and implementers. While this mechanism is quite informal, it is both quick and comprehensive.

Product Test

The project manager's best friend is his daily adversary, the independent product-testing organization. This group checks machines and programs against specifications and serves as a devil's advocate, pinpointing every conceivable defect and discrepancy. Every development organization needs such an independent technical auditing group to keep it honest. In the last analysis the customer is the independent auditor.

Architect - Needed Disciplines

Having in mind that we have separated responsibility for functional specification (Architect) from responsibility for building a fast, cheap product (Builder), now we have to answer the question of: how we will achieve the best outcome of our project? The fundamental answer is thoroughgoing, careful, and sympathetic communication between architect and builder.

Nevertheless there are little details that must to be observed for the Architect in order to get the ideal result, and these are:

Interactive Discipline

The architect has two possible answers when confronted with an estimate that is too high: cut the design or challenge the estimate by suggesting cheaper implementations. This latter is inherently an emotion-generating activity. The architect is now challenging the builder's way of doing the builder's job. For it to be successful, the architect must:

Remember that the builder has the inventive and creative responsibility for the implementation; so the architect suggests, not dictates;

Always be prepared to suggest a way of implementing anything he specifies, and be prepared to accept any other way that meets the objectives as well;

Deal quietly and privately in such suggestions;

Be ready to forego credit for suggested improvements.

Normally the builder will counter by suggesting changes to the architecture. Often he is right—some minor feature may have unexpectedly large costs when the implementation is worked out.

Self-Discipline—The Second-System Effect

An architect's first work is apt to be spare and clean. He knows he doesn't know what he's doing, so he does it carefully and with great restraint.

As he designs the first work, frill after frill and embellishment after embellishment occur to him. These get stored away to he used "next time." Sooner or later the first system is finished, and the architect, with firm, confidence and a demonstrated mastery of that class of systems, is ready to build a second system.

This second is the most dangerous system a man ever designs. The general tendency is:

To over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one.

Tendency to refine techniques whose very existence has been made obsolete by changes in basic system assumptions.

How does the architect avoid the second-system effect? Well, obviously he can't skip his second system. But he can be conscious of the peculiar hazards of that system, and exert extra self-discipline to avoid functional ornamentation and to avoid extrapolation of functions that are obviated by changes in assumptions and purposes.

Sunday, September 18, 2011

Basic Information

JavaScript is a case-sensitive language. Many client-side JavaScript objects and properties have the same names as the HTML tags and attributes they represent. While these tags and attribute names can be typed in any case in HTML, in JavaScript they typically must be all lowercase.

JavaScript’s strings (and its arrays) use zero-based indexing.

When the JavaScript interpreter starts, it creates a new global object that has: Properties like undefined, Infinity, and NaN , Functions like isNaN(), parseInt(), and eval(), Constructor functions like Date(), RegExp(), String(), Object(), and Array() and Global objects like Math and JSON.

In client-side JavaScript, the Window object serves as the global object for all JavaScript code contained in the browser window it represents. This global Window object has a self-referential window property that can be used instead of this to refer to the global object.

Primitive values

Primitives are immutable: there is no way to change (or “mutate”) a primitive value. Primitives are also compared by value: two values are the same only if they have the same value. The primitive values are:

Numbers, strings, or booleans.

null and undefined.

Numbers

Arithmetic in JavaScript does not raise errors in cases of overflow, underflow, or division by zero. Check the following table:

Case

Result

Overflow

It could be either:Infinity or -Infinity

Infinity as one operand

It could be either:Infinity or -Infinity

Division by Zero

It could be either:Infinity or -Infinity

0/0

NaN

For more complex mathematical operations check the Math object.

Boolean values

The Boolean values are: true and false, aside of that when a value different from boolean has to be converted to one JavaScript converts it to true or false base on the following table:

Values

Convert to

undefined, null, 0, –0, NaN, ""

false

All other values, including all objects (and arrays)

true

null and undefined

Both does not neither properties nor methods and indicate an absence of value. Comparing both using “==” we will get “true”

null

undefined

Description

It is a language keyword

It is the value of variables that have not been initialized. The value you get when you query the value of an object property or array element that does not exist. It is also returned by functions that have no return value, and the value of function parameters for which no argument is supplied.

Represent

Program-level, normal, or expected absence of value

system-level, unexpected, or error-like absence of value

typeof

"object"

"undefined"

Object

It is a collection of properties where each property has a name and a value. an unordered collection of named values. It is mutable.

Objects are not compared by value, are compared by reference: two object values are the same if and only if they refer to the same underlying object. Core JavaScript defines the following useful classes.

Name

Definition

Array

Represents an ordered collection of numbered values.

Function

A function is an object that has executable code associated with it.

Date

Defines objects that represent dates.

RegExp

Defines objects that represent regular expressions.

Error

Defines objects that represent syntax and runtime errors that can occur in a JavaScript program.

Saturday, September 17, 2011

Conceptual Integrity

I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas. Unfortunately most programming systems reflect big conceptual disunity. Usually this arises not from a serial succession of master designers, but from the separation of design into many tasks done by many men.

Taking into account that the purpose of a programming system is to make a computer easy to use, the ratio of function to conceptual complexity is the ultimate test of system design. Neither function alone nor simplicity alone defines a good design.

Simplicity and straightforwardness proceed from conceptual integrity. Every part must reflect the same philosophies and the same balancing of desiderata. Every part must even use the same techniques in syntax and analogous notions in semantics. Ease of use, then, dictates unity of design, conceptual integrity.

How to get conceptual integrity ?

Conceptual integrity dictates that the design must proceed from one mind, or from a very small number of agreeing resonant minds. Schedule pressures, however, dictate that system building needs many hands. Two techniques are available for resolving this dilemma:

The first is a careful division of labor between architecture and implementation.

The second is the new way of structuring programming implementation teams such as surgical team.

Separation of labor between architecture and implementation.

It is a very powerful way of getting conceptual integrity on very large projects.By the architecture of a system, I mean the complete and detailed specification of the user interface. For a computer this is the programming manual. For a compiler it is the language manual. For a control program it is the manuals for the language or languages used to invoke its functions. For the entire system it is the union of the manuals the user must consult to do his entire job.

The architect of a system, like the architect of a building, is the user's agent. It is his job to bring professional and technical knowledge to bear in the unalloyed interest of the user, as opposed to the interests of the salesman, the fabricator, etc.

Architecture must be carefully distinguished from implementation. As Blaauw has said, "Where architecture tells what happens, implementation tells how it is made to happen."

Objections to this technique

This is criticized with the following questions:

Are not the architects a new aristocracy, an intellectual elite, set up to tell the poor dumb implementers what to do?

The answer to this must be yes and no.Yes, in the sense that there must be few architects, their product must endure longer than that of an implementer, and the architect sits at the focus of forces which he must ultimately resolve in the user's interest. If a system is to have conceptual integrity, someone must control the concepts. That is an aristocracy that needs no apology.

No, because the setting of external specifications is not more creative work than the designing of implementations. It is just different creative work. The design of an implementation, given an architecture, requires and allows as much design creativity, as many new ideas, and as much technical brilliance as the design of the external specifications. Indeed, the cost-performance ratio of the product will depend most heavily on the implementer, just as ease of use depends most heavily on the architect.

Has not all the creative work been sequestered for this elite, leaving the implementers as cogs in the machine?

Similarly, I observe that the external provision of an architecture enhances, not cramps, the creative style of an implementing group. They focus at once on the part of the problem no one has addressed, and inventions begin to flow. In an unconstrained implementing group, most thought and debate goes into architectural decisions, and implementation proper gets short shrift.

Won't one get a better product by getting the good ideas from all the team, following a democratic philosophy, rather than by restricting the development of specifications to a few?

I will certainly not contend that only the architects will have good architectural ideas. Often the fresh concept does come from an implementer or from a user. However, all my own experience convinces me, and I have tried to show, that the conceptual integrity of a system determines its ease of use. Good features and ideas that do not integrate with a system's basic concepts are best left out. If there appear many such important but incompatible ideas, one scraps the whole system and starts again on an integrated system with different basic concepts.

Thursday, September 8, 2011

How to take forward really big projects

Small sharp team concept is not a viable solution because it is too slow. So we have to find other options.

The dilemma

For efficiency and conceptual integrity, one prefers a few good minds doing design and construction. For large systems one wants a way to bring considerable manpower to bear, so that the product can make a timely appearance.So how can these two needs be reconciled?

Harlan Mills's Proposal

Mills proposes that each segment of a large job be tackled by a team, but that the team be organized like a surgical team and not a hog-butchering team.

Surgical team

One does the cutting and the others give him every support that will enhance his effectiveness and productivity.

Hog-butchering team

Each member cutting away on the problem.

Surgical Team - Roles and work division

The following are the roles of 10 people that contribute in well differentiated and specialized roles on a programming team built on the surgical model:

The surgeon

Mills calls him a chief programmer. He needs great talent, ten years experience, and considerable systems and application knowledge, whether in applied mathematics, business data handling, or whatever.

His main responsibilities are:

Defines the functional and performance specifications .

Designs, codes and tests the program.

Writes its documentation.

He needs to have effective access to a computing system which not only runs his tests but also stores the various versions of his programs, allows easy file updating, and provides text editing for his documentation.

The copilot

He is the alter ego of the surgeon, able to do any part of the job, but is less experienced. His main responsibilities are:

Shares in the design as a thinker, discussant, and evaluator. The surgeon tries ideas on him, but is not bound by his advice.

Knows all the code intimately.

Researches alternative design strategies.

He may even write code, but he is not responsible for any part of the code.

The administrator.

The surgeon is boss, and he must have the last word on personnel, raises, space, and so on, but he must spend almost none of his time on these matters. Thus he needs a professional administrator who handles money, people, space, and machines, and who interfaces with the administrative machinery of the rest of the organization. He can serve two teams.

The editor.

The surgeon is responsible for generating the documentation; for maximum clarity he must write it. This is true of both external and internal descriptions.

The editor, however, takes the draft or dictated manuscript produced by the surgeon and criticizes it, reworks it, provides it with references and bibliography, nurses it through several versions, and oversees the mechanics of production.

Two secretaries.

The administrator and the editor will each need a secretary; the administrator's secretary will handle project correspondence and non-product files.

The program clerk.

He is responsible for maintaining all the technical records of the team in a programming-product library. The clerk is trained as a secretary and has responsibility for both machine-readable and human-readable files.

All computer input goes to the clerk, who logs and keys it if required. The output listings go back to him to be filed and indexed. The most recent runs of any model are kept in a status notebook; all previous ones are filed in a chronological archive.

The specialized function of the program clerk relieves programmers of clerical chores, systematizes and ensures proper performance of those oft-neglected chores, and enhances the team's most valuable asset—its work-product.

The toolsmith.

File-editing, text-editing, and interactive debugging services are now readily available, so that a team will rarely need its own machine and machine-operating crew.

But these services must be available with unquestionably satisfactory response and reliability; and the surgeon must be sole judge of the adequacy of the service available to him. He needs a toolsmith, responsible for ensuring this adequacy of the basic service and for constructing, maintaining, and upgrading special tools—mostly interactive computer services—needed by his team.

His job is to see to the tools needed or wanted by his surgeon, without regard to any other team's needs. The tool-builder will often construct specialized utilities, catalogued procedures, macro libraries.

The tester.

The surgeon will need a bank of suitable test cases for testing pieces of his work as he writes it, and then for testing the whole thing. The tester is therefore both an adversary who devises system test cases from the functional specs, and an assistant who devises test data for the day-by-day debugging. He would also plan testing sequences and set up the scaffolding required for component tests.

The language lawyer.

By the time Algol came along, people began to recognize that most computer installations have one or two people who delight in mastery of the intricacies of a programming language. And these experts turn out to be very useful and very widely consulted. The talent here is rather different from that of the surgeon, who is primarily a system designer and who thinks representations. The language lawyer can a neat and efficient way to use the language to do difficult, obscure, or tricky things.

Often he will need to do small studies (two or three days) on good technique. One language lawyer can service two or three surgeons.

How It Works

The team just defined meets the desiderata in several ways. Ten people, seven of them professionals, are at work on the problem, but the system is the product of one mind—or at most two, acting uno animo.

Notice in particular the differences between a team of two programmers conventionally organized and the surgeon-copilot team.

First, in the conventional team the partners divide the work, and each is responsible for design and implementation of part of the work. In the surgical team, the surgeon and copilot are each cognizant of all of the design and all of the code. This saves the labor of allocating space, disk accesses, etc. It also ensures the conceptual integrity of the work.

Second, in the conventional team the partners are equal, and the inevitable differences of judgment must be talked out or compromised.Since the work and resources are divided, the differences in judgment are confined to overall strategy and interfacing, but they are compounded by differences of interest—e.g., whose space will be used for a buffer. In the surgical team, there are no differences of interest, and differences of judgment are settled by the surgeon unilaterally.

These two differences—lack of division of the problem and the superior-subordinate relationship—make it possible for the surgical team to act uno animo. Yet the specialization of function of the remainder of the team is the key to its efficiency, for it permits a radically simpler communication pattern among the members.

Saturday, September 3, 2011

Identifying the joys and woes inherent to the system programming.

The Joys of the Craft

Following we can find some answers to the questions: Why is programming fun? What delights may its practitioner expect as his reward?

The sheer joy of making things.

The pleasure of making things that are useful to other people.

The fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles, playing out the consequences of principles built in from the beginning.

The joy of always learning, which springs from the nonrepeating nature of the task.

The delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff.

The Woes of the Craft

One must perform perfectly.

Other people set one's objectives, provide one's resources, and furnish one's information. One rarely controls the circumstances of his work, or even its goal.

Designing grand concepts is fun; finding nitty little bugs is just work. With any creative activity come dreary hours of tedious, painstaking labor, and programming is no exception.

One finds that debugging has a linear convergence, or worse, where one somehow expects a quadratic sort of approach to the end. So testing drags on and on, the last difficult bugs taking more time to find than the first.

The product over which one has labored so long appears to be obsolete upon (or before) completion.

Why more software projects have gone awry for lack of calendar time?

Our techniques of estimating are poorly developed.

Our estimating techniques fallaciously confuse effort with progress.

Because we are uncertain of our estimates.

Schedule progress is poorly monitored. Techniques proven and routine in other engineering disciplines are considered radical innovations in software engineering.

When schedule slippage is recognized, the natural (and traditional) response is to add manpower. Like dousing a fire with gasoline, this makes matters worse, much worse.

About creative activity

Dorothy Sayers, in her excellent book, The Mind of the Maker, divides creative activity into three stages:

The idea,

A book, then, or a computer, or a program comes into existence first as an ideal construct, built outside time and space, but complete in the mind of the author.

The implementation,

It is realized in time and space, by pen, ink, and paper, or by wire, silicon, and ferrite.

The interaction.

The creation is complete when someone reads the book, uses the computer, or runs the program, thereby interacting with the mind of the maker.

Demythologizing of the man-month.

The number of months of a project depends upon its sequential constraints. The maximum number of men depends upon the number of independent subtasks. From these two quantities one can derive schedules using fewer men and more months. (The only risk is product obsolescence.)