Agile IQ

Wednesday, 22 November 2017

Ramayana, India’s aadi kavya or first (grand) poem, written in Sanskrit around 1000 – 500 BC; is a book of epic significance in the sub-continent. Although the story of Rama, the eponymous hero of this book is well known across India ever since; the book in any of the Dravidian languages – the language of the people in the South of India, was not available till the 12th century, almost 2000 years later. The book being available in the language of the common people had a lot of cultural and social significance. To quote Wikipedia, “Ramayana is not just a story: it presents the teachings of ancient Hindu sages in narrative allegory, interspersing philosophical and ethical elements”. Till that time, only the elite few belonging to the royalty or the upper castes could learn Sanskrit and understand the philosophy of this epic. Ramayana in local language democratized this knowledge.

Thunchath Ramanujan Ezhuthachan (popularly known by his surname) wrote Ramayana in the Malayalam language. He faced an interesting challenge when he started to write it. The Malayalam script at that time had only 30 characters and was not enough to represent words from Sanskrit, which has 52 characters. To solve this, he created a new script with 51 characters which is used even today as the modern Malayalam script. Using this new script, he could express the philosophy of the Sanskrit text clearly in Malayalam.

In my last blog about programming microservices, we discussed about three characteristics which developers find tough to get it always right. Functional Composition, where we compose higher level functions from first order functions, is one of them. Programmers are used to the more conventional way of composing structurally. Structural composition usually creates more dependencies, which don’t go very well with microservices.

Functional composition can be expressed better with a programming language which is not procedural/imperative. The most popular programming languages – C, C++ & Java are conventionally imperative. A functional programming language is very suitable, yet some may find its demanding paradigm shift a bit too much. A multi-paradigm language may be a better go-between option.

There are some more features of some programming languages that the developer may use to express functionally. The function closure in javascript simplifies the data exchange between functions, as against the block closure found in most programming languages.

…

//records A, B & C available in this scope

function someFunction () {

…

function innerFunction () {

//access records A & B

//update records A & B

};

//access record C

//update record C

};

…

In the example given above, inner functions can access & modify data from the outer functions without explicitly passing them as parameters. In imperative languages, this is usually achieved by parameter passing, often creating the need for many data structures. Other conventional alternatives are using global or class variables.

Functional composition often requires the functions to be passed as parameters/objects to other functions. Most programming languages support this through various techniques like function pointers, call-backs, OOPS or dynamic types. Support for lambdas or anonymous functions (with a fair share of syntactic sugar) can be a big plus for expressing better in this case.

…

function sumOfPowers (a, b, fn) {

return fn(a) + fn(b);

};

var sumOfSquares = sumOfPowers(x, y, x=>x*x);

var sumOfCubes = sumOfPowers(x, y, x=>x*x*x);

…

C, C++ (till version 10) and Java (till version 7) don’t support lambdas. Multi-paradigm languages like JavaScript, python and all functional programming languages support lambda.

Programmers may consider these features when choosing a programming language for functional composition. Those who don’t, may end up with an application composed structurally and with more dependencies than originally thought.

Sunday, 12 November 2017

I remember watching the movie Avatar, by James Cameron at IMAX. Seeing the richly built world of Pandora on a larger than life screen was awe inspiring. Like every true fan, I looked beyond the state of the art CGI and was hooked on to Na’vi, the language of the natives, that James created for his movie.

Reviewers from movie magazines asked James the need of creating a new language. James explained that he had many requirements which necessitated it. First was for the symbolism, the metaphorical references to oppression, colonialism and annihilation of the indigenous communities and the philosophy of the nourishing, all providing, omnipresent female. James used voiceless consonants in the language, as used by many primordial languages of Asia and Americas and modeled Na’vi based on the Mayan and Polynesian dialects to achieve this.

He also wanted the language to allow his actors to emote expressively, in addition to it being nice to hear and easy to speak. The language uses varying tones and vowels of varying length to achieve an expressive yet pleasing intonation. The new language helped him to not to limit his creativity based on the conventions of English.

When developing Microservices, it is relatively easy to do what is required. Components and RESTful interfaces are some of the low hanging fruits. As we develop, we discover that it is difficult not to do what is not required. The hard-to-avoid antipatterns are structured composition, centralised data and about managing mutability.

Microservices need to be composed functionally, not structurally. Teams fall back on structural composition as an incidental effect of using a programming language with a procedural/imperative style. The most popular programming languages - C, C++ and Java are imperative and coding in them let you compose structurally, usually in a pattern of some central providers and many contextual users. The problem with structured composition is that it creates dependencies (on the providers) which makes it difficult in creating independent and loosely coupled Microservices. A litmus test for your composition would be to look at the module names and check whether they are in English or geekish. For e.g. module names like iNode, checkpoint etc. suggests a structured composition where as names like replication, backup suggests a functional approach.

Centralised data is a consequence of dependencies created by building your application structurally. Decentralising requires intuitive frameworks, which may need to be created for the popular programming languages like C, C++ and Java, using some reactive programming techniques.

Most of the popular programming languages have mutable data types, which creates a need for managing mutability to enable concurrency. This is an incidentally complex problem in C, C++ and Java, requiring locks and identifying critical sections. Even the best implementations have deadlocks and race conditions, in spite of these synchronised methods. Is there a simpler alternative?

In the next part of this blog, we will look at how to avoid these antipatterns and the features of some programming languages that would help us to do so.

What lessons, if any, can we learn from this, and use in the world of software design?

Any software has three sets of characteristics; the functional part which is about the features, the operational part which deals with aspects like security, performance and concurrency and the developmental aspects of being readable, easy to debug and extensible. These three characteristics are orthogonal, i.e. changing one will not usually affect the other. Since they are orthogonal, it is not very intuitive to consider them together, say security and being extensible, at the same time.

A good design is one where these three characteristics are visited iteratively, and one at a time.

Let us continue with the same example to do some iterative design for the developmental and operational aspects.

A good programmer once told me that loops should do nothing, but only loop. Let us refactor the loop code by extracting it as a method for better readability.

...

for(intn=0;n<cols;n++)

{

printColumn(s, rows, cols, n);

}
...

Next, we look at the concurrency design, and find that the print functions cannot assure it. We can use some indirection here and achieve concurrency as follows:

...

publicstaticvoid encrypt(String s)

{

encryptThreadSafe(s, new PrintStreamThreadSafe(System.out));

}

publicstaticvoid encryptThreadSafe(String s, PrintStream p)

{

…

for(intn=0;n<cols;n++)

{

printColumn(s, rows, cols, n, p);

}

p.println("");

}
...

We will look at another operational aspect now, by securing our code. In my experience, tools are an invaluable asset in secure design. You may use a combination of code scanning tools, like Coverity and application scanning tools, like Nessus for this.

In our code example, let us take care of a common security flaw, handling the null input, as follows

...

publicstaticvoid encrypt (String s) throws IllegalArgumentException

{

if(s == null)

{

thrownew IllegalArgumentException("Null input");

}

...

In our small example, after two or three iterations of handling the operational and developmental parts, our iterative design is satisfactory.

In bigger implementations, we would have to do more number of iterations to be satisfied with our design.

We can visualize the state of our design, any time, as below:

Design Radar

Initially, the design would be skewed towards functionality. After some iterations of addressing the operational & developmental concerns, we would have a more well-rounded(!) design.

We can configure our CI system to regularly generate such snapshots of our design; where the functional indices may be based on the function test passing rate and suitable tools, like Coverity, Structure-101, Nessus, JMeter, Valgrind etc. may be used to suggest the operational and developmental health.

When we find that our operational and developmental design is poor, it would be better to pause adding new functions and have a Zero-Feature sprint to catch up.

Monday, 14 December 2015

Code like a sloth

A programmer’s guide to incremental coding

Set a timer (in your mobile) for 21 minutes.

We will do a programming puzzle now. It is from HackerRank, a popular
competitive programming site. While we are at it, when the timer fires; we will
stop what we are doing (bang!) and come back to this blog.

Answer the following questions. If the answer to any question is “No”,
skip the rest of the questions and read on.

1.Does your code compile (as applicable)?
Any runtime error?

2.Do you know what all cases/scenarios are
handled by your code? Are you highly confident about it?

3.Do you know how your code may behave for
any unhandled case/scenario? Are you highly confident about it?

4.Does the code pass the test cases when
you click the “Run Code” button?

5.Go ahead
and submit your code. The site will run more tests on your code now. Are the
results as expected?

Coding in a sprint/iteration is also
somewhat like this exercise. Software product development is essentially
problem solving in nature. Coding is like solving a puzzle. I love solving puzzles and that is why I am a programmer. In the sprint, the programmer starts
developing an increment, assuming a solution. The assumption can be tested only
by coding it; thus the solution evolves while coding. Somewhere during this
process, the time box of the iteration ends. At this time, the only question
that matters is how well done is your increment? We would understand later in this
discussion that it is not really about how much of the planned work we
completed or what percentage (ugh!) of the feature we coded.

Did the titular reference to the sloth get
you curious? Let me explain the use of this metaphor.

The sloth is a much misunderstood animal.
Many would think about it as a lazy dozer. It may be surprising to know that
they don’t sleep that much, but only for 8 or 9 hours a day. Being still for
most part of the day is simply, the defense against their worst predator – the
eagle.

The eagle is a very formidable predator,
swift and with a keen eye for a prey. Being inconspicuous is probably your best
defense against something that can dive bomb and snatch you off a branch, just
like that. They have some adaptations that help them to be unseen, like their
fur which is very suitable for algal growth. From the sky, they would probably
look like some dull branch or fruit. Their lazy, yet amazingly simple
defense has been so effective that the sloths has been around for 100 million
years or so (to put that in perspective, some of our earliest known ancestors
are only 6 million years old) and did not require any further adaptation for
natural selection all this time.

Incremental coding is like the sloth;
mostly misunderstood, although amazingly simple in its scheme.

Let us attempt the same puzzle by doing incremental coding. I have grouped the activities in a sequence of six steps as follows:

1.Fail
loud and clear

Make clear decisions about what scenarios are included and what are excluded. In incremental development, it is more important to decide what not to do than what to; be it for the product, the iteration or the increment.

Apple decided not to support SD cards in iPhone. Contrast it against the multiple usability issues reported on android versions till date related to SD card support. The android community also agrees that their support is not (yet) full and promises some improvements in the Marshmallow version.

Once we decide about the exclusions, define our behavior it to it clearly. It is suggested to fail loud and clear, like throwing an exception in Java.

As we start, let us exclude everything and fail with a special string, "_E". Don't forget to test it with a simple test case like encrypt("xyz") and check for the expected failure.

publicstaticvoid encrypt(String s)

{

System.out.println("_E");

}

2.Start
with the simplest case

We will start with the simplest case. In this case, a two letter string would be good enough to start. We will implement it in the simplest way (as shown below) and yes! we would test it by calling encrypt("ab") and expecting "a b" as the output.

…

if(s.equals("ab"))

System.out.println("a b");

else

System.out.println("_E");

…

3.Build
upon it

Now let us build upon our "early win".

Let us implement encrypt("cd"), again with a simple hack.

publicstaticvoid encrypt(String s)

{

System.out.print(s+" ");

if(s.equals("ab"))

System.out.println("a b");

elseif(s.equals("cd"))

System.out.println("c d");

else

System.out.println("_E");

}

We can use the same approach for any other tow-lettered string. However, we know there are a large number of such combinations to be handled in if-else. So we refactor the code as shown below. (Note: I have retained the commented code to illustrate the guideline of don't burn the bridges. We can fallback to the commented code if we get stuck.)

…

/*if(s.equals("ab"))

System.out.println("a b");

else if(s.equals("cd"))

System.out.println("c d");*/

if(s.length() == 2)

{

System.out.println(s.charAt(0)+" "+s.charAt(1));

}

else

System.out.println("_E");

…

A 3 lettered string would be encoded in a 2x2 matrix. Thus the first column would be char[0] followed by char[2] and the next column will have only char[1]. We can reuse the code for the two lettered string and extend it as follows:

We notice that the code handling the five-lettered input is a rather long line and we could refactor it so as to print the output column-wise, as shown below:

…

System.out.print(s.charAt(0)+""+s.charAt(3));

System.out.print(" ");

System.out.print(s.charAt(1)+""+s.charAt(4));

System.out.print(" ");

System.out.print(s.charAt(2));

System.out.println("");

…

4.Observe
the patterns

Let us reuse the code which handles the five lettered string and enhance it to handle the six lettered string.

…

elseif(s.length() == 6)

{

System.out.print(s.charAt(0)+""+s.charAt(3));

System.out.print(" ");

System.out.print(s.charAt(1)+""+s.charAt(4));

System.out.print(" ");

System.out.print(s.charAt(2)+""+s.charAt(5));

System.out.println("");

}

…

We observe that the above code prints each column. The characters in each column also follows a pattern; viz. char[0] and char[0+3] for the 0th column, char[1] and char[1+3] for the 1st column etc. We refactor the code based on this pattern:

…for(intn=0;n<3;n++)

{

System.out.print(s.charAt(n)+""+s.charAt(n+3));

if(n<2)

System.out.print(" ");

}

…

We also observe the pattern that the maximum number of characters in a column is equal to the number of rows (2 in this case). We refactor again based on this pattern.

…

for(intn=0;n<3;n++)

{

for(intm=0;m<2;m++)

{

System.out.print(s.charAt(n+m*3));

}

if(n<2)

System.out.print(" ");

}

When we reuse this code for a seven lettered input, we would get an OOB exception as shown below. This is because the code would try to access charAt(7).

…

for(intn=0;n<3;n++)

{

for(intm=0;m<3;m++)

{

System.out.print(s.charAt(n+m*3)); --> OutOfBounds
exception!!

}

if(n<2)

System.out.print(" ");

}

…

We can avoid the exception with a simple bounds check.

…for(intm=0;m<3;m++)

{

if(n+m*3 >= 7)

break;

System.out.print(s.charAt(n+m*3));

}

5.Combine
logically

To handle an eight-lettered input, we would refactor the bounds check as below.

…

for(intm=0;m<3;m++)

{

if(n+m*3 >= s.length())

break;

System.out.print(s.charAt(n+m*3));

}

…

We could see that strings of 7 <= length < 10 will be encoded using a 3x3 matrix and we can refactor the code for all these cases as follows:

…

{

introws = 3;

intcols = 3;

for(intn=0;n<cols;n++)

{

for(intm=0;m<rows;m++)

{

if(n+m*cols >= s.length())

break;

System.out.print(s.charAt(n+m*cols));

}

if(n<cols - 1)

System.out.print(" ");

}

System.out.println("");

}

…

We will reuse the code to handle strings of 10 <= length < 13 (which needs a 3x4 matrix) by only changing the number of columns.

…

elseif(s.length() < 13)

{

introws = 3;

intcols = 4;

…

Now let us compute the rows and columns as suggested in the problem statement.

…

introws = (int) Math.floor(Math.sqrt(s.length()));

intcols = (int) Math.ceil(Math.sqrt(s.length()));

…

However, to handle a string of 13 characters, we will have adjust the rows as follows:

…

if(rows*cols < s.length())

rows++;

…

Thus the logic of our encrypt function would have evolved as follows:

…

introws = (int) Math.floor(Math.sqrt(s.length()));

intcols = (int) Math.ceil(Math.sqrt(s.length()));

if(rows*cols < s.length())

rows++;

for(intn=0;n<cols;n++)

{

for(intm=0;m<rows;m++)

{

if(n+m*cols >= s.length())

break;

System.out.print(s.charAt(n+m*cols));

}

if(n<cols - 1)

System.out.print(" ");

}

System.out.println("");

…

And yes! we would handle the spaces also ...

s = s.replaceAll("\\s", "");

6.Reflect

The programmer may code only for half of the time; in the other half, one should reflect. Reflection may refer to any activity in which the programmer re-looks at the code including testing, refactoring or pairing.

In our example above, let us reflect by adding some more test cases, including some long strings and some special strings (only white spaces etc.).

Let us also refactor, the loop looks like a good place to start with.

To test your understanding of incremental coding, take another programming puzzle. Set the timer and start coding. When the timer fires, answer the set of questions which we have seen in the beginning of this blog. And ask yourself, is your increment well done?