April
- Implementation Patterns, Kent Beck (3rd time)
- The Power of Habit: Why We Do What We Do in Life and Business, Charles Duhigg
- How to Win Friends and Influence People, Dale Carnegie
- Understanding the Four Rules of Simple Design, Corey Haines (2nd time)
- La llamada de Cthulhu (The Call of Cthulhu), El horror de Dunwich (The Dunwich Horror) and Las ratas en las paredes (The Rats in the Walls), H. P. Lovecraft
- The Tales of Beedle the Bard, J. K. Rowling

Saturday, December 24, 2016

"...finding and expressing symmetry is a preliminary step to removing duplication. If a similar thought exists in several places in the code, making them symmetrical to each other is a first good step towards unifying them"

In this post we'll look at an example of expressing symmetry as a way to make duplication more visible. This is the initial code of a version of a subset of the Mars Rover kata that Álvaro García and I used in a refactoring workshop some time ago:

This code is using conditionals to express two consecutive decisions:

Which command to execute depending on a command code.

How to execute the command depending on the direction the rover faces.

These decisions are repeated for different responsibilities, displacing the rover and rotating the rover, so the code presents a Case Statements smell.
Since those decisions are completely independent, we could mechanically refactor the code to start using polymorphism instead of conditionals. This way we'll end having two consecutive single dispatches, one for each decision.

Looking carefully more carefully, we can observe that in the case of the rover's displacement, the sequence of two decisions is clearly there:

However, that sequence is not there in the case of the rover's rotations. In this case there's a third decision (conditional) based on the command type which as, we'll see, is not necessary:

This difference between the two sequences of decisions reveals a lack of symmetry in the code.
It's important to remove it before starting to refactor the code to substitute the conditionals with polymorphism.

I've seen many cases in which developers naively start extracting methods and substituting the conditionals with polymorphism starting from asymmetrical code like this one. This often leads them to create entangled and leaking abstractions.

Notice how the Direction class contains a decision based on the encoding of the command. To be able to take that decision, Direction needs to know about the encoding, which is why it's been made public in the Command class.

This is bad... Some knowledge about the commands encoding has leaked from the Command class into the Direction
class. Direction shouldn't be taking that decision in the first place. Neither should it know how commands are encoded.
Moreover, this is a decision that was already taken and doesn't need to be taken again.

How can we avoid this trap?

We should go back to the initial code and instead of mechanically applying refactoring, we should start by removing the asymmetry from the initial code.

You can see one way of doing it in this video:

After this refactoring all cases are symmetrical (they present the same sequence of decisions)

We've not only removed the asymmetry but we've also made more explicit the case statements (Case Statements smell) on the command encoding and the duplicated switches on the direction the rover is facing:

Now it's clear that the third decision (third nested conditional) in the original rover's rotations code
was unnecessary.

As we said at the beginning, symmetry is a very useful concept that can help you guide refactoring. Detecting asymmetries and thinking why they happen, can help you to detect hidden duplication and, as in this case, sometimes entangled dimensions of complexity[1].

Then, by removing those asymmetries, you can make the duplication more visible and disentangle the entangled dimensions of complexity. The only thing you need is to be patient and don't start "obvious" refactorings before thinking a bit
about symmetry.

Some members of the team wanted to learn Clojure, so we started a small Clojure/ClojureScript study group.

We created a slack called clojuresai where I'm posting some readings (we're reading Clojure for the Brave and True) and exercises (we're working through a selection of exercises from 4Clojure) each week and where they can ask doubts about the exercises and readings, and comment things they find interesting.

I'd like to especially thank Ángel and Magento Barcelona for letting me do the talk today from their office (their internet connection is much better than mine).

Recently, Miguel de la Cruz (thanks a million Miguel!!) has joined our study group as a tutor. He knows a lot of Clojure so he will help answering doubts and giving some talks that we'll hopefully share with you soon, as well.

In this post, we'll show how to create your own effect by defining one that writes to the local storage.

Let's have a look at an event handler using that effect that writes to the local storage:

Notice the a key-value pair having the key :write-localstore in the effects map returned by the event handler. This key-value pair is telling re-frame to do a side-effect, which is uniquely identified by that key and requires the data in the corresponding value. But, how does re-frame know how to action on the :write-localstore effect?

We have to use re-frame's reg-fx function to associate the effect key, (:write-localstore), with an effect handler, (local-store/write function):

reg-fx receives two arguments: the key that identifies the effect and the function which actions the side-effect, the effect handler.

When an event handler returns an effects map which contains a given effect key, the effect handler associated with it using reg-fx will be called to action on the effect, passing it the value associated to the effect key in the effects map.

The local-store/write function is passed a map containing the key-value pairs it has to write to the local store.

Finally, this is how we'd test the event handler using it:

being check-writing-to-local-storage-contains and check-writing-to-local-storage:

And that's all we need to know in order to create a custom effect in re-frame.

"History is not a list of names and dates. It's a way of thinking that can be powerful and illuminating."

"Social interactions really do influence what scientist produce."

"How a community tells its history changes the way it thinks about itself."

"A historical perspective on science can help physicists understand what is going on when they practice their craft, and it provides numerous tools that are useful for physicists themselves."

"Physics is a social endeavor."

"Research is done by people. And people have likes and dislikes, egos and prejudices. Physicists, like everyone else get attached to their favorite ideas and hang on them perhaps long after they should let them go."

"The history of science can help dismantle the myth of the purely rational genius living outside the everyday world. It makes physics more human."

"And a more human physics is a good thing. For starters, it makes physics more accessible."

"A field in which people are aknowledged as people is much more appealing than one in which they are just calculating machines."

"Physics only work when people talk to each other and communication s not always easy."

"Everything seems obvious in retrospect."

"The history of physics can remind us how difficult is to justify ideas that now seem obvious."

"Complexity, not simplicity , has ruled the practice of science."

"Every discovery has come out of a messy mix of people, ideas, accidents and arguments."

"Students and young researchers are often heartened to learn that physics is hard work and that it is ok for their own efforts not to look like a text-boo presentation. Messiness is a standard. Mistakes are normal. The results of physics are not self-evident."

"The history of physics suggests that there are usually several ways to approach a problem."

"Turning complexity into good physics requires creativity. You can never tell what weird idea will help clarify a confusing observation or provide the key to interpreting an equation. History uncovers the strange stew of concepts that were necessary for the development of physics."

"The interplay of various approaches is what brought us a modern view."

"Strange but ultimately useful perspectives come from fields and disciplines apparently distant from the problem at hand."

"The history of science shows how important it is for scientists across different fields to talk to each other. Conversations among separate groups are healthy. Apparently isolated problems are often closedly tied together, and you never know where you will find the weird idea that solves your difficulty."

"The best strategy for encouraging diverse ideas is to cultivate a diverse community."

"Underrepresented groups that offer different ways of thingking are often the source of fresh insights and novel methods."

"Underrepresented groups are usually marginalized because of cultural inertia or deliberate decisions made long ago."

"The diversity of ideas and interpretations serves as a reminder that physics is a work in progress. Knowledge is provisional. There are always new ways to tackle a problem, and there is always more to be learned."

"Accepting uncertainty would require changes in how science is taught."

"Curiosity should be revered, and everyone should be encouraged to ask, what else?"

"If scientist are not explicit and honest about their doubts, a crisis of confidence arises when that uncertainty is revealed."

"Physics wasn't always as it is."

"The flip side of accepting physics will be different in the future is accepting that it was different in the past. Everyone has a tendency to assume that the way things are now is the norm. But history makes it clear that things were not always this way. An understanding of why people used to think differently is a powerful tool for understanding people today. By drawing attention to older, usnpoken assumptions, history shows us how to start paying attention to our own."

"A knowledge of the historic and physical background, gives the kind of independence from prejudices of his generation from which most scientists are suffering" Einstein

"..they should study the history of those ideas and understand the circumstances in which they were justified and found useful."

"History trains you to think critically about received ideas. History provides evidence of roads not taken."

"Science's plurality of interpretation can make the history of science a resource for modern scientific research." Hasok Chang

"Putting complementary science into practice demands difficult self-examination. Thinking deeply about assumptions and accepted knowledge can be hard to do in professional scientific contexts, but history is a mode in which it's encouraged."

"The simple realization that people used to think differently can be quite powerful."

"Physics doesn't have rigid rules."

"Scientist simply don't follow a rigid, linear problem-solving system. Sometimes they start with a hypothesis, sometimes with a strange observation, sometimes with a weird anomaly in an otherwise straightforward experiment."

"... a scientist must be an "unscrupulous opportunist", adopting and adapting various approaches as new challenges arise."

"History teaches that knowledge is not fixed."

"Engaging with history will teach you to understand ideas on their own terms."

"Historical thinking makes its subject dynamic. It helps you think about science as a series of questions rather than a series of statements. Those questions will continue into the future, and it is helpful to know what has been asked so far."
"In the end, history of science exposes scientists to new ways of thinking and forces them to reexamine what is already known. Such intellectual flexibility is essential for any discipline, but it's particularly important for fields as influential and authoritative as physics and other sciences."

However, as we said, to build a program that does anything useful, it's inevitable to have some side-effects and/or side-causes. So, there will be many cases in which event handlers won't be pure functions.

These impure event handlers are hard to test. In order to test them, we'll have to somehow spy the calls to the function that is doing the side-effect (the dispatch). Like in the case of side-causesfrom our previous post, there are many ways to do this in ClojureScript, (see Isolating external dependencies in Clojure), only that, in this case, the code required to test the impure handler will be a bit more complex, because we need to keep track of every call made to the side-effecting function.

In this example, we chose to make explicit the dependency that the event handler has on the side-effecting function, and inject it into the event handler which becomes a higher order function. Actually, we injected a wrapper of the side-effecting function in order to create an easier interface.

Notice how the event handlers, evolve and start-stop-evolution, now receive as its first parameter the function that does the side-effect, which are dispatch-later-fn and dispatch, respectively.

When the event handlers are registered with the events they handle, we partially apply them, in order to pass them
their corresponding side-effecting functions, dispatch-later for evolve and dispatch for start-stop-evolution:

These are the wrapping functions:

Now when we need to test the event handlers, we use partial application again to inject the function that does the side-effect, except that, in this case, the injected functions are test doubles, concretely spies which record the parameters used each time they are called:

This is very similar to what we had to do to test event handlers with side-causes in re-frame before having effectful event handlers (see previous post). However, the code for spies is a bit more complex than the one for stubs.

Using test doubles makes the event handler testable again, but it's still impure, so we have not only introduced more complexity to test it, but also, we have lost the two other advantages cited before: local reasoning and events replay-ability.

Whereas, in our previous post, we saw how coeffects can be used to track what your program requires from the world (side-causes), in this post, we'll focus on how effects can represent what your program does to the world (side-effects). Using effects, we'll be able to write effectful event handlers that keep being pure functions.

Let's see how the previous event handlers look when we use effects:

Notice how the event handlers are not side-effecting anymore. Instead, each of the event handlers returns a map of effects which contains several key-value pairs. Each of these key-value pairs declaratively describes an effect using data. re-frame will use that description to actually do the described effects. The resulting event handlers are pure functions which return descriptions of the side-effects required.

In this particular case, when the automaton is evolving, the evolve event handler is returning a map of effects which contains two effects represented as key/value pairs. The one with the :db key describes the effect of resetting the application state to a new value. The other one, with the :dispatch-later key describes the effect of dispatching the :evolve event after waiting 100 microseconds. On the other hand, when the automaton is not evolving, the returned effect describes that the application state will be reset to its current value.

Something similar happens with the start-stop-evolution event handler. It returns a map of effects also containing two effects. The one with the :db key describes the effect of resetting the application state to a new value, whereas the one with the :dispatch key describes the effect of immediately dispatching the :evolve event.

The effectful event handlers are pure functions that accept two arguments, being the first one a map of coeffects, and return, after doing some computation, an effects map which is a description of all the side-effects that need to be done by re-frame.

Notice how by using effects we don't need to use tests doubles anymore in order to test the event handlers. These event handlers are pure functions, so, besides easier testing, we get back the advantages of local reasoning and events replay-ability.

:dispatch and :dispatch-later are builtin re-frameeffect handlers already defined. It's possible to create your own effect handlers. We'll explain how and show an example in a future post.

Wednesday, November 2, 2016

First of all, the people in Codesai.
I met Carlos in 2011, because he was teaching the first TDD course I attended. Carlos is a humble and honest person that tries very hard to make sure his actions match what he believes. Later I've known him as a quiet, open-minded leader who is always willing to listen what other people have to say, and this permeates the whole Codesai team.
During the last two years, I had the opportunity to meet most of Codesai's apprentices. They are very enthusiastic and eager to learn. I had great fun participating with them in some Software Craftsmanship Gran Canaria events and talking about code and life in general in Codesai's apartment.

I share Codesai's values and feel very confortable with their culture and communication style because the team creates a healthy and safe environment which respects the diversity of its members and the debate of ideas.

I met the rest of the team last week during Codesai's summit (aka #flejesai16) and it was a confirmation that I'm in the right place.

Second, the opportunity to learn from experienced XP practicioners.
I've been developing software for 6 years now. I've learned a lot, most of it by reading books, practising on my own, attending courses and from colleagues, both from the community and at work. However, I felt the need of having a mentor, because I think I'll learn more and better than if I keep working on my own. I've found some great mentors in Codesai who acknowledge that I need a safe environment to get experience step by step.

Finally, the last reason is more personal.
I live in Barcelona but I'm from Canary Islands. Codesai is a remote-first team but the core of Codesai's team is currently in my hometown in the Canary Islands. So being a part of Codesai also gives me the chance of visiting my family more often.

Monday, October 24, 2016

Event handlers, collectively, provide the control logic in a re-frame application. Since the mutation of the application state is taken care of by re-frame,
these event handlers can be pure functions that given the current value of
the application state as first argument and the event (with its payload)
as second argument, provide a new value for the application state.

This is important because the fact that event handlers are pure functions brings great advantages:

Local reasoning, which decreases the cognitive load required to understand the code.

Easier testing, because pure functions are much easier to test.

Events replay-ability, you can imagine a re-frame application as a reduce (left fold) that proceeds step by step.
Following this mental model, at any point in time, the value of the application state
would be the result of performing a reduce over the entire collection of events dispatched
in the application up until that time, being the combining function for this reduce
the set of registered event handlers.

However, to build a program that does anything useful, it's inevitable to have some side-effects and/or side-causes. So, there would be cases in which event handlers won't be pure functions.

In this post, we'll focus on side-causes.

"Side-causes are data that a function, when called, needs but aren't in its argument list. They are hidden or implicit inputs."

Let's see an example:

The event handler for the :word-ready-to-check event, that we are registering using reg-event-db, is not pure because it's using the value returned by JavaScript's Date.now function instead of getting it through its arguments. To make matters worse, in this particular case, this side-cause also makes the event handler untestable because JavaScript's Date.now function returns a different value each time it's called.

To be able to test this event handler we'll have to somehow stub the function that produces the side-cause. In ClojureScript, there are many ways to do this (using with-redefs, with-bindings, a test doubles library like CircleCI's bond, injecting and stubbing the dependency, etc.).

In this example, we chose to make the dependency that the handler has on a function that returns the current timestamp explicit and inject it into the event handler which becomes a higher order function.

In the previous code example, notice that the event handler now receives as its first parameter a function that returns the current timestamp, time-fn.

And this would be how before registering that event handler with reg-event-db, we perform a partial application to inject JavaScript's Date.now function into it:

Using the same technique, we can now inject a stub of the time-fn function into the event handler in order to test it:

This is the code of the home-made factory to create stubs for time-fn that we used in the previous test:

We've seen how using a test double makes the event handler testable again, but at the price of introducing more complexity.

The bottom line problem is that the event handler is not a pure function. This makes us lose not only the easiness of testing, as we've seen, but also the rest of advantages cited before: local reasoning and events replay-ability. It would be great to have a way to keep event handlers pure, in the presence of side-effects and/or side-causes.

Since re-frame's 0.8.0 (2016.08.19) release, this problem has been solved by introducing the concept of effects and coeffects. Effects represent what your program does to the world (side-effects) while coeffects track what your program requires from the world (side-causes). Now we can write effectful event handlers that keep being pure functions.

Thanks to coeffects, we can eliminate the side-cause, passing the timestamp input through the argument list of the event handler. The resulting event handler is a pure function:

This event handler receives, as its first parameter, a map of coeffects, cofx, which contains two coeffects, represented as key/value pairs. One of them is the current timestamp which is associated to the :timestamp key, and the other one is the application state associated to the :db key. The second argument is, as in previous examples, the event with its payload.

The map of coeffects, cofx, is the complete set of inputs required by the event handler to perform its computation. Notice how the application state is just another coeffect.

This is the same event handler but using destructuring (which is how I usually write them):

How does this work? How does the coeffects map get passed to the event handler?

When you use reg-event-db to associate an event id with the function that handles it, its event handler, that event handler gets as its first argument, the application state, db.

While event handlers registered via reg-event-fx also get two arguments, the first argument is a map of coeffects, cofx, instead of the application state. The application state is still passed in the cofx map as a coeffect associated to the :db key, it's just another coeffect.
This is how the previous pure event handler gets registered:

Notice the second parameter passed to reg-event-fx. This is an optional parameter which is a vector of interceptors. Interceptors are functions that wrap event handlers implementing middleware by assembling functions, as data, in a collection. They "can look after cross-cutting concerns helping to "factor out commonality, hide complexity and introduce further steps into the 'Derived Data, Flowing' story promoted by re-frame".

In this example, we are passing an interceptor created using re-frame's inject-cofx function which returns an interceptor that will load a key/value pair (coeffect id/coeffect value) into the coeffects map just before the event handler is executed.

Second, we factor out the coeffect handler, and then register it using re-frame's reg-cofx. This function associates a coeffect id with the function that injects the corresponding key/value pair into the coeffects map. This function is known as the coeffect handler. For this example, we have:

Since the event handler is now a pure function, it becomes very easy to test it:

Notice how we don't need to use tests doubles anymore in order to test it. Thanks to the use of coeffects, the event handler is a pure function, and we can just pass any timestamp to it in its arguments.

More importantly, we've also regained the advantages of local reasoning and events replay-ability that comes with having pure event handlers.

In future posts, we'll see how we can do something similar using effects to keep events handlers pure in the presence of side-effects.

Using literals in your tests can have some advantages, such as, readability and traceability.

While this is true when the data are simple, it's less so when the data are nested, complex structures.
In that case, using literals can hinder refactoring and thus become an obstacle to adapting to changes.

The problem with using literals for complex, nested data is that the knowledge about how to build such data is spread all over the tests. There are many tests that know about the representation of the data.

In that scenario, nearly any change in the representation of those data will have a big impact on the tests code
because it will force us to change many tests.

This is an example of a test using literals, (from a ClojureScript application
using re-frame, I'm developing with Francesc),
to prepare the application state (usually called db in re-frame):

As you can see, this test knows to much about the structure of db.

There were many other tests doing something similar at some nesting level of the db.

To make things worse, at that moment, we were still learning a lot about the domain, so the structure of the db was suffering changes with every new thing we learned.

The situation was starting to be painful, since any refactoring provoke many changes in the tests, so we decided to fix it.

What we wanted was a way to place all the knowledge about
the representation of the db in just one place (i.e., remove duplication),
so that, in case we needed to change that representation, the impact of the change would be
absorbed by changing only one place.

A nice way of achieving this goal in object-oriented code, and at the same time making
your tests code more readable, is by using test data builders which use the builder pattern, but how can we do these builders in Clojure?

After creating builder functions for some other data used in the project, our test started to read better and to be robust against changes in the structure of db.

For instance, this is the code of the test I showed at the beginning of the post, but now using builder functions instead of literals:

which not only hides the representation of db but also eliminates details that are not used in particular tests, while still being as easy to read, if not easier, than the version using literals.

We have seen how, by composing builder functions and using them in our tests, we managed to reduce
the surface of the impact that changes in the representation of data might have on our tests.
Builder functions absorb the impact of those changes, and enable faster refactoring, and, by doing so, enable adapting to changes faster.

We managed to make a rough version but we didn't have time to test it and make it nice.

When I got home I redid the exercise from scratch using a mix of TDD and REPL Driven Development.

First, I coded a couple of evolution rules. These are their tests:

and this is their code:

Next, I coded the cellular automata evolution, these are its tests:

and the resulting code:

Once I had the evolution working, I started to work on the view, using Figwheel and a hard-coded list of cellular automaton states on the db. This way I focused only on rendering the list of cellular automaton states.

With that working, I played on the REPL to create a subscriber that reacted to changes on the cellular automaton states making the view render.

This is the resulting code of the view:

the code of the subscriber:

and the default db:

Then, I used TDD to write the event handlers.

The use of effects keeps re-frame handlers pure. They allow us to avoid making side effects. We just have to describe as data the computation that will be made instead of doing it. re-frame takes care of that part.

These are the handlers tests:

these are the handlers:

Notice, how in order to dispatch to another handler (a side effect) we just have to add a key-value pair to the map of effects returned by the handler. In this case, I used the :dispatch and :dispatch-later effects which are part of re-frame's built-in effects. You can also create and register your own effects (I will talk about it in some future post).

To see the whole picture this is the code that registers the handlers:

I think that effects and coeffects (I'll talk about them in a future post) are just great!

They make testing handlers logic very easy (since handlers keep being pure functions) and avoid having to use test doubles.

Finally, everything is put together in the core namespace:

You can see the cellular automaton evolving in this video (I used garden library to add some CSS to make it look nicer):

In this post I've tried to, somehow, describe the process I followed to write this code, but the previous gists just reflect the final version of the code. If you want to follow how the code evolved, have a look to all the commits on Github.

Wednesday, September 28, 2016

To finish with the series of exercises we've been doing lately around the Lights Out kata, I decided to use reagi library to raise the level of abstraction of the communications between the Lights and the LightsGateway components.

The main change here is that the ApiLightsGateway component keeps a lights-stream which it's initialized in its start function and disposed of in its stop function using reagi.

I also use, reagi's deliver function to feed the lights-stream with the response that is taken from the channel that cljs-http returns when we make a post request to the server.

Next,the lights name space:

Notice how the dependency on core.async disappears and the code to update the lights atom is now a subscription to the lights-stream (look inside the listen-to-lights-updates! function). This new code is much easier to read and is at a higher level of abstraction than the one using core.async in previous versions of the exercise.

Now the lights-view name space:

Here I also used reagi to create a stream of clicked-light-positions. Again the use of FRP makes the handling of clicks much simpler (in previous versions a callback was being used to do the same).

Another change to notice is that we made the view a component (LightsView component) in order to properly create and dispose of the clicked-light-positions stream.

Finally, this is the core name space where all the components are composed together:

This was a nice practice to learn more about doing FRP with reagi. I really like the separation of concerns and clarity that FRP brings with it.

You can find my code in these two GitHub repositories: the server and the client (see the master branch).

Write a failing test (using more complicated examples than the typical ones you use when doing only TDD).

Explore and triangulate on the REPL until I made the test pass with some ugly complete solution.

Refactor the code to make it more readable.

I've noticed that when I mix TDD and RDD, I end up doing bigger steps and keeping less tests than when doing only TDD. I think this is because most of the baby steps and "triangulation tests" are done on the REPL which gives you a much faster feedback cycle.

This way I wrote both encode and decode functions.

Once I had then working with a few examples, I added property-based tests using Clojure test.check library (a QuickCheck like library for Clojure) to automatically explore more examples by checking that the property of "decoding and encoded number returns the number" held for 1000 randomly generated examples.

I think that these three techniques are complementary and its combination makes me more productive.

To document the development process I commit after each TDD green step and after each refactoring. I also committed the REPL history.

April
- Implementation Patterns, Kent Beck (3rd time)
- The Power of Habit: Why We Do What We Do in Life and Business, Charles Duhigg
- How to Win Friends and Influence People, Dale Carnegie
- Understanding the Four Rules of Simple Design, Corey Haines (2nd time)
- La llamada de Cthulhu (The Call of Cthulhu), El horror de Dunwich (The Dunwich Horror) and Las ratas en las paredes (The Rats in the Walls), H. P. Lovecraft
- The Tales of Beedle the Bard, J. K. Rowling

This time instead of doing everything on the client,
we used the Compojure library to develop a back end service that reset and flipped the lights,
and a simple client that talked to it using cljs-http and core.async libraries.

And finally the lights namespace which is in charge of talking to the back end and updating the lightsatom:

Notice how we used the pipe function to take the values that were coming from the channel returned by the call to cljs-httphttps://github.com/r0man/cljs-http's post function and pass them to the channel from which the code that updates the lights state is taking values.