Vukoje's blog about software development

A decade ago I had many, many questions and doubts about programming. How does programming look in real life on big projects? Is it too hard? Will I be any good at it? Over the last few months I have been mentoring talented junior programmers in my company from whom I’ve come to have great expectations. This is what motivated me to write this post and share it with young talents with whom I will hopefully work one day soon.

Learn, learn, learn

You will learn some things in the university, but they are not enough to make you excellent. You should keep learning new things all the time because technology is complex and constantly changing. For every subject you are interested in there is a fabulous book out there full of knowledge.

Once a friend told me “If I read 10 pages of an IT book every day, imagine how many books I will have read in few years, and what I will know”. Well, of course, I didn’t keep up with that pace, but I pressured myself to always read a lot. I keep a public list of things I read as a public CV and a motivator to keep going.

Technical blogs are becoming a main source of information and more and more often people publish books by refining content already published on their blogs. So pick a favorite RSS reader and start building you feed collection that will keep you up to date. For a start, here are some of my favorites:

You become an excellent programmer in you free time

Formal education is a good start, but it’s not enough. Work experience is mandatory, but you only advance in practical stuff you solve at your desk. It takes time and passion to go deeper into problems and get your eureka moments. Don’t expect that others, teachers or colleagues, will teach you how to code.

This doesn’t mean that you will have no free time. It only means that programing will be one of your hobbies and that you will annoy your non-programming friends with your technical discussions.

Learn how to write code

If you are going to be a programmer, you will be staring at code editor and writing code most of your work day. You should write code like a pro. Some programmers tend to think that architecture and fancy patterns are important and that code is just something in between, but I would strongly disagree. I’ve never seen app with great architecture and crappy code.

Not having enough time is a common excuse for writing bad code. Avoid excuses. Once you master the skill of clean code it will come naturally to you, at no cost.

So how do you write code like a pro? Well read a book of course! I read Refactoring by Martin Fowler but I’ve heard that Clean Code and Code Complete are also great. Actually I remember an older guy at the university who told me to read Refactoring and that it will make me twice the programmer I was. Till this day I believe this to be truth.

There are also code conventions and code analysis tools for specific platforms. They will come in handy. Why waste time and energy on trivial decisions when you can follow guides.

Is it hard?

Well, it can be, when not done the right way. You could end up with an application that is very hard to maintain because of the huge complexity and a lack of tests or documentation. On the other hand, when development is done properly, programming seems more like a fun game than work. As you get better you will build better apps and your job will be easier.

Sometime you end up with a problem you just cannot solve. In that case you either solve it and feel like you’re on top of the world, or somebody else does and you learn from him or her.

Do the hard stuff

Some guys constantly run away from tough tasks. My advice is to do just the opposite, ask for the toughest. Tough problems will make you a tough programmer and speed up your learning. Also, if you go for the hard things you will never be bored at work.

It’s ok to be scared

So you are starting to feel comfortable as a programmer and you are successfully completing your tasks. It feels good. But then one day a new task comes along. A new task requires new technologies, tools and domain knowledge you don’t have. You get scared. You doubt if you will you be able to complete the task. It’s OK to be scared. The only difference between you and a more experienced guy is that he has been scared many times before and probably successfully completed such tasks and now he believes in himself. Everybody feels like a phony at some point.

It isn’t only about code

Writing code is not all there is to being a programmer. Your job is to create successful software. In order to do that you will be testing, writing specs, talking to customers, organizing tasks, installing machines, motivating and mentoring coworkers… Even carrying equipment and much, much more… Don’t underestimate value of pure business domain knowledge. It can be much more valuable than programming skill.

There are always guys who insist that their job is only to write code and not know anything about the business domain or anything else, but they are usually jerks.

Don’t be a jerk

Because nobody wants to work with one. If you behave like one you will be alone in solving problems although it much easier to do it together. Be a humble programmer.

There are only two hard things in Computer Science: cache invalidation and naming things. - Phil Karlton

Yes, naming is hard but it is also very important. We tend to think that big complex architecture is the software design and that type and member naming along the way is just something that is hard and that we screw up. My point here is that naming is the software design and that on daily basis to me it is more important than architecture.

If naming is good:

it will be easier to understand the code and get the big picture

API will be simpler to user and there will be less bugs

there will be a greater chance that code for some new feature goes to the right place and that software will evolve gracefully instead of becoming a big ball of mud

In book Domain Driven Design, Eric Evans describes the term Ubiquitous language for the practice of building up a common, rigorous language between developer and users, and we can think of this as naming on steroids. Also Obfuscation can demonstrate the importance of naming because obfuscated code has worst possible names so that is almost not human readable.

Naming and Cohesion

As applied to object-oriented programming, if the methods that serve the given class tend to be similar in many aspects, then the class is said to have high cohesion. In a highly cohesive system, code readability and the likelihood of reuse is increased, while complexity is kept manageable. - Cohesion

IMO types that have high cohesion are easy to name. Low cohesion types are hard for naming because usually they are schizophrenic and do many (unrelated things). So we can think of a good name as a necessity that will force good design decisions.

Code example

I want to give an example of bad naming that irritated me enough to write this post. Here are some JavaScript DOM Event methods.

cancelable - Returns whether or not an event can have its default action prevented

preventDefault() - To cancel the event if it is cancelable, meaning that any default action normally taken by the implementation as a result of the event will not occur

defaultPrevented - Returns a boolean indicating whether or not event.preventDefault() was called on the event.

Last year I gave a talk State of the art logging where I discussed logging, logging libraries and tools for logs analysis. I wanted to go a step back and share with you my notes on designing a logging library. I actually written down what was important for me and later decided to go with custom logger implementation that wraps NLog.

When I talk about logging I always tell a difference between tracing and event logging. By tracing I mean logs with high detail data useful for debugging app. Trace is available only in “tracing mode” – trace data is not collected by default. By event logging I mean data that application outputs to some persistent store and which are used for monitoring application behavior (e.g. crash reports).

Tracing

Tracing must have high performance:

It should have no performance impact if not turned on

String formatting should not be done if tracing is turned on (Tracing client code never formats string because formatting consumes time and it might not be needed)

It should have very small memory consumption when tracing is turned off.

When turned on it must not endanger app pool execution (e.g. production tracing must not cause out of memory exception).

Tracing should be enabled per session in web applications (multitenant or not) but should also work in cases where session doesn’t exist (tracing win services, tracing web applications before session initialization).

Tracing should support priority levels and labels (categories).

Tracing should support “Start – End” style logs with usingstatement. This will make tracing API better (smaller and more readable code) and make performance monitoring easier (time of method execution). It should also enable hierarchy displaying of trace data (similar to Call stacks in debugging).

If Tracing component uses external components this must be hidden (our trace will wrap them) because otherwise it would introduce high coupling to that external component and make any changes very hard.

Trace should support logging to file. File system organization for web multitenant applications should be like this: Root folder/tenant name/user name/date/request time stamp.log Programmer should be able to configure simpler folder structure in simpler cases (Windows service that doesn’t have tenant or user). Root folder should be configurable. If session is not yet initialized or is lost, files should be logged to Root folder/tenant name/.

In web applications it should be possible to trace each web request in separate trace file. In other cases programmer should be able to decide when to split file (per day, month, never…).

Trace should have buffer of last N relevant trace calls (probably only logs with highest priority). N constant should be possible to configure by programmers. Beside this buffer of last N relevant logs we might keep full trace of last HTTP request. This can be useful when tracing is not turned on and application breaks. In this case we might still want full trace form request that broke the app. This solution might compromise memory so we must be careful with it.

Trace files older by some period should be deleted in some time intervals.

Tracing must not break in production! Breaking is only ok in debug mode. E.g. if programmer supplies trace with invalid parameters, Trace should break only in debug mode.

Event logging

Event log should contain information about events in application that are of interest to people monitoring application health and configuration status.

It started happening while I was still at college. We students have informally divided into two groups, us that were constantly digging for more knowledge and them that were telling us that we will just be stupid programmers. We wanted to be great at software development, but they said that we will be zombies staring at out monitors for the rest of our lives. They had different future in mind for them, where they would be very smart without any effort, work little and be very good paid for that. My favorite quote from them was "I won't be a programmer, I will be a consultant to other programmers" ... straight from collage. Imagine that!Or "I will stay at the collage and teach others on the advanced software development techniques"... techniques they have never experienced.

Think of some person you admire for what they have achieved in their profession. Do you think they earned your respect by being talented or with their passion and hard work? Do you think they were stupid to push so hard?

Albert Einstein was committed to his work but I never heard anybody says that he was staring at formulas for the whole day. Nobody complained that he was always lost in his taught or that his clothes were always wrinkled. And what about Jimmy Hendrix, when you think of him, do you think he was a freak locked up in a room all day long with his guitar? What about Michael Jordan, can you even imagine his commitment to basketball? Do you know he was rated as untalented player when he was in high school?

The reality check

The key point that I am trying to make here is that you can't be great at anything if you don't commit to it. There is a myth of talent and intelligence, saying that success only depends on it and that you either have it like gift from the God or not at all. From my personal experience and people around me I believe that talent and intelligence have nothing to do with success. Fortunately, research results confirm my thoughts:

"Research now shows that the lack of natural talent is irrelevant to great success. The secret? Painful and demanding practice and hard work. ... The critical reality is that we are not hostage to some naturally granted level of talent. We can make ourselves what we will. Strangely, that idea is not popular. People hate abandoning the notion that they would coast to fame and riches if they found their talent. But that view is tragically constraining, because when they hit life's inevitable bumps in the road, they conclude that they just aren't gifted and give up."

And for all of you dreaming of building a startup and getting rich, you should be prepared for hard work because according to Paul Graham, determination is the key factor for startup success.

"We learned quickly that the most important predictor of success is determination. At first we thought it might be intelligence. Everyone likes to believe that's what makes startups succeed. It makes a better story that a company won because its founders were so smart. The PR people and reporters who spread such stories probably believe them themselves. But while it certainly helps to be smart, it's not the deciding factor. There are plenty of people as smart as Bill Gates who achieve nothing.In most domains, talent is overrated compared to determination—partly because it makes a better story, partly because it gives onlookers an excuse for being lazy, and partly because after a while determination starts to look like talent."

"Deliberate practice is activity designed specifically to improve performance, often with a teacher’s help; it can be repeated a lot; feedback on results is continuously available; it’s highly demanding mentally, whether the activity is purely intellectual, such as chess or business-related activities, or heavily physical, such as sports; and it isn’t much fun.….most of us, as adults, are just doing what we’ve done before and hoping to maintain the level of performance that we probably reached long ago. Deliberate practice requires that one identify certain sharply defined elements of performance that need to be improved, and then work intently on them."

If you don't believe it, see Deliberate Practice in Software Development, excellent presentation that takes scientific data to show that in any work (sport, art, engineering…) there is no substitute for hard work and practice.

Einstein advices

So to get back to Einstein, he left us with his 10 universal principles that can be applied to any work where someone wants to achieve excellence. So here are some of the things one of the greatest scientists of all time had to say:

On Determination

“I have no special talent. I am only passionately curious.”“It's not that I'm so smart; it's just that I stay with problems longer.”

On Value

“Strive not to be a success, but rather to be of value."

On Experience

“Information is not knowledge. The only source of knowledge is experience.” “A person who never made a mistake never tried anything new.”

On Self-improvement

“Insanity: doing the same thing over and over again and expecting different results.” “You have to learn the rules of the game. And then you have to play better than anyone else.”

Conclusion

So the question is do you want to be excellent at your work and what are you doing about it? Are you better than others? Do you have skills that will bring value to your employers/clients? What are you currently trying to learn/improve?

Few days ago I saw same ordinary scene on street while going to work that got me thinking... about code off course :) Car driver was complaining to parking service employee about a parking ticket he has just received. He said that he was late for only 15 minutes (parking ticked is valid for 2 hours and drivers are notified with SMS 15 minutes before expiration). At the first moment I taught that 15 minutes isn't that much of a big deal, but then I started thinking what about 20 minutes? Still OK? And 30 minutes? Too much?

The real question is where to draw the line? Because we obviously can't do this, the right answer is that there should be no tolerance. OK maybe 60 seconds :)

You said something about code?

Unfortunate driver reminded me about a question I often encounter on my work. Is all code important? Should really all code follow best coding practices and coding standard?

Same day I heard that driver complaining, my collage and I asked ourselves if small repeated blocks of code in Unit Test Mock classes are really important?

If we assume that all code is not important (which will usually happened) you will be constantly wondering where to draw the line between important and not so important code? The answer is same as for parking time overdue, the line can't be drawn. You will end up with universal excuse for having sloppy code, although you have adopted best practices and maybe you even created a full coding standard.

But could we have saved time if we could draw the line?

I think not. If some code is less important, rarely used, and doing trivial staff, than it's usually easy to implement and hard to get it wrong. This means that code probably won't need any refactoring to enhance its design. I said at my previoustwo presentations, and I will say it again, I can write code by coding standard in same time as if I didn't code by standard. In fact I can maybe write good code even faster than bad code because I'm already constrained by the standard so I don't have to wonder or think about lowest level solutions (Event design, choosing between Class and Struct etc.).

I hope that this statement provokes you to show that I am no better than you because I know that everyone can write good code with just a little focus and discipline.

As programmers fight every day with deadline pressures to deliver meaningful software to customers on time, there is one code principle that can speed up and simplify their effort. This principle is called YAGNI, which is short for "You Aren't Going To Need It".

YAGNI is the principle in extreme programming stating that programmers should not add functionality until it is necessary, meaning that you should always implement things when you actually need them, and never when you just foresee that you need them.

The reason for enforcing this principle is that anticipation in software is very hard and usually leads codeoff track, while also spending valuable resource - time. Anticipation leads to half baked implementation that you maybe didn't need in the first place, and that implementation can turn out to be constraint on implementing some vital functionality that you really need. So what you have is lose-lose situation, you wasted your resources and made situation harder for yourself.

Often you will be building some class and you’ll hear yourself saying "We’re going to need...".Resist that impulse, every time. Always implement things when you actually need them, never when you just foresee that you need them.

The best way to implement code quickly is to implement less of it. The best way to have fewer bugs is to implement less code.

You’re not gonna need it!

The art of delaying

Any problem can be solved if you can delay it long enough, but of course you can't always delay things.

You must remember those situations where Microsoft or some other vendor solved all your problems shortly after you implemented your custom solution. These are really painful moments because usually vendor does it much better than you did and it usually comes free of charge.

Or maybe you remember those painful to implement tasks that altered your core architecture before you found out that they weren't really that important for business and that they will probably be thrown away?

Hart of the justification of YAGNI is that many of these potential needs end up not being needed, or at least not in the way you'd expect. By not doing them until they are really needed you'll save a good deal of effort.

YAGNI and The Big Design

As developers, I think we also tend to be far too optimistic in assessing the generality of our own solutions, and thus we end up building elaborate OOP frameworks around things that may not justify that level of complexity. To combat this urge, I suggest following the YAGNI doctrine. Build what you need as you need it, aggressively refactoring as you go along; don't spend a lot of time planning for grandiose, unknown future scenarios. Good software can evolve into what it will ultimately become.

To make design of your whole big project before you start coding it you need to know/assume most of the requirements. This is the only way you can plan application design that will last for next few years. Reason why YAGNI won't let you create big upfront designs is because they are almost always wrong. They are wrong simply because it is impossible to foresee all user requirements and think of all possible variations of Use Cases and potential design problems. Even if you could do all of this, requirements will change before you finish your project. What you should do instead is start from some basic design that will evolve to finally product. Basically you should design a minimum that works, without over engineered grand designs of feature.

What about prototypes?

YAGNI is in direct conflict with concept of software prototyping, but the difference is that when you build a prototype you know that you are doing it because you don't know the real requirements (you know that you don't know). This is also why you should throw away your prototypes, or at least evolve them to real code.

In every agile development early customer feedback is very important. Today I will talk about even earlier feedback… I mean feedback from your team members. This is earliest feedback you can get and maybe most important, because often this is only technical feedback you can have. If you are building some API or planning some architecture, than at that point other programmers are your customers as your API users.

The question is why is this feedback needed any way? Thes because variations, combination and potential code usage scenarios are uncountable. Ok, maybe they can be counted, but rest assure that when the time comes, you will not have them all in you head. One more thing we want to avoid with this is over-engineering.

So tell your ides to your teammates and ask them for opinion. If you do this, you will often hear the phrase "The problem with that is...". And there it is, one future gotcha you have just avoided.

In the past I was annoyed with this feedback that was constantly proving my "super" ideas wrong, but today I cherish them as a proven strength of positive critics. Always give, cherish and encourage positive critics in your team.

Another way to look at this early feedback is as early WTF?!. Instead of living some other developer thinking many WTF!? while trying to understand your code, you cam tell your ides and wait to here WTF!?.

Sanity Checks

Programmers, as pure technicians, tend to lose focus to business value and end goal of software they are building. Once they dive deep in to code they can forget customer needs, deadlines, weak plans, planned designs… heck we will forget our friends and families. You can say that, from business perspective, developer have gone mad and this is why regular sanity checks are very important.

This is exactly why in Scrum you need Scrum Master whose main role will be “sanity checker” (team conscience) who will be checkingdeveloper sanity on daily scrum meetings. On these meetings every developer answers 3 questions:

What have he done from last meeting?

What will he be doing until next meeting?

Does he have any problems with his work?

After everyone has answered these 3 questions, whole team should be informed about each other’s work but also it should be obvious if someone have gone insane. For example, developer could answer that he has not been doing what was planned and has started to build his own parsing tool in some experimental technology that will be supper cool.

After this madness is discovered, everything is easy. Scrum Master asks simple question: “Why?” and keeps repeating it until developer realizes his mistake. This is known as 5 Whys technique.

Do your team sanity checks and check yourself daily. Focused team, building minimum that customer required can do amazing things, so go and be amazing.

In my previous blog post, I have stated that communication is very important for programmers, and maybe essential for successful teams. Today I want to clarify this subject by defining jet another guideline: Two heads are smarter than one. What this means is that two people are more likely to solve some problem faster and better.

Let’s put this in context of real programmer’s working day: you are trying to solve some problem, you cannot find good enough solution or you cannot decide which alternative solution to choose or how to organize your code, you aren’t sure will underlying technology cause problems…

What can you do about it?

You can do your best to focus, you can “google” solution all day long, you can pool out your hair trying to fit all that information in your head at once… or you can just ask for help person sitting next to you.

Often it is enough to try to explain some problem to another person and to see for yourself where you have gone wrong. If this doesn’t do the trick, your colleague already has advantage over you in solving the problem because he is approaching it from a different angle. It can be that colleague is more expirience with that kind of probelm, or that he is the developer who wrote original peace of code. Maybe he has already solved that problem, maybe he read something about it on the Internet, or maybe he knows that someone else in firms knows solution. It may also be that he is smarter than you, which again is great thing because what’s important is that problem is solved with little effort and that you learned something.

If all of these positive sides of communication didn’t solve the problem in first 2 minutes, you and the colleague are left to use power of brain storming, to inspect solution variations and alternatives. The catch with programming is that it is very complex and that if some problem is not solved 100% than it isn’t solved at all. You can’t say to your customer, “I can obtain all data you need but I don’t know how to show them to you, can you pay me half?” Sometimes you solve 95% of the problem for few minutes, and can be left struggling to solve rest 5% for days. Maybe your colleague has 50% of the solution, but it includes your missing 5%.

Other great thing about this rule is that motivates people to share problems. Beside the advantage of collective problem solving, it is great means for informing others about project state and sharing knowledge patterns in form problem -> solution. This way, some programmer is less likely to get stack solving problem that someone have already solved.

Few years ago my friend Aleksandar Mirilovic gave me an advice "No mater how small and easy project is, never work alone". At first I wasn't happy with idea but today it is an apsolute truth for me. What happend meanwile is that I have been working on mantaining a large project for two years. During that time I learned that some times you colleague can solve the problem you are working on in few minutes and that without his help you would be working on a problem for whole day.

So if it is that simple, why don’t we use it then?

Maybe it is because of our ego, because we like challenges or because we are afraid we will look stupid. The real chalange is to finish you task in simplet and fastest form using all available resources, and colleague sitting nexto to you is friendliest and most user friendly resource you will ever have. Use your colleague and don’t worry, he will share his adventures with you soon enough. Programming is hard enough, don’t make it harder on your self, use all the help you can get.

When you want to do something, next question always comes
up “What should we do next?” The answer will differ if you are making
decision what module to develop in next few months, or you are wondering what
functionality to code next.

Choosing next module

When you pick a development task for a longer period, always
pick the hardest one. This approach is also called “Swallow the biggest frog first”. This means that if you
will be developing some software in period of one year, in first quarter you
should tackle all biggest risks and most difficult implementations. These hard
problems must be solved before project is completed and if you delay them:

they
will only become harder because
already implemented components introduced higher complexity

you
will not have sufficient time to
solve them because you already burned big amount of time solving less important
issues

it is
more likely that importantfeedback will come too late when too
much time and money is spent

This approach will lead to early pain in
project, where you have large problems right on project start. You should
cherish early pain because alternative is slow and painful project death.

Choosing next functionality

When you are choosing next feature the code, choose the
smallest/simplest one that will work. If you choose simplest possible task,
you will be able to:

implement,
test and check in your code easily
and quickly

you will reduce chances for making a mistakebecause
work is small and simple

you will
be less stressed because
there will be less things that you have to hold in your head

Often when something is being built, the question comes up
“How far should we go with this?” If you are building application data access
layer, the questions can be:

How
much database vendors should we support?

How
many data types should we support?

Should
caching be integrated automatically?

Should
code support mapping to domain objects?

Should
this mapping support multiple domain objects view of the same system and
perform transparent memory synchronization of them?

What
about concurrency control?

You could go with all or nothing solution and spend few
years building data access layer. This approach is also called gold plating, and is always wrong approach because
customers won’t wait 10 years for you to achieve perfection in data access. The
right answer is to do a minimum that works. In previous example you probably
won’t need support for multiple types of database nor advance domain object
mapper.

If you do a minimum that works, you have:

Satisfied
all requirements

Spent
minimum amount of time and money

You
have minimal amount of code/documentation to maintain

Because
you finished faster, feedback will also come faster

If you
were doing a wrong thing, easier it will be to get back on track

Project
leader can do better time estimation if you just build what he requested
from you

You
will have more time to work on other things that need more attention

This is true for
code, as it is true for documentation. If you try to gold plate your requirements you will end up with such amount of documentation that no one will want to read it.
Also to much trivial information can hide important information. It is
situation when you can’t see forest from trees. Look at this
excellent experiment that shows that you can do more with fewer
requirements (Thanks to Vojin for link).

It is important to stress out that doing minimum that works doesn't imply minimum quality. I can only think of three situations where low code quality is acceptable:

solution is to small for quality to matters (e.g. project shorter than a month).

solution is for onetime use and it will be thrown away after (e.g. application for data migration)

solution must be finished in unrealistic short period (e.g. to satisfy important client's urgent need)