Daily Software Development

One of the agile communities mocking terms of the old-fashioned Waterfall development technique is, “Waterfail”. It is called this, because the community credits this technique with being part of why a lot of projects fail.

The main difference between this technique and most agile techniques, is that waterfall does one big flow of everything. It does not loop through the process as most agile techniques do. It attempts to have everything planned out so that you’re just going through the motions toward success.

I’ve been posting each of the topics from the NimblePros software craftsmanship calendar as we get to the month. With each of these, I am mentioning that you should work on trying to follow the good practice or avoid the anti-pattern. Since this year’s calendar is on anti-patterns, it included Waterfail as something to avoid.

So for the month of August, I am going to recommend trying to move away from Waterfall. I’m not saying that you should suddenly move from a Waterfall project to some form of an Agile project. At least start looking into the possibility. Read a book on agile, go to an agile or software craftsmanship user group, access online resources for learning agile techniques, or attend a conference with an agile track.

I really liked how well this image turned out, and if you didn’t notice, we wrote “Waterfall”, but we cut part of the first “L” to make it appear to say “Waterfail”. Just one of our subtle little tricks in the calendar.

Almost three years ago, Steve Smith, Rich Henning, and I founded the Hudson Software Craftsmanship Group (HudsonSC) in Hudson Ohio. We scheduled our first meeting for August 19, 2009 and we had plenty of people show up. Since that time, similar groups have sprung up in NE Ohio and other areas around the country. It’s really neat to watch as the software craftsmanship community grows. There are other groups, dojos, workshops, etc. all over the place.

Our group has been a model of other groups around the area as well. Brian Friesen attended one of our meetings as a bit of research before starting the Knoxville Software Craftsmanship Group. Brian Friesen is quite dedicated, because the drive up to HudsonSC is over 8 hours. What that means is, if you’re in that area and want to attend a meeting with a dedicated software craftsman, check their website regularly they’ve always got their meetings posted.

Our number of attendees fluctuates between highs of 20-30 people down to lows of 5-10 people. We’re usually somewhere between 10 and 20 attendees. These numbers are great for a group like ours. If you get much larger than this, it becomes nearly impossible to have a discussion.

How Software Craftsmanship Groups Stand Out

We’re not eyes-front, pay attention to a speaker kinds of groups. If that’s what you’re looking for, HudsonSC is not for you. There are plenty of groups that meet every month across the world that have this format. We focus on discussions and self-organization. Our members suggest topics they want to discuss. We try to get everyone contributing instead of just taking notes on someone else’s “wisdom”. Everyone in our group has something to bring to the table and adds value to the community as a whole. You can take notes after the meeting.

HudsonSC’s Agenda:

Introduction

Lightning Talks, Show and Tell, and Opening Discussions

Open Spaces and/or Group Discussions

Exercises (usually programming)

Our members bring laptops to the meetings so that we can write some code and pair with other people. Our group is not buried in their laptops nor is it tweeting away. When you attend a software craftsmanship group, you participate in the event. You are the speaker!

Third Anniversary

As I mentioned at the beginning of this post, it’s been nearly 3 years since we founded the group. Our third anniversary meeting has been scheduling and you can sign up for it now. The event is going to be held on August 15, 2012.

To celebrate this event, we’re going to have a spectacular evening or talking, discussions, and programming exercises.

I look forward to seeing you at our next HudsonSC. If you’re not in the area, go check out your nearest software craftsmanship group!

Programmers are talented, smart, skilled professionals. We put in lots of our own effort to educate ourselves and stay up on current technologies. We work hard and we feel great satisfaction in our achievements. We donate our time to charities when the opportunity arises. We give back to our local communities. All things considered, there are a lot of really great programmers in the world.

I lead a team of software developers. Does that mean I am the “best” programmer on the team? Certainly not. It just means that I lead the team. As the leader of the team, it is my job to: inspire, encourage, trail blaze, and motivate my team to be the best they can be. So who is the best programmer on my team? I don’t know, and I don’t care. There shouldn’t really be a “best” programmer on the team. Everyone on the team is great and working hard.

A team that really meshes and works well together doesn’t let egos get in the way. It doesn’t matter if you’re a rock star developer. When you’re on the team, you’re part of the team. You might be the JavaScript expert, the master of SQL queries, or the guy who can refactor anything. When you’re part of the team, you should try to use your strength, but don’t let that interfere with the team’s flow.

Sometimes it makes sense for a more experience developer to work with a less experienced developer. A common first instinct is that the senior person is mentoring the junior person, but it’s actually going both ways.

Each of the two developers is bringing a lot to the table. Their different experiences and views allow them to each approach the problems differently, ask different questions, and apply different, existing knowledge.

If either person’s ego gets in the way, it would prevent the 2-way knowledge transfer in addition to creating friction between them.

One Developer and one Junior Developer can pair on a task and share knowledge. Does one act like the other one is just along for the ride? No.

Your pair partner, is watching your back, guiding the team, and keeping the pair honest. Which one does that job? Both. If you had one person drive the whole time, you’re not going to see the benefits of pair programming.

If you let your ego get in the way, you might not let the junior developer write any of the code. They may not know every library, every design pattern, or even the intricacies of the language they’re using, but those guys can write some great code.

I don’t know how the Junior Developers are on your team, but ours write great code and show a high level of professionalism in their work.

And sometimes you get a Junior Developer pairing with the Developer Intern. How the pairs split up doesn’t really matter. Who is working with whom doesn’t really matter. Each and every member of the team is bringing something to the table, and it is this interaction that is making the team as effective as it is. These two are as productive a team as any other pairing that we’ve got.

Someone is probably reading this and thinking I am crazy for letting a Junior Developer and a Developer Intern pair together. They’re good though, and they really get great stuff done.

We can accomplish a lot of stuff when our team works individually, however, we can accomplish a lot more as a group. You might be the “best developer” on your team, but if you keep thinking and acting that way, you’ll be missing out on a lot. I learn a lot when I pair with any member of my team. It doesn’t matter how much experience the person has. Everyone knows something I don’t, and no one approaches the problems exactly the same way I would.

There is no “Ego” in “Agile Team”, so don’t bring yours to an agile team and expect good results.

When writing software, we often have to write comments to explain why we did something. I try to write my code such that I don’t have to comment it. In fact, anything other than a temporary (less than a few minutes) comment I write, I consider to mean that I made a mistake.

We could get into endless discussions of when we should comment, what we should comment, and how much we should comment. I’ve had plenty of discussions about comments with people with various opinions on the subject.

If you write “TODO”, it tells the reader that the code is incomplete and that wasn’t intentionally left the way it is. There just wasn’t enough time. If you write “HACK”, it admits that you wrote the code badly and it needs to be rewritten.

These types of comments fit in very well with what I tell people about comments. Each one I write and leave in the codebase means that I didn’t write the code as well as I wanted to. Each one is my mistake. The more comments I write, the worse the code is.

The Bad

One codebase that I was working on had a lot of these comments. It seemed they were all over the codebase. There was the other problem that copy-paste coding was rampant, so I am sure that many of these were copied and pasted from previous locations. Heck some of the even said to remove the duplication. The irony? Those comments about removing the duplication were copied with each new instance of the code that was copied.

I checked Visual Studio’s task list of these comments just to see how bad it was.

1131. 1131 tasks that were indicated with “TODO” and “HACK” comments. That takes more than just writing a lot of these comments. That requires writing them and never going back to fix them or complete them.

The Better

Check your codebase right now. Do you have any of these comments lying around? Is it only a handful? If so, you should go try to clean them up. That’s easy and will help you out down the road.

Anytime that you’re going to write a “TODO” comment, think about why you’re putting it there? Is it something that’s really needed? If so, make yourself a note somewhere else. Add a story to go back and make the change later. Get it in the system where you really track your work.

If you know that you’re not going to get to it any time soon, don’t bother with the comment. It will only clutter up your system. If it’s not important enough for you to come back and get it soon, why are you even recording it? If it’s important later, you will be working in that part of the codebase again and you can go back later and fix the code as part of your other work.

Certainly once you get that many of the comments, you should have long earlier realized that there is something that you need to work on. That’s just a ridiculous number of TODO and HACK comments for one codebase.

Has anyone ever see more than that? Anyone think I am crazy or wrong? Tell me about it in the comments!

More Code Audit Nuggets

Keep watching for more interesting nuggets of stuff that I’ve seen in codebases.

In software development, it is common to have users authenticate using usernames and passwords. In order to handle the authentication process, we must store this information in some way. Obvious, passwords should never be stored in clear text. Toward that end, when we store them, we encrypt, hash, salt, etc. to keep them safe.

While these steps cannot protect against all forms of attack, they’re a good base of protection to have in place. We don’t want to just give this information away to anyone who wants it. Yes, some people could take the non-clear-text passwords and obtain the original password from it, but we’ve certainly set the barrier higher.

Security is all about setting more and larger hurdles in place for people to jump over. The only secure information is information that doesn’t exist, and that becomes inherently insecure as soon as it does exist. If information is accessible to someone, it means that it’s possible for someone else to access that information also.

We just try to make things more difficult and limit that damage. Or at least some of us do.

I’ve made plenty of my own mistakes in the past. Some of the applications I wrote in college have security holes, which seem glaring to me now, but I’d not even noticed at the time I wrote the code. I’m sure that most people reading this would say the same thing about their own code.

The Bad

One of the systems I had the pleasure of seeing the code for, at its core, was a customer portal. It even had its own custom ecommerce solution. (These details while talking about security issues will have some of you facepalming already.)

The developers knew that they needed to encrypt the passwords, so they wrote some sql, dbo.encrypt, to encrypt passwords before storing them in the database. Insert a password and out comes a jumbled mess that no one could read.

Users will often forget passwords, and will need some way of recovering from this situation. I recommend resetting passwords when this happens. This application took a different approach.

Also stored in the database was some other useful sql, dbo.decrypt, which would take any of the encrypted passwords and decrypt them. Well that’s obviously a security issue, but the worst thing about this isn’t just that it’s reversible. It’s that this is also in the database. That means that we have the lock and the key together. If someone gets read-only database access, but not access to the application’s source code, they still have everything.

It’s like have a lock and keeping the key inside the lock for convenience. I wouldn’t want to have to call a locksmith if I ever lost the key, so I just keep it here. It makes opening the lock faster too!

The Better

Don’t allow reversing of stored passwords. Use 1-way only when dealing with passwords. This way brute force techniques are required to figure out a password.

If the system can reverse the password back to the original, anyone who has obtained access to the system can retrieve all of the passwords.

This isn’t a new idea. You’re not the first developer to set up authentication in an application. There are plenty of existing authentication systems out there. Don’t reinvent the wheel if you don’t have to. Some systems are safer than others. Ask around. Talk with security experts (I am not one of them) about these if your application requires really high hurdles.

More Code Audit Nuggets

Keep watching for more interesting nuggets of stuff that I’ve seen in codebases.

When and how much time to spend refactoring out code is one of the best questions that a budding software craftsman will ask. It’s one I’ve asked and answered many times. There isn’t one specific answer that is better than all of the others. It always depends on your personal preference, the restrictions of you client, company, or team, your code base, your language, your version control, and many other issues.

One rule, which is the one that I follow and encourage others to follow, is the Boy Scout Rule. It’s based on the “leave the campsite better than you found it” idea. It’s often said to be a line related to the scouts, but it seems to work well for code also.

If you have some old code that needs refactoring, I would guess it’s not tested. Since it’s not tested, you run the risk of creating bugs even if you’re testing it while refactoring. Some small changes will need to be made in order to test the code. You could miss something. That makes it dangerous. This is why you want to keep your refactoring to small pieces at a time. When and where you refactor are the next questions.

The Boy Scout Rule answers these questions nicely. You want to refactor the code you’re working on now. It’s fresh in your mind. You know how it’s supposed to work since you’re in there making changes now. It obviously is a volatile place, which should be cleaned up. You’re in there changing it now aren’t you?

Plus your current change might make things worse if you don’t do a bit of refactoring first!

The 2011 NimblePros Software Craftsmanship Calendar featured the Boy Scout Rule in the month of July, so for the rest of July, try to do small pieces of refactoring as you go into parts of a legacy codebase. Rewrite some code, write a test if you can, update an outdated comment (if you don’t just remove it), or even just write a better variable name.

It’s all about incremental improvements. The agile community should be loving this rule, since it is all about incremental changes and improvements.

Enjoy the rest of July! Don’t forget that you should also be avoiding Feature Creep.

I think we’ve all heard about the great security policies that companies implement.

Companies want you changing your password every other day, without repeating any passwords you’ve ever used, without using any characters from your first name or last name, without having any numbers associated with your address, phone number, birth date, the current year, or your age, and they want your password to be exactly 12 characters.

All of these rules are really funny. I don’t think I need to tell too many people that there are better ways of creating passwords.

All of that aside, those rules are kind of interesting. I use a lot of different passwords. I change the important ones. I don’t change the unimportant ones.

The Bad

When companies implement these policies, they like to have software in place to enforce the policies. Now is a great time to discuss how not to manage these policies.

In this instance the company requires that passwords be changed every 2 months. The passwords cannot contain more than 2 consecutive characters from the person’s first and last name (challenging to avoid as a user creating a password). You cannot use any of your last 3 passwords.

I was going to be rewriting this logic in a new system. I asked a few questions about this. One important questions is, if the password needs to be updated if the person’s name changes. For example they could have gotten married and may not be violating the password not containing consecutive name characters issue. Since I can’t find out what their password is in code, I have to just make them change the password.

This company obviously cares about securing this information since it’s implementing this password policy.

The application is using salted, hashed passwords when storing them in the database. That’s good.

I later noticed how the existing system handled password history. Clear text passwords stored with the username all in a table in the database. Every single password every user had ever used on the site. This is a scary pile of passwords that likely get used all over the Internet by these users. It even included the current password being used by that user.

The Better

These should not be stored this way! If you have to keep passwords around, keep the hash only. Also, get a new salt with each new password. Treat these as passwords (they are!) Don’t be foolishly insecure with this stuff. If someone accessed that database, they would have usernames and passwords for people to try all over the Internet. That would be a huge disaster.

If you have to keep that kind of information around, at least don’t have it in clear text. It would also be a good idea to limit the information that is there by removing the old passwords that are no longer relevant. Since they only care about the last 3 in the history, there is no reason to keep any more than that. Remove the old ones.

To be honest, I also don’t think that those types of password policies are all that secure anyway. I am sure that there is a weak link somewhere in the human element that would render the policy useless anyway.

More Code Audit Nuggets

Keep watching for more interesting nuggets of stuff that I’ve seen in codebases.

Static methods are very useful and powerful tools when writing code. As I am sure many people have pointed out, these tools should be used sparingly and cautiously.

Statics are dangerous if they have dependencies. This is because they’re hard to test. Or so I thought. Keep in mind, that just because you can mock statics does not mean that you should be using a lot of statics.

The Bad

I saw a great case of this misuse of statics in some cross-cutting code. It would be a great place to use Aspect Oriented Programming, but was instead a mess of statics and dependencies.

This cross-cutting code was being used throughout the application. There was barely a section of the site that did not use this code. Not once in the code had anyone unit tested any of the code that used these methods. It wasn’t being mocked, so there wasn’t really a way to unit test anything using that code.

Starting things off is a class. As you can probably have guessed from this post’s title, it has a static constructor. In the constructor, it accesses a database and creates an instance of the class. Each of the methods in this static class take in parameters and then write to the file system.

I was trying to write code in this code base, but I could not make any changes yet. I needed to get that monster abstracted so that I could work around it.

The Better

For cross-cutting concerns like this, it’s common to use statics. That’s OK. Not ideal, but OK. You just need to be careful. Try to limit the damage.

If done nicely, the Singleton isn’t horrible. Just keep in mind that tests will need to mock out that singleton when you’re using it.

For this class, I wrote a wrapper around it that implemented an interface. My code depended solely on that interface, and I mocked the interface in my tests. This let me minimize the impact of the change. In the long run, It will be important to make sure that the wrapper gets used throughout the codebase. Anywhere the static was used, it should not depend on the wrapper. This is a quick, easy solution that slowly improves the code.

In my opinion, that’s the best way to clean up any code base. Just fix a piece at a time. Make sure that when you touch any other part of the code that you use that same wrapper and eventually things will start looking a lot better.

More Code Audit Nuggets

Keep watching for more interesting nuggets of stuff that I’ve seen in codebases.

I am coining a new term and have just googled to make sure that it’s not written anywhere yet. “Massive Multiplied Methods” did not appear in a Google search at all. I establish that I am likely the first person to post those 3 words together in that order on the Internet.

Sometimes developers don’t do that. Occasionally, we can find places where people completely ignored those two.

The Bad

Imagine a site whose content comes from editors uploading lists of content stored in spreadsheets. Each spreadsheet, for some reason, has to have only 1 type of content (even though the data all has the same fields).

Now imagine that when the system reads each row from the spreadsheet, it needs to map each value onto a specific record from the database. The data is normalized, so what was a string in the excel file needs to link to the correct other table in the database.

The user selects which type of content is being uploaded and then the system will call the method for uploading that content. Yes, it’s just one method. Any other methods called are part of the .NET Framework. Making things worse, the methods are long. Very long. Each one was about 1,500 lines of code.

There were 5 different types. Each of these types has its own 1,500 line method. That’s about 7,500 lines of code for these.

And the best part about those 5 methods? They’re all identical except for 1 string. Literally, the only difference between these methods is the magic string literal used for the type.

The Better

I am not sure what to say here other than. Go learn about parameters in methods and how to create methods to break an algorithm into logical parts. Wow. Just. Wow.

It’s bad enough that the string could have just been in the spreadsheet to begin with, but did it really require 5 separate methods?

To top it off, you could have just accessed the value from the drop down list the user selected from to indicate the type. This was all in a code behind file anyway, so it could have just grabbed that value. (It shouldn’t have been there, but this would at least be slightly better.)

I don’t know how people let copy and paste operations get that bad. To be honest, I am amazed that the methods still matched. I guess people just avoided changing the code because it was too ugly. (That’s a mistake, because we need to clean up the scary code.)

More Code Audit Nuggets

Keep watching for more interesting nuggets of stuff that I’ve seen in codebases.

We’ve all been there before. Our project is moving along just fine. The release date is in sight. We’re on track to get everything pushed out right on schedule. Then everything is derailed by a last minute must have new feature. Maybe it’s the differentiator that will raise your product out of the crowd. Perhaps it’s just one small feature that will only push back the release by a day or two. When you’ve finished that one, however, will you have any additional must have features?

This is called Feature Creep. It’s when new features get added to the scope, and often, these features will cause delays in the project.

As a proponent of agile development, I approve of clients adding new features, however, it’s important to understand that adding in a feature should mean replacing a previous feature or you’ll have to push back the release date. It’s important to understand that with many (certainly not all) software projects, you can update the software at some point within the next few months with new features. It’s important to release something to the client. Adding on more and more new features right before a release will just mean a delayed release.

The NimblePros 2012 Software Craftsmanship Anti Patterns calendar for the month of July is about Feature Creep. So for the month of July, make sure that you watch to make sure that your scope is not ever increasing and pushing off release dates. Shipping is a feature, and your product should have that feature also. If you keep pushing off the release date, it will never ship.

Once the airbag, rocket boosters, seat belts, and parachute are added to this bike it will be done. Oh. And an ejector seat. We also need to add spokes, so I can put a baseball card in them. We need a bell too. And a horn. It won’t take long to paint it blue will it? Can we add padding to the seat? That’s the last feature. I promise. Oh I forgot about the mud flaps. Once that’s done we can ship it. As long as there are pegs on the back for passengers.