I made mistakes trying to chase new features rather than perfect existing ones.

Luke Picciau - 2018-11-06

Hi everyone. Its been a little while since I posted news on my project
PikaTrack. I know a lot of you expressed interest on
Reddit as well as Lobsters. I thank you all for the support and I think its about time I explain
how things have been going over the last month.

If you have been watching my commits you can see that there hasn’t been much activity in the past
few weeks. After I announced the first live version of the website and gained a huge rush of
users I put all my effort in to trying to add new and exciting features. I know what I have now
is pretty basic but I was on the verge of adding a lot of really exciting things that I couldn’t
wait to get in to everyone’s hands.

On October 21st I had finished a feature that I know many are waiting on. The feature enabled the
website to display badges on your activity to show where you had improved on your previous
efforts. After the feature was working I tested it a few times on my development machine and it
seemed to be working fine so I pushed it to production and thats where things went bad. A
combination of a missing docker volume mount and the server running out of memory caused the
website to become unusable and data left half processed. After spending an hour with my hair on
fire trying to work out what happened I realized the best solution was to roll everything back a
few hours to a backup.

In reality this probably wasn’t the worst thing in the world. Its likely no one even noticed the
few hours my website was down but for me this made me stop and rethink how I have been developing
software. If I ever want to run a serious website and have users trust me to not lose their data
I need to make some changes. This is the main reason I haven’t developed any new features in the
past few weeks. I have come up with a list of everything I have been doing wrong and what actions I
should make to solve these issues.

What went wrong

Prioritizing new features over improving old ones

New features are fun, new features make people happy and in my pursuit of making something people
liked I neglected fully finishing existing features when there was a chance to make something new.
My software only handled happy paths and many features had ways to break them resulting in bad UI
and an error 500 displayed on the screen. I can see that some users have submitted invalid data that
I didn’t handle properly. This invalid data posed no security or stability issues for the website
but its no fun at all for the person trying to use the website.

In the future I need to think really hard about when features are actually finished and every
realistic edge case is covered so the user cannot possibly get things wrong or hit something
that wasn’t expected or supported. It may also be worth getting someone else to try using things
before they go live but on a single person project having a separate tester is a bit of a luxury.

Neglecting CI

When I started the project I quickly set up automated testing with GitLab CI. The problem is I never
took the time to actually write the tests to be run by the CI. Again this came from my desire to get
new stuff out to people as fast as possible. And for a few months that worked but now I am facing
the consequences of neglecting testing.

In the future all features should have tests at least covering the common use and ideally covering
edge cases as well.

Having a poor deployment process

When I first started setting up deployment I had a goal in mind that I wanted the app to be super
easy for anyone to host on their own without having to be a developer of the app themself. To
achieve that I picked Docker with Docker Compose. These two seemed like the simple choice for
something that would just work for everyone.

What I have now is not what I wished for. The app is not easy for anyone to host on their own. It’s
not even easy for me to host. Each deploy requires multiple manual steps that must be run in order
that should have been automated. Configuration of the application is complex and requires inside
knowledge of how the app works.

This comes down mostly to my lack of knowledge of docker deployment. In the past I have used
Capistrano for deployment which has worked amazingly well but I chose not to for this project as I
thought docker would be a better fit for my goals.

I still think docker and docker compose is the right choice but I need to spend more time
researching how to make it work and automate my processes better.

That’s all for this update. Although its a little negative, this will not hold the project back in
to the future. I just wanted to share what I have been working on and maybe this can help others out
to avoid the same mistakes. After a little bit of cleaning stuff up I’ll be back to the fun stuff
and sharing more progress updates!