Tuesday, January 14, 2014

Over the course of a night and morning I built a small flask app (Flask is a micro web framework for Python. Django is similar but more heavy-duty). It is essentially a dictionary. You can search for an entry by its keyword, or view the helptexts for all the entries, or (if you are logged in) add and remove entries. It is a straightforward derivative of excellent the Flaskr tutorial.

For me, I am really proud of this app even though it is dirt simple and not much to look at because this is the first time I really interacted with a database in a programming language.

The intent was to make a database of documentation for the AVR Atmega328P microcontroller. However, since then I have decided to move to using a mediawiki (sample page).

The app may be seen live at salty-retreat-5363.herokuapp.com. The username is "admin" and the password is "default". The database gets wiped at least once every 24 hours.

If you'd like to run the app locally and play around with source code, please see https://github.com/nouyang/WTFisThisRegister for detailed instructions. That page also details how to deploy this app (or similar ones, such as Flaskr) to Heroku. I used http://dillinger.io/ to play around with markdown online and learn it and actually document this project with a proper README.

Things Learned

Here are some things I learned in the course of making this:

Gitignore files need to be created before you run "git add ." -- in general, the line telling it to ignore a file needs to exist before you added it to the version control, git won't automagically remove it for you.

Virtualenv does NOT like spaces in your directory path! This was tricky to debug as even in -v verbose mode the full path for my virtualenv packages was shortened and so I couldn't even notice that my path had spaces in it.

Databases use cursors.

The Process

Here are some images from the process. Click on them to view them in full resolution.

1. First I started out just using a dictionary (no database involved) and a single hello.py file.

2. I then followed the Flaskr tutorial. After doing so I wanted to modify it so that the text was monospaced and preserved whitespaces, because it doesn't by default.

I used the "pre" tag at first.

but soon switched to using CSS.

This actually caused me a lot of headache later troubleshooting why there was random whitespace in my design, because I didn't restrict where the CSS was applied enough. So much whitespace:

6. I learned about initializing databases, namely by dumbly thinking I needed to include the flaskr.db file and no schema.sql file in the repository to allow other people to get it working on their systems. Actually, I was just missing a line, "init_db()", before "app_run()". In the end I am still keeping it without that line so that I have persistence in the database (I can stop the server and restart it and the database entries will still be there).

All in all I learned a lot! Do browse through the source code and play it with yourself if you'd like.

For example, you might create the file at ~/.gitignore_global and add some rules to it. To add this file to your cross-repository configuration, run git config --global core.excludesfile ~/.gitignore_global. Local per-repository rules can be added to the .git/info/exclude file in your repository. This method can be used for locally-generated files that you don't expect other users to generate.

Okay, so now I want to pull the config parameters (passwords, admin usernames, secretkeys) out into a separate file that is gitignored.
Done. Created a file called databaseconfig.py, added the line "from databaseconfig import secretkey, username, password" to the main python file.

WHOA good point I forgot to turn debug off! Whaaaa. Okay that is terrible of me.

# configuration

DEBUG = False

WAH. still sad.

Let's try to fix foreman by adding the var to .env... Does that work?

.env does not like spaces around the equal sign assignment operator!

S3_KEY=mykey

NOT

S3_KEY = mykey

Oh! Make sure to add "import os" to the main python file if we are using os.environ. DUH.

Note: Only today I realized that in pasting URLs from the location bar in chromium-browser (ubuntu 12.10) into blogger, the URLs are automatically turned into hyperlinks, but they remain plain old text and I have to manually turn them into hyperlinks when I copy them from firefox, my main browser :( Such a weird specific bug.

Ah? It still doesn't work?? Why? It works locally on foreman, but not remote on heroku.

Great, now my config works in with both "foreman start" and "python WTFisThisRegister.py".

Now to get databaseconfig to show up in heroku. Do I want to?
No. A cleaner solution instead of all this is to use try, importerrors. Since the present of databaseconfig.py can be used to indicate that we are running it locally.

Okay, still works locally but not remotely. Maybe it is a port issue. Sucks that these tiny changes debugging production (heroku environment) take forever to try (30 seconds for each time I git push heroku master)

2014-01-12T05:58:48.956357+00:00 heroku[web.1]: State changed from crashed to starting

Note that the way I have it set up -- foreman locally will run with Debug = False, heroku will run with Debug = false, and python locally will run with Debug = True. A rather peculiar state of affairs, but fine for me.

Let's debug the download-from-github install for the Readme.
$ virtualenv --distribute --no-site-packages venv
The --no-site-packages flag is deprecated; it is now the default behavior.
New python executable in env/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...
Error [Errno 2] No such file or directory while executing command /home/nrw/projects/t...env/bin/easy_install /usr/share/python-vi...p-1.1.debian1.tar.gz
...Installing pip...done.