Noobing it since forevrrrrr

Tag: coding

I’ve begun thinking about how to deploy my app. It would have been easy as a one-pager, but my app has a PostgreSQL database, and it is a social networking site, which means I should prepare for lots and lots of users (assuming people pick it up at all).

Rooting around in this direction turned up this helpful video that tells you about different ways to host your app. It also turned up the idea of “workers”. Heroku and DigitalOcean use workers. What are these? I don’t know yet. Of course there are definitions, as usual. But definitions try to be one-stop shops and catch-all solutions, and in catering to everyone, they cater to no one at all. Ok, I exaggerate. They cater to some. Dictionaries do a pretty decent job although they don’t tell you about nuances, connotations, and other baggage that comes with words.

While researching workers and deployment, I realised that I have to look at sharding, and possibly logical sharding in particular. Also connection pools, database caching, clustering, cloud formation, load balancing, docker/virtualisation, and batching writes to the database. In short, it seems, I have to learn some devops.

We meet again, prong of epistemology. To go from Flask to deployment means learning a whole critical bunch of things.

The bad thing is, I don’t know how much I should know about devops, so I’m groping around in the dark, and groping around for all kinds of different things. To mix two idioms, I’m a drowning man in the dark clutching at straws. The good thing, I suppose, is that I think I know the primary question here: how many requests can a worker handle? It’s a surprisingly hard question to answer, and I have to answer it so I know if the cheapest plan on DigitalOcean is suitable for a starting social networking web app. I’ll root around some more.

Deployment also made me think about marketing, which led me to this video, which reminded me that I have to cite open source code, which means I have to review all my code and try to remember where each part of it came from if I didn’t write it myself.

At the same time, I’ve learnt and exercised enough Flask to know that I should pick up Django and rebuild the entire app in it, and that I should also learn JavaScript, because a lot of interactions, like socketio, depends on it.

Does the prong of epistemology become larger and larger as knowledge increments? At this point, I should be working in a team with proper front end and back end developers and designers, right? But with only basic Python, Flask, and SQL under my belt, what team roughing it out in the savage wilderness of live business operations would want me? “Not good enough”, that old refrain.

I was totally confused over how to use flask-security, and researching that made me super confused about sessions vs. cookies vs. tokens, and researching that brought me to the topic of the model-view-controller (MVC). Long story short, I found a pretty good video that introduces the paradigm:

After watching it, I realised I could reduce my code by maybe 90% if I abstracted away huge repetitive bunches of it. I knew this in the back of my mind before but was too caught up learning the basic basics to think about it clearly. Anyway, after watching a few more videos, I think this is one of the better introductions. It provides a brief enough history that segues smoothly into contemporary practice.

I’m still completely baffled with flask-security and slightly more clear about sessions-cookies-tokens. RTFM not working, especially for flask-security.

I picked code up in part to get a better paying job than anything on offer in the arts. Now I have Python, Flask, and SQL with me. However, after going through job sites for a few days, I’ve discovered that most jobs are for front end developers with HTML5, CSS, and Javascript.

There is a lot of demand for Javascript using AngularJS, React, NodeJS, etc. So, the fastest way to get a job is to pick up front end web dev/design skills.

If you stick with Python and want to do some back end work, you’ll need Django, and NoSQL, typically MongoDB. Not Flask. Exceedingly few hiring companies use Flask.

Ruby on Rails appears quite a bit too. Also a lot of demand for UI/UX. But that floats into a more distant skill set.

I should close the loop on this series. Part 1 and Part 2, if you’re interested.

The conclusion is, I haven’t mastered it, but I’ve definitely figured out the basics. I thought I’d need to learn more from CS350 for some time before getting here. Instead I found myself going back to just two of the videos in Charles Severance’s “Using Databases with Python” several times and jogging memory with code I’d worked with during that course. I’m forgetful so it was great to have his videos on hand. I also spent a lot of time googling, reading, and YouTubing tutorials.

Basically, I had to draw from everywhere at this point, and the range was really, really wide.

One uses Python to concatenate randomly selected strings from various lists to form grammatically correct English sentences and then displays a new one in the middle of a webpage at regular intervals.

The other draws from the IMDB database, banishes all USA and Western European films, randomly picks 10 from the rest of the world, and displays them along with their trailers from YouTube. This one was not so trivial for me. There was a lot of junk data in the IMDB database to clean up, and I had to work out how to scrape YouTube to embed the trailers on my site.

I can’t say if it’ll continue this way. Already there are certain things that are falling into doubt. For instance, CS350 leverages AngularJS a little bit to do Flask stuff, and in the course of reading all the other sources I found out that there is a lot of uncertainty surrounding the future of AngularJS right now. That particular case is not terribly important. Just a small example. We can use other JS frameworks anyway.

What’s next for me? I’m going to build up that film site of mine. Add users to it, work on some formulae running in the back end. Get more familiar with PostgreSQL. Get a better handle on how to mash Flask up with Bootstrap. And then maybe learn Cordova next year to build mobile apps.

Coding is great fun, especially since I use it to engage real world problems. For example, the winner-takes-all effect. Films from USA and Western Europe, and especially from Hollywood, saturate almost every single one of our media environments. My film site bans all USA and Western European films in order to allow the diversity of the rest of the world’s film heritage to come to the fore. Turns out, it’s wonderful. Using it, I discovered amazing, beautiful films from Azerbaijan, Indonesia, Malaysia, Turkey, and elsewhere. I think there was even one from Antigua and Barbados. It’s not ready for the public yet, but I hope to put it up somewhere just for show-and-tell soon.

Another big reason that drove my learning was the oppressive pecuniary pressures in the art world. I wonder if I’m ready for an entry level/junior position yet. I have no idea. Time to try looking around for a job now as well, I guess. At the very least, I’ll get a better idea of what the world wants.

I’ve made crazy progress in the last couple of weeks. As a result of that, I’ve managed to get my grubby hands onto SQL and I’ve even wrestled with which database management system to use. I used Digital Oceans’ succinct article to settle on PostgreSQL.

Unfortunately, unlike the SQLite I’m a bit more familiar with, PostgreSQL databases (DB) don’t come in one handy portable file. I spent maybe a couple of hours poking around the PostgreSQL directory wondering where the DB was. Then I googled it and read some articles a bit more closely and figured out it’s more complicated than a single manifestation in a single locus.

So, using the vocabulary I had, I googled “how to export PostgreSQL database” and a couple of other variants related to it. Didn’t work. But at some point, I found out that you have to do a dump. That gets you a file you can move around. But not many were good at explaining it, or even the concept of it. Finally, I read the documentation here and thought, this should be easy. But once I’d done a pg_dump, I couldn’t find the file. I spent another couple of hours trying out various things.

Then, finally, finally, I YouTubed “postgresql dump” or something like that and found this silent video. I watched it carefully and realised that the dump is created in the directory from which I work in the command line. And that has not been mentioned in all the previous places I’d looked in, including the documentation. Maybe it’s obvious to experts. But beginners need something a bit more explicit.

I first looked at Flask and Django because people said that if I want to use Python on a webpage I’d need a web framework. Turns out, the simpler thing to learn is not a web framework but a template engine.

What is a template engine? Who the hell knows? Do you think I knew as a beginner? Why do all these tutorials call it that? Tutorials for beginners, no less. And if you want to even learn one, Jinja2 in my case, you can barely find a tutorial that focuses on it. Most Jinja2 tutorials involve a great deal of Flask.

So here’s the thing, Jinja2 is the medium that weaves between Python and a webpage on the front end. It allows us to show results from a Python function in the back end on a web page. It can’t do everything but, sometimes, it’s more than enough for us.

As far as I know, Jinja2 remains an integral part of the Holy Trinity of Flask-Jinja2-Werkzeug. Pip install Flask would get you the entire Trinity. However, you don’t need to learn all of Flask to put Python functions on the web. You do still need to import flask in your script and learn the basics of where to put your files and how to name your folders the way Flask likes it, but after that, it can be simple. Here’s an example.

In your index.html file, which must be placed in the templates folder according to Flask’s preferences, you can have a string like, say:

And that’s it. Your string on the webpage randomly displays a type of jam.

You can do other things with it, of course, and you’ll need some more knowledge to better appreciate what it can do. But my point is, for beginners to get to here, we first need to tunnel through mountains of other stuff, a lot of which is unnecessary (even if nice) simply because the current ecology of pedagogical resources don’t design for noobs.