Also, I think his lecture on biological underpinnings of religiosity that is a part of this course is not in the playlist, I think because its pretty controversial, you can find it as an old recording here

Needed some usual things for internet operations these days like system monitoring, smart logging, graphing and alerting, high availiability, cloud, ids/ips, and mucho security.

Did this many times for random projects but this one was more serious and so I decided to rethink everything with focus on the security infrastructure.
Manuals were read and quite a few quick projects were made and there is more to come so this post will be updated with time.
I'll quickly go through the novel things I implemented and novel thoughs I have on the subject. This post will be a mess and there is a lot to write,

Quick logistics for nodejs apps

this is boring and already exists but I like the way I do things.

lego is a node app plugin system, works by looking for node_modules modules with certain prefix, supports dependency trees and a central config file.

System monitoring?

munin is a collection of perl? scripts from '98 ran by crontab. UGH. nagios is.. yeah, same thing.

went with logstash -> elasticsearch -> kibana for data collection/graphing and riemann for alerting. Didn't figure out active system checks yet (a job for nagios usually. stuff like pinging, checking if websites render correctly, potentially crawling them and such)

all kinds of apps write to logstash. my node apps, log data collectors (lumberjack), local system checks IDS, and such, it works well. Data from everywhere is aggregated and graphed together. I've written something simmilar to this stack as one off apps for debugging complex architecture a bunchoftimes, was very happy to find out that someone actually already works on such a thing pretty seriously.

didn't like collectd as it seemed like something that hangs out in a munin/nagios oldies crowd. so I wrote:

a simple probe that collects data from machines locally and sends it to a central server (in my case logstash) via udp json.

Geographical internet stuff

csi-internet is there for you if you want to draw some computer stuff on a map. Also good to impress your girlfriend if she got used to cmatrix. As a test of a graphing engine it can ping around, do traceroutes and draw heatmaps. Messy atm as looking at pictures was more fun then looking at good code. work in progress. works on top of d3 and datamaps.

Firewalls

Writing tons of iptables rules is for robots and not humans and so I'd fuck it up eventually.
pyromaniac is a thing that renders JSON into iptables commands.

this program is highly specific to my scenario (machines behind a NAT hypervisor with strict control of each connection) I'm showing an aproach here more then sharing my code as something that will work for you.

IDS/IPS

I really like this part, but will write it up later when I have more to show

these might be used in combination with some simple IR or just cameras (need more CPU but could double as optical flow sensors) used to locate becaons for landing or grabbing things

Ideally, both sensors should have an unobstructed front view on pitch dimension, with aditional unobstructed jaw dimension for LIDAR.
Both sensors need to be able to independantly rotate on pitch axis for compensation of the angle of the drone when flying, and for scanning up and down (going in through windows, looking down when landing, etc) maybe rangefinder should also rotate on jaw axis on some fast servos for looking around independant of the drone orientation. not sure yet.

--

IR and ultrasound were also options, they are cheap but tricky and unreliable, (ultrasound behaves extra badly due to propeller noise on drones)
These things can be compensated for, but laser rangefinding will always afaik give more accuracy and reliability at a price.

people often ask me why I own a drone, what is it for, and then half of the time they say that I could use it to record weddings and get some cash flowin'

If mention of recording weddings doesn't kill all my will for conversation, I'll say drones are so simple to make these days that I find it hard to justify not owning one.
also, at the time there is no regulation and no one ever counts on you owning a flying robot, so its a sweet spot to own one.
a drone is a flying sidekick, it can, in theory, do whatever a flying sidekick can do. it can enter through windows and steal jewlery with its tiny robot hands. Sounds ridicilous but technically it is not hard to do. In the following years we'll have to rethink space and what can be done, and before that happens, I get to be a kind of a superhero.

today, drones are very primitive. I'll fix some parts that I care about. and if I make anything useful, Im guessing drone related kickstarter projects would be welcome.

a few ideas,

work on being more autonomus, gesture recognition, landing on backpack, automatic charging, simple commands like follow me, land and observe, and such

picking up bread in the morning

ground locomotion, wheels or legs!

stringing a rope somewhere high or for a tyrolean traverse - I can be a very slow batman and I always wanted to string a hammock up between two buildings.

guarding a camp spot and scarring bears away

looking for people

planting and picking up sensors

security? (I'm terrified of security robots working against me, I'm excited about security robots working for me - maybe I won't touch this.)

obvious things like scouting or mapping, it would be nice to have something that creates a google maps or openstreetmap layer as it flyes around, and it shouldn't be too hard to implement, in theory :)

few concrete things to work on, sorted by priority

integrate a real computer (starting with beaglebone in my case, will need to reverse and implement sbus and naza CAN bus protocols)

long range modem - something like xbee xtend and/or sim cards for control over gsm

So, we'd like to carry a boat for going down rivers and exploring curious things on water, while being self sufficient with food and camping equipment. Giving up on the boat and building bamboo rafts was one of the propositions:) We ended up buying a small 12kg inflatible kayak that packs in a 60L backpack,

I've discovered today that we are completely defeated by this awesome guy 200 years ago.

Halkett had long been interested in the difficulties of travelling in the Canadian Arctic, and the problems involved in designing boats light enough to be carried over arduous terrain, but robust enough to be used in extreme weather conditions.
the hull of the boat could be worn as a cloak, the oar used as a walking stick, and the sail as an umbrella

Spurred on by the successful testing of the boat-cloak, Halkett designed a larger version that folded into a knapsack. When inflated, it could carry two men, operating a paddle on each side, and when deflated served as a waterproof blanket to allow the users to camp on wet ground
Although the Admiralty saw no use to which Halkett's designs could be put in general naval service, this larger design was extremely well received by explorers

Ended up sneaking past a few guards, one of them taking a shower. The building below us has 20-30 stories, I'm guessing the crane was 80-100m high? A long climb. Where do crane workers go to toilet? Also. my god! I want to do this again.

Flickr tag lshblog_crane will be displayed here, if you have javascript.

We are talking to a random french couple, answering common questions and ending up explaining that we've met by going into the subway tunnels through an airduct. They mention seeing some photos of us climbing a building in hongdae park and I wonder if that was the night when the crazy guy in the underwear climbed it with us. At some point I say something like

remember that girl from yesterday that wanted us to smuggle diamonds from africa? I think she might be a stripper.

To which a guy asks us if those are real events or are we talking about a spy movie, and I realize how weird our conversation sounds from another persons perspective, and how most of the things that have been happening would sound, and I conclude that we did good.

I bought a climbing equipment and all places are reacheable now. I'm living with Kanghee, and a great group of friends are around.
We are planning to buy a ~15 meter sailboat and sail through philipines to papua new guinea next year. we'll see how that goes. we'll have a lot to learn before doing this.

It was super easy. a lot of japanese react as if someone is doing something insane and exciting and are happy about it. Most of them are also too shy to stop and they would even cover their smiling faces if I maintained eye contact for too long <3

Izu peninsula has two sides, people call them a female and male side, one is full of fine sand beaches and the other is made of rough steep vulcanic rocks.. I'm guessing that strong waves tend to hit it from the female side.

My favorite ride was this businessman, it seemed that what was happening was a way bigger adventure for him then for me. He would start laughing histerically out of excitement whenever I'd speak. So naturally, I wanted to speak as much as possible but that was hard as he didn't understand any english.

Arrived to Irozaki port-o fairly quickly and started a hike up the hill towards where the jungle theme park is supposed to be.

Found bunch of abandoned buildings and infrastructure at the top but couldn't find the main dome, ended up exploring the peninsula.

Climbed an antenna tower to look around (kinda. half way. it was scarry. note to self - learn how to make a harness out of climbing rope, I could have secured myself). found the dome - its not a dome. didn't look at the online photos good enough, and found a coast that looked so amazing that it stole my attention. I wanted to quickly check it out before heading out to the park but the place was great and I ended up spending the night.

Everything seemed totally devoid of life but it turns out, the place becomes a spiderland during the night. I don't know what they eat, each other? I tucked my shirt in my pants.

I'm super happy with the funny-shoes. Japanese people say b ooooooooooo ninja shoesb and they are right! they are quiet and great for climbing and I have a lot of control on the rough terrain. a downside is that you freak people (including yourself) out, and need to talk about them a lot. its not worth it in the city.

It was a great night. I realized that I was alone for the first time in three months. There was a huge lighthouse spotlight sweeping the panorama and big ships passing by on their way towards port of Tokyo.
The next morning I discovered that all the bread I bought is filled with chocolate and that all the cans I have are the same discusting fish. Had a breakfast that stimulated my taste buds and left to quickly explore the rest of the theme park.

Turns out, a few years have passed and big parts of the park are overgrown now. It was quite late and quite hot and quite overgrown and there was noone around to encourage me so I decided to keep going towards Kyoto and give up on getting inside.

on the way there I couldn't pass on the opportunity to sneak into some classuy star hotel onsen for the third time.

This forest was always shifty, but its popularity grew due to a contraversial book, The Complete Manual of Suicide where it is described as the perfect place to die.
Over 100 bodies per year are discovered there, a lot more probably don't ever get found.

To make things comically creepy, the forest lies on some large magnetic rocks, which make compasses useless. my gps seems to rely on a digital compass, so I quickly rehersed the night before on how to orientate myself by the sun.

The place is fairly touristic, it has a few hiking trails, and it lies just below mount Fuji. When we got there they told us that a black bear was sighted today exactly where we were going. We ended up yelling and clapping our hands a lot to scare it away. We walked a lot, not knowing exactly where we should go, and got off the trail a lot, relying on a bad tourist map.

A small culture grew around suicides in the forest. for example, people that think that they might change their minds, walk in leaving a trail of (usually nylon) string, which makes parts of the forest look really strange, with nylon strings going everywhere. The whole place is full of strange objects that seem to have been used in ritual like situations. Compass and gps ended up working well for the most part, with occasional crazyness.

I've been hoping to write this for a while, its not a very exciting project so I've been working on this when I'm tired.

its a bit unusual, nodejs-blosxom, ~400 lines of coffescript. this is very much a work in progress, I'd advise you to use it as inspiration more then as a code you run. templates related to my own homepage are included in the repo. might move them laters.

features

each post is a markdown file (maybe I should check out ORG mode files?)

posts are in a git repo, so they can be written offline and synced with the web later

I wanted something like google plus circles, with restricted tags for particular people

implicit tagging by placement .md posts in folders

explicit tagging/metadata for a particular post by adding a JSON line as a first line in a file

tag filtering

no javascript on the clientside

rss for arbitrary tag combination

immediate todo

full set operations on tags

think about collapsing large posts when they are initially displayed..

also, don't render ALL matching posts, do the next/prev button thing.

saving of blog post metadata JSON back to files

color private tags differently

wiki (some way to link to other posts or tags (just extend the markdown?))