Needed some usual things for internet operations these days like system monitoring, smart logging, graphing and alerting, high availiability, cloud, ids/ips, and mucho security.

Did this many times for random projects but this one was more serious and so I decided to rethink everything with focus on the security infrastructure.
Manuals were read and quite a few quick projects were made and there is more to come so this post will be updated with time.
I'll quickly go through the novel things I implemented and novel thoughs I have on the subject. This post will be a mess and there is a lot to write,

Quick logistics for nodejs apps

this is boring and already exists but I like the way I do things.

lego is a node app plugin system, works by looking for node_modules modules with certain prefix, supports dependency trees and a central config file.

System monitoring?

munin is a collection of perl? scripts from '98 ran by crontab. UGH. nagios is.. yeah, same thing.

went with logstash -> elasticsearch -> kibana for data collection/graphing and riemann for alerting. Didn't figure out active system checks yet (a job for nagios usually. stuff like pinging, checking if websites render correctly, potentially crawling them and such)

all kinds of apps write to logstash. my node apps, log data collectors (lumberjack), local system checks IDS, and such, it works well. Data from everywhere is aggregated and graphed together. I've written something simmilar to this stack as one off apps for debugging complex architecture a bunchoftimes, was very happy to find out that someone actually already works on such a thing pretty seriously.

didn't like collectd as it seemed like something that hangs out in a munin/nagios oldies crowd. so I wrote:

a simple probe that collects data from machines locally and sends it to a central server (in my case logstash) via udp json.

Geographical internet stuff

csi-internet is there for you if you want to draw some computer stuff on a map. Also good to impress your girlfriend if she got used to cmatrix. As a test of a graphing engine it can ping around, do traceroutes and draw heatmaps. Messy atm as looking at pictures was more fun then looking at good code. work in progress. works on top of d3 and datamaps.

Firewalls

Writing tons of iptables rules is for robots and not humans and so I'd fuck it up eventually.
pyromaniac is a thing that renders JSON into iptables commands.

this program is highly specific to my scenario (machines behind a NAT hypervisor with strict control of each connection) I'm showing an aproach here more then sharing my code as something that will work for you.

IDS/IPS

I really like this part, but will write it up later when I have more to show

these might be used in combination with some simple IR or just cameras (need more CPU but could double as optical flow sensors) used to locate becaons for landing or grabbing things

Ideally, both sensors should have an unobstructed front view on pitch dimension, with aditional unobstructed jaw dimension for LIDAR.
Both sensors need to be able to independantly rotate on pitch axis for compensation of the angle of the drone when flying, and for scanning up and down (going in through windows, looking down when landing, etc) maybe rangefinder should also rotate on jaw axis on some fast servos for looking around independant of the drone orientation. not sure yet.

--

IR and ultrasound were also options, they are cheap but tricky and unreliable, (ultrasound behaves extra badly due to propeller noise on drones)
These things can be compensated for, but laser rangefinding will always afaik give more accuracy and reliability at a price.

async object or json pattern matching/validation used for validating function/process arguments, api call permissions, messages, etc. supports serialization/deserialization of patterns themselves, which I used to implement a generic db query language on top of this. for now, tests are documentation

random things, my laptop is using it to watch and react to system wide events, like lid being closed or connection to the internet being established, my cellphone is running it in order to connect and sync with my computers and it is what I use to control lights in my appartment or display data from remote servers on my desktop

I've been hoping to write this for a while, its not a very exciting project so I've been working on this when I'm tired.

its a bit unusual, nodejs-blosxom, ~400 lines of coffescript. this is very much a work in progress, I'd advise you to use it as inspiration more then as a code you run. templates related to my own homepage are included in the repo. might move them laters.

features

each post is a markdown file (maybe I should check out ORG mode files?)

posts are in a git repo, so they can be written offline and synced with the web later

I wanted something like google plus circles, with restricted tags for particular people

implicit tagging by placement .md posts in folders

explicit tagging/metadata for a particular post by adding a JSON line as a first line in a file

tag filtering

no javascript on the clientside

rss for arbitrary tag combination

immediate todo

full set operations on tags

think about collapsing large posts when they are initially displayed..

also, don't render ALL matching posts, do the next/prev button thing.

saving of blog post metadata JSON back to files

color private tags differently

wiki (some way to link to other posts or tags (just extend the markdown?))