Posts Tagged ‘Musings’

There’s a reason why many people stick to a programming language and reuse it whenever possible: it’s hard work to learn another one! So once you’ve learned one well, and in due time really, really well, then you get a big payback in the form of a substantial productivity boost. Because – as I said – it’s really a lot of hard work to become familiar, let alone proficient, with any non-trivial technology – including a programming language or an OS.

Here’s me, a few weeks ago:

I was completely overwhelmed by trying to learn Node.js, AngularJS, SocketStream, CoffeeScript, and more. To the point of not being able to even understand how it all fits together, and knowing that whatever I’d try, I would stumble around in the dark for ages.

I’ve been through the learning curve of three programming languages before. I like to think that I got quite good at each of them, making code “sing” on occasion, and generally being able to spend all my time and energy on the problem, not the tools. But I also know that it took years of tinkering. As Malcolm Gladwell suggests in his book Outliers, it may take some 10,000 hours to become really good at something. Maybe even at anything?

Ten. Thousand. Hours. Of concentrating on the work involved. That sure is a long time!

But here’s the trick: just run, stumble, and learn from each re- / disrecovery!

That doesn’t mean it’s easy. It’s not even easier than the last time, unfortunately: you can’t get good at learning new stuff – you still have to sweat it through. And there are dead ends. Tons of them. This is the corner I painted myself into a couple of days ago:

Let me explain:

I thought I was getting up to speed with CoffeeScript and the basics of AngularJS

… even started writing some nice code for it, with lots of debug / commit cycles

big portions were still clear as mud to me, but hey… I got some stuff working!

The trouble is: instead of learning things the way all these new tools and technologies were meant to be used, by people far more knowledgeable than me, I started fitting my old mental models on top of what I saw.

The result: a small piece, working, all wrapped into code that made it look nice to me.

That’s a big mistake. Frameworks such as AngularJS and SocketStream always have their own “mindset” (well, that of their designers, obviously) – and trying to ignore it is a sure path to frustration and probably even disaster, in the long term.

Ya can’t learn new things by just mapping everything to your old knowledge!

Yesterday, I realised that what I had written as code was partly just a wrapper around the bits of NG and SS which I started to understand. The result: a road block, sitting in the way of real understanding. My code wasn’t doing much, just re-phrasing it all!

So a few days ago, I tore most of it all down again (I’m talking about HouseMon, of course). It didn’t get the startup sequence right, and it was getting worse all the time.

Guess what? Went through a total, frustratingly disruptive, re-factoring of the code. And it ended up so much simpler and better… it blew me away. This NG + SS stuff is incredible.

Here’s me, after checking in the latest code a few hours ago:

No… no deep understanding yet. There’s no way to grasp it all in such a short time (yes, weeks is a short time) – not for me, anyway – but the fog is lifting. I’m writing less code again. And more and more finding my way around the web. The AngularFun project on GitHub and this post by Brian Ford are making so much more sense now.

I’ve also decided to try and change one more habit: I’m going back to the vim editor, or rather the MacVim version of it. I’ve used TextMate for several years (now v2, and open source), but the fact is that pointing at things sucks. It requires eye-hand coordination, whereas all command-line and keyboard-driven editing activity ends up settling itself in your muscle memory. Editors such as vim and emacs are amazing tools because of that.

Now I don’t know about you, but when I learn new things, the last thing on earth I’d want is to spend my already-strained mental energy on the process instead of the task at hand.

So these days, I’m in parallel learning mode: with head and spine working in tandem :)

I learned to program in C a long time ago, on a PDP11 running Unix (one of the first installations in the Netherlands). That’s over 30 years ago and guess what… that knowledge is still applicable. Back in full force on all of today’s embedded µC’s, in fact.

I’ll spare you the list of languages I learned before and after that time, but C has become what is probably the most widespread programming language ever. Today, it is the #1 implementation language, in fact. It powers the gcc toolchain, the Linux operating system, most servers and browsers, and … well, just about everything we use today.

It’s pretty useful to learn stuff which lasts… but also pretty hard to predict, alas!

Not just because switching means you have start all over again, but because you can become really productive at programming when spending years and years (or perhaps just 10,000 hours) learning the ins and outs, learning from others, and getting really familiar with all the programming language’s idioms, quirks, tricks, and smells.

C (and it its wake C++ and Objective-C) has become irreplaceable and timeless.

Fast-forward to today and the scenery sure has changed: there are now hundreds of programming languages, and so many people programming, that lots and lots of them can thrive alongside each other within their own communities.

While researching a bit how to move forward with a couple of larger projects here at JeeLabs, I’ve spent a lot of time looking around recently, to decide on where to go next.

The web and dynamic languages are here to stay, and that inevitably leads to JavaScript. When you look at GitHub, the most used programming language is JavaScript. This may be skewed by the fact that the JavaScript community prefers GitHub, or that people make more and smaller projects, but there is no denying that it’s a very active trend:

In a way, JavaScript went where Java once tried to go: becoming the de-facto standard language inside the browser, i.e. on the client side of the web. But there’s something else going on: not only is it taking over the client side of things, it’s also making inroads on the server end. If you look at the most active projects, again on GitHub, you get this list:

There’s something called Node.js in each of these top-5 charts. That’s JavaScript on the server side. Node.js has an event-based asynchronous processing model and is based on Google’s V8 engine. It’s also is phenomenally fast, due to its just-in-time compilation for x86 and ARM architectures.

And then the Aha-Erlebnis set in: JavaScript is the next C !

Think about it: it’s on all web browsers on all platforms, it’s complemented by a DOM, HTML, and CSS which bring it into an ever-richer visual world, and it’s slowly getting more and more traction on the server side of the web.

Just as with C at the time, I don’t expect the world to become mono-lingual, but I think that it is inevitable that we will see more and more developments on top of JavaScript.

With JavaScript comes a free text-based “data interchange protocol”. This is where XML tried to go, but failed – and where JSON is now taking over.

My conclusion (and prediction) is: like it or not, client-side JavaScript + JSON + server-side JavaScript is here to stay, and portable / efficient / readable enough to become acceptable for an ever-growing group of programmers. Just like C.

Node.js is implemented in C++ and can be extended in C++, which means that even special-purpose C libraries can be brought into the mix. So one way of looking at JavaScript, is as a dynamic language on top of C/C++.

I have to admit that it’s quite tempting to consider building everything in JavaScript from now on – because having the same language on all sides of a network configuration will probably make things a lot simpler. Actually, I’m also tempted to use pre-processors such as CoffeeScript, Jade, and Stylus, but these are really just optional conveniences (or gimmicks?) around the basic JavaScript, HTML, and CSS trio, respectively.

It’s easy to dismiss JavaScript as yet another fad. But doing so by ignorance would be a mistake – see the Blub Paradox by Paul Graham. Features such as list comprehensions are neat tricks, but easily worked around. Prototypal inheritance and lexical closures on the other hand, are profound concepts. Closures in combination with asynchronous processing (and a form of coding called CPS) are fairly complex, but the fact that some really smart guys can create libraries using these techniques and hide it from us mere mortals means you get a lot more than a new notation and some hyped-up libraries.

I’m not trying to scare you or show off. Nor am I cherry-picking features to bring out arguments in favour of JavaScript. Several languages offer similar – and sometimes even more powerful – features . Based on conceptual power alone, I’d prefer Common Lisp or Scheme, in fact. But JavaScript is dramatically more widespread, and very active / vibrant w.r.t. what is currently being developed in it and for it.

But it’s only fitting that today is also a great time to celebrate. Because this weblog has reached…

That’s right. Eight hundred articles on this weblog, published on a daily basis – at midnight Central European Time. About all the fun in physical computing and the electronics, hardware, software, and mechanics surrounding it – collected as a stream of notes describing my adventures in this geeky world of techno-babble.

The posts on this daily JeeLabs Weblog are – by definition – organized as a timeline. Some of it will inevitably be wildly obsolete by now, other parts are timeless and still useful, but if you didn’t tag along from the beginning in 2008, then it might all be somewhat overwhelming – or ridiculously frustrating, depending on your expectations.

There are a few ways to find your way around here. First of all, there are six categories (some of them quite large by now): AVR, Hardware, Musings, News, Software, and the catch-all Uncategorized. A somewhat more fine-grained structure is available through 28 “tags”, shown at the top of each post. There’s an updated listing of them on the intro page.

Another approach to finding your way around here, is to use the Search box at the bottom right of each page. It works quite well, but don’t search too broadly or you’ll get a result with hundreds of posts.

The third place to look for interesting posts is on the “Café“, and especially the pages in the hardware wiki, many of which have a Related Weblog Posts section at the end. I’ve tried to add links to new pages there whenever I could, although there’s still lots of room for improvement.

And last but not least, there’s the forum area, which is getting more and more traffic these days, so it could be a good place to search and ask questions. I tend to participate at least once a day (although quite a bit less in the two months ahead).

If shopping for JeeNodes and other JeeStuff is your kind of thing, don’t forget that the shop will close for one month, starting two weeks from now, and that there’s a discount / sale right now for current customers and contributors. See this page for all the details.

I’d like to thank everyone who in some form or other voiced their appreciation and encouraged me to push further. Techie as it all may be, this weblog is still about people, creativity, and shared passions. Because without all that, there is no point.

Scattered over the coming two months, Liesbeth and I will break away and travel in Europe for a while, and I’ll fill in the rest of this long summer by relaxing, learning, tinkering, experimenting, working on top-secret stuff, and cooking up all sorts of crazy hardware and software for future JeeLabs projects and products.

These daily weblog posts will resume on September 1st.

Until then, for your entertainment, a tale of two worlds: Splitscreen.

I’m already tracking most of my inventory, including pictures and locations, but the problem was that these locations were too broad. Rummaging through various piles of, ehm, junk can get boring and tedious after a while!

Now I’ve got to think about where everything should go. Shop products in one place, projects-in-progress in another, lab supplies in a third, not to mention the shipping department.

Jee Labs is about fun, not size, after all – I don’t really want to expand beyond the current 4 x 7 meter space currently in use (apart from a few large supply boxes and tools in the basement), which is going to require a bit of self-discipline. There’s still a fair bit of unused wall space, so that’s good.

Last month, we (Liesbeth and I) spent a delightful time in the Provence:

As usual, that means I get to switch to my French persona, as we visit new places and meet delightful people of all ages. Lots of baguettes, cheese, wine, and fruit – as usual. Life is embarrassingly good this way…

We visited a friend of a friend at this amazing spot in the Provence:

I’ve never been in a house before, on top of a hill, with a 360° view towards all the surrounding mountains. Here’s the side of the house where we stayed:

It was an incredible place to be.

The geek in me couldn’t resist peeking at the way electricity is hooked up there:

The black box at the bottom right is an earth leakage circuit breaker, which trips at… 350 mA! IOW, it doesn’t protect people, only the house wiring! – luckily, a modern 10 mA version has also been added.

Lots of personal stories, which I won’t bore you with. Been back for over a week now, but still mostly in vacation mode. I’m looking forward to going into lots of Jee Labs projects again, starting tomorrow…

A few more images from the country which gave us the phrase Joie de vivre and a lifestyle I truly love:

P.S. Liesbeth took all the pictures, roughly 600 in all – I just tagged along to relax and enjoy…

One year ago, the first serious PCB designs were “taped out” (heh, if that isn’t an anachronism by now!) – this is when the first batch of JeeNode v3 boards was produced, with all the ports and pins that have by now become a standard around here.

One year later, there are 4 JeeNode variants and over 20 “plugs” / add-ons – all part of a happy JeeFamily :)

What’s next? Well, I don’t have a crystal ball. But I do know what’s coming next because of some recent projects behind the scenes … and I can tell you that there will be several new plugs starting mid-August.

Another announcement I’d like to make now, is that after the summer more of the production will be out-sourced (here in the Netherlands), to free my time for work on new hardware and software development.

As you probably know, Jee Labs is just me, moi, and myself – with a few great people helping out behind the scenes. The major difference with traditional companies is that I’m neither driven by a boss, nor (primarily) by revenue, but by interest. Which means that you can have a considerably larger influence on where Jee Labs is going than you might think… all you need to do is speak up, preferably in the discussion forum, and point out neat / useful / practical stuff. I won’t guarantee that I’ll follow everyone’s lead, but I’m as keen as anyone to go where the neat stuff is regarding physical computing.

Speaking of neat stuff…

Franz Achatz sent in a great email today, describing what he’s been doing, complete with pictures and screen dumps. Here’s the latest addition to his RFM12B-based WSN – a fridge sensor (posted with permission):

All in a neat little box, with the GSM-type antenna sticking out:

The sensor is a 1-wire Dallas sensor, to allow tracking the current temperature inside the fridge.

And here’s the software side of it, all created by Franz with the current JeeMon software:

Given how young JeeMon currently is, I’m amazed to see just how much it can already be made to do…

The story I’d like you to take home from this is not how great JeeNodes or JeeMon are (they’re not, they are still far too young and simplistic), but how much freedom you have when everything is open source, hardware as well as software.

It’s time for me to start winding down (with 30..39°C of humid heat, it’s almost a necessity, even…). There will be one or two more queued-up posts on the weblog, and then it’ll be set to read-only mode. In fact, all of internet will become read-only a few days from now, as far as I’m concerned. I’ll be away only part of this summer, but even when I’m in I won’t respond to emails – sorry.

If you ever get bored, there are now 550 posts on this weblog – feel free to browse around, and enjoy :)

A couple of days ago an email titled “Au secours! LCD ne marche pas” came in, with this picture:

Two JeeNode tinkerers at the Computational Art department of the UdK in Berlin, looking sad and disappointed. And they guessed – correctly! – that sending an email from Germany, written in English, and with a French title would draw my attention :)

The email contained enough technical details to be able to resolve everything with a quick email reply from me (it was the brightness pot, which can be finecky!).

Then this came back:

Yeay – they look happy again! :)

(above pictures shown with permission)

Isn’t that what support is all about? It’s easy to forget that at the end of the day, support is for (and by) people, not technology.

One of the things I was totally ignorant about when starting out on the Jee Labs adventure, was the whole process of running an internet shop. The real physics and mechanics of it, not just some imagined “process”.

Of course it was clear from the outset that it would be about packaging and shipping stuff. But what does it come down to, on a day-to-day basis? Does it add much overhead? What do you need? Is it tedious / boring?

To start at the end: no, it’s actually fun! You get to give something to people. And a surprising number of names on the orders even come back once in a while, which tells me that someone, somewhere appreciated this and liked it enough to get even more JeeStuff. Which is very rewarding: I get to come up with stuff and make it, and then I get to “give” it to people all around the world (of course it’s a sale, but for me it still feels like giving).

So what happens after the obvious assembly of boards and packaging of kits, etc? Well, I pick all the required items, and put them in a padded envelope – with lots of sizes to pick from:

Little did I know about how much room all that “sealed air” takes up!

To keep shipping costs as low as possible, I try to always fit all the stuff into one envelope. In fact, I think I’ve only had to ship in a cardboard box once until now, although I do have some extra large envelopes (25x38x3 cm) for bigger workshop packages, etc. And there are limits to this type of frugal packaging, as someone pointed out:

So much for that Carrier Box. What did that postman do? Stand on it?

This is now solved by sending Carrier Boxes in slightly larger padded envelopes, btw. Apparently that gives just enough added cushioning to prevent this sort of damage.

Oh, and here’s a fun detail – check out the Jee Labs postal scale:

Seriously. That’s how each package here is weighed, to determine how much postage to apply. I can’t think of a nicer way to honor the makers of roughly a century ago. A timeless piece of engineering, with cast iron foot and all. Here’s a puzzle for you, if you haven’t seen such a postal scale before: try to figure out how it can have two ranges: 0..50 g and 0..250 g – it’s a very clever yet simple trick, as with all great inventions.

The next step is to add postage, which is now done with these state-of-the-art (ahem) digital stamps:

The convenience being that I’ve got a whole pile of them printed in advance, for each rate – instead of having to transfer up to 8 stamps.

And then it’s off to the mailbox: a 3-minute walk if I get everything done before 17:00 (5 pm), or a 10-minute walk to a more central mailbox which gets emptied at 19:00 (7 pm).

So there you have it – a peek into the Jee Labs kitchen, eh, I mean shipping department :)

I run a webserver for Jee Labs and a couple of other web sites and services.

I wanted to add support for logging the last hours of a new “#jeelabs” IRC channel. IRC might not be a good fit for me personally, but since several people have asked for it, I do want to support it. With say the last 3 days of that discussion logged as some public web-pages, I could drop by once in a while and try and add to the discussion going on at that time.

The “eggbot” system was suggested to me as a way to accomplish this:

Ok. Server is Debian, so “aptitude install eggdrop” should work, right? Debian is good at package management.

Wrong.

Turns out my server is still running Debian Etch, and “Etch is end-of-lifed”, as I just found out. The server ISP’s repository is gone. No updates, no installs, no security fixes, nothing.

Ok. Better upgrade Debian Etch to Debian Lenny first.

I’ve been there before, I know how to do it. Takes a few hours, but then it’s done. Server is running Lenny now, and all the main servers and services are still running fine (impressive!). This is a server in Germany, btw.

That was yesterday (well, ehm… night).

Today, quick check. Server ok. Weblog ok. Forum ok. Cool.

Oops. Ikiwiki isn’t working anymore. That’s the software I use to convert the Markdown pages into the Cafe web site. Fully scripted, based on Perl.

Ikiwiki wasn’t installed as Debian package, because the one in Debian was waaaay too old. Even on Lenny, it lags (by over a year – Debian Lenny still has 2.x, Ikiwiki has been at 3.x for some time now).

Ok, so let’s try to re-install it from source. By now, usually all my warning systems go into in high alert mode, because source installs are a different ball game (even with a “scripted” language such as Perl, how ironic!).

The Ikiwiki installation page is reassuringly short. Three steps. First two look pretty easy, right?

More than 10,000 lines of text have scrolled over my terminal window so far. This sort of drivel:

More than two hours have passed.

More than 180 Mb of disk space has been consumed.

I’m forced to watch closely, because disk space is running low on the root partition. And this is running as superuser, so it’ll bring down the server if it fills up the disk.

Pinch me, am I really in the year 2010?

My complaint is not that some things are complex and require a lot of work and understanding.

My complaint is that some things are NOT complex and yet you get treated to an incredible amount of nonsense. Not to mention the fact that the problem isn’t solved until you invest in figuring out every detail of lots of different kinds of packages.

The IRC logging? Not solved yet. I can probably write a 200 line script with not a single dependency on other code which does what I need: track the last 100 hours of an IRC channel disscussion as a web page.

The Ikiwiki setup for the Jee Labs Cafe? Broken. One week of work to write a Markdown parser and generate the static pages myself, would be my estimate. Probably well under 1000 lines of code, all in a single script. Three orders of magnitude less disk space, and easier to adjust to my needs.

I’m still waiting for the second step of the Perl install to complete. It’s running a “BigInt” package without proper library support, apparently. Falling back to “Math::BigInt::FastCalc”, and running tests which take ages:

Could someone help me understand where a static wiki page generator needs to do cryptographically secure factorization of large primes or something?

I’m stumped. Three hours waiting now. I give up.

I’ve aborted the (non-) install. With 220 Mb disk space gone, and no clue how to get it back. Oh yes, wait, it’s all in “~/.cpan” of course – how obvious. Thank you, Ikiwiki & Perl, for breaking down. Thank you, Eggdrop, for making simple things hard. And thank you Debian Linux, for not giving me the option to go back to my previous setup.

And you know what ticks me off? All of the above can be avoided. It has been solved.

But we’re living in medieval times, clearly. Most people haven’t even heard of Starkits. What a cruel joke.

Ah, wait. Maybe I’m the only one who runs into issues like these. Maybe everyone else has servers which just work, and to which you can add functionality without getting stuck. Doing everything you want. And a breeze to upgrade, of course.

That must be it. It’s just me. I have totally unrealistic expectations.

Apologies for the rant. Normal transmissions will resume tomorrow…

Update – thanks to a backport tip in the comments, Ikiwiki is now working again.

There are tons ideas floating around here at Jee Labs (thanks in no small part to a few people joining in on the brainstorming side of things). Never a dull moment!

One direction I’d like to explore a lot further, is coming up with kits which include a really nice enclosure. Even geeks sometimes like to show off their creations in such a way that not all the guts are spilling out, with wires everywhere, and everyone else scared of messing up… or even just touching something.

Because I like to get a real physical feel for things at times, I ordered a large set of ABS enclosures in a wide range of shapes, varieties, sizes, and colors:

See? That’s what “research” looks like! ;)

Now there are many ways to create enclosures, also with acrylic panels, aluminum plates, and PCBs. Even 3D printing is starting to become an option. Not to forget my favorite foam board, and more environmentally friendly materials such as cardboard and wood.

In many cases, it’s possible to fit a project + board into an enclosure after the fact – especially if the enclosure is made specifically for that purpose. And sometimes you just get lucky and find a case which turns out to fit existing boards really well with only a little extra tweaking:

But sometimes it’s easier to pick an enclosure first, and then see whether a proper board can be designed to create a perfect fit.

This is what the JeeLink did, and it ended up becoming a really practical gadget, IMO:

Anyway, with this new “box full of boxes” at Jee Labs, I hope to be able to design more kits in the future which are not only fun to build, but also end up looking good and practical for daily use in and around the house. We’ll see.

“Making stuff fit” is often a neat puzzle. But it’s also easy to overlook details – always the crucial ones, of course.

This post is a question, but maybe I’m already answering it by posing the question in the first place …

The question is: what to do with the Jee Labs source code?

As you may remember, I went through a few mad gyrations recently w.r.t. setting up a community site using TikiWiki, and then quickly abandoning it again.

In the end, I’m glad I didn’t buy into TikiWiki. Too many compromises, and for my particular needs, far fewer benefits than I had originally hoped to see.

The current Jee Labs Café that has replaced it is very crude, visually, because I haven’t yet done the work to bring over the layout and style sheets from this weblog. But far more importantly, IMO, is that the information is finally organized in a sensible way – until someone tells me otherwise, anyway!

The pages are maintained by subversion (svn), a version control system which is well-known and widely used by software developers (as are the older “cvs” and the newer “git”), although I do find that a surprising number of people are not used to the whole idea of “source code control systems” and “code repositories”. Frankly, I couldn’t live without something like svn anymore, these days. It’s the only way to develop software and not lose your mind w.r.t. change management and long term maintenance – IMO.

Maybe I should go into the features and benefits of such an approach in another post.

My point here is that the entire Café website is managed by svn. This means that all older versions of pages, sources, documents, pictures, etc. are available. At least as importantly in this context, is that the pages can be edited by anyone with write access to the repository. It’s pretty simple: get a copy (“check out”) of the website, make changes, and write it back to the repository (“check in”). Writing is only allowed to those with a user name and password in the repository, so this is wiki-like, but not publicly writeable.

The interesting tidbit about the Café website is that all changes are automatically published on the web. I’m using ikiwiki as tool to take care of all that. Thats why all the pages for the Café can be maintained as simple Markdown text pages, and how the result ends up as HTML, with links and embedded images. I’m only using a very small subset of ikiwiki’s features, btw.

Another great side-effect of using ikiwiki is that the entire Café website consists of static web pages. No server load, inifinite scalability for free.

In practice, it’s all working out really well. I can easily update, extend, and maintain the Café website this way. And if anyone wants to participate and contribute more content for the wiki (one can always hope, eh?) then that would be very easy to fit in, since subversion can be used over the internet.

As far as I’m concerned, the daily weblog and the Café / wiki are in good shape for a long time to come.

There are two other areas. One is the Talk discussion forum, which uses bbPress. I’m not terribly excited by its functionality, nor by its (lack of) progress, but it’s holding out fine, sooo… if it ain’t broken, don’t fix it.

The fourth area of public Jee Labs activity is the source code being developed for use with JeeNodes and JeeMon. It’s been in my subversion repository for a while, then I moved it to Google Code, en then I moved it back to my subversion repository again, which is where it has been for many months now.

As fas as the repository goes, this way of managing source code works well, at least for me. But there are some sharp edges. For one, the web-browsing interface is based on ViewVC – which is a bit quirky, and which turns out to be CPU resource hog on my server. Probably just web-crawlers, constantly triggering CGI execution of the ViewVC Python scripts.

Also, I currently have no issue tracking system in place. Would be nice to get something going one day.

I’ve been looking into GitHub, which seems to attract a lot of attention (and open source projects), these days. It has git-based code repositories, with an optional svn interface, and it offers a wiki, issue tracker, and download area. Reminds me a bit of SourceForge, which seems to have fallen out of grace, these days.

My main concern with GitHub is that it’s yet another big central site, offering a mix of features and functions to attract as many developers as it can, with no guarantees that it won’t be gone a few years from now, or start doing “targeted” ads, or selling email addresses, or other nasty stuff.

There is, however, one tempting option out there: GitHub + Jekyll – Jekyll is similar to ikiwiki, in that it takes text files (including Markdown, yeay!) and generates a website from it. Jekyll even offers a migration path for WordPress sites. The interesting bit is that GitHub supports Jekyll out of the box, and that it can be set up to work with custom DNS domains. IOW, I could merge this daily weblog, the Café, and the code repository into a single consistent system. The result could be one or two static websites (hm, I guess that rules out blog comments), fully under version control, and using the Markdown format I’ve already standardized on. Needs more thought!

As it currently stands, I think I’m going to stick with WordPress + subversion, which is rock solid and has some really nice software tools for it (like Versions on the Mac). Maybe just integrate the source code areas with the Café, and use ikiwiki to allow browsing the latest version on the web, right next to the supporting doocumentation.

It feels a bit like “going it alone” by placing everything in my own subversion repository and maintaining it all on a privately managed server, but all I can say is that a lot of what I’ve done in the past has been around a lot longer – and still is! – than some of those fads…

This is weblog post number …

Yes, five hundred!

If you’ve been following along, you know what I do, and why. And my views on OSH and OSS.

My motivation for the daily weblog format comes from a guy called Seth Godin, who – surprise! – writes a daily blog (for many years now). I find his never-ending stream of insights absolutely delightful and inspiring.

So what does it take to write about something I care about, day in, day out? Surprisingly little. The trick is to stop chasing quick results. And to stop chasing big results. The drive comes from within. The challenge comes from the problem. The goal is to understand and to solve. You start with a puzzle, you end up with learning something new. The journey is the reward, to quote Steve Jobs – something I profoundly agree with.

This weblog isn’t a race. To the top, more readers, fame, success, fortune, or even to get the most posts in. This weblog is a dedication, to those who explore and invent, and to those who teach and inspire. Day in, day out.

It’s a lifetime thing.

Check out the following story…

Driveby culture and the endless search for wow – by Seth Godin, March 2010

The net has spawned two new ways to create and consume culture.

The first is the wide-open door for amateurs to create. This is blogging and online art, wikipedia and the maker movement. These guys get a lot of press, and deservedly so, because they’re changing everything.

The second, though, is distracting and ultimately a waste. We’re creating a culture of clickers, stumblers and jaded spectators who decide in the space of a moment whether to watch and participate (or not).

Imagine if people went to the theatre or the movies and stood up and walked out after the first six seconds. Imagine if people went to the senior prom and bailed on their date three seconds after the car pulled away from the curb.

The majority of people who sign up for a new online service rarely or never use it. The majority of YouTube videos are watched for just a few seconds. Chatroulette institutionalizes the glance and click mentality. I’m guessing that more than half the people who started reading this post never finished it.

This is all easy to measure. And it drives people with something to accomplish crazy, because they want visits to go up, clicks to go up, eyeballs to go up.

Should I write blog posts that increase my traffic or that help change the way (a few) people think?

Should a charity focus on instant donations by texting from a million people or is it better to seek dedicated attention and support from a few who understand the mission and are there for the long haul?

More and more often, we’re seeing products and services coming to market designed to appeal to the momentary attention of the clickers. The Huffington Post has downgraded itself, pushing thoughtful stories down the page in exchange for linkbait and sensational celebrity riffs. This strategy gets page views, but does it generate thought or change?

If you create (or market) should you be chasing the people who click and leave? Or is it like trying to turn a cheetah into a house pet? Is manipulating the high-voltage attention stream of millions of caffeinated web surfers a viable long-term strategy?

Mass marketing used to be able to have it both ways. Money bought you audience. Now, all that buys you a mass market is wow and speed. Wow keeps getting harder and dives for the lowest common denominator at the same time.

Time magazine started manipulating the cover and then the contents in order to boost newsstand sales. They may have found a short-term solution, but the magazine is doomed precisely because the people they are pandering to don’t really pay attention and aren’t attractive to advertisers.

My fear is that the endless search for wow further coarsens our culture at the same time it encourages marketers to get ever more shallow. That’s where the first trend comes in… the artists, idea merchants and marketers that are having the most success are ignoring those that would rubberneck and drive on, focusing instead on cadres of fans that matter. Fans that will give permission, fans that will return tomorrow, fans that will spread the word to others that can also take action.

Culture has been getting faster and shallower for hundreds of years, and I’m not the first crusty pundit to decry the demise of thoughtful inquiry and deep experiences. The interesting question here, though, is not how fast is too fast, but what works? What works to change mindsets, to spread important ideas and to create an audience for work that matters? What’s worth your effort and investment as a marketer or creator?

The difference this time is that driveby culture is both fast and free. When there’s no commitment of money or time in the interaction, can change or commerce really happen? Just because you can measure eyeballs and pageviews doesn’t mean you should.

In the race between ‘who’ and ‘how many’, who usually wins–if action is your goal. Find the right people, those that are willing to listen to what you have to say, and ignore the masses that are just going to race on, unchanged.

Yesterday’s post was an attempt to explain what I’m doing, and how the bigger issues cause me to wander around a lot, working on secondary projects while trying not to stray too far from the main direction – which is to experiment with fun stuff in the home, around the topics of energy use and environmental monitoring. And a whiff of domotics… when it serves a useful purpose.

Ok, so Jee Labs is about JC’s Environmental Electronics. Doh.

Today I’d like to go into why I’m working on this stuff.

Whenever you ask people why they do what they do, the usual answers are: money, prestige, influence. But the most exciting answers in my book are from those who chase their dreams: because they can or because they want to see where it leads to. Fortunately, these answers do come up, once in a while.

Here are some “why” answers from me:

Why environmental? Because we’re on a dangerous course. I’m ashamed of what my species is doing, yet I share full responsibility. Unfortunately, I don’t know how to change the rest of the world. Them. Out there. But maybe I can change the small world I live in. Me, my family and friends. My living space.

Why electronics? Because it’s what I loved doing when I was a teenager. It was my biggest passion, before computers took over that spot. I would love nothing more than share that passion. If I can somehow reach some kid, somewhere, to discover the magic of exploration and invention, then that would be fantastic.

Why microcontrollers? Because they bring together everything I like: electronics, logic, code, mechanical design. And because nowadays, they are so low-cost and so darn easy to work with. Incredibly robust (hey, you can plug ‘em in backwards!) yet infinitely malleable (its all code, just change the flash memory!).

Why wireless? Because wireless is as close to magic as technology will ever get. Making things happen somewhere else with invisible power, literally!

Why sensors? Because it’s about time our technology started paying more attention to the “real” world out there. Out with the big and noisy machines, which operate in a strictly controlled fashion. The future belongs to sentient systems, which fit in, investigate, respect, respond to, take care of, and even protect our most valued aspects of life.

Why networks? Because this world is about information. Data which does not reach the right places and persons, has no value.

Why the home? Because that’s where people live. Factories, offices, and commutes are all artifacts of the industrial revolution. That was long ago. We’re living in the internet revolution now. Being in a specific place to make something happen is losing its grip on our lives.

Ok, so maybe that last one is pushing things a bit … :)

Now some more focused why’s …

Why JeeNodes? Because Arduino’s got it almost right, but shields are simply not modular enough to encourage real mix-and-match tinkering. Single-purpose shields are made for consumption and they restrict needlessly (try stacking them, and feel the pain!).

Why Ports and Plugs? Same reason, really. Because I want everyone to be able to experiment with combinations of sensor and actuator functions. JeeNodes are not about consuming (“I create a neat combination kit with all sorts of choices fixed in advance and you build a copy of it”) but about de-constructing and re-constructing stuff. Analyze and synthesize. Take it apart, combine it in other ways. Go and try something new, please!

Why RFM12B’s? Because they are low cost and more than capable enough. Mesh, frequency hopping, TDMA, sure… if you want to dabble in complexity, go for it. Go swim in network protocol “stacks”. Add in a micro-kernel to deal with all the required parallelism. Go overboard in failure modes and recovery mechanisms. Use beefier chips. But count me out. I can live with imperfect packet delivery, and simple manual configuration of a few dozen nodes. I cheerfully pass w.r.t. all this “self-inflicted complexity”.

Why 3.3V? Because more and more of the new and interesting sensors operate only up to 3.6V or so. And wireless chips, and Ethernet chips. And because LiPo batteries are very good power sources: very low self-discharge, very fast recharge times, and available in a huge range of sizes and capacities.

Why JeeMon? Because I want the software equivalent of a breadboard to explore lots and lots of ideas, and it doesn’t exist – not equally simple and equally powerful as a breadboard, not to my knowledge anyway. I think we haven’t even scratched the surface of software design yet, and the potential for real modularity and simplification. Hardware is much further along, in that respect.

Why Tcl? Because there seems to be nothing quite like it, in terms of simplicity, expressive power, flexibility, robustness, portability, scalability, and deployment. I don’t mean in terms of each individual issue, but in terms of the combination of those aspects. As a package deal, Tcl embodies a surprisingly clever and effective set of trade-offs. I could probably dismiss Tcl on every single issue in isolation, and name another language which would be preferable – but no single language can go where Tcl goes.

Why multi-platform? Because I want to create interesting solutions on the desktop as well as on small Linux boards. I’m fascinated by the idea of moving solutions around, modularizing larger systems into loosely coupled sub-systems, and migrating some of those pieces to dedicated miniature hardware platforms.

Why open source? Because it simplifies my life – I can work in the open, share and discuss everything, and benefit from every mode of collaboration imaginable. And because it simplifies your life. If you don’t like what I do, you have three options, instead of just the first one: 1) ignore me, 2) take what you like and
change everything else, and 3) make your case and bring up arguments to steer me into a better direction. I’m against lock-in, so if there’s anything I can do to further reduce inter-independencies, let me know.

Why no standards, such as XML or ZigBee? Because in this context, standards make no sense. The context is an environment where you can choose every data structure and every side of the communication. In a world where everyone speaks a different language, you need dictionaries, translators, and interpreters. They are all essential, useful, and valuable. I should know, I speak 4 languages, 3 of them regularly within my own family (the fourth being English). But the compactness of well-chosen words and the intricacy of their nuances really take off when you’re totally on the same wavelength. XML has many virtues, but “impedance matching” and compactness are not amongst them. Standards stand in the way of creativity. XML and ZigBee add no value in this context, just tons of complexity, which then creates its own set of problems and distractions.

Speaking of complexity…

This post is starting to become a little too complex as well. So let me summarize and simplfy it as follows: why am I doing all this stuff at Jee Labs? To share my excitement, to convince those interested in technology that there are infinitely many fascinating adventures ahead in the land of Physical Computing, to give me an interesting and useful context to try out lots of new software ideas, and … for the sheer fun of hacking around and learning.

Oh, and because I can, and want to see where it leads to, of course :)

PS. The “normal” weblog posts will resume tomorrow, i.e. how to set up JeeMon for the reflow project.

If you’ve been following this daily weblog for a while, you may have noticed that it’s all over the place – as if I’m working on everything and nothing. At the same time!

But there’s a pattern, dear reader.

Not fully worked out, not fully planned, not static, but still… there is.

The pattern is that I’m currently trying to automate some of the stuff I need to do here to keep Jee Labs running. The shop has been growing steadily, which is great because it means I can keep doing this – which is exactly what I want to do. Indefinitely, preferably.

But the shop needs to run smoothly, so that I don’t end up becoming its slave. That means the daily production work needs to be automated as much as possible. Only then will it be possible for me to work on all sorts of fun projects, keep up this daily weblog, and fill it with – hopefully – interesting topics, day in day out.

This “self-automation” is why I created the Flash Board, for example. And why I’m redoing the reflow controller to work more reliably. I’ve also been automating like crazy recently to try and stay on top of this huge pile of parts called (haha!) inventory.

I have not lost track of the main focus of Jee Labs – the direction where it all started: energy use tracking and environmental monitoring around the house. It’s still my main focus. And now that the basic hardware works, with lots of configurations and sensors to play with, the next frontier is the software.

But software is a very finicky beast. With hardware, you hook up a few parts and it starts working – after some soldering and/or tinkering, evidently. Software is both primitive and complex in comparison: primitive, in the sense that you have to create these big house-of-cards constructions to get anywhere. Complex, because each of the ingredients is usually a massive chunk of code.

So I’m taking a lot of time to think through numerous aspects of the software. JeeMon is the house I’m building, but its core structure needs to support ideas which haven’t even been invented yet. In terms of software, that means it has to be very modular. I’ve currently got a few components in place, and the binding structure and modularity trade-offs are starting to become clear.

What I don’t have right now, is a clear enough view on the data storage side of things. So all the little JeeMon experiments so far have been side-stepping the issue of persistence (the IT word for “storage”). I’m showing things on screen. Great, but of limited utility.

What’s the big deal with persistence? Well, the moment software includes a storage mechanism, you get into the issue of how to make changes. Suppose you have a working system, and you want to change it in some fundamental way because of a new insight. How do you deal with the data it has already stored? It sounds trivial, but I think it’s everything but trivial – on a very fundamental level.

Storage is a big deal. It is crucial. And it comes up even with something as simple as displaying a moving average. How do you deal with a system restart when there are moving averages in graphs you want to show, for example?

Software development and persistence create opposing forces. Development means just that: to progress from one insight to the next as you go along and extend your understanding of the deeper issues in the problem space at hand. And then, ideally, to implement solutions in better – sometimes completely new – ways. As a developer, I constantly tear my software apart, to put it back together in improved ways (I probably do it 10x more often than most developers). This is a learning process, and the result – IMO – leads to simplicity, elegance, and almost as a side effect: robustness.

Persistence is the elephant in the room. It opposes change. Data saved on disk (or flash memory) has a structure, and changing that structure can be anything from awkward to nearly impossible. That’s why starting from scratch is so easy. That’s also why version 2 of anything in software can be so elusive. It’s not just data, btw – code is data too, in this context. Try folding a new idea into an existing bit of code …

Maybe I’m overstressing this a bit – m a y b e. But this is the main reason why I work on completely unrelated issues at times, such as streamlining the shop activities.

In the meantime, as background process, I keep exploring scenarios for the software and collecting insights from what others do or even just ask for.

So there you have it: ISP programmers, solder reflow controllers, even tangential activities such as 3D printing and CNC milling, they all get addressed here at Jee Labs. Meanwhile, I try to figure out the best way forward for many far-reaching design choices w.r.t. JeeMon.

The good news is that I think this detour is coming to an end. I think a simple, small, modular, and fun way of tying all sorts of hardware together via software is starting to shape up – in a vague hand-waving style right now, but that’s just a matter of pushing the ideas into code. And doing the grunt work.

Neither the ideas nor the code are the hard part. Ideas are cheap and plentiful. Code is easy and can be created gradually. No, the really hard part is to come up with a “pervasively modular” architecture. I.e. how to set up a system (and hence make choices) which can evolve, even in ways not yet imagined.

What I’m after is not a solution, but a tool. And that takes a bit more time…

As the well-known Chinese proverb goes:

I’m not a teacher, but the fish doesn’t interest me nearly as much as improving the whole process of fishing.

Not so long ago, junk comments on the Jee Labs daily weblog started rising sharply:

Flagged by Akismet, but still a pain because I have to clean them up once in a while – and manually skim over each one to pick out any false positives. No fun, especially knowing that all this spamming is scripted – regardless how little effort it takes, life’s too short for this sort of nonsense!

Fortunately, there are additional tools for WordPress to prevent most of this junk from even reaching Akismet (which does a terrific job, btw).

As you can see at the end of the graph, the spam log is clean again. Five incoming junk comments in two weeks – that, I can deal with :)

It is now possible to print postage stamps with a label printer, instead of having to constantly buy stamps and place up to 8 (!) of them on the packages going out at Jee Labs. Not to mention keeping 4 different stamps around.

Until now, the only other option was to buy a fairly expensive franking machine which weighs the packages and prints stamps on them, but it doesn’t handle thicker packages, so I’d end up printing on separate labels anyway – and essentially waste most of the investment while still doing the same as before.

I’ll probably keep using stamps for the small shipments. But no more reams of stamps per envelope, new shipping options in reach, and no more running out of specific stamps!

So far so good, I think yesterday’s community site setup is working out fairly well!

Here’s one of the pages, as seen by anonymous visitors:

The same page, once you pick a username and register:

As you can see, the site can be extended to include more features for registered users (there are lots more!).

TikiWiki has 12,723,940 options – I’ve only tweaked 83 of ‘em so far…

In fact, I’ve been turning off features, stripping menus, changing permissions, and simplifying the CSS theme, to try and regain some uncluttered screen space. I may have gone too far in some areas, but there’s definitely room for further simplification in others.

There are built-in anonymous, registered, editor, and admin “groups”, so we can make this thing as open as possible w.r.t. participation, while sharing the maintenance burden later on if this site becomes hyperactive.

I still find web-based page entry and editing extremely tedious. But I’m willing to stick to this wiki formatting convention, and I can easily use my own text editor to prepare new pages off-line when it gets too painful.

I’ve moved the site to its definitive cafe.jeelabs.net URL (www.jeelabs.net and jeelabs.net will also work).

The talk.jeelabs.net forum and the wiki/jeelabs.net wiki will be kept around in read-only mode for a month or so, until all the main content has been moved over.

This new setup is hosted on the Jee Labs server again. No user signup in some big wiki farm, no tracking of activity (I don’t do big-brother style “analytics” – not now and not ever…).

The interesting aspect of TikiWIki is that it is even more powerful than Wikidot, but that almost each individual feature can be turned off, so that I can pick exactly what’s needed to create a convenient site, without all the bells and whistles which would only distract from creating useful content and discussing important topics. The flip side is that it’s quite a large set of config options to navigate through!

I’ve replaced the Wikidot with this new site, again at test.jeelabs.org – you are hereby invited to sign up and try it out (all user names are available, since this is a standalone setup). I can assure you that it is an order of magnitude more tedious for me as it probably is for you to have to go through all this hassle again…

As before – this is a test. Final decision as to whether this becomes the new Jee Labs community site will be made within the next 10 days … I sure hope this is it!

After having played around a bit with the recently installed Forum + Wiki site, I’m having some serious doubts about switching over to it. Here’s the Sandbox page, as example:

Looks great, right? So what’s the problem?

It’s too powerful and it’s too complex.

The #1 feature which drew me to Wikidot, is its unification of a discussion forum with a wiki. The forum is very good: hardly any setup needed, good threaded discussions, good notification options, and good admin interface. Perfectly usable, right out of the box. As I described here, the combination of a timeline-oriented forum and a project / topic oriented wiki sounds like the pefect way to merge the two major areas one would like to have on a community-based site.

But the wiki, if you pardon my French, sucks…

It’s an attempt to shoehorn formatting options, style elements, and content management features into a wiki-like syntax which just doesn’t make sense. Tables, tabbed views, modules, footnotes, formulas, … the list doesn’t end, but the confusion just increases as you go along. There are numerous ways to create links, yet I can’t make internal and external links appear differently on the page. The way to change a link’s title depends on the type of link used. Some features are modules, others are built-in. I don’t know about others, but as I’ve been trying out a few things, I’m not getting more and more used to the syntax – which to me is a sign that it lacks regularity.

In terms of site structuring features, there is too much functionality: sections w/ table-of-content, tags, categories, and parent-child page relationships. The problem with this is that every contributor will choose a different mechanism when adding new pages. Some people will put everything into a single page, others will try to create “islands of personal hierarchies”, twisting the available features to match a personal preference for specific pages. I don’t fancy spending my time editing pages and adjusting structures to try and unify things. And I’m only going to end up stepping on everyone’s toes if I change something they spent ages perfecting to look “just right”.

Don’t get me wrong. There is a lot to like about Wikidot.

But for an open community site, which aims to help lots of people find their way around, while letting active members share and contribute highly informative nuggets of knowledge(link broken as of 2010-04-05), I think the Wikidot format has (far!) too many degrees of freedom. Style should be pleasing (which is evidently a personal preference) but most of all it should be uniform. This daily weblog gets information across because its style was defined long ago and has become unimportant – both for me as writer, and for its readers, judging by the friendly emails I keep receiving.

A drawback which bothers me more than I expected: posting on the forums requires signing up with Wikidot.

The other thing I found out is that browser-based editing is tedious. There’s not much Wikidot can do about that (it does explain why I prefer email-based discussions over web forums).

I’m not quite happy with that setup. The PROJECTS, DOCS, and CODE links point to stuff I’m working on and sharing through this site, but there’s a bit of a disconnect with what others are doing. It’d be nice to be able to maintain a list with other projects derived from (or perhaps just inspired by) the ones at Jee Labs. Likewise, the code I’m kicking into the world is only part of the story – what about changes and extensions by others? Or sample code? Wouldn’t it be nice to have everything available in a single place?

And then there’s the somewhat confusing separation between the TALK discussion forums and the WIKI pages. Both of them intended to help people share and exchange articles, code, images, etc.

Lastly, TALK is of course about asking questions, but wouldn’t it be nice to tie in much more to the pages to which these questions apply, and to have the answers easily reachable from those same pages?

I’d like to try something new. Better get it over with early on, and benefit as soon as possible from an improved community exchange site.

Meet the new experimental Forum + Wiki site:

This is a unified forum + wiki setup. That concept in itself risks being so confusing that I’ve decided to start a Guidelines page to describe what I’m trying to do.

Will it work? I don’t know. Let’s review this a few weeks from now …

The trouble with a site change like this is that it’s going to be disruptive. Especially since I’m not yet 100% sold on adopting the above site and throwing out the current forum and wiki.

The current plan is to proceed as follows:

Allow everyone to explore and try out the new site, which is now public at http://test.jeelabs.org/

Keep the current forum and wiki links on this weblog intact for another week or two.

If the new test site is deemed effective, make it the default one.

Create read-only archived versions of the forum and wiki for reference.

Enjoy the new site, make it rock, and move on!

I hope this works out. Let me know what you think, especially if there is any show-stopper or other issue!

I ran out of little zip-lock bags. The ones I put small 6-pin headers in, and such.

So I ordered a few more by direct mail, from a Dutch office supplies shop on the web.

Sure enough, one day later – a big courier delivery truck from DHL stops by and delivers a 25×35 cm box with the requested goods:

They could have dropped it in a padded envelope, added €1.60 postage, and dropped it in the mail. Which – in the Netherlands – is guaranteed to reach me in 24 hours, IOW just as quickly, along with all my other mail. Delivered to my doorstep by our friendly mailman, who services the whole neighborhood … on bike.

Update – I’ve re-ordered some more (slightly larger) bags from another Dutch supplier suggested by someone after reading this post. Not only did that supplier do the right thing – they even alerted me to the fact that another item would have pushed postage up to an unreasonable level, and proposed to omit that extra item. Hurray for vendors who use their common sense when serving their customers!