Welcome to ned
Productions (non-commercial personal website, for commercial company see ned Productions
Limited). Please choose an item you are interested in on the left
hand side, or continue down for Niall's virtual diary.

Niall's virtual diary:

If you have any comments/questions/criticism of my virtual diary,
you can email me at the address at the bottom of the page.

For a deep, meaningful moment, watch this dialogue
(needs a video player) or for something which plays with your
perception, check out this picture. Try moving your eyes around - are
those circles rotating???

It's rare to get all five cygnets in the same physical location as their parents nowadays. They tend to prefer being off on their own somewhere nearby. But this morning they were all napping until I disturbed them for this photo. If it's any consolation, I was quick, they were back to napping half a minute later.

I'll be speaking at the C++ users group Dublin this coming Monday 17th September. Entire evening will be just me, 90 minutes, no other speakers. I just finished the slides for the talk just there, it shouldn't be too boring for folk hopefully. And it's the only talk I'll give in all of 2018, usually I speak twice a year at conferences, but this year has been unusually quiet for me. Too busy with standards paper writing!

Cold this morning. But pretty. I do wish they'd turn on the heat in the seminary ...

Monday 10th September 2018: 7.29pm. I know I've said it here many times before, but I'll say it again that somebody really needs to make the Lensman series of books into a TV series or something. It's the great grand daddy of most contemporary sci fi. Witness this:

Galactic Patrol (1937): Big space battle with defensive shields, beam weapons, maulers, torpedoes, tractor beams, escape pods. Hero uses his cunning and Jedi Lens powers to repeatedly evade capture by utterly evil multi-galaxy empire across multiple planets, each with its own unique species, governments and terrain. After successfully stealing plans for the enemy's much more advanced space drive and returning to the Jedi council Patrol, thus saving Civilisation from annihilation, he is released from command service and allowed to roam free to work in any fashion he chooses. He uses this to master mind control of those weak of mind, learned from advanced training by an elder race who birthed all the younger races and who manufactures an individual Lens for everyone elite enough to get into the Patrol.

(Yes, Star Wars and Babylon 5 borrowed heavily from Lensman)

Gray Lensman (1939): Our hero pays a visit to the second galaxy where the evil empire comes from, and saves a civilisation under attack who have the technology to transport entire planets (they transport theirs to our galaxy). Using said new technology, planet weapons get invented (just fire small planets at the enemy), as well as black hole weapons (note black holes hadn't been invented in physics by 1939). The enemy meanwhile deploys wormhole drive capable of traversing huge distances in an instant (yes, that's warp speed, again unknown to physics in 1939). Oh, and our hero considerably furthers his powers, and is joined in his level of abilities by members of several other very alien races, including one which are nebulous gas clouds (who consistently get the best and funniest lines throughout the series).

(Yes, the multicultural everybody-is-equal-but-different liberal consensus based Federation of Star Trek also came from Lensman, in fact a lot of Star Trek is clearly lifted from Lensman. Quite a few of the side plots read exactly like a classic Star Trek episode)

Second Stage Lensman (1941): Evil empire, whose primary planets were destroyed end of the last book by black hole and planet weapons and a huge Patrol armada, drops a huge fleet by wormhole into the Earth system along with its own planet weapons in order to take revenge. Luckily our hero and friends have just in time deployed a sunbeam weapon which redirects the entire output of our sun at stuff (yes, a full fat stellar converter weapon, a Death Star!), and the entire invading fleet of tens of thousands of monster attack ships, plus planet weapons and all, is wiped out. Realising knocking out evil empire's primary planets wasn't enough, our band of Jedi Lens heroes set to work figuring out who is the ancient advanced race behind the evil empire, the implacable foe of the ancient advanced race sponsoring Civilisation. Where, conveniently, both ancient races don't fight each other directly, but via the younger races they sponsor. Yes, that's basically Babylon 5 again.

I'll stop there, as that's how far I'm currently into the books, though I have read them before. One is just stunned at how transformational these books were for sci fi. Before them, is classic Jules Verne type sci fi. Then these came along, and defined all the sci fi tropes you've endlessly seen since e.g. the gruff, big strong guy big into fighting; the geeky scientist type big into analysis and numbers; women being saved, women doing the saving. Literally you could fill multiple pages with box ticking. And it all started with the Lensman.

Now, I'd remind anyone tempted to go read them that the prose is truly awful, everybody speaks like a 1930's gangster, everybody of course smokes and drinks constantly, and the language is quite archaic and not politically correct by today's standards. There is also a strong dose of eugenics and race purity in there, as was the zeitgeist. That said, for the time they were written, they are relatively colour blind, and women are not done down relative to men, but it is a very old fashioned form of feminism. Like the kind typical in society by the 1960s. However if you can push through all of that, they're definitely worth reading. And an eye opener.

There was to have been a book after the final one Children of the Lens, but it was felt unpublishable at that time. By the end of that book, the children have become a new species, more like gods in terms of power and remoteness from humans, or indeed all other life. Not so easy to publish after World War 2. Plus apparently there was a lot of incest amongst the new species as they multiplied themselves into a new elder race to shepherd the lesser lifeforms in the universe after the two ancient elder races departed beyond the rim. But we don't know for sure, it was never published.

Sunday 9th September 2018: 4.04pm. For the last four weekends, been ever so slowly building a new house server inside a VM running on the old house server. Got so close this weekend to switching it over, but just fell short. I've set up a rsync cronjob to keep old server mirrored to new server, so as soon as I get home next weekend, hopefully I can finally migrate as the old server is increasingly crashing out while I am away which means everything stops until I get home to raise it from the dead again. It's not the hardware, the hypervisor remains running, it's the VMs, some of which were installed in 2013 just when Clara was born, and have been major version upgraded which is always a bad idea. They all need to be swept out and replaced with latest across the board, should restore stability.

The linked paper which has been submitted to the C and C++ standards committees, which I spent four months doing stakeholder dialogue to arrive at in its proposed form, if accepted, could utterly transform how failure handling is implemented for all computing systems in the world. Yes, I either go big or go home. If it actually happens though ... wow, I actually would have changed the world in perpetuity!

Gorgeous if cold morning. You can see how low the sun is getting, long shadows in the mornings now, won't be long before autumn. I wonder when cygnets turn into swans? Apparently they must learn to fly before winter, else they get abandoned by their parents. Which is harsh.

Sunday 2nd September 2018: 1.14am. Rereading the Lensman series of SciFi books, they are the progenitor of all modern SciFi. In the second book our hero has just mastered mind control as he explores his Jedi Lens powers, used it to infiltrate an enemy base, and convert half its people from the dark to the light side. No, Star Wars didn't invent that. It borrowed it from the Lensman books written before the second world war. Anyway that's a truly great SciFi series, the sheer scope and scale. Terrible terrible prose mind you. But definitely enjoying the second book a ton!

Remember in http://www.nedprod.com/Niall_stuff/vdiary/archives/1535287474 when I estimated that the 8Tb shingled drive copy would take 7.52 days to complete? Well, it's been running 5.8 days now, and has completed 79%. On that basis it should complete after 7.34 days, not 7.52. Still, that's just over a week. Boy are these shingled drives slow for writes!

Wednesday 29th August 2018: 6.45pm. Tesco Mobile Ireland just flipped on 4G for me free of charge, which was nice of them. Speed difference:

Tescos had some speciality melons this week costing a fortune. Going to try them this week, take a risk. Right most melon is a standard European Galia melon for comparison.

Sunday 26th August 2018: 12.44pm. I've been very slowly, bit by bit, doing a little bit of work when home each weekend to replace the main house server, whose install of Debian EOLed last June.

Right now my ZFS pool is kept within a VM running FreeNAS, as back when I set this up I didn't trust ZFS on Linux (for good reason). Now I do, so I've set up a mirrored ZFS pair of those cheap shingled SMR 8Tb Seagate drives, and I set the old pool rsyncing to the new pool last night.

Those shingled Seagate drives have a 20Gb PMR acceleration cache, and writes of up to that much go quickly, indeed for random 4Kb i/o these drives have amazing benchmarks for spinning rust, because the drive ignores the randomness and just writes your random i/o into sequential blocks in the PMR cache. So, perversely, for small bursts of random writes, these drives outperform any spinning rust drive out there by orders of magnitude.

However, once the PMR cache fills, then writes must block on SMR, which is dog slow. I knew from conveyancing tests for each of these drives that we are really talking slow here, it takes nearly a week to run badblocks on them. But I had no idea until now how slow ZFS would be on these, or more specifically, if one of these 8Tb drives fails, how long does it take to resilver a replacement?

I can tell you now that ZFS is writing to each of these SMR drives at about 11.3Mb/sec! So it'll take approximately 7.52 days to resilver one of these 8Tb drives. That's probably an overestimate, my new server is running inside a VM on the current server, and rsync is running over two virtualised network interfaces, so there's going to be overhead there. Still though, that's an order of magnitude slower than a PMR drive. If the single remaining drive failed during resilver, game over for all your data.

Are SMR 8Tb drives therefore worth it? If you buy the Seagate SMR drive in its Backup Hub Pro USB case, it costs €170 or so (just extract the drive from its case, it's far cheaper than buying these drives loose). The next cheapest 8Tb PMR drive I could find is a Western Digital Purple which is currently selling for €240, which is 40% more expensive.

I am half tempted to triple mirror these SMR drives, that reduces chance of pool loss to nothing. But I am also minded that Linux lets me stitch together drives easily, so I can create a faked 8Tb drive from 3x 3Tb drives which I have spare, and thus guard against failure during a SMR drive resilver. In other words, I can buy that week through more work by me at the time. So, for now, onwards and upwards!

Been trying this oat derived milk alternative after I read about it in the Economist. I can't stand the soya or almond based stuff, undrinkable. This stuff, however, is acceptable, even in coffee as well as cereal. Real milk is still better, and about a quarter the cost, but this stuff is okay.

Successfully made it through the maze! Second picture we are on top of the castle in the middle.

Thursday 26th July 2018: 7.53am. I did something to my knee two weeks ago and it's being very slow to heal. Other knee is getting sore now too. I suspect it's that 3kg I gained a month or two ago during conference and standards committee time. With all the daily walking I now do, and being older, an extra 4% of weight wreaks disproportionate havoc. This sucks because now I must lose 3kg, which means starving myself next month or so. Sigh.

Decided to fire the starting pistol on the next round of hard drive upgrades for my ZFS array, I've got 800Gb still free which won't be filled for over a year yet, but it takes me at least three months to buy drives as I need to stagger their purchase over a long time to ensure I never buy two drives from the same production batch, plus there is a non-trivial soak testing period after arrival to ensure the postman didn't drop them too hard. Plus I can see that the 8Gb Seagate ST8000DM004, which is very reliable according to https://www.backblaze.com/blog/hard-drive-stats-for-q1-2018/, has taken a recent price tumble under 200 euro which suggests it is finally time to buy.

As much as those 3Tb WD Red's failed in droves before, their warranty issued replacements appear to be much better. 38k hours is 4.3 years of continuous operation. Those WD Red's failed before at around 17k hours, so we're heading past double that now. And I still have a spare WD Red with 40k hours on the clock, but had shown no signs of old age when I replaced it.

Now I appreciate a mere 10Tb array could fit onto a single mirrored pair of 12Tb drives with room to spare, but that would be well over a grand to buy, and as much as it would save me 20w of electricity or so which is €45/year in electricity, the math says that I'm better replacing one of the 3Tb pairs with 8Tb pairs instead, thus adding 5Tb. It'll take eight years (yes, eight!) to fill 5Tb at our current rate, during which I'd expect those 3Tb drives to fail, in which case we'll have plenty of ready to go spares. And if one of the 4Tb pairs go, I can always fake a 6Tb drive spare by joining two 3Tb spares.

So, in short, I don't expect to have to think about that ZFS array again after this upgrade for many years to come. If anything, in fact, I really ought to start considering mirror triples, as they vastly reduce the risk of drive failure ever being a problem, plus they eliminate the need for keeping spares at the cost of extra electricity. It's a tough call though, no drive failures since 2015 makes triple mirrors seem a bit overkill for my needs. I guess, though, everybody thinks that until they experience a dual mirror pair failure, and then they're hosed.