John Marino has created something very useful: a graphical tool for Hammer file history. It’s called ‘Slider’, and it uses curses to work in a terminal. It shows historic versions of files and can restore those old versions as needed. This was already possible in Hammer, of course, but it required a sequence of commands that were not straight-forward. I’ve been slow enough posting it that version 2.0 is already out, offering a way to see files that no longer exist, but are still in history. (i.e. deleted some time ago) ‘Time Machine’ sounds like the best name, but that seems to be taken.

One way to keep file history on an very active Hammer disk from eating up all the space: more snapshots. This may seem counterproductive, but disk pruning eliminates historical data between snapshots, so you can keep older data at the cost of some temporal accuracy.

If you really, really want to make sure you aren’t pulling in any parts of X when installing dports, and you’re building from source, there’s a few options you can set to keep X11 off your system. You can even go farther.

It’s possible, if you are several releases (years) behind, to end up with a DragonFly system that can’t compile and install the current release, due to incremental changes over time. It’s rare, but it could happen now between, say, version 3.4 and 4.0. The usual solution would be to incrementally upgrade in order, which is a lot of building and updating. The alternative is the new installworld-force option from Matthew Dillon that forces a new set of binaries into place. Use as a last resort.

This page, Varialus et Anisoptera, set up by… I’m not sure of the real name but it’s ‘varialus’ on IRC – has a detailed description of the DragonFly install process and installation of MATE, plus extra notes. I always find these sorts of cheatsheets entertaining.

It seems pkg 1.3.6 was slightly scrambled. If you happen to have built and installed it, John Marino has special instructions on how to update to 1.3.7. If you are on DragonFly 3.8, you can follow those instructions now, and if you are on 3.9, that repo should be ready for an update in the next few days.

Several people (including me) have been getting bit by a problem: when performing an installworld with a changed kernel, the vn kernel module is loaded, but it was built by the previous kernel and may cause problems when it doesn’t match up.

To fix that, vn is now built in, instead of being a separate module. The rescue initrd (which is what is being mounted when it has this problem) is now installed via a ‘make rescue‘ command that can wait until a successful installworld and reboot.

While Matthew Dillon was testing the new up-to-256-processor support for DragonFly, he added a few sysctls, one of which helps qemu performance when emulating a lot of processors. I note it here in case it’s helpful to someone else.

A note for everyone: use Hammer default on a very busy filesystem, and you will eat a lot of disk space since all file changes are recorded. (I’ve done this to myself a few times.) Francois Tigeot has a list of tips on how to keep that from happening.

If you’re building ports, it will treat OpenSSL as a dependency and bring in whatever version is available. If perhaps you want to use the version of OpenSSL installed as part of your base system, Robin Hahling has the answer for how. (This probably works on FreeBSD too.)

If you noticed the lack of a GUI DVD image for the 3.6 release of DragonFly, I posted a followup note on the users@ list that talks about the steps to get X installed. It’s not much work, with pkg set up.

Hammer’s ability to stream to remote disks is great, but what if you have storage that uses some other file system? Antonio Huete Jimenez put together a shell script that will dump out the contents of a Hammer PFS, for upload to whatever. Read the README for the details.

Recent updates to tzcode apparently fixed a long-standing time zone bug in DragonFly. POSIX says the America/New_York timezone is picked as default if nothing else has been selected. That didn’t happen in DragonFly – until recently. If your timezone seemed to suddenly jump to U.S. Eastern time, that’s because you never picked before.

There are no binary packages built for dports, on DragonFly 3.7, for 32-bit machines, at this time. Pierre Abbat found this out. You can build from source, of course, or just use 3.6 packages. Don’t forget -DBATCH to avoid getting asked for build options when building from source.

TrewGrip, another item in my quest for interesting keyboards I don’t use.

4043 bytes to recreate a mid-80s IBM PC. There are less bytes of data in the program than there were transistors in the CPU that it emulates. It can run MS Flight Simulator. It was for the International Obfuscated C Code Contest, which should surprise you not at all. (via)

True Nuke Puke Story. My mine coworkers once did something similar to a copier repairman; got him so worried about going underground that he had a panic attack when he had to step on the hoist. We had to get a new repairman.

If you want to track the bleeding edge of DragonFly, which is currently version 3.7, I happened to describe it in a reply to Filippo Moretti, on users@. Long-time users will know this/do this already, but it’s worth repeating just because new users may not realize how easy it is.

The system install went normally. I rebooted before performing ‘make upgrade’, as noted in UPGRADING and elsewhere.

I already have dports installed, so a binary upgrade should be possible. I had heard of people with older version of pkg, having trouble getting it to notice upgrades. I rebuilt pkg, and ran ‘pkg upgrade’. A number of the updates coredumped. Here’s one example:

After the upgrade, I had two problems: PHP wasn’t working for the website, and some programs would segfault.

The random segfault was fixable by forcing a binary upgrade of all packages. Since there were some programs on the system that were still new enough that the version number was the same as on the remote repository, pkg didn’t upgrade them. Those packages were linked against old versions of system libraries that predated the locale changes in DragonFly 3.6, so they’d crash. Forcing the update for all packages fixed the issue.

The other problem, PHP on the web server, is not new to me. The binary package for PHP does not include the module for Apache. The solution is to build from source with that option selected. I understand that pkg is destined to support (some?) port options in the future. There’s also an immediate workaround for locking it.

However, the port would not build because of a security issue. The binary package installed without any warning. This, I am told, will change to pkg giving you the option to install if you are aware of the security problem, and whether it really affects you. (which is just what I want, yay!)

Anyway, other than the system changes biting me because I didn’t realize some packages weren’t updated, it went very quickly. That is the reason for binary updates through pkg, or at least a major one.

Things are very quiet this week; I’ve had nothing to post for some days – DragonFly or even for other BSDs. The end of the year has most people distracted, I think. This makes it a good time to bring up something that’s been bothering me: the state of software firewalls in BSD. The pf utility is a BSD advantage; I’ve heard people say “I used iptables on Linux and pf is a much better alternative.” I know that’s anecdotal, but there it is. Here’s the question, and the reason I’m writing this: which pf?

DragonFly has a version of pf equivalent to what was shipped in OpenBSD 4.4. FreeBSD has a version equivalent, I think, to OpenBSD 3.8 4.5’s pf, and it has been further modified. NetBSD has a similar, older pf, but there’s people working on a NetBSD-specific version called npf, which isn’t yet ready. And of course, OpenBSD has its version of pf. If you feel good about these different alternatives, you call it divergence. If you don’t feel good about it, you call it fragmentation.

Compare this to OpenSSH – it works the same on each platform. There’s no confusion on how to configure it, or interoperability problems. It would be wonderful to have the equivalent for pf, where other BSD platforms would import a portable version. This software firewall is a strength, and it’s much easier to tout it when there’s only one.

I doubt there’s a way to bring it all back to one source tree. There’s a lot vested in the different forks out there. You know what would take a lot less effort: a compatibility test suite. Agreeing on a common syntax and set of functions would make life easier for every end user. It would incidentally make vendors a lot happier, too. Even if a user or vendor wasn’t hoping to move between BSD flavors, a test suite would still guarantee a certain known level of functionality for any BSD release.

How likely is this? I don’t know. But I want to bring up the notion before it gets missed. Now is a good time, with each pf version still being relatively close to one another.

Compatibility is Hard. Contrary to popular belief, Microsoft Word documents are not backward or forward compatible, from release to release.

From that previous link: Why Microsoft Word Must Die. The worst problems to troubleshoot are when someone says “Word/Excel is acting funny”. There’s so many intermediate layers of software in those programs that it’s difficult to find the actual data and the actions being performed on it, much less troubleshoot any process.

Apple ][ DOS source code. I don’t have anything I can actually do with the source, but there’s a 1977 price list pictured in the the article that shows some interesting numbers: A 4Kb RAM system costs about $1300, and the prices just go up from there.

If you’ve seen my previous two reviews of Michael W. Lucas’s ‘Mastery’ books – DNSSEC Mastery and SSH Mastery – then you can guess what this will be: his newest book, focusing on a single software topic. This time it’s sudo.

The one downside of reading this book: I now am aware I’m using sudo wrong. Perhaps not wrong, but not anywhere near its potential. Sudo – and I’m not the only person who has experienced this – is used as a “Let’s install sudo so we don’t have to tell anyone the root password”. Sudo works for that sort of thing, but there’s a lot more possibilities.

Sudo is designed to be deployable across multiple systems, as part of a security policy. It’s an easy way to create purpose-shaped roles with different users, especially with users that have specialized skills and tasks, like database maintenance.

Obviously I think better of sudo after reading the book; there’s a lot of program capabilities of which I was unaware, but it’s the book that sells them. Michael W. Lucas’s humor is on display again, to break up some very technical material. Here’s some bits, pulled out.

Remember that “syntactically valid” is not the same as “does what you want.”

Pressing Q tells visudo to break sudo until you log in as root and fix it. Do not press this button. You won’t like it.

Here I create the TAPEMONKEYS alias for the people who manage backups.

And if Carl tries to configure Oracle on the PostgreSQL server, senior sysadmin Thea needs to have sharp words with him. Probably involving a tire iron.

The book is in-depth enough to cover more complex topics like using sudo and Active Directory, and sudo as an intrusion detection tool, of all things.

The usual reasons to buy a Mastery book are all still there: it specifically mentions working on BSD systems instead of pretending Linux is the only system out there. It’s available through a DRM-free seller (Smashwords) in addition to Amazon. It’s a self-published effort, not shovelware. It’s available now as an ebook, and in physical form soon. Lucas talks about it on BSDNow 010, too.

I have one last nontechnical note. Since these Mastery books are working into a series, I’d like to see a whole printed run of visually matching books. Something with the equivalent of the O’Reillyanimals or the Pelican or even Little Blue Books common look and feel.

Google has a Shell Style Guide. Which equates to a Bash Style Guide, but that’s OK. Shell scripts are sometimes considered the most disposable form of programming, so it’s good to see a full guide. (via)

Remember that PDF compression/scanning error I mentioned a few weeks ago? Xerox has fixed it for their devices, and the patch is available. One of the ways to apply the patch is to print the file to a copier, via LPR/raw. Firmware updates via printing – that seems like a good and bad idea all at the same time.

JAVASCRIPT MESS. 45 different computers all emulated in Javascript. A Javascript Mac SE, for instance.

Your unrelated link of the week: Mighty Taco radio ads. Mighty Taco is a Mexican fast food place from Buffalo, New York, USA. It’s about as authentically Mexican as fast food from a city on the edge of Canada can be, which is ‘not much’. I’ve always loved the food, though, and the commercials are just the right mix of amateur joke and commercial advertising.

Linus Torvalds swears a lot. The problem is not ‘office politics’ as he sees it, but that if you swear all the time as the leader of a project, it becomes commonplace. Linus really has to really freak out for people to notice something new. There’s other issues, like how other people emulate the behavior, but I’m pointing out the ‘verbal base sweariness’ of a project affects the entire tone.

Quine Relay, where programming languages write each other. The Ouroboros illustration is appropriate. (via many places)

Your unrelated link of the week: Bones Don’t Lie. An anthropologist who blogs about various discoveries of human remains. I really enjoy blogs where someone is talking about a subject they care about – not to sell a product, not to be paid (directly), but just because they like the topic and they want to share it with others. Of course I would think that, wouldn’t I?

It’s possible your Internet service provider uses a non-routeable IP range (like 10.*) and occasionally your border device picks that up via DHCP by accident instead of an Internet address. If that happens to you, and you’re using DragonFly as your border gateway, it’s possible to prevent it with PF dhclient.

If you’ve been reading the Digest for a while, you’ve seen me talk about the value of hosting or running your own services. It’s not too much of a surprise in my case; if you are working on an open-source operating system, you want to run it. It’s good to get the experience, and you can run programs the way you want, instead of picking from whatever vendors happen to sell you.

The PRISM disclosure, which I am going to assume everyone is familiar with at this point, is another facet. Every time you use another company for your email, your entertainment, your software, and so on, their information on you can be accessed. This isn’t a problem that can be fixed by going from one webmail provider to another. You can shop around, but notice that the author in that link effectively throws his or her hands in the air and says, “there’s no way out” by the end of the article. This is because corporations work as collecting agents for the government, even if they don’t plan to do so.

That sounds drastic, but there’s legal frameworks in every country for governments to require companies to give up data on any person, on request. It happens. I’ve seen it myself; I worked for Time Warner for several years, tracking down cable modem user information and handing it over as compelled by law. I know the lawyers at TW Corporate didn’t like doing it, but they didn’t have a choice. (I have some horrifying stories about what people would do to themselves and each other.)

Companies are increasingly working to create services to sell, not products to buy. A service never stops being consumed, so it forms an ongoing revenue stream. I’m not saying this is bad; I firmly believe that a financial incentive to be paid improves services. However, as only a consumer, you can end up not owning what you use. Other people have pointed this out, and I don’t want to sound like a frothing crazy person… but it is relevant, though not necessarily as catastrophic as some people pronounce.

What I’m working towards here is a reminder that you should run your own software, and running it on DragonFly is the best way. (Or some other operating system, I guess. If you have to.) Instead of trying to figure out what the least-bad commercial option can be, run it yourself. Good for privacy, good for learning. I know that’s not an option for everyone; fighting with Sendmail (for instance) is not an activity that many people pick voluntarily. But, if you’ve been thinking of setting up a replacement for Google Reader, or hosting your own mail, or own blog, etc… there’s never a better time than now.

(Follow all those links for some good information; consider it an early Lazy Reading post)

The ‘amd64′ specific parts of kernel architecture have been removed, since x86_64 covers all that. As a side effect of other changes, John Marino warns that upgrading DragonFly from a version older than 3.4, to a version newer than 3.4, will require an intermediate step of going to 3.4 first. e.g. If your machine is a DragonFly 3.0 system, you will need to upgrade to 3.4 before moving to, say, 3.6 once it is out. This won’t matter for some months, since the next release is months off.

The Colby Walkmac, which predates the Mac Luggable. Linked to because it includes good pictures of what the (external) hardware was like. I find all the old ports interesting, since it’s all USB and the occasional eSATA these days… not that I’m complaining! I’ve never had a good experience with a 9-pin serial port. (via)

I get worried when remotely rebooting a server in a different town or even state. In Praise of Celestial Mechanics covers much more stressful circumstances: interplanetary reboots. Does Voyager 1 or 2 have an ‘uptime’ function?

The equivalent of what you are doing right now, 20 years ago. I personally never got to see this; my experience was MUDs. Speaking of which…

Playing with powerlines. I used to work at a company that used these lines for data transfer. It was neat technology, but it sure wasn’t easy to set up. Imagine wiring a city but only being able to use Ethernet hubs. Not switches, hubs. That, combined with undersized ARP caches/MAC tables, made it really difficult.

OpenVPN on FreeBSD, which will come in handy for at least several readers, I’m sure, as the directions should apply to any BSD.

Since dports uses FreeBSD ports as a base, adding something to FreeBSD ports means it will show in dports, too. However, it doesn’t have to go that way. It’s possible to have dports packages that exist only in dports. If you have changes to a port that make it compile on DragonFly, that can be added too. For all of that, go to the dports issues page on GitHub.

Michael W. Lucas recently wrote and self-published a new book, DNSSEC Mastery. He asked me to review it, and I’ve been reading it in bits and starts over the past few very busy weeks.

First, the background: If you’re not familiar with the acronym, it’s a method of securing DNS information so that you can trust that domain name information is actually from the machine that’s supposed to provide it. DNS information is basic to Internet operation, but it traditionally has been provided without any mechanisms to deal with misinformation or malicious use. This seems to happen with protocols that have been around for many years, as any mail administrator can tell you…

In any case, ‘DNS poisoning’ (or as Wikipedia calls it, ‘DNS Spoofing‘) attacks such a basic part of how the Internet works that it will completely bypass any security methods that assume name information is correct. DNSSEC is a way to deal with that. It introduces public-key encryption into the process of sharing and updating DNS information. The idea has been around for a while, but it’s only been completely implemented recently.

DNSSEC Mastery goes over this history, and through the setup required to get (recent) BIND working with DNSSEC. Lucas seems to be starting a series of ‘Mastery’ books, where he covers all the territory around a specific topic. This one, like his previous title, is exactly what it says. As long as you have some existing clue around zone files and DNS, the book will take you from no DNSSEC at all to fully implemented in less than 100 pages. (well, at least in the PDF version, but that gives you an idea of the size.)

Use it to learn, or use it as a quick reference – either way will work. If you have any DNS server(s) to manage, you’re the target audience. I expect DNS without these security extensions will go the way of telnet vs. ssh.

A book covering things like new encrypted hash zone record types is going to be a bit dry, but there’s an appropriate sprinkling of humor through the book. I’ve reviewed other Lucas books before, and I’ve got another on my plate right now, but this is the same: there’s plenty of funny to make the lessons go down easier.

Ansible seems to be a configuration management system that’s lighter than puppet or salt. I had a student talking about it in my class tonight. BSD users Hubert Feyrer and Michael W. Lucas have both posted about it recently. Anyone want to repeat their experiences?

If you’re running DragonFly-current, which right now means version 3.3 and very soon 3.5, you are probably running pkgsrc. If you want to transition to dports, this pair of posts from John Marino will tell you how.

Peter Hansteen has an extensive writeup of how he has managed the bsdly.net spam blacklists. Normally I’d stick this article in the Lazy Reading links, but the article is good enough to call out separately. It’s excellent not just for the mechanical aspects of how the blacklists were maintained, but for his strict description on how the process is simple, verifiable, and transparent. That last item, transparency, is how many anti-spam groups fall down.

For anyone who is a student considering Google Summer of Code this year: this timeframe we’re in right now is listed by Google as time for “students discuss project ideas with mentoring organizations”. This is the perfect time to find out what the people in an organization are like, and get early feedback on your project ideas.

Chances are, if you’re submitting a proposal for an idea from an org’s project list, you’re one of a number of students all trying for the same thing. The best way to get accepted instead of any other applicant is to be the person they already know.

I hope you like reading; there’s some very meaty links this week. Go get a cup of tea and settle in. You drink tea, don’t you? You ought to.

Reading about KDE’s repository near-meltdown makes me think we need more checks for DragonFly. We have the advantage of Hammer, of course, which would help in the same way that the linked article names ZFS as a ‘fix’. (via multiple places)

We know that Apple will reject apps it disagrees with. Google also will do so. Has there ever been a program rejected from pkgsrc or (FreeBSD/OpenBSD) ports on content grounds? Not that I know of – anyone remember differently? I’d argue that’s a favorable point for the BSD packaging systems, though it may just be that no application has tested those boundaries yet.

Your unrelated comics link of the week: Tom Spurgeon of the Comics Reporter asked people for their lists of webcomics that could go in a ‘Hall of Fame’. The resulting list is a lot of really, really good material. Go use up a few hours reading.

Apple is Losing the War – Of Words. I’m not interested in it for Apple, but rather the casual reference to the huge quantity of astroturfing going on, all the time, from major tech companies.

Following up on my earlier tweetspam post: World’s Best Spam. Remember, recommendations from others is the most effective persuasion method to get people to buy, so there’s a big economic incentive to create positive recommendations. (via)

I managed to come up with a lot of links this week, somehow, despite the start of the class I’m teaching in addition to normal work. And Summer of Code’s coming up! And we’re due for a release relatively soon! I may appear somewhat… stretched over the next few weeks.

Your unrelated link of the week: I’m the Computer Man. I always thought the mid-1990s were sort of a Internet/computer teenager phase. Everything had potential but everything was also awkward. (via I forget, sorry!)

If you are on Windows, you probably use PuTTY for ssh. It saves everything in the registry, which can occasionally mean losing all your configuration. There’s manual ways to save it, but there’s also PuTTYtray. (I’ve used portaPuTTY in the past, but it seems to be missing/no longer updated.)

“Don’t Be A Stranger“, musing on how there isn’t enough meeting strangers through the Internet any more. Here’s the odd thought I had while reading that article: I couldn’t pick most of the other DragonFly developers out of a lineup, but I’ve been working and talking with some of them for a decade.

Updates: from discussion in IRC about this sort of distributed authentication (maybe ‘authentication distribution’ is a better phrase): Tools like puppet or FreeIPA may also be useful. From seeing other conversations about this, it looks like there’s a lot of solutions to pick from, of varying difficulty, and none canonical. That’s both good and bad.