06/03/15

Guess I've been neglecting my blog in more ways than I thought...hadn't updated this progression since my post back in October 2013. But, have two recent prescriptions to add to the mix....one with my regular optometrist, and a second done as part of seeing a neuro-opthamologist at the Mayo Clinic as part of my ongoing quest to see if I have SCA, what it might be, and what its future holds for me....

Felt like there should've been a Feb 2014 in there somewhere.....but I don't have access to all my files at the moment, something I'm going to want to do something about when I get home. But, nothing in my usual cloud places ....

But, wonder which is right and which should I get in glasses....well, actually, I had already ordered new glasses after the May eye exam. Had they been ready in about 7 days, I'd be wearing them right now. But, didn't get the call about them until yesterday. Wonder if I'd get new glasses with the latest prescription now....there is a Lenscrafters here.... or wait until I get home. The new prescription actually looks like its in the same ballpark as what I had been told....hadn't actually looked at my May prescription until just now as well...

Meanwhile...does the Spinocerebellar Ataxia quest continue...

But, first where should I go for supper tonight?

Well, I opted to not visit Lenscrafters at all, though I had several wonderful suppers at Chester's Kitchen & Bar, including once from 'Garden Patio Seating' (where I was kittycater-corner from Lenscrafters.)

When I inquired back home in Manhattan, it was explained that the prescription needed to be converted from positive scale to negative scale to compare to last month's prescription. After conversion, the difference was considered within the range of what would happen if I were to get tested every day.

According to this blog, I received this PowerSource 400 on January 21st, 2008 - click here

In answer to the question at the end of that post, the answer was yes. There are 5 other PowerSource 400's in service in my home.

Anyways...back on October 28th (my 45th birthday ), about an hour after getting into bed there was a power blink....I recall being woken by the PowerSource 400 in the bedroom, but I was still able to breath, so I eventually went back to sleep. I woke in the morning to find that half my network had been down for quite some time, though I didn't see any alerts on the security system control pad.

This being the "network closet" UPS...it was providing backup power for my DSL modem, its associated router, the switch that connects the various rooms in my home together, namely the "living room" and the "back bedroom". Not sure which AP the "side bedroom" was connected to. Though the power outage knocked out my laser printer, which I haven't felt the urge to find out why. Really need to work on finding a more accessible location for it, or work on reversing the entropy in my nuclear room(s)... especially since my other printer is circling the drain, and not sure when I'll get around to getting the new inkjet printer that I've been eyeing.

Restarting the UPS, was when I finally got alerts from my security system. It was a combination of events that happened when I cycled power and events that had been waiting for the return of connectivity. The security system's primary connection is broadband, but it also has a phone line for backup (there's a cellular option, but its been out of stock whenever I've felt the urge to splurge for it....though I kind of have cellular backup anyways....)

It was the return of power that caused it to report loss and recovery of the phone line, along with the queued messages of broadband and power outages, with restores of all... Its built in backup battery had kept it up during the outage (approx 8.5 hours). The phone line from the base station is connected to an ObiHai OBi110. Which has an FXO port to allow selection of analog phone service, or in the event an outage pass through to this. In normal operation the OBi110 is making my Google Voice available to my regular phones, where the OBI110 is configured to use my other Internet gateway. The FXO port is connected to a Cobra PhoneLynx Bluetooth Cell to Home Phone Adapter which is associated with a phone with AT&T prepaid service.

Though I plan to at a later date replace both the OBi110 and PhoneLynx and cellphone with an At&t Wireless Home Phone Base. Which I have sitting, waiting for when I get around to it.

The plan is the OBi110 will move to the living room for use there, though its in associating with the planned inkjet printer replacement.

So, its time to replace the batteries...guess I'll do research on where I can get a pair. Hmmm, it uses the same kind as the pair I had replaced in my SmartUPS 1400 back on October 10th (it went around October 5th, and I ordered them on the 6th). The site provides discounts if you buy multiples. I got 5% off for getting 2, would've gotten 7% off for getting 4, or 8% off for getting 6 (9% for 8 and 10% for 10+).

I hesitate, since I'm waiting to see how the recent problem with BillMeLater goes...so needs to be sometime after Nov 5th to order. Meanwhile a storm comes in on November 3rd, and power is blinking like crazy...and I'm constantly restarting the PowerSource 400. Finally I take it out of service, and everything is just in a regular surge strip. Things remain stable (dbox stayed up straight) until this morning. But, feel that I need to put the PowerSource 400 back into service before my annual Thanksgiving trip.

After much procrastination, and lots of searching. I settle on ordering 4 such batteries from an eBay seller on November 14th. Plan to replace the batteries in my second PowerSource 400 soon, since it was purchased on December 31st, 2008 and provides backup power for my CPAP. (see here).

And finally, this morning, I decided that I've put it off too long and to replace it this morning.

Which was a challenge....

There was supposed to be a page 3, but all I had typed was the page break...so had to figure out how to open it up again. Aside from the 4 corner screws, I remembered that there's a screw behind the sticker in the middle. But, I forgot the rivets in the bottom, and that both parts need to be extracted.

Once opened, I started to disconnect the batteries, and once again threw sparks and added another mark to my screwdriver. Taking a step back, I decided there's an order of operation involved here. The answer is start with the outside connections, and then the inside ones. And, then reverse it. This time I knew the side panels interlock with the batteries to help lock them into place, so made sure the wires were clear, I noted that the neural wire for the front outlet had a kink, I made sure that all the wires were in the channel while buttoning things back up.

Surrounding this, was the challenge of getting it out from behind my headboard, though while I was down there, I found a couple missing packages of mask diffusers, I knew I wasn't using them up that quickly, but couldn't figure out why I kept running out. There was one Xyrem pill bottle, I had one night knocked both down...luckily while the empty (first dose) one went all the way down, the other was just wedged between mattress and headboard in upright position. Since I use bottles longer than a month, I usually go by how grungy the caps get...since I customize the bottles, so I can tell the two apart by touch, which matters when one has a larger dose than the other....and the caps get grunge depending on how many nights I overfill (have always had a problem measuring drinks that way.... ) Or how often they get tipped...

There were also as expected a bunch of nasal inhalers of various types...such as ones from: Olbas, Benzedrex, and Vicks.

10/05/14

Probably less major effort to merging customizations into this update, partly because it wasn't that long ago that I merged 5.0.9 into my site. When there are diff patches, I tend to merge those into my site...while the bigger releases its figure out how to reapply my customizations to it. Though there wasn't a huge amount of change between 5.0.x and 5.1.x, within the areas that I had made customizations to.

Some were also obsoleted by updates. And, some of the customizations I should convert to using methods that exist in the newer versions and abandon my old patches. Such as how to add additional smilies (where I'm using a combination of the two methods.) Or skin customizations (like custom meta or javascript), where I continue to patch foward from older sites but have done through the Advanced tab on my newer site.

One area of extra work was updating my skins to the new versions, not as bad as 4.x to 5.x was, but there were enough changes that I had dig around a bit to see what was going on and what was still needed.

The old skins should still work unchanged, but sometimes things break between versions...or things change slightly. I use three main skins for this setup. I had some time ago, made my own copy of 'custom' to avoid constantly recustomizing it after any update. And, I had settled on making a skin based on 'evocamp' for another. It was originally based on 'emerald' which was a 3rdparty skin, so there hadn't been a separate copy then. Which is probably why I didn't make my own copy of the 'photoblog' skin. Though since I used the Advanced tab and such to make most of the customizations to it, there was minimal adjustments to make to it. So, after updating my copies of the 'evocamp' and 'custom' skins. I shoved the new version up to my web host.

Ran the updates, and it largely worked. There were some oddness with $baseurl, $htsrv_url, and $allow_redirects_to_different_domain. Eventually came up with values of the first two that seemed to work right, and set the last to 'always'. Multidomain is kind of messy still. If $baseurl, changes with HTTP_HOST, Blog URL doesn't work unless none of my blogs use $baseurl. Plus there were oddities with logins or backoffice. Perhaps there needs to be another option for $allow_redirects_to_different_domain, and that is to allow them to a configured list of allowed domains.

The final bit was to make some readjustments to style.css, for my main site (based on 'custom'.)

And, then one more bit...the more and next-page toolbar buttons are missing....back patch those into _quicktags.plugin.php, even though the internal version didn't change.

05/01/14

Keep seeing this annoying message on FreeBSD, even though back on December 20th, 2013....I had set "security.bsd.unprivileged_mlock=1" in /etc/sysctl.conf to try to finally address this problem.

The default RLIMIT_MEMLOCK resource limit is 64k, which I would think is more than sufficient.

So, it was time to research this problem in more depth.

Found that there's a DEBUG_SECURE_MEMORY define to see how much memory its trying to allocate. Which its trying to allocate some multiple of 16k blocks, which it later refers as pages. Which I seem to recall is Windows?, Solaris is 8k and most other systems are 4k (my FreeBSD system, its 4k). Well, its only trying (and failing) to mlock 16k. So, I tried overriding the constant to 4k. But, this also failed.

I had skimmed the man page, where it says:

Since physical memory is a potentially scarce resource, processes are limited in how much they can lock down. A single process can mlock() the minimum of a system-wide ``wired pages'' limit vm.max_wired and the per-process RLIMIT_MEMLOCK resource limit.

If security.bsd.unprivileged_mlock is set to 0 these calls are only available to the super-user.

Well, on my system vm.max_wired defaults to 1323555 and RLIMIT_MEMLOCK (ulimit -l) is 64.....so limit is 64k, right?

Wrong...delving into the Kernel source...I found that it first checks that the requested amount + the amount it already has doesn't exceed RLIMIT_MEMLOCK, and then that the requested amount + the amount wired system wide ("vm.stats.vm.v_wire_count") is not greater than "vm.max_wired".

Well, when I looked at vm.stats.vm.v_wire_count it was 2020311....its already got more than vm.max_wired, wired!

I feel a PR coming on....

1323555 (which is about 5GB) is said to be 1/3 of some maximum. I have a 16GB system, probably not contiguous...and there's probably some amount reserved....but 2020311 is about 7.7GB.

I did a "sysctl vm.max_wired=2097152", and it took it (so put that into /etc/sysctl.conf, too.) and now gnome-keyring-daemon can start without that message.

02/13/14

Some time before I bought one, I had often wished I had a Travel Router during my travels. And, I know I looked at whether it would be possible to use my Linux laptop as such.

But, then on November 25th, 2011, Black Friday....meaning I was at Chicago TARDIS at the time....I had on the spur of the moment ordered the ZuniConnect ZTRP150 WiFi Travel Router with USB Charging by ZuniDigital from NewEgg.com.

As I recall, it then sat around for months until I finally set it up, in preparation for possible use on my next trip....Gallifrey One in 2012.

The turn hotel ethernet into WiFi for all my gadgets was really nice, since most hotels only allow one device per room to register for its free or pay wireless. (though I heard some allow pay per device....) But, my reasoning at the time was the large number of Eye-Fi cards that I have in my collection, which are unable to connect to such WiFi, even when its free and its just an EULA page that needs to be accepted.

Otherwise, I wasn't too WiFi dependent gadget heavy then....I could use 3G on my smartphone, and my Kindles all did 3G or something (either exclusively or later with WiFi...I had started carrying the Kindle Fire, original, on trips....so it was the first that would benefit from a travel router.)

It was November 23, 2012 that I started my journey into the world of Chromebook (I had preordered it, and that's when it shipped....at first I was traveling with both my Linux laptop and the Chromebook, but for some time now...I've been going with just the Chromebook. Which has been challenging, like now I can't add SSIDs to my eye-fi cards on the road. So, I have to hope that I got the right ones pre-added to the cards. Along with some of the ones I know what will work, like the SSID for my MiFi2200 or later tethering off of my HTC One (I got the 5GB tethering plan, because lower tiers weren't eligible for employee discount and I had been looking to upgrade from MiFi....but hadn't found a reasonable pay as I go, but can't be activated because I don't live in an area that's covered by it.

Though I did consider exiting the Smartphone crowd and getting a contract Mobile Hotspot, but there isn't anything in an Android 4.3/4.4+ equivalent to iPod Touch....or WiFi only smart phone ???

Now, I guess I've been lucky with the ZuniConnect, which has two modes, Router or WISP. It has both a WAN and LAN port, so there's lots of different ways it could be used.

Long before this, I already had a RoadWarrior travel Ethernet cable in my carry on....so I wasn't stuck if the room only had a jack. And, the router mode was all I needed. It was pretty much plug it in and go everywhere that I stayed.

That was until my previous trip....the one to visit my brother and parents for Christmas. The hotel I stayed at didn't have Ethernet in the room. I had never looked at WISP, but knew it was something I was going to need to use eventually.

Well, it was a bust, because WISP is largely a different configuration in the router, because I would see ZuniConnect or something as an SSID sometimes, but not be able to connect to it. Not sure I know how to connect Chromebook to WPS, or if its possible. etc. But, in the end found that the only way to configure WISP is through ethernet. And, none of my devices had ethernet ports. (I'd still be screwed if I had a MacBook Air along....)

So, I made a note to investigate alternative Travel Routers and to acquire a USB Ethernet adapter for my Chromebook, etc. I eventually got both as part of a larger order from Amazon.com on Jan 22, 2014. I got a "Plugable" USB Ethernet adapter, because it was specifically listed as an adapter from Chromebook. And, I got a TP-Link TL-WR702N, Which sounded like it also did all I wanted, had been favorably reviewed and I've been pretty happy with the TP-Link TL-WR1043ND router that handles my Cox connection to the world....doesn't do all the stuff I liked doing with DD-WRT (though I could DD-WRT it)...but its been rock solid, and since I've moved to running nginx reverse proxy on a DMZ host, the 16 port forwarding limitation isn't an issue. QoS might start to become a concern though. But, I still primarily do that through DD-WRT on AT&T connection to the world. Some day I think I want to try pfSense....

But, that hotel stay wasn't a problem since they're WiFi access was controlled by a password that is given out at check-in....so I could connect all my devices to the WiFi without problems.

Anyways....these items sat around in their packages, until the night before I was to depart for Gallifrey One 2014.

The USB Ethernet adapter just worked and wasn't a problem (though I haven't registered its MAC with my network, which only does reserved DHCP ... its on my list to create a guest network, which can be helpful for discovering MAC address of devices that don't have them printed anywhere on them. But, it hasn't been an issue with wireless devices, since those failed attempts show up in my radius log. Which probably also shows up in the dhcp log (I suppose I should set those logs to forward to zen, so I can see them and add them to the appropriate files in CFEngine 3 repository....still haven't gotten cf-runagent working though.)

The TP-Link TL-WR702N was another story. Again it looks like it needs to be configured manual for WIFi Bridging through its ethernet port, but the ethernet port being dual mode LAN or WAN, its out of scope for its built-in DHCP (which is also disabled by default...) Also of annoyance was that its SSID was fixed, couldn't tailor it to my convention, but rather its own convention ending with the last 3 octets of its MAC. Where the default password is the last 4 octets. It does allow you to change the password, along with other encryption settings, or go open. It has a dropdown list for channel, which had defaulted to AUTO. But, it won't allow you leave the page until its been changed to be the same channel as the selected WiFi. Seems its a flaw with all WISP, that they attach to BSSID.... I have two APs at home, both with the same SSID, but different BSSID (of course) and different channels. Yet, my bedroom is still in a hole....

Could be interesting in a hotel environment where there's going to to be many different BSSID/Channels, which might change throughout the stay.

But, I ran into a problem. I couldn't not get my Chromebook to connect to the Ethernet port. It didn't do DHCP, but Chromebook has options to set things manually, but Chromebook still wouldn't connect. I suspect there's something Chromebook expects to get answers for to determine that the connection makes sense, and the TP-Link doesn't do it. At first I thought the Chromebook was expecting a fully usable Internet connection....which doesn't make sense, since its able to use captive portals, though often the captive portals only block http/https initially, or provide/leak enough to satisfy my Chromebook.... That was until this Gallifrey One trip....

01/20/14

I went to Chicago TARDIS 2013 this year, after having missed going to Chicago TARDIS 2012. This being the weekend after the 50th Anniversary, I was determined to make this convention.

This year things were interesting, in that I flew out of Manhattan at 7:05am, arriving in Chicago around 9am. Fortunately, I didn't have any trouble being able to check in right away when I arrived at the hotel. It was kind of strange being picked up and riding alone in a stretched limo. But, I guess its what was available when I showed up, and nobody else was making a similar trip around that time. I had booked shared ride service from Windy City Limos, like I had the previous two Chicago TARDISes. Return trip was in black sedan. I had also booked Black Car in Manhattan for the ride to and from the airport.

While getting settled in the hotel room, before heading down to see about lunch....I discovered to my horror that I the charger for my Panasonic Lumix DMC-FZ200 was not among my luggage. I did have a couple of what should be fully charged battery packs, but they would not last the weekend.

I later spent some time researching local options on what to do....I settled on the possibility of experiencing Target on Black Friday, where they were featuring the Nikon Coolpix L320 for $99.99 or the Nikon Coolpix L820 for $199.99. I was leaning towards the L320, since this wasn't likely to become one of my regular cameras, so price was a big factor. Alternatively was to see if they carried extra chargers for my camera, since they carried the Panasonic FZ70K....though I later found out that it wasn't the same battery. They had this camera on sale for $299.99, IIRC. Also the Coolpix L320 or L820 had the advantage to me that they took AA batteries (4).

I knew Target was going to be one of the stores opening Thursday night for Black Friday....though I wasn't sure when I would make the dash to try to salvage my trip.

Later on Thursday night, after getting my badge and then checking out the reception (when I saw other people heading the same way, but continue out across to Target.) After getting some food and drink, making a loop around the space, I decided that I would make a dash over to Target and see what's what.

It was chaos in Target, the electronics/cameras counter was pretty much inaccessible since everybody there was after iPad's of various models and sizes. I wandered around a bit to see if things might settle down as they started reporting certain iPads were now out of stock. Occurred to me that I should see about some cheaper water for use in the hotel room, but they didn't have any small packs of still purified or spring water available, so I got a 6 pack of flavored water. Guess I could do that instead of mixing Crystal Light to go with my medication.... As I walked back, I noticed that one of the end of aisle displays was full over Nikon Coolpix L320's. Since that was the most likely candidate if I was going to buy a camera for salvage the trip....its what I got.

It then took forever to snake out from one opposite corner of store to the other to check out. I had intentionally brought my Target Red card this trip, which I otherwise usually leave at home when I travel. Plus I hadn't shopped at Target for a while, in anticipation of this possibility I would later make another trip, to get refill pages for the album that I keep the purchased photos and such I get at Chicago TARDIS, and a bottle of shampoo as the hotel shampoo was bothering my scalp.....

I knew I had an 8 pack of Lithium Photo AA's in my luggage. I went back to my hotel room and unpacked the new camera and spent sometime getting familiar with it (using the included AA batteries.) Since I had owned Nikon Coolpix cameras before...namely the Coolpix S80 (Much earlier on there was a Coolpix 5500, but I don't recall what it was like anymore), I found the menus and such largely familiar, so I tweaked up some settings to fit the kind of shooting I was going to do this weekend. And, set up to give it a test.

The reception was still going on, so I went back in and shot some pictures around the room to get a feel for its controls and behavior. And, downed some more snackage before calling it a night.

01/19/14

So, near the end of July, I started investigating (once again) on replacing my HP Photosmart 8450xi (which was now over 8 years old....bought it on June 30th, 2005 - Back from Vacation Tech Buying Spree?...setup on July 9th, 2005 - link

I had started looking some time before this, but was put off for a bit due to my experiences with the Brother DCP-7065DN -- link, since it seemed most of the choices out there were GDI and I'm moving to more and more heavily FreeBSD as my primary operating system.

Especially since it appears that 'box' finally called it quits on December 2nd, before I had started my journey home from Chicago TARDIS that day....and orac is inching close to its end, as the pair of ST2000DL003's which evidently only had 1 year warranties from June/September 2012 started going shortly into the new year. I was trying to use ddrescue to force sector remapping on the first drive, when the other drive has decided to vanish permanently. I had thought it was was DM's that had 1 year and DL's that had 5 years, perhaps I had it backwards....or its a question of when I purchased them, or how they were packaged.

Checking my order history, I purchased one drive on June as a bare drive and later in September as a retail kit. I haven't yet pulled the drives, so I can't look up the serial number for the vanished one, but Seagate's website says the one that is responding is out of warranty. Even if the other drive is still under warranty, not sure I want to deal with getting it exchanged for a refurb to create a solo 2TB drive. Can't think of not wanting raid given what I'll likely use it for. And, not sure I'd buy a different 2TB drive to be its mate (and it won't work with my other 2TB arrays, since its an advanced format 2TB drive...while the lraidz2 pool on zen used legacy format 2TB drives (which limits options of growing it non-destructively.)

Fortunately, I had copied one of the big volumes from it over to zen (along they way it got corrupted, so had been trying to copy it back from zen when the other drive died). And, files of the other volume (my pyTiVo store) should all be in backup, where I don't have space on zen to restore them yet.... I have pyTiVo on zen, but the content under it is different...and larger, so much that it is currently not being backed up. I haven't gotten made much progress on building the second backup server....guess I'll need to look at this sooner than later.

And, now it seems the other 2TB RAID-1 array on orac is dying. I just went ahead and failed the drive that was giving it issues. Not sure what to do with it...suppose I could try ddrescue on it and see what happens. The big volume on it had also been copied over to zen, so guess I'll update my HSTi's to point to zen instead of orac for their content. Another used to be for Time Machine backups, but I had moved that over to zen when I set up the new work laptop to do Time Machine backups on my home network. I was using that space as overflow from pyTiVo. And, another was for backups of various things, which I had stopped adding to as new backups are going to zen now. Its things like regular backups of my websites at dreamhost and 1and1, my router configs, serial console servers, and some other backups. I was also replicating some directories on zen to orac as backup (left over from when zen was a Windows 7 PC....which saved me from losing everything when it scrambled itself.)

01/08/14

Today this message appeared, and I knew that I needed to find a socket with a QLIM smaller than QLEN=8, but couldn't remember what the formula was.

But, the topic had come up on the bind-users list back on November 14th, 2013, where the messages was about '16 already in queue'.

Where for months before this I had been getting messages for '10 already in queue', and the only tcp socket I found that might be a problem The only thing with a QLIM of 10 was the submission port on sendmail, which didn't make sense...and bumping it up didn't help.

And, searching my system for the pcb was a bust (using lsof ‑i ‑Tfs | grep LISTEN or netstat ‑LAan)

Reducing end digits until I got matches, resulted in matches that didn't seem to fit.

So, I tried to ignore it....

When it popped up on the bind-users list. The discussion went to that the tcp-listen-queue default is 10. But, it didn't seem to apply in my case, until later when I did see some messages for "5 already in queue", because the base bind in FreeBSD 9.2 is 9.8.4-P2 where the default tcp-listen-queue is 3. It was changed to 10 in bind-9.9.

Anyways, when the thread came up on bind-users list, I decided that I needed to really dig for the answer. Searching through the kernel source, I eventually found my answer.

Couldn't figure out how to change the listen queue in it through its configuration file, so I stopped using it. And, the messages stopped. I had filled out the proxy settings in chromium with squid for http & https and ss5 for Socks5....and evidently some update around the same time as when I upgraded to FreeBSD 9.2 (or perhaps FreeBSD 9.2 made the message show up for dmesg?). Switching to using squid for all protocols fixed it.

Meanwhile...while I was looking for that old message, which I had posted back on November 20th, 2013. I stumbled upon some older threads on freebsd-stable.

I was searching on home computer, where I'm subscribed to the list, while my work email isn't subscribed to the list... and all my old freebsd list emails have since been purged. Still trying to get my email back under control after switching providers...both personally and at work. Plan to let an old personal domain expire once the migration is fully done, but its going so slowly that I let it auto-renew last year...and perhaps forgetting to change to the default 2 year auto renew to 1 year was intentional? New expiration date is November 20th, 2015. It was an early domain that I had registered, before I knew that '-'s in domains are considered bad. There were a number of different blogs that I would try to leave comments at, and the comments would claim to go to moderation but actually get discarded. The owner of one site eventually responded saying the system automatically does that to domains with '-'s in them, since most of them are spam. But, he'll whitelist my domain for the future. (IIRC, it was about a different antispam patch he had written for our blogging platform, functionality that never made it into newer releases and hadn't gotten updated. Wishing something like it was back again.)

That made me wonder if another site, running under my employer's domain...with a '-' in it, was rejecting my comments under my work email account, because it has a '-' in it. Switching to the form without the '-', and the comments would appear. I suggested to the site owner that he should remove that filter or at least whitelist our employer's domain.

The threads were older, and associated with upgrading to FreeBSD 9.2....first thread was started on August 1st, 2013. Was for "8 already in queue", and later indicated that the system was for backups and did outgoing rsync's and also did NFS and Samba. The discussion talked of strangeness of only having a queue limit that small, and that the default limit (128) is like 20 times that. The last reply to the thread was October 7th, 2013. Another thread started on September 30th, 2013 for "193 already in queue", with the last reply on November 12th, 2013.

The main hanging point again was that the pcb couldn't be found...and the suspicion is that its how daemons fork processes to listen to sockets and/or to handle requests, plus that they might create all these things and then use fork to detach to run in the background. The last thread was about using dtrace to maybe see if the process could be found that way.

I've been meaning to play around with that, but when I had last tried...found that its a module, and kldload dtrace wasn't the right way to load it.... its kldload dtraceall Guess I've rebooted since then, so it should be right (and done automatically in /boot/loader.conf.) Guess when I have time....

So, I wonder if I should reply to one or both of the threads....but first, its been a while since I blogged....so here I am.

As for today's message?

QLEN = 8 => QLIM = 5

At first I looked for the full address:

Shell

# netstat -LaAn | grep fffffe006acd9310

nothing

trimming, I eventually got:

Shell

# netstat -LaAn | grep fffffe006acd

fffffe006acdb7a0 tcp4 0/0/5 *.5666

fffffe006acdb3d0 tcp4 0/0/128 *.587

fffffe006acdcb70 tcp4 0/0/50 *.445

fffffe006acdc3d0 tcp4 0/0/128 *.621

nrpe? Hmmm, did that one new disk check push me over?

What else is 5?

Shell

# netstat -LaAn | grep '/5 '

fffffe012b909b70 tcp4 0/0/5 *.10143

fffffe006acdb7a0 tcp4 0/0/5 *.5666

fffffe006aab2b70 tcp6 0/0/5 *.5666

fffffe006abd17a0 tcp4 0/0/5 *.9032

fffffe019a1503d0 tcp4 0/0/5 *.873

fffffe012b9093d0 tcp6 0/0/5 *.873

fffffe006abd0000 tcp6 0/0/5 *.2049

fffffe006abd03d0 tcp4 0/0/5 *.2049

....

10143, imapproxyd - wasn't accessing roundcube

9032, there shouldn't be anything accessing pyTiVo

2049, NFS hmmm....well, my MacBook Air might be doing a PowerNap and doing its TimeMachine backup to the NFS share on my FreeBSD server.

873, rsyncd - BackupPC is constrained against running more than 3 jobs at once, and at most 3 against this server (I break up my [bigger] systems so its not all backed up at once, using lockfile in DumpPreUserCmd, though I have exceptions on this server so that certain rsync shares aren't blocked if a really long backup is running (recently had an incremental take 1 day and 11 hours - at least on my FreeBSD/ZFS system I have a comamnd in DumpPreShareCmd to take a snapshot.... a couple of weeks earlier, I had an incremental take 1 day and 15.5 hours.

Tweaked some sysctl's, and deleted some old snapshots seems to have sped things back up.

Probably NRPE

So some of the messages convert to:

QLEN => QLIM
==== ====
193 128
16 10
10 6
8 5
5 3

OTOH, "8 already in queue" is what the first thread in August had, and he had added about being a backup server that does output rsync and had also mentioned NFS (and Samba).

Additionally, in the output looking for QLIM == 5, were these lines

Shell

unix 0/0/5 /tmp/.org.chromium.Chromium.wpVy4H/SingletonSocket

unix 0/0/5 /tmp/ksocket-beastie/klaunchere40501.slave-socket

unix 0/0/5 /home/beastie/.pulse/zen.lhaven.net-runtime/native

unix 0/0/5 /tmp/.esd-1000/socket

unix 0/0/5 /tmp/seahorse-QY4SIO/S.gpg-agent

unix 0/0/5 /var/run/samba/nmbd/unexpected

When I was previously looking for QLIM == 6, there were only the two tcp sockets, so it was only 50-50 on picking the culprit, and since the other was minidlna which I haven't done more than build/install it so far. It was really only the one socket to explain it, and it did clear up immediately once I stopped using it.

As for NRPE, there doesn't seem to be a way to change it easily....so I'll just see if the problem continues to happen, before investigating other solutions.

11/06/13

The Chuck Yerkes Award is presented annually in recognition of outstanding individual contributions in [system administration] online forums. It was created after Chuck Yerkes' untimely death in 2004 to memorialize the mentorship he provided countless systems administrators through his helpful and accurate posts to systems administration mailing lists.

Since 2009, LOPSA (The League of Professional System Administrators) determines and presents the award to someone who followed Chuck's example in their contributions to system administration online forums--whether mailing lists, web forums or chat rooms. 2005-2008 awards were presented by the USENIX Association.

According to alerts coming from IRC, I found that I got the award during the "Opening Remarks and Awards" portion at LISA '13 conference in Washington, DC.

I knew I had been nominated, but hadn't heard anything after that...so I'd be curious to see what get's written up on me officially, etc.

11/04/13

So, the announcement of FreeBSD 9.2 came out on Monday [September 30th], which I missed because I was focused on my UNMC thing. But, once it appeared, I knew that I was going to want to upgrade to it sooner than later.

From its highlights, the main items that caught my attention were:

The ZFS filesystem now supports TRIM when used on solid state drives.

The ZFS filesystem now supports lz4 compression.

DTrace hooks have been enabled by default in the GENERIC kernel.

But, I did start this upgrade on October 4th....where for an unknown reason, I launched the freebsd-update process on cbox, the busier of the two headless servers. I suspect I went with doing the upgrade on my headless servers, because they are entirely running on SSD and would likely see the benefit of lz4 compression. And, perhaps I did cbox, because it was the system that could most gain from lz4.

It took a couple iterations through freebsd-update, before I got an upgrade scenario that could proceed. And, it took a long time given the high load that is cbox.

That is cbox is an Atom D2700 (2.13GHz, dual core) processor. And, cacti (especially with the inefficient, processor/memory intensive percona monitoring scripts -- might help if only scrpt server support worked, and wasn't just a left over from what it was based on.) being the main source of load. That is usually in the 11.xx area, except during certain other events (like, since 3.5, when cf-agent fires...cbox is set to run at a lower frequency than my other systems.) or when the majority of logs get rotated and bzip'd. And, there's also some impact when zen connects to rsyncd each day for backuppc. But, these spikes weren't that significant. Though the high load would cause cf-agent runs to take orders of magnitude longer than other systems, including its 'twin' dbox.

Also ran into a problem (again?) where a lot of the differences that freebsd-update needed resolved were differences in revision tags....some as silly as '9.2' vs '9.1', others had new time stamps or usernames, but seldom any changes to the contents of the file. Which I then discovered a problem from having some of these files under cfengine control. cfengine would revert these files back to having '9.1' revision strings, which confused the freebsd-update. I ended up updating all the files in cfengine to have the 9.2 versioning, though I thought about just removing/replacing it with something else entirely, though wasn't sure the impact that would have on current/future freebsd-update upgrades.

Though it did seem to cause problem with the other two upgrades, where it would say that some of these files were now removed and asked if I wanted to remove these. Which doesn't make sense, since it didn't say that with the first upgrade. It was probably just angry that these files already claimed to be from FreeBSD 9.2.

It also didn't like that I use sendmail, therefore my sendmail configs are specific to my configuration, or that I use cups, so printercap is the one auto-generated by cups, etc.

But, once it got to where it would let me run my first "freebsd-update install". I ran it, rebooted, ran it again, rebooted, updated stuff (though it didn't complain as much, perhaps because some of the troublesome kernel mod ports had corrected the problem of installing into /boot/kernel, or perhaps enough stayed the same between 9.1 and 9.2, that things didn't freak out like before. And, this includes the virtualbox kernel mod, when I did the upgrade on zen, and later mew. But, I re-installed these ports and lsof. I did a quick check of other services, and then upgraded the 'zroot' zpool to have feature flags (which now means it no longer has a version, apparently instead of jumping the numbers to distinguish from Sun/Oracle it has eliminated having version numbers (for beyond 28) and having flags for the features added since. Wonder if the flags capture all has changed since 28, since I thought there have been other improvements internal that aren't described by version numbers. Namely, I seem to recall that there have been improvements in recoverability....namely it had been suggested, when I was trying to recover a corrupt 'zroot' on mew, to try finding a v5000 ZFS live CD. Which I don't think I ever found, and gave up anyways when I concluded the level of corruption was too great for any hope of recovery and that I needed to resort to a netbackup restore, before the last successful full get's expired. Though being that it was nearly 90 days old, the other two month fulls didn't exist due to system instability that eventually caused the corrupted zpool (eventually found to be a known bad revision of the Cougar Point chipset and a bad DIMM...things seem to finally be stable from using a SiI3132 SATA controller instead of the on board, and getting that bad DIMM replaced....was weird that it was a Dell Optiplex 990, purchased new over a year after the problem had been identified and a newer revision of the chipset was released. I did eventually convince Dell support to send me a new motherboard and replace the DIMM. The latter was good, since I had to use DIMMs from another Dell that had been upgraded, so I had less memory for a while. But, while at first I did use the onboard SATA again, eventually I started having problems that would result in losing a disk from the mirrored zpool, to eventually causing a reboot where they would both be present again [though gmirror would need manual intervention]....and moving back to the SiI3132 has finally gotten things stable again. Though the harddrives in mew are SATA-III, so it would've been desirable to have stayed on the SATA-III onboard ports, where it was these ports that were the main source of problems in the prior defective version. Perhaps the fact that the prior version had a heatsink and the new version didn't, wasn't because they didn't need it to try to compensate for the problems caused by over-driving the silicon for the SATA-III portion. But, an oversight with the newer revision motherboard. The problem did tend to occur in the early morning hours on the weekend, when not only is there a lot of daily disk activity, but there is also a lot of weekly disk activity, etc. Oh well.)

So, after upgrading the zpool, and reinstalling the boot block/code. I then rebooted the system again. I had already identified the zfs filesystems where I had 'compression=on', so had written a script to change all these to 'compression=lz4'. Which I now ran.

Now instead of subjecting some poor random forum to a long rambling thought, I will try to consolidate those things into this blog where they can be more easily ignored profess to be collected thoughts from my mind.