Meta

Archive for tech

To date we have had 4 of 5 linux fedora 23 updates completed, so far all without issue.
Of particular note was an update from a troublesome system running fedora 21. Over 20 attempts to upgrade this to fedora22 failed due to some mounting issue failure during the upgrade. After each failed attempt, I decided to stop and wait for fedora 23 to come along.

Eventually it did, I tried it and it failed.. 5 times. But at least the error messages were different this time.
Seemed to be a conflict with .f22 and .f23 packages. It would fail after “running transaction test” and reboot
I tracked down /var/lib/dnf and rm *fc22* -f and tried it again…
Success!
It went passed the transaction test to the Running transactions. 3839 steps later (and 50 minutes later), it restarted into fedora 23 without error!
What a relief. Powerup to login time is 53 seconds. Could be less with a SSD Boot drive… will think about that in the future.

It’s always a good day when a new Fedora Linux gets released. Today was Fedora 23. There was no real rush to upgrade… we were already on kernel version 4.2 and in all but one case, everything was working well and smoothly.

The one exception was a fedora 21 installation that absolutely would not upgrade to version 22. It would get midway through the process, and reboot back into 21. Very frustrating the last 6 months or so.

The goal now was to skip 22 and upgrade directly to 23. But first, an upgrade on a couple of noncritical systems first, just to test out the process and learn what to expect.

I took a look at it was 27% of the way downloading approx 2.7GB of windows 10.
It completed and it was time to start the upgrade.
at 12:10 the system restarted and the install began
at 12:47 it restarted, 30% complete
at 13:13 it restarted, 75% complete
at 13:25 it restarted and I logged in
at 13:33 the desktop appeared, but with only 1 display at a low resolution
another restart and it found the nvidia geforce GT430 drivers, card and 2 displays and set them to the correct 1680×1050 pixel resolution.
Overall about 1.5-2 hours after download and it was complete.
Over the next 20-30 minutes or so it was slow to respond. mouse movements, clicks, etc were all slow. Disk I/O was 100%
Now, 1 hour after, CPU use is low, about 10%, memory use is 2.5GB (or 63% of the 4GB onboard), Disk use is now much lower 0-10%

A quick test of must have programs showed all to work: thunderbird, stellarium, ms office 2013 (after 5 minutes of reconfiguration), adobe reader, firefox, chrome and more. So far so good.
Keyboard and mouse responsiveness are also better now. Pretty much the same as before with Win7.

We have successfully updated from Linux fedora 21 to fedora 22, 5 systems now, all without any issues whatsoever. The 6th and final system however had failed, 10 times now.
The upgrade process starts off well with
fedup –network 22
downloading 1800+ packages and completing successfully
The next step is to reboot into the upgrade.
That works.
Welcome to Fedora 21, reached target local file system, mounting drives, reached target system upgrade
Then it reboots back into Fedora 21, too fast to see what error was on the screen.

Time after time this repeated. Logs showed nothing helpful.
In desperation, I videod the screen and watch it frame by frame in playback and came across these errors:

Linux Fedora 22 was released this morning and two servers have been upgraded without issue… the smoothest upgrades yet.
Total time to complete the upgrade has been about an hour. About 10 minutes of that was the actual download of the 1000+ packages, and the remainder of that was the reboot and actual installation of packages.

The process goes something like this:
“yum update” to get to the latest point of the old stuff
“df -h” to make sure there is enough room everywhere, especially in the /boot partition
“fedup –network 22” and off you go
about 10-15 minutes later the 900-1300 pacakges will have been downloaded, and the system is set to reboot into the upgrade process
reboot
packages get installed, then the cleanup then another reboot.
The servers come up all the way as they are supposed to. DONE!

Both are now running Linux kernel 4.0.4-301.fc22.i686+PAE
yum has been deprecated so now we use dnf instead.

As a side note, our normal method to ssh in has broken. Teraterm v4.74 now comes up with an error
unexpected ssh2 message(80) on current stage(6)

I tracked down an upgrade to v4.86 and it too shows the same error.
Time to google “programs similar to teraterm” and find something called putty
putty works! it is also configurable and will probably be the default terminal client until teraterm gets fixed.

Updated: 2015 May 27
add a third server upgrade that went flawlessly. This one took about 55 minutes, probably due to its 1750+ packages that needed upgrading.

Updated: 2015 May 28
add a fourth server upgrade that went flawlessly. This one took about 30 minutes, with over 2500 packages that needed upgrading.

Our Internet Service Provider (ISP) has recently notified us that they have started using internet usage caps. Yay!
It gets better
They notified us halfway through the billing cycle. Surely that is an illegal practice.
Oh by the way, starting two weeks ago we are now charging for your use.
Needless to say we are not impressed.

What we have done is change our usage practice.
This may be noticed with the allsky1 system in particular.
We used to upload lossless .PNG format files and the resulting .AVI video of the nights images, plus an hourly update for those wanting to do what current sky conditions are in our area.
As of now, we are only uploading lossy .JPG files and creating the .AVI video on the server itself, rather than upload it.
Hourly updates have also been cancelled.
This should take daily usage from 100MB down to 36MB or monthly from 3GB to 1GB

There have been a few days of problems with the new software code needed, but I think it has been sorted out now.

Apologies to one and all.. we have been experiencing some web server hardware issues over the last few weeks.
Hopefully things are stable again for the time being.
We are looking at some new(er) hardware in the short term and will see how that goes.

Leo Laporte on Twit.tv was recently talking about a new internet connected talking Barbie with the infamous “Math is Hard!” scandal.
Well.. math is not hard and this is simple arithmetic.

The first nice weekend to come along and the allsky1 camera system gets a reno.
Currently the heater (running on a timer), is 33 ohms at 24 vdc creating approx 18 watts of heat for a dome that is 14″ in diameter.

The new dome is 4″ in diameter, having a lot less air to heat up. So we want to reduce the amount of heat generated.

The volume of a sphere is
V=(4/3)πr3 where the radius is 7″ and 2″.
7″ gives 1400 in^3 but this is only 1/2 a sphere, so 700 in^3
2″ gives 33 in^3/2 or 16 in^3, a reduction of 700/16 =43 times the heated air volume.

So we should be able to pull down the heat wattage from 18 down to next to nothing, ie like 1 or 2.
But then again there are other factors, like the thermal mass of the dome itself plus the camera body (which has a cooled camera, so we do not want to heat that up if possible.

Our server had a power supply fail, taking with it many sectors of the 320GB SATA hard drive. The bad sectors contained a lot of the mysql database.
Luckily we had a week old backup of the database and it restored ok. Unfortunately other databases did not fare so well and we are still forced to run under a limping mode innodb_force_recovery = 2 inside of /etc/my.cnf Now we try to identify bad tables and remove them to get the database server to run again.

In the meantime we are learning far more about databases then we would like to, yet not enough to easily identify and fix the problem.

The latest in the Linux Fedora distribution was released Tuesday, 2014 Dec 09.

It now comes in three major versions, workstation, server and the new cloud. Not sure what that is but what the heck. Progress!

We started with the most non-critical server and ran the internal upgrade process with:

fedup –network 21 –product=server

That ran for a short time, downloading a GB of so of files. A reboot followed by about 1 hour of installing and updating. Another reboot and Fedora 21 was up and running. Hmm. There seems to be an issue. Everyone on windows and imacs have lost file sharing to the server via samba.

Hmm.. a few hours of troubleshooting revealed two possible issues, one of which was the new firewalld process was active and blocking connections.

That was fixed with the command

firewall-cmd –add-service=samba

and firewall-cmd –list-services

to confirm what was allowed through.

The 2nd issue was that most client computers needed to have the static IP of the server put into their hosts files before the file sharing connections came back.

Still not sure why or if this was a side affect from the samba Domain Controller being updated.

And as of yet, have yet to try out some of the new Fedora 21 features, like cockpit. Sounds interesting.

Adventures in hard drive cloning.
Starting point: a linux based file server with a 320GB SATA drive as a boot drive.
Overkill for sure as less than 20GB or 30GB was actually in use and needed.

In addition there was a WDGreen 3TB data drive and two WDGreen 1TB backup drives.

The issues at hand: The older drives generally take more power and we are trying to conserve; there wasn’t enough space on the two backup drives to backup the data drive; splitting the backups into two parts was a more complex system than it should be; rebooting times, while fast, could be better.

So we decided to replace the spinning boot drive with an SSD. However, there is always a problem trying to clone a larger drive to a smaller drive. Luckily, the 320GB drive was not using a lot of its space, so we rebooted into GPARTED, a safe repartitioning utility. The large root and home partitions were shrunk down to about 40GB each. Shutdown, attached the USB adapter to the 120GB SSD and used a cloning utility that came with the SSD. No luck. It aborted saying the source was bigger than the destination.
Well, that was expected.

Then we rebooted into Clonezilla, another utility, and went through the localdisk-localdisk option and it completed very quickly but with an error. Rebooting to the SSD showed a Grub error and failure.
Tried Clonezilla again and this time went with the clone partition-partition option on the three active partitions. That went well, no errors, and the resizefs part of the process worked well.

A reboot came up fast, and the login prompt was there in about 15 seconds (compared to about 60-70 with the spinning boot drive).

That went very well. Once the correct process was discovered it took maybe 15 minutes.
The old boot drive gets put into an antistatic bag and put on a shelf for awhile.

The next step in the project was to remove the two 1TB backup data drives, install the new 3TB 7200rpm drive and rsync everything from the current 3TB data drive to the new one. That took about 40 hours. When completed we ran the rsync again, to catch up on the last 40 hours of updated files.

Once reasonably sure all of the data was across, we edited the /etc/fstab file and swapped the mounting points of the two drives, saved and rebooted.
15 seconds later the main data drive was up and running on the newer, faster data drive.

The old data drive is now the new data backup drive (runs a little slower, buts that’s ok), with an uptodate copy of everything.

A couple more minutes to reenable the backup bash scripts and all is operational again.

End point:
The new system should be faster to start, faster to respond, faster to read and write data to, enough space for the data backup and use less power to boot.
Two 1TB drives average 3.3W each = 6.6W – 3TB W 4.1W = 2.5 Watts saved plus 320GB @ 2.5W = 5 watts total less 120GB SSD @2W = 3 watt savings. Hmm.. not as much as I thought.

One minor task left is to reboot into GPARTED and reexpand the root and home partitions to completely fill up the SSD.

This is the Raspberry Pi, a very small computer that has some great potential in replacing larger desktop computers and use a heck of a lot less power.
I bought one a few months ago and finally got some time to plug it in and see what it can do.

The HDMI video went into the TV, a wired ethernet cable was plugged in, the wireless USB dongle was removed, a USB powered hub was connect and powered, a USB keyboard and mouse was connected, then finally, the Pi power was plugged in.

It booted up quickly into Noobs 1.3.5 and I chose Raspbian as a generic distribution to use. It reformatted the 8GB SD memory card and installed itself over the next 20 minutes, then restarted.
Still works!
Fired up a terminal program, got its IP address and SSH’d in from another computer. Still so far so good. Updates followed with sudo apt-get update then upgrade and 30 minutes later it was uptodate.

We then changed the dhcp address to static, rebooted and all was still good.
Used startx to fire up the graphical shell and still all was good.
Installed a remote desktop package with the commands: sudo apt-get install tightvncserver;
then ran tightvncserver to set the access passwords, then created a shell script to autostart it on boot: /etc/init.d/vnc.sh
rebooted again and was able to use an ultra-vnc viewer from another computer to remotely access the graphical desktop.
Wow! This is all going well!

The next step was to get the wireless usb dongle plugged in and configured and that worked as well.
That was done, then the wired cable was disconnected and the Pi restarted again. Still all is well!

Now we just have to find some purpose for it 🙂
Our first choice is to act as a data logger for our RadioJove radio telescope system. That will require RadioSkypipe windows software to run on linux. We will start checking out methods to accomplish that.