Hosting von

Donnerstag, 30. April 2015

Avocent (formerly Cyclades) is a supplier for various datacenter management tools. They are best known for their rackmounted power distribution units and their serial console servers. Both devices run Linux and have been around for years.

Both the now EOL'd devices from Cyclades as well as the newer devices from Avocent can powercycle devices either through a serially attached smart PDU or through IPMI. Every device under the Advanced Console Server (ACS) label can control IPMI devices with a recent firmware.

While the functionality of the attached PDUs is quite well documented, there's no matching documentation for the IPMI interface. The web-interface works but the logic is mostly inside the AcsWeb webserver binary.
There's a cyc_ipmicmd binary but that one doesn't offer any --help functionality to explain how to call it.

For future reference here's the missing man page:

cyc_ipmicmd(1) General Commands Manual cyc_ipmicmd(1)
NAME
cyc_ipmicmd - utility for power cycling servers via IPMI
SYNOPSIS
cyc_ipmicmd SERVER COMMAND
DESCRIPTION
The cyc_ipmicmd utility is a wrapper around /bin/ipmitool which allows to send IPMI
power commands such as On, Off, Status and Cycle to configured devices.
The server address as well as necessary authentication data is taken from
/etc/IPMIServer.conf.
CONFIG FILE FORMAT
The /etc/IPMIServer.conf file contains the necessary data to successfully send IPMI
commands to remote devices.
Each line contains one remote server definition with the following colon separated
fields:
- Numerical server ID (starting at 1)
- IP address
- Authentication Type (none, password, md2, md5)
- Access Level (user, operator, admin)
- Username
- Password
- Alias (human readable name)
An example line might look as follows:
1:192.168.0.1:password:operator:user:pass:Example Server:
SERVER parameter
The server parameter is the numerical server ID taken from the first field of the
configuration file.
COMMAND parameter
The command parameter is numerical code which specifies which command is being
sent to the remote IPMI device.
0 Off
Poweroff the server
1 On
Poweron the server
2 Status
Reportpower status
3 Cycle
Powercycle the server
EXAMPLES:
Powercycle the first server:
cyc_ipmicmd 1 3
AUTHOR:
Andreas Thienemann

Freitag, 24. Januar 2014

I'm trying to add TPM support to the initrd images on Fedora 20. The idea is that the cryptokeys for the harddrive encryption are only handed out to cryptsetup as long as the whole bootchain is unmodified.
The tpm-luks package seems to be able to do the trick but needs to be adapted to the current boot procedure where systemd is part of the initial ramdisk image.

rd.break - Break out into a shell at pre-defined points in the bootprocess and continue after exiting the shell.

Especially the shell is a good helper but I was looking for something which allows me to poke around while systemd is presenting the user with a password prompt. For this I wanted a shell into the initramfs which allows for out-of-band access. A simple sh started in the background with input and output redirected to /dev/ttyS1 should do the trick I decided.

A nice feature of dracut is that it allows to include arbitrary files into the initramfs through thee dracut injection feature. This way a new systemd service file can be added which will take care of starting the serial shell:

Donnerstag, 16. Mai 2013

Introduction

dss_cli is a small command line program written in Python which can serve as the base for automating tasks on the Open-E Data Storage Server. A sysadmin can use it to control regular maintainance from the shell instead of having to log into the web-interface through a browser.
It can access the existing API via SSH and provides missing functionality by interfacing with the web-server on the DSS appliance. It is using both mechanize and Beautiful Soup to make it resiliant to changes in the webinterface. While it was originally written on a DSS v6, initial tests showed that it mostly works on the DSS v7 release as well.

Background

The "Data Storage Server" from Open-E is a linux based software appliance. After installing the software on a server, the server can then offer NAS and iSCSI storage to attached clients and is manageable through a web-interface.

One interesting feature of the appliance is, that it does offer failover for both iSCSI exported block devices as well as for NFS shared folders, something which makes it very interesting for Bawue.Net. The active/passive failover pair should give us better availability for maintenance as one half of the failover pair can be taken down for maintenance without affecting the virtual machines using the filer as a storage.

During testing of the DSS v6 system we did notice however a certain lack of functionality: The webinterface is great to manage the servers, create volumes, export these and set them up for replication. But using the webinterface is a manual process full of repetitive steps while the tasks at hand call for automation to reduce operator errors and to allow configuration through tools like puppet.

In order to help with automation, the DSS appliance offers an API/CLI access via ssh: Generate a key, connect to the server via ssh and pass some commands:

Unfortunately, the API is incomplete: It does allow for a lot of automation tasks, it does not export all the functionality to create working failover volumes and destroy them again. If there are plans to use the DSS filer as a storage backend for any kind of automated creation of virtualized servers these functions are sorely needed to prevent the need for manual interaction.

In order to address this lack of functionality, I wrote dss_cli, a command line client aimed at owners and administrators of DSS appliances in order to support all daily administration tasks needed on these filers.

Future plans

Provide a second tool to combine common steps for creation of iSCSI and NAS targets in a cluster.
Otherwise I am also taking nominations for needed functionality.

Installation

The current code is available on GitHub::ixs/dss_cli and is published under the GPLv2. Preqrequisites to running the dss_cli command is a recent Python installation with the Paramiko module for SSH connectivity and mechanize and Beautiful Soup for the web-scraping functionality.

Installation is simple: Download the latest code, unzip it in a new directory and edit config.ini to reflect your environment.
The [failovergroup] section contains your failover pairs, one group per line.
The example below defines one failovergroup called main, containing the dss1 and the dss2 filer.
The [dss1] and [dss2] section define their address, their admin passwords, the ssh_key needed for the API functionality and whether they are the primary or the secondary host in the failover group.

Usage

./dss_cli --help
Usage: dss_cli [options]
Command Line Interface to interact with an Open-E DSS Storage Server
Options:
-h, --help show this help message and exit
-f FILE, --file=FILE Configuration file to use
-l, --list List all commands available
-g, --failovergroup List all configured failover groups
-d, --debug
Use --list to get a list of all supported commands. Each command should
support the --help parameter to get a list of accepted arguments.

Running ./dss-cli -l dss1 does give a list of all commands supported on that device:

Example

The following commands would serve to create a failover iSCSI volume on dss1 and dss2:

Create the logical volumes on both filers as part of the arc_vol_000 volume group. Command line arguments are create_iscsilv <vg_name> <size> blockio
The size argument is specified in 32MB blocks. 150GB * 1024 / 32 = 4800.

Assign the created volume to the just created iSCSI target on both systems. The server will report back with a randomly generated SCSI id for the LUN. Make sure to pass this one when assigning the volume on the secondary system. These ids need to be the same.

Donnerstag, 23. Februar 2012

As I am travelling quite a bit I am often in the situation that I have a hotel room with a network cable and Internet access, but I have a notebook and a smart phone with me and the only device with an Ethernet connector is the notebook.

When I saw the ASUS WL-330N3G wireless mobile router I immediately bought the gadget. It's a small and portable router with 3 main uses for me:

Connect a 3G stick to it and the router will share the UMTS Internet via cable and a wifi network.

Connect it to the Ethernet cable from the hotel and use the internet via the wifi network.

Connect the router to a wireless hotel hotspot, authenticate and then use the single device hotspot account for accessing the Internet from both wired and wireless devices.

When playing around with the router you can brick the device (e.g. uploading incorrectly rebuilt firmware from the GPL sources).
If this happens, the device will not boot correctly any more but will be flashing it's power LED and possibly the network activity LED as well. The device can be put into rescue mode however where it will take a tftp uploaded firmware file and flash it:

Connect the router to the Ethernet port of your computer.

Connect the router to your USB port or the supplied USB power adapter.

Press the restore button on the back of the device and hold it for 10 seconds or so.If you are running tcpdump on the Ethernet interface you should see ARP requests from 192.168.1.1 for 192.168.1.20.

Use tftp to upload a firmware file to the device:Under Windows this can be done with "tftp -i 192.168.1.1 put WL-330N3G_1.0.2.0.trx".Under Linux you can use the "tftp 192.168.1.1" command and then send the following commans: "binary", "trace", "rexmt 1" and finally "put WL-330NG_1.0.2.0.trx" to upload the firmware image.

Wait...

The tftp server on the device will send OACK and reboot

At this point the device has recovered and you'll be able to log in again...

Dienstag, 13. September 2011

Sometimes you want to see if a connection to a remote system is still alive or you just want to keep it alive by transmitting some data.
I've found the following little shell one-liner to be quite useful:

Even though I wasn't at FrOSCon for Fedora but for my employer Booking.com, who was one of the FrOSCon sponsors, I still managed to spend quite some time with the fellow Fedora people. I even got roped into providing a talk during the Fedora Activity Day.

As a topic I chose func, a remote execution framework we have been using quite successfully at Booking for automating a lot of processes.

Having seen these, I naturally had to try the new F15 release for myself to see what this is all about.

Having booted it up in a virtual machine I am now ready to proclaim that Fedora 15 does not suck. The system greets me the same way prior releases did and the only noticable difference is a different release name and newer kernel release.

However, in the few minutes I spent with the new release I have to say that I already found two bugs:
After switching to the graphical login screen with gdm3, the screen becomes garbled. Suddenly the background is full of scanlines reminding me of the old days when I didn't had the correct modeline in the /etc/X11/X11.conf file.
I was assured however on IRC, that my hardware is not broken and this is the normal design for the Alpha release and the final one will not destroy my display. Thank god for that.

The other bug is that after ignoring the garbled screen and actually logging in, the desktop has become totally broken and hideous. Black and grey bars are alternating and hurting my eyes, functionality is missing, I have to switch constantly to the command line to change settings etc...

Again however the friendly people on the IRC channel came to the rescue and informed me that this is a known bug (tracked in the ohh so aptly named F15GnomeFAIL tracker) and it is actually not a software bug but human error.
It seems I just picked the wrong iso image thinking that the Desktop-ISO is actually for the Desktop. It looks like this is a common mistake and the real Desktop Spin is further down on the Download page. Silly me, we obviously have to better educate our Target Audience to pick the right image.

While I am actually very interested in seeing what the Gnome3 shell actually has to offer, this is not the release to do so. I seriously ask myself if this is the most unfinished and broken-by-default Fedora release there ever was. The Go/No-Go meeting should have just taken the hard decision and delayed the F15 release by 6 months. Maybe then this would look more like a release and less than a trainwreck.
I am basically disgusted at this point.

Freitag, 28. Januar 2011

Snom is the maker of pretty decent VoIP phones running Linux. I have had a Snom 360 for some time now and am reasonably happy with it.

The Snom phones do support SNMP but their SNMP daemon is severly limited. It only supports GETs on a small number of OIDs, doesn't support WALK and standard MIBs like the system-MIB are not supported. The Snom Wiki has a list of the supported OIDs and a description how to enable SNMP on the phones.
The limited support makes autodetection by network management systems or MRTG's cfgmaker fail. In order to chart this data, a manually created template is therefore needed.

Traffic Monitoring (bytes) a Snom Phone

The Snom phone exports all it's interfaces aggregated. This means all vlans and locally generated traffic. The only traffic not exported is the traffic generated on the loopback interface and the traffic bypassing the phone completely via the internal switch. The latter means that the traffic of the machine connected to the PC/passthrough port is not monitored.
The MRTG template to chart the incoming and outgoing bytes is the following. The IP Address 192.168.2.124 would have to be changed, as well as the descriptive details.

Some other values worth charting could be CPU load and free memory or the number of registered extensions. This could be useful for tracking down errors. Unfortunately, mrtg is unable to chart this correctly out of the box and needs some help converting the data.
This is therefore left as an excercise to the reader.

This USB passthrough feature is also available with many other desktop virtualization solutions, e.g. KVM and Qemu. Nevertheless it seems VirtualBox is favoured by a large number of users who are installing VirtualBox only to find that they cannot actually make their USB devices visible to the guest operating system. The common problem seems to be that they checkboxes next to the devices are grayed out, preventing the user from marking them to be added to the guest.

There are a large number of forum articles and blog posts available which all claim to have a solution to the issue. Very often the suggested solution is to change the mount options for /proc/bus/usb in fstab or add an appropriate entry. Sometimes it is suggested to mount usbdevfs to /sys/bus/usb/drivers. Some report success by editing certain udev rules so that files the in the procfs belong to the user executing the VirtualBox binary.

All these solutions have one thing in common:They are all wrong!
The fact that they are mindlessly repeated by posters in a large number of user-centric web forums does not help at all.
It is still wrong!I said it before but it still is true: Web forums are full of cargo-cult users: No idea what they are doing but trying and talking about it in the hope that it will achieve something.

The right solution is actually very simple. All that is needed is to add the user running VirtualBox to the vboxusers group:

Afterwards we returned to the Hotel and spent a few hours in the lobby, chatting some more with the other Fedora people sharing the same hotel. As the night was still young we moved outside to a few tables in front of the hotel, enjoying the warm summer night and exchanging some more gossip and ranting about the things in Fedora which made us unhappy.

After all, a very nice first day where I could catch up with old friends and had the ability to make some new acquaintances.

Samstag, 27. Juni 2009

I enjoy you immensly, it's been great meting old friends and making some new ones. Furthermore you are perfect for catching up with some former colleagues, other developers and for talking about lingering issues in a much more sensible setting than a mailing list filled with people with too much time for pointless bickering.

Ignore the people claiming that you're at the wrong time, the wrong location or the wrong anything. Sure, you're not always next door and sometimes you're even at the other end of the world. I won't be attending you in such cases, but there's always a FUDCon closer by which is worth it. Ignore the haters, they are just cramping your style.

But talking about style, I would really, really pretty please with sugar on top have you offering a more relaxed setting for conducting chats between a small group of people or just one-on-one talks. You have something called a lounge, but it's not really conductive for staying longer, the chairs are horrible. And your little brother FUDPub is too wild. No time there.
So please FUDCon, improve your style a bit and make your "lounge" a real lounge. I'll love you for that even more.

Dienstag, 9. Juni 2009

I've been traveling to Amsterdam today and as usual for airtravel, you spend an awful lot of time with the security theater. Today cost me about ~15 minutes. Unlike the normal horror stories however, today was a notable exception. In fact, what happened today at the security checkpoint of the Stuttgart airport was a very interesting experience.

I was asked to take my notebook out of my bag and put it on the belt by itself. Easily done. Usually the security guys ask you to switch it on for a moment. No idea why that is though.
Anyway, this time it was a bit different, the security guy asked me if the notebook sporting the Linux advertisement (lovingly applied by Alex Maier) is actually running Linux. After confirming this and stating that it's only natural as I've been with Red Hat in the past, was wearing my Spacewalk Hacker shirt and am still doing Fedora work, the guy was very happy as he seemingly could vent his frustration with Linux at someone knowledgable.

The security guy was telling me that he's been using Linux in the past, but it's just not user friendly enough. His pet peeve was the need to mount and unmount removable media.
Especially the unmounting was a major hassle for him as it makes working with the system difficult for the inexperienced user.
I explained to him the technical reasons why the system is acting as it is, how the filesystem cache is playing it's part and the need to sync data before being able to remove a disk and how the eject button on his cd drive is disabled. After having explained the technical details of the kernel, I told him that the current Desktops do automounting of CDs and other removable media but that I wasn't sure about removal. This should at least partly solve his problem with the mounting. About unmounting, I have no idea as it has been more than 5 years since I last touched a CD. Fast Networks and PXE all but obsoleted optical media for my use-cases.

After this was cleared up, he mentioned another problem he considered important: The claimed amount of technical knowledge needed to expertly use linux.
I tried explaining my point of view, that a certain amount of technical knowledge is immensely helpful in order to understand the system and thus being able to spot problems and fix them accordingly. Without the technical knowledge, which can be picked up rapidly by the way, the user would be forced to always get help from a third party for each small problem. Not optimal either.
At the same time however, I stressed the fact that the current distributions are all trying very hard to be usable, even for the inexperienced user.

As a good deed for the day, I mentioned that F11 is being released today and that he should give the Live-CD a shot, he might like it. The security guy was countering that the Live-CD might be nice, but what he would really like is a Live-USB media with persistent storage. Luckily, Fedora can score big time here and satisfy that requirement: Live USB with added persistence was one of the main features touted at last year's Linuxtag.

As I had to leave for my plane which was starting to board, I left a business card with my personal email address and asked
the guy to please report back on his experience with the Live-USB. Feedback is always good, especially if it is about a failure in our system.
It might give us some nice usability data and show where we can improve our documentation or presentation.
Anyone in Fedoraland interested in the possible follow-up?

But as interesting that chat was, no good deed goes unpunished: I'll have to find out now where these imbeciles in Schiphol lost my luggage with my documents and all clothes for tomorrow's meeting and my hayfever medicine.

Sonntag, 7. Juni 2009

Today it's voting day for me. As Hendrik already mentioned, it's time to vote for the European Parliament.

Besides that, I had to fill in three local ballots. One for my town council, and two times for the regional council.

But there's an even more important vote going on: Fedora has three elections running, one for the next Release name, one deciding about 5 seats on the next FESCO and three seats for the Board.

Unfortunately my preferred candidate, Cthulhu, wasn't running for any of the elections which meant I indeed had to settle for the lesser evil.

In case you haven't voted yet, do so at https://admin.fedoraproject.org/voting/.
You can find out about the candidates and their platforms at the nominataion page for FESCO or the nomination page for the Board.
Another good way of understanding the candidates and their intentions is the Questionaire every candidate was asked to fill in. This was the first time we've ever tried gathering questions from the Fedora contributors and submitting them to the candidates. The answers are very interesting and can be found in either plain text or as an OpenOffice Spreadsheet. My suggestion would be to look at the spreadsheet, it's easier to compare the different stances of each candidate.
And then, there are of course the Town Hall Meeting transcripts. The #fedora-townhall is a moderated IRC Chanel where the candidates are taking questions from the audience and try to answer them. The logs can be found on the normal Fedora Election pages.
There are a lot of worthwhile candidates, not the least your's truly. So don't waste your vote.