Friday, June 24, 2011

We saw earlier how easy it is to set up munin in FreeBSD. Now, what if a system that you wish to monitor is located somewhere beyond your firewall(s) perimeter ? You could install munin-node and let the whole world to grab your system statistics or become a victim of a future exploit! You could on the other hand tunnel the traffic via ssh.

Openssh has the ability to create a tunnel to encapsulate another protocol in an encrypted session. Which means that you can pretty much pass any traffic you want, even bypass firewall restrictions. Lets try it out. First you have to set up munin-node on the target host.

host# telnet localhost 4949
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
# munin node at host.my.domain
Now lets try to set up the tunnel. From the munin master initiate a ssh tunnel

ssh -2 -N -L 5000:host.my.domain:4949 user@host.my.domain
What we just did ? We created a tunnel to host.my.domain as user and from tcp port 5000 to tcp port 4949. Try connecting to the localhost from munin-master.

host# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
# munin node at host.my.domain
Beautiful! Now lets add this node to the monitor servers. Edit /usr/local/etc/munin/munin.conf in the munin-master, and add the new host.

[this host]
address 127.0.0.1
use_node_name yes
[host.my.domain]
address 127.0.0.1
port 5000
use_node_name yes
There a couple of things though that need improvement. First, you have to type a password and second what happens if the ssh session is terminated.

Setting up ssh with key exchange
Setting up ssh authentication with key exchange is not only easier, it is also more secure. Log on to the master-node with the account you wish to create the key and issue the following command:

gkontos@hp>ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/gkontos/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/gkontos/.ssh/id_rsa.
Your public key has been saved in /home/gkontos/.ssh/id_rsa.pub.
The key fingerprint is:
ed:3e:d7:62:48:7e:2b:f1:d5:94:e3:13:ee:7a:fa:aa gkontos@mydomain.loc
The key's randomart image is:
+--[ RSA 2048]----+
Good, now you should 2 files, one is id_rsa.pub and it contain your public key, the other one is id_rsa and it has the private key to encrypt data to the remote server.
To enable auto ssh login without being prompt for a password, create the ./~ssh/authorized_keys2 on the munin-node and copy the public key into it. Try logging again

ssh -2 -N -L 5000:host.my.domain:4949 user@host.my.domain
You should now be able to log in without typing any password.

Maintain the ssh tunnel
Autossh is a nice program that monitors and restart ssh sessions and tunnels. It is also very easy to install.

autossh -2 -fN -M 20000 -L 5000:localhost:4949 user@host.my.domain
That should do it. If you check you will see that the tunnel is up. Autossh will monitor the connection and will attempt to connect again if it is lost. So, unless you reboot the machine your tunnel will be up for ever.
I was thinking of writing a startup script because on occasions I do reboot my machines. But because I m too lazy I use a crontab entry like this

Thursday, June 23, 2011

There is always a need to monitor your systems performance. System statistics can help you fine tune a system and can also warn you of possible issues that could lead to a system misbehavior. Munin is a very nice tool for graphing system statistics using the rrdtool and it is based on a client server model. Munin master is used to collect information from Munin nodes. Fortunately Munin has been ported to FreeBSD.

On the Munin Server, let's install the Munin node port on the same system since we want to monitor it as well:
# cd /usr/ports/sysutils/munin-node
# make install

Note: install munin-node port on all machines you wished to monitor.

Now, we have to configure our webserver. In this case I assume that apache is being used and Munin has been installed in /usr/local/www/munin:
# vi /usr/local/etc/apache22/httpd.conf
### [START] Munin
Alias /munin "/usr/local/www/munin/"

<Directory /usr/local/www/munin>
Options none
AllowOverride All
Order Deny,Allow
Deny from all
Allow from all
</Directory>
### [END] Munin

On the Munin Server, you can see AuthUserFile is required:
# cat /usr/local/www/munin/.htaccess
AuthUserFile /usr/local/etc/munin/munin-htpasswd

Plugins that munin-node uses are usually to be found in /etc/munin/plugins (or /etc/opt/munin/plugins). If the directory is empty you will need to fill it. The directory should have been filled by the package installation script or by you when you read the INSTALL instructions.

You can fill it manually by symlinking to files in /usr/share/munin/plugins (or /opt/munin/lib/plugins). Or automatically by running munin-node-configure --shell | sh -x. This will which plugins it thinks are suitable on your system and make the symlinks.

After making all the symlinks restart munin-node.

Then wait 5 to 10 minutes before re-loading the munin web pages to see graphs.

Did you restart munin-node?

Restarting munin-node is a rather heavy operation requiring running all the plugins as part of the startup. Therefore munin-node does not restart itself when the contents of the plugin directory changes. So after making a change in the plugin directory you need to restart munin-node.

There is a bug in a good number of versions of the Debian (and Ubuntu) munin package that did not restart munin-node after running munin-node-configure. A manual restart is needed in this case.

Then wait 5 to 10 minutes before re-loading the munin web pages to see graphs.

Inconsistent names for the node on the master and on the node

If your node has plugins and is restarted the next possibility is: it is likely because the server and the node have inconsistent name information for the node. Munin wants the host names to match between its configuration and what the munin-node calls itself.

This means that this machine knows itself to be named lorbanery.langfeldt.net. If the name shown is not what you expected, you need to configure the correct name in munin-node.conf:

host_name lorbanery.langfeldt.net
On the master you configure lorbanery like this:

[lorbanery.langfeldt.net]
address 10.1.0.2
This makes the names identical. If you had put lorbanery in the square brackets the result would be no graphs because munin expects the whole name to be the same, and the whole name isn't lorbanery.

Restart munin-node, then wait 5 to 10 minutes before re-loading the munin web pages to see graphs.

I just read the above answer and there still aren't any graphs

Then it's time to get more advanced. Consider the following protocol exchange for lorbanery aka lorbanery.langfeldt.net. You should of course use the host name of your host, as configured in the munin.conf file. Please make very sure that you use the whole and exactly the same filename as configured in munin.conf.

This is the actual exchange used with munin-nodes that understands the nodes command. The nodes command asks the munin-node which hosts it has information for, then asks it to list the plugins that represent lorbanery.langfeldt.net. Lastly it fetches the df results from lorbanery.langfeldt.net.

If the output of the list command with the host name behind is empty, there are no plugins installed for that host. And that's the reason there are no graphs.

If there are plugins listed by the list command then you have some other problem. Please contact the users mailing-list.

Node does not "allow" master to telnet

You have not added allow to node's munin-node.conf file. See munin-node.conf. Ensure you use the reg ex syntax as prescribed there.

Saturday, June 4, 2011

The Valentines day for 2011 is just a Monday away. Do you have a date? Or do you ever experience dating a programmer? Somewhat like a computer scientist. Why try to date one?Don't date a programmer.

Programmers are geeks. They are nerds. They are anti-social. They are hackers and will just hack your Facebook account. They are self centered. They spent their whole time in front of a computer, typing some weirdo code. They end their line with a semicolon. They are just plain weird.
But if you still insist and want to take the risk, do some adventure and try it. Try and date a programmer!

Maybe it is an adventure, because a programmer, a computer scientist is no ordinary person. Yes, maybe they are geeks and nerds, but have you ever heard of Chuck?

He's a nerd, just do the math.

Programmers think outside the box. They use their critical thinking skill to formulate some algorithms. It definitely needs going out of the box to do it. Thus it solves the problem of cliche. They are wild thinkers and can execute an outside of the box move. A move that is new to the eyes, definitely not a cliche. Who says proposing using a game is overused?

If thinking outside of their box is not enough, how is thinking inside the other box? How about thinking inside your box? Programmers need rigorous testing for their programs to work well. To do it, the programmer must simulate the position of the projected user and use their program. It is thinking like the other person.

With it, that programmer can see things in your perspective. A perspective different from his, but he will definitely understand the way you think.

They also understand the concept of strict rules. Programmers need to have the correct syntax in their program for it to compile and run. They understand how important rule is and not following it can lead to a bad result.

Unlike some people, they also know that a certain "if" word exist and it can always be there. They know that some things or event might not happen with that "if". Life is full of uncertainty and you will never know what will happen next. When an "if" event happen, they know how to accept it.

Everything happens for a reason, and programmers know it. Every effect has a cause, and every cause has an effect. A small problem that is disregarded might be a bigger problem and may affect later. A problem ignored, is not a problem solved. Programmers also know that perfection is such an impossible to achieve. Because they know, that even though their program is well made, sooner or later, a bug will appear. A problem that needs an update to fix. A problem is better solved than ignored.
Computers do not understand the language of the people, but the programmers can understand the language of the computer. Programmers are flexible. Try to date a programmer and you will not have a hard time fitting in, because they will understand you with ease.

Simple things are appreciated by programmers. And they do not take it for granted. It is because the programs made by them are somewhat taken for granted. The simple interface of the program is just a mask to a much more inside. The complex code that is written by the programmers is hidden to the user unlike the wires and circuits of engineers that can easily be seen by naked eye.

Yes, those are some qualities of a programmer. Programmers that are sometimes stereotyped by misconceptions.