Archive for the ‘Programming’ Category

Sometimes we have to administrate this operating systems such as FreeBSD / AIX / HP UX or even Mac OS server where by default due to historical reasons or for security bash shell is not avialable. That's not a common scenario but it happens so if as sysadmin we need to create for loops on ksh it is useful to know how to do that, as for loop cycles are one of the most important command line tools the sysadmin swiss army knife kind of.

So how to create a for loop (cycle) in ksh (Korn Shell)?

The most basic example for a KSH loop shell is below:

#!/bin/ksh
for i in 1 2 3 4 5
do
echo "Welcome $i times"
done

Add the content to any file lets say ksh_loop.ksh then make it executable as you do in bash shells

$ chmod +x ksh_loop.ksh
$ ksh ksh_loop.ksh

The overall syntax of the for loop ksh command is as follows:

for {Variable} in {lists}
do
echo ${Variable}
done

Hence to list lets say 20 iterations in a loop in ksh you can use something like:

#!/bin/ksh
for i in {1..20}
do
echo "Just a simple echo Command $i times";
# add whatever system commands you like here
done

Example for some useful example with KSH loop is to list a directory content so you can execute whatever command you need on each of the files or directories inside

#!/bin/ksh
for f in $(ls /tmp/*)
do
print "Iterating whatever command you like on /tmp dir : $f"
done

Other useful for loop iteration would be to print a file content line by line just like it is done in bash shell, you can do that with a small loop like belows:

Most of people nowadays are using Google as a default search engine for different reasons, some because of its popularity and most importantly because Google is considered the Largest and most used and superior in quality Internet Search Engine in the World as of time of writting this post.

However as most (if not all) Companies and Corporations nowdays in this terrible freedom enslaving Globalistic world, Google's popularity is not only a result of their superiority of service but a consequence of 17 years constant branding (or better to say user brainwashing) to addict peoples arround the world to their colorful logo and their Loud name GOOGLE (contains 2G's 2O's 1L and 1E or as it might be associated subsconsciously = Le, 2 G, 2 O or LE 2 GO – looks pretty much like the English LET GO – doesn't it??) .

Color programming has become a popular way for companies and brands to keep our minds affected the Colorful addiction and has been definitely quite seriously researched throughout World War I and World War II and Post war times by Governments and Psychological institute studies some of which tried to call it and name it as NLP (Neuro Linguistic Programming) and made a pseudo-science that is more of a spiritual new age occult thing than really based on facts. I guess some of this studies were put in action by most if not all big companies to influence their customer / users.

Google's Logo

Some example that such a mind psyche manipulation techniques are reality could be seen by simply comparing the Color Gamma of Google's Logo, Microsoft Flag (See your start button logo), Ebay Logo, Google Play's logo, Microsoft (Windows)Flag logo, Windows Media Player, Google Chrome, Microsoft Visual Studio, Apple's Old Logo … etc. etc. and a number of key applications we use daily is not a coincidence but a smartly crafted methodology to make people regularly exposed to combination of colors be psychologically influenced and attracted by the Colorful flag message not only in these company Software programs but also in daily products we buy from Shops / Supermarkets.

Windows Flag (Also present on every Windows PC in the start button and during PC boot)

However color programming that Google inflicts (is not really a prooved science but is based on a number of occult gnostic believes combined with some pseudo scientific research on associative human thinking).

Hence these colorful manipulation / programming technique so much adopted by companies is believed to effect on adult people and kids throughout the world because even from kids we're being exposed to this colors in many children books, cartoons cartoons and most importantly by the market which produces this "interesting" entertaining fun colorful kid toys.

Apple's Computer Company Inc Old Logo

In other words these corporations are trying to influence and bring up a postive reaction in us by using these colorful logos propaganda because it has to do with how people were learned to think from the Kinder Garden onwards, especially in democratic countries and probably less in ex-communist totalitarian countries (because they were excluded more or less fromt he global rest of the world for many years), though color programming was partially used also in ex-communist contries too but in a less agressive way than in democratic countries.

With all said above I guess / assume Google (creators) or some of the management and CEO personal undertstood this people's natural daily inflicted love for colorfulness quite well not only because of its psychological materrial meaning but also because Google founders Larry Page and Sergey Brin have been told a mystical teaching about colors that is being preached heavily in recent years by pseudo scientific (occult) companies like Herrmann Brain Dominance Institute HBDI the infamous business module so much loved and adopted by modern businesses – (HBDI) as well as the colors hidden meanings as believed in secret (closed) societies such a masonry fraternities and their front organizations such as Rainbow Girls, Shriners etc..

Logo of Masonic Order Rainbow girls together with Masonic Square and Compass famous symbol

Probably the importance of exposed Google Logo colors is not a coincidence but same colors order is important for both Jewish (because of the Rainbow's importance in the Holy BibleOld testament as a sign of covenant God give between God and man as a promise the world will be no more destroyed by Water Flood), that is still strictly followed by Jewish.

Besides that the Google logo colors are also to be seen as one of central symbols of Free Masonry the Eastern Star (note that it is the same as the Satanists Pentagram), Partially in Gay Lesbian and Homosexual Movement's flag logo and many new founded companies that are adopting and embedding occult meaning it in their company brands exposing them selve who are their masters.

An important Masonic Star (Same is seen in Lavey's Church of Satan but in a circle)

Homosexuals movement logo (flag)

With all said, if you're like me and you want to continue use Google as a Search engine and not switch to Something like Russian's Yandex / Yahoo or DuckDuckGo etc. and want to keep using Google but disable annoying Color propaganda of Google Logo and the Google Doodle's that exposes you to people and events that are often anti-Christian (or never related to Christianity) but showing a pop-culture idolizing personalities many of whom are known for their liberal ideas and non-christian world view (e.g. indoctrinating you slowly but surely with secularism and globalistic) ideas, or if you want me to be "politically incorrect" to save yourself from Google Brainwashing.

Strangely perhaps not a co-incidence this colorfulness and order of color, very much resembles the Buddhist prayer flags used heavily by Buddhists (hanged on their large temples), I guess this is not strange since nowdays the Western culture is crazy of integrating Eastern teachings into their Post-Christian "enlightenment" world.

I've tried a couple of methodologies to get rid of Annoying Google colorful logo that started to be really irritating for me by using DOOD Remove Firefox plugin which was supposed to be able to allow me change the Google Logo site to whatever I like but the link pointing to the FF DOOD Remove plugin was broken, I then tried using a simple hack with GreaseMonkey and My Google Logo and Background changer on my Firefox Version 38.0.1 but as the script used to be last tested reported working on 18-05-2013 as of time of writting this article it didn't work nomore because of some changes in how google serve its Logo Images. I knew there is a way to also dump the logo using some of the embedded Firefox Javascripts but as I was lazy and don't have a good understanding on JS, I decided to not continue that way and instead tried another method with Adblock Plus's custom Filterint capabilities that is able to filter any picture or object out of a html web page.

UsingABP to remove Google Logo turned to be the easiest and only working solution to remove Google's Logo, to do you need to:

1. Have AdBlockPlus extension for Firefox installed first
2. Enter ABP Options menu by either right-clicking the extension's icon and choose Filter Preferences as shown in prior screenshot.3. Go to the Add filter Group button (Type Google and then, click Add Filter button and enter google.*/logos/doodles/, as shown in below screenshot

Then give it a try just open www.google.com and you will not see anymore the Google logo, unfrotunately the Logo flashes for less than a second until it is removed but this is the best I can achieve at current point.

I've also done a short research on how is possible to Remove Google logo if using Google's Chrome browser and it seems the only meaningful thing I found here is again DOOD Remove (extension) for Chrome.
I've give DOOD Remove for Chrome a try and it works fine, however when you want to open Google you get a short second flash of the Google Logo (after typing google.com in URL bar) before you get the logo wiped so though this works the solution is unfortunately not perfect, the other option to remove the Google Logo branding from searches in Chrome was also to use the Greasemonkey + Userscript as well as Tampermonkey + DoodleGone's script as explained in this post however after enabling TamperMonkey + DoodleGone, the Google logo still was appearing in a behind the search bar (so this worked not), the Google Chrome version where I tried this solution is Google Chrome 43.0.2357.81 (Official Build) m (32-bit).

So finally it turned out that in order to Disable Google's Kindergarden like looking Logo in Chrome (which is manipulately crafted to inflict kiddish feelings in yourself), I had to Adblock Plus Methodology as described above.
The ABP method to get rid of the ugly google logo also seems to be the only Method that works to remove the Google Logo on Mac OS's (Safari browser).

Hope my article helps others who want to Ban the Google Doodles Logo in your favourite web browser the only condition is to have the Adblock Plus for the browser.

It is a very useful thing if you have to administer a large group of Linux / UNIX servers and you're not running web monitoring such as Nagios / Icinga / Munin / monit to Raise Email or other kind of Alert / Warning notices to periodically check the system load avarage with uptime command and log to a file and if necessery send emails in case if some server too high load avarage is matched on servers.

Below little script could be a basis for development of a script to raise email SMS alerts in case if a Linux / BSD / UNIX system gets CPU overloaded.

Getting load avarage is easy with uptime command, however since nowadays Linux servers are running on multiple CPU machines and Dual cores, returned load avarage shows only information concerning a single processor. Of course seeing overall CPU server load is possible with TOP / TLoad command / HTOP and a bunch of other monitoring commands, but how you can get a CPU percentage server load using just /proc/stat and bash scripting? Here is hwo:

Something very useful that most Apache LAMP (Linux Apache MySQL PHP) admins should know ishow to check Apache Webserver uptime and MySQL server running (uptime).Checking Apache / MySQL uptime is primary useful for scripting purposes – creating auto Apache / MySQL service restart scripts, or just as a quick console way to check what is the status and uptime of Webserver / SQL.

My experience as a sysadmin shows that lack of Periodic Apache and MySQL restart every week or every month often creates sys-admin a lot of a headaches cause (Apache / NGINX / SQL server) starts eating too much memory or under some circumstances leads to service or system crashes. Periodic system main services restart is especially helpful in case if Website's backend programming code is writetn in a bad and buggy uneffient way by unprofessional (novice) programmers.
While I was still working as Senior SysAdmin in Design.BG, I've encountered many such Crappy Web applications developed by dozen of different programmers (because company's programmers changed too frequently and many of the hired Web Developers ,were still learning to program, I guess same is true also for other Start-UP Web / IT Company where crappy programming code is developed you will certainly need to keep an eye on Apache / MYSQL uptime. If that's the case below 2 quick one liners with PS command will help you keep an eye on Apache / MYSQL uptime

Note that the server you're working is shared with other developers they might be able to steal your username / password by using a simple process list command such as:

ps auxwwef
…

In my case security is not a hot issue, as I'm the only user on the server (and only concern might be if someone hacks into the server 🙂

Then assuming that you have a plain text file with all your administered servers, you can easily use sshpass in a Bash Script loop in order to run, lets say a package upgrade across all identical Linux version machines:

Change the command you like to issue across all machines with the string "apt-get …"
Above command can be used to keep up2date all Debian stable server packages. What you will do on servers is up to your imaginations, very common use of above line would be if you want to see uptime /netstat command output across all your network servers.

I've been working for a customer that has about 450 hundred servers running all kind of different Web Applications / Mail services / FTP / Web Statistics etc.. I have a single user / password access to all of the servers (using LDAP authentication) – just like many of my colleagues and by default most of servers shell set is Korn Shell (/bin/ksh) is automatically run on ssh login time ( as it is set as the default shell ). I have nothing personally against Korn Shell but as mainly script in Bash Shell last 10+ years and I'm used to Bash's terminal behavior so much, each time I login to the servers I had to run manually run :

/bin/bash

This becomes really annoying, because I have to login to so many servers daily and do various stuff, so I got pissed off at the end and I wanted to change default shell set (only for my current) user to BASH, since there are other users that work on servers and its not a good idea to change globally servers global shell to bash, because such action could hurt some poor colleague love for KSH 🙂

A note to make here is the servers are primary running SuSE SLES / CentOS and Debian and RHEL Linux, but this bash one liner would be working on any other Linux release which has a bash >=2.0, I've tested it exactly on bash 3.2.25.

Since SSH client doesn't have a non-interactive way to pass on password from command line (for security reasons), to make above bash one liner loop to deploy bash as a default shell (mass) on all servers I have to paste (my Sotred in Memory Copied password) the SSH remote login server password 450 times each time I',m prompted by SSH to input password manually (this takes about 20) minutes but it is a worthy time, since from then on /bin/bash runs automatically on server login time. One important note to make is the </dev/nullhere is necessery because otherwise ssh is having troubles with reading the host in the loop. If you try running above cmd without </dev/null you, the loop is about to end after first login attempt. Changing default shell to multiple servers is not done only to servers with default ksh, but for any servers with default csh shell or any other obscure shell to bash (if bash is installed) is achieved by above command.

As you can guess above while bash loop, can easily be modified to run any other command to a bunch of hundred or thousand servers of your choise, the only inconvenience you will have is you have to paste the password for each server part of loop to which ssh is creating connection.
This example works on Linux but should work fine on any other platform that have /bin/bash (or /usr/local/bin/bash) installed if on FreeBSD / NetBSD, it should work on HP-UX and SunOS / Solaris hosts which have bash installed too.

Once you've launched the loop and added /bin/bash to run on login to check on next login where you're really running bash shell there is the:

echo $SHELL/bin/bash

If you want keep / grep output of shell variable $SHELL:

MYSHELL=`ps -hp $$|awk '{echo $5}'`

Anyhow on some Linux hosts this variable might show bash and still you might be running old shell ksh / csh, the better way to check your SHELL is with ps -p $$ command (nowdays not known by many admins)

ps -p $$PID TTY TIME CMD
866 pts/0 00:00:00 bash

Now I can happily use bash on all the servers every time I login. Cheers and Thanks God ! 🙂

Above line greps for documentroot prints 2nd column of the matchi (which is the Apache server set docroot and then removes any " chars).

However I faced the issue that parsed string contained in $documentroot variable there was mysteriously containing r – return carriage – this is usually Carriage Return (CR) sent by Mac OS and Apple computers. For those who don't know the End of Line of files in UNIX / Linux OS-es isLF – often abreviated as n – often translated as return new line), while Windows PCs use for EOF CR + LF – known as the infamous rn. I was running the script from the server which is running SuSE SLES 11 Linux, meaning the CR + LF end of file is standardly used, however it seem someone has editted the httpd.conf earlier with a text editor from Mac OS X (Terminal). Thus I needed a way to remove the r from CR character out of the variable, because otherwise I couldn't use it to properly exec tar to archive the documentroot set directory, cause the documentroot directory was showing unexistent.

Opening the httpd.conf in standard editor didn't show the r at the end of
"directory", e.g. I could see in the file when opened with vim

DocumentRoot "/usr/local/apache/htdocs/site/www"

However obviously the r character was there to visualize it I had to use cat command -v option (–show-nonprinting):

There are plenty of Apache Performance Optimization things to do on a new server. However many sysadmins miss .htaccess mod_rewrite rules whole optimization often leads to a dramatic performance benefits and low webserver responce time, making website much more attractive for both Search Engine Crawlers and End User experience.

Normally most Apache + PHP CMS systems, websites, blogsetc. are configured to use various goodies of .htaccess files (mostly mod_rewrite rules, directory htpasswd authentication and allow forbid directives). All most popular open-source Content management systems like Drupal, Joomla, WordPress, TYPO3, Symphony CMS are configured to get use .htaccess file usually living in the DocumentRoot of a virtualhost ( website/s ) – httpd.conf , apache2.conf /etc/apache2/sites-enabled/customvhost.com or whichever config the Vhost resides…

It is also not uncommon practice to enable .htaccess files to make programmers life easier (allowing the coder to add and remove URL rewrite rules that makes URL pretty and SEO friendly, handle website redirection or gives live to the framework like it is the case with Zend PHP Framework).

However though having the possibility to get the advantages of dynamically using .htaccess inside site DocRoot or site's subdirectories is great for developers it is not a very good idea to have the .htaccess turned on Production server environment.

Having

AllowOverride All

switched on for a directory in order to have .htaccess enabled, makes the webserver lookup for .htaccess file and re-read its content dynamically on each client request.
This has a negative influence on overall server performance and makes Apache preforked childs or workers (in case of mpm-worker engine used) to waste time parsing .htaccess file leading to slower request processing.

Now CMS uses the previous .htaccess rules just as before, however to put more rewrite rules into the file you will need to restart webserver which is a downside of using rewrite rules through theInclude directive. Using the Include directive instead of AllowOverride leads to 7 to 10% faster individual page loads.

I have to mention Include directive though faster has a security downside because .htaccess files loaded with Include option (uses mod_include) via httpd.conf doesn't recognize <Directory> … </Directory> set security rules. Also including .htaccess from configuration on Main Website directory, could make any other sub-directories .htaccess Deny / Allow access rules invalid and this could expose site to security risk. Another security downside is because Include variable allows loading a full subset of Apache directives (including) loading other Apache configuration files (for example you can even override Virtualsthost pre-set directives such as ErrorLog, ScriptAlias etc.) and not only .htaccess standard directives allowed by AllowOverride All. This gives a potential website attacker who gains write permissions over the included /var/www/website/.htaccess access to this full set of VirtualHost directives and not only .htaccess standard allowed.

Because of the increased security risk most people recommend not to use Include .htaccess rules, however for those who want to get the few percentage page load acceleration of using static Include from Apache config, just set your Included .htaccess file to be owned by user/group root, e.g.:

A colleague of mine (Vasil) asked me today, how he can recursively chmod to all files in a directory whileexclude unreadable files for chmod (returning permission denied). He was supposed to fix a small script which was supposed to change permissions like :

Another useful Linux find use for scripting purposes is listing only all files presented in current directory (simulating ls), in case if you wonder why on earth to use find and not a regular ls command?, this is useful for scripts which has to walk through millions of files (for reference see how to delete million of files in same folder with Linux find):

find . ! -name . -prune

./packages
./bin
./package

"! -name . " – means any file other than current directory

prune – prunes all the directories other than the current directory.

A more readable way to list only files in current folder with find is – identical to what above cmd:

find ./* -prune

./packages
./bin
./mnt

If you want to exclude /mnt folder and its sub-directories and files with find by using prune option: