Search This Blog

Sunday, December 12, 2010

In a previous article, I described a KVM installation of the new RHEL 6 distro. My next article describes some production XFS considerations. Now that RHEL 6 is running, let's see how it is different under the hood from its predecessor RHEL 5. This was a good (fast and inexpensive) way for all of you to give it a go. In this article, I actually draw conclusions after performing a physical bare metal installation of RHEL 6 on a Dell PowerEdge 1950 server.

The very first impression I got by booting the system is a familiar conservative GNOME 2.28 theme. It works, but it shows its age when compared to the most recent 2.32 GNOME version of its Fedora 14 cousin. The same can be said for the KDE environment: RHEL 6 is at KDE 4.3.0, miles away from KDE 4.5.3. Frankly, if you really want the latest and the greatest on your desktop/laptop, head for the Fedora distro and leave RHEL for your server silicon.

Nevertheless, RHEL 6 does have some desktop improvements over RHEL 5. In summary, it includes support for Kernel Mode Setting (KMS), DR12, OpenGL 3D. The nouveau driver is there, and this means that you are more likely to get a more decent display (my Dell Precision 490 and E6410s did wearing Nvidia gear, although your mileage might vary, depending on your hardware).

On the scripting front-end, Perl has moved to version 5.10.1 (RHEL 5.5 is still on the archaic 5.8.8) and Python is on 2.6.5 (5 was on 2.4.3). For Python funs, there is nothing in the official RHEL repository for Python 3, however, I would expect that in the near future, the EPEL repository and/or the IUS Community Project will package something suitable. An improvement on standard utilities like grep is noticeable (RHEL 6 is on version 2.6.3, whereas RHEL 5 used the ancient 2.5.1). It still beats me why a major commercial distribution (5) still uses 2.5.1. Grep 2.5.1 is so buggy and untrustworthy that I am very surprised why somebody has not screamed at the RedHat release engineers to push an upgrade for something as vital as grep! I understand it is good to be conservative, but not to that extent!

At the heart of the operating system runs a 2.6.32 kernel (2.6.32-71.7.1.el6.x86_64 is the RHEL 6 version at the time I am writing this article), with back-ported features and fixes from the 2.6.33 and 2.6.34 vanilla/mainline source kernel versions. There is a big scalability metric change over the 2.6.18 based kernels in a number of different areas, as shown on the table below.

The big increase in the number of supported CPUs shows a shift of the RedHat development team towards NUMA architectures. With multi-core processors raising the limit every year, suddenly 192 CPU systems are within the range of many mid-range shops and the trend will continue in the future. The same can be said for "fat RAM" nodes, a popular choice for speedy RDBMS and science number crunching platforms. Ticketed-spinlocks, Read-Copy-Update (RCU), and Transparent Huge Pages are some of the techniques the RHEL kernel employs to drive the scalability for CPU and RAM to these levels.

In my honest opinion, the greatest problem in terms of RHEL scalability lies in the area of filesystems. The inadequate 16 TB of the officially supported ext3 of RHEL 5 is now raised to 100 TB in RHEL 6, by means of providing the option of officially supported XFS. RHEL 6's (and RHEL 5.6's) ext4 is still on 16 Tbytes and the only thing it provides is slightly better performance and much improved fsck times.

I was always curious to do a head to head comparison between ext3 and ext4 on the same OS platform, so I fired up a RHEL 6 test node on a Dell PowerEdge 1950 with 8 cores and 16 GB of RAM and setup a RAID 50 (21 x SATA 1Tb) 12 TB partition on hardware RAID controller. Using iozone I provide the comparative results of writing a 32 Gig file (2 x size of RAM) on that 12 TB partition.

The results indicate that ext4 is not such a big improvement over ext3. However, when I mungle some inodes and bring the fs into an inconsistent state to simulate an fs failure, fsck-ing the ext3 filesystem takes me 2.5 hours, whereas doing the same on the ext4 restores the same filesystem in just under 14 minutes! That's a big difference, so for all of you that run multi-TB ext based fs-es in production environments, ext4 is a big plus only for that reason.

XFS is a much more scalable filesystem that ext4. The latter is clearly a marginally better performing filesystem than ext3 and is a temporary step to the roadmap for a more scalable next generation solution. My view is that RedHat is moving to the right direction, but it needs a lot more than the 100 TB it currently supports on XFS, to get a good slice of the Enteprise storage market. Just think of all of these people that run their Netapps/ Panasas solutions or all the folk that really love their ZFS on Solaris. The market is there, RedHat has the experties and it should address the problem really soon. My comment addresses also the clustered filesystem area, where GFS2 is traditionally employed by RedHat. Apart from minor differences in RHEL6 GFS2 bundling, I read no substantial improvements in that area.

Moving on to a more realistic workload comparison between RHEL 5 on ext3 and RHEL 6 on ext4, I could not think of a better example than a heavy duty RDBMS task. Thus, I utilized a MySQL 5.1.50 server on RHEL 5.5 box (2.6.18-194.26.1.el5) to perform a heavy duty task on the 12 TB partition (booting from two sets of identical system disks, in order to switch between RHEL 5 and 6). The workload comes from the field of bioinformatics. The task had 3 steps:

Step 1: Unzipping the entire EMBL release 105 nucleotide database. The release consists of 105 .gz compressed files of total size approximating 138 GB. A small Perl script that forks 8 instances (1 for each available processing core) unzips each one of the 105 files in sequence, without saturating the SATA I/O controller. The end result is just under 1 TB of disk space with all the files uncompressed.

Step 2: A second Perl parsing script utilizes the DBI interface and the DBD::mysql driver to read all the entries to an InnoDB MySQL table. That's right, the whole lot of 194 million entries are going into a single InnoDB table.

Step 3: A text index is created on certain columns of the InnoDB MySQL table that concern the feature table of the sequence database. This is mainly a processor/memory intensive task.

This ensures an overall low latency for I/O tasks which can yield notable performance differences over the default cfq scheduler. The results of this benchmark are given below and they show substantial improvements in RHEL 6.

In total, RHEL 5 took 36.3 hours to complete the workload. In contrast, RHEL 6 reduced the time of completion to 29.1 hours, a staggering difference. The greatest difference can be attributed to the last step (MySQL creation stage). Step 2 is in the middle, mainly due to the difference in Perl versions and drivers (5.8.8 in RHEL 5 versus 5.10.1 in RHEL 6).

An additional area where I normally look into when I benchmark systems is that of network device and protocol performance. In my production environment, I have to use NFS heavily. This is a mixed blessing, as NFS was, is and will be a grey area, when it comes to tuning and performance but on the other hand, it is one cheap and easy way to distribute large data sets into a moderate amount of nodes. RHEL 6 introduces two major differences in this area:

Default use of NFS v4: Amongst RHEL 6 boxes, the default NFS protocol version is v4. RHEL 5 used v3 by default. I am not going to write on essay on NFS v4 features, I am sure most of you know your NFS or can look it up yourselves. The great difference here is that you now have solid IPv6 and encryption support for NFS v4, as well as the ability to combine it with FS-Cache. This latter feature can increase the performance in some data distributions scenarios.

Kernel networking enhancements: These include improved drivers for FCoE (Fiber Channel over Ethernet) and RDMA support over 10 G Ethernet (RoCE) and Infiniband for low latency networking.

In order to place some of these features to the test, I have setup a small computer cluster of three RHEL 6 clients (8 x core clients) to talk to a single RHEL 6 NFS server over NFS v4. The interconnect was a 10G link point-to-point connections, using Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection adapters. An FS-Cache setup was employed to read-only access 551 files of 105 GB. The files were data sets for the NCBI Blast database utility, a popular tool for life scientists. Under RHEL 5, I used to NFS v3 mount them as read-only to the client nodes. Lots of sysadmins rsync those files to local client disks for better performance. This is also my normal practice , however, here I want to test the two different NFS version scenarios: NFS v3 without FS-Cache in RHEL 5 versus NFS v4 with FS-Cache in RHEL 6.

The results were a bit strange. At the beginning, when I actually had one or two clients accessing the NFS imported data in RHEL 6, things were substantially slower with FS-Cache than the RHEL 5 scenario. However, after rebooting to clear any previous cache effects, as soon as I started hitting the NFS imported data with all three clients and launched 5 NCBI Blast jobs on each of them (3x5=15 jobs) simultaneously, things started looking better on RHEL 6. This tells me exactly what the RHEL 6 manual states: FS-Cache and NFS v4 are not performance winners all the time over simple NFS v3, if the client-server traffic is low. However, when you start loading the interconnect, FS-Cache is a good compromise between client reads and network overhead (all these context switches caused by your card).

I am re-looking into this test case, as I want to make sure that I get things right before I post numbers and I will update the blog when I am certain.

That's it for this week. The next article will look into virtualization (KVM) performance, the other big area where RHEL 6 is supposed to give us great benefits.

Tuesday, December 7, 2010

I think the world of media has been taken hostage by an attention seeker...or maybe the media empires have decided to use an attention seeker to spice up their news. The second scenario seems more probable, even if it seems to be a cynical view.

I am NOT against Wikileaks. I find it useful as a third resource of evidence-based news records, which can help to cross-reference bits of information, when in doubt about various news bits coming from other sources. Despite the fact that sometimes I (and many others) question the source ethics and the authenticity of its published records. This seems to be a general problem with the Internet. It contains a lot of information, but not all of it is credible or useful. The recent diplomatic cable leaks may serve as evidence of various foreign policy misconducts, but if I raise the question "Does a reasonably educated and well informed person need Wikileaks to know the deal behind the US foreign policy, to infer the ties between China and North Korea?", the answer I will expect is a definitely negative. Thus, I question the noise made behind the Wikileaks "revelations".

In fact, I have financially supported Wikileaks (before their accounts where shut down). What I am really against is the behaviour of its founder, for various reasons.

Julian Assange has a background of ethical hacking. There is no universal definition of the term "ethical hacker". I tend to think of it as a reference to a person with advanced technology skills, that puts them into good use to reveal the truth or inform people about potentially harmful situations, seeking no financial gain or other rewards from any affected parties. In the information security world, we have the classical argument of software vulnerability disclosures. Some people argue that all vulnerabilities should be made public, whereas others disagree and are of the view that vulnerabilities should be disclosed only to a limited number of parties, on a strict need-to-know-to-fix basis. Personally, I support the second view, and I never disclosed software vulnerabilities in public.

If I draw a parallel line to the software vulnerability disclosure issue, it would run along the Wikileaks disclosure of vital US sites around the world. I strongly dislike this action, even if I am not a US citizen (or US Government employee). The reason is simple and it has nothing to do with the breaching of any National Security policy. After all, if someone is really determined to do something nasty, surely they will find the resources to do harm without the Wikileaks disclosures. However, any reasonable person understands that revealing strategic infrastructure locations (some of which are not only US based, but they serve collectively many nations) is an act that adds very little to the truth. It is simply a reckless action, bound to also draw the attention of less serious folk with malicious intent.

An equally noteworthy issue with the Wikileaks case is that of the Denial of Service (DoS) attacks they had on their domain. I am not sure whether the slowness experienced on their domain was due to persistent DoS attacks or simply by the strong demand (probably by both) in anticipation of the forthcoming document leaks, but this is a strong lesson in distributing important information in a scalable and secure way. This seems to be the job of Peer to Peer (P2P) protocols and not a number of static HTTP/FTP servers mirrored around the world, which is the usual approach. Torrents distributing the content were of course active from day 1.

My final comment concerns the good old face value of the information origin. I will use an example that comes from the Linux/Unix sysadmin world. Every security conscious sysadmin (and user) that uses third party binary package repositories makes sure to validate them via either a secure hash algorithm based key (MD5, SHA) or public repository key prior installing them into computer systems. These mechanisms make it more difficult for someone to maliciously alter the contents of the binary package and make you install something nasty in your computer. However, they are not a panacea. We have had cases where world famous open source packages have been compromised. Nevertheless, this is a rare event, and each time we download a Linux kernel, an Apache binary or our latest IDE from our Linux distribution, we trust that the keys have not been compromised. This trust is there because we know that our favourite Linux distribution has capable folk to look after security issues, so we do use the good old face value rule.

Wikileaks has appeared so far to be a human-centric entity around the face of Julian Assange. Hence, it would be fair to say that Julian represents to the world the face value of Wikileaks, even if there are probably dozens of people behind the scenes that work to make Wikileaks tick.

It is also understandable that people that reveal the truth are also the subject of massive attacks at every level. Mr Assange had been hiding for quite a long time. I find that a bit odd. If Bob Woodward and Carl Bernstein managed to stay alive by revealing one of the largest scandals of the US political history during the horrible Nixon era, I am sure Assange could find ways not to hide. In the same way, I am sure that the lack of transparency and instant communication during the Nixon times could make it easier for someone to attack journalists then. And they did, but somehow, the journalists and the papers stood up to the challenge. No hiding was necessary.

In the same way, if Assange is not willing to understand that he has to face the Swedish prosecutors and clear his name, he will never gain the face value he needs to be trusted. Sweden is not known to be a corrupted state, so if the "rape" and "sexual misconduct" allegations are constructed to halt him down and he cares about the truth, he should raise to the challenge and pass the public face of Wikileaks to someone else. Sooner or later, he will face the facts.

Thursday, November 11, 2010

A previous article outlined the new things of the RHEL 6 beta for server administrators. Today, RHEL 6 was released by RedHat. A good way to test the new environment is to take advantage of virtualization technology and install a virtual RHEL 6 instance. I am a user of the Fedora project in my laptop environment, as it is close to my RHEL production line, its free and it works well on my Latitude E6410 system. So, time to get our hands dirty and go through a proper RHEL 6 installation.

As a rule of thumb, you will need about 12 Gigs of free space to have a working RHEL 6 beta environment (3 of them will be used to download and store the RHEL 6 beta ISO image and the rest will be the install itself). A computer with at least 4 Gigs of RAM and 2 processing cores is a must, if you need to keep working with your host OS and other guests at the same time.

My Fedora Core 14 distro provides a KVM virtualization environment. After downloading the RHEL 6 x86_64 image from RedHat Network RHEL 6 Channel, I fire up '/usr/bin/virt-manager' from the command line as the root user. For those of you that prefer to start things graphically, the Virtual Machine Manager is normally situated under 'Applications'->'System Tools' in a standard FC 12 installation. Either way, I am greeted by the Virtual Machine Manager startup screen, I click on the 'Create a new virtual machine' button on the upper left corner, as shown below.

The first VM building screen shown above allows me to name my new guest OS (I like to avoid spaces in the names so I type 'RHEL6' rather than 'RHEL 6') and since I have already downloaded the DVD ISO image, I choose to install from 'Local installation media'. I press the 'Forward' button and the second VM building screen appears (shown below).

In the next step, I inform the VM manager about the whereabouts of the RHEL 6 DVD ISO images, the guest OS type ('Linux') and the version (FC14 and updated KVM versions on FC12 and FC13, will show the option for 'Red Hat Enterprise Linux 6').

The next VM building screen (third in sequence) defines the allocation of RAM and CPU resources. You know that with RAM, the more the better. My E6410 is fitted with 8 Gigs of RAM and quad core processor, so I can allocate a modest 2 Gigs of RAM and 2 cores.

The configuration of the virtual machine's disk image follows as shown below. On 'Step 4 of 5', I select the second option 'Select managed or other existing storage'.

When you hit the 'Browse' button, you will get to the Virtualization Manager volume manager, shown in the next screen caption. The virt-manager organizes images in virtual storage pools. I have two in my system, the default one (which depending on your partitioning options can be under /var) and a user defined one (vm1), which I made for the special purpose of storing virtual disk images. I select the user defined one, where I know I have enough space and I allocate approximately nine and a half Gigs to make room for my RHEL 6 virtual disk image (/vm1/RHEL6.img).

When I can, I like to use the qcow2 QEMU Copy On Write image format, due to its versatility and performance. The Allocation is set to 0 MB, as we do not need to pre-allocate all of the 9.5 Gigs. Instead, the virtualization engine will allocate space on demand. If you pre-allocate, things can be a little bit faster. In my case, I prefer to optimize disk space usage over speed.

One final step prior firing up the virtual machine remains. Step 5 of 5 confirms that we should be all set to go. Note the network parameters (expand the 'Advanced options') that indicate a NAT bridged interface. 'Virt Type' should be set to 'kvm' and 'Architecture' to x86_64. In that way, you can have optimal results and expand into more than 4 Gigs of RAM, if you can afford the RAM.

That's all, we hit the 'Finish' button. The virtual machine will boot the virtual BIOS and load the emulated Local CDROM/ISO image of the RHEL 6 distribution. At the initial greeting GRUB screen, it is wise to choose to install with the Basic video driver installation option, to avoid any undesired video emulation effects.

After the RHEL 6 greeting screen and the choice of language/keyboard, you should get a warning, which relates to the process of initializing the virtual drive image. This is quite normal and provided you have selected the right virtual image, go ahead and 'Re-initialize all'.

You can then choose a default partitioning scheme (Use All Space) and click Next.

This should get you to the software configuration screen. What you choose to install its up to you. For the purposes of the evaluation, I went for a Basic Server installation with the option to customize the packages ('Customize now'). I made sure that I took my time to select database servers, the KDE and GNOME graphical environments and most of the development tools. Thinks can be easily added at the post-installation stage.

At this point, set the root password, and let the routines perform the installation for you. The system will eventually reboot and enter the familiar First-boot stage, where you need to make additional users, set the time and date (recommended to use NTP). One important difference I came across from previous RHEL versions is that the First Boot has stopped being annoying in the process of trying to convince you to register with RHN. That has been left as a post install step. In fact, this is the first thing I do on the new system, using the rhnreg_ks utility, as shown below.

After doing that, and issuing a 'yum -y update', it turns out that already there are about 150 Megs worth of updates to be done. Latest reported kernel as of today (11/11/2010) is 2.6.32-71.7.el6 (Oct 27 2010).

Let the fun begin! Future articles will present the first impressions.

Tuesday, May 18, 2010

Late this spring, Redhat released a beta of the 6th version of their Enteprise Operating System. As I have personally appreciated RHEL 5's stability on my server production environment for many years, I also started realizing that the distro's 2.6.18 kernel (with feature enhancements from latter kernels) started showing its age. It is my personal view that Redhat spent a lot (too much?) time with version 6 and should have already have had it in production. But, first impressions, indicate that it was worth waiting, as the beta seems quite solid and feature rich. Whilst Redhat's release notes are more complete than my text here in terms of listing the features, the new beta comes with a few long awaited 'goodies' for system administrators. I highlight the ones that are most important for a server admin (IMHO):

In the filesystems area the production (supported) version of the 4th Extended Filesystem (ext4) and XFS filesystems are notable. A technology preview (means not supported, run at your own risk) of the btrfs filesystem shows the way ahead. The ext4 is a much needed performance and feature oriented revamp of the ext3 filesystem, but the way ahead is really brtfs (and maybe other filesystems such as the recently 2.6.34 kernel included ceph filesystem ). Finally, are you thinking of using Solid State Drive (SSD) devices as storage to speed up system (OS) disk I/O? Well, good news, because 'block disgard' (mark inactive blocks to prevent/reduce 'wear levelling') is now available.

On the storage area (but at different level), the block layer has now introduced utilities to get to limit (aka adapt/optimise) I/O activity according to the storage device settings (I am not sure I would personally call these as 'I/O limitæ tools, I/O aligners or even better 'I/O storage tunners' would do them justice :-) ). The idea is simple but quite important: Imagine certain storage utilities (mkfs, lvm...) becoming aware of the optimum way to place/align data on the storage device, so that they can boost performance.

In the virtualization field, Redhat made clear from 5.4 and 5.5 that is moving away from XEN in favour of KVM. It seems that XEN's overall good performance on paravirtualized guests did not come above its implementation complexity and at the end, things went the KVM way. So, those of you who seek support for XEN hosting, you will either have to stick to RHEL 5 (XEN hosting will be supported throught 5's support life) or start planning your XEN-to-KVM migration now (for help to migrate Xen guests to KVM, have a look at this page to make sure you switch your system from Xen to a KVM hypervisor and then to convert your guest keep an eye on the Fedora project git hosted Xen to KVM converter (virt-v2v). ) To make up for the rather forced migration, Redhat has introduced a lot of performance tunning patches on the KVM hypervisor.

Another important inclusion in the RHEL 6 toolkit is the inclusion of the Corosync Cluster Engine. The new engine brings a lot of improvements over the RHEL 5 cluster tools. Amongst the notable ones are a redesigned cluster administration interface and the ability to run clusterized KVM managed guest as managed services.

The next part of this article series will go through a standard installation of RHEL 6 on a virtual KVM platform.

Thursday, May 6, 2010

Greece mourns the death of three working people. People that woke up to go to work like all of us and ended up in their coffins, just because various thugs wanted to "protest" and considered that the lives of ordinary citizens were worth nothing.

These thugs who call themselves "anti-authoritarians" have a long story in Greece. A story of destruction and violence. They are the same people that destroy Greek universities exploiting the flawed University asylum rules, the same people that destroy properties built with sweat and blood, and now the murderers that destroy and kill in the name of anti-capitalism and anti-authoritarian spirit.

These thugs are for long now being "cuddled" by certain organizations and political parties such as KKE (which did not consider to place something appropriate in their international page calling the event just a "provocation" and does not recognise the Greek constitution) that kept and keep the country behind.

Monday, February 22, 2010

Recently, I came across Bruce Schneier's view on the Google "Chinese" hacking attacks. In essence, he analyzes the situation mentioning various facts (the existence of backdoors to various systems due to wire tap requirements) and concludes about the dangers of building the ultimate surveillance system and establishing a police state.

I think that the title of the article is a bit misleading though. In my honest opinion, the title should instead read : "US fails to handle the inside threat problem in critical surveillance infrastructures". I will explain my point of view.

Schneier's view is fundamentally correct. Indeed, we need to be careful about the procedures and the persons who handle surveillance systems (I am not going to discuss here whether we need these systems, this is a big issue. For the purposes of highlighting the insider issue I axiomatically accept that we do need them). This essentially points to the insider problem, a big issue that is left to the side in the process of securing surveillance infrastructures.

I do not want to reveal details I do not know about, but from my experience as a system administrator and security researcher, I understand that it takes a lot more than knowing the existence of a backdoor to a system to exploit it. Google might have indeed left backdoors to Gmail, this creates a vulnerability. To exploit this vulnerability a catalyst is needed and that is the person who knows the failsafes (procedures/passwords) and hands them over (intentional misuse) or a person that is naive enough to design procedures that are too open (accidental misuse). Without these factors, it is difficult for me to accept that Google's systems could have been compromised.

Interestingly enough, Schneier mentions the high profile wiretap case of the Greek government in June 2004. In my view, the problem was not that Ericsson enabled the surveillance functionality into the ground stations for the Greek government, nor the fact that Vodafone Greece had issues with their procedures for enabling the issues. The very fundamental issue is that someone took a decision to consult/allow important members of the Government to discuss government issues using a normal private carrier and ordinary phones, with no further security failsafes (further encryption, VPN and other mechanisms). That is essentially a policy mistake made by an insider ( an advisor that the Government trusted to secure the communication channel or the lack of him).

I often argue that for every external breach of security, there is almost always an internal reason (naive users, laizy IT admins, absence of policies from CIOs). The same is true for surveillance systems. The funny thing is that these problems have been highlighted by many US government funded workshops of insider threats and other gatherings around the world. And my question is: "When will people listen to insider threat researchers?"

Wednesday, January 20, 2010

(Disclaimer: I advise Promon AS on some issues, so my opinion about Integrated Application Protection could be biased. In that case, feel free to point out tools from other sources that achieve the same functionality. I am also not on their payroll, so this not a sales pitch).

The recent targetting of Google, Adobe and IE flaws raised a lot of eyebrows and gives plenty of thought to the security analyst. This has nothing to do with the specific Microsoft/Adobe vulnerabilities involved in the case. Software will always have vulnerabilities. It has also nothing to do with infrastructure vulnerabilities (spoofed SMTP headers, DNS targetting, more to come in a latter arcticle ). They will always be there. The question is why setups with highly capable people fail to address attacks that might be zero day (or near zero day) but they are still preventable.

In order to give an answer to this question, I can't help but notice the shortcomings of common desktop anti-virus/spyware packages. Today, most people have a desktop/server anti/virus/spyware package installed on their computer to defend themselves against various types of malware. The vendors that produce these tools do a tremendous amount of work to ensure that they protect you against many types of threats. In essence, they have industrialised the process of producing signatures (non ambigous ways of describing the malware) and then they throw a bit of heuristics to compensate for the things the signature will not catch (you cannot know everything in advance). This combination is good to catch most types of attacks, but not all (many of these products did not detect the recent aurora attacks for various reasons).

If we rely daily on computers for critical tasks, is a "good for most things" type of protection acceptable? Personally for me, it is certainly not. Sure, I use known desktop and servers anti/virus products and I always recommend them, but I always say their development strategy has many pitfalls. I am going to outline them in the next paragraphs and then prescribe a solution to these problems.

Anti-malware scanners tend to use heuristics to address (amongst other issues) the polymorphic payload of malware. Is is interesting to note that the recent "Aurora" style attacks used either malware with different payloads or different payload wrappings. This can downgrade the ability of the most carefully crafted heuristic signatures of anti-malware vendors. In fact, heuristic signatures are also responsible for "false positives", a known issue not only in the anti-malware world but also in the field of Intrusion Detection/Prevention (IDS/IPS).

Take the fact of the previous paragraph and combine it with the variety of attack techniques, operating system modules and the even greater number of applications and combine it with the need for an infrastructure to safely distribute (on almost a daily basis) the updates for the heuristics threshold and signatures and you get...the chaos. In my view, this is the very reason behind the fall back of many anti-malware vendors in the race of malware writers and
anti-malware analysts. They just can't and will not (I dare to say) keep up with these techniques. Their approach is useful but it needs a more focused complement.

The next reasonable question is what can be a desirable complement. The answer is obvious. What is the most valuable component in the software stack that interacts and allows access to data? Yes, the application. And after the mail browser, the most critical (and for long time universal) application that gets data from the Internet and sends data to the Internet is the web browser. Application integrated protection is the key complement most people need to render stack overflow and other types of similar attacks useless.

It would also be good to minimize the reliance on a continuous update distribution channel. Infrequent updates would satisfy bad Internet connectivity scenarios, as well as lazy user and system administrators.

There are many companies out there that offer integrated application protection. And a notable one is the Norway based Promon.

Promon's Integrated Application Protection (IAP) achieves this functionality. The concept behind Promon's technology is unique and has a sound theoretical basis: If the data and the information flow are the most valuable things, the most important thing is to stop information leaking out and in to the application, making a virtual shield around it. The details of this shield are of course a trade secret and proprietary, so I hate I cannot analyze the details here. However, the important thing is that their tool caters for minimum installation hassle, minimum user intervention and a small number of updates.

I was curious to find out whether their tool could stop the "Aurora" attack payloads. It turns out that it does. showing the technology in action. Here is a video that demos their technology against "Aurora" payloads (thanks Sondov!):

On the left hand side you can see a Vmware instance simulating the victim's system and on the right hand side is the bad guy that likes to hand out malware payloads. Watch what happens on the first instance. The victim's system connects to 'evil.com' (effectively exploiting of the IE vulnerability) and then various things happen to the poor desktop. The second part repeats the experiment under the shield of Promon's guard. The application immediately detects the exploit by sensing essentially suspicious information flow in and out of IE and then it drops the bad application.

I will try to outline the technology behind promon's concept in a latter article. My point is to show that effective technology to protect (and complement other tools) vital applications in cyberinfrastructures does exist and is quite effective against professionally designed and executed attacks such as those of the recent "Aurora" wave.