Tuesday, October 27, 2009

It has been a long time since I used torque. I though I pen down some common commands I will used

pbsnodes -l (List node names and their state. If no state is specified, only nodes in the DOWN, OFFLINE, or UNKNOWN states are listed. Specifying a state string acts as an output filter. Valid state strings are "active", "all", "busy", "down", "free", "offline", "unknown", and "up". )

pbsnodes -o (Add the OFFLINE state. This is different from being marked DOWN. OFFLINE prevents new jobs from running on the specified nodes. This gives the administrator a tool to hold a node out of service without changing anything else. The OFFLINE state will never be set or cleared automatically by pbs_server; it is purely for the manager or operator.)

pbsnodes -c (Clear OFFLINE from listed nodes)

pbsnodes -a (All attributes of a node or all nodes are listed. This is the default if no flag is given)

Monday, October 26, 2009

We are using OpenFarics for our HPC. According to the FAQ, "With OpenFabrics (and therefore the openib BTL component), you need to set the available locked memory to a large number (or better yet, unlimited) -- the defaults with most Linux installations are usually too low for most HPC applications that utilize OpenFabrics"

Sunday, October 25, 2009

If you are suffering from excessive slowness from your netbook or your old and slow notebook, you may want to tweak the CPU frequency governor to performance mode instead of the general mode. How do you do that? If you are using Linux Mint, you have to install a utilities called cpufrequtils

Thursday, October 22, 2009

The configuration file at /etc/security/limits.conf help in settings limits to processes or logins oe more. This helps in preventing a rogue applications in consuming too much processes that might cause the servers to crash.

* soft nproc 400

* hard nproc 500

(If you hit 400 processes, the system will warn you. If you hit 500, the system will block you)

Alternatively if you do not wish to set a limit to the number of processes that can be run on your servers due to the particular application, you can also

* soft nproc unlimited

* hard nproc unlimited

You do not need to reboot to effect the settings, just relogin and the users will be subjected to the /etc/security/limits.conf configuration file

Sunday, October 18, 2009

When you are doing High Performance Computing or Cloud, the concept of workload is very important. Basically there are 4 kinds of workloads (Taken and summarise from MOAB Admin Guide 5.2)

Batch Workload - With a Batch Job, the job is submitted to a job queue, and is run somewhere on the cluster as resources becomes available

Interactive Workload - Requestors are interested in immediate response and are generally waiting for interactive request to be executed before going on to other activities. To manage interactive jobs, the focus is usually on setting aside resources to guarantee immediate execution or at least a minimal wait time for interactive jobs

Calendar Workload - Calendar workload must be executed at a particular time and possibly in a regular periodic manner. There must be guarantee of resource availability at the needed time to allow calendar jobs to run as required.

Service Workload - Service workload processes externally generated transaction requests while a scheduler or resource allocation mechanism provides the distributed service with needed resources to meet target backlog or response targets to the service. Examples are web farms etc.

Step 1b: A better method in my opinion will be by using the repository method
Create a virtualbox.repo file at /etc/yum.repos.d/# touch /etc/yum.repos.d/virtualbox.repo# vim virtualbox.repo-------------------------------[virtualbox]
name=VirtualBoxbaseurl=http://download.virtualbox.org/virtualbox/rpm/fedora/$releaseverenabled=1gpgcheck=0---------------------------------# yum check-update

Step 2: Run setup file for Virtualbox# /etc/init.d/vboxdrv setup

Step 3: Add yourself as a user in the vboxusers group at /etc/group# usermod -G vboxusers myserid

Step 4: USB Fix for VirtualBox # vim /etc/group----------------------------------------------# create a new group and add your name to itusb:x:502:myserid----------------------------------------------

Friday, October 16, 2009

Step 1: If you wish to change to another compiler like PGI (in this example) from the Intel Compiler,
You can issue the commands# export CC=pgcc# export CXX=pgCC# export F77=pgf77# export FC=pgf90

To check that the openmpi is using the PGI compiler# /usr/local/openmpi/bin/mpicc --showme# /usr/local/openmpi/bin/mpiCC --showme# /usr/local/openmpi/bin/mpif77 --showme# /usr/local/openmpi/bin/mpif90 --showme

Thursday, October 15, 2009

eyeos is an open source web desktop following the cloud computing concept. It is mainly written in PHP, XML, and Javascript. It acts as a platform for web applications written using the eyeos Toolkit. It includes a Desktop environment with 67 applications and system utilities. It is accessible by portable devices via its mobile front end. eyeos lets you upload your files and work with them no matter where you are. It contains applications like Word Processor, Address Book, PDF reader, and many more developed by the community

Wednesday, October 14, 2009

Intel® Data Center Manager (Intel® DCM) SDK provides power and thermal monitoring and management for servers, racks and groups of servers in data centers. Management Console Vendors (ISVs) and System Integrators (SIs) can integrate Intel® DCM into their console or command-line applications and provide high value power management features to IT organizations

Intel® DCM features include:

Built-in policy based intelligent heuristics engine that can maintain group power capping while dynamically adapting to changing server loads and minimizing performance impact of workloads

Uses Intel® Intelligent Power Node Manager for node power and thermal management

Designed as an SDK to integrate into existing management software products by means of a web service application programming interface (WSDL API)

Tuesday, October 13, 2009

This special edition eBook "High Performance Computing for Dummies" from Sun and AMD shares details on real-world uses of HPC, explains the different types of HPC, guides you on how to choose between different suppliers, and provides benchmarks and guidelines you can use to get your system up and running.

Saturday, October 10, 2009

Cloud Computing is gaining a lot of publicity. I have attended a few myself. But up to now even if you gather the experts on this, even the definition is abstract. Maybe one of the best way is to ask a question and try to answer it

So what is Cloud Computing?

Cloud Computing can be referred to as IT as a service. In the same way, a utility company delivers electricity for the end users for their specific and unique consumption, the challenge is whether Cloud Computing can deliver cheaper, faster, superior user experiences.

What are the essential components of Cloud?

Service Catalogue - From my limited understanding, Service catalogue is the standardised IT offering you are offering to the customers. Maybe it is just some virtualised infrastructure for development, maybe it is a development environment like Hadoop, or even maybe it is even a Windows environment inside for an application requirement....

Automation - By automation, what I meant was the ability to auto-provisioned by users for the standardised environment they require. In other words, there is some sort like a self-service portal where the users can select their choices from. Another essential atribute of automation comes in the form of workload analysis. The ability of the Cloud Computing infrastructure to meet the demand based on the workload analysis

Virtualisation - Virtualisation is very essential as many image format of the computing environment or the service catalogue can be kept efficiently. It is also a mean where the image can be deployed very quickly and the environment consistently kept for the particular user. Bare-metal provisioning is not impossible for the Cloud Environment, it is just less efficient and more time consuming as the nodes have to be reformatted with the OS and the image. Of course you can dual-boot or triple-boot. If your number of images are small or have very standardised user base, it is still manageable.

What are the catalysts pushing industry to Cloud Computing?

I think the current IT infrastructure limitation will gets more and more glaring as the explosion of services and data are harder to manage in a distributed environment. On average, about 85% of a compute resource remain idle. This is definitely not the best use of resources.

Another is the amount of budget spent on maintaining instead of adding new capabilities is another "push" factor. With a consolidated environment, cost will definitely be more managed.

I guess the last catalyst will be is the rising perception of the users. IT services is going to be just a utility. IT is so pervasive now like electricity. If you check in a hotel and there is no Internet, it is perceived as very out-of-touch. Rising expectation will definitely put demands on IT resources and yet the need to be flexible to provision with peak and trough of computing demands. This is the impetus given to the Cloud