Linux - ServerThis forum is for the discussion of Linux Software used in a server related context.

Notices

Welcome to LinuxQuestions.org, a friendly and active Linux Community.

You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!

Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.

If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.

Having a problem logging in? Please visit this page to clear all LQ-related cookies.

Introduction to Linux - A Hands on Guide

This guide was created as an overview of the Linux Operating System, geared toward new users as an exploration tour and getting started guide, with exercises at the end of each chapter.
For more advanced trainees it can be a desktop reference, and a collection of the base knowledge needed to proceed with system and network administration. This book contains many real life examples derived from the author's experience as a Linux system and network administrator, trainer and consultant. They hope these examples will help you to get a better understanding of the Linux system and that you feel encouraged to try out things on your own.

I have a VM with Ubuntu 12.04, RAM 4GB, processor shares are higher than the other VMs.

I use this VM for only one website, "Based on Joomla"

When I issue the "top" command I see that CPU are 100% all the time, apache2 and mysql are consuming it.

Sometimes, and all of a sudden, the access to the website gets REALLY slow, but pinging to the server is normal.
Browsing to the hosted website is dead, even SSH to the server is very slow, While the CPU Consumption is nothing.

I checked the logs for the mysql and the apache but nothing shows there.

So what I do is that I restart the services and the website is fired up again, and CPU gets 100% again.

I have a VM with Ubuntu 12.04, RAM 4GB, processor shares are higher than the other VMs.

I use this VM for only one website, "Based on Joomla"

When I issue the "top" command I see that CPU are 100% all the time, apache2 and mysql are consuming it.

Sometimes, and all of a sudden, the access to the website gets REALLY slow, but pinging to the server is normal.
Browsing to the hosted website is dead, even SSH to the server is very slow, While the CPU Consumption is nothing.

I checked the logs for the mysql and the apache but nothing shows there.

So what I do is that I restart the services and the website is fired up again, and CPU gets 100% again.

What to say? The VM ran out of memory and then out of swap at which point it started killing processes. 4 GB RAM "should be enough" to run Joomla so the question is what system and service configuration modifications you made (for example some people think that when Apache complains about memory they should multiply all configuration values w/o diagnosis as if that should help), what "one website, Based on Joomla" means (is it brand new, does it contain a gazillion badly written homebrewn scripts, is it a migration meaning, did it previously run on its own physical hardware, is it well-developed, etc, etc?), if system and daemon logs show errors / clues and what metrics you're going to watch using which tools (Apachetop, mtop aka MySQL top, Atop, Dstat, collectl, etc, etc)...

Do read me saying "the question is" as an invitation to be methodical about it and be verbose sharing information:
- Which system and service configuration modifications have you made?
- What were the HW specs of the old location?
- Did the web site exhibit problems there too?
- What is the Apache version? Is it up to date?
- Did you increase any values in your Apache configuration?
- What worker type is Apache running?
- What is the PHP version? Is it up to date?
- How do you use PHP? As FastCGI or mod_php?
- Did you increase any resource settings in your PHP configuration?
- What is the Joomla version? Is it up to date?
- Does the web site run any outdated / risky / unnecessary mambots / agents / extensions / components?
- Have you looked at system and daemon logs for errors and clues?
- Do you know what the standard requirements for Joomla are? In which way do the configuration files deviate from those standards?
- Which documentation do you use to troubleshoot the problem?
- What system counters are you going to watch over time to start diagnosing the problem?
- Which tools are you going to use?

I know you are going to hate me for this,
But I'm kinda newbie with this, I generally do not alter any config in apache, i only set virtual hosts on it.

- i made no system modification, i only changed my.cnf as suggested my the mysqltuner suggestions.
- I honestly don't know what was the older config, i got the backup files from the old host.
- The old website was rumning very slow on the old host, but i haven't heard of a problem on that old host.
- Apache version is 2.2.22, I only updated it yesterday.
- The worker is prefork
- php version 5.3.10
- I haven't made any modification to the php conf.
- I can't really tell if the website running any outdated version/risky components.
- Joomla is 2.5.6
- I tried to search for any clues and logs on the system, but couldn't find anything on neither apache, mysql, or syslog.

Just let me know if it is an apache configuration so i can dig in and try to find a solution and make some changes,

I'm sorry if I'm waisting your time, but I might make use of any kind of any kind of hints and clues.

Performance tuning is more than tweaking just one setting in each configuration file: there is no "magical" key to turn to make everything right and you will have to work hard to understand all the aspects involved. For example if you made no alterations then Apache will be running with all modules enabled and with default settings, PHP may be wasting memory on Innodb while it's not being used and the site may be rife with mambots or SEF rewrites that bug everything down. I'm going to ask you put in some effort and do your own research first: search the 'net and check the Apache, PHP, MySQL and Joomla web sites for performance-related recoomendations. Post what URI's you judge relevant here, along with the contents of your Apache, PHP and MySQL configuration files (strip out comments by running grep -v ^# /path/application.conf|grep .) and the sites settings you find in the Joomla administrator menu. With respect to the latter you should of course obfuscate any identifying information. Meanwhile run Logwatch on the available system and daemon logs, let ApacheTop and MySQLTop run as long as possible, run FPA (easier overview of Joomla administrator settings) and save all information for posting (text preferred over screen shots).

Again thank you very much for bearing with me, and thank you for your patience.

As instructed, I googled for tuning apache configuration, I have found as you said, that the apache is running all three modules, while it was compiled only with prefork module, so I have disabled the worker and the event modules, thinking of course that they might consume some memory.

I have found as you said, that the apache is running all three modules, while it was compiled only with prefork module, so I have disabled the worker and the event modules, thinking of course that they might consume some memory. This is the tutorial I followed for the apache tuning: http://cloudinservice.com/tune-apache-performance-using-mpm-prefork-module/

OK. Let's start with the web server first. The link you point to is good because that it explains the method behind deriving values based on your actual situation. However it doesn't explain all values (see for instance this or this) and the effect it has (or should have) on how you configure your my.cnf next. Also I don't know if you logged process and memory data before and after the changes or if you actually derived your posted values from it and if you looked at changes in resource usage. The thing is that adjusting (in small increments) by making changes and comparing data is the kind of process you need to go through: take the "original" httpd.conf and log process and memory usage and consider that your baseline data. Create a backup of httpd.conf, make changes, restart the web server, hit the server with traffic (there's tools for that), log process and memory usage. Then compare both process and memory usage sets. Rinse, repeat.

Also note your httpd.conf lists several includes. While they may not hold surprises it's good to actually inspect them and ensure they don't. In the meanwhile I do hope you still collect data with at least ApacheTop and MySQLTop because the longer the period the better averages you have to base decisions on. Also please check your FPA extensions output (well done for posting) against http://docs.joomla.org/Vulnerable_Extensions_List please and correct the listed access rights (777 is too much), see http://docs.joomla.org/Security_Chec...le_permissions for an explanation (do read the whole page though).

Do persevere! This may seem like small steps now but once you notice the effect it has you know the effort wasn't wasted.

I don't know how to thank you for your replies, and I can't say how much it makes me happy to know that I might be somehow on the right track to solve my problems,

I have changed more on the apache config, the site is a little bit getting improved, stalling a little but not freezing, memory consumption is much better, I might increase the limits on the prefork module a little bit until I get a satisfied results.

As for the apachetop, I am running it till now, but wanted to share this results with you, as honestly I searched for the tool to have something useful to learn about it, but didn't hit anything:

But As I understand from what I see, it is http requests, 2xx OK, 4xx contains 404 not found.
And All: number of requests?

but what should I make use of it?

And what is R on the third row stands for?

Do you think that I may reduce the timeout to less time and increase MaxClients directive?
Coz I see from the top command, that apache processes are staying alive and dies after some time, at which time the website is a little bit stales.

I have changed more on the apache config, the site is a little bit getting improved, stalling a little but not freezing, memory consumption is much better, I might increase the limits on the prefork module a little bit until I get a satisfied results.

Usually you'll want to measure, lower values, put the pressure on, measure again. Also consider the other processes you still need to look at before enthusiastically tuning things for "performance". Because what if new insights mean the values in php.ini and my.cnf actually need to be increased? Doing that and increasing httpd.conf values simultaneously may be actually bad for performance. That's why you need to log data.

Quote:

Originally Posted by Tarikc

I really wish I will have to mark this problem as solved

What's the rush? And think of it as gaining knowledge instead.

Quote:

Originally Posted by Tarikc

This the continued apache top monitoring statistics:
I'm stopping it now coz I've read it consumes some memory.

No, you're running with default settings, should be OK.

Quote:

Originally Posted by Tarikc

But As I understand from what I see, it is http requests, 2xx OK, 4xx contains 404 not found.
And All: number of requests?
but what should I make use of it?
And what is R on the third row stands for?

The README for ApacheTop is pretty clear about things: how long it's been running, two lines accumulated data and two lines of stats in the last 30 seconds. Note nearly 25 per cent of your traffic isn't 2xx with over 20 per cent of redirects (usually 301: Moved Permanently / 302: Moved Temporarily). This may be due to .htaccess rewrites (or in httpd conf's we didn't see) or some SEO plug-in: best look at the 3xx messages in access_log.

Quote:

Originally Posted by Tarikc

Do you think that I may reduce the timeout to less time and increase MaxClients directive?

Coz I see from the top command, that apache processes are staying alive and dies after some time,

That is the effect of the MPM you chose. Having children start costs resources too but OTOH having children die also cleans up memory, keepalives, etc, etc. Have a go at the official Apache MPM docs.

Quote:

Originally Posted by Tarikc

at which time the website is a little bit stales.

Observed how?

Quote:

Originally Posted by Tarikc

But this is just me , or you suggest to try it myself and check?

Heh. You catch on pretty quickly.

BTW one way to look at process details is to use 'top' (or atop, htop or mpstat) but if you need a quick way to isolate the current memory usage of say a process called "apache2" then you could also use 'ps':

Code:

/bin/ps -C apache2 -oppid,pid,pcpu,pmem,sess,rss,vsz,nice,pri,args

In the long run it's better to run any capable SAR like Atop, Dstat or collectl.