Apache fcgi tuning php heavy app high ram usage

6 posts / 0 new
Last post
#1 Mon, 06/01/2009 - 06:30

Apache fcgi tuning php heavy app high ram usage

Hello, I am trying to configure Virtualmin GPL Apache 2 and fcgi, running at a Debian 4.0 Etch virtuozzo VPS, 512 MB ram, to a demanding php4 application that needs php memory_limit of 64 MB. I have read the Apache doc [0] and after reading about the PHP_FCGI_CHILDREN bug [1] [2], will use only Apache and, maybe, some other fcgi configurations [3]. Given that [4] explanations warns to avoid double spawning management (apache and php), maybe using DefaultMaxClassProcessCount, and reading doc about Virtualmin in low memory systems [5], I still have doubts regarding how to calculate memory usage, then correct configurations to avoid ram swapping at my vps. Currently, it seems to have still 240 MB ram free, but running the apache benchmark tool [6] caused a really ugly crash when load reached some amount. Almost all services crashed. A few minutes later, many of them restarted, but Virtualmin did not. It was more easy to reboot, then. But what would happen if it was an unnatended flood? I would like to limit resources usage, and gracefully degrade performance in a slashdot event (previously happened already, at past published blog articles) How do I calculate fcgi and apache configurations, given the php memory_limit 64 MB? How monitor virtualmin service itself and all configured services (in a crash event as described)? Apache seems to be monitored [7], but how about others? Regards. Andre Felipe Machado http://www.techforce.com.br

[0] http://httpd.apache.org/docs/2.0/mod/worker.html [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=431799 [2] http://bugs.php.net/bug.php?id=40286 [3] http://www.virtualmin.com/forums/limit/10/limitstart/10/virtualmin/fcgi.... [4] http://www.virtualmin.com/forums/virtualmin/limit-php-processes-per-doma... [5] http://www.virtualmin.com/documentation/id,virtualmin_on_low_memory_syst... [6] http://httpd.apache.org/docs/2.0/programs/ab.html [7] http://www.virtualmin.com/documentation/id,virtualmin_module_configurati...

Mon, 06/01/2009 - 06:53

It seems that the collectinfo.pl background process [7] is the responsible to restart crashed important services.
But how configure it to restart each desired service, virtualmin itself included?
It seems to have a fixed small list of monitored services, not including all managed services.
Andre Felipe Machado

[7] http://www.virtualmin.com/documentation/id,virtualmin_module_configurati...

Mon, 06/01/2009 - 07:23 (Reply to #2)


Well, I've seen a number of strange issues arise when using Virtuozzo, especially when no swap is configured.

512MB of RAM is generally a decent amount, but it may not be handling all the usage spikes if there isn't any swap to fall back on.

So, we could do some math and determine how to configure Apache -- but anything we come up with will only be account for Apache itself, not factors outside of that.

There are the cron jobs you're seeing, the fact that clamav and spamassassin both use a hefty amount of RAM, BIND seems to increase in size every day it's used, and things like Mailman just go nuts with your RAM usage :-)

A few thoughts:

1. One key will be to disable anything you don't need. A lot of people don't require Postgres and Mailman, for instance.

2. You may want to reduce the frequency at which collectinfo.pl runs. You can do that in System Settings -> Module Config -> Status Collection. It defaults to running every 5 minutes -- but it just collects stats. You could even tune that down to running once or twice a day, outside of business hours.

3. Similar with bw.pl, which runs once an hour (but I think you'll have to tweak the cron job manually to reduce it's frequency).

4. You might consider looking at the "Virtualmin on low memory" guide to see if anything there offers any help: http://www.virtualmin.com/documentation/id,virtualmin_on_low_memory_syst...

5. Of course, if you're able to enable swap on your system, that could really help too!

As far as monitoring your system -- I personally use monit to keep an eye on the various services running, available RAM and CPU, and so forth. If there's a problem, it can be configured to try and fix it. But it'll also send you an email.

Virtualmin does have some status monitoring built in, though I'm not sure if it can restart itself, I'll take poke around a bit on that :-)

Mon, 06/01/2009 - 09:22 (Reply to #3)

Thanks for your hints.
Regarding swap, if the hosting provider is unwilling to enable swap partition, I will have to use swap file [8] [9]. I will wait for their answer before creating a swap file.
But previous experience with apache+php+ezpublish using swap was terrible. They <b>must</b> stay at ram, leaving swap only to avoid crashes for a few seconds spikes. Completely useless running at swap for more than a short period spike causing 2 x 64MB apache+php processes at swap.
The doc virtualmin on low memory system [5] contains valuable hints, but does not show how to <i>calculate</i> the configuration parameters.
I would like to calculate apache and fcgi parameters using the conditions
[li]ram available to app[/li]
[li]php.ini memory_limit (constrained by the app)[/li]
[li]desired number of simultaneous users visiting site (each comprising a few http connections)[/li]
How do I calculate the [0] and [10] parameters?
Each visitor http connection will be translated at one apache fcgi child thread?
Each visitor browser will be translated at one apache child (server)?
By reading [11], the MaxProcessCount fcgid parameter is key to protect against memory over usage.
Also, [10] states that for php, MaxRequestsPerProcess must be limited to 500.
But will each fcgid php process be an apache child thread, then?

The mailman is a virtualmin required dependency, unfortunately. I have to manually disable its service.
The statistics collector seems to not use too much ram. But I would like to configure the collectinfo.pl to monitor more services, even virtualmin itself.
I will study the suggested monit.

Andre Felipe Machado

[8] http://www.techforce.com.br/news/linux_blog/swap_file_x_swap_partition
[9] http://www.go2linux.org/swap-file-vs-swap-partition
[10] http://fastcgi.coremail.cn/doc.htm
[11] http://2bits.com/articles/apache-fcgid-acceptable-performance-and-better...

Wed, 06/03/2009 - 09:29 (Reply to #4)

Virtuozzo does not &quot;expose&quot; swap to a VPS until it is required.
Also, if hosting provider configured it, RAM and swap are burstable.
If during an usage burst, more ram is required, it <i>can</i> make more ram available until a hard limit and then offer more swap until another configured hard limit.
These limits are from the hosting plan, and can only be examined (privvmpages), by a simple cat command of /proc/user_beancounters.
Values are in memory pages, 4 KB in x86 hw.

<i>[color=#FF6600]Virtuozzo also does not allow enabling swap files.[/color]</i>

So, swap is &quot;dynamic&quot; and not visible all times, and the size is negotiated with the hosting provider. It is the hosting plan...

How virtualmin behaves in such &quot;dynamic&quot; swap behaviour?
Other sw could be configured for the &quot;invisible&quot; limits of the virtuozzo vps hosting plan, but how about the Virtualmin?

Andre Felipe Machado

[0] http://wiki.openvz.org/Proc/user_beancounters

Wed, 06/03/2009 - 10:59 (Reply to #5)
Joe's picture

<div class='quote'>How virtualmin behaves in such &quot;dynamic&quot; swap behaviour?
Other sw could be configured for the &quot;invisible&quot; limits of the virtuozzo vps hosting plan, but how about the Virtualmin?</div>

I have no idea what you're talking about. ;-)

Virtualmin doesn't &quot;behave&quot; any way with regard to swap (nor do any other services or applications). Swap is just more memory. Virtualmin doesn't do anything special for memory. It allocates it when it needs it (via Perl), and it gives it up when it doesn't (also via Perl).


Check out the forum guidelines!

Topic locked