When I was first setting up my Linode server, I spent quite a bit of time tweaking the Apache config files for performance. I ran a few tests with ab and kept a close eye on the graphs in Munin to see what worked and what didn’t. The single most effective change I made was changing KeepAliveTimeout. This controls the number of seconds Apache will wait for a subsequent request before closing the connection. It’s like a bartender serving drinks and waiting 15 seconds after serving a client in case they want something else before helping the next customer. You end up with a lot of waiting.

This chart shows the results of reducing KeepAliveTimeout from the default of 15 to 2. I’m sure you can see when the change took place because all the busy servers go away.