Sitemap Generator WordPress Plugin Hits Server Memory Limits

Google (XML) Sitemap Generator Plugin is a popular wordpress plugin which can generate a search engine compatible sitemap of your blog in seconds … but it was leading to Dreamhost to terminating our wordpress scripts and resultant internal server errors.

Internal Server Error Causes

Following investigation into cause of our persistent internal server errors, Dreamhost support kept informing us repeatedly to identify the wordpress scripts which were tripping the server.

I’ve checked into why you’ve been receiving these internal server errors, and it seems your scripts have been getting automatically killed by our Process Watcher due to your sites going over Memory limits on the shared server.

I would highly recommend that you follow the steps in the following wiki article in order to reduce your usage: http://wiki.dreamhost.com/Finding_Causes_of_Heavy_Usage

Also, please be aware that just because this site is the one getting killed, it isn’t necessarily the problematic one. For instance, if Site A is using 90% of your allotted memory usage, and� then Site B attempts to use an additional 15%, Site B will be 5% over and will get it’s script killed.

I’d also recommend looking into any 3rd party plugins you may be running, especially if you happen to be running WordPress installs as they can be� notoriously poor at memory management. (Secret Tip: Removing them from� the plugins directory is more effective then simply disabling them) Lastly, you may want to look into DreamhostPS, as you will be able to raise your memory limits to whatever limit you’d like.

Sitemap Generator WordPress Plugin and Memory Usage

So how much memory was Sitemap Generator consuming everytime it generated a sitemap of over 3000 urls? Around 25MB!

And to top that it was eating up that much memory every time we edited, deleted or created new articles. That was a setting which is usually enabled to have the most current sitemap.

And when I often tried to rebuild the sitemap manually, I got this error

Basically the script was taking too much memory (from the entire server which also hosts other sites) everytime it generated a sitemap with so many urls and the Dreamhost Process Watcher used to stop the scripts automatically due to your sites going over Memory limits, which basically stopped all WordPress processed till some tech support guy manually released the process watcher and activated the site. It was only thanks to caching that the site traffic sustained despite these errors.

Though this happened several times over the last few weeks, I was able to identify the plugin causing the memory usage spikes only recently. Thereafter I deactivated the plugin. It is obvious that if we need to run this plugin with all that memory usage, it is not possible on shared hosting and you need to upgrade to Dreamhost Private Server where the memory limit is as high as you set it.

Advanced Sitemap Settings

But the sitemaps are essential to let search engines know about your latest urls. The plugin author has created a wonderful FAQ which explains all these issues. Some which relates to us is

• Limit the number of posts If you have problems with the maximum execution time or memory limit you can limit the number of posts which will be included in the sitemap. Newer posts are included first so your sitemap will stay up-to-date.


• Increase the memory limit Building the sitemap needs a lot of memory. If the memory size is limited via configuration and the script can’t finish the sitemap, you can try to increase this limit by entering a higher value. The values are in megabytes so you can start with “2M” for smaller sites and raise the number until it works. However, it could be that you don’t have the permission to change this value so if it still doesn’t work and you already tried a very high value like 16M, you will need to contact your webhoster and ask him to raise it for you.


• Increase the maximum execution time Like the memory, the maximum execution time can also be limited. If the script doesn’t finish, try to set the time limit to “0″ which means unlimited or a high value like “30″ seconds.

Since it is already consuming too much memory, raising the memory or script execution time might not help. So we can continue to use the plugin after limiting the number of posts, which seems like the most practical solution as of now.

We also upgraded from PHP4 to PHP5 and both these fixes should take care of our internal server errors. Keeping our fingers crossed …

NOTE: The Google (XML) Sitemap Generator plugin is a� very useful and highly recommended plugin. This article is for informational purpose to inform that large wordpress blogs will require larger memory to generate sitemaps and sometimes it may be beyond allowed memory limits on shared hosting servers. So either you upgrade to VPS hosting or tweak the sitemap plugins to work on a limited basis.

Related articles you might like ...