Our Gravity Form is creating a MAJOR load bottleneck on our site

We have new Nagios monitoring in place to track issues with yankeemagazine.com, and this new monitoring happened to coincide with the release of the latest Great Yankee Giveaway yesterday, with an email to 17,000 readers around 2:15 p.m.

Almost immediately we began to see issues with Apache saturation – almost every Apache process on two running servers was consumed with HTTP POSTs to /giveaway, and these processes appeared to hang on for quite a long time, meaning that access to Yankeemagazine.com quickly because very sluggish as there were no available processes. This wasn't a server load issue, so no additional servers were spawned.

Can you give me some background on the system you put in place to process giveaway entries? Are there any known issues that might describe the behaviour that we saw yesterday? We are running a simple gravity form that is embeded in a post that is displaying in a feed at this URL: http://www.yankeemagazine.com/giveaway

I have new info on this topic... It is the add-ons (Aweber and WP Registration) that are the real problems for me. With those activated, the site barely loads.

Actually the issue with us has to do directly with the amount of submissions that have been passed through a given form. We cleaned out the form that was processing slow (it had roughly 20,000 submissions in the database) and the form then processed very quickly. We did some stress testing and have discovered the following:

I've done some stress testing on the /giveaway form, POSTing entries from the command line with cURL to see where or if things slow down as number of entries increases.

With the entries cleared out for the Yankee Giveaway, the POST takes 0-1 seconds.

Once the number of entries grows to 1500, this slows to 2-3 seconds.

Once the number of entries grows to 2500, this slows to 3-4 seconds.

At 3000 entries this slows to 4-5 seconds.

At 3700 entries this slows to 6-7 seconds.

At 4400 entries this slows to 8-10 seconds.

I stopped the process at 4400.

So the question is how do we make it so we don't have this slow down effect when we are collecting lots of entries for our popular giveaways....this seems to be an inherient problem with the build of the plugin. Some response from Gravity forms regarding this issue would be nice....this is kind of urgent for us.