Lately, we've been having some growing pains with ScrumDo. Occasionally, our apache+mod_wsgi stack on our application servers would just get completely overwhelmed with an amount of traffic it should have been able to handle easily.

A couple days worth of debugging wasn't getting us closer to an answer, and we weren't giving our customers the experience they deserved. So we decided to give uwsgi+nginx a try instead of apache. After all, that's what pretty much every python shop will tell you to use these days anyways.

We wanted to make sure that we were actually doing better. We decided to put both apache and nginx/uwsgi application servers behind our load balancer at the same time and direct half our traffic to either.

Since we use New Relic to monitor our application, we could use it to perform this A/B test.

First, we created a brand new application in New Relic for the nginx/uwsgi install so we could keep our stats separate.

Then we picked out a few stats to track and added them to a custom dashboard. We went with Appdex (New Relic's overall metric of user satisfaction), request response times, and page load times. The results are below. While looking at them, keep an eye on the y-axis of the middle charts since left and right don't match.

As you can see, the right side (nginx+uwsgi) performed significantly better and should lead to much happier users.

Also remember, that we've been running Apache for years, and have tweaked the config numerous times to optimize it. This was our first stab at an nginx/uwsgi config, so there may be more performance we can squeeze out with a similar amount of tweaking, making it even better.

Also note, that during the 24 hour timeframe in those graphs, we did not experience that mysterious problem we were originally trying to fix. That's likely just chance since it didn't happen every day.