Google AppEngine - Java vs Python Performance Comparison
I am quite convinced that Google AppEngine is a very significant piece of "cloud computing" technology. More than other cloud components, it has been designed to scale from small, low request count websites to the big suckers. That is, until you come across your first Loading Request in the world of the AppEngine Java (and friends - Scala, Groovy, etc).
I found out about Loading Requests in AppEngine, when I pushed my first Gaelyk application to it's home on AppEngine. I've been keeping an eye on the performance of my sites and applications using Pingdom monitoring, and unfortunately I saw something that made me feel a little sad when I saw the results.
|Platform||Avg Response||Slow Average||Fast Average|
|AppEngine Vosao CMS (JVM Java)||1380 ms||2063 ms||938 ms|
|AppEngine Lift Framework (JVM Scala)||506 ms||600 ms||438 ms|
|AppEngine Gaelyk Site (JVM Groovy)||11,597 ms||12,776 ms||10,565 ms|
|AppEngine Site Proxy (Python)||504 ms||713 ms||372 ms|
Comparing the performance of my Gaelyk skeleton site (WIP @ sparkz.distractable.net) with my own (pretty rough) Python AppEngine Proxy, the Vosao CMS running the Sidelab site, and the Lift examples on AppEngine you can see that Gaelyk really lags behind in terms of performance.
Update 2010-04-25: As comments around the place have suggested, note that the Scala Lift vs Groovy Gaelyk isn't really an apples vs apples comparison. In my view this is primarily because I was running tests against the established appspot lift samples. I provided more detail on this in a comment below
Now, I had pretty much decided that I was going to use Groovy and Gaelyk in some future projects, but these loading requests really throw a cat amongst the pigeons. I believe some people are using cron jobs to keep the AppEngine JVM warm and prevent it from unloading, but this practice is discouraged by Google (understandably). Considering also that my pingdom check interval is 5 minutes if it's unloading between those requests, then I don't hold much hope until Google do something about this.
To that end, there are a number of things in the pipeline that will probably help things along. The ability to precompile was introduced in AppEngine 1.2.8, and while it does make a difference when you enable it looking at my detailed pingdom logs requests are still taking almost 10 seconds to complete. Additionally, there is an issue worth starring that if implemented will hopefully enable people to reserve a JVM instance. Doing this would bring response times of my very simple Gaelyk app down to around the 250ms time. Hurrah.
In the meantime though, I think I have to seriously consider either going back to Python for AppEngine development or investigate Scala and Lift. Any opinions? I definitely know more people who are passionate about Scala, but really, it's still JVM based and definitely nowhere near as mature as Python. Excuse me now while I wipe my tears shed for Groovy and Gaelyk away.
While I'm doing that though, here are a couple of other useful resources on the topic:
- Google App Engine Loading Requests Happen Very Frequently @ StackOverflow
Update 2010-04-25: I have posted a follow up to this post in an effort to clear up a few points of confusion. Additionally, it looks at the whether making requests to your site every minute (instead of every 5 minutes) reduces the impact of a loading request.
Update 2010-12-17: Make sure you check out the AppEngine 1.4.0 release as it has some solutions to the problems that I experienced when writing this post.