Daily Archives: May 30, 2009

Reducing Data Store Contention in App Engine get_or_insert

I was giving a demo at the Apple Academix Conference during a keynote where I use a Google App Engine application to simulate a clicker-like activity in a course. It is a simple multi-user number guessing game. Whenever the audience participation was too good and too many people started clicking on the application I would get errors – it made the demo in the middle of my talk lose a bit of credibility – but it was also funny.

I finally saw the error and spent a few hours to track the issue down. I was getting the following message:

TransactionFailedError: too much contention on these datastore
entities. please try again.

I had been using get_or_insert() pretty much with reckless abandon and wondered if it was smart enough to optimistically do a get() and see if the object was there before starting a transaction to do the get() and then put() if necessary. But given the behavior I was seeing – it looked like it was starting a transaction every time.

I had 20 participants and after the first click in the application, there were no new objects in the application and 20 people clicking within a 10 second interval were causing the error to occur. I suspect but cannot confirm that at some level the AE framework marks you as abusing transactions and then it takes a while before load drops down that it lets you once again use transactions.

So I decided to reduce my transaction abuse and build an optimistic get or insert that does the get() first and then if the record does not exist do the get_or_insert(). This way once objects start to exist there will be no more transactions. And yes, my next optimization is to use memcache.

So this was my approach.

Continue reading