Go Spiny Norman, Go!

Update 14 Oct 2011: The green line from my graphs below has just appeared in the AppEngine admin console's Instances graph, as "Billing". Well actually it's the minimum of my green line and the blue "Total Instances" line, ie: it defines what I show as the pink area. Anyway, that's really, really useful, thanks to the AppEngine team from us developers!


---


tl;dr version: You do *not* have to minimise your app's instance count in order to keep your appengine costs in an acceptable range. What you do have to do is to set Max Idle Instances to a low number. If you do that, your app can cost you about the same as it costs you now, possibly less. You don't need multithreading in order to achieve decent pricing. You will see advice like this: "Forget about the scheduler. Turn on multithreading ASAP". That is wrong.

----

It looks like the Spiny Norman Test was successful.

Recall the hypothesis:

Hypothesis: Ignoring the 15 minute cost for spinning up new instances, the price we pay should be the pink area on the graph. That is, the moment by moment minimum of (total instances) and (active instances + Max Idle Instances). If Max Idle Instances is Automatic, then there is no green line, and we pay for the area under the blue line.

I proposed 
1 - First test that we pay for the area under the blue line when Max Idle Instances is Automatic.
2 - Next, test that we pay for the pink area when Max Idle Instances is set to something.

I ran two tests. Firstly, I ran Spiny Norman with Max Idle Instances set to Automatic. Secondly, I ran Spiny Norman with Max Idle Instances set to 2.

Test 1: Max Idle Instances set to Automatic

Here's the instance graph for the day:

The area under the blue line looks to be roughly 7 * 20 = 140 hours. The billing says:

Resource	Used	Free	Billable	Charge
Frontend Instance Hours $0.04/Hour	130.46	28.00	102.46	$4.10

Looks about right! So if you leave Max Idle Instances set to Automatic, you'll pay for the entire blue area.

Test 2: Max Idle Instances set to 2

Here's the instance graph for the day of this test:

That, believe it or not, is an identical run of Spiny Norman. Why the huge blowout of instances in the second part of the day? No idea, maybe the Max Idle Instances = 2 setting caused the scheduler to get upset? In any case, here we care about the yellow line, active instances, not the blue line. Here's a modified graph with the predicted "pink area" as above, for yellow line + 2:

Spiny Norman uses negligible instance time (very low Active line), he just causes total (idle) instances to blow out. So it looks like that area is around, say, 0+2 instances * 20 hours? So approx 40 hours.

And the billing says:

Resource	Used	Free	Billable	Charge
Frontend Instance Hours $0.04/Hour	44.90	28.00	16.90	$0.68

That's about right!

Conclusion

I could do some more in depth tests, changing the behaviour of Spiny Norman over the course of a day, playing with the Max Idle Instances setting, but I think these two tests show the state of play pretty adequately. Hypothesis supported.

So what this means, especially for those of us watching our pennies, is that even though we can't stop the scheduler kicking off massive amounts of instances, we can control whether we pay for them or not. Make sure that you set Max Idle Instances to a fixed (low!) number. For my self funded projects I'll be setting it to 1, and that'll do.

Leaving Max Idle Instances on Automatic, the default, is a mistake you'll regret very, very quickly.

Of course the billing rules will probably change tomorrow. Ah well.