Towards a general theory of JVM dancing (GAE/J)

Following my previous blog post about JVM dancing in the Google App Engine/J environment I decided to do some small testing to see how often the described "JVM switching" occurs.

The test was conducted by sending one HTTP-request per second to the same GAE/J application during a period of roughly four hours (3.7 hours). All requests were sent from the same IP-address and supplied the same JSESSIONID cookie (emulating all requests being sent from the same user within the same session). The GAE/J app that received the requests did not have any traffic other than the traffic generated by the test.

A total of 13285 requests were successfully served during this period. The requests were served by three distinct JVM:s (as identified by Runtime.getRuntime().hashCode()):

• JVM A served 9273 requests (69.8 % of all requests)
• JVM B served 4004 requests (30.1 %)
• JVM C served 8 requests (0.1 %)

During these four hours of testing the serving of the requests switched between the JVM:s as follows:

• 160 successive requests served by JVM A
• 2764 successive requests served by JVM B
• 1 request served by JVM C
• 1240 successive requests served by JVM B
• 21 successive requests served by JVM A
• 2 successive requests served by JVM C
• 208 successive requests served by JVM A
• 2 successive requests served by JVM C
• 368 successive requests served by JVM A
• 1 request served by JVM C
• 5499 successive requests served by JVM A
• 2 successive requests served by JVM C
• 3017 successive requests served by JVM A

In total 12 JVM switches took place. Which in this test translates into one JVM switch per 1107 requests (switch probability of 0.09 %).

The shortest time before a JVM switch was one second - but please note that the granularity of the test was one second since one request was sent per second.

The longest time before a JVM switch was 1.5 hours (5499 seconds).

It is probably very safe to assume that the "JVM dancing" pattern is heavily dependent on a variety of factors, including traffic patterns, general load of the GAE/J system, etc. So please take the above testing as one data point from one test - NOT an attempt to provide the general theory of JVM dancing :-)