Thursday, April 16, 2009

Know your CLASSPATH in Google App Engine/J

Continuing the exploration of the Google App Engine Java environment I decided to explore what's in the CLASSPATH of the live GAE/J environment (this differs from the CLASSPATH of the development environment).

I did the following: 

First I created a list of classes that might be available in the GAE/J live environment. The list of potential classes was created by compiling a list of all classes available in the development environment. This gave a list of 4391 class names.

From the original list of 4391 classes in the development environment I was able to successfully load 3689 classes of those using Class.forName(...) in the live environment.

After loading each class I used reflection to recursively find references to other classes using the following method:

Assume a class called A:

public class A {
  public void doSomething(B bObject) { ... }
  public C getC(int i) { ... }
}

By enumerating all methods of class A using the reflection API an looking at the return types and parameters passed to the methods we can infer the existence of classes B and C. We can then load classes B and C and find additional classes using the same method.

Using this method recursively I was able to load an additional 220 classes giving a total of 3909 classes (3689 + 220).

For each of the 3909 loadable classes I used queried for the JAR that each file was loaded from using Class.forName(...).getProtectionDomain().getCodeSource().getLocation().toString()

Of the 3909 loadable classes 2023 originated from JARs uploaded to the system (/WEB-INF/lib/*.jar). This gives a total of 1886 (3909 - 2023) loadable "GAE/J standard classes" -- that is classes that are available given an empty /WEB-INF/lib/. Please note that the total number of available classes most probably is larger since the methodology described does not catch all available classes.

Based on this analysis described above the GAE/J JVM appears to load classes from the following locations:
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/*.jar
  • JDK Base Classes (1332 classes)
  • /base/java_runtime/user-privileged.jar (1 class)
  • /base/java_runtime/runtime-shared.jar (107 classes)
  • /base/java_runtime/restricted-class-stubs.jar (31 classes)
  • /base/java_runtime/appengine-api.jar (415 classes)
What follows is a list of packages that are loaded from each location/JAR:

JDK Base Classes

  • java.*
  • javax.*
  • org.w3c.dom.*
  • org.xml.sax.*

Classes in /base/java_runtime/user-privileged.jar

  • com.google.apphosting.runtime.security.shared.CustomURLClassLoader

Packages in /base/java_runtime/runtime-shared.jar

  • com.google.apphosting.api.*
  • javax.servlet.*
  • javax.servlet.http.*
  • javax.servlet.jsp.*
  • javax.servlet.jsp.el.*
  • javax.servlet.jsp.jstl.core.*
  • javax.servlet.jsp.jstl.fmt.*
  • javax.servlet.jsp.jstl.sql.*
  • javax.servlet.jsp.jstl.tlv.*
  • javax.servlet.jsp.tagext.*

Packages in /base/java_runtime/restricted-class-stubs.jar

  • com.google.apphosting.runtime.security.shared.stub.java.awt.*
  • com.google.apphosting.runtime.security.shared.stub.java.awt.datatransfer.*
  • com.google.apphosting.runtime.security.shared.stub.java.awt.event.*
  • com.google.apphosting.runtime.security.shared.stub.java.beans.*
  • com.google.apphosting.runtime.security.shared.stub.java.io.*
  • com.google.apphosting.runtime.security.shared.stub.java.lang.instrument.*
  • com.google.apphosting.runtime.security.shared.stub.java.net.*
  • com.google.apphosting.runtime.security.shared.stub.java.nio.*
  • com.google.apphosting.runtime.security.shared.stub.java.nio.channels.*
  • com.google.apphosting.runtime.security.shared.stub.java.nio.channels.spi.*
  • com.google.apphosting.runtime.security.shared.stub.java.text.*
  • com.google.apphosting.runtime.security.shared.stub.javax.naming.*
  • com.google.apphosting.runtime.security.shared.stub.javax.swing.text.*
  • com.google.apphosting.runtime.security.shared.stub.javax.tools.*

Packages in /base/java_runtime/appengine-api.jar

  • com.google.appengine.api.*
  • com.google.appengine.api.datastore.*
  • com.google.appengine.api.images.*
  • com.google.appengine.api.mail.*
  • com.google.appengine.api.mail.stdimpl.*
  • com.google.appengine.api.memcache.*
  • com.google.appengine.api.memcache.stdimpl.*
  • com.google.appengine.api.urlfetch.*
  • com.google.appengine.api.users.*
  • com.google.appengine.repackaged.com.google.common.base.*
  • com.google.appengine.repackaged.com.google.common.base.genfiles.*
  • com.google.appengine.repackaged.com.google.common.base.internal.*
  • com.google.appengine.repackaged.com.google.common.collect.*
  • com.google.appengine.repackaged.com.google.common.io.*
  • com.google.appengine.repackaged.com.google.common.primitives.*
  • com.google.appengine.repackaged.com.google.common.util.*
  • com.google.appengine.repackaged.com.google.io.base.*
  • com.google.appengine.repackaged.com.google.io.protocol.*
  • com.google.appengine.repackaged.com.google.io.protocol.proto.*
  • com.google.apphosting.api.*
  • com.google.apphosting.utils.servlet.*
  • com.google.storage.onestore.v3.*
  • javax.cache.*
  • javax.mail.*
  • javax.mail.event.*
  • javax.mail.internet.*
  • javax.mail.search.*
  • javax.mail.util.*
  • org.apache.geronimo.mail.handlers.*
  • org.apache.geronimo.mail.util.*
That's all of the CLASSPATH analysis for now!

Hope you didn't miss today's great GAE/J news: four hours ago the Grails project lead Graeme Rocher got Grails up and running on the AppEngine/J. That is excellent news for all us Google App Engine- and Grails-fans! I really do believe the scalability of GAE/J in combination with the high productivity of the Grails framework will be a killer combination! Exciting times! :-)

10 comments:

  1. Great work!
    Keep it going ;-)

    Franck Arnulfo

    ReplyDelete
  2. Thanks for your kind words!

    I've posted a follow-up to this blog post here with a list of available classes that are not in the white-list.

    ReplyDelete
  3. Awesome!

    One thing I'd also like to mention... You can override classes in the appengine jars if you include them as deployed source files, so one can assume the classpath ordering as:
    source files
    appengine apis
    war/WEB-INF/lib

    ReplyDelete
  4. It’s really a nice and useful piece of info. I’m satisfied that you simply shared this useful info with us. Please stay us up to date like this. Thanks for sharing.
    Commodity tips
    Stock Market
    Share Market
    Nse Tips
    Bse Tips

    ReplyDelete
  5. Wow, awesome blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your website is great, let alone the content!
    Ncdex Tips || Indian Stock Market || Bullion Tips || Nifty Future Tips || Nifty Option Tips || Future Tips

    ReplyDelete
  6. A good informative post that you have shared and thankful your work for sharing the information. Got some appealing information and would like to give it a try. Applaud your work and keep sharing your information.Really impressed! Everything is very open and very clear clarification of issues.
    Call Option || Option Tips || Stock Option Tips || Options || Nifty Options

    ReplyDelete
  7. This is a great post ! it was very informative. I look forward in reading more of your work.
    Share market tips
    Intraday Nifty Future Tips
    Share and Earn

    ReplyDelete
  8. I had no trouble navigating through all the tabs as well as related info. The site ended up being truly simple to access.
    share market
    stock market
    Indian Stock Market Tips

    ReplyDelete
  9. I like concept of your post. Very creative post. Best of luck and waiting for some new ideas. I am looking for such useful information and I found here a lot of it.
    Buy Property
    Sell Property
    Invest in Property
    Find real estate agent

    ReplyDelete
  10. your blog suggestion is knowledge full information keep continue provide information very helpful .
    mcx tips with 100 accuracy

    ReplyDelete