Monday, April 27, 2009

Getting started with Grails 1.1.1-SNAPSHOT on Google App Engine/J

Official Grails support for Google App Engine/J is coming with Grails 1.1.1 that is set to be released this week according to a recent Twitter post from Grails project lead Graeme Rocher.

I'm really looking forward to this release - the combination of Grails (high productivity) and GAE/J (high scalability) is a very nice match.

If you're eager to get going before the official Grails 1.1.1 release - follow this small guide to build Grails 1.1.1-SNAPSHOT from source and deploy your first Grails GAE/J application.

# Step 1: Fetch and build Grails 1.1.1-SNAPSHOT
cd
mkdir grails-1.1.1-build
cd grails-1.1.1-build
git clone git://github.com/grails/grails.git
cd grails/grails
export ANT_OPTS="-Xmx512m"
ant jar
export GRAILS_HOME="$(pwd)"
export PATH="${GRAILS_HOME}/bin:${PATH}"
grails
# Version string should show "Grails 1.1.1-SNAPSHOT"

# Step 2: Create Grails-GAE/J application
grails create-app gaetest
cd gaetest
grails uninstall-plugin hibernate
emacs grails-app/conf/BuildConfig.groovy
# Add: google.appengine.sdk="/path/to/appengine-java-sdk-1.2.0"
grails install-plugin app-engine
# Fix a bug in the app-engine plugin.
emacs ~/.grails/1.1.1-SNAPSHOT/projects/gaetest/plugins/app-engine-0.5/scripts/_Events.groovy
# Add <sessions-enabled>true</sessions-enabled> after <version>${appVersion}</version>
grails install-templates
emacs src/templates/war/web.xml
# Remove the <welcome-file-list> definition
grails app-engine run
# Now browse to http://localhost:8080/

# Step 3: Deploy Grails-GAE/J application
grails set-version 1
emacs grails-app/conf/Config.groovy
# Add: google.appengine.application="app-id"
grails app-engine deploy
# The app should now be deployed at http://app-id.appspot.com/

Saturday, April 25, 2009

Three GAE/J issues reported

I've reported three (very minor) issues to the Google App Engine issue tracker:

#1414: NullPointerException when deploying with web.xml = <web-app />

#1415: .java/.jsp files uploaded to the live environment?

#1416: Extra JAR:s being added to WEB-INF/lib/ and uploaded on each deployment

These three issues were discussed in some detail in my blog post "Deployment on GAE/J and the minimal GAE/J application".

Available classes not listed in the GAE/J white-list

An application running in the Google App Engine/J has restricted access to the classes in the Java standard library (the JRE). The classes that are available are listed in the official JRE Class White List.

As described in my earlier blog post "Know your CLASSPATH in Google App Engine/J" the JRE Class White List is not a complete list of all classes available by default in the GAE/J environment.

Based on the methodology described in the previous post I've compiled a list of classes that are available by default in the GAE/J (defined as being loadable using Class.forName(...) given an empty /WEB-INF/lib/).

I've excluded all classes listed in the standard JRE Class White List. Furthermore, I've removed all stub classes in com.google.apphosting.runtime.security.shared.stub.

This is the list of classes with the originating JAR:s within parenthesis:
  • com.google.appengine.api.NamespaceManager (appengine-api.jar)
  • com.google.appengine.api.datastore.Blob (appengine-api.jar)
  • com.google.appengine.api.datastore.DataTypeTranslator (appengine-api.jar)
  • com.google.appengine.api.datastore.DataTypeUtils (appengine-api.jar)
  • com.google.appengine.api.datastore.DatastoreApiHelper (appengine-api.jar)
  • com.google.appengine.api.datastore.DatastoreConfig (appengine-api.jar)
  • com.google.appengine.api.datastore.DatastoreFailureException (appengine-api.jar)
  • com.google.appengine.api.datastore.DatastoreNeedIndexException (appengine-api.jar)
  • com.google.appengine.api.datastore.DatastoreService (appengine-api.jar)
  • com.google.appengine.api.datastore.DatastoreServiceFactory (appengine-api.jar)
  • com.google.appengine.api.datastore.DatastoreServiceImpl (appengine-api.jar)
  • com.google.appengine.api.datastore.DatastoreTimeoutException (appengine-api.jar)
  • com.google.appengine.api.datastore.Entity (appengine-api.jar)
  • com.google.appengine.api.datastore.EntityNotFoundException (appengine-api.jar)
  • com.google.appengine.api.datastore.EntityTranslator (appengine-api.jar)
  • com.google.appengine.api.datastore.FetchOptions (appengine-api.jar)
  • com.google.appengine.api.datastore.ImplicitTransactionManagementPolicy (appengine-api.jar)
  • com.google.appengine.api.datastore.Key (appengine-api.jar)
  • com.google.appengine.api.datastore.KeyFactory (appengine-api.jar)
  • com.google.appengine.api.datastore.KeyTranslator (appengine-api.jar)
  • com.google.appengine.api.datastore.Link (appengine-api.jar)
  • com.google.appengine.api.datastore.PreparedQuery (appengine-api.jar)
  • com.google.appengine.api.datastore.Query (appengine-api.jar)
  • com.google.appengine.api.datastore.QueryIterator (appengine-api.jar)
  • com.google.appengine.api.datastore.QueryResultsSource (appengine-api.jar)
  • com.google.appengine.api.datastore.QueryTranslator (appengine-api.jar)
  • com.google.appengine.api.datastore.ShortBlob (appengine-api.jar)
  • com.google.appengine.api.datastore.Text (appengine-api.jar)
  • com.google.appengine.api.datastore.Transaction (appengine-api.jar)
  • com.google.appengine.api.datastore.TransactionImpl (appengine-api.jar)
  • com.google.appengine.api.datastore.TransactionRunner (appengine-api.jar)
  • com.google.appengine.api.datastore.TransactionStack (appengine-api.jar)
  • com.google.appengine.api.datastore.TransactionStackImpl (appengine-api.jar)
  • com.google.appengine.api.images.Composite (appengine-api.jar)
  • com.google.appengine.api.images.CompositeImpl (appengine-api.jar)
  • com.google.appengine.api.images.CompositeTransform (appengine-api.jar)
  • com.google.appengine.api.images.Crop (appengine-api.jar)
  • com.google.appengine.api.images.HorizontalFlip (appengine-api.jar)
  • com.google.appengine.api.images.ImFeelingLucky (appengine-api.jar)
  • com.google.appengine.api.images.Image (appengine-api.jar)
  • com.google.appengine.api.images.ImageImpl (appengine-api.jar)
  • com.google.appengine.api.images.ImagesService (appengine-api.jar)
  • com.google.appengine.api.images.ImagesServiceFactory (appengine-api.jar)
  • com.google.appengine.api.images.ImagesServiceFailureException (appengine-api.jar)
  • com.google.appengine.api.images.ImagesServiceImpl (appengine-api.jar)
  • com.google.appengine.api.images.ImagesServicePb (appengine-api.jar)
  • com.google.appengine.api.images.Resize (appengine-api.jar)
  • com.google.appengine.api.images.Rotate (appengine-api.jar)
  • com.google.appengine.api.images.Transform (appengine-api.jar)
  • com.google.appengine.api.images.VerticalFlip (appengine-api.jar)
  • com.google.appengine.api.mail.MailService (appengine-api.jar)
  • com.google.appengine.api.mail.MailServiceFactory (appengine-api.jar)
  • com.google.appengine.api.mail.MailServiceImpl (appengine-api.jar)
  • com.google.appengine.api.mail.MailServicePb (appengine-api.jar)
  • com.google.appengine.api.mail.stdimpl.GMTransport (appengine-api.jar)
  • com.google.appengine.api.memcache.ErrorHandler (appengine-api.jar)
  • com.google.appengine.api.memcache.Expiration (appengine-api.jar)
  • com.google.appengine.api.memcache.InvalidValueException (appengine-api.jar)
  • com.google.appengine.api.memcache.LogAndContinueErrorHandler (appengine-api.jar)
  • com.google.appengine.api.memcache.MemcacheSerialization (appengine-api.jar)
  • com.google.appengine.api.memcache.MemcacheService (appengine-api.jar)
  • com.google.appengine.api.memcache.MemcacheServiceException (appengine-api.jar)
  • com.google.appengine.api.memcache.MemcacheServiceFactory (appengine-api.jar)
  • com.google.appengine.api.memcache.MemcacheServiceImpl (appengine-api.jar)
  • com.google.appengine.api.memcache.MemcacheServicePb (appengine-api.jar)
  • com.google.appengine.api.memcache.Stats (appengine-api.jar)
  • com.google.appengine.api.memcache.StrictErrorHandler (appengine-api.jar)
  • com.google.appengine.api.memcache.stdimpl.GCache (appengine-api.jar)
  • com.google.appengine.api.memcache.stdimpl.GCacheEntry (appengine-api.jar)
  • com.google.appengine.api.memcache.stdimpl.GCacheException (appengine-api.jar)
  • com.google.appengine.api.memcache.stdimpl.GCacheFactory (appengine-api.jar)
  • com.google.appengine.api.urlfetch.FetchOptions (appengine-api.jar)
  • com.google.appengine.api.urlfetch.HTTPHeader (appengine-api.jar)
  • com.google.appengine.api.urlfetch.HTTPMethod (appengine-api.jar)
  • com.google.appengine.api.urlfetch.HTTPRequest (appengine-api.jar)
  • com.google.appengine.api.urlfetch.HTTPResponse (appengine-api.jar)
  • com.google.appengine.api.urlfetch.ResponseTooLargeException (appengine-api.jar)
  • com.google.appengine.api.urlfetch.URLFetchService (appengine-api.jar)
  • com.google.appengine.api.urlfetch.URLFetchServiceFactory (appengine-api.jar)
  • com.google.appengine.api.urlfetch.URLFetchServiceImpl (appengine-api.jar)
  • com.google.appengine.api.urlfetch.URLFetchServicePb (appengine-api.jar)
  • com.google.appengine.api.users.User (appengine-api.jar)
  • com.google.appengine.api.users.UserService (appengine-api.jar)
  • com.google.appengine.api.users.UserServiceFactory (appengine-api.jar)
  • com.google.appengine.api.users.UserServiceFailureException (appengine-api.jar)
  • com.google.appengine.api.users.UserServiceImpl (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.CharEscaper (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.CharEscaperBuilder (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Escaper (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.FinalizableReference (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.FinalizableSoftReference (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.FinalizableWeakReference (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Function (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Hash (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Join (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Joiner (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Objects (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Pair (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Preconditions (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Predicate (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Predicates (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Receiver (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.ReferenceType (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.Supplier (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.X (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.genfiles.IntArray (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.base.internal.Finalizer (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.AbstractIterator (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.AbstractMapBasedMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.AbstractMapEntry (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.AbstractMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ArrayListMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.AsynchronousComputationException (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.BiMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.CancelableReferenceCache (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ClassToInstanceMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Collections2 (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Comparators (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ComputationException (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ConcreteMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ConcurrentMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Constraint (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Constraints (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.CustomConcurrentHashMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.EnumBiMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.EnumHashBiMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.EnumMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ExpiringReferenceCache (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ExpiringReferenceMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingCollection (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingConcurrentMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingIterator (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingList (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingListIterator (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingMapEntry (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingObject (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingQueue (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingSet (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingSortedMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ForwardingSortedSet (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.HashBasedTable (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.HashBiMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.HashMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.HashMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Hashing (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableBiMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableBiMapBuilder (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableCollection (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableEntry (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableList (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableMapBuilder (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableMultimapBuilder (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableSet (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ImmutableSortedSet (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Iterables (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Iterators (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.LinkedHashMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.LinkedHashMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.LinkedListMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ListMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Lists (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.MapConstraint (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.MapConstraints (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.MapDifference (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.MapMaker (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Maps (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Multimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Multimaps (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Multiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Multisets (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.NpeThrowingAbstractMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.NullOutputException (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ObjectArrays (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Ordering (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.PeekingIterator (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Platform (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.PrefixMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.PrefixTrie (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.PrimitiveArrays (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ReferenceCache (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.ReferenceMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Serialization (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.SetMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.SortedArraySet (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.SortedSetMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.StandardBiMap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.StandardListMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.StandardMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.StandardRowSortedTable (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.StandardSetMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.StandardSortedSetMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.StandardTable (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Synchronized (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Table (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.Tables (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.TreeBasedTable (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.TreeMultimap (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.TreeMultiset (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.UncheckedThrower (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.collect.UnmodifiableIterator (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.io.ByteArrayDataInput (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.io.ByteArrayDataOutput (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.io.Bytes (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.io.Closeables (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.io.InputSupplier (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.io.MultiInputStream (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.io.OutputSupplier (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.primitives.IntQueue (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.primitives.IntStack (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.util.Base64 (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.common.util.Base64DecoderException (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.base.Buffers (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.base.FileSystem (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.base.IORuntimeException (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.base.PerThreadByteCounter (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.base.ServerAddress (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.base.VarInt (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.MessageSet (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.MessageVisitor (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.Protocol (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.ProtocolMessage (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.ProtocolMessageEnum (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.ProtocolSink (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.ProtocolSource (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.ProtocolSupport (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.ProtocolType (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.RawMessage (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.UninterpretedTags (appengine-api.jar)
  • com.google.appengine.repackaged.com.google.io.protocol.proto.ProtocolDescriptor (appengine-api.jar)
  • com.google.apphosting.api.ApiBasePb (appengine-api.jar)
  • com.google.apphosting.api.ApiProxy (runtime-shared.jar)
  • com.google.apphosting.api.DatastorePb (appengine-api.jar)
  • com.google.apphosting.api.DeadlineExceededException (runtime-shared.jar)
  • com.google.apphosting.api.UserServicePb (appengine-api.jar)
  • com.google.apphosting.runtime.security.shared.CustomURLClassLoader (user-privileged.jar)
  • com.google.apphosting.utils.servlet.SessionCleanupServlet (appengine-api.jar)
  • com.google.apphosting.utils.servlet.TransactionCleanupFilter (appengine-api.jar)
  • com.google.storage.onestore.v3.OnestoreEntity (appengine-api.jar)
  • java.awt.datatransfer.DataFlavor (JDK)
  • java.awt.datatransfer.MimeType (JDK)
  • java.awt.datatransfer.Transferable (JDK)
  • javax.activation.ActivationDataFlavor (JDK)
  • javax.activation.CommandInfo (JDK)
  • javax.activation.CommandMap (JDK)
  • javax.activation.CommandObject (JDK)
  • javax.activation.DataContentHandler (JDK)
  • javax.activation.DataContentHandlerFactory (JDK)
  • javax.activation.DataHandler (JDK)
  • javax.activation.DataSource (JDK)
  • javax.activation.FileDataSource (JDK)
  • javax.activation.FileTypeMap (JDK)
  • javax.activation.MailcapCommandMap (JDK)
  • javax.activation.MimeType (JDK)
  • javax.activation.MimeTypeParameterList (JDK)
  • javax.activation.MimeTypeParseException (JDK)
  • javax.activation.MimetypesFileTypeMap (JDK)
  • javax.activation.URLDataSource (JDK)
  • javax.activation.UnsupportedDataTypeException (JDK)
  • javax.cache.Cache (appengine-api.jar)
  • javax.cache.CacheEntry (appengine-api.jar)
  • javax.cache.CacheException (appengine-api.jar)
  • javax.cache.CacheFactory (appengine-api.jar)
  • javax.cache.CacheListener (appengine-api.jar)
  • javax.cache.CacheLoader (appengine-api.jar)
  • javax.cache.CacheManager (appengine-api.jar)
  • javax.cache.CacheStatistics (appengine-api.jar)
  • javax.cache.EvictionStrategy (appengine-api.jar)
  • javax.mail.Address (appengine-api.jar)
  • javax.mail.AuthenticationFailedException (appengine-api.jar)
  • javax.mail.Authenticator (appengine-api.jar)
  • javax.mail.BodyPart (appengine-api.jar)
  • javax.mail.EventQueue (appengine-api.jar)
  • javax.mail.FetchProfile (appengine-api.jar)
  • javax.mail.Flags (appengine-api.jar)
  • javax.mail.Folder (appengine-api.jar)
  • javax.mail.FolderClosedException (appengine-api.jar)
  • javax.mail.FolderNotFoundException (appengine-api.jar)
  • javax.mail.Header (appengine-api.jar)
  • javax.mail.IllegalWriteException (appengine-api.jar)
  • javax.mail.Message (appengine-api.jar)
  • javax.mail.MessageAware (appengine-api.jar)
  • javax.mail.MessageContext (appengine-api.jar)
  • javax.mail.MessageRemovedException (appengine-api.jar)
  • javax.mail.MessagingException (appengine-api.jar)
  • javax.mail.MethodNotSupportedException (appengine-api.jar)
  • javax.mail.Multipart (appengine-api.jar)
  • javax.mail.MultipartDataSource (appengine-api.jar)
  • javax.mail.NoSuchProviderException (appengine-api.jar)
  • javax.mail.Part (appengine-api.jar)
  • javax.mail.PasswordAuthentication (appengine-api.jar)
  • javax.mail.Provider (appengine-api.jar)
  • javax.mail.Quota (appengine-api.jar)
  • javax.mail.QuotaAwareStore (appengine-api.jar)
  • javax.mail.ReadOnlyFolderException (appengine-api.jar)
  • javax.mail.SendFailedException (appengine-api.jar)
  • javax.mail.Service (appengine-api.jar)
  • javax.mail.Session (appengine-api.jar)
  • javax.mail.Store (appengine-api.jar)
  • javax.mail.StoreClosedException (appengine-api.jar)
  • javax.mail.Transport (appengine-api.jar)
  • javax.mail.UIDFolder (appengine-api.jar)
  • javax.mail.URLName (appengine-api.jar)
  • javax.mail.event.ConnectionAdapter (appengine-api.jar)
  • javax.mail.event.ConnectionEvent (appengine-api.jar)
  • javax.mail.event.ConnectionListener (appengine-api.jar)
  • javax.mail.event.FolderAdapter (appengine-api.jar)
  • javax.mail.event.FolderEvent (appengine-api.jar)
  • javax.mail.event.FolderListener (appengine-api.jar)
  • javax.mail.event.MailEvent (appengine-api.jar)
  • javax.mail.event.MessageChangedEvent (appengine-api.jar)
  • javax.mail.event.MessageChangedListener (appengine-api.jar)
  • javax.mail.event.MessageCountAdapter (appengine-api.jar)
  • javax.mail.event.MessageCountEvent (appengine-api.jar)
  • javax.mail.event.MessageCountListener (appengine-api.jar)
  • javax.mail.event.StoreEvent (appengine-api.jar)
  • javax.mail.event.StoreListener (appengine-api.jar)
  • javax.mail.event.TransportAdapter (appengine-api.jar)
  • javax.mail.event.TransportEvent (appengine-api.jar)
  • javax.mail.event.TransportListener (appengine-api.jar)
  • javax.mail.internet.AddressException (appengine-api.jar)
  • javax.mail.internet.AddressParser (appengine-api.jar)
  • javax.mail.internet.ContentCheckingOutputStream (appengine-api.jar)
  • javax.mail.internet.ContentDisposition (appengine-api.jar)
  • javax.mail.internet.ContentType (appengine-api.jar)
  • javax.mail.internet.HeaderTokenizer (appengine-api.jar)
  • javax.mail.internet.InternetAddress (appengine-api.jar)
  • javax.mail.internet.InternetHeaders (appengine-api.jar)
  • javax.mail.internet.MailDateFormat (appengine-api.jar)
  • javax.mail.internet.MimeBodyPart (appengine-api.jar)
  • javax.mail.internet.MimeMessage (appengine-api.jar)
  • javax.mail.internet.MimeMultipart (appengine-api.jar)
  • javax.mail.internet.MimePart (appengine-api.jar)
  • javax.mail.internet.MimePartDataSource (appengine-api.jar)
  • javax.mail.internet.MimeUtility (appengine-api.jar)
  • javax.mail.internet.NewsAddress (appengine-api.jar)
  • javax.mail.internet.ParameterList (appengine-api.jar)
  • javax.mail.internet.ParseException (appengine-api.jar)
  • javax.mail.internet.PreencodedMimeBodyPart (appengine-api.jar)
  • javax.mail.internet.SharedInputStream (appengine-api.jar)
  • javax.mail.search.AddressStringTerm (appengine-api.jar)
  • javax.mail.search.AddressTerm (appengine-api.jar)
  • javax.mail.search.AndTerm (appengine-api.jar)
  • javax.mail.search.BodyTerm (appengine-api.jar)
  • javax.mail.search.ComparisonTerm (appengine-api.jar)
  • javax.mail.search.DateTerm (appengine-api.jar)
  • javax.mail.search.FlagTerm (appengine-api.jar)
  • javax.mail.search.FromStringTerm (appengine-api.jar)
  • javax.mail.search.FromTerm (appengine-api.jar)
  • javax.mail.search.HeaderTerm (appengine-api.jar)
  • javax.mail.search.IntegerComparisonTerm (appengine-api.jar)
  • javax.mail.search.MessageIDTerm (appengine-api.jar)
  • javax.mail.search.MessageNumberTerm (appengine-api.jar)
  • javax.mail.search.NotTerm (appengine-api.jar)
  • javax.mail.search.OrTerm (appengine-api.jar)
  • javax.mail.search.ReceivedDateTerm (appengine-api.jar)
  • javax.mail.search.RecipientStringTerm (appengine-api.jar)
  • javax.mail.search.RecipientTerm (appengine-api.jar)
  • javax.mail.search.SearchException (appengine-api.jar)
  • javax.mail.search.SearchTerm (appengine-api.jar)
  • javax.mail.search.SentDateTerm (appengine-api.jar)
  • javax.mail.search.SizeTerm (appengine-api.jar)
  • javax.mail.search.StringTerm (appengine-api.jar)
  • javax.mail.search.SubjectTerm (appengine-api.jar)
  • javax.mail.util.ByteArrayDataSource (appengine-api.jar)
  • javax.mail.util.SharedByteArrayInputStream (appengine-api.jar)
  • javax.mail.util.SharedFileInputStream (appengine-api.jar)
  • javax.servlet.Filter (runtime-shared.jar)
  • javax.servlet.FilterChain (runtime-shared.jar)
  • javax.servlet.FilterConfig (runtime-shared.jar)
  • javax.servlet.GenericServlet (runtime-shared.jar)
  • javax.servlet.RequestDispatcher (runtime-shared.jar)
  • javax.servlet.Servlet (runtime-shared.jar)
  • javax.servlet.ServletConfig (runtime-shared.jar)
  • javax.servlet.ServletContext (runtime-shared.jar)
  • javax.servlet.ServletContextAttributeEvent (runtime-shared.jar)
  • javax.servlet.ServletContextAttributeListener (runtime-shared.jar)
  • javax.servlet.ServletContextEvent (runtime-shared.jar)
  • javax.servlet.ServletContextListener (runtime-shared.jar)
  • javax.servlet.ServletException (runtime-shared.jar)
  • javax.servlet.ServletInputStream (runtime-shared.jar)
  • javax.servlet.ServletOutputStream (runtime-shared.jar)
  • javax.servlet.ServletRequest (runtime-shared.jar)
  • javax.servlet.ServletRequestAttributeEvent (runtime-shared.jar)
  • javax.servlet.ServletRequestAttributeListener (runtime-shared.jar)
  • javax.servlet.ServletRequestEvent (runtime-shared.jar)
  • javax.servlet.ServletRequestListener (runtime-shared.jar)
  • javax.servlet.ServletRequestWrapper (runtime-shared.jar)
  • javax.servlet.ServletResponse (runtime-shared.jar)
  • javax.servlet.ServletResponseWrapper (runtime-shared.jar)
  • javax.servlet.SingleThreadModel (runtime-shared.jar)
  • javax.servlet.UnavailableException (runtime-shared.jar)
  • javax.servlet.http.Cookie (runtime-shared.jar)
  • javax.servlet.http.HttpServlet (runtime-shared.jar)
  • javax.servlet.http.HttpServletRequest (runtime-shared.jar)
  • javax.servlet.http.HttpServletRequestWrapper (runtime-shared.jar)
  • javax.servlet.http.HttpServletResponse (runtime-shared.jar)
  • javax.servlet.http.HttpServletResponseWrapper (runtime-shared.jar)
  • javax.servlet.http.HttpSession (runtime-shared.jar)
  • javax.servlet.http.HttpSessionActivationListener (runtime-shared.jar)
  • javax.servlet.http.HttpSessionAttributeListener (runtime-shared.jar)
  • javax.servlet.http.HttpSessionBindingEvent (runtime-shared.jar)
  • javax.servlet.http.HttpSessionBindingListener (runtime-shared.jar)
  • javax.servlet.http.HttpSessionContext (runtime-shared.jar)
  • javax.servlet.http.HttpSessionEvent (runtime-shared.jar)
  • javax.servlet.http.HttpSessionListener (runtime-shared.jar)
  • javax.servlet.http.HttpUtils (runtime-shared.jar)
  • javax.servlet.http.NoBodyOutputStream (runtime-shared.jar)
  • javax.servlet.http.NoBodyResponse (runtime-shared.jar)
  • javax.servlet.jsp.ErrorData (runtime-shared.jar)
  • javax.servlet.jsp.HttpJspPage (runtime-shared.jar)
  • javax.servlet.jsp.JspApplicationContext (runtime-shared.jar)
  • javax.servlet.jsp.JspContext (runtime-shared.jar)
  • javax.servlet.jsp.JspEngineInfo (runtime-shared.jar)
  • javax.servlet.jsp.JspException (runtime-shared.jar)
  • javax.servlet.jsp.JspFactory (runtime-shared.jar)
  • javax.servlet.jsp.JspPage (runtime-shared.jar)
  • javax.servlet.jsp.JspTagException (runtime-shared.jar)
  • javax.servlet.jsp.JspWriter (runtime-shared.jar)
  • javax.servlet.jsp.PageContext (runtime-shared.jar)
  • javax.servlet.jsp.SkipPageException (runtime-shared.jar)
  • javax.servlet.jsp.el.ELException (runtime-shared.jar)
  • javax.servlet.jsp.el.ELParseException (runtime-shared.jar)
  • javax.servlet.jsp.el.Expression (runtime-shared.jar)
  • javax.servlet.jsp.el.ExpressionEvaluator (runtime-shared.jar)
  • javax.servlet.jsp.el.FunctionMapper (runtime-shared.jar)
  • javax.servlet.jsp.el.ImplicitObjectELResolver (runtime-shared.jar)
  • javax.servlet.jsp.el.ScopedAttributeELResolver (runtime-shared.jar)
  • javax.servlet.jsp.el.VariableResolver (runtime-shared.jar)
  • javax.servlet.jsp.jstl.core.ConditionalTagSupport (runtime-shared.jar)
  • javax.servlet.jsp.jstl.core.Config (runtime-shared.jar)
  • javax.servlet.jsp.jstl.core.LoopTag (runtime-shared.jar)
  • javax.servlet.jsp.jstl.core.LoopTagStatus (runtime-shared.jar)
  • javax.servlet.jsp.jstl.core.LoopTagSupport (runtime-shared.jar)
  • javax.servlet.jsp.jstl.fmt.LocaleSupport (runtime-shared.jar)
  • javax.servlet.jsp.jstl.fmt.LocalizationContext (runtime-shared.jar)
  • javax.servlet.jsp.jstl.sql.Result (runtime-shared.jar)
  • javax.servlet.jsp.jstl.sql.ResultImpl (runtime-shared.jar)
  • javax.servlet.jsp.jstl.sql.ResultSupport (runtime-shared.jar)
  • javax.servlet.jsp.jstl.sql.SQLExecutionTag (runtime-shared.jar)
  • javax.servlet.jsp.jstl.tlv.PermittedTaglibsTLV (runtime-shared.jar)
  • javax.servlet.jsp.jstl.tlv.ScriptFreeTLV (runtime-shared.jar)
  • javax.servlet.jsp.tagext.BodyContent (runtime-shared.jar)
  • javax.servlet.jsp.tagext.BodyTag (runtime-shared.jar)
  • javax.servlet.jsp.tagext.BodyTagSupport (runtime-shared.jar)
  • javax.servlet.jsp.tagext.DynamicAttributes (runtime-shared.jar)
  • javax.servlet.jsp.tagext.FunctionInfo (runtime-shared.jar)
  • javax.servlet.jsp.tagext.IterationTag (runtime-shared.jar)
  • javax.servlet.jsp.tagext.JspFragment (runtime-shared.jar)
  • javax.servlet.jsp.tagext.JspIdConsumer (runtime-shared.jar)
  • javax.servlet.jsp.tagext.JspTag (runtime-shared.jar)
  • javax.servlet.jsp.tagext.PageData (runtime-shared.jar)
  • javax.servlet.jsp.tagext.SimpleTag (runtime-shared.jar)
  • javax.servlet.jsp.tagext.SimpleTagSupport (runtime-shared.jar)
  • javax.servlet.jsp.tagext.Tag (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagAdapter (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagAttributeInfo (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagData (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagExtraInfo (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagFileInfo (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagInfo (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagLibraryInfo (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagLibraryValidator (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagSupport (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TagVariableInfo (runtime-shared.jar)
  • javax.servlet.jsp.tagext.TryCatchFinally (runtime-shared.jar)
  • javax.servlet.jsp.tagext.ValidationMessage (runtime-shared.jar)
  • javax.servlet.jsp.tagext.VariableInfo (runtime-shared.jar)
  • org.apache.geronimo.mail.handlers.HtmlHandler (appengine-api.jar)
  • org.apache.geronimo.mail.handlers.MessageHandler (appengine-api.jar)
  • org.apache.geronimo.mail.handlers.MultipartHandler (appengine-api.jar)
  • org.apache.geronimo.mail.handlers.TextHandler (appengine-api.jar)
  • org.apache.geronimo.mail.handlers.XMLHandler (appengine-api.jar)
  • org.apache.geronimo.mail.util.ASCIIUtil (appengine-api.jar)
  • org.apache.geronimo.mail.util.Base64 (appengine-api.jar)
  • org.apache.geronimo.mail.util.Base64DecoderStream (appengine-api.jar)
  • org.apache.geronimo.mail.util.Base64Encoder (appengine-api.jar)
  • org.apache.geronimo.mail.util.Base64EncoderStream (appengine-api.jar)
  • org.apache.geronimo.mail.util.Encoder (appengine-api.jar)
  • org.apache.geronimo.mail.util.Hex (appengine-api.jar)
  • org.apache.geronimo.mail.util.HexEncoder (appengine-api.jar)
  • org.apache.geronimo.mail.util.QuotedPrintable (appengine-api.jar)
  • org.apache.geronimo.mail.util.QuotedPrintableDecoderStream (appengine-api.jar)
  • org.apache.geronimo.mail.util.QuotedPrintableEncoder (appengine-api.jar)
  • org.apache.geronimo.mail.util.QuotedPrintableEncoderStream (appengine-api.jar)
  • org.apache.geronimo.mail.util.RFC2231Encoder (appengine-api.jar)
  • org.apache.geronimo.mail.util.SessionUtil (appengine-api.jar)
  • org.apache.geronimo.mail.util.StringBufferOutputStream (appengine-api.jar)
  • org.apache.geronimo.mail.util.UUDecoderStream (appengine-api.jar)
  • org.apache.geronimo.mail.util.UUEncode (appengine-api.jar)
  • org.apache.geronimo.mail.util.UUEncoder (appengine-api.jar)
  • org.apache.geronimo.mail.util.UUEncoderStream (appengine-api.jar)
  • org.apache.geronimo.mail.util.XText (appengine-api.jar)
  • org.apache.geronimo.mail.util.XTextEncoder (appengine-api.jar)

Sunday, April 19, 2009

Deployment on GAE/J and the minimal Google App Engine/J application

Continuing the exploration of the Google App Engine/J I decided to take a look at how the deployment process works.

I started by creating a tiny GAE/J application consisting only of one JSP file. In addition to the JSP file the GAE/J environment requires two additional files: the standard Java deployment descriptor web.xml and the GAE/J specific appengine-web.xml. The files required for our application are hence: index.jsp, WEB-INF/appengine-web.xml and WEB-INF/web.xml.

The smallest possible valid appengine-web.xml contains only the required elements appengine-web-app/application and appengine-web-app/version. If you want to use HTTP-sessions you're required to add an element appengine-web-app/sessions-enabled set to true. The appconfig-web.xml is documented here.

This is the smallest possible valid appengine-web.xml:
<appengine-web-app>
<application>your-app-id</application>
<version>1</version>
</appengine-web-app>


The smallest possible valid web.xml file that is accepted by GAE/J contains only the root node web-app. For some reason <web-app /> is not accepted whereas the equivalent <web-app></web-app> is. The GAE/J documentation covering web.xml is found here.

The smallest possible web.xml that is accepted by GAE/J is:
<web-app></web-app>


Now that we have the JSP-file and our minimal web.xml and appengine-web.xml we're ready to deploy.

To make the working of the deployment process as clear as possible I choose to deploy by issuing the deploy command directly rather than relying on the Ant scripts provided by the GAE/J SDK. In order to deploy we issue the command:
java -cp /some/path/to/appengine-java-sdk-1.2.0/lib/appengine-tools-api.jar com.google.appengine.tools.admin.AppCfg update .


The application should now be deployed.

If we examine our GAE/J "home dir" (see this previous post about "GAE/J home dirs") after deployment we'll find the following files:

  • /base/data/home/apps/[app-id]/[version-id]/index.jsp

  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/classes/org/apache/jsp/index_jsp.java

  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/classes/org/apache/jsp/index_jsp.class

  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/web.xml
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/appengine-web.xml
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/jasper-compiler-5.0.28.jar
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/jakarta-jstl-1.1.2.jar
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/commons-el-1.0.jar
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/jakarta-standard-1.1.2.jar
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/jasper-runtime-5.0.28.jar
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/ant-launcher-1.6.5.jar
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/commons-logging-1.1.1.jar
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/ant-1.6.5.jar


Note that a couple of JAR:s have been to our application. Apache Jasper runtime (+ dependencies) is used by the servlet (org.apache.jsp.index_jsp) that is generated from our JSP-file (index.jsp). But what about the other JAR:s (say ant, ant-launcher and jasper-compiler) - should those really be required in the live environment? Is the uploading of those files a bug?

The web.xml that is uploaded to the live environment is altered during the deployment process. Servlet-mappings are added to the original web.xml to match JSP filenames against the servlets that are generated from said JSP files -- in this case the URL-pattern /index.jsp is mapped to the servlet org.apache.jsp.index_jsp.

Note also that both the original JSP file (index.jsp) and the corresponding generated servlet Java file (index_jsp.java) are uploaded although those files should not be needed in the live environment. This can be seen either as a feature (in the sense you have a backup of your source code in the live environment) or as a bug (in the sense that only what's required in the live environment should be in the live environment).

This concludes the exploration of the GAE/J deployment process this time.

I'd like to hear from you what you think about the three (very minor) issues discussed -- <web-app /> not accepted as a valid web.xml + JSP:s and Java-files uploaded to live environment + additional JAR:s uploaded of which some redundant (?). Are any of these bugs that should be reported? What do you think? Please leave a comment.

I should probably add that I'm very impressed with how smooth the deployment process is handled when using GAE/J. In particular I have not yet seen a case where deploying a new version means downtime visible for the end-user. That's very nice!

Thursday, April 16, 2009

Know your CLASSPATH in Google App Engine/J

Continuing the exploration of the Google App Engine Java environment I decided to explore what's in the CLASSPATH of the live GAE/J environment (this differs from the CLASSPATH of the development environment).

I did the following: 

First I created a list of classes that might be available in the GAE/J live environment. The list of potential classes was created by compiling a list of all classes available in the development environment. This gave a list of 4391 class names.

From the original list of 4391 classes in the development environment I was able to successfully load 3689 classes of those using Class.forName(...) in the live environment.

After loading each class I used reflection to recursively find references to other classes using the following method:

Assume a class called A:

public class A {
  public void doSomething(B bObject) { ... }
  public C getC(int i) { ... }
}

By enumerating all methods of class A using the reflection API an looking at the return types and parameters passed to the methods we can infer the existence of classes B and C. We can then load classes B and C and find additional classes using the same method.

Using this method recursively I was able to load an additional 220 classes giving a total of 3909 classes (3689 + 220).

For each of the 3909 loadable classes I used queried for the JAR that each file was loaded from using Class.forName(...).getProtectionDomain().getCodeSource().getLocation().toString()

Of the 3909 loadable classes 2023 originated from JARs uploaded to the system (/WEB-INF/lib/*.jar). This gives a total of 1886 (3909 - 2023) loadable "GAE/J standard classes" -- that is classes that are available given an empty /WEB-INF/lib/. Please note that the total number of available classes most probably is larger since the methodology described does not catch all available classes.

Based on this analysis described above the GAE/J JVM appears to load classes from the following locations:
  • /base/data/home/apps/[app-id]/[version-id]/WEB-INF/lib/*.jar
  • JDK Base Classes (1332 classes)
  • /base/java_runtime/user-privileged.jar (1 class)
  • /base/java_runtime/runtime-shared.jar (107 classes)
  • /base/java_runtime/restricted-class-stubs.jar (31 classes)
  • /base/java_runtime/appengine-api.jar (415 classes)
What follows is a list of packages that are loaded from each location/JAR:

JDK Base Classes

  • java.*
  • javax.*
  • org.w3c.dom.*
  • org.xml.sax.*

Classes in /base/java_runtime/user-privileged.jar

  • com.google.apphosting.runtime.security.shared.CustomURLClassLoader

Packages in /base/java_runtime/runtime-shared.jar

  • com.google.apphosting.api.*
  • javax.servlet.*
  • javax.servlet.http.*
  • javax.servlet.jsp.*
  • javax.servlet.jsp.el.*
  • javax.servlet.jsp.jstl.core.*
  • javax.servlet.jsp.jstl.fmt.*
  • javax.servlet.jsp.jstl.sql.*
  • javax.servlet.jsp.jstl.tlv.*
  • javax.servlet.jsp.tagext.*

Packages in /base/java_runtime/restricted-class-stubs.jar

  • com.google.apphosting.runtime.security.shared.stub.java.awt.*
  • com.google.apphosting.runtime.security.shared.stub.java.awt.datatransfer.*
  • com.google.apphosting.runtime.security.shared.stub.java.awt.event.*
  • com.google.apphosting.runtime.security.shared.stub.java.beans.*
  • com.google.apphosting.runtime.security.shared.stub.java.io.*
  • com.google.apphosting.runtime.security.shared.stub.java.lang.instrument.*
  • com.google.apphosting.runtime.security.shared.stub.java.net.*
  • com.google.apphosting.runtime.security.shared.stub.java.nio.*
  • com.google.apphosting.runtime.security.shared.stub.java.nio.channels.*
  • com.google.apphosting.runtime.security.shared.stub.java.nio.channels.spi.*
  • com.google.apphosting.runtime.security.shared.stub.java.text.*
  • com.google.apphosting.runtime.security.shared.stub.javax.naming.*
  • com.google.apphosting.runtime.security.shared.stub.javax.swing.text.*
  • com.google.apphosting.runtime.security.shared.stub.javax.tools.*

Packages in /base/java_runtime/appengine-api.jar

  • com.google.appengine.api.*
  • com.google.appengine.api.datastore.*
  • com.google.appengine.api.images.*
  • com.google.appengine.api.mail.*
  • com.google.appengine.api.mail.stdimpl.*
  • com.google.appengine.api.memcache.*
  • com.google.appengine.api.memcache.stdimpl.*
  • com.google.appengine.api.urlfetch.*
  • com.google.appengine.api.users.*
  • com.google.appengine.repackaged.com.google.common.base.*
  • com.google.appengine.repackaged.com.google.common.base.genfiles.*
  • com.google.appengine.repackaged.com.google.common.base.internal.*
  • com.google.appengine.repackaged.com.google.common.collect.*
  • com.google.appengine.repackaged.com.google.common.io.*
  • com.google.appengine.repackaged.com.google.common.primitives.*
  • com.google.appengine.repackaged.com.google.common.util.*
  • com.google.appengine.repackaged.com.google.io.base.*
  • com.google.appengine.repackaged.com.google.io.protocol.*
  • com.google.appengine.repackaged.com.google.io.protocol.proto.*
  • com.google.apphosting.api.*
  • com.google.apphosting.utils.servlet.*
  • com.google.storage.onestore.v3.*
  • javax.cache.*
  • javax.mail.*
  • javax.mail.event.*
  • javax.mail.internet.*
  • javax.mail.search.*
  • javax.mail.util.*
  • org.apache.geronimo.mail.handlers.*
  • org.apache.geronimo.mail.util.*
That's all of the CLASSPATH analysis for now!

Hope you didn't miss today's great GAE/J news: four hours ago the Grails project lead Graeme Rocher got Grails up and running on the AppEngine/J. That is excellent news for all us Google App Engine- and Grails-fans! I really do believe the scalability of GAE/J in combination with the high productivity of the Grails framework will be a killer combination! Exciting times! :-)

More JVM-dancing in the Google App Engine/J environment

Following up to my previous blog posts (#1, #2) about "JVM dancing patterns"/JVM-switching I've done some additional experimenting.

My latest round of testing included 118,679 successfully served requests during a period of roughly 48 hours.

The requests were served by a total of 54 distinct JVM:s (as identified by Runtime.getRuntime().hashCode()).

The longest period during which only one single JVM served all requests was 2.5 hours (8,995 seconds - a new record).

I tried altering the number of requests sent per second to see the impact on the number of JVM:s involved in serving the requests, and also the frequency of JVM-switching. As expected the number of JVM:s involved in serving requests rose quickly as the number of requests per second increased. Furthermore, the frequency of JVM-switching increased significantly as the number of JVM:s involved increased. When the number of requests per second was really high the requests seemed to be more or less randomly distributed among the participating JVM:s.

I'll keep tinkering and I'll report any results here! :-)

Wednesday, April 15, 2009

HTTP-sessions and the Google App Engine/J - some implemention details

The Google App Engine/J environment gives you access to the standard Java mechanism for handling HTTP-sessions - namely the familiar HttpSession (javax.servlet.http.HttpSession).

The first thing to note about the GAE/J HttpSession is that by default you're only allowed read-access to the session.  If you want write to the session -- that is calling setAttribute(..., ...) -- you're required to setup session handling by adding <sessions-enabled>true</sessions-enabled> to your WEB-INF/appengine-web.xml.

Once you've enabled sessions you'll have HttpSession:s that behave like you're used to. Well, they behave like you're used to -- you can getAttribute(...) and you can setAttribute(..., ...) -- but behind behind the scenes there are a few implementation details to be aware of.

As described in a previous post two consecutive HTTP-requests sent by the same user to the same GAE/J-app may very well be served by two different JVM:s. An application developer would under normal circumstances assume that what he writes to the session in request N is available for retrieval from the session in request N+1 (assuming that the time between the two request is reasonably small). But in the case of GAE/J those two requests might be served by different JVM:s. Luckily, the GAE/J infrastructure makes sure that the session data is shared across all JVM:s serving your requests.

The sharing of HttpSession data is achieved by temporarily storing  your session data in the App Engine Datastore and memcache

After your servlet has processed the request and returned its reponse a serialized version of the HttpSession is stored in the App Engine Datastore and memcache. If subsequent requests are routed to other JVM:s this data will be read to re-create the HttpSession.

A Datastore entity of kind "_ah_SESSION" is created for each new HttpSession. The entity key is "_ahs" + session.getId(). Each _ah_SESSION entity has two properties "_values" and "_expires".

The "_values" property is simply the serialized byte[] representation of a HashMap that includes the the session data.

The "_expires" property is the absolute time-to-live of the HttpSession and is expressed in milliseconds since January 1, 1970 (Unix time expressed in milli-seconds). The default time-to-live is 24 hours, but can be configured by altering . The _expires field is updated each time the session is active (_expires = System.currentTimeMillis() + 24 * 60 * 60 * 1,000 = System.currentTimeMillis() + 86,400,000).

Automatic removal of expired ah_SESSION entities (_ah_SESSION:s where _expires < System.currentTimeMillis()) did not make it to the initial GAE/J release, but it's probably safe to assume that it will be fixed in an upcoming release.

I'm planning a follow-up blog post about the corresponding loading and retrieval of HttpSession data from the memcache.

This is what I've found out about GAE/J session handling so far - please leave a comment if you have any additions and/or corrections!

Monday, April 13, 2009

Towards a general theory of JVM dancing (GAE/J)

Following my previous blog post about JVM dancing in the Google App Engine/J environment I decided to do some small testing to see how often the described "JVM switching" occurs.

The test was conducted by sending one HTTP-request per second to the same GAE/J application during a period of roughly four hours (3.7 hours). All requests were sent from the same IP-address and supplied the same JSESSIONID cookie (emulating all requests being sent from the same user within the same session). The GAE/J app that received the requests did not have any traffic other than the traffic generated by the test.

A total of 13285 requests were successfully served during this period. The requests were served by three distinct JVM:s (as identified by Runtime.getRuntime().hashCode()):
  • JVM A served 9273 requests (69.8 % of all requests)
  • JVM B served 4004 requests (30.1 %)
  • JVM C served 8 requests (0.1 %)
During these four hours of testing the serving of the requests switched between the JVM:s as follows:
  • 160 successive requests served by JVM A
  • 2764 successive requests served by JVM B
  • 1 request served by JVM C
  • 1240 successive requests served by JVM B
  • 21 successive requests served by JVM A
  • 2 successive requests served by JVM C
  • 208 successive requests served by JVM A
  • 2 successive requests served by JVM C
  • 368 successive requests served by JVM A
  • 1 request served by JVM C
  • 5499 successive requests served by JVM A
  • 2 successive requests served by JVM C
  • 3017 successive requests served by JVM A
In total 12 JVM switches took place. Which in this test translates into one JVM switch per 1107 requests (switch probability of 0.09 %).

The shortest time before a JVM switch was one second - but please note that the granularity of the test was one second since one request was sent per second.

The longest time before a JVM switch was 1.5 hours (5499 seconds).

It is probably very safe to assume that the "JVM dancing" pattern is heavily dependent on a variety of factors, including traffic patterns, general load of the GAE/J system, etc. So please take the above testing as one data point from one test - NOT an attempt to provide the general theory of JVM dancing :-)

The JVM Dance: Follow your app from JVM to JVM on GAE/J

When you upload an app to the Google App Engine/J environment HTTP-requests sent to your app will be served by multiple JVM:s (or more specifically, requests will be served by one or more JVM:s).

This means that two consecutive HTTP-requests sent by the same user to the same GAE/J-app may very well be served by two different JVM:s.

The fact that multiple JVM:s are serving your requests might not be apparent since the GAE/J infrastructure makes sure that all objects you store in the session (HttpSession) are available across all JVM:s serving your app (this is done by temporarily storing your session objects in the datastore). On the other hand, you might notice that code within static blocks will be executed once for each JVM loading your code.

To identify which JVM that is serving a request, use the following code:
int currentJvm = Runtime.getRuntime().hashCode();

I've written a small JSP-file that will show you which JVM that is serving the current request, and alert you when two consecutive HTTP-requests are being served by different JVM:s -- let's term this jumping between JVM:s the "GAE/J JVM dance" :-)
jvmdance.jsp: http://pastebin.com/f127200f1

Sunday, April 12, 2009

Determining if your code is executing in the GAE/J environment/sandbox

When moving your apps to the GAE/J platform you might be required to do some tweaking to make your application comply with the restricted sandbox environment of the Google App Engine (no write access to disk, read access limited to "user.dir", no thread creation, etc.).

Here is a quick way to determine if your code is executing in the GAE/J environment:
boolean onGoogleAppEngine = getServletContext().getServerInfo().startsWith("Google App Engine");
getServerInfo() currently returns "Google App Engine/1.2.0" when running in the live GAE/J environment. When running in local development mode it returns "Google App Engine Development/1.2.0".

Please comment if you know of any better/cleaner way to determine if your code is executing in the GAE/J sandbox.

User home dirs ("user.dir") on the Google App Engine/J

A Java application running on the Google App Engine is only allowed to read files stored inside the user's home dir. The path to this directory is stored in the read-only system property "user.dir" (System.getProperties().get("user.dir")).

Trying to read files outside of your home dir will raise the following exception:
java.security.AccessControlException: access denied (java.io.FilePermission ... read)
The path of the "user.dir" on Google App Engine/J follows the following structure:
/base/data/home/apps/[app-id]/[version-id]
The version-id appears to be a version identifier followed by a dot (".") and then an 18 digit random number.

A new home directory is created each time you deploy your app using "appcfg.sh update".

Combining the fact that you're allowed to read your own home dir with the fact that a new deployment gives rise to a new home dir gives you the following method to get the exact date and time of your last deployment:
Date deployTime = new Date(new File((String) System.getProperties().get("user.dir")).lastModified());

Saturday, April 11, 2009

Header filtering/modification by Google Frontend

Google Frontend (GFE) appears to remove the header fields "Accept-Encoding", "Keep-Alive" and "Connection". Furthermore, it seems to add ",gzip(gfe)" to the User-Agent string.

Compare what's sent by the browser ...

Host: your-app-id.appspot.com
User-Agent: OriginalUserAgentString
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: sv-se,sv;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: JSESSIONID=session-id-string

... to what's received by the GAE/J application after filtering by Google Frontend (GFE) ...

Host: your-app-id.appspot.com
User-Agent:
OriginalUserAgentString,gzip(gfe)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: sv-se,sv;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Cookie: JSESSIONID=session-id-string

As I've understood it GFE takes care of compressing the response, so removing "Accept-Encoding" makes sense. I guess ",gzip(gfe)" is added to the User-Agent to indicate to the back-end that the response will be compressed before being sent to the client.

Interactive Groovy console running on GAE/J

If you want to quickly test Groovy code on the GAE/J, then this small Groovy console way is a neat way to do it:
http://groovyconsole.appspot.com/
Try executing the following code to get some information about the GAE/J environment:

def p = System.getProperties()
p.keySet().each { println "${it}=${p[it]}" }
def r = Runtime.getRuntime()
println "processors=${r.availableProcessors()}"
println "freeMemory=${r.freeMemory()}"
println "maxMemory=${r.maxMemory()}"
println "totalMemory=${r.totalMemory()}"

GAE/J running Jetty

Judging from stack traces GAE/J appears to be running a modified Jetty servlet container.

This is a full stack trace from the GAE/J environment:

java.lang.NullPointerException
at ...(...:...)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:237)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)
at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:63)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:125)
at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:235)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:4547)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:4545)
at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24)
at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:359)
at com.google.net.rpc.impl.Server$2.run(Server.java:792)
at com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java:56)
at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java:489)
at com.google.net.rpc.impl.Server.startRpc(Server.java:748)
at com.google.net.rpc.impl.Server.processRequest(Server.java:340)
at com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.java:422)
at com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319)
at com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java:290)
at com.google.net.async.Connection.handleReadEvent(Connection.java:419)
at com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.java:733)
at com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java:207)
at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:101)
at com.google.net.rpc.RpcService.runUntilServerShutdown(RpcService.java:249)
at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run(JavaRuntime.java:373)
at java.lang.Thread.run(Unknown Source)

HTTP User-Agent used by GAE/Java

Fetching a file over HTTP using GAE/Java is easy:
URL url = new URL("http://www.example.com/file.txt");
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
The request sent by the Google App Engine servers will have the following properties:
HTTP_ACCEPT_ENCODING = gzip
HTTP_REFERER = http://your-app-id.appspot.com/
HTTP_USER_AGENT = AppEngine-Google; (+http://code.google.com/appengine)
REMOTE_ADDR = 64.233.172.6 (probably varies)
SERVER_PROTOCOL = HTTP/1.1
Please note that the referer field contains the app-id of the application that generated request, which allows for identification of the source of the request.

The Java environment used for the Google App Engine

This is the environment used by the Google App Engine for Java:

java.specification.version=1.6
java.vendor=Sun Microsystems Inc.
line.separator=\n
java.class.version=50.0
java.util.logging.config.file=WEB-INF/logging.properties
java.specification.name=Java Platform API Specification
java.vendor.url=http://java.sun.com/
java.vm.version=1.6.0_13
os.name=Linux
java.version=1.6.0_13
java.vm.specification.version=1.0
user.dir=/base/data/home/apps/[your-app-id]/[random-id]
java.specification.vendor=Sun Microsystems Inc.
java.vm.specification.name=Java Virtual Machine Specification
java.vm.vendor=Sun Microsystems Inc.
file.separator=/
path.separator=:
java.vm.specification.vendor=Sun Microsystems Inc.
java.vm.name=Java HotSpot(TM) Client VM
file.encoding=ANSI_X3.4-1968

Seems like GAE/Java is running Sun's Java 1.6.0.13 under Linux.
All according to System.getProperties().

So what about number of available processors and memory?

Runtime.getRuntime().availableProcessors()=1337 ("elite" joke)
# "the maximum number of processors available to the virtual
# machine; never smaller than one"

Runtime.getRuntime().freeMemory()=6293011 (6 MB)
# "an approximation to the total amount of memory currently
# available for future allocated objects, measured in bytes"

Runtime.getRuntime().maxMemory()=104857600 (105 MB)
# "the maximum amount of memory that the virtual machine will
# attempt to use, measured in bytes"

Runtime.getRuntime().totalMemory()=104857600 (105 MB)
# "the total amount of memory currently available for current
# and future objects, measured in bytes"

Friday, April 10, 2009

HOWTO: Getting started with GAE/Java

Getting started with Google App Engine for Java is easy - downloading the SDK and deploying your first demo app can all be done in a few commands.

Assuming you've already created an app-id using the Google App Engine web interface, getting started is as easy as ...

# Download and unzip the SDK
curl -O http://googleappengine.googlecode.com/files/appengine-java-sdk-1.2.0.zip
unzip appengine-java-sdk-1.2.0.zip
cd appengine-java-sdk-1.2.0
cd demos/guestbook/
# Start the local test server
ant runserver
# Now browse to http://localhost:8080/ and make sure the app works.
# Try editing war/guestbook.jsp and make a few changes.
# Add your app-id to appengine-web.xml: <application>your-app-id</application>
emacs war/WEB-INF/appengine-web.xml
../../bin/appcfg.sh update war/


Now your applications should be deployed and ready for requests at http://your-app-id.appspot.com/.