GraniteDS 2.0 now supports server push on Google App Engine

We are close to releasing GraniteDS 2.0 RC1, hopefully the last release before GA. It includes a big refactoring of the Gravity core to simplify the implementation of new providers, and in particular there is a new Gravity provider for Google App Engine based on Memcache.

An example of this Google App Engine push implementation is available at http://gdsgaepush.appspot.com, once again with admin/admin or user/user. If you open two browsers on the application, the changes made on from side should be also updated on the other. The eclipse project can be downloaded here.

The GAE environment is very interesting because it is fully clustered: each request can be unpredictably handled by a different server instance. The only reliable way to communicate between different requests/users is by using the datastore or memcache (or a combination of both). In a first messaging implementation, we have chosen the memcache API that allows fast communication between processes and is easy to use. The main drawback is that it is not a reliable store and that client subscriptions or messages could be lost under high memory usage, but for non critical use it might be acceptable.

The implementation is relatively simple: Gravity subscriptions and channels are stored in the cache with a configurable expiration delay. Each channel maintains its current number of pending messages in the cache. Publisher threads just increment this number and store new messages in the cache, and subscriber threads poll the channel counter at regular intervals and get existing messages from the cache when there are available. The poller thread by default polls the channel every 500ms and has a total duration of 20s (under the 30s limit of GAE).

It's important to note that due to the use of memcache, only serializable objects can be sent in Gravity messages (but it's also the case with JMS). Besides that simple limitation, all features of Gravity (client <-> client, server -> client, selectors, ...) can be used with GAE (see documentation here).

The configuration of the GAE provider can be done as follows :

Define the Gravity GAE servlet in WEB-INF/web.xml :

<servlet>
    <servlet-name>GravityServlet</servlet-name>
    <servlet-class>org.granite.gravity.gae.GravityGAEServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
    <servlet-name>GravityServlet</servlet-name>
    <url-pattern>/gravityamf/*</url-pattern>
</servlet-mapping>

Setup the GAE Gravity factory in WEB-INF/granite/granite-config.xml :

<gravity 
  factory="org.granite.gravity.gae.GAEGravityFactory 
  channel-idle-timeout-millis="1800000">
  <configuration>
    <gae polling-interval="500">
  </configuration>
</gravity>

Finally define the GAE service adapter in WEB-INF/flex/services-config.xml :

<service id="gravity-service"
    class="flex.messaging.services.MessagingService"
    messageTypes="flex.messaging.messages.AsyncMessage">
    <adapters>
        <adapter-definition id="gae" class="org.granite.gravity.gae.GAEServiceAdapter"/>
    </adapters>

    <destination id="exampleDestination">
        <channels>
             <channel ref="my-gravityamf"/>
        </channels>
        <adapter ref="gae"/>
    </destination>
</services>

This new server push provider complements the current existing support for AMF remoting and JDO persistence and makes GraniteDS 2.0 a complete platform for deploying Flex applications on Google App Engine.

As always, feedback and comments are welcome.

GraniteDS 2.0 on Google App Engine

Google App Engine is the buzzword of the week for Java developers, so of course we have tried to make GraniteDS 2.0 work on it.

After fixing a few issues we managed to have an application working correctly, that you can see at http://gdsgae.appspot.com (log in with admin/admin or user/user). The application is relatively simple, but it demonstrates a complete integration between GraniteDS, Spring MVC and JDO/DataNucleus. The Eclipse project can be downloaded here.

Having a working environment in the local development server has been quite easy, but two bugs in the GAE production environment prevented GDS to work correctly. First there is a bug in XML/XPath support (GAE 1255) that we use for configuration parsing, and we had to add support for Xalan as an alternative XPath provider. Second there is a bug in request.getInputStream (GAE 1339) that throws EOFExceptions and that we could work around by buffering the stream in the AMF servlet.

So the first good news is that the latest GraniteDS 2.0 snapshot perfectly works in GAE if you add xalan.jar and serializer.jar (from the Apache Xalan distribution) in your WEB-INF/lib folder.

Then by pure coincidence we had been working lately on JDO and DataNucleus support in GDS (thanks a lot to Stephen More for his work on it) so it was natural to try it with the GAE datastore. The second good news is that GraniteDS fully supports JDO detached objects from GAE, and that all Tide features can work with JDO annotated entities (including lazy loading).
There seem to be a lot of bugs in the GAE persistence support though when not using Key objects as entity ids. So we have added the GAEKeyConverter that can serialize GAE keys to strings in ActionScript, and support can be configured in granite-config.xml with :

<converters>
  <converter type="org.granite.messaging.amf.io.convert.impl.GAEKeyConverter"/>
</converters>

JDO annotated entities has also been implemented in the GDS Eclipse builder to generate AS3 classes from a JDO model and it works with the Google Eclipse plugin. If you also use Flex Builder, it's recommended to setup the order or the builders in the project properties as follows: Java, GraniteDS, Enhancer, Flex, GAE Project validator, GAE Webapp validator.

So we're happy to be able to add Google App Engine to the list of the Java platforms supported by GraniteDS, after Tomcat, Jetty, JBoss 4/5, GlassFish V2/V3 and WebLogic.