What's New in GraniteDS 2.0.0 RC1

Main New Features:

Granite Data Services 2.0.0 Release Candidate 1 has just been released and comes with four main new features:

1. Google App Engine support: GDS applications may now be deployed on Google's infrastructure. See details in William's related posts on this blog.
2. Servlet 3.0 preview: Gravity now implements data push with Servlet 3.0 API and may be tested with Glassfish v3 preview (lastest promoted builds only). See below for details.
3. Configuration MBeans: you may now use your JMX console in order to manage several GraniteDS options and watch for runtime configuration details. See below for details.
4. JDO annotations support: Gas3 now generates suitable ActionScript3 code for JDO annotated entities and the DataNucleusExternalizer takes care of JDO's specific annotations.

Granite-config.xml Changes (MUST READ):

The granite-config.dtd has changed in order to make the config more readable: as a general rule, all configuration attributes use now a "dashed" convention: "annotedwith" is now spelled "annotated-with", "instanceof" "instance-of", etc. There is also a new "gravity" section whose meaning content is detailed in "Gravity Changes" below.

The new DTD is online here and will help you with a DTD aware XML editor.

Gravity Changes:

Gravity code was getting messy, with several duplicated source sections, and hard to maintain. With the addition of Servlet 3.0 and GAE support, things were going even more harder, so the code has been cleaned up and widely refactored. However, the new code has been fully tested and shouldn't bring any problem.

A new feature in this rewriting is the use of a thread pool for publishing messages. Previously, a published message was delivered to subscribed channels in a synchronous way, using the publish request container thread (delivering means only duplicating the message for all channels subscribed to the message's topic and adding each copy to each channel's queue). This operation was potentially time consuming with a large number of channels connected to the same topic and could block the caller container thread for a significant amount of time. A thread pool is now used for that purpose in all servlet implementations (Tomcat, JBossWeb, Jetty and Servlet 3.0) except for GAE.

Instead of servlet parameters, Gravity configuration now goes in a dedicated section of granite-config.xml. Here is an example of all options with there default values:

<gravity
factory="org.granite.gravity.DefaultGravityFactory"
channel-idle-timeout-millis="1800000"
long-polling-timeout-millis="20000"
reconnect-interval-millis="30000"
reconnect-max-attempts="60">

<thread-pool
core-pool-size="5"
maximum-pool-size="20"
keep-alive-time-millis="10000"
queue-capacity="2147483647" />

</gravity>

This section is purely optionnal and you may omit it if you accept default values. Full documentation will be provided later but here is some comments on the most important options:

• channel-idle-timeout-millis: the elapsed time after which an idle channel (pure producer or dead client) may be silently unsubcribed and removed by Gravity. Default is 30 minutes.
  
• long-polling-timeout-millis: the elapsed time after which an idle connect request is closed, asking the client to reconnect. Default is 20 seconds. Note that setting this value isn't supported in Tomcat/APR configurations.
• thread-pool attributes: all options are standard parameters for the Gravity ThreadPoolExecutor instance.
  

You may also use the new Gravity MBean in order to change most of these options at runtime (see below).

Servlet v3 support is new in this RC1 and should be considered as beta software. The only up-to-date implementation of the new API at this time is in Glassfish v3 trunk and you may download it here (Servlet 3.0 support has been tested with glassfish-v3-preview-b47-windows.exe and may or may not work with other builds).

Configuring the GDS chat sample for Glassfish v3 is a two steps process:

1. Edit the graniteds/examples/env.properties and set SERVER_HOME to your glassfishv3/glassfish directory and SERVER_HOMER_DEPLOY to ${SERVER_HOME}/domains/domain1/autodeploy.
2. Copy the content of the graniteds/examples/graniteds_chat/resources/WEB-INF/servlet3-web.xml into the web.xml file. Then, use the "deploy" target of chat build file and start Glassfish (see Glassfish documentation).

You should then be able to connect to http://localhost:8080/graniteds-chat/ and test the application.

Configuration MBeans:

This RC1 comes with two new MBeans for GDS runtime management:

• GraniteConfig MBean: this mbean will show an almost complete configuration state of GDS at runtime, with several things that were not accessible previously (except with a debugger): externalizers used so far, associated with serialized classes, enabled or disabled Tide components, etc. The only available operation provided by this MBean is a reload button: it will re-read your deployed granite-config.xml file and take care of your modifications. Note that everything may not be reconfigured at runtime but this may be at least a great tool for finding issues, for example, with externalizers configuration.
  
• GravityConfig MBean: this mbean will display the current state of Gravity, letting you modify at runtime timeouts and pool sizing. You'll be also able to restart Gravity: note however that this is only an emergency feature because all undelivered queued messages will be lost!
  

To access these MBeans with JBoss, browse to http://localhost:8080/jmx-console and scroll to the "org.granite" section.

GraniteDS 2.0 now supports server push on Google App Engine

We are close to releasing GraniteDS 2.0 RC1, hopefully the last release before GA. It includes a big refactoring of the Gravity core to simplify the implementation of new providers, and in particular there is a new Gravity provider for Google App Engine based on Memcache.

An example of this Google App Engine push implementation is available at http://gdsgaepush.appspot.com, once again with admin/admin or user/user. If you open two browsers on the application, the changes made on from side should be also updated on the other. The eclipse project can be downloaded here.

The GAE environment is very interesting because it is fully clustered: each request can be unpredictably handled by a different server instance. The only reliable way to communicate between different requests/users is by using the datastore or memcache (or a combination of both). In a first messaging implementation, we have chosen the memcache API that allows fast communication between processes and is easy to use. The main drawback is that it is not a reliable store and that client subscriptions or messages could be lost under high memory usage, but for non critical use it might be acceptable.

The implementation is relatively simple: Gravity subscriptions and channels are stored in the cache with a configurable expiration delay. Each channel maintains its current number of pending messages in the cache. Publisher threads just increment this number and store new messages in the cache, and subscriber threads poll the channel counter at regular intervals and get existing messages from the cache when there are available. The poller thread by default polls the channel every 500ms and has a total duration of 20s (under the 30s limit of GAE).

It's important to note that due to the use of memcache, only serializable objects can be sent in Gravity messages (but it's also the case with JMS). Besides that simple limitation, all features of Gravity (client <-> client, server -> client, selectors, ...) can be used with GAE (see documentation here).

The configuration of the GAE provider can be done as follows :

Define the Gravity GAE servlet in WEB-INF/web.xml :

<servlet>
    <servlet-name>GravityServlet</servlet-name>
    <servlet-class>org.granite.gravity.gae.GravityGAEServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
    <servlet-name>GravityServlet</servlet-name>
    <url-pattern>/gravityamf/*</url-pattern>
</servlet-mapping>

Setup the GAE Gravity factory in WEB-INF/granite/granite-config.xml :

<gravity 
  factory="org.granite.gravity.gae.GAEGravityFactory 
  channel-idle-timeout-millis="1800000">
  <configuration>
    <gae polling-interval="500">
  </configuration>
</gravity>

Finally define the GAE service adapter in WEB-INF/flex/services-config.xml :

<service id="gravity-service"
    class="flex.messaging.services.MessagingService"
    messageTypes="flex.messaging.messages.AsyncMessage">
    <adapters>
        <adapter-definition id="gae" class="org.granite.gravity.gae.GAEServiceAdapter"/>
    </adapters>

    <destination id="exampleDestination">
        <channels>
             <channel ref="my-gravityamf"/>
        </channels>
        <adapter ref="gae"/>
    </destination>
</services>

This new server push provider complements the current existing support for AMF remoting and JDO persistence and makes GraniteDS 2.0 a complete platform for deploying Flex applications on Google App Engine.

As always, feedback and comments are welcome.