Development environment for a Google App Engine application

There is integration with Google App Engine for several IDEs and you can also interact with the deployment tools from plain command line if you have that disposition.

I'm an Eclipse guy, and here's how I've done to get a good setup to be able to develop in an efficient way using Eclipsen, Maven, CVS, Hudson and GAE.

Fisrt of all, install the Google App Engine SDK from http://code.google.com/intl/sv-SE/appengine/docs/java/tools/eclipse.html and follow the install instructions.

You should get a new toolbar in Eclipse with icons for creating GAE web projects, profiling them and for deploying to the cloud. If you create a new GAE application you can have it deployed in Googles data centers in a few minutes. All you need is a standard Google Account.

A few tips:

• You access your application at http://localhost:8888/ by default.
• In your dev environment a light weight data store will be kept in WEB-INF\appengine-generated\local_db.bin. This store is permanent over server restarts and must be deleted if you wish a clean state. Indicies are automatically generated in WEB-INF\appengine-generated\datastore-indexes-auto.xml. You access the development console at http://localhost:8888/_ah/admin where you can inspect and make queries on the data store and work with task queues and more.
• In addition to the standard JEE files such as web.xml there are a few specific config files for GAE projects. Inspect WEB-INF\appengine-web.xml where you setup logging and turn on sessions. By default you will not create web sessions when accessing you application

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
    <application>rankington</application>
    <version>1</version>
    
    <sessions-enabled>true</sessions-enabled>
    
    <!-- Configure java.util.logging -->
    <system-properties>
        <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
    </system-properties>
    
</appengine-web-app> 

If you wish to complicate your environment with some continous integration for compilation, bug, test and coverage reporting I'll outline how you could integrate your GAE application with Hudson.

I've fiddled around with the default directory structure to make a more Maven compliant tree for dependency management. That is no problem since the App Engine plugin uses the Eclipse .classpath to resolve the structure.  So create src/main/java src/main/resources /src/test/java src/test/resources and src/main/webapp. Create a pom.xml in your project root and setup your Maven dependencies. To make this easy, install the Eclipse Maven plugin which makes pom-editing a no-brainer. You will need a few special dependencies for App Engine. Be careful about which version of the plugin you use to match the Maven dependency.

<dependency>
   <groupId>com.google.appengine</groupId>
   <artifactId>appengine-api-1.0-sdk</artifactId>
   <version>1.4.0</version>
   <type>jar</type>
   <scope>compile</scope>
</dependency>

My Maven dependencies are:

It took me some time to figure out how to add unit testing, findbugs reporting and code coverage to my Maven build. So hopefully you can reuse something like this for your pom file. I've removed most of the code dependencies specific for my project for clarity

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>se.noren.rankington</groupId>
  <artifactId>Rankington</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>Maven Quick Start Archetype</name>
  <url>http://maven.apache.org</url>
  <ciManagement>
  </ciManagement>
  <build>
      <plugins>
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-compiler-plugin</artifactId>
              <version>2.3.2</version>
              <configuration>
                  <source>1.5</source>
                  <target>1.5</target>
            </configuration>
          </plugin>
          <plugin>
              <groupId>org.codehaus.mojo</groupId>
              <artifactId>surefire-report-maven-plugin</artifactId>
              <version>2.0-beta-1</version>
          </plugin>
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-checkstyle-plugin</artifactId>
              <version>2.6</version>
          </plugin>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>cobertura-maven-plugin</artifactId>
            <version>2.2</version>
            <configuration>
                <formats>
                    <format>xml</format>
                </formats>
            </configuration>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>cobertura</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>          
      </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.4</version>
      <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-beans</artifactId>
        <version>3.0.5.RELEASE</version>
        <type>jar</type>
        <scope>compile</scope>
    </dependency>

  </dependencies>
  <reporting>
      <plugins>
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-checkstyle-plugin</artifactId>
              <version>2.6</version>
          </plugin>
          <plugin>
              <groupId>org.codehaus.mojo</groupId>
              <artifactId>findbugs-maven-plugin</artifactId>
              <version>2.3.1</version>
              <configuration>
                <findbugsXmlOutput>true</findbugsXmlOutput>
                <findbugsXmlWithMessages>true</findbugsXmlWithMessages>
                <xmlOutput>true</xmlOutput>
                  <!-- Optional directory to put findbugs xml report --> 
                  <findbugsXmlOutputDirectory>target</findbugsXmlOutputDirectory>
                  <effort>Max</effort>
                  <threshold>Low</threshold>
            </configuration>
          </plugin>
      </plugins>
  </reporting>

</project>

So once alright in Eclipse, commit your application to a version control system. I use CVS, but this will work fine with Subversion as well. In the example below I commited App Engine project Rankington on branch Develop.

Now you could build your GAE application from a command line system with Maven installed with something like

CVSROOT=:pserver:rankington@213.xxx.xxx.xxx:/cvsrepo
export CVSROOT
cvs login
cvs co -r Develop Rankington
cd Rankington/
mvn package

This will generate a directory structure with your compiled classes, a packaged war-file ready for deployment to Google App Engine, directories with reports of your unit tests, static code analysis bug possibilities and code coverage.

Go to subdirectory target and you'll find everything



drwxr-xr-x 11 johan johan     4096 2011-04-16 14:28 ./
drwxr-xr-x  8 johan johan     4096 2011-04-16 14:28 ../
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 classes/
drwxr-xr-x  2 johan johan     4096 2011-04-16 14:28 cobertura/
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 generated-classes/
drwxr-xr-x  2 johan johan     4096 2011-04-16 14:28 maven-archiver/
drwxr-xr-x  8 johan johan     4096 2011-04-16 14:28 Rankington-1.0-SNAPSHOT/
-rw-r--r--  1 johan johan 23081938 2011-04-16 14:28 Rankington-1.0-SNAPSHOT.war
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 site/
drwxr-xr-x  2 johan johan     4096 2011-04-16 14:28 surefire-reports/
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 test-classes/
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 war/

Next time we'll look at how to integrate this into Hudson.

Building a Google App Engine application

I've been poking around lately with an application for creating competitions and tournaments with your friends. The main purpose is to keep track of results and creating rankings.

Similar to chess ratings, when you beat a better ranked opponent you will gain a lot of ranking while beating a less ranked opponent will not affect your ranking as much.

For test data I've been using Premier League results over some time period and calculating the teams rankings as functions of the teams performance.


I've been using Google App Engine (GAE) as deploy target and trying to take advantage of what the GAE infrastructure can offer. I'd thought I'll write some blog posts about pros and cons of using GAE and how my technology stack has worked out.

So what is GAE? GAE is a cloud based hosting service running applications written in Java or Python. In contrast to for example Amazon EC2 you have no control over what OS or web/application server you are running on. This might be a problem if you wish to have total control. For my purposes it is perfect since I don't need to worry about server and OS upgrades etcetera.

The main advantage is that you can focus entirely on the application and infrastructure like scaling of more servers and database backends comes for free. The pricing model is also very generous. App Engine is free to use up to certain quota limits of daily traffic measured on server requests, data store accesses, memcache data transfers etcetera. These limits are so ridiculously high that an average small to medium application can never breach them. If your application becomes a success you will start paying for the extra traffic.

So what can you run on GAE? If focusing on Java applications you must conform to the JEE container specs. What we got to work with is a JEE servlet container supporting most of the specification. Your application is a standard Java web application contained in war archive with the ordinary directory structure of WEB-INF etcetera.

A GAE application runs in a sandboxed environment with constraints like no forking of threads, no access of the file system and other IO tasks and not a complete Java SE library to work with, the AWT libraries are for example not bundled. But there are work arounds for most practical problems. For asynchronous jobs there is for example an API for creating long running tasks. Similarly there are alternatives to lacking JEE standards like JMS messages queues.

Naturally, you don't want to write to much code relying on propritary Google APIs to be able to move to other hosting systems. So when you hear that there is no standard relational SQL database to work with you might fell uneasy!

Data storing relies on Googles BigTable which you might see as a big and extremely fast lookup table. Fortunately GAE provides different access mechanisms to BigTable. If you're familiar with Java ORM frameworks like Hibernate or JPA you can feel at rest since there are JPA and JDO bindings so that you can be data store agnostic in your code.

Obviously, you can leverage a lot of Google APIs inside your application. For example reusing the the identification framework of Google Accounts or using Gmail as email infrastructure.

I'll post some notes on examples of the APIs and more info on GAE deployment.