Showing posts with label maven. Show all posts
Showing posts with label maven. Show all posts

Friday, 24 January 2014

Setup Jenkins project from Git repository with Findbugs, Cobertura and Checkstyle

I created a Jenkins setup for my current Java hobby project (a site on physical units intended to make ads money, check it out at http://www.all-about-units.com) yesterday and noticed that my old note on how to setup Jenkins (then it was written for Hudson) is a bit out of date. The Jenkins configuration menues for generating Findbugs static analysis, Cobertura code coverage and Checkstyle format control were completely new. In addition to that, I've migrated to Git since then so there are a few steps also involved in getting Git running as code repository within Jenkins.

Here's a workflow for setting up Jenkins with these features.

Install Jenkins and plugins

First of all, if you haven't installed Jenkins the easiest way is to download the war archive from http://jenkins-ci.org/. This article is based on Jenkins version 1.532.1. You can install in in a server container like Tomcat or Jetty, but the easiest way to get started is to run it from the command line like 

java -jar jenkins.war

I created a small bash script which also changes the port since I have another server using the default 8080 port.

#!/bin/sh

nohup java -jar jenkins.war --httpPort=8081

So now you can access Jenkins via http://localhost:8081 or similar.

Now we must install some plugins which are not part of the core distribution. Go to the settings of Jenkins and find the Plugins section.

Here you should at least find the following plugs and install them





Creating Jenkins job

Create a new Jenkins job. My project handles its dependencies via Maven 3 which suits Jenkins very well. Choose the Maven 2/3 option.


In the next step, choose Git as SCM. If you have your Git repository on the same server as Jenkins you can simply add the file path to the repository. Otherwise you setup the security credentials here as well. Also, choose if you want to build master or some other branch.


We haven't talked about the pom.xml for the project yet, but it will be configured to use some Maven goals. So in the Maven goals section, add goals for test package site.

Under Build settings, check the boxes for Publish Checkstyle analysis resultsPublish FindBugs analysis results and Publish duplicate code analysis results.


In the Post-build Actions add a step for Cobertura reporting.


Maven will generate the Cobertura report to the Jenkins workspace under path /target/site/cobertura/coverage.xml so add this in the configuration of the action.



Now you should have a Jenkins job setup ready for building.


Before building we must make sure the pom.xml of the project has the correct plugins and reports configured.

Maven pom.xml configuration

After all the dependency specifications in the pom.xml add something similar to this. In this setup Cobertura is configured in the package phase and that's why we added that Maven goal above. If you hadn't added this in the pom file, you could have added separate Maven goals for Cobertura in Jenkins as well.

<build>
   <finalName>Unitconversion</finalName>
   <plugins>
      <plugin>
         <groupId>org.codehaus.mojo</groupId>
         <artifactId>findbugs-maven-plugin</artifactId>
         <version>${findbugs.version}</version>
      </plugin>
      <plugin>
         <groupId>org.codehaus.mojo</groupId>
         <artifactId>cobertura-maven-plugin</artifactId>
         <version>${cobertura.version}</version>
         <configuration>
            <formats>
               <format>xml</format>
            </formats>
         </configuration>
         <executions>
            <execution>
               <phase>package</phase>
               <goals>
                  <goal>cobertura</goal>
               </goals>
            </execution>
         </executions>
      </plugin>
   </plugins>
</build>
<reporting>
   <plugins>
      <plugin>
         <groupId>org.codehaus.mojo</groupId>
         <artifactId>findbugs-maven-plugin</artifactId>
         <version>${findbugs.version}</version>
      </plugin>
      <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-checkstyle-plugin</artifactId>
         <version>${checkstyle.version}</version>
      </plugin>
      <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-surefire-report-plugin</artifactId>
         <version>${surefire.reportplugin.version}</version>
      </plugin>
      <plugin>
         <groupId>org.codehaus.mojo</groupId>
         <artifactId>cobertura-maven-plugin</artifactId>
         <version>${cobertura.version}</version>
         <configuration>
            <formats>
               <format>xml</format>
            </formats>
         </configuration>
      </plugin>
   </plugins>
</reporting>
Run the job and everything should work out of the box. Do another build to start getting history graphs like below of the trend of failing tests, checkstyle warnings, issues found by Findbugs and te code coverage statistics from Cobertura.




Saturday, 16 April 2011

Hudson - Continuous Integration for a Google App Engine application

The last blog post described how to configure a Maven project for a Google App Engine application. 
To build and deploy the Maven artifacts you will need some command line hacking.

Hudson CI can be used to escape the command line for all this. Hudson is a continuous integration system for building and testing your projects. It has some cool features as distributed building and much more.

To install Hudson, or Jenkins as the main fork has rebranded it now after Oracle came into clinch with the open source community, simply download the war-archive from either the Jenkins or Hudson web site. My installation is old so I use a Hudson build.You can either deploy it in a web container such as Tomcat or simply use the built in bootstrap. To bootstrap the war archive

java -Dhudson.udp=32850 -jar hudson.war --httpPort=9090 --daemon --logfile=/home/johan/hudson/hudson.log

Access Hudson via http://localhost:8080/.

Goto Manage Hudson -> Configure System and enable Maven under the Maven subsection. Goto Manage Hudson -> Manage Plugins and install the plugins you need. In my case I have installed Cobertura Plugin for code coverage, Findbugs for static code analysis, Maven2 Project Plugin, checkstyle for Java code validation and the CVS plugin. You might need to bring in some sub dependencies like Static Analysis Collector Plug-In, static Analysis Utilities.

Now create a new job, under your job, click Configure and setup appropriate version control config. In my case CVSROOT=:pserver:rankington:xxxx@213.xxx.xxx.xxx:/cvsrepo
and correct CVS module and branch.

Under Build, choose correct Maven installation, probably /usr/bin/mvn. Then setup your Maven build goals. In this case the goals are, enable debug, clean, compile, test, create war archive and generate reports.

-X
clean
package
findbugs:findbugs
checkstyle:checkstyle



This will make Maven checkout your code, compile it, run the Maven plugins for creating xml reports for Findbugs and Checkstyle.

Now add Post-build Actions to integrate these reports into Hudson. Enable Publish Findbugs analysis results from file **/target/findbugsXml.xml

Add similar report integrations by enabling publishing the following reports target/surefire-reports/*.xml for JUnit tests, **/target/site/cobertura/coverage.xml for Cobertura code coverage and so forth. 


Now build your Hudson job by clicking Build Now and the Maven goals are executed to checkout, build, test and creates reports of the project. Then the post action goals kicks in and updates the dashboards of Hudson to show the results of the build.

Cobertura allows you to see on a package level the unit test coverage and the possibility to drill down on package and file level. Similar graphs and drilling can be done on Findbugs, Checkstyle and JUnit test reports.


Cobertura reporting on file level
JUnit test reports over time, red indicates test cases have failed during those builds.
Checkstyle reports Java code issues and the interface makes drilling easy.
In this example Hudson will generate a war archive ready for deployment to a web or application server.


The Maven build could be extended to deploy the war at a local server to also run the web tests as a part of the Hudson job.






If you wish to also incorporate production deployment in the Hudson process you could use the Google App Engine scripting possibilities. Add a build step which does something like this to upload your newly built war archive to GAE.

appengine-java-sdk\bin\appcfg.cmd --email user@gmail.com --passin password update myapp/war

The same script can be used to download logs from production via
appengine-java-sdk/bin/appcfg.sh request_logs myapp/war mylogs.txt

Theres a bunch of more handy commands for scripting GAE if you run through the docs at http://code.google.com/intl/sv-SE/appengine/docs/java/tools/uploadinganapp.html.

Next time I'll try to describe what the production environment offers in addition to the local development environment.

Development environment for a Google App Engine application

There is integration with Google App Engine for several IDEs and you can also interact with the deployment tools from plain command line if you have that disposition.

I'm an Eclipse guy, and here's how I've done to get a good setup to be able to develop in an efficient way using Eclipsen, Maven, CVS, Hudson and GAE.

Fisrt of all, install the Google App Engine SDK from http://code.google.com/intl/sv-SE/appengine/docs/java/tools/eclipse.html and follow the install instructions.

You should get a new toolbar in Eclipse with icons for creating GAE web projects, profiling them and for deploying to the cloud. If you create a new GAE application you can have it deployed in Googles data centers in a few minutes. All you need is a standard Google Account.

A few tips:
  • You access your application at http://localhost:8888/ by default.
  • In your dev environment a light weight data store will be kept in WEB-INF\appengine-generated\local_db.bin. This store is permanent over server restarts and must be deleted if you wish a clean state. Indicies are automatically generated in WEB-INF\appengine-generated\datastore-indexes-auto.xml. You access the development console at http://localhost:8888/_ah/admin where you can inspect and make queries on the data store and work with task queues and more.
  • In addition to the standard JEE files such as web.xml there are a few specific config files for GAE projects. Inspect WEB-INF\appengine-web.xml where you setup logging and turn on sessions. By default you will not create web sessions when accessing you application

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
    <application>rankington</application>
    <version>1</version>
    
    <sessions-enabled>true</sessions-enabled>
    
    <!-- Configure java.util.logging -->
    <system-properties>
        <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
    </system-properties>
    
</appengine-web-app> 

If you wish to complicate your environment with some continous integration for compilation, bug, test and coverage reporting I'll outline how you could integrate your GAE application with Hudson.

I've fiddled around with the default directory structure to make a more Maven compliant tree for dependency management. That is no problem since the App Engine plugin uses the Eclipse .classpath to resolve the structure.  So create src/main/java src/main/resources /src/test/java src/test/resources and src/main/webapp. Create a pom.xml in your project root and setup your Maven dependencies. To make this easy, install the Eclipse Maven plugin which makes pom-editing a no-brainer. You will need a few special dependencies for App Engine. Be careful about which version of the plugin you use to match the Maven dependency.

<dependency>
   <groupId>com.google.appengine</groupId>
   <artifactId>appengine-api-1.0-sdk</artifactId>
   <version>1.4.0</version>
   <type>jar</type>
   <scope>compile</scope>
</dependency>

My Maven dependencies are:

It took me some time to figure out how to add unit testing, findbugs reporting and code coverage to my Maven build. So hopefully you can reuse something like this for your pom file. I've removed most of the code dependencies specific for my project for clarity

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>se.noren.rankington</groupId>
  <artifactId>Rankington</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>Maven Quick Start Archetype</name>
  <url>http://maven.apache.org</url>
  <ciManagement>
  </ciManagement>
  <build>
      <plugins>
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-compiler-plugin</artifactId>
              <version>2.3.2</version>
              <configuration>
                  <source>1.5</source>
                  <target>1.5</target>
            </configuration>
          </plugin>
          <plugin>
              <groupId>org.codehaus.mojo</groupId>
              <artifactId>surefire-report-maven-plugin</artifactId>
              <version>2.0-beta-1</version>
          </plugin>
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-checkstyle-plugin</artifactId>
              <version>2.6</version>
          </plugin>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>cobertura-maven-plugin</artifactId>
            <version>2.2</version>
            <configuration>
                <formats>
                    <format>xml</format>
                </formats>
            </configuration>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>cobertura</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>          
      </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.4</version>
      <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-beans</artifactId>
        <version>3.0.5.RELEASE</version>
        <type>jar</type>
        <scope>compile</scope>
    </dependency>
  </dependencies>
  <reporting>
      <plugins>
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-checkstyle-plugin</artifactId>
              <version>2.6</version>
          </plugin>
          <plugin>
              <groupId>org.codehaus.mojo</groupId>
              <artifactId>findbugs-maven-plugin</artifactId>
              <version>2.3.1</version>
              <configuration>
                <findbugsXmlOutput>true</findbugsXmlOutput>
                <findbugsXmlWithMessages>true</findbugsXmlWithMessages>
                <xmlOutput>true</xmlOutput>
                  <!-- Optional directory to put findbugs xml report --> 
                  <findbugsXmlOutputDirectory>target</findbugsXmlOutputDirectory>
                  <effort>Max</effort>
                  <threshold>Low</threshold>
            </configuration>
          </plugin>
      </plugins>
  </reporting>
</project>
So once alright in Eclipse, commit your application to a version control system. I use CVS, but this will work fine with Subversion as well. In the example below I commited App Engine project Rankington on branch Develop.

Now you could build your GAE application from a command line system with Maven installed with something like

CVSROOT=:pserver:rankington@213.xxx.xxx.xxx:/cvsrepo
export CVSROOT
cvs login
cvs co -r Develop Rankington
cd Rankington/
mvn package

This will generate a directory structure with your compiled classes, a packaged war-file ready for deployment to Google App Engine, directories with reports of your unit tests, static code analysis bug possibilities and code coverage.

Go to subdirectory target and you'll find everything



drwxr-xr-x 11 johan johan     4096 2011-04-16 14:28 ./
drwxr-xr-x  8 johan johan     4096 2011-04-16 14:28 ../
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 classes/
drwxr-xr-x  2 johan johan     4096 2011-04-16 14:28 cobertura/
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 generated-classes/
drwxr-xr-x  2 johan johan     4096 2011-04-16 14:28 maven-archiver/
drwxr-xr-x  8 johan johan     4096 2011-04-16 14:28 Rankington-1.0-SNAPSHOT/
-rw-r--r--  1 johan johan 23081938 2011-04-16 14:28 Rankington-1.0-SNAPSHOT.war
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 site/
drwxr-xr-x  2 johan johan     4096 2011-04-16 14:28 surefire-reports/
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 test-classes/
drwxr-xr-x  3 johan johan     4096 2011-04-16 14:28 war/

Next time we'll look at how to integrate this into Hudson.