Picking is the process of finding objects in you scene based on user input. Most commonly, you want to determine what object a user has clicked with his mouse. In this case the 2D mouse coordinates serve as a reference on the view port to identify the projected objected that has been clicked. A similar scenario is a first person shooter, where the gun serves as the picking pointer, however in world space coordinates. If you shoot your gun, the trajectory of the bullet is traced through your scene and collisions are detected.

A screenshot showing a hex map in an LWJGL application with a selected tile as an example for 3D picking

OpenGL Selection Mode vs. Ray Picking

There are two common ways of implementing picking. The first one makes use of a special OpenGL feature in which you render your scene in a so-called “selection mode”. In selection mode, OpenGL does not render to the frame buffer but gathers depths information of the objects with is later read at the point where your picking occurs. The second approach is called “Ray Picking” and is independent from OpenGL and simulates a line that is shot through the scene until it hits an object. I personally prefer the second approach for the following reasons:

• You don’t need an additional render pass. A render pass can take some considerable amount of time and special clipping/culling code is required to only render the small part of the scene which you want to perform the picking on. In addition, you render a less complex scene (omitting textures, lighting, normals, etc) in selection mode if possible which can add a high degree of complexity to your drawing routines (imagine breaking up display lists). Ray picking obviously requires some code to trace the ray through your scene, but this is usually less complex.
• Ray picking can be done independently from the rest of the game code as a simultaneous and “read-only” operation on the scene graph. In some environments such as AWT/JOGL where the input devices are handled by a separate thread, your picking code can run as part of the handler thread and does not impact the performance of the rest of the game. This comes in handy especially in scenarios where you need to perform picking very often (eg. mouse hovering, target laser on the in-game gun).
• In pretty much all Java environments, calling OpenGL takes a considerable amount of time due to JNI. It is therefore beneficial to minimize the number OpenGL calls. Rendering in selection mode would mean additional OpenGL calls that I rather substitute with ray picking.

However, there are scenarios where ray picking is simply inferior or impractical. Consider picking in a scene with a lot of small details (eg. tree with little branches and leaves) or if you need to know the individual polygon/triangle that you picked. I will therefore cover both approaches, starting with ray picking.

Ray Picking

Ray picking is the process of shooting a line (ie. “ray”) from the camera through the 2D viewscreen (where the 3D scene is projected on) into the scene until it hits an object. To do so, we need to know the camera and the point on the viewscreen (eg. mouse cursor position). The first point is trivial, however the second point is a bit more difficult to determine. Let’s say we have a 2D point on the viewscreen (x, y) and now want to map this point into world coordinates. One way would be to make use of the inverted viewing matrix, but inverting a matrix is usually an expensive undertaking. Instead, we compute the position of the plane of the viewscreen in world space and map the 2D point on this plane and from there into world space.

The algorithm in pseudo code follows below. Note that cameraLookAt is the 3D point where the camera looks at (as used on glLookat), cameraPosition is the current position of the camera in world space and cameraUp is the up vector of the camera.

x = getMouseX() // scalar
y = getMouseY() // scalar

view = cameraLookAt - cameraPosition // 3D float vector
normalize view

h = crossProduct( view, cameraUp ) // 3D float vector
normalize h

v = crossProduct( h, view) // 3D float vector
normalize v

ok, so far for h and v. We now need to compute the lengths of h and v in the view port. In my example I assume that you set up your frustum by a field of view angle in degrees (fovy) as commonly the case with glFrustum. Let nearClippingDistance be the distance to the near clipping plane and width/height the ratio of the view port width divided by the viewport heigth.

// convert fovy to radians
rad = fovy * PI / 180
vLength = tan( rad / 2 ) * nearClippingPlaneDistance
hLength = vLength * (width / height)

scale v by vLength
scale h by hLength

The two scalars vLength and hLength in combinaton with v and h help spanning the view port plane extending from the center point. At this point it is worth mentioning that it may make sense to only compute h and v when you update the camera orientation because the normalization involves a square root which is a relatively expensive operation (actually tan may also be avoided).

Now it is time to map the 2D mouse coordinates onto the view port plane.

// translate mouse coordinates so that the origin lies in the center
// of the view port
x -= width / 2
y -= height / 2

// scale mouse coordinates so that half the view port width and height
// becomes 1
y /= (height / 2)
x /= (width / 2)

// linear combination to compute intersection of picking ray with
// view port plane
pos = cameraPos + view + h*x + v*y

// compute direction of picking ray by subtracting intersection point
// with camera position
dir = pos - cameraPos

That's pretty much it. We now got pos as the intersection point of the picking ray in the view port plane and the picking ray direction dir. This describes a line that can now be used to intersect with the individual objects of the scene.

// brute force
for all objects in the scene
  test for intersection and keep closest
end for

So by intersecting the picking ray with all your visible objects in your scene while keeping track of the distance, you determine the object that is hit by your mouse cursor.
Checking all objects in our scene may not be very efficient though. So in a first attempt I recommend to test only objects that lie (partly) within your viewing frustum. Under normal circumstances you got that code already in your scene graph as part of the software culling routine when pumping obects into the render pipeline.

I hope this helped. If you liked the post or if you got questions, ideas hints or found a problem, please leave a comment. I am trying to answer as quickly as possible. Also, if you want me to send you the source code of this example please let me know, I am happy to share the code.

For this tutorial, I am using

• jogl-b633-2012-01-23_20-37-13, and
• gluegen-b480-2012-01-23_16-49-04.

This is how your project setup should look like:

So one you got the binaries, unpack the ZIP file somewhere. You will notice the jar folder with a number of JAR files. All you need is in that jar folder, because the DLL binaries are included in the JARs.

Create a new eclipse project (name doesnt matter) and add the following JARs on the classpath:

• gluegen-rt-natives-windows-amd64.jar
• gluegen-rt.jar
• jogl-all-natives-windows-amd64.jar
• gluegen-rt.jar

These files are the native part of the OpenGL binding and are linked via Java Native Interface (JNI). Since the DLLs are precompiled as part of the JAR files, you are almost ready to go.

Once you got that set up, create a package de.schabby.jogl.helloworld and copypaste the following two classes in the newly created package.

package de.schabby.jogl.helloworld;
 
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
 
import javax.media.opengl.GLCapabilities;
import javax.media.opengl.GLProfile;
import javax.media.opengl.awt.GLCanvas;
import javax.swing.JFrame;
 
public class HelloWorld 
{
    public static void main(String[] args) 
    {
    	// setup OpenGL Version 2
    	GLProfile profile = GLProfile.get(GLProfile.GL2);
    	GLCapabilities capabilities = new GLCapabilities(profile);
 
    	// The canvas is the widget that's drawn in the JFrame
    	GLCanvas glcanvas = new GLCanvas(capabilities);
    	glcanvas.addGLEventListener(new Renderer());
    	glcanvas.setSize( 300, 300 );
 
        JFrame frame = new JFrame( "Hello World" );
        frame.getContentPane().add( glcanvas);
 
        // shutdown the program on windows close event
        frame.addWindowListener(new WindowAdapter() {
            public void windowClosing(WindowEvent ev) {
                System.exit(0);
            }
        });
 
        frame.setSize( frame.getContentPane().getPreferredSize() );
        frame.setVisible( true );
    }
}

The Render.java class looks like follows. It implements GLEventListener which is the call-back implementaiton by JOGL to do all OpenGL rendering.

package de.schabby.jogl.helloworld;
 
import javax.media.opengl.GL2;
import javax.media.opengl.GLAutoDrawable;
import javax.media.opengl.GLEventListener;
import javax.media.opengl.glu.GLU;
 
class Renderer implements GLEventListener 
{
    private GLU glu = new GLU();
 
    public void display(GLAutoDrawable gLDrawable) 
    {
        final GL2 gl = gLDrawable.getGL().getGL2();
        gl.glClear(GL2.GL_COLOR_BUFFER_BIT | GL2.GL_DEPTH_BUFFER_BIT);
        gl.glLoadIdentity();
        gl.glTranslatef(-1.5f, 0.0f, -6.0f);
        gl.glBegin(GL2.GL_TRIANGLES);		
        gl.glVertex3f(0.0f, 1.0f, 0.0f);	
        gl.glVertex3f(-1.0f, -1.0f, 0.0f);	
        gl.glVertex3f(1.0f, -1.0f, 0.0f);	
        gl.glEnd();				
        gl.glTranslatef(3.0f, 0.0f, 0.0f);
        gl.glBegin(GL2.GL_QUADS);           	
        gl.glVertex3f(-1.0f, 1.0f, 0.0f);	
        gl.glVertex3f(1.0f, 1.0f, 0.0f);	
        gl.glVertex3f(1.0f, -1.0f, 0.0f);	
        gl.glVertex3f(-1.0f, -1.0f, 0.0f);	
        gl.glEnd();				
        gl.glFlush();
    }
 
 
    public void displayChanged(GLAutoDrawable gLDrawable, boolean modeChanged, boolean deviceChanged) 
    {
    	System.out.println("displayChanged called");
    }
 
    public void init(GLAutoDrawable gLDrawable) 
    {
    	System.out.println("init() called");
        GL2 gl = gLDrawable.getGL().getGL2();
        gl.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
        gl.glShadeModel(GL2.GL_FLAT);
    }
 
    public void reshape(GLAutoDrawable gLDrawable, int x, int y, int width, int height) 
    {
    	System.out.println("reshape() called: x = "+x+", y = "+y+", width = "+width+", height = "+height);
        final GL2 gl = gLDrawable.getGL().getGL2();
 
        if (height <= 0) // avoid a divide by zero error!
        {
            height = 1;
        }
 
        final float h = (float) width / (float) height;
 
        gl.glViewport(0, 0, width, height);
        gl.glMatrixMode(GL2.GL_PROJECTION);
        gl.glLoadIdentity();
        glu.gluPerspective(45.0f, h, 1.0, 20.0);
        gl.glMatrixMode(GL2.GL_MODELVIEW);
        gl.glLoadIdentity();
    }
 
 
	public void dispose(GLAutoDrawable arg0) 
	{
		System.out.println("dispose() called");
	}
}

By right-clicking on the main() method and chosing "run as application", Eclipse will execute the program. You should see something like this:

That's it so far. Please let me know if you have questions or comments.

• recent JOGL binary distribution download
• precompiled Windows Java OpenGL bindings
• you just can not find the place where to download JOGL

I had a couple of hours on my own recently and though it would be a good thing to catch up with the latest developments in the Java+OpenGL environment. So I looked for the download of JOGL (Java Bindings for OpenGL) just to realize, that I is actually quite difficult to find a recent binary distribution of JOGL!

Eventually, after a while I found it: http://jogamp.org/deployment/autobuilds/master/.

There you find an extensive list of directories containing builds, mostly for different binding projects such as gluegen, joal, etc. Search for “jogl” to jump to the JOGL builds.

The downloads are basically a list of (probably automated) builds. So I hope that this will be the place for the next months to get update-to-date JOGL binaries. The builds that have the suffix “-master” in their directory name appear to contain only documentation.

Please also note that for running JOGL you also need gluegen. I personally use the 2012-01-23 release at the moment. This is

• gluegen-b480-2012-01-23_16-49-04
• jogl-b633-2012-01-23_20-37-13

which seem to work.

So the guys at JogAmp.org do actually provide precompiled binaries, they just make it difficult to find them.

Now that you downloaded JOGL, you may be interested in my JOGL Hello World example (includes how to set up the classpath and DLLs right).

Please leave a comment if it helps or my resource gets out of date.

2010-10-20 EDIT: updated download URL.
2010-11-03 EDIT: updated download URL again to point to autobild directory.
2010-11-19 EDIT: added reference to my hello world example with JOGL
2011-07-27 EDIT: updated link to JOGL downloads
2012-01-27 EDIT: updated link to JOGL downloads, added note about gluegen

Es stellt sich also folgende Frage:

• Session unterdrücken in Tomcat / JSP
• Tomcat keine Sessions
• JSP / Tomcat ohne Sessions

Im Prinzip ist die Lösung sehr einfach. Man muss nur zwei einfache Regeln befolgen.

1. Mache nur nur request.getSession() calls wenn Du wirklich eine neue oder die bestehende Session brauchst (Login, etc.). Überlege Dir auch, ob nicht request.getSession(false) reicht (= nur eine bereits bestehende Session zurückgeben, aber keine neue Anlegen falls noch keine existiert).
2. Setze in allen JSPs die Direktive <%@ page session="false" %> um das automatische Erzeugen einer Session im JSP-Code zu unterbinden. Wenn Du die Session doch in einer JSP brauchen solltest, erstelle sie vorher in einem Controller.

1. BufferedReader verbessert die Lesegeschwindigkeit aus anderen Readern (z.B. FileReader um von der Festplatte zu lesen) drastisch,
2. BufferedReader hat praktischer Weise eine readLine() Methode, mit deren Hilfe man ganze Zeilen lesen kann.

In Prinzip ist der BufferedReader auch nur ein Reader. Das heisst er hat die typischen read(…) Methoden. Intern aber, arbeitet der BufferedReader auf einem Speicherbereich (dem Buffer), den er erst füllt, bevor seine read(…) Methoden darauf arbeiten können. Das heisst, dass der BufferedReader größere Datenblöcke auf einmal lesen kann und die dann nach und nach an den äußeren Reader weiterleiten kann. Das ist insbesondere sinnvoll bei Readern die von wiederum von Low-Level Readern lesen, die jeweils bei größeren Datenblöcken performant sind, wie typischer Weise von der Festplatte oder vom Socket.

Ein typischer Gebrauch eines BufferedReaders sieht so aus:

InputStreamReader inputStreamReader = new InputStreamReader(new FileInputStream (textFile), encoding );
BufferedReader bufferedReader = new BufferedReader (inputStreamReader);

Uebrigens, JDK heisst Java Development Kit, und SDK Software Development Kit. Es nimmt sich aber defacto nichts.

Ist man nun bei SUN gelandet, gibt es drei Varianten zum downloaden.

• Java SE: steht fuer Standard Edition ist ziemlich sicher das was Du suchst wenn Du das klassische Javapaket fuer den Deskop runterladen willst (JRE und JDK)
• Java ME: steht fuer Mobile Edition und beinhaltet eine auf fuer Handies zugeschnittene Java Plattform (mit Compiler) mit reduzierten Bibliotheken
• Java EE: steht fuer Enterprise Edition und ist deutlich umfangreicher als die Standard Edition und enthaelt zusaetzliche, business nahe Tools wie den Glassfish Application Server

Ich hoffe das hilft.

• Maven tests ausschalten
• mvn ignoriere test fehler

Wenn man trotzdem seinen maven snapshot bauen möchte, müssen aber die tests failures ignoriert werden. Das wird mittels einer Property direkt auf der Kommandezeile bewerkstelligt. Da ich dieses Property andauernd vergesse, möchte ich es mir hier selbst und der Allgemeinheit nochmals vor Augen führen:

mvn -Dmaven.test.failure.ignore=true install

Hope that helps…

After extracting Cassandra to some folder (on my Windows box I placed it directly in D:\cassandra), the only file you need to edit is conf/storage-conf.xml. While Cassandra is engineered to run on a large number of machines in a network, we start it here as a single node with the default parameter set, so that most of the settings are ok for now.

If your are not on a Unix-like system, you need to update the folders where Cassandra is supposed to store the data. If your using Windows (like me), then find the following lines in conf/storage-conf.xml and change the paths to something sensible

<CommitLogDirectory>/var/lib/cassandra/commitlog</CommitLogDirectory>
<DataFileDirectories>
      <DataFileDirectory>/var/lib/cassandra/data</DataFileDirectory>
</DataFileDirectories>
<CalloutLocation>/var/lib/cassandra/callouts</CalloutLocation>
<BootstrapFileDirectory>/var/lib/cassandra/bootstrap</BootstrapFileDirectory>
<StagingFileDirectory>/var/lib/cassandra/staging</StagingFileDirectory>

like for example my settings:

<CommitLogDirectory>D:/cassandra/data/commitlog</CommitLogDirectory>
<DataFileDirectories>
       <DataFileDirectory>D:/cassandra/data/data</DataFileDirectory>
</DataFileDirectories>
<CalloutLocation>D:/cassandra/data/callouts</CalloutLocation>
<BootstrapFileDirectory>D:/cassandra/data/bootstrap</BootstrapFileDirectory>
<StagingFileDirectory>D:/cassandra/data/staging</StagingFileDirectory>

Let’s take Cassandra for a spin and check if she starts up correctly. For Mac OS, Linux, etc. users, simply change to the bin directory of Cassandra and run ./cassandra. As an aside for the impatient, I start Cassanda with sudo to avoid trouble with the Cassandras system.log.

Windows users, however, that use the command line (meaning not Cygwin) cannot start it just like that. The cassandra.bat didnt work for me on my Vista box if executed with bin being the current working directory (probably due to the CASSANDRA_HOME environment variable that get’s incorrectly set in the batch file). BUT it works perfect if you call bin\cassandra.bat from Cassandra’s main directory above bin. So if you are on Windows, change to the directory where you extracted Cassandra and execute bin\cassandra.bat.

Cassandras output on startup will look similar to this (here on Mac OS):

Schabbys-MacBook-Pro:bin johannes$ sudo ./cassandra
Schabbys-MacBook-Pro:bin johannes$ Listening for transport dt_socket at address: 8888
DEBUG - Loading settings from ./../conf/storage-conf.xml
DEBUG - Syncing log with a period of 1000
DEBUG - opening keyspace Keyspace1
DEBUG - adding Super1 as 0
DEBUG - adding Standard2 as 1
DEBUG - adding Standard1 as 2
DEBUG - adding StandardByUUID1 as 3
DEBUG - adding LocationInfo as 4
DEBUG - adding HintsColumnFamily as 5
DEBUG - opening keyspace system
DEBUG - INDEX LOAD TIME for /Users/johannes/cassandra/data/system/LocationInfo-1-Data.db: 0 ms.
DEBUG - INDEX LOAD TIME for /Users/johannes/cassandra/data/system/LocationInfo-2-Data.db: 0 ms.
DEBUG - INDEX LOAD TIME for /Users/johannes/cassandra/data/system/LocationInfo-3-Data.db: 0 ms.
INFO - Replaying /Users/johannes/cassandra/commitlog/CommitLog-1257980407451.log
DEBUG - Replaying /Users/johannes/cassandra/commitlog/CommitLog-1257980407451.log starting at 117
DEBUG - Reading mutation at 117
DEBUG - replaying mutation for system.L: {ColumnFamily(LocationInfo [Generation,])}
INFO - Flushing Memtable(LocationInfo)@228828460
DEBUG - Submitting LocationInfo for compaction
INFO - Completed flushing Memtable(LocationInfo)@228828460
INFO - Compacting [/Users/johannes/cassandra/data/system/LocationInfo-1-Data.db,/Users/johannes/cassandra/data/system/LocationInfo-2-Data.db,/Users/johannes/cassandra/data/system/LocationInfo-3-Data.db,/Users/johannes/cassandra/data/system/LocationInfo-4-Data.db]
DEBUG - index size for bloom filter calc for file  : /Users/johannes/cassandra/data/system/LocationInfo-1-Data.db   : 256
DEBUG - index size for bloom filter calc for file  : /Users/johannes/cassandra/data/system/LocationInfo-2-Data.db   : 512
DEBUG - index size for bloom filter calc for file  : /Users/johannes/cassandra/data/system/LocationInfo-3-Data.db   : 768
DEBUG - index size for bloom filter calc for file  : /Users/johannes/cassandra/data/system/LocationInfo-4-Data.db   : 1024
DEBUG - Expected bloom filter size : 1024
INFO - Compacted to /Users/johannes/cassandra/data/system/LocationInfo-5-Data.db.  0/255 bytes for 0/1 keys read/written.  Time: 150ms.
DEBUG - collecting Generation:false:4@3
DEBUG - collecting Token:false:16@0
INFO - Saved Token found: 160533723849634883377008460059010504450
DEBUG - Starting to listen on 127.0.0.1:7001
DEBUG - Binding thrift service to localhost:9160

I think that’s it. Leave a comment if you run in trouble or check the nice If Something Goes Wrong page in the Cassandra Wiki.

I have been toying around with Cassandra for quite some time now. From all NOSQL databases I have seen (and there are quite a few already as Michael pointed out to me earlier), Cassandra seems to be the most promising one to me for reasons that are definitely worth discussing, but are here be beyond the scope of this post.

Data Model

Cassandras data model has been described more than once. In contrast to the descriptions above, I will try to follow a more javaish view which I find easiest and most powerful to work with. I thereby start describing Cassandras data model as nested hash maps.

The way in which data get’s stored in key/value based databases like Cassandra strongly resembles the use of ordinary hash maps. To recall, hash maps store data for a (unique) key. The key is also later used to retrieve the data from the hash map. For example, in order to map string keys to byte arrays you would write in Java

Map<String, byte[]> map = new HashMap<String, byte[]>();

This principle stays the same with Cassandra. However, in Cassandra you do not have a single hash map but up to three layers of nested hash maps! What does the mean? Imagine you dont store your values in a single byte array for each key, but again in a hash map, like

Map<String, Map<byte[], byte[]>> map = new HashMap<String, Map<byte[], byte[]>>();

This way you would partition the data you want to store as key/value pairs that are first filled in the data hash map. The data hash map then gets inserted in the higher-order hash map for a given key string. Similarly, to retrieve a value, you would provide the key string and get the data hash map from which you would extract the value you are interested in.

Let us further assume that we dont want to store the key/value pairs as two individual values, but coupled in a class called “Column” so that our data model would look like this:

Map<String, Map<byte[], Column>> map = new HashMap<String, Map<byte[], Column>>();

Where Column is defined as:

class Column {
    byte[] name;
    byte[] value;
    long timestamp;
 
    public Column(byte[] name, byte[] value) {
         this.name = name;
         this.value = value;
         this.timestamp = System.currentMillis();
    }
}

This is already pretty close to what is called a Column Family in Cassandra. You need to restrain yourself from deriving something from the name “Column”. Also ignore timestamp which is used by Cassandra to avoid data inconsistency and which shall not bother us here.

Before we go on, let us have a look on a concrete example on how you would need to work with this kind of data structure. Let us assume we want to store the profile data of a single user for some imaginary social networking website.

/* data model to store user profiles */
Map<String, Map<byte[], Column>> user = new HashMap<String, Map<byte[], Column>>();
 
/* create a user 'schabby' */
user.put("schabby", new HashMap<byte[], Column>());
 
/* fill in some profile data for user 'schabby' */
Column age = new Column("age".toBytes(), new byte[]{ 27b });
user.get("schabby").put(age.name, age);
 
Column realName = new Column("real name".toBytes(), "Johannes Schaback".toBytes());
user.get("schabby").put(realName.name, realName);
 
Column nationality = new Column("nationality".toBytes(), "German".toBytes());
user.get("schabby").put(nationality.name, nationality);

Again, do not get confused by the use of the byte arrays where normal string would make more sense. This is to resemble the Cassandra data model as close as possible. You will later realize that it’s actually quite nifty to keep the inner hash map byte based for the price of manually converting everything to byte arrays.

If we want to retrieve values from our data structure, we would need to do as follows:

byte age = user.get("schabby").get("age".toBytes()).value[0];
String realName  = new String(user.get("schabby").get("real name".toBytes()).value);
String nationality= new String(user.get("schabby").get("nationality".toBytes()).value);

And this is it. There is not much more conceptual stuff to understand in order to use Cassandra. So we are now ready to project this structure to Cassanda terminology.

Column Family

Cassandra structures its data model in keyspaces, Column Families (CF), Columns and SuperColumns.

A keyspace is a namespace to group Column Families and can be compared to a schema or single database in the SQL world. A keyspace contains one or more Column Families.

A Column Family can be seen as a multidimensional hash map like the one in our example above. In the SQL analogy, you may see a Column Family as a single table that belongs to a schema, however this comparison will not take you far. It is really more a dynamically growing and shrinking hash map rather than a table with fixed columns. Still, in Cassandras terminology you speak of rows when you refer to the hash map that you get for a key string.

Rows are accessed by string keys and each row – which can be seen as a “data hash map” – has several columns. Each column within a row is a bundled pair of a byte array key (a.k.a name) and its byte array data field (a.k.a. value) very similar to our example.

Depending on your configuration, you can let Cassandra apply a sorting scheme to impose an order over your columns in a row. This enables to query ranges over columns. For example, imagine a telephone book from which you want to retrieve all names starting with “Smi”. In Java terms, this could be compared to using SortedMap instead of Map. But we sticked to Map for simplicity here.

SuperColumns

The cool thing about Cassandra is its support for an additional hash map layer. This additional layer is added to the Column layer and enables you to store and access your data as a hash map in a hash map in a hash map, or in other words, as a three dimensional hash map. This additional hash map is called a SuperColumn (SC)

In our Java-like example, a Column Family with SuperColumns look like

Map<String, Map<byte[], SuperColumn>> superColumn 
     = new HashMap<String, Map<byte[], SuperColumn>>();

where SuperColumn is again a hash map over columns like

class SuperColumn extends HashMap<byte[], Column>
{
}

Again, I want to point out that the actual SuperColumn definition in Cassandra is different and that this explanatory definition is not too accurate, but nicely serves the illustration purpose.

Similar to normal Columns, the values within a SuperColumn are also stored in an order depending on your configuration, enabling to cut out slices from your SuperColumns.

To continue our social networking site example, let us have a look on how SuperColumns are used to store the friend and relations of the user ‘schabby’.

/* create ColumnFamily with SuperColumns */
Map<String, Map<byte[], SuperColumn>> columnFamily = new HashMap<String, Map<byte[], SuperColumn>>();
 
/* prepare a SuperColumn for 'schabby' */
columnFamily.put("schabby", new HashMap<byte[], SuperColumn>());
 
/* create SC to store friend info */
SuperColumn friends = new SuperColumn();
 
/* fill in some friends */
Column friend1 = new Column("friend_1".toBytes(), "Merry".toBytes());
friends.put(friend1.name, friend1);
 
Column friend2 = new Column("friend_2".toBytes(), "Robert".toBytes());
friends.put(friend2.name, friend2);
 
Column friend3 = new Column("friend_3".toBytes(), "Susan".toBytes());
friends.put(friend3.name, friend3);
 
/* finally store SC in Colunm Family */
columnFamily.get("schabby").put("friends".toBytes(), friends);

We are free to create another SuperColumn in the same Column Family to store other list-like data for ‘schabby’, for example his inbox.

/* ... continued example */
 
SuperColumn inbox = new SuperColumn();
 
/* add two mails to inbox */
Column mail1 = new Column("Hi Schabby".toBytes(), "I hope you are well! Cheers, Nick".toBytes());
inbox.put(mail1.name, mail1);
 
Column mail2 = new Column("Welcome".toBytes(), "some message body".toBytes());
inbox.put(mail2.name, mail2);
 
columnFamily.get("schabby").put("inbox".toBytes(), inbox);

Retrieving the mails from the inbox is straight forward:

/* continued example */
 
SuperColumn inbox = columnFamily.get("schabby").get("inbox".toBytes());
 
for(byte[] subject: inbox.keySet())
{
   String body = inbox.get(subject);
   // do something with subject/body
}

And this is it. I hope this enlightened your understanding of Cassandras data model. It’s not that difficult all in all, especially when you start using it.

Please leave some comments for corrections and feedback.

The updated API got more coherent and consistent. Its changes mainly address the new authentication model and the application store in which you can feature your own applications based on Zanox data.

Now for the purpose of this post, I sum up how to connect to Zanox SOAP API using Java. There are several steps to do, mostly administrative in nature.

Get the keys

Zanox implements a shared key authentication model. Therefore, you need to get yourself the right keys (public and private) to get started. Here’s the page that describes how to get them. Once you created your own application, you will receive the following keys that are necessary prerequisites for your code

• zanox Application ID
• Connect ID
• Public key
• zanox secret key (aka. private key)

Generate Zanox SOAP Beans From WSDL

Once you have the keys, you need to generate the classes from the WSDL. For the purpose of this post, we used the (currently latest) WSDL on

http://api.zanox.com/wsdl/2009-07-01

with the endpoint at

http://api.zanox.com/soap/2009-07-01

We used wsimport to generate the classes as follows

wsimport -d bin -s src -p com.zanox.api.namespace._2009_07_01 http://api.zanox.com/wsdl/2009-07-01

This should generate you a bunch of class in in the src folder.

Example Code to Connect to Zanox SOAP API

In order to run our example code, you need to setup your classpath with the following libraries (actually I am not sure if they are really all necessary)

• XmlSchema-1.4.3.jar
• asm-2.2.3.jar
• cxf-2.1.4.jar
• geronimo-activation_1.1_spec-1.0.2.jar
• geronimo-annotation_1.0_spec-1.1.1.jar
• geronimo-jaxws_2.1_spec-1.0.jar
• geronimo-stax-api_1.0_spec-1.0.1.jar
• geronimo-ws-metadata_2.0_spec-1.1.2.jar
• jaxb-api-2.1.jar
• jaxb-impl-2.1.9.jar
• jaxb-xjc-2.1.9.jar
• saaj-api-1.3.jar
• saaj-impl-1.3.2.jar
• wsdl4j-1.6.2.jar
• wstx-asl-3.2.6.jar
• xml-resolver-1.2.jar

Alrighty, we got all we need to fill in the last bricks. The example code conncets to the API and retrieves the sales of the specified date. The API holds far more methods for different purposes, but they are quit similar in use and once you got the sale-example working (even though you might not generate any sales yet) you can easily infere the correct usage and parameter sets for all other methods.

Create two classes: ZanoxAuthenticationDto and SoapZanoxFacade. The first DTO is just to hold the authentication data and looks like this:

public class ZanoxAuthenticationDto
{
	private String applicationId;
	private String connectId;
	private String sharedKey;
	private String privateKey;
 
 // ... according getter and setter methods
 //  are left out for simplicity reasons
}

In the SoapZanoxFacade class we do the actual work and contain all following code snippets.

We first define a couple of helper methods. The most important one is certainly the one to compute the correct signature that is sent to Zanox for every request.

    public static String createSoapSignature(final String serviceName, final String serviceMethod, final String timestamp, String nonce, String secreyKey) 
    {
        final String HMAC_SHA1_ALGORITHM = "HmacSHA1";
 
        // Acquire an HMAC/SHA1 from the raw key bytes.
        SecretKeySpec signingKey = new SecretKeySpec(secreyKey.getBytes(), HMAC_SHA1_ALGORITHM);
 
        // Acquire the MAC instance and initialize with the signing key.
        Mac mac = null;
        try 
        {
            mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
            mac.init(signingKey);
        } 
        catch (Exception e) 
        {
            throw new RuntimeException("trouble while initiating/signing", e);
        }
 
        String input = serviceName.toLowerCase() + serviceMethod.toLowerCase() + timestamp + nonce;
 
        // Compute the HMAC on the digest, and set it.
        String signature = new BASE64Encoder().encode(mac.doFinal(input.getBytes()));
        return signature;
    }

Another one to format the timestamp

    public static String createSoapTimestamp() 
    {
        String fmt = "yyyy-MM-dd'T'HH:mm:ss.000'Z'";
        SimpleDateFormat df = new SimpleDateFormat(fmt, Locale.US);
        df.setTimeZone(TimeZone.getTimeZone("UTC"));
        String date = df.format(new Date());
        return date;
    }

one to converte java.util.Calendar to XmlGregorianCalendar

    public static XMLGregorianCalendar toXmlGregorianCalendar(Calendar c)
    {
    	if( !(c instanceof GregorianCalendar) )
    	{
    		throw new ZanoxWsException("Calendar instance must be of type GregorianCalendar");
    	}
 
    	try
		{
			return DatatypeFactory.newInstance().newXMLGregorianCalendar((GregorianCalendar)c);
		} 
    	catch (DatatypeConfigurationException e)
		{
    		throw new ZanoxWsException(e);
		}    	
    }

and the last one to generate a random string of specified length (aka. nonce)

    public static String createNonce(int length)
    {
    	final String CHARS = "01234567890abcdefghijkmlmnopqrstuvwxyz";
    	StringBuilder sb = new StringBuilder();
    	for(int i=0; i < length; i++)
    		sb.append(CHARS.charAt((int)(Math.random()*CHARS.length())));
 
    	return sb.toString();
    }

The final code snipped finally retrieves the sales:

	/**
	 * Retrieve the first 10K sales for day <code>calTime</code>. 
	 * Paging is hard-wired to first page (in other words: no paging)
	 */
    public void getSales(Calendar calTime, ZanoxAuthenticationDto auth)  
    {
        String timestamp = createSoapTimestamp();
        String nonce     = createNonce(20);
        String signature = createSoapSignature("PublisherService", "GetSales", timestamp, nonce, auth.getPrivateKey());
        XMLGregorianCalendar date = toXmlGregorianCalendar(calTime);
 
        GetSalesRequest getSalesRequest = new GetSalesRequest();
        getSalesRequest.setConnectId(auth.getConnectId());
        getSalesRequest.setDate(date);
        getSalesRequest.setTimestamp(timestamp);
        getSalesRequest.setPage(0);
        getSalesRequest.setItems(10000);
        getSalesRequest.setSignature(signature);
        getSalesRequest.setNonce(nonce);
 
        GetSalesResponse response = null;
 
        try 
        {
            response = getPublisher().getSales(getSalesRequest);
        } 
        catch (Exception e)
        {
            throw new ZanoxWsException("error while fetching sales from Zanox SOAP API", e);
        } 
 
        // TODO: do something sensible with the data...
        for( SaleItem item: response.getSaleItems().getSaleItem())
        {
        	System.out.println(item.getProgram().getValue()+"\t "+item.getSubPublisher().getId()+"\t"+item.getClickId()+" "+item.getAmount()+" "+item.getLptId()+" "+item.getCommission());
        }
    }
 
    private PublisherSoapPortType getPublisher() 
    {
        final PublisherService pubs = new PublisherService();
        return pubs.getPublisherSoapPort();
    }

To run the code, create a main methode where you set up you authentication data and call getSales on SoapZanoxFacade.

    public static void main(String[] args) 
    {
    	ZanoxAuthenticationDto dto = new ZanoxAuthenticationDto();
    	dto.setConnectId("ASL22DKAS223SKD55KAgD");
    	dto.setSharedKey("JY89AFLLA(RKJZZ");
    	dto.setPrivateKey("asdkljah893qisdnfsyniweoi8ynw893nw");
    	dto.setApplicationId("sdfs4w9p4uao8oadf9fusm9d");
 
    	Calendar c = Calendar.getInstance();
    	c.set(2009, 9, 2);
    	SoapZanoxFacade l = new SoapZanoxFacade();
    	l.getSales(c, dto);
    }

This should get you going with the API. All other SOAP requests are built in a similar fashion.