Beyond the Data Grid: Coherence, Normalisation, Joins and Linear Scalability (QCon)

Normalisation is, in many ways, the antithesis of typical cache design. We tend to denormalise for speed. Building a data store (rather than a cache) is a little different: Manageability, versioning, bi-temporal reconstitution become more important factors. Normalisation helps solve these problems but normalisation in distributed architectures suffers from problems of distributed joins, requiring iterative network calls.

We’ve developed a mechanism for managing normalisation based on a variant of the Star Schema model used in data warehousing. In our implementation Facts are held distributed (partitioned) in the data nodes and Dimensions are replicated throughout the query-processing nodes. To save space we track ‘used’, or as we term them ‘connected’ data, to ensure only useful objects are replicated.

This model was presented at the QCon 2011 and at the Coherence SIG.

You can find the slides here (Powerpoint – 7MB).

See Also:

Posted on January 27th, 2011 in Coherence, Talks


  1. Ashwin Jayaprakash June 16th, 2011
    23:58 GMT

    Nice presentation… especially about handling complex graphs using the snowflake schema.

  2. ben June 17th, 2011
    6:13 GMT

    Thanks Ashwin. Appreciated. QCon actually published the slides today with me rambling on in the background! I’m not sure if it’s a good thing or not though. http://www.infoq.com/presentations/ODC-Beyond-The-Data-Grid

  3. Vasily Stepanov October 11th, 2011
    10:16 GMT

    Could you update links to the slides, because both of them are dead? I attended your presentation at JavaOne. and It was very interesting.

  4. ben October 11th, 2011
    15:09 GMT

    Hi Vasily

    I’m very pleased you found it useful. I’ve updated the links on this page so you should now be able to download the ppt or pdf. Sorry they are so large. I’ve also pointed to the version I presented @JavaOne which is slightly different to this one.

    B

  5. Lyndon Adams November 29th, 2011
    14:13 GMT

    Brilliant work. I enjoy reading all your work.

  6. ben November 29th, 2011
    15:21 GMT

    Thanks Lyndon. Appreciated!

    B

Have your say

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>




Safari hates me
IMPORTANT! To be able to proceed, you need to solve the following simple problem (so we know that you are a human) :-)

Add the numbers ( 11 + 14 ) and SUBTRACT two ?
Please leave these two fields as-is: