Managing Versioning
This is the first in a series of posts describing some useful patterns for implementing Coherence data grids.
Most non-trivial caches need to version their objects. There are number of reasons for wanting this:
- Versioning provides a historic record of changes.
- By linking versioning with the wall-clock / business times (i.e. bi-temporal) views of the system at previous points in time can be recomponsed. This is important for providing consistent views over your data.
- Versioning allows concurrency to be managed through Multi-Version Concurrency Control (MVCC)
However simply adding versions to your objects (more precisely your object key) has the downside that you can no longer look up the value via it’s business key: you must know the business key as well as the version of the object that you want.
Key = [Business Key][Version]In Coherence accessing objects via their key directly is far more performant than doing a query (see The Fallacy of Linear Scalability) so it is preferable to keep the latest version of the object available via its business key alone. There are two common approaches to solving this problem: The Latest/Versioned pattern and the Latest Version Marker pattern.
Using Latest and Versioned Caches
The first approach is to define two caches for every object. The Latest… cache and the Versioned… cache. The key of the ‘latest’ cache is simply the business key:
Latest Cache Key = [Business Key]This cache only ever contains the latest object. The ‘versioned’ cache contains all versions of the object with a, usually monotonically incrementing version embedded in the key:
Versioned Cache Key = [Business Key][Version]Writes must be directed at the ‘Latest’ cache and a Coherence Trigger is used to copy the object reference to the ‘Versioned’ cache adding the version onto the key as it does so. This is demonstrated in the first figure opposite.
The disadvantage of this approach is a memory inefficiency arising because the latest object exists in both Latest and Versioned caches. When the object is written the same reference can be used to save space, however the backup copies in each cache will be different instances and, should a node be lost, and process of recreating the primary from the backup copy will create new instances by default further eating memory. It is therefore advisable to use the LatestMarker pattern below when memory is a concern. The advantage of this approach is that it reduces the number of records in the latest caches which makes filter operations faster when they operate only on ‘Latest’ data (a common use case in most applications).
Checklist:
- Define two cache schemes based on the masks Latest* and Versioned* ensuring that they are in the same CacheService.
- In the Latest* scheme specify a trigger to forward objects to the versioned cache, incrementing the version as it does so.
- Specify KeyAssociation (Affinity) on the business key of the Latest* cache across both caches.
- Write a trigger that adds a monotomically incrementing version to the business key as it copies the value’s reference to the Versioned cache. You’ll need to use direct backing map access to avoid reentrancy problems (I’ve discussed the issues of reentrancy in Coherence before. See Merging Data And Processing: Why it doesn’t “just work”). The code sample below is provided for reference.
Using Versioned Cache Only With a Latest Version Marker
A second approach to solving the same problem is to only use a single cache with the key format:
Key = [Business Key][Version]but specifying that the latest version of an object has a special version marker:
KeyLatest = [Business Key][LatestVersionMarker]As clients are aware of the LatestVersionMarker (for example -1 is common) they can always access the latest value directly by calling:
cache.get([businessKey][-1])This approach does not suffer from the issues of duplication associated with separate Latest and Versioned caches but has the disadvantage that versioned data is in the same cache as latest data, marginally slowing down filters. Just reiterating that again: in this pattern there is only one copy of the latest object. The one with the latest marker. This is different to the latest/versioned pattern where the latest object will exist in both caches (so twice) so that the versioned cache can contain all versions of that object.
Checklist:
- Create a cache with a KeyAssociation on the business key (i.e. the key parts without the version number). Add a trigger that replaces the current value for the “LatestMarker” with the new object whilst copying the old value to a key with the appropriate real version. You’ll need to use direct backing map access to avoid reentrancy problems (I’ve discussed the issues of reentrancy in Coherence before [link]). See code sample below.
Implementing the trigger to avoid reentrancy issues
The below code outlines one mechanism for moving objects (in this case for the Latest/Versioned pattern) from one cache to the other using direct backing map access.
public void copyObjectToVersionedCacheAddingVersion(MapTrigger.Entry entry) {
// I'm assuming that you are tracking the version, and incrementing it, in your object
// Also note that it's more efficient to just take the version out rather than deserialise
// the whole object but this way is more succinct
MyValue value = (MyValue)entry.getValue();
MyKey versionedKey = (MyKey)value.getKey();
BinaryEntry binaryEntry = (BinaryEntry)entry;
Binary binaryValue = binaryEntry.getBinaryValue();
Map versionedCacheBackingMap = binaryEntry.getContext().getBackingMap("VersionedCacheName");
versionedCacheBackingMap.put(toBinary(versionedKey), binaryValue);
}
Summary
Both approaches provide direct access to latest objects (most importantly key-based access to the most commonly used ‘latest’ version) without requiring knowledge of the version itself. Both retain a history of versions, something that is important for locking, MVCC and snapshotting (I’ll be writing more about these later). Affinity (Key Association) is used to ensure that the versioning process is entirely local to the JVM doing the write.
Related Posts
8 Comments
Jump to comment form | comments rss | trackback uri