Evolving Your Cache Objects Online

Introduction
Oracle Coherence provides the ability to evolve your data objects over time, without an outage in your data grid, or affecting applications that access the data in the grid. Essentially, multiple versions of a particular cache object, say a Customer, can exist in a data grid at the same time while you perform an upgrade. Also multiple clients that understand the different versions, can access this data as well.

This does require a little bit of forethought and planning, but its well worth it. Using the Evolvable interface along with the PortableObject interface plus a methodical approach will get us there.

If you Google “using evolvable portable objects”, you can see a number of great posts about this feature already, but what I wanted to do is explain how you could do this with a simple working example.

The Scenario
In my example I will have a Customer object that starts off with four attributes: customerId, name, region and creditLimit. What we want to do is to upgrade the application, without an application or grid outage, to include loyalty card functionality which requires an additional attribute (loyaltyCard) on the Customer object.
(This is a simple change, but the concepts can be applied to much more complex changes!)

Person object Evolution

To prepare for this scenario our objects in the grid must already :

  1. Implement the Evolvable interface via extending the AbstractEvolvable abstract class.
  2. Implement getImplVersion() (from above abstract class), to specify the evolving versions of objects.

I’m also using the new @Portable annotation to make them use the Portable Object Format (POF) serialisation. You could just use Serializable, but its good practice to get used to using POF as its smaller, faster and portable across Java,.NET and C++.

Here is a snippet from the V1 Person Object.

@Portable
public class Customer
        extends AbstractEvolvable
     {
     public Customer(int customerId, String name, String region,
			float creditLimit) {
		super();
		this.customerId = customerId;
		this.name = name;
		this.region = region;
		this.creditLimit = creditLimit;
	}

    // setters/getters/equals omitted

    @Override
    public int getImplVersion()
        {
        return VERSION;
        }

    /**
     * Version identifier for Evovlable.
     */
    public static final int VERSION = 1;

    public static final int CUSTOMER_ID = 1;
    public static final int NAME = 2;
    public static final int REGION = 3;
    public static final int CREDIT_LIMIT = 4;

    public static final String CACHE_NAME = "Customer";

    @PortableProperty(CUSTOMER_ID)
    private int    customerId;

    @PortableProperty(NAME)
    private String name;

    @PortableProperty(REGION)
    private String region;

    @PortableProperty(CREDIT_LIMIT)
    private float creditLimit;
    }

Here is a snippet from the V2 Person Object.

@Portable
public class Customer
        extends AbstractEvolvable
     {
     public Customer(int customerId, String name, String region, float creditLimit, String loyaltyCard)
        {
        super();
        this.customerId  = customerId;
        this.name        = name;
        this.region      = region;
        this.creditLimit = creditLimit;
        this.loyaltyCard = loyaltyCard;
        }

    // setters/getters/equals omitted

    @Override
    public int getImplVersion()
        {
        return VERSION;
        }

    /**
     * Version identifier for Evovlable.
     */
    public static final int VERSION = 2;

    public static final int CUSTOMER_ID = 1;
    public static final int NAME = 2;
    public static final int REGION = 3;
    public static final int CREDIT_LIMIT = 4;
    public static final int LOYALTY_CARD = 5;

    public static final String CACHE_NAME = "Customer";

    @PortableProperty(CUSTOMER_ID)
    private int    customerId;

    @PortableProperty(NAME)
    private String name;

    @PortableProperty(REGION)
    private String region;

    @PortableProperty(CREDIT_LIMIT)
    private float  creditLimit;

    @PortableProperty(LOYALTY_CARD)
    private String loyaltyCard;
    }

The process we are going to use to execute this is:

  1. Setup two different directory structures (or jars), one with the v1 classes and the other with the v2 classes.
  2. One by one, we shutdown the cache servers running v1 code (checking StatusHA in between) and start them up using the new v2 classes. This will update the implementation class version.
  3. When all cache servers are running with v2 code we are able to run v2 clients against the data grid. As v2 clients access data and deserialize and serialize, they will save data in the new data version.
  4. During this time, both old clients and new clients can access the data, with older clients not aware of the new atrributes, but ensuring any “future data” (that was set by v2 clients) they don’t know about is preserved.
  5. When all objects have both the data version and implementation version set to v2, our upgrade is complete and we can then retire our v1 clients.

The diagram below outlines this approach.

Data Version and Impl Version
It is important to note that even when all cache servers are running v2 of the Person class (Impl version), that only until the data is deserialized and then serialized will the data version be updated to equal the Impl version. Normal partition transfers are binary, so they don’t result in data version changes. In my example, after we have upgraded all cache servers to v2 we will run some Entry Processors (EP) that will modify data and as part of the process upgrade the data version to match the Impl version.

The really cool thing is that we can also still run an EP that only understands v1 classes and data, and the v2 data (with extra loyaltyCard attribute) will be maintained and the data versions upgraded to v2.
Enough explaining, lets see this in action.

The Example
I’ll post a link to the example source code below, but effectively i’m using ant for all cache servers and clients. This allows it to be run on any O/S that supports ant and Coherence. Just remember to set the COHERENCE_HOME environment variable before you run ant.

If you run ant without any arguments from the directory with build.xml in, you will see the following options:

help:
[echo] Valid Targets are:
[echo] clean - clean
[echo] build - build the source code
[echo] cache-server-v1 - Run a cache server with v1 classes
[echo] cache-server-v2 - Run a cache server with v2 classes
[echo] populate-cache - Populate the cache with 100 v1 Customer objects
[echo] customer-count - Count count of customer objects
[echo] customer-region-update - Update credit limit by region
[echo] customer-card-update - Update loyalty card by region

Run ant clean build to build the environment.

To upgrade the cache servers, carry out the following

  1. Startup 3 cache servers all user v1 classes using ant cache-server-v1 in 3 different terminal windows.
  2. Next, in another terminal, run ant populate-cache which adds 10,000 objects into the cache using only V1 classes.
  3. When the populate finishes, run ant customer-count to startup a small GUI that shows the total customers and the different Data and Impl versions. You can see that ad the moment all the data is V1 and all the classes are V1.

    Initial Customer Counts

  4. Now we are going to shutdown each v1-cache server in turn and startup a v2-cache server in its place. In normal operation, you would check the StatusHA value to ensure that its safe to do the next cache server. E.g. Coherence has rebalanced partitions. (For more information in StatusHA, see here).
  5. Use CTRL-C to kill the first cache server. Wait a couple of seconds and then run ant cache-server-v2. Look at the GUI and you will see that approx 1/3 of the data is now in V2 Impl. This is because with 3 cache servers, the data is distributed across all cache servers evenly.

    Data Totals After One V2-Cach Server Started

  6. Now do the same for the second and third cache server terminals. Use CTRL-C, wait for a few seconds, start v2-cache server using ant cache-server-v2. Note the changes in the totals in between each shutdown and startup. The resulting GUI show now look like the following with all Impl versions now V2.

    Totals After All Cache Server V2

Now the cache servers are updated to V2 we need to upgrade all the data to V2. As mentioned previously, when the data is deserialized and serialized, it data version will be upgraded to match the Impl (class) version.

To do this, carry out the following

  1. On a separate terminal run ant customer-card-update. This will display a small GUI through which you can run an EntryProcessor (below) to update the new loyaltyCard attribute of customers of a particular region. (There are 25% of all customers in each region.)
    @Portable
    public class UpdateLoyaltyCardProcessor extends AbstractProcessor {
        @Override
        public Object process(Entry entry)
            {
            Customer c = (Customer)entry.getValue();
            c.setLoyaltyCard("CL" + c.getCustomerId());
            entry.setValue(c);
    
            return null;
            }
    }
    
  2. If you click the “Update North Region” button, you will see that when this completes, 25% of the data is now in V2 format as it was processed via the Entry Processor.

    North Region Updated

  3. Continue updating the other regions until all the data is converted to V2.

    All Data Converted to V2

  4. Now close the Loyalty Card Update GUI.

All the data is now V2 and all classes are now V2.

Lastly, say that we will have an old client that has not yet been upgraded to understand the new V2 classes. We can still run this old V1 client code against the V2 data without causing any problems.

To do this, carry out the following

  1. In the same terminal window run ant customer-region-update. This will display a small GUI through which you can run an EntryProcessor (below) to update a regions credit limit by 10%.
    @Portable
    public class UpdateCreditLimitProcessor
            extends AbstractProcessor
        {
        @Override
        public Object process(Entry entry)
            {
            Customer c = (Customer)entry.getValue();
            c.setCreditLimit(c.getCreditLimit() * 1.1f);
            entry.setValue(c);
    
            return null;
            }
        }
    
  2. When you click any of the buttons, the EntryProcessor will run against the V2 cache servers correctly, without causing any data loss.
  3. You could also run this EntryProcessor while there are V1 and V2 cache servers running.

Conclusion
Hopefully through this small example, you’ve now got an idea of how you can evolve your cache objects online. In my example I’ve just used standard in memory backing maps (cache storage). The same principles can be applied if your caches have read-write backing maps associated with them, e.g. read from database on miss, write to database via JPA etc. Again, there is a bit of discipline needed for this, but the rewards are great.

You can can download the code for this example here.

Advertisements
This entry was posted in Examples, Uncategorized and tagged . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s