Using Coherence for Reference Data in ADF

Introduction

I’m always trying to think of different ways Coherence can  benefit when developing applications. One of the more unusual and useful ways my colleague and I came up with was to use Coherence, rather than a database, to store reference data for ADF based applications. Doing this means you can get extremely fast, in memory access, to the data you need to display in your ADF apps, therefore increasing performance and scalability.

Oracle ADF is an Application Development Framework from Oracle, which allows you to develop web and mobile-based applications quickly and easily using a rich set of components. Part of ADF is the concept of a data control, which can be attached to components to display or process data. Data controls can be based upon sources such as web services, EJB’s, RESTful web services, JBO/BC4J, etc.

To see how this would help, consider the example of an ADF application querying country codes from the database. If you had these in the DB, you could use in-built controls to access that data via a BC4J, which is fine, but every ADF page that accesses this data will incur overhead of DB access, O/R mapping, etc. Image it all being in memory and accessible very quickly providing a more scalable platform for heavy reference data access. Also, what if you wanted a fast Google-like autosuggest? Getting data for this from the database as you type would not be a good idea!

I’m going to assume a bit of ADF and Java knowledge here for brevity. For my example I’m using JDeveloper 11.1.1.6.

Creating the ADF Project
First create a new Application and choose Fusion Web Applications (ADF). Take the defaults and finish.

Create ADF Project

Make sure you add coherence.jar to your project libraries for the Model project. You will find this in the following location: <Middleware-home>\oracle_common\modules\oracle.coherence\coherence.jar. You can use Coherence for evaluation purposes by accepting the OTN license agreement) from http://www.oracle.com/technetwork/middleware/coherence/downloads/index.html.

Create the Data Control

Now we’re going to create a JavaBean class to hold our country details. On the empty Model project, right-click and select New. Choose Java->Java Class. Name this Country and click on OK.

Add the following attributes:

//
private String shortCode;
private String description;

We then use JDeveloper to automatically generate the required code for the JavaBean.   Under the Source menu, choose Generate Accessors, Generate equals() and hashCode(), and Generate Constructors from Fields. Choose both fields. Also make sure this object implements java.io.Serializable  for Coherence to store. For sorting, implement the following inner class:

//
public static class DescriptionComparator implements Comparator<Country> {
        @Override

        public int compare(Country o1, Country o2) {
            return o1.getDescription().compareTo(o2.getDescription());
        }
}

Now we’ve created this, we want to create a class in the same Model project that we can expose as a Data Control. We will call this CountryLister.java.  See the code below:

//
import com.tangosol.net.CacheFactory;

import com.tangosol.net.NamedCache;

import com.tangosol.util.filter.AlwaysFilter;
import com.tangosol.util.filter.LikeFilter;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Set;

/**
 * Class to be deployed as data control
 */
public class CountryLister {

   public CountryLister() {
    }

    // handle to the cache
    private static NamedCache nc;

    // cache name
    public static final String CACHENAME = "Country";

    public List<Country> findAllCountries() {
        List countries = new ArrayList();
        long start, end;
        start = System.currentTimeMillis();

        Iterator it =
            nc.entrySet(AlwaysFilter.INSTANCE, new Country.DescriptionComparator()).iterator();
        while (it.hasNext()) {
            Map.Entry result = (Map.Entry)it.next();
            countries.add((Country)result.getValue());
        }
        end = System.currentTimeMillis();
        System.out.println("*** Total time was: " + (end - start) +
                           " ms for findAll");

        return countries;
    }

    /**
     * Returns all countries that match a like filter.
     * Note: Depending upon the size of the data in Coherence you may need an index.
     *
     * @param name the filter
     * @return the array that matches
     */
    public  List findCountriesByDescription( String description) {
        List countries = new ArrayList();
        long start, end;
        String pattern = "%" + (description != null ? description : "") + "%";
        // could be more efficient to not use % at beginning and adding index
        LikeFilter filter = new LikeFilter("getDescription", pattern, true);

        start = System.currentTimeMillis();
        // get all countries and add them

        Iterator it =
            nc.entrySet(filter, new Country.DescriptionComparator()).iterator();
        while (it.hasNext()) {
            Map.Entry result = (Map.Entry)it.next();
            countries.add((Country)result.getValue());
        }
        end = System.currentTimeMillis();
        System.out.println("*** Total time was: " + (end - start) +
                           " ms for string: " + pattern);

        return countries;
    }

    static {
     // initialze as storage disabled
      System.setProperty("tangosol.coherence.distributed.localstorage", "false");

      // Following four lines only required if multicast doesn't work
      //System.setProperty("tangosol.coherence.localhost", "localhost");
      //System.setProperty("tangosol.coherence.ttl","0");
      //System.setProperty("tangosol.coherence.wka", "localhost");
      //System.setProperty("tangosol.coherence.wka.port", "8088");

      nc = CacheFactory.getCache(CACHENAME);
    }
}

We will also need a class to start our cache server – StartCacheServer.java. In this case I’ve just referenced our cache through Cachefactory.getCache() (which will start a cache server by default) and loaded the data in directly in one operation. There are better ways to do this, but just taking the easier path today!. Create this in the Model project.

//
import com.tangosol.net.CacheFactory;
import com.tangosol.net.NamedCache;

import com.tangosol.util.extractor.ReflectionExtractor;

import java.util.HashMap;
import java.util.Map;

/**
 * Start a coherence cache server process and load data.
 * Normally you separate the cache servers and data loading processes.
 * But just done this the quick and easy way for the moment.
 */
public class StartCacheServer {
    public static void main(String[] args) {
        // Following four lines only required if multicast doesn't work
        //System.setProperty("tangosol.coherence.localhost", "localhost");
        //System.setProperty("tangosol.coherence.ttl","0");
        //System.setProperty("tangosol.coherence.wka", "localhost");
        //System.setProperty("tangosol.coherence.wka.port", "8088");

        CacheFactory.ensureCluster();
        NamedCache nc = CacheFactory.getCache(CountryLister.CACHENAME);
        Map buffer = new HashMap();
        System.out.println("Cache Started, Loading data");

        if (nc.size() == 0) {
           nc.putAll(loadBuffer());
        }

        System.out.println("Data loaded. Size is " + nc.size());

        try {
            Thread.sleep(Long.MAX_VALUE);
        } catch (InterruptedException e) {
        }
    }

     private static Map loadBuffer() {
        Map buffer = new HashMap();
        buffer.put("AD",new Country("AD","Andorra"));
        buffer.put("AE",new Country("AE","United"));
        buffer.put("AF",new Country("AF","Afghanistan"));
        buffer.put("AG",new Country("AG","Antigua"));
        buffer.put("AI",new Country("AI","Anguilla"));
        buffer.put("AL",new Country("AL","Albania"));
        buffer.put("AM",new Country("AM","Armenia"));
        buffer.put("AN",new Country("AN","Netherlands"));
        buffer.put("AO",new Country("AO","Angola"));
        buffer.put("AQ",new Country("AQ","Antarctica"));
        buffer.put("AR",new Country("AR","Argentina"));
        buffer.put("AS",new Country("AS","American"));
        buffer.put("AT",new Country("AT","Austria"));
        buffer.put("AU",new Country("AU","Australia"));

        // you get the idea.  I’ve left out many of these.

        buffer.put("ZM",new Country("ZM","Zambia"));
        buffer.put("ZR",new Country("ZR","Zaire"));
        buffer.put("ZW",new Country("ZW","Zimbabwe"));

        return buffer;

     }
}

You can download country codes from http://www.iso.org/iso/list-en1-semic-3.txt and then use the following awk script to create the data.

cat country_codes.txt | awk -F\; \
'{ printf "buffer.put(\"%s\", new Country(\"%s\",\"%s\"));\r\n",$2,$2,$1 ;'}

Now we will create a Data Control. Select the CountryLister.java file and right-click and choose Create Data Control. Once this is completed it should look something like this.

Application Navigator

Use the Data Control to create a Country List

Next we are going to create a page that will use this data control. In Application Navigator, select the View Controller project, right-click and choose New -> Web Tier -> JSF -> JSF Page. Name it as CountryList and ensure you check Create as XML Document.

Create JSF Page

Drag a Panel Stretch Layout from the Layout panel from the component palette. In the Data Controls panel, expand CountryLister and then expand findAllCountries(). Drag Country to the center facet and choose Table->ADF Read Only Table. Check Enable Sorting. Save All.

Now before we run, we need to startup our cache server. In your model project choose StartCacheServer.java, right-click and Run. Once that starts, select CountryList.jspx, right-click and Run.

What you should see is something like the following. I know, not amazing looking (thats for you to make nicer!) but every time you access this data control data is coming from Coherence so its fast! (see the timings, without indexes in the JDeveloper Log)
Have a think how you could apply this data control to other components such as drop down boxes, etc.

Country List

If you get a message like “No storage-enabled nodes exist” when you try to run, it is likely that your machine doesn’t support multicast.  You can un-comment  the following in your StartCacheServer.java and  static code in CountryLister.java to use Well Known Addresses (WKA).

        System.setProperty("tangosol.coherence.localhost", "localhost");
        System.setProperty("tangosol.coherence.ttl","0");
        System.setProperty("tangosol.coherence.wka", "localhost");
        System.setProperty("tangosol.coherence.wka.port", "8088");

Linking the Data Control to the Auto-Suggest ADF feature

The last part is to link this data control into the (Google-like) Auto Suggest feature of ADF so we can start typing and see a list of suggested countries.  For this I need to do a couple of things:

  1. Create a new JSF page called AutoSuggest.jspx.
  2. Drag a Panel Stretch Layout to this page.
  3. Drag an Input Text common component to the center panel of the page.
  4. Drag an Auto Suggest Behavior Operation into the newly created Input Text. (From the Operations concertina menu on the right)
  5. Click on the SuggestedItems dropdown (not SuggestItems), in the properties for the autoSuggestBehavior and click Edit.
    SuggestedItems
  6. Click New to create a new ManagedBean and use the following settings. (Probably worth setting Scope to something more appropriate like Session)
    Create Managed Bean
  7. Click New for method and call it getAutoSuggest.
  8. Use the following code for the newly generated AutoSuggestBean.java:
    // include your imports for Country and CountryLister
    
    import java.util.ArrayList;
    import java.util.List;
    
    import javax.faces.model.SelectItem;
    
    public class AutoSuggestBean {
        private CountryLister cl;
    
        public AutoSuggestBean() {
            cl = new CountryLister();
        }
    
        public List getAutoSuggest(String string) {
    
            List<Country> suggestedList =
                cl.findCountriesByDescription(string);
    
            // now convert to a list of SelectItems
            List items = new ArrayList();
            for (Country country : suggestedList) {
                items.add(new SelectItem(country.getDescription()));
            }
            return items;
        }
    
    }
    
  9.  That’s it!  Now when we run this, as we type we can see suggestions of countries that match.  I’m using a LikeFilter in Coherence and putting % around the values I type, but you could as easily only look for values that start with what you type.

Auto Suggest List

In the example above, we have used storage-disabled clients, which allow us to separate the data access from the cache servers. It enables us to scale out the cache servers to provide more data capacity and processing capacity. The nice thing about Coherence is that when you are querying the data, it will do this in parallel across all available cache servers!

There are many extra things to consider before putting this into production including (but not limited to):

  1. Adding indexes to improve performance (from the 20ms to get countries as I typed!),
  2. Implementing near caching getting even faster in memory access speed;
  3. Using a proper cache server and cache config; and
  4. Using EvolvablePortableObjects for fast serialization and allowing schema evolution of objects.

Thanks to Andrew Rosson for his original thoughts around this and Chris Muir for some ADF help.

About these ads
This entry was posted in Examples and tagged , . Bookmark the permalink.

3 Responses to Using Coherence for Reference Data in ADF

  1. Tapash Ray says:

    ‘No storage-enabled nodes exist” should not be related to unicast or multicast. Coherence works fine on both, multicast is recommended as Coherence relies heavily on network protocols, especially for large clusters.
    You will get this error because you set the localstorage to false for the cluster node. For this kind of implementation, where you want the cluster node to act as cache nodes, you would want the localstorage to be set to true and for nodes that you want to use as TCP extend nodes, which don’t store data, you want to set this to false.

    • Thanks for your comment. You are right it what you say, but i’ve seen on many machines, mainly laptop’s in workshops, that multicast is disabled by default. (Particularly on some corporate machines with standard environment installed – SOE).
      If you try to create a cluster using multicast in these situations, the processes don’t see each other, from the cluster perspective, and you can get ‘no storage-enabled nodes exist’.
      Tim

      • Tapash Ray says:

        If I remember it right from my experience with Coherence last year, there is a multicast tester script that Coherence provides out of box. This can be used to test if multicast is enabled on the network.
        Also, isn’t it that multicast/unicast come into play when there are cluster nodes distributed across multiple physical machines across the network ? On a single machine, it would not matter much (or at all), just that the cluster will not be machine-safe.
        On this post, there is another thing to note – it is good that you are using a POJO class, wouldn’t it be better to use a dynamically created ViewObject with a DB query, accompanied with read-through caching.
        Thanks
        Tapash

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s