CoherenceRx: Reactive Extensions for Oracle Coherence

We are excited to announce the release of the CoherenceRx open source project which provides a reactive API for Coherence. It utilizes the Coherence Async API (requiring Coherence 12.2.1) and is based upon the RxJava Library.

There is an official Announcement on the Coherence Blog and you can take a look at the CoherenceRx project and examples on GitHub.

For more about reactive see reactivex.io and RxJava.

 

 

Posted in News | Tagged , , , | Leave a comment

Coherence 12.2.1 Cluster Discovery Made Easy

Introduction

In Coherence 12.2.1 a number of changes were made to make working with ports in Coherence easier. These changes allow multiple clusters (including different versions from 12.2.1 on), to share the same cluster port, and in most cases ports and addresses do not need to be explicitly set. This makes cluster discovery easier for both TCMP and Coherence*Extend clients.

We will first summarise these changes and then go through some practical examples.

New Features

  1. Coherence now includes a common cluster port that is used for multicast, WKA discovery as well as proxy service discovery via the names service.
  2. Unicast ports (for each member) use automatically assigned ephemeral ports by default.
  3. Address resolution has been simplified to help with cluster discovery. Coherence automatcially selects the IP which is routable to the IP’s on the WKA list.  We also have the ability to store the full WKA list under a single DNS entry.

Coherence now has a registered port number, 7574, with the Internet Assigned Numbers Authority (IANA). (See here for the registration.) This means that other products/ processes “should” not use this port.  Since multiple Coherence clusters can share this port then there should be no reason to ever change the value.

Cluster names are the preferred mechanism to distinguish clusters, and having multiple clusters share a port adds no overhead once you’ve discovered the cluster.

Example

Lets see this in action. I have a 12.2.1.0.1 installation on my Mac. We do have OOTB scripts to start cache servers, etc. but i’m going to do this by interacting directly with the jar files so we can see it working.

Change to your lib directory in your Coherence install and run the following. (From 12.2.1 onwards when you using -jar coherence.jar you get a Cache Server rather than the console!)

$ cd /u01/oracle/product/coherence1221/coherence/lib
$ java -jar coherence.jar

You should see the following in the logs indicating the (automatically generated) cluster name and the default cluster port. Also you can see the ephemeral port generated for the member is 56839 in this case.

... (thread=main, member=n/a): Started cluster Name=timmiddleton's cluster,
ClusterPort=7574
...
ThisMember=Member(Id=1, Timestamp=2016-05-03 11:56:53.071,
Address=X.X.X.X:56839, MachineId=20521, Location= ...)

Now that you can leave the cluster port as is, the cluster name becomes even more important in identifying a Coherence cluster. When I issue a netstat we can see that Coherence is listening on all interfaces on IPV4/6 and UDP as we have multicast enabled by default.

$ netstat -an | grep 7574
tcp6   0   0   xxx::aebc:32ff:.7574   *.* LISTEN
tcp6   0   0   xxxx::4c2:20ff:f.7574  *.* LISTEN
tcp6   0   0   xxxx::1%lo0.7574       *.* LISTEN
tcp6   0   0   xxxx:b400:c70:54.7574  *.* LISTEN
tcp4   0   0   127.0.0.1.7574         *.* LISTEN
tcp4   0   0   X.X.132.149.7574       *.* LISTEN
tcp4   0   0   X.0.0.55.7574          *.* LISTEN
tcp6   0   0   ::1.7574               *.* LISTEN
udp46  0   0   *.7574                 *.*

I can now start a second cache server but specify the cluster name with
-Dcoherence.cluster=TimsCluster. We can see that the clusters are sharing the cluster port but each using a separate ephemeral member port.

$ java -Dcoherence.cluster=TimsCluster -jar coherence.jar
... Started cluster Name=TimsCluster, ClusterPort=7574
ThisMember=Member(Id=1, Timestamp=2016-05-03 12:40:47.186, Address=X.X.X.X:57943

Now if we run CohQL using the default cluster, you will see that the first terminal shows the cluster member has joined.

$ java -cp coherence.jar com.tangosol.coherence.dslquery.QueryPlus
CohQL: insert into 'test' key(1) value('one');
CohQL: select count() from 'test';
Results
1

Now restart CohQL and specify a cluster name of TimsCluster.

$ java -Dcoherence.cluster=TimsCluster -cp coherence.jar com.tangosol.coherence.dslquery.QueryPlus
CohQL: select count() from 'test';
Results 0

The result is zero as we have connected to a separate cluster by specifying the cluster name.

Extend Clients

Extend clients can query the name service on the known cluster port and ask for the address of a proxy server.  This has been possible since 12.1.2 but in 12.2.1 combined with the new example coherence-cache-config.xml file this has been made a lot easier.

Note: It is now recommended that if you require proxy clients, all cache servers also run the proxy service. This allows for better scalability of cluster data storage as well as aggregate client proxy bandwidth.

The new cache config has two things that help:

  1.  A proxy-scheme with no acceptor config means it will listen on an ephemeral port like members do.
      <proxy-scheme>
        <service-name>Proxy</service-name>
        <autostart>true</autostart>
      </proxy-scheme>
    
  2. A remote-cache-scheme, that has no initiator-config will lookup the required proxy service name by asking the name service. To reach the name service the proxy client must share the same operational configuration, and as a result the discovery information for the cluster. In our case it will connect to the name service on port 7574 and ask for the address of the proxy service.
      <remote-cache-scheme>
        <scheme-name>thin-remote</scheme-name>
        <service-name>RemoteCache</service-name>
        <proxy-service-name>Proxy</proxy-service-name>
      </remote-cache-scheme>
    

Another advantage of the changes that were made to the cache config in 12.2.1 is that you can specify -Dcoherence.client option which will change the way the client connects to the cluster.  You can choose either direct which means cluster client or remote which means extend client. Direct is the default. (You can also set -Dcoherence.profile to near for a near cache or thin for a normal client.)

The following, with using the default cache config, will ensure CohQL will connect as an extend client.

$ java -cp coherence.jar -Dcoherence.client=remote com.tangosol.coherence.dslquery.QueryPlus

Its worth extracting the cache config from coherence.jar and taking a look at how it works. Its a good model to utilize and extend. To do this issue:

jar xvf /path/to/coherence.jar coherence-cache-config.xml

Well Known Addresses (WKA)

Up to now i’ve been using the default discovery mechanism which is multicast. If this is not allowed in your environment you can utilize WKA which has also been made a lot easier in 12.2.1 in that it no longer contains ports, only IP addresses.  The cluster port also applies to WKA as well as multicast cluster, therefore you can just specify a list of hosts and a cluster name to discover your cluster.

Lets change to using WKA by specifying -Dcoherence.wka=127.0.0.1. 

$ java -Dcoherence.wka=127.0.0.1 -jar coherence.jar
...
Started cluster Name=timmiddleton's cluster, ClusterPort=7574
...
WellKnownAddressList(
 127.0.0.1
 )

You can see the the cluster port is still used, but now we see the well known address list. This is not a good WKA address as such because its only on the localhost, but you could of course specify your hostname instead.

In this example I have used a system property to define the WKA, but it is best practice to put this in the operational configuration and ensure all members use the same configuration.

Where the real power is here is that say you have clusters running on host1, host2, host3 and host4 you could add these entries to a DNS entry called production.servers and then connect to the cluster using the following:

# Connect to HRCluster
$ java -Dcoherence.wka=production.servers -Dcoherence.cluster=HR ...
# Connect to GLCluster
$ java -Dcoherence.wka=production.servers -Dcoherence.cluster=GL ...

Summary

The idea with all of these changes is to make configuring clusters simpler and to avoid hard coding ports which help with resolving port conflicts. Its also important to note that all existing “legacy” configurations will still work and there may be cases where you need to specify port ranges for firewall access. See Setting Up a Cluster in the Coherence Documentation.

The key here is that the cluster name is now very important and you should set it always!

Further Reading

A number of resources are available to help you understand the changes.

Enjoy

Posted in New Features, Uncategorized | Tagged , , , | Leave a comment

Run the WebLogic & Coherence Multitenant Examples

Introduction

As part of the WebLogic 12.2.1 release a new new feature called “WebLogic Server Multitenant” has been introduced.  I’ve taken a couple of paragraphs from the Oracle WebLogic Server Documentation as it explains it nicely.

Multitenancy in WebLogic Server provides a sharable infrastructure for use by multiple organizations. These organizations are a conceptual grouping of your own choosing, which you can think of as tenants. By allowing one domain to support multiple tenants, WebLogic Server MT improves density and achieves a more efficient use of resources while eliminating the hurdles typically present when trying to share multiple applications: runtime cross-application impact, security differences, data co-mingling, and administrative challenges.

WebLogic Server MT provides resource isolation within a domain partition, an administrative and runtime slice of a WebLogic domain that is dedicated to running application instances and related resources for a tenant. Domain partitions achieve greater density by allowing application instances and related resources to share the domain, WebLogic Server itself, the Java virtual machine, and the operating system while isolating tenant-specific application data, configuration, and runtime traffic.

Coherence, when used in conjunction with WebLogic Server is fully supported in a multitenant environment.  When you deploy GARs (Grid Archives) and applications using Coherence, the services started within the applications are isolated by domain partition allowing multiple tenants to run the same Coherence application, but have private data and services.

Coherence & WebLogic Server Multitenant Example

When we released Coherence and WebLogic 12.1.2 back in August 2013, I showed how to run the examples we shipped which demonstrate how to use the new “GAR” model. See here for the details.

The standard examples are still there but we have included an example with a fully fledged Coherence & WebLogic Server application in a multi tenant domain.

The rest of this post will walk you though, in varying levels of detail, how to install and run these examples.

STEP 1 – Download and install JDK and WebLogic Server Generic

Download the Generic installer from OTN as well as the supported JDK for your platform. At the time of writing this it is 1.8.0_51 or above.

After you have installed the JDK, go do the directory where you downloaded WebLogic Server. You should see a file named: fmw_12.2.1.0.0_wls_Disk1_1of1.zip.

Unzip this file which will create a file called fmw_12.2.1.0.0_wls.jar.

Follow the installation instructions in my original post here. When you have completed the install, remembering to choose “Complete Install”,  return to this post as some of the directories have changed slightly in this version.

STEP 2- Viewing the Examples Home Page

Now thats you’ve installed the JDK and WebLogic Server (with Examples:) ), we will take a look at the examples home page.  In my case I installed WebLogic into the following directory:

    c:\Oracle\Middleware\1221

Using your browser, open up the following file, (replacing the WebLogic Server home with yours)

    c:\Oracle\Middleware\1221\wlserver\samples\server\index.html

The page should look similar to the following:

examples_home_page

Examples Home Page

You can see the second example is the Multitenant example. If you click on the link you will see the instructions for running the example as shown below.

mt_instructions

Multitenant instructions

The instructions are quite detailed and explain how the example works.I’ve included a summary of how to run/ build below.

STEP 3 – Building and Start the Multitenant Domain

To build the domain, open a command prompt and issue the setWLSEnv.sh/cmd file for your environment.

E.g. for Windows

cd \Oracle\Middleware\1221\wlserver\server\bin
setWLSEnv.cmd

Linux/Mac

cd /u01/Middleware/1221/wlserver/server/bin
. ./setWLSEnv.sh

Once you have set your environment, change to the following directory:

 c:\Oracle\Middleware\1221\wlserver\samples\server\examples\src\examples\coherence\managed-coherence-servers\multi-tenant

Once here, you can issue the following command to build the domain. You will be asked a few questions. Be sure to check your hostname is correct, otherwise the build may not succeed. You can just press enter to leave the ports as default.

ant deploy

The output should be similar to below:

ant_deploy

ant deploy

At the end of the build, you will see instructions showing you how to run the sample applications and get the the WebLogic Server Admin Console.

build_complete

Build complete

STEP 4 – Run The Applications

There are two demonstration applications deployed.

Contact Application
This application shows the same contact application we shipped with previous versions, deployed unchanged in a multi tenant environment. The data for each for the domain partitions (tenants) is totally isolated. You can go to either of the URLs below and see the different data:

In the example above we are using URL prefixes for convenience to determine the different domain partitions, company1 and company2, but in normal environments you could use Virtual Targets behind a load balancer and map them to http://company1.com/example-web-app and http://company2.com/example-web-app.

Stores Application
The stores application shows an example of sharing data between domain partitions in multi tenant application.

The concept of a “shared cache” between domain partitions is not enabled out of the box for obvious reasons but the administrator can configure this if required.

You can also go to the Admin console (URL in the output about) to see how this configuration is setup. A few sample screens are shown below.

domain_partition

Domain Partition Setup

deployments

Deployments

STEP 5 – Stopping the Domain

You can issue the following to stop the enter WebLogic Server domain:

    ant shutdownMultiTenant

Conclusion

“WebLogic Server Multitenant” is an exciting new feature and is a great way to achieve more density and better utilise your WebLogic Server environments and is fully supported by Coherence applications.

For more information on configuring Coherence in a multitenant environment see the WebLogic Server / Coherence Documentation.

 

 

 

 

Posted in Examples | Tagged , , , | Leave a comment

Oracle Coherence on Docker

Happy new year to all!

Its going to be a busy year and I’m planning on doing a number of posts early this year to go into more depth into new 12.2.1 features. Stay tuned for these.

In the meantime, one of our Coherence Engineers has just published a great post outlining how to run Coherence on Docker.   See http://thegridman.com/coherence/oracle-coherence-on-docker for the full post.

JK covers building Coherence Docker images, networking considerations including Coherence*Extend, JMX management, Persistence and Federation.

Well worth a read.

 

Posted in Related Posts, Uncategorized | Tagged , | Leave a comment

Running the Coherence 12.2.1 Persistence Examples

In this post I’ll show you how to get started with the new Coherence Persistence feature in 12.2.1 by working through the examples that ship with the product.

Firstly a quick overview of what Persistence is about.

Persistence Introduction

Persistence is a set of tools and technologies that manage the persistence and recovery of Coherence distributed caches. Cached data is persisted so that it can be quickly recovered after a catastrophic failure or after a cluster restart due to planned maintenance. Persistence and federated caching can be used together as required.

Persistence can operate in two modes:

  • On-Demand persistence mode – a cache is manually persisted and recovered upon request using the snapshot manager. The snapshot manager is exposed as an MBean interface that provides operations for creating, archiving, and recovering snapshots of a cache.
  • Active persistence mode – In this mode, cache contents are automatically persisted on all mutations and are automatically recovered on cluster/service startup. The snapshot manager can be used in active persistence mode to perform on-demand snapshots of a cache.

See the the Online Documentation or our Persistence YouTube Screencast for more information.

Installation

Follow the instructions in my post from last month on how to install Coherence with the examples.  Once you have done this, then continue on to run the Persistence examples.

Setting your Environment 

Now that you have completed the installation above you need to open a command prompt or terminal window and setup your environment with the following environment variables:

  • JAVA_HOME  – points to the directory you install the JDK
  • COHERENCE_HOME – points to the base directory you installed Coherence to

Note: If you are running Coherence 12.2.1.0.1 or greater, you will not need to set the COHERENCE_HOME variable.

Open a terminal and change to the examples/java directory under your Coherence home.

E.g. for Windows: (assuming you installed into c:\Oracle\Middleware\1221)

SET JAVA_HOME=c:\jdk\jdk1.8.0_65
cd \Oracle\Middleware\1221\coherence\examples\java
set COHERENCE_HOME=%cd%\..\..

Or for *NIX / Mac (using bash)

SET JAVA_HOME=/u01/jdk1.8/
cd /u01/product/coherence1221/coherence/examples/java
export COHERENCE_HOME=`pwd`/../..

Note: You don’t need to set JAVA_HOME on Mac.

Next run the following to check you have everything setup correctly:

bin\build contacts
bin\build persistence

You should see something like the following:

building contacts from com/tangosol/examples/contacts ...
Please refer to readme.txt for instructions on how to run the contacts example

If you don’t, check that you have JAVA_HOME and COHERENCE_HOME set correctly.

Hint for Windows users: Once you have completed the above, just type start in a command window and press enter. This will open another window with the same environment set. Do this a couple of times as we will need a number of command prompts.

Running the Examples

There are a number of different examples you can run to get a feel for how persistence works. Take a look in the readme.txt and search for ‘Persistence example’. The examples you can run are:

  1. Basic snapshot operations – shows how to issue Persistence Operations from Java
  2. Display Persistence notifications – shows how to use JMX Notifications to subscribe and display Persistence operations
  3. Running Persistence operations in parallel
  4. Advanced example – There are also SFTP snapshot archiver advanced example. (Leave that until you understand the concepts)

The readme.txt explains the steps to run each example, but I have included the Basic Snapshot operations example below for convenience.

The basic example shows how to call snapshot operations from Java.
Persistence is configured to store data in the ‘persistence-data’ directory under java examples home. Refer to resource/persistence/tangosol-coherence-override.xml for details of the defined persistence-environment.

First, build the persistence example using the following:
(Only required once, Ignore if you already did this above)

bin\build contacts
bin\build persistence

Next, start one or more cache servers using in separate terminals using:

bin\run-cache-server persistence

Next, in a separate terminal run the persistence example using:

bin\run persistence

The example will run and output similar to the following will be displayed:

------BasicSnapshotOperations example begin------

------Initial data size is 0------

------create new snapshots------
      create new snapshot snapshot-empty
      snapshots = [snapshot-empty]

------Populated data size is 10000------
      create new snapshot snapshot-10000
      snapshots = [snapshot-10000, snapshot-empty]

------recover snapshot------
      cleared cache, size = 0
      Recover snapshot snapshot-10000 containing 10000 entries
      cache size is now 10000
      Recover snapshot snapshot-empty containing 0 entries
      cache size is now 0

------removing snapshots------
      snapshots = []

------BasicSnapshotOperations example completed------

You can inspect the code in src/com/tangosol/examples/persistence directory. The Driver.java file is the initial class for all persistence examples.

You will also see messages on the run-cache-server terminal indicating that Persistence operations are underway.

Note: Don’t forget to run through the other examples!

Viewing Persistence Using JVisualVM Plugin

Its worth installing the JVisualVM Plugin and take a look at the Persistence tab as well as Caches tab while you are running the examples.  See my post in the new JVisualVM Plugin for 12.2.1 for more information.

This post goes into a bit more depth on the MBeans that are available for working with Persistence through JMX connections.

Conclusion

Persistence is a great new feature and the examples show you how to get started with it and some of the functionality available.

Take a look at the Online Documentation for Persistence and email me let me know if you are interested in a more in-depth guide to Persistence as i have a lab guide that I can share.

Enjoy

 

 

Posted in Examples | Tagged , , | Leave a comment

New Features in Coherence 12.2.1 JVisualVM Plug-in

Along with Coherence 12.2.1, we introduced a new version of the JVisualVM Plug-in supporting major new Coherence functionality such as Persistence and Federation, as well as including small incremental improvements.

Persistence

As per our release notes: “Coherence persistence is a set of tools and technologies that manage the persistence and recovery of Coherence distributed caches. Cached data is persisted so that it can be quickly recovered after a catastrophic failure or after a cluster restart due to planned maintenance.”

I’ll be doing an in-depth post on Persistence soon, but take a look at the Coherence Documentation for more information.

When you connect to a cluster that has either on-demand (the default), or active persistence enabled, you will see a new persistence tab displayed.

Persistence Tab

We can see we have one service, PartitionedCache, which has active Persistence. The majority of the columns in the table are used for active persistence and show usage and latency information as well as the number of snapshots saved as well as status information

Persistence Menu OptionAs is shown in the menu option to the right, you can right-click on a service and issue the Persistence operations listed. These operations call JMX operations via the PersistenceCoordinator MBean.

Note: A Snapshot is a consistent copy of an entire service and can be used to recover a service to a known state.  Snapshots can be archived to central locations as an entire set and can be validated for consistency.

We can also see graphs of active space used and information on additional latency that may have been incurred using active mode. All these statistics are available in raw form under the Service MBean.

Persistence operations generate standard JMX notifications and we have added support to easily display these in the plug-in on the “Notifications” tab. If I create and then recover a snapshot you can see the notifications being generated.

Notifications

Federation

Again, from the release notes: “Federated caching replicates and synchronizes cache data across multiple geographically dispersed clusters. Cached data is replicated across clusters to provide redundancy, off-site backup, and multiple points of access for application users in different geographical locations.”

Its a really exciting new feature that simplifies cross-cluster replication greatly. I’ve run the Federation demo, which is part of the OOTB Examples explained in this post, and connected to ClusterA via JVisualVM.

For more information on Federation see the Coherence Documentation.

Federation Tab

You can see that Federation details such as bandwidth utilization other statistics are displayed for the participants (clusters) in the topology. Various operations can also be carried out against the participants such as starting, pausing and resuming replication as well as forcing a replication of all data.

See the MBean reference for more details on the underlying Federation MBeans.

Other Updates to the Plug-in

Minor UI Update
In the upcoming 12.2.1.0.1 patch, you will see a minor update to the plugin UI to more correctly align it with the general JVisualVM plugins.

Report Node State
Report Node StateOn the Members Tab, you can issue a ‘Show Node State’, which will run a Thread Dump remotely for that node which can me used to help diagnose issues.
Details

Display Near Cache Details
When selecting a cache which is a near-cache, then information on the “near” portion will be displayed in the plug-in.

Near Cache

Hope you enjoy using the plugin and feel free to provide feedback and suggestions on how you use it and we can improve it.

Just as a reminder, when you install Coherence via the installer, the plug-in is available at $COHERENCE_HOME/coherence/plugins/jvisualvm/coherence-jvisualvm.nbm.

For details on how to install the plugin-in see the Documentation.

Posted in New Features | Tagged , , | Leave a comment

Coherence / WebLogic Server 12.2.1 Workshop now in Adelaide, Melbourne & Sydney

Just a quick update to let you know the Free Coherence/WebLogic Server 12.2.1 hands-on workshop will also be in the following Australia Cities on the dates below:

  • Adelaide – Tuesday 1st December
  • Melbourne – Wednesday 2nd December
  • Sydney – Thursday 3rd December

The workshop is primarily focused on Coherence but we will cover some WebLogic Server new features in general and as they relate to Coherence.

Please contact me at tim dot middleton at oracle dot com for registration details for any of the above cities.

The summary of the workshop is below.

Workshop Overview

Bringing, their own laptops, attendees will have hands-on experience with the latest release of Coherence 12.2.1 and WebLogic Server 12.2.1 by exploring various examples and hands-on demonstrations including:

  • Data access, data loading, Java8, Lambda, Streams, Generics, Security
  • Data operations including: querying, aggregating, listening and processing
  • Federated Caching for continuous available across sites
  • Cache Persistence for automated recovery and cluster snapshot creation
  • Accessing Coherence via REST though a AngularJS application
  • Integrating Coherence with WebLogic Server 12cR2 Multi-tenancy, Coherence Grid-Archives (GAR) and issuing Rolling redeploy
  • Using the Coherence JVisualVM Plug-in to monitor Coherence Development environments
Posted in Workshops | Tagged , , , | Leave a comment