Using the NameService in Coherence 12.1.2

Introduction

In this post I would like to talk about a new feature in Coherence 12.1.2 called the NameService.  The NameService provides a greater level of flexibility in the way that you can configure and use proxy servers when you use Coherence*Extend.
See the 12.1.2 documentation for more detailed information.

Configuring Coherence*Extend in pre-12.1.2 Environments
Before 12.1.2, to configure a “Proxy Server” (or to be pedantic, a Proxy Service), you would configure a proxy-scheme in your cache configuration.

  <proxy-scheme>
      <service-name>ExtendTcpProxyService</service-name>
      <thread-count>5</thread-count>
      <acceptor-config>
        <tcp-acceptor>
          <local-address>
            <address>198.168.1.5<</address>
            <port>9099</port>
          </local-address>
        </tcp-acceptor>
      </acceptor-config>
      <autostart>true</autostart>
    </proxy-scheme>

The above says we want to listen on address 198.168.1.5, port 9099 for incoming Coherence*Extend connections.

For a client you would then need to configure a remote-cache-scheme for your Coherence*Extend clients. An example is below:

<remote-cache-scheme>
  <scheme-name>extend-dist</scheme-name> 
  <service-name>ExtendTcpCacheService</service-name> 
  <initiator-config>
    <tcp-initiator> 
      <remote-addresses>
        <socket-address> 
          <address>198.168.1.5</address> 
          <port>9099</port>
        </socket-address> 
       </remote-addresses> 
       <connect-timeout>10s</connect-timeout>
    </tcp-initiator> 
    ...  
    </initiator-config> 
</remote-cache-scheme>

This configuration works fine, but it does mean we have to hard-code IP addresses in our server and client cache configuration files, as well as it makes it more difficult to add or remove proxy servers, without changing configuration.

Goals for the NameService
The NameService was introduced to help with some of the above challenges, and provide the following benefits in the way you configure Coherence*Extend:

  1. Remove addresses and ports from cache config for both the proxy-scheme and remote-* schemes.
  2. Provide the ability for multiple proxy services within a JVM be able to share a listen port.
  3. Support a large number of applications and proxies deployed to a single cluster node.
  4. Deploy proxies without a JVM restart.

 
Architecture
The NameService is a very light-weight service that runs on each node, irrespective of if there is a proxy server on that node. The NameService listens on ‘sub-port’ 3 of the unicast listen port. It is responsible for accepting requests from clients looking for a proxy service by name, and returning the address of the required proxy service.

The diagram below, outlines how the different components of the NameService work together.
NameService Architecture

When you startup a Coherence 12.1.2.0.1 cluster, (you need to use at least the first patch), you can see the name service starting up and listening on sub-port 3.

... ommitted ...
Group{Address=224.12.1.0, Port=12100, TTL=4}
MasterMemberSet(
  ThisMember=Member(Id=1, Timestamp=2014-03-21 11:59:11.573, Address=10.1.2.3:8088, MachineId=40577, 
Location=site:,machine:my-machine,process:55076, Role=CoherenceServer)
...
2014-03-21 11:59:14.899/4.588 Oracle Coherence GE 12.1.2.0.1 <Info> 
(thread=NameService:TcpAcceptor, member=1): TcpAcceptor now listening for connections on 10.1.2.3:8088.3

To define a proxy server that uses the NameService, you just need to provide an empty tcp-acceptor element. E.g. you don’t specify the listen port.

  <proxy-scheme>
    <service-name>ExtendTcpProxyService</service-name>
    <thread-count>5</thread-count>
    <acceptor-config>
      <!-- empty tcp-acceptor means to use the name service -->
      <tcp-acceptor/>      
    </acceptor-config>
    <autostart>true</autostart>
  </proxy-scheme>

When we startup a node with a proxy service defined, we will see the following messages:

  1. One indicating the NameService is started on sub-port 3 on the unicast listen address, and
  2. One indicating The proxy service starting up on an ephemeral sub-port.

The proxy service then registers its address with the NameService and is therefore known to all NameService members on all nodes.

2014-03-21 11:59:30.899/4.484 Oracle Coherence GE 12.1.2.0.1 <Info> (thread=NameService:TcpAcceptor, member=2): 
TcpAcceptor now listening for connections on 10.1.2.3:8090.3
...
2014-03-21 11:59:30.899/4.588 Oracle Coherence GE 12.1.2.0.1 <Info> (thread=Proxy:ExtendTcpProxyService:TcpAcceptor, member=2): 
TcpAcceptor now listening for connections on 10.1.2.3:8090.35545

The key thing is that we don’t actually need to know the address of the proxy server, just its service name.

Our client then defines the following remote-cache-scheme that defines the ‘proxy-service-name’ we want to connect to as well as an address-provider to find that address. The address provider is defined in the operational override file.

  <remote-cache-scheme>
    <scheme-name>extend-dist</scheme-name>
    <service-name>ExtendTcpCacheService</service-name>
    <!-- name of the remote proxy service to connect to -->
    <proxy-service-name>ExtendTcpProxyService</proxy-service-name>
    <initiator-config>
      <tcp-initiator>
        <name-service-addresses>
          <address-provider>extend-address-provider</address-provider>
        </name-service-addresses>
      </tcp-initiator>
      <outgoing-message-handler>
        <request-timeout>30s</request-timeout>
      </outgoing-message-handler>
    </initiator-config>
  </remote-cache-scheme>

The address provider effectively points to the unicast listen ports only, or WKA if you have defined this.

 <address-providers>
      <!-- this address provider of the cluster -->
      <address-provider id="extend-address-provider">
        <socket-address>
          <address >10.1.2.3</address>
          <port    >8088</port>
        </socket-address>
        <socket-address>
          <address >10.1.2.4</address>
          <port    >8088</port>
        </socket-address>
        <socket-address>
          <address >10.1.2.5</address>
          <port    >8088</port>
        </socket-address>
      </address-provider>
    </address-providers>

The client then connects to sub-port 3 (the NameService port) of one of the addresses in the address list and says its looking for a proxy service with the name ‘ExtendTcpProxyService’. The NameService will then (taking into account load balancing, etc) return the address of one of the proxy servers for the client to connect to.

In the messages below, you can see that the client originally connected to the NameService on sub-port 3 and then was directed to a proxy service with the appropriate name.

2014-03-21 12:59:30.899/4.588 Oracle Coherence GE 12.1.2.0.1 <D5> (thread=main, member=n/a): Connecting Socket to 10.1.2.3:8088.3
2014-03-21 12:59:30.899/4.588 Oracle Coherence GE 12.1.2.0.1 <Info> (thread=main, member=n/a): Connected Socket to 10.1.2.3:8088.3
2014-03-21 12:59:30.899/4.588 Oracle Coherence GE 12.1.2.0.1 <D5> (thread=main, member=n/a): Connecting Socket to 10.1.2.4:8089.35545
2014-03-21 12:59:30.899/4.588 Oracle Coherence GE 12.1.2.0.1 <Info> (thread=main, member=n/a): Connected Socket to 10.1.2.4:8089.35545

Summary
There are a fair few words above, but to summarise in dot-points.

  • In the nodes that proxy servers need to run on you specify an empty tcp-acceptor element.
  • On starting a node, the name service is started and listens on sub-port 3 of the unicast listen port (-Dtangosol.coherence.localport).
    Oracle Coherence GE 12.1.2.0.1  (thread=NameService:TcpAcceptor, member=4): 
    TcpAcceptor now listening for connections on 10.1.2.3:8088.3
  • The proxy service starts, registers its service-name with the NameService and listens on a random ephemeral sub-port of the unicast listen port. E.g.
    Oracle Coherence GE 12.1.2.0.1  (thread=Proxy:ExtendTcpProxyService:TcpAcceptor, member=4): 
    TcpAcceptor now listening for connections on 10.1.2.3:8088.60563
  • An address-provider is specified in the tangosol-coherence-override.xml file which points to the unicast-listen ports for the name service.
  • A client then uses the address provider client configuration to contact the name services and specifies the service-name of the proxy service it wishes to contact.
  • The name service returns the location of a proxy server to the client, and the client connects.

The NameService is great feature allowing you to simplify the management and setup of proxy servers to support Coherence*Extend.

I’m sure you will find it useful.

Advertisements
This entry was posted in Examples and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s