Medium Post: Monitoring Clusters During Rolling Restarts

We have just published a post on Medium on how to monitor rolling restarts of Coherence clusters using a new example just released in the latest SNAPSHOT build of Coherence CE.

Carrying out a rolling restart to apply application changes or patches to a cluster is a common way to eliminate downtime for Coherence applications. As part of this process, as described in the documentation, checking StatusHA of partitioned services is essential to ensure that services have rebalanced all the data and are in a safe state to continue before the next cluster member is restarted.

View the Medium post for the full article which includes step-by-step instructions on how to use the example which can be found here.

Enjoy!

Posted in Examples | Tagged , , | Leave a comment

Coherence VisualVM Plugin 1.1.0 Released

We are pleased to announce the latest release of Coherence VisualVM Plugin which includes updated support for Coherence CE 21.06 including following additional features:

  • A new experimental option allowing you to analyze log files that have been created from cache servers using the new “Partition Events Logging” feature. This capability, introduced in Coherence CE 21.06, creates additional logging to identify times that partitions are unavailable to to events such as index builds, partition transfers and persistence events.
  • In Coherence CE 21.06, new index related JMX attributes have been added to the StorageManager MBean and you can now view them in the VisualVM Plugin in the Cache Storage tab.
  • Various other minor improvements and bug fixes
Partition Logging Analysis

Please see the Original Medium Post or visit the Coherence VisualVM release page.

Enjoy!

Posted in News | Tagged , , | Leave a comment

Coherence Community Edition 21.06 Released!

While I was on my two week road trip up the North of Western Australia, Coherence CE 21.06 was released!

On our road trip up north of Western Australia

My holiday was great, but now I’m back at work, I’d like to share some exciting new features introduced in the Coherence CE 21.06 release. The full list is available at the Coherence CE release page, but I have included a few of these below:

  • The Repository API
  • Durable Events (Experimental)
  • Topics Improvements
  • Partition Event Logging
  • Non-Blocking Entry Stores

Repository API

This new higher-level API for Coherence implements the Repository Pattern and provides Domain Driven Design (DDD) friendly way of accessing Coherence on top of the NameMap API.

Below is a simple example of a CustomerRepository:

public static class CustomerRepository 
        extends AbstractRepository<Integer, Customer> {

        private NamedMap<Integer, Customer> customers;

        public CustomerRepository(NamedMap<Integer, Customer> customers) {
            this.customers = customers;
        }

        @Override
        protected NamedMap<Integer, Customer> getMap() {
            return customers;
        }

        @Override
        protected Integer getId(Customer customer) {
            return customer.getId();
        }

        @Override
        protected Class<? extends Customer> getEntityType() {
            return Customer.class;
        }
    }

To use this, with Coherence we can do the following:

CustomerRepository customers = new
    CustomerRepository(Session.create().getMap("customers"));

// create a new customer
customers.save(new Customer(1, "Tim", "Address", Customer.GOLD, 1000));

// find the customer
Customer c = customers.get(1);

// update the customer address
customers.update(1, Customer::setAddress, "New Address");

There is so much more you can do with the Repository API, including, but not limited to: searching, aggregations, streaming and integration with our Spring and Micronaut implementations.

There is also an Async implementation of the API as well and it it works nicely with standard Java applications as well as applications that use CDI. As Aleks Seovic mentioned in his Medium post:

“The Repository API was introduced to make the implementation of data access layer within the applications easier, regardless of which framework you use to implement applications that use Coherence as a data store.”

Original Medium Post

See the full documentation here.

Durable Events (Experimental)

This new experimental feature extends the Coherence events features so you never lose an events if your client has disconnected and re-connects, allowing clients to keep track of event versions and request replay of events

See the Durable Events documentation and sample guide here.

Topics Improvements

There has been some significant changes to the Coherence Topics feature to support explicit client acknowledgements. You can browse the documentation here.

Partition Events Logging

This new feature allows you to get more information on when and for how long partitions may be blocked due to partition transfer, restore or any other reason. There is extensive documentation on how to enable and understand the messages here.

Non-Blocking Entry Stores

Coherence provides a means of integrating with underlying data sources using a number of existing strategies; namely, read-through, write-through, write-behind and refresh-ahead. Coherence now also provides a NonBlockingEntryStore interface for integrating with data sources that provide non-blocking APIs.

This new strategy is similar in nature to write-behind, as it is asynchronous to the original mutation, however does not require a queue to defer the call to the store and immediately passes the intent to store to the implementer. The implementer can in-turn immediately call the non-blocking api of the data source and on success or failure a future can pass that information to the provided StoreObserver via onNext or onError respectively.

Resources

Enjoy!

Posted in New Features | Tagged , , , , , | Leave a comment

Coherence Spring 3.0.0 M1 Released

We are excited to announce the Milestone 1 release of the Coherence Spring Project!

The project has had a new lease of life since Gunnar Hillert joined the Coherence team. With his wealth of Spring knowledge our Spring support has been rebooted and now includes support for the following (and much, much more!)

  • Coherence Bootstrap API
  • Spring Data repositories using the brand-new Coherence Repository API introduced in Coherence CE 21.06 M2
  • Spring Boot Starter
  • Spring Cache abstraction
  • Dependency Injection for various Coherence types such as caches, maps, sessions, filters
  • Coherence Event Listeners

For full information on the release, see the Release Notes or the Medium article from Gunnar.

See the following for more information:

Enjoy!

Posted in News | Tagged , , , | Leave a comment

Replay: Managing State in Elastic Microservices

Another great presentation from Aleks Seović on Coherence, Helidon and Microservices.

Well worth grabbing a beer, wine or soft-drink and listen to Aleks’ presentation to the
Denver Java Users Group.

Links:

Enjoy!

Posted in Screencasts | Tagged , , , , , | Leave a comment

Building Scalable Microservices for Java using Helidon and Coherence CE

Was great to listen to Aleksander and Dimitri talk about “Scalable Microservices for Java using Helidon and Coherence CE” in the Oracle Developer Live – Java Innovations online session a couple of days ago.

A lot of great information about Helidon and Coherence and how to combine them to create Microservices based applications.

The YouTube replay is below plus a few useful links to useful information!

Links:

Enjoy!

Posted in Presentations | Tagged , , , | Leave a comment

Open Source Coherence VisualVM Plugin v1.0.1 Released

A new and updated version of the Open Source Coherence VisualVM Plugin has just been released!

This new release contains a number of new features (as well as a few bug fixes) which will make using the Plugin much easier.

You can view the release notes here or read on to find out more.

Options Panel

We have added a new Options panel to allow easier setting of various functionality rather than having to specify properties on command line. Once the Plugin is installed, you can access via VisualVM->Preferences on Mac or Tools->Options on Windows or Linux.

VisualVM Options

Taking Multiple Thread Dumps

You have previously been able to take thread dumps by right-clicking on a member on the Members tab, but now you can also choose Generate Multiple Thread Dumps. This is useful especially in a support situation or you just want to understand what the thread are doing.

Taking Multiple Thread Dumps

Saving and Copying Thread Dumps

Going hand in had with the above, we have enhanced the thread dump dialog to include additional node and cluster information as well as give you the ability to save or copy the contents to the clipboard.

Experimental Features

We have added some “experimental” features, that can be enabled in the new options panel, that may be included as standard features down the track. Please feel free to provide feedback on any of these below:

  • Added experimental tab to display Coherence Topics information
  • Added experimental option to issue thread dump across all members
  • Added experimental “Cluster Snapshot” Tab which gives a text based snapshot of the cluster which you could cut/paste to assist in diagnosis of issues.

Installing the Update

If you already have the plugin installed, then within VisualVM Choose Tools -> Plugins from the main menu and click on Check for Updates if it doesn’t already show in the Updates tab.

If this is your first time using the Coherence VisualVM Plugin, see the GitHub Page for instructions to install.

Enjoy!

Posted in New Features | Tagged , , , , | Leave a comment

Monitoring Coherence in Docker with Grafana

Coherence provides the ability to expose Metrics via HTTP in a number of formats, Prometheus being the default. With this feature it is very easy to monitor a Coherence using Grafana.

In this post I will show you how to run the Coherence Docker image and view metrics via Grafana using docker-compose to start the following images:

  • Coherence Community Edition (CE) version 20.12
  • Grafana
  • Prometheus

The Docker images will expose the following ports:

  • Coherence
    • 1408 – gRPC Proxy port
    • 9612 – Coherence metrics endpoint to be scraped by Prometheus
    • 20000 – Coherence*Extend port
    • 30000 – Management over REST port
  • Grafana
    • 3000 – Grafana UI
  • Prometheus
    • 9090 – Prometheus UI

For more information please see https://github.com/tmiddlet2666/coherence-docker-metrics

What will you need?

  • Docker Desktop for Mac or the equivalent Docker environment for you O/S.
  • docker-compose
  • Git

Clone the example repository

git clone https://github.com/tmiddlet2666/coherence-docker-metrics.git
cd coherence-docker-metrics

Download the Grafana Dashboards

For Linux or Mac run the following to download the dashboards from the Coherence Operator GithHub page.

cd coherence-docker-metrics/grafana
./download-dashboard.sh
cd ..

For Windows:

  1. Clone the Coherence Operator repository – https://github.com/oracle/coherence-operator.git
  2. Copy all the *.json files from coherence-operator\dashboards\grafana to the directory grafana\dashboards under cloned coherence-docker-metrics repository.

Start the Docker Containers

from the coherence-docker-metrics directory run:

docker-compose up -d
Creating coherence-docker-metrics_grafana_1    ... done
Creating coherence-docker-metrics_coherence_1  ... done
Creating coherence-docker-metrics_prometheus_1 ... done

docker ps --format '{{ .ID }}\t{{.Image}}\t{{ .Names }}'

8067f3e38f61	grafana/grafana	coherence-docker-metrics_grafana_1
7bef8a89862f	prom/prometheus	coherence-docker-metrics_prometheus_1
07aeb39eb036	oraclecoherence/coherence-ce:20.12	coherence-docker-metrics_coherence_1

Note: I have included abbreviated output above but you can use docker ps to show the full output.

Access Grafana

Access the Grafana Dashboard using the following URL: http://localhost:3000/d/coh-main/coherence-dashboard-main

Default username/password is admin/admin

Main Dashboard

You can now explore the metrics exposes via Coherence.

How it Works

If you take a look at the docker-compose.yaml file you will see a number of sections for each of the Docker Images.

Coherence

The Coherence image exposes metrics via setting the coherence.metrics.http.enabled=true and ensures that the default metrics port, 9612 is exposed amongst other ports 30000 for management over REST, 20000 for Coherence*Extend and 1408 for gRPC proxy.

coherence:
  image: oraclecoherence/coherence-ce:20.12
  environment:
    - coherence.member=storage
    - coherence.machine=localhost
    - coherence.metrics.http.enabled=true
  ports:
    - 30000:30000
    - 20000:10000
    - 1408:1408
    - 9612:9612

Prometheus

The Prometheus image includes the /prometheus/prometheus.yaml file and exposes the 9090 port.

prometheus:
  image: prom/prometheus
  volumes:
    - ./prometheus/prometheus.yaml:/etc/prometheus/prometheus.yaml
  command:
    - '--config.file=/etc/prometheus/prometheus.yaml'
  ports:
    - 9090:9090

The prometheus.yaml file contains the following to scrape the default Coherence metrics port every 15s:


global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'Prometheus'
    static_configs:
      - targets: ["localhost:9090"]
  - job_name: 'coherence_cluster'
    static_configs:
      - targets: ["172.17.0.1:9612"]

Grafana

Finally, the Grafana image creates a Prometheus datasource and loads the Dashboards:

grafana:
  image: grafana/grafana
  volumes:
    - ./grafana/grafana.ini:/etc/grafana/grafana.ini
    - ./grafana/datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml
    - ./grafana/dashboards.yaml:/etc/grafana/provisioning/dashboards/dashboards.yaml
    - ./grafana/dashboards:/etc/grafana/provisioning/dashboards/dashboards
  ports:
    - 3000:3000

Conclusion

You can see from the above that it is very easy to expose and captures the metrics from a Coherence Cluster. For more information see the following:

Enjoy!

Posted in Examples | Tagged , , , , , , | Leave a comment

Access Coherence Data using GraphQL

Helidon version 2.2.0 has just been released and supports the MicroProfile GraphQL specification from the Eclipse Foundation. Using Coherence’s MicroProfile integration with Helidon, it is now easy to access Coherence data via a GraphQL endpoint.

Having worked on updating Helidon to support the MicroProfile GraphQL Spec last year, I’m looking forward over the next couple of months to share tips, tricks and examples of how this works in detail.

Please take a look at my Medium Post where where I take you through updating the Coherence To Do example with a GraphQL endpoint.

Enjoy!

References:

Posted in Related Posts | Tagged , , , , , | Leave a comment

Coherence Community Edition 20.12 Released

Coherence Community Edition (CE) 20.12 has just been released. The full Medium post can be found here, but I have included some summary information below.

This release is the second major 6 monthly release of Coherence CE and includes some exciting new features and capabilities:

  • Bootstrap API – A new simple API for configuring and starting Coherence cluster members – see the documentation
  • GraphQL Support – Along with Helidon 2.2.0, you can expose GraphQL endpoints to access and mutate your Coherence Data – see the Helidon blog for more information. Look out for a Coherence example early in the new year
  • Parallel Recovery for Coherence Persistence – A great enhancement to the Persistence feature to decrease the startup and recovery times for Coherence clusters – see the documentation
  • Micrometer Metrics – Support for Micrometer metrics – see the documentation

Have a safe and happy holiday break and see you in the new year!

Posted in Uncategorized | Tagged , , , | Leave a comment