Kafka MessageLogger

Before deciding to use a specific logger, it’s worth considering where the message and other data logging fits into your overall strategy. For example if you are using ODS, you should use the MessageLogger implementations provided by IPF Processing Data, specifically see Processing Data Egress.

Message-logger-impl provides a Kafka implementation of MessageLogger, and its specific logMessage implementation is using a SendConnector to send the messageLogEntry to a Kafka topic:

This uses the IPF core MessageLogEntry.

The default configuration for this logs messages to a topic called MESSAGE_LOG, but this can be changed via the ipf.conf, defaults shown below:

message.logger {
  send-connector {}
  kafka {
    producer {
      topic = MESSAGE_LOG
      restart-settings = {
        min-backoff = 1s
        max-backoff = 5s
        random-factor = 0.25
        max-restarts = 5
        max-restarts-within = 10m
      }
      kafka-clients {
        client.id = message-logger
      }
    }
  }
}

Dependencies

A starter is provided which will wire in both Kafka and Mongo loggers

<dependency>
    <groupId>com.iconsolutions.ipf.core.messagelogger</groupId>
    <artifactId>message-logger-starter</artifactId>
</dependency>

Alternatively you can bring in the specific Kafka dependency only:

<dependency>
    <groupId>com.iconsolutions.ipf.core.messagelogger</groupId>
    <artifactId>message-logger-kafka</artifactId>
</dependency>

Configuration

To configure the MessageLogger implementation you wish to use you will need to set the following in your application configuration file:

message.logger.type=kafka
message.logger.enabled=true

Exception Propagation

The behaviour when exceptions occur when attempting to send to Kafka is configurable via the following property:

message-logger.send-connector.message-logging-behaviour

BACKPRESSURED_IGNORE_FAILURES - exceptions logged, backpressure respected (default)
BACKPRESSURED - propagates exception
FIRE_AND_FORGET - exceptions are ignored