File Connector Quickstart

Often you might want to configure a connector to load files from a specific destination. This guide explains how to do this.

Step 1: Add connector-file dependency

The dependency to add to your pom.xml is:

<dependency>
    <groupId>com.iconsolutions.ipf.core.connector</groupId>
    <artifactId>connector-file</artifactId>
</dependency>

If importing the Icon BOM, or using the Icon BOM as a parent, there’s no need to supply a separate version.

Step 2: Configuration

Connector configuration - in general - is heavily config-driven.The configuration allows us to specify:

  • Files directory

  • File loading interval

Here’s an example of a configuration block for FileIngestionConfiguration.

tips { (1)
  process-participant.enabled = true
  file-ingestion-connector {
    file-ingestion {
      files-directory = "/static/files" (2)
      directory-id = "TIPS" (3)
      initial-delay = 5s (4)
      interval = 1h (5)
      archive-subdirectory = "archive" (6)
      failed-subdirectory = "failed" (7)
      processing-subdirectory = "processing" (8)
      timestamp-archived-and-failed-files = true (9)

      restart-settings { (10)
        min-backoff = 1s
        max-backoff = 10s
        random-factor = 0.1
        max-restarts = 20
        max-restarts-within = 10m
      }

    }
  }
}
1 This is known as the config root path and will be referenced in the code. It indicates where in the application’s configuration to look for this File Connector Transport’s settings
2 Path of the directory with files
3 ID of the directory
4 Initial load delay
5 Loading interval
6 Optional parameter to specify where processed files will be archived; if unspecified, will default to 'archive'
7 Optional parameter to specify where failed files will be stored; if unspecified, will default to 'failed'
8 Optional parameter to specify where files will be processed from; if unspecified, will default to 'processing'
9 Optional parameter to add timestamp of pattern 'yyyyMMdd_HHmmss' to filenames when moved to archived/failed directories (i.e. 'file_20251020_143530.xml'). Useful when ingesting files with same name multiple times to avoid overwriting; if unspecified, will default to 'false'
10 Optional restart parameters; if these are not set, default values are set

Step 3: Create File Connector Transport Configuration

The FileIngestionConfiguration class acts as a wrapper for the HOCON configuration which we defined in Step 2, the application accesses the configuration via this class.

Here’s how to create a FileIngestionConfiguration class for configuring a Local Directory Connector Transport:

var fileConfiguration = FileIngestionConfiguration.create("file-ingestion-connector", actorSystem.classicSystem().settings().config());
fileConfiguration.setFilesDirectory(tempDirectory.toString());
1 This retrieves the application configuration from the actor system. The actorSystem can typically be injected as a Spring bean.
2 This indicates the path to the config where the application should retrieve configuration values from.

Step 4: Create File Connector Transport

Here’s an example of how a LocalDirectoryConnectorTransport can be created:

var connectorTransport = LocalDirectoryConnectorTransport.builder()
                .withName("file-ingestion-transport") (1)
                .withActorSystem(actorSystem)
                .withFileIngestionConfiguration(fileConfiguration) (2)
                .build();
1 Give the ConnectorTransport a meaningful name
2 Provide a File Connector Transport Configuration (see Step 3 for an example)