Scheduling a File Polling Job

This section will guide you through how to set up a file polling job in your application using IPF File Poller.

Step 1: Add dependency

Add the following dependency to your pom.xml:

<dependency>
    <groupId>com.iconsolutions.ipf.filepoller</groupId>
    <artifactId>ipf-file-poller-scheduler</artifactId>
</dependency>

ipf-file-poller-api comes bundled with ipf-file-poller-scheduler, so you don’t need to add this dependency separately.

If importing the Icon BOM, or using the Icon BOM as a parent, there’s no need to supply a version.

Step 2: Configure a polling job

Multiple polling jobs can be created per file poller application. To create a polling job that will be scheduled by the File Poller Scheduler and executed by the File Poller Scheduled Job Execution Handler, add the following configuration to your configuration file (e.g. ipf.conf) using the example below as a guide:

ipf.file-poller {
  application-id = default-id (1)
  rehydrate-scheduler-jobs = true (2)
  pollers = [ (3)
    {
      cron= "*/10 * * ? * *" (4)
      job-schedule-seconds = "20" (5)
      file-path = "/app/share/ipf/inbound" (6)
      changed-file-job-reschedule-policy = ALWAYS (7)
      file-content-hash-buffer-bytes = 8192 (8)
      file-processing-parallelism = 128 (9)
      file-processing-buffer = 500 (10)
      patterns = [ (11)
        "*.xml" (12)
      ]
    }
  ]
}

ipf.persistent.scheduler.past-acceptance-window = 30s (13)

1	The file poller application identifier
2	Parameter that determines if the file poller will rehydrate all jobs scheduled by the File Poller Scheduled Job Execution Handler at application startup.
3	The list of polling jobs that can be configured for this file poller application
4	The cron schedule for this polling job
5	The scheduling delay in seconds for each job that will be scheduled by the File Poller Scheduled Job Execution Handler. This delay fulfills two functions: Provides a job re‑scheduling window: creates a short buffer during which a job scheduled for a file can be cancelled and re‑scheduled if the file changes between polling cycles. This requires setting the delay larger than the polling interval so the cancellation window spans at least one polling cycle. Mitigates missed job executions: it reduces the chance that a job will be skipped when heavy application load causes delays that push the scheduled execution time into the past.
6	The location of the files to be retrieved and processed by the File Poller Scheduled Job Execution Handler.
7	The changed file reschedule policy. See the Changed File Job Re-schedule Policy section for more details about the available policies.
8	The buffer size in bytes for the BufferedInputStream used in file content hashing
9	The maximum number of retrieved files that can have their content and metadata hashed concurrently.
10	Buffer size controlling the maximum batch of retrieved files (after hashing) for processed status determination.
11	A list of wildcard filters that can be configured for this polling job.
12	A wildcard pattern specifying that only `.xml` files are retrieved and processed by the File Poller Scheduled Job Execution Handler for this polling job.
13	An optional parameter provided by the IPF Persistent Scheduler which allows scheduled jobs to be triggered retroactively. Can be used in conjunction with the job scheduling delay mentioned above to further reduce the chance that a job will be skipped under heavy application load.

Step 3: Implement the File Poller Adapter

After configuring our polling job, we need to create an implementation of the FilePollerAdapter interface, which will contain the logic we want to be called whenever a job scheduled by the File Poller Scheduled Job Execution Handler is executed. An example of this is shown below:

@RequiredArgsConstructor
public class FilePollerAdapterImpl implements FilePollerAdapter {

    private final NotificationSender notificationSender;

    @Override
    public CompletionStage<Void> triggerFileProcessing(FileDetails fileDetails) {
        File file = new File(fileDetails.getFilePath());
        FilePolledNotification filePolledNotification = FilePolledNotification.builder()
                .fileName(file.getName())
                .configName("pain.001.001.09")
                .fileProvider("local")
                .filePath(fileDetails.getFilePath())
                .build();
        return notificationSender.send(filePolledNotification)
                .thenApply(ignored -> null);
    }
}