IPF File Poller Scheduler
This provides a scheduler that is polling the files from a configured location(s). Based on some configured accepted patterns it is filtering the files. The scheduler is checking if a file was previously processed by keeping track on the file name, hashed metadata, and hashed content.
Metadata changed? |
Content changed? |
Meaning |
Action |
|
No |
file has not been modified |
|
|
No |
file has not been meaningfully modified |
|
|
Yes |
Assumed not possible |
|
If the file is new, or it has been modified, the File Poller Adapter is notified. The file poller is doing this by setting up a scheduler, The schedulerId will be saved in the database.
If a file has been modified and there is already a scheduled job, the job will be cancelled and another job schedule will be created for a fresh guard window. This allows for files that are in the process of being copied but have not completed, to be picked up only when they have finished and the content is no longer changed during the guard window.
Configuration
| Existing file pollers which are found during application startup are cancelled and file pollers are reloaded from configuration each time the application starts. |
The base config is ipf.file-poller.pollers and it takes a list of pollers to allow polling from multiple directories. This allows polling to be split across processing entities where each entity would be configured to read from a separate directory.
Default
Config |
Type |
Comment |
Default |
|
cron |
Property which defines the cron for the scheduler poller to run. |
"0 0 1 ? * *" |
|
cron |
Property which defines the time in seconds to schedule the job that is processing a file to run. |
"10" |
|
String |
Property which defines the location where the files should be polled from. |
"/tmp" |
|
list of strings |
Property which defines the valid patterns for a file name format. |
patterns = [ "*.txt" ] |
|
String |
Property which identifies the file poller application instance. If multiple file poller applications share the same database, this can be used to associate file poller scheduler jobs to a specific application instance |
"default-id" |
|
Boolean |
Property that determines whether the file poller will send a command to rehydrate all persistent scheduler jobs. Depending on the other modules bundled with the file poller module, you may choose to disable this if you have another module in your application which you’d like to be responsible for rehydrating persistent scheduler jobs. |
true |