Documentation for a newer release is available. View Latest

Bulk Archiver

The bulker has functionality for archiving produced bulk files. This will place a copy of the produced bulk file in an archive location.

Current supported locations are:

  • LOCAL

  • S3

Config Type Default Comment

ipf.bulker.archiver.file-system

String

local

Property that defines the file system that the bulk files will be copied/archived to.

Valid values are s3 or local

ipf.bulker.configurations[0].archive-path

String

N/A

Property that defines the directory or S3 bucket the files will be copied to on the file system specified above.

When enabling archiving to S3 the following config must also be provided:

ipf.file-manager.s3 {
    enabled = true
    region = "us-east-1"
    upload-parallelism = 1
    credentials {
        access-key-id = "accessKey"
        secret-access-key = "secretAccessKey"
    }
}