Documentation for a newer release is available. View Latest

MongoDB Starter

Dependency

The following dependency is provided by default for flo-based projects (created using the Scaffolder)

<dependency>
    <groupId>com.iconsolutions.ipf.core.platform</groupId>
    <artifactId>ipf-common-starter-mongo</artifactId>
</dependency>

If you’re developing your own project using MongoDB as a data store and using Spring Reactive, you can include the above dependency to provide enhanced MongoDB integration.

Doing so adds:

  • Retries

  • SSL Configuration

  • Additional Converters

This page outlines the various different configuration options for the functionality provided by the above dependency.

Retries

When using the MongoDB Starter, the default retry configuration below helps handle transient failures (like network issues or rate limits) in Spring Data Reactive Repositories and can be customised as required.

Config Type Comment Default

ipf.mongodb.max-retry

Integer

Maximum number of retry attempts if error code is contained in the list of retryable-error-codes

3

ipf.mongodb.retryable-error-codes

List of Integers

Comma separated list of error codes for which retries should be attempted

[16500]

ipf.mongodb.delay-duration

Duration

Delay until the next retry attempt, specified as a duration.

For more details on the supported format, see: github.com/lightbend/config/blob/main/HOCON.md#duration-format

500ms

SSL Configuration

If securing a connection to MongoDB server, this can be done by setting the below configuration.

Config Type Comment Default

ipf.mongodb.set-ssl-context

Boolean

Configure the SSL context based on the below configuration parameters

false

ipf.mongodb.ssl-context.key-store-location

String

File path or resource location of the keystore file for MongoDB SSL/TLS connections

n/a

ipf.mongodb.ssl-context.key-store-password

String

Password required to access the keystore defined by ipf.mongodb.ssl-context.key-store-location

You should use secrets rather than storing plain text credentials directly in the property

n/a

ipf.mongodb.ssl-context.key-store-type

String

The type of the keystore defined by ipf.mongodb.ssl-context.key-store-location

n/a

ipf.mongodb.ssl-context.key-password

String

Specifies the password required to access the private key within the keystore.

You should use secrets rather than storing plain text credentials directly in the property

n/a

ipf.mongodb.ssl-context.trust-store-location

String

File path or resource location of the trust store used to establish trust for SSL/TLS connections to MongoDB

n/a

ipf.mongodb.ssl-context.trust-store-type

String

The type of the truststore defined by ipf.mongodb.ssl-context.trust-store-location

n/a

ipf.mongodb.ssl-context.trust-store-password

String

Password to access the trust store file.

You should use secrets rather than storing plain text credentials directly in the property

n/a

Additional Converters

The following additional converters are registered as MongoCustomConversions with Spring Data when adding the ipf-common-starter-mongo dependency:

Converter Conversion

Decimal128ToBigDecimalConverter

source.bigDecimalValue()

BigDecimalToDecimal128Converter

new Decimal128(source)

Decimal128 is a mongo bson type

DateToOffsetDateTimeConverter

ofInstant(source.toInstant(), ZoneOffset.UTC)

OffsetDateTimeToDateConverter

Date.from(source.toInstant())

DateToIsoDateTimeConverter

new IsoDateTime(source.toInstant().atOffset(ZoneOffset.UTC))

IsoDateTimeToStringConverter

source.getDateTimeString()

StringToIsoDateTimeConverter

new IsoDateTime(source)

CommandIdToStringConverter

SerializationHelper.objectToString(source)

StringToCommandIdConverter

SerializationHelper.stringToObject(s, CommandId.class)

SupportingContextToStringConverter

SerializationHelper.objectToString(source)

StringToSupportingContextConverter

SerializationHelper.stringToObject(s, SupportingContext.class)

Client-Side Field Level Encryption (CSFLE)

IPF supports MongoDB’s Client-Side Field Level Encryption capability, which allows end-to-end encryption to protect sensitive or personally-identifying (PI) data.

⚠️ Important Considerations ⚠️

Please note that CSFLE encryption keys are not ephemeral. That is, they are not generated on a per-session basis, but are long-lived. In other words, losing encryption keys means losing access to your data.

Before starting to use CSFLE, please ensure that you have robust processes in place for managing keys and access to keys.

Here is an non-exhaustive list of changes to the application that need extra consideration when CSFLE is enabled:

Event Impact

An encryption key is lost

Data cannot be recovered.

An encryption key is compromised

You must decrypt all fields that were encrypted with the compromised key, atomically update them with a new encrypted value derived from a new non-compromised key, and then update IPF to use the new key ID.

A field that was previously not considered sensitive is now sensitive

You must encrypt all existing instances of the field using the new encryption key, and update IPF’s encryption schema to include the new field.

Basic Configuration

The snippet below shows the minimal configuration required to configure CSFLE for IPF:

ipf.mongodb.csfle {
  enabled = true (1)
  key-vault-namespace = "encryption.__keyVault" (2)
  key-id = "fITzChBUQFWiAjyWAvvdbg==" (3)
  kms-providers {
    //pick the right one for your environment from one of local/aws/gcp/azure/kmip:
    local { (4)
      key = "key-here"
    }
    aws { (5)
      accessKeyId = "key-id-here"
      secretAccessKey = "secret-access-key-here"
    }
    gcp { (6)
      email = "my@gcp.email"
      privateKey = "my-private-key"
    }
    azure { (7)
      tenantId = "tenant-id-here"
      clientId = "client-id-here"
      clientSecret = "client-secret-here"
    }
    kmip { (8)
      endpoint = "my.kmip.com" //uses default KMIP port 5696
    }
  }
  extra-options {
    cryptSharedLibPath = "/path/to/mongo_crypt_v1.so" (9)
    cryptSharedLibRequired = true
  }
}
1 Enable CSFLE (disabled by default)
2 The namespace (database.collection) containing the key vault
3 The shared IPF key ID to use (see Specifying a Key ID for the format, and also What Is Encrypted and How to Override the Default Encryption Key)
4 Local KMS example
5 AWS KMS example
6 GCP KMS example
7 Azure KMS example
8 KMIP example
9 Path to the MongoDB encryption shared library used to encrypt the data. Download here (select crypt_shared under "Package")

Specifying a Key ID

Key IDs are specified as the base64 encoded equivalent of the _id of the key. This is usually a UUID. For example, if a key entry in the MongoDB vault is:

{
    _id: UUID('634211da-561d-4e34-b958-554b5bef3565')
    //...other fields here...
}

The equivalent Key ID would be the base64 encoded version of the bytes represented by the UUID string (not the UUID string itself). Here are some code snippets to convert a UUID’s bytes to its base64 equivalent.

Language Snippet

Shell

 
$ echo "634211da-561d-4e34-b958-554b5bef3565" | xxd -r -p | base64
Y0IR2lYdTjS5WFVLW+81ZQ==

Java

 
String uuid = "634211da-561d-4e34-b958-554b5bef3565".replace("-", "");
String base64 = Base64.getEncoder().encodeToString(HexFormat.of().parseHex(uuid));
System.out.println(base64);

Python

 
from base64 import b64encode
uuid = "634211da-561d-4e34-b958-554b5bef3565".replace("-", "")
base64 = b64encode(bytes.fromhex(uuid)).decode()
print(base64)
keyAltNames are not supported.

What Is Encrypted and How to Override the Default Encryption Key

By default, only one data key (defined in key-id above) is required to encrypt all fields. The table below shows all the fields that IPF Engineering has identified as potentially containing PI data, and has decided to encrypt if CSFLE is enabled.

The table also shows the configuration key you can set to use a different key ID to encrypt each field. The general format is ipf.mongodb.csfle.mongodb.[collection].[field]:

Collection Field to encrypt Config key to override default key ID

journal

eventPayloads.payload.value

ipf.mongodb.csfle.key-ids.journal.eventPayloads.payload.value

durable-state

state.value

ipf.mongodb.csfle.key-ids.durable-state.payload.content

pds

pdsContent

ipf.mongodb.csfle.key-ids.pds.pdsContent

processes

content

ipf.mongodb.csfle.key-ids.processes.content

componentEntity

content

ipf.mongodb.csfle.key-ids.componentEntity.content

summaries

creditorName

ipf.mongodb.csfle.key-ids.summaries.creditorName

summaries

debtorName

ipf.mongodb.csfle.key-ids.summaries.debtorName

summaries

creditAccount

ipf.mongodb.csfle.key-ids.summaries.creditAccount

summaries

creditorAccount

ipf.mongodb.csfle.key-ids.summaries.creditorAccount

summaries

debtorAccount

ipf.mongodb.csfle.key-ids.summaries.debtorAccount

summaries

debitAccount

ipf.mongodb.csfle.key-ids.summaries.debitAccount

summaries

initiatingPartyName

ipf.mongodb.csfle.key-ids.summaries.initiatingPartyName

summaries

searchFields.creditorName

ipf.mongodb.csfle.key-ids.summaries.searchFields.creditorName

summaries

searchFields.debtorName

ipf.mongodb.csfle.key-ids.summaries.searchFields.debtorName

summaries

searchFields.creditAccount

ipf.mongodb.csfle.key-ids.summaries.searchFields.creditAccount

summaries

searchFields.creditorAccount

ipf.mongodb.csfle.key-ids.summaries.searchFields.creditorAccount

summaries

searchFields.debtorAccount

ipf.mongodb.csfle.key-ids.summaries.searchFields.debtorAccount

summaries

searchFields.debitAccount

ipf.mongodb.csfle.key-ids.summaries.searchFields.debitAccount

payments

initiatingPartyName

ipf.mongodb.csfle.key-ids.payments.searchFields.initiatingPartyName

payments

searchFields.creditorName

ipf.mongodb.csfle.key-ids.payments.searchFields.creditorName

payments

searchFields.debtorName

ipf.mongodb.csfle.key-ids.payments.searchFields.debtorName

payments

searchFields.creditAccount

ipf.mongodb.csfle.key-ids.payments.searchFields.creditAccount

payments

searchFields.creditorAccount

ipf.mongodb.csfle.key-ids.payments.searchFields.creditorAccount

payments

searchFields.debtorAccount

ipf.mongodb.csfle.key-ids.payments.searchFields.debtorAccount

payments

searchFields.debitAccount

ipf.mongodb.csfle.key-ids.payments.searchFields.debitAccount

payments

initiatingPartyName

ipf.mongodb.csfle.key-ids.payments.searchFields.initiatingPartyName

Encrypting additional collections

To encrypt additional collections, add an encryption-schemas configuration block under ipf.mongodb.csfle and use the normal Encryption Schemas syntax:

The only departure from the MongoDB reference above is that you should not prefix the collection name with the database name. IPF will automatically determine this from the ipf.mongodb.url. 
ipf.mongodb.csfle {
  //other config here e.g. KMS, shared lib path
  encryption-schemas {
    myCollection {
      bsonType = "object"
      properties {
        myProperty {
          bsonType = "string" (1)
          keyId = [
            {
              "$binary" {
                base64 = ${my-special-key-id} (2)
                subType = "04"
              }
            }
          ]
          algorithm = "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic" (3)
        }
      }
    }
  }
}
1 See the full list of types here (under "Alias")
2 Or reuse ${ipf.mongodb.csfle.key-id}
3 Or use AEAD_AES_256_CBC_HMAC_SHA_512-Random. More info here.

Troubleshooting

CSFLE can be difficult to set up. Here are some common error messages you may encounter in the log and potential resolutions:

Error Resolution 
java.lang.IllegalStateException: executor not accepting a task

or

Error while opening candidate for CSFLE dynamic library [some path]

The mongocrypt library can not be found. Check ipf.mongodb.csfle.extra-options.cryptSharedLibPath

 
com.mongodb.crypt.capi.MongoCryptException: not all keys requested were satisfied. Verify that key vault DB/collection name was correctly specified.

Ensure that ipf.mongodb.csfle.key-vault-namespace is correctly pointing to the key vault collection in the database

 
com.mongodb.crypt.capi.MongoCryptException: HMAC validation failure

The defined KMS provider key is different to the one that was initially used to write data to the database