Identity Resolution
Identity Resolution is an application that provides features for comparing user-supplied names and addresses to known name and addresses, and providing a response indicating whether the comparisons are a match or not. Typically, this is used in situations where the Creditor name and address provided by a third party is compared by the Creditor Bank against the name and address held against the quoted Creditor Account, though it can be used in any situation where a comparison is needed.
Features
Comparison
Comparison is where we can take two values, an actual known value, and a user-submitted comparison value, and compare them. These values can be a persons name, or an address, and each comparison query can contain one or more comparisons of each type. A response is returned indicating if the query as a whole is a match, containing scores for each comparison of how well those two things match.
This comparison is achieved by calling out to a netowl product called Namematcher. Despite the name, this product is able to compare data types beyond peoples names, including addresses, organizational names, places, and more.
A single comparison query to Identity Resolution can contain many comparisons, and depending on the NetOwl product being used, this may be split into many NetOwl comparison requests.
The comparison queries from IPF to Identity Resolution would typically by done using the client library, but the transport is http based and could also be a custom approach provided the comparison query matches the api spec.
The comparison queries from Identity Resolution to NetOwl Namematcher are also http which talks the NetOwl API.
Core Functionality
Comparisons can be done on fields of different types, for example a persons name, or their address. A single comparison query may include any number of comparisons of each type.
The implementation for comparison is achieved using NetOwl Namematcher, which will give a score (0..1) for each field being compared.
Thresholds for each field are expected along with the comparison query fields, and if any of the comparison scores for a single query is below the threshold, then the comparison match is false.
Table Based Comparisons
NetOwl supports tables of data for performance as well as for adding things like aliases and custom matching rules. Identity resolution API supports both table based and non table based matching. This is enabled/disabled by either setting or not setting the "tableName" and "fieldName" fields as seen in the Open API documentation.
| Identity Resolution does not handle the creation or insertion of data into NetOwl tables, this should be handled by the client side implementation. |
NetOwl Implementation
NetOwl Namematcher allows us to compare a single field of a single type in a single request, and returns a score between 0 and 1 indicating how similar those values are, where 1 is an exact match, and 0 is no match.
Licensing Restrictions
The current licence restricts us to NetOwl Namematcher, which supports only a single field comparison at a time, per request, and also restricts us to Latin text comparisons.
Namematcher does support comparing a single value against more than one known actual value, which we do not currently make use of. This is potentially useful when supporting name comparisons for joint accounts where only a single name has been provided for the payment.
Client Library
The comparison client sends comparison queries via http to Identity Resolution, and receives comparison responses, handling marshalling/unmarshalling of the json payloads.
Clients could call the http endpoints directly, but should prefer using this client as it abstracts and simplifies a lot of the required setup.
The client library is configurable, and at a minimum needs to know how to reach Identity Resolution to make comparison queries.
identity-resolution.comparison.http.client {
host = "localhost"
port = 8080
}TLS
One-way
If Java cacerts already exists, you may only need to apply the following comparison client configuration.
identity-resolution.comparison.http.client {
host = null
port = null
endpoint-url = "https://host:port/identity/compare"
}
The host and port must be set to null, and the full comparison path /identity/compare must be set in endpoint-url
|
Alternatively, you’ll need a trust-store, and you won’t need to override endpoint-url.
identity-resolution.comparison.http.client {
host = "localhost"
port = 8443
ssl {
trust-store-location = "/absolute/path/to/trust-store.p12"
trust-store-password = "password"
trust-store-type = "PKCS12"
}
}Mutual
identity-resolution.comparison.http.client {
host = "localhost"
port = 8443
ssl {
trust-store-location = "/absolute/path/to/trust-store.p12"
trust-store-password = "password"
trust-store-type = "PKCS12"
key-store-location = "/absolute/path/to/key-store.p12"
key-store-password = "password"
key-store-type = "PKCS12"
key-password = "password"
}
}| Identity Resolution, or its ingress must also be configured to accept Mutual TLS. |
Metrics
The following metrics are exposed for identity comparison.
| Key | Type | Description |
|---|---|---|
|
Counter |
A count of the number of comparison requests received via the api. It does not count the number of fields for comparison, so a comparison of a Name and Address is considered one request. |
|
Counter |
A count of the number of comparison request matches. A single request to compare both a Name and Address would have a single response indicating a match, or no match. |
These metrics are enabled on the back-end (identity-resolution), but in theory they could also work client-side, when the client is using the comparison-client for comparisons. Depend on com.iconsolutions.ipf.identityresolution:comparison-metrics alongside com.iconsolutions.ipf.identityresolution:comparison-client. The app using this client will also need to be a spring boot application, with micrometer, and aop enabled.
NetOwl
Comparison
NetOwl Namematcher is used to fulfil comparison functionality. The minimum configuration required for the connector is:
identity-resolution.comparison.netowl {
default.http.client {
host = "localhost"
port = 8080
}
table.http.client {
host = "localhost"
port = 8080
}
}It is expected that NetOwl Namematcher would be deployed within the same pod as Identity Resolution, in which case the above configuration may not be required as it is also the default.
Special Handling
The netowl API currently does not handle comparisons that only contain special characters. For example the following request would fail to Netowl:
{
"actual": {
"type": "FULL_ADDRESS",
"value": "10 Downing Street, SW1A 2AA"
},
"comparison": ".",
"minimumScoreToMatch": 0.75
}To handle this we filter out any of these comparisons before sending to Netowl and instead return a score of 0.0.
Special characters can be configured by providing a list of characters to the identity-resolution.comparison.netowl.exclusion.special-characters config parameter. The default list consists of Basic Latin special characters:
identity-resolution.comparison.netowl.exclusion.special-characters =
[
"/", "\\", "!", "@", "#", "$", "%", "^", "&", "*", "(", ")", "\"", "{", "}", "_", "[", "]" ,"|", "?", "<", ">", ",", "."," "
]| To add or remove items from the above list, you’d need to re-define the entire list with your additions or omissions as there’s no simple way to modify a list in HOCON. |
TLS
One-way
If Java cacerts already exists, you may only need to apply the following NetOwl comparison client configuration - using the default endpoint as an example.
identity-resolution.comparison.netowl.default.http.client {
host = null
port = null
endpoint-url = "https://host:port/api/v2/_compare"
}
The host and port must be set to null, and the full comparison path /api/v2/_compare must be set in endpoint-url
|
Alternatively, you’ll need a trust-store, and you won’t need to override endpoint-url.
identity-resolution.comparison.netowl.default.http.client {
host = "localhost"
port = 8443
ssl {
trust-store-location = "/absolute/path/to/trust-store.p12"
trust-store-password = "password"
trust-store-type = "PKCS12"
}
}Mutual
identity-resolution.comparison.netowl.default.http.client {
host = "localhost"
port = 8443
ssl {
trust-store-location = "/absolute/path/to/trust-store.p12"
trust-store-password = "password"
trust-store-type = "PKCS12"
key-store-location = "/absolute/path/to/key-store.p12"
key-store-password = "password"
key-store-type = "PKCS12"
key-password = "password"
}
}| NetOwl Namematcher, or its ingress, must also be configured to accept Mutual TLS. |
Official Documentation
Once NetOwl has been started, you can access the product documentation by visiting the NetOwl url in a browser.
For example, if deployed locally (using local-e2e-test as an example) then you can point your browser at localhost:8081/docs/index.html
Actuator
The module netowl-actuator is included in the identity-resolution application, and adds a spring boot actuator info contributor and health indicator to the application.
The actuator components work out of the box with defaults, but at a minimum, identity-resolution must know where to find NetOwl. This url will likely be the same as the one used for the comparison connector.
| This configuration must be supplied if either health and/or info is enabled. |
netowl {
base-url = "http://localhost:8080"
}| Property | Example | Description |
|---|---|---|
|
The base url required to be able to communicate with NetOwl. |
Info
The /actuator/info endpoint includes the NetOwl product details (name and version) if NetOwl is reachable. It also includes any errors with the connectivity to the NetOwl application, including errors specific to NetOwl, such as licensing and configuration issues.
The info endpoint will contain a netowl section
{
"netowl": {
"error": "system is unlicensed",
"product": "NameMatcher",
"version": "4.9.11.0"
}
}Configuration
| This is the default info contributor configuration, you don’t need to explicitly add this configuration unless you need different values. |
management.info.netowl {
enabled = true
timeout = "2S"
}| Property | Example | Description |
|---|---|---|
|
|
NetOwl info is enabled by default, but can be disabled with this property. When disabled, the NetOwl product details are not included in the info. |
|
|
The default timeout is 2 seconds, which is the duration we will wait for NetOwl to respond when checking product details. If the timeout is exceeded, the info will include NetOwl product name and version, but they will be marked as |
Health
The /actuator/health endpoint includes the same information as provided in the info contributor, and also includes the health of the NetOwl product used for comparisons. The overall application status will indicate as DOWN if NetOwl is considered to be DOWN. For NetOwl to be considered UP it must respond to a /api/v2 request with a status of 200 and the product details.
The health endpoint will contain a netowl section
{
"netowl": {
"details": {
"error": "system is unlicensed",
"product": "NameMatcher",
"version": "4.9.11.0"
},
"status": "DOWN"
}
}Configuration
| This is the default health indicator configuration, you don’t need to explicitly add this configuration unless you need different values. |
management.health.netowl {
enabled = true
timeout = "2S"
}| Property | Example | Description |
|---|---|---|
|
|
NetOwl health is enabled by default, but can be disabled with this property. When disabled, the status of NetOwl is not considered, and health will indicate as |
|
|
The default timeout is 2 seconds, which is the duration we will wait for NetOwl to respond when checking health. If the timeout is exceeded, NetOwl is considered to be |
Licencing
Licences are provide by NetOwl, and we currently use a development licence for NetOwl Namematcher, restricting us to a single comparison thread, and three instances. The current development licence is bundled with the netowl-testcontainer module for integration testing.
{
"product": "NameMatcher",
"type": "Development",
"thread limit": 1,
"*": {
"lifespan": [
"2022-01-01 00:00Z",
"2023-01-01 00:00Z"
],
"features": [
"Latin"
]
},
"verification": 1778757171272520714
}The licence needs to exist inside the running container at /var/local/netowl-data/license.key for NetOwl Namematcher to work, including the hosted documentation. For the testcontainer, it’s mounted directly into the container from the classpath, when deploying with docker compose the licence can be mounted from a local filesystem volume. In a kubernetes deployment the licence could come from a config map, as seen in identity-resolution-deployments.
Alternatively the licence can be POSTED via http, for example:
curl --request POST \
--url http://localhost:8081/api/v2/license-key \
--header 'Content-Type: application/json' \
--data '{
"product": "NameMatcher",
"type": "Development",
"thread limit": 1,
"*": {
"lifespan": [
"2022-01-01 00:00Z",
"2023-01-01 00:00Z"
],
"features": [
"Latin"
]
},
"verification": 1778757171272520714
}'The licence could be posted from an init container, or even submitted manually using the NetOwl UI.
After submitting the license, the UI will change, and also include links to the official documentation.
Updating NetOwl
The NetOwl documentation has an installation guide which includes a section on upgrading from previous versions. There’s also a version history page listing the main changes in each version.
Typical steps for absorbing a new version of NetOwl Namematcher are:
-
NetOwl binaries and licences are obtained from NetOwl, and the binary should be manually uploaded into Nexus.
-
Run the netowl-namematcher Jenkins job, which checks out and runs the netowl-namematcher Jenkins pipeline. This produces a
registry.ipf.iconsolutions.com/netowl-namematcherdocker images tagged withlatestand the version used to run the job. -
Update the netowl-testcontainer module with the correct version; integration tests will now use the latest version and can be used to verify Identity Resolution works with the Namematcher update.
-
Update the e2e_functional.yml in local-e2e-test with the correct version, and run the e2e tests to really be sure that everything is working correctly.
NetOwl Application TLS
NetOwl Namematcher is an Apache Tomcat based web application, as a result, enabling TLS will either need to be done before building the docker image, or after the fact by replacing the Tomcat server.xml file with one configured correctly for TLS. A keystore will also need to be present under the NetOwl installation directory.
For more information see the official documentation, which has a section on TLS.
Bundle TLS configuration into the docker image
Before building the NetOwl Namematcher docker image, place a keystore file in the installation directory under /server/. Then edit the Tomcat server.xml in the installation directory, under /server/conf/server.xml You will need the following:
<Connector port="8443" keystoreFile="keystore.jks" keystorePass="password"
scheme="https" secure="true" clientAuth="false" maxThreads="150"
SSLEnabled="true" sslProtocol="TLS" protocol="org.apache.coyote.http11.Http11NioProtocol" />Build the docker image.
Mount TLS configuration into the container
Mount the keystore into the NetOwl Namematcher container under /usr/local/netowl-inst/server. Then mount a server.xml, replacing the one inside the container at /usr/local/netowl-inst/server/conf/server.xml, and containing the following.
| Copy the server.xml from the installation directory for a reference config |
<Connector port="8443" keystoreFile="keystore.jks" keystorePass="password"
scheme="https" secure="true" clientAuth="false" maxThreads="150"
SSLEnabled="true" sslProtocol="TLS" protocol="org.apache.coyote.http11.Http11NioProtocol" />In a docker compose file, this might look like:
netowl-namematcher:
image: releases-registry.ipfdev.co.uk/thirdparty/netowl-namematcher:4.9.11.0
container_name: netowl-namematcher
ports:
- "8081:8080"
- "8443:8443"
volumes:
- ./config/netowl-namematcher/license.key:/var/local/netowl-data/license.key
- ./config/netowl-namematcher/keystore.jks:/usr/local/netowl-inst/server/keystore.jks
- ./config/netowl-namematcher/server.xml:/usr/local/netowl-inst/server/conf/server.xmlIdentity Resolution Application
Identity Resolution is a Spring Boot based application, and it is intended to be deployed alongside Namematcher to satisfy the comparison feature. As it is a spring boot application, all the usual spring application properties are supported.
Hocon is the primary way to configure identity-resolution, although application.properties files are also supported.
For configuration of NetOwl connectivity, see the NetOwl section.
Metrics
Prometheus compatible application metrics are exposed under /actuator/prometheus
Feature and implementation specific metrics (e.g. comparison or netowl) may also be exposed under this endpoint, and are documented in the relevant sections.
Actuator
The health and info endpoints also include the NetOwl dependency. See the NetOwl Actuator section for more information.
Identity Resolution comes with the following default actuator endpoint configuration.
management.endpoints.web.exposure.include = [health, info, prometheus]If you wish to disable health and/or info endpoints, you will need to do so explicitly, by changing the array of includes in an application.conf.
management.endpoints.web.exposure.include = [prometheus] //Look ma, no healthTLS
Identity Resolution is a Spring Boot application, which accepts a set of server.ssl.* properties for configuring TLS.
if server tls is enabled, also configure comparison client tls