Documentation for a newer release is available. View Latest

IPF Dynamic NodePort Service

Overview

IPF Dynamic nodePort is a Kubernetes service that dynamically allocates a nodePort for containers via an initContainer. It consists of two components: a creator and a watcher.

The creator is responsible for creating the service and allocating the nodePort, writing the configuration to a file that can be included in Akka applications.

File generated by the creator 
IPF_NODEPORT = %v
IPF_MANAGEMENTPORT = %v
IPF_NODENAME = %s
akka.remote.artery.canonical.port = ${IPF_NODEPORT}
akka.remote.artery.canonical.hostname = ${IPF_NODENAME}
akka.management.http.hostname = ${IPF_NODENAME}
akka.management.http.port = ${IPF_MANAGEMENTPORT}

The watcher monitors the state of the pods and services, ensuring that the dedicated service is properly cleaned up when its corresponding pod is deleted.

Deployment

RBAC

For the IPF Dynamic nodePort to function correctly, it requires specific Kubernetes permissions. These permissions are defined in the examples/permissions directory, which contains both ClusterRole and Role-based permissions.

RBAC permissions need to be deployed prior to deploying the creator and watcher components; otherwise, the applications will not function correctly.

Example permissions for default namespace 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ipf-dynamic-nodeport-service
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: ipf-dynamic-nodeport-service
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["services"]
    verbs: ["get", "list", "watch", "delete", "create"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch", "patch"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ipf-dynamic-nodeport-service
  namespace: default
subjects:
  - kind: ServiceAccount
    name: ipf-dynamic-nodeport-service
    namespace: default
roleRef:
  kind: Role
  name: ipf-dynamic-nodeport-service
  apiGroup: rbac.authorization.k8s.io

Creator

The creator component is designed to be deployed as an init container alongside the IPF application.

It will automatically expose a pod via a dynamically allocated nodePort through a Kubernetes service.

You need to configure it with a service template that will be used to create the service.

ConfigMap example for the nodeport service creator 
apiVersion: v1
kind: ConfigMap
metadata:
  name: ipf-dynamic-nodeport-service-creator
data:
  default.yaml: |
    apiVersion: v1
    kind: Service
    metadata:
      name: {{.PodName}}
      labels:
        creator: ipf-dynamic-nodeport-service-creator
        app: {{.PodName}}
    spec:
      externalTrafficPolicy: Local
      internalTrafficPolicy: Local
      publishNotReadyAddresses: true
      type: NodePort
      selector:
        ipfPodName: {{.PodName}}
      ports:
        - name: akka-management
          port: 8558
          targetPort: 8558
          protocol: TCP
          nodePort: 0
        - name: akka-artery
          port: 55001
          targetPort: 55001
          protocol: TCP
          nodePort: 0

Default values should be sufficient for most use cases, but you can customize the configuration as needed. For example, you can add new services that require a dynamically allocated nodePort or change the target service name.

Keys in the example that contain the value '{{.PodName}}' are used to generate the proper mapping between the pod and the service; therefore, they should not be changed.

The last step is to deploy the creator as an init container in your IPF application.

It will need a shared volume to write the configuration file that can be used by the IPF application, and a section in the pod spec to add an init container that will run the creator.

The shared volume can be mounted to the /tmp directory, and it can be an 'emptyDir' volume type with a sizeLimit of 1 MB, as the file written is very small.

By default, the creator will write the configuration file to the /tmp directory; however, you can adjust this to any path you want by using an application switch.

      initContainers:
        - name: service-creator
          image: ipf-dynamic-nodeport-service-creator:latest
          volumeMounts:
            - name: shared
              mountPath: /tmp
...
      volumes:
        - name: shared
          emptyDir: {}

This will create a file /tmp/nodeport.conf that can be dynamically included by an IPF application by adding include /nodeport/nodeport.conf to the beginning of the application.conf file (adjusting the file path to the mount location as needed).

Full example of the IPF application with the creator init container 
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ipf-example-discovery-mongodb
  labels:
    app: ipf-example-discovery-mongodb
    product: ipfv2
data:
  application.conf: |
    include "file:///nodeport/nodeport.conf"
    ipf.mongodb.url = "mongodb://ipf-mongo:27017/ipf"
    akka.cluster.seed-nodes = []
    akka {
      extensions = ["akka.management.cluster.bootstrap.ClusterBootstrap"]
      discovery {
        method = akka-mongodb
        akka-mongodb.uri = ${ipf.mongodb.url}
      }
      management {
        cluster.bootstrap {
          contact-point.filter-on-fallback-port=false
        }
      }
    }
    akka.management.http.bind-port = 8558
    akka.remote.artery.bind-port = 55001
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ipf-example
  labels:
    product: ipfv2
spec:
  replicas: 1
  selector:
    matchLabels:
      product: ipfv2
  template:
    metadata:
      labels:
        product: ipfv2
    spec:
      serviceAccountName: ipf-dynamic-nodeport-service
      imagePullSecrets:
        - name: registrysecret
      initContainers:
        - name: service-creator
          image: ipf-dynamic-nodeport-service-creator:latest
          imagePullPolicy: Never
          command:
            [
              "/ipf-dynamic-nodeport-service-creator",
              "--output",
              "/nodeport/nodeport.conf",
            ]
          volumeMounts:
            - mountPath: /nodeport
              name: cache-volume
      containers:
        - name: ipf
          image: iconsolutions/example-discovery-mongodb:latest
          ports:
            - containerPort: 8558
            - containerPort: 55001
          volumeMounts:
            - mountPath: /nodeport
              name: cache-volume
            - name: application-config
              mountPath: /example-project-app/conf/application.conf
              subPath: application.conf
      volumes:
        - name: cache-volume
          emptyDir: {}
        - name: application-config
          configMap:
            name: ipf-example-discovery-mongodb

Watcher

The watcher component monitors deleted pods that have the label product=ipfv2. When such an event is detected, the watcher will automatically attempt to clean up the corresponding service, which shares the same name as the deleted pod.

Additionally, the watcher runs at a fixed interval (every 5 minutes) to scan for potentially missed events. This background scan targets all services labeled with creator=ipf-dynamic-nodeport-service-creator.

This service can be deployed as a standalone deployment in the cluster, and no further configuration is required.

Deployment example for the nodeport service watcher 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ipf-dynamic-nodeport-service-watcher
  labels:
    product: ipfv2
spec:
  replicas: 1
  selector:
    matchLabels:
      product: ipfv2
  template:
    metadata:
      labels:
        product: ipfv2
    spec:
      serviceAccountName: ipf-dynamic-nodeport-service
      containers:
        - name: service-watcher
          image: ipf-dynamic-nodeport-service-watcher:latest
          imagePullPolicy: Never

Configuration

Priority Order (Precedence of Configuration Sources)

Configuration values are applied in the following priority order:

  1. CLI Flags (e.g. --namespace, --scan-interval)

  2. Environment Variables (e.g. NAMESPACE, SCAN_INTERVAL)

  3. Configuration File (config.yaml)

  4. Default Values (e.g., "default" for namespace, 5m for scan-interval)

Configuration search files locations and priorities

Creator

  1. /etc/ipf-dynamic-nodeport-service-creator/

  2. $HOME/.config/ipf-dynamic-nodeport-service-creator

  3. .

Watcher

  1. /etc/ipf-dynamic-nodeport-service-watcher/

  2. $HOME/.config/ipf-dynamic-nodeport-service-watcher

  3. .

Configuration options

Watcher

To configure the watcher, create a config.yaml file:

namespace: my-namespace
scan-interval: 2m

Alternatively, set the following environment variables:

NAMESPACE=my-namespace
SCAN-INTERVAL: 2m

Time can be specified as:

  • 1 (seconds)

  • 1s (seconds)

  • 1m (minute)

  • 1h (hour)

Creator

To configure the creator, create a config.yaml file:

pod: test-pod
output: /nodeport/nodeport.conf
configmapkey: default.yaml
akka-artery-management-port-name: akka-artery
akka-management-port-name: akka-management

Alternatively, you can set the following environment variables:

POD=test-pod
OUTPUT=/nodeport/nodeport.conf
CONFIGMAPKEY=default.yaml
AKKA-ARTERY-MANAGEMENT-PORT-NAME=akka-artery
AKKA-MANAGEMENT-PORT-NAME=akka-management