Appearance

Powered by Platform One

Kustomize - User Guide and Best Practices ​

Summary ​

Kustomize is a command-line based, Kubernetes-native configuration management solution that allows layering of declarative YAML files to build deployments.

Getting Started ​

# simplified invocation
$ kustomize build [<OVERLAY PATH>]

Structure ​

Basic Structure

A basic application may be structured like the following:

.
├──  README.md
├──  base
│   ├── 󰅩 kustomization.yaml
│   └──  resources
│       ├── 󰅩 deployment.yaml
│       └── 󰅩 service.yaml
└──  il2
    ├──  base
    │   ├── 󰅩 kustomization.yaml
    │   └──  patches
    │       └── 󰅩 deployment-patch.yaml
    └──  overlays
        ├──  prod
        │   └── 󰅩 kustomization.yaml
        └──  staging/
            └── 󰅩 kustomization.yaml
  • "Base" directories form a base and contain the configuration of common/default resources.
  • Other directories (i.e., overlays) will inherit or patch the base.

NOTE

Base layers may be nested. In the above example, the IL2/base directory contains common resources for all IL2 deployments of this application. However, its configuration is layered on top of the root-level base directory, which contains configurations applicable to ALL Impact Levels.

Mission DevOps (MDO) Structure

The MDO structure convention is to organize the overlays as follows:

└──  <IMPACT LEVEL>
    ├──  base
    │   ├── 󰅩 kustomization.yaml
    └──  overlays
        ├──  <ENVIRONMENT>
        │   └── 󰅩 kustomization.yaml

Impact Levels

  • IL2
  • IL4
  • IL5
  • JWICS
  • NIPR
  • SIPR

Environments

An application may have environments in different Impact Levels.

  • Staging
  • Prod

Kustomize ​

At the core of Kustomize are files called Kustomizations. They are YAML-formatted and define a Kubernetes Resource Model (KRM) object, which describes how to generate or transform other KRM objects.

Definitions vary, but may look similar to the following:

kustomization.yaml

yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

commonLabels:
  protect: keycloak
  app: yarn-world
  authentication: istio-auth

resources:
  - resources/deployment.yaml
  - resources/service.yaml

Additional files called resources define specs for Kubernetes resources. Typical resources are the following:

  • Deployments
  • Services

Files or directories containing resources are referenced by Kustomizations. The below are resources in the example Kustomization above:

  • base/resources/deployment.yaml
  • base/resources/service.yaml
  • il2/base/patches/deployment-patch.yaml

Components are similar to resources, but contain configuration logic to conditionally include them in different overlays.

NOTE

Resources do not have to be organized into a directory called 'resources.'

Step-by-Step Guide ​

The primary usage for Kustomize is to generate a kustomization target. This is a good way to quickly validate syntax for objects. It can also be piped into kubectl to generate a cluster.

  1. Changes to manifest repositories (e.g., hello-manifests ) should always be verified with: kustomize build [< *PATH* >].

    a. If the PATH argument is not supplied, it will default ot the current directory. Kustomize will attempt to generate the resources specified in the kustomization.yml (or kustomization.yaml) at PATH.

yaml
$ kustomize build  yarn-world/il2/overlays/staging/
apiVersion: v1
kind: Service
metadata:
  labels:
    app: yarn-world
    authentication: istio-auth
    protect: keycloak
  name: web
spec:
  ports:
  - name: http
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: yarn-world
    authentication: istio-auth
    protect: keycloak
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: yarn-world
    authentication: istio-auth
    protect: keycloak
  name: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: yarn-world
      authentication: istio-auth
      protect: keycloak
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: yarn-world
        authentication: istio-auth
        protect: keycloak
    spec:
      containers:
      - image: registry.il2.dso.mil/platform-one/devops/hello-pipeline/yarn-world:bd2e7b11
        imagePullPolicy: IfNotPresent
        name: web
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 25m
            memory: 128Mi
          requests:
            cpu: 10m
            memory: 16Mi
      imagePullSecrets:
      - name: code-il2-pull-creds
      terminationGracePeriodSeconds: 15

Common Errors ​

Invalid Paths ​

The following errors may be displayed if the path is not correct:

PATH is to kustomization.yaml Parent Directory

$ kustomize build yarn-world/il2/overlays/staging/kustomization.yaml
Error: Error creating new loader with git: url lacks host: yarn-world/il2/overlays/staging/kustomization.yaml, dir: got file 'kustomization.yaml', but 'yarn-world/il2/overlays/staging/kustomization.yaml' must be a directory to be a root, get: invalid source string: yarn-world/il2/overlays/staging/kustomization.yaml
  • The path provided should be a directory containing a Kustomization YAML, not the path to the Kustomization YAML.
  • In this example, update to yarn-world/il2/overlays/staging/.

No kustomization.yaml Found at PATH

$ kustomize build yarn-world/il2/overlays/
Error: unable to find one of 'kustomization.yaml', 'kustomization.yml' or 'Kustomization' in directory '/Users/tonyngo/repos/hello-manifests/yarn-world/il2/overlays'
  • The path provided does not contain a Kustomization YAML.
  • Update the path to a directory containing a Kutomization YAML.

Powered By Platform One