Stichword-Archiv: OpenShift

Kubernetes: Distributing Pods of a Deployment across nodes

Mai 17, 2022 9:14 am Veröffentlicht von

Sometimes you need to ensure that the pods of a deployment are not deployed to the same node. To achieve this, you can use the pod anti-affinity and configure it so that pods do not get deployed to pods of the same deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: game
  name: game
  namespace: arcade
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: game
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: game
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - game
            topologyKey: kubernetes.io/hostname
      containers:
      - image: quay.io/mdewald/s3e
        name: s3e

This pod anti-affinity definition will not deploy any 2 pods of the deployment onto the same node.

During the roll-out, additional pods are created before old pods are removed. If you have the same number of nodes as replicas, that means the roll-out won’t happen: No node is available to suffice the criteria to deploy an additional pod. So in the best case, you should have more nodes available than the deployment requires replicas.

You can work around this problem by changing from requiredDuringSchedulingIgnoredDuringExecution to preferredDuringScheduilingIgnoredDuringExecution:

podAntiAffinity:
  preferredDuringSchedulingIgnoredDuringExecution:
  - podAffinityTerm:
      labelSelector:
        matchExpressions:
        - key: app
          operator: In
          values:
          - game
        topologyKey: kubernetes.io/hostname
    weight: 100

However, this would allow some of the pods of the deployment to land on the same node during a roll-out of the deployment. After the roll-out, they will be distributed one pod per node again.

If you absolutely don’t want to ever have 2 pods of the same deployment run on the same node but don’t have more nodes than replicas, it can be an option for you to migrate from a Deployment to StatefulSet, which will first terminate each pod before creating a new one:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: game
  name: game
  namespace: arcade
spec:
  replicas: 2
  selector:
    matchLabels:
      app: game
  serviceName: ""
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: game
    spec:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - game
            topologyKey: kubernetes.io/hostname
      containers:
      - image: quay.io/mdewald/s3e
        name: s3e

This will ensure no pod of the StatefulSet is scheduled to the same node. If you have the same number of nodes as replicas in the StatefulSet the rollout will do the following: One by one, the pods will be removed and the replacement will be scheduled to the same node before the next pod is removed.

Building Kubernetes Operators

November 17, 2020 4:33 pm Veröffentlicht von

In the past months ad Red Hat I dove a bit into operator development. During learning how to build Kubernetes operators myself I published a quick start article on opensource.com . Using Operator SDK is a good way to kick-start your own Kubernetes operator.

While the post itself is based on an older version of the SDK and usage has changed, you should be able to get such an example operator up and running quickly using this post and a later version of the Operator SDK.

While working on different operators, I wrote down patterns and best practices I learned or discovered in the different projects, which I wrote down on a blog post on openshift.com. The CFP CFP: How to build a Kubernetes operator that doesn’t break production is mainly based on this blog post. If you want to build your own Operator, those two blog posts may help you build an operator that is easy to build, maintain and run.