DaemonSets:

 

DaemonSets: Running System Services on Every Node

📅 Published: August 2026
⏱️ Estimated Reading Time: 12 minutes
🏷️ Tags: Kubernetes, DaemonSets, System Services, Node Monitoring, DevOps

---

Introduction: What is a DaemonSet?

A DaemonSet is a Kubernetes controller that ensures a copy of a Pod runs on every node in your cluster. When you add a new node, the DaemonSet automatically deploys the Pod on that node. When you remove a node, the Pod is garbage collected.

Think of a DaemonSet as a "node daemon manager." It runs background services that need to be present on every node, such as monitoring agents, log collectors, or network proxies.

Key characteristics:

• Runs one Pod per node: Exactly one Pod per eligible node

• Auto-scaling: New nodes get the Pod automatically

• Auto-healing: Pods are recreated if they fail

• Node selection: Can run on specific nodes using nodeSelectors, tolerations, and affinities

---

Part 1: Why Use DaemonSets?

Common DaemonSet Use Cases

Monitoring and Observability:

• Prometheus Node Exporter: Collects node metrics

• Datadog Agent: Monitors applications and infrastructure

• Fluentd/Fluent Bit: Collects and forwards logs

Networking:

• Calico, Weave, Flannel: Network plugins for pod networking

• Cilium: eBPF-based networking and security

Storage:

• Ceph, GlusterFS: Distributed storage clients

• CSI Drivers: Container Storage Interface drivers

Security:

• Falco: Runtime security monitoring

• AppArmor, SELinux: Security policy enforcement

System Management:

• Node Problem Detector: Detects node issues

• kube-proxy: Kubernetes network proxy

---

Part 2: DaemonSet vs Other Controllers

Aspect	DaemonSet	Deployment	StatefulSet	Job/CronJob
Pods per node	Exactly one (per node)	Many (any node)	Many (any node)	One or many
Pod identity	Random	Random	Ordered	Random
Scaling	Auto with nodes	Manual	Manual	Manual
Use case	Node-level services	Stateless apps	Stateful apps	Batch jobs

DaemonSet vs Deployment

text

Deployment:
Node 1: [Pod A] [Pod B]
Node 2: [Pod C]
Node 3: [Pod D] [Pod E] [Pod F]

DaemonSet:
Node 1: [Pod X]
Node 2: [Pod X]
Node 3: [Pod X]

Every node gets exactly one Pod from the DaemonSet.

---

Part 3: DaemonSet YAML Structure

Basic DaemonSet Example

yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
  labels:
    app: fluentd
spec:
  selector:
    matchLabels:
      app: fluentd
  template:
    metadata:
      labels:
        app: fluentd
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch
        env:
        - name: FLUENT_ELASTICSEARCH_HOST
          value: "elasticsearch.logging.svc.cluster.local"
        - name: FLUENT_ELASTICSEARCH_PORT
          value: "9200"
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: dockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: dockercontainers
        hostPath:
          path: /var/lib/docker/containers

Node Selection with nodeSelector

yaml

spec:
  template:
    spec:
      nodeSelector:
        kubernetes.io/os: linux

Tolerations

yaml

spec:
  template:
    spec:
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      - key: node.kubernetes.io/not-ready
        operator: Exists
        effect: NoExecute
        tolerationSeconds: 60

Updating a DaemonSet

yaml

spec:
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1

yaml

spec:
  updateStrategy:
    type: OnDelete

---

Part 4: Real-World DaemonSet Examples

Example 1: Fluentd Log Collector

yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: logging
spec:
  selector:
    matchLabels:
      app: fluentd
  template:
    metadata:
      labels:
        app: fluentd
    spec:
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: dockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        env:
        - name: FLUENT_ELASTICSEARCH_HOST
          value: "elasticsearch.logging.svc.cluster.local"
        - name: FLUENT_ELASTICSEARCH_PORT
          value: "9200"
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: dockercontainers
        hostPath:
          path: /var/lib/docker/containers
      serviceAccountName: fluentd
      terminationGracePeriodSeconds: 30

Example 2: Prometheus Node Exporter

yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      hostNetwork: true
      hostPID: true
      containers:
      - name: node-exporter
        image: prom/node-exporter:v1.7.0
        args:
        - --path.procfs=/host/proc
        - --path.sysfs=/host/sys
        - --path.rootfs=/host/root
        ports:
        - name: metrics
          containerPort: 9100
          hostPort: 9100
        volumeMounts:
        - name: proc
          mountPath: /host/proc
          readOnly: true
        - name: sys
          mountPath: /host/sys
          readOnly: true
        - name: root
          mountPath: /host/root
          readOnly: true
      volumes:
      - name: proc
        hostPath:
          path: /proc
      - name: sys
        hostPath:
          path: /sys
      - name: root
        hostPath:
          path: /

---

Part 5: DaemonSet Commands

bash

# List DaemonSets
kubectl get daemonsets
kubectl get ds

# List DaemonSets in all namespaces
kubectl get ds --all-namespaces

# Describe a DaemonSet
kubectl describe daemonset fluentd

# Create a DaemonSet
kubectl apply -f fluentd-daemonset.yaml

# Update a DaemonSet
kubectl apply -f fluentd-daemonset.yaml

# Delete a DaemonSet
kubectl delete daemonset fluentd

# Scale a DaemonSet (if using nodeSelector to limit nodes)
# DaemonSets scale automatically with nodes

# Check DaemonSet status
kubectl rollout status ds/fluentd

---

Part 6: DaemonSet Best Practices

Resource Limits

Always set resource requests and limits for DaemonSet Pods to prevent them from consuming too many resources.

yaml

resources:
  requests:
    cpu: 100m
    memory: 100Mi
  limits:
    cpu: 200m
    memory: 200Mi

Tolerations

Use tolerations to run DaemonSets on control plane nodes:

yaml

tolerations:
- key: node-role.kubernetes.io/master
  operator: Exists
  effect: NoSchedule

Node Affinity

Use node affinity to target specific node types:

yaml

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: node-type
          operator: In
          values:
          - gpu

Update Strategy

Use rolling updates for zero-downtime updates:

yaml

updateStrategy:
  type: RollingUpdate
  rollingUpdate:
    maxUnavailable: 1

Priority Class

Set priority class to ensure DaemonSet Pods are scheduled:

yaml

priorityClassName: system-node-critical

---

Part 7: DaemonSet Troubleshooting

Pod Not Running

bash

# Check DaemonSet status
kubectl get ds fluentd

# Check Pod status
kubectl get pods -l app=fluentd

# Check Pod logs
kubectl logs fluentd-abc123

# Check Pod events
kubectl describe pod fluentd-abc123

Common Issues

Issue	Solution
Node not eligible	Check nodeSelector, tolerations, and node conditions
Pod won't start	Check logs for errors, resource limits
Pod evicted	Increase resource limits
Node not ready	Check node status, kubelet logs

---

DaemonSet vs Deployment Quick Reference

Aspect	DaemonSet	Deployment
Purpose	Run one Pod per node	Run multiple Pods across cluster
Scaling	Auto-scales with nodes	Manual scaling
Pods per node	Exactly one	Multiple
Use case	Node services	Application workloads
Update	RollingUpdate	RollingUpdate

---

Summary

Aspect	DaemonSet
Purpose	Node-level system services
Pods per node	Exactly one
Scaling	Auto-scales with nodes
Use cases	Monitoring, logging, networking, storage, security
Update strategy	RollingUpdate or OnDelete
Node selection	nodeSelector, tolerations, node affinity

DaemonSets are essential for running system-level services that need to be present on every node. They are the preferred way to deploy monitoring agents, log collectors, network plugins, and storage drivers in Kubernetes.

---

Learn More

Practice DaemonSets with hands-on exercises in our interactive labs:
https://devops.trainwithsky.com