Volumes And Storage Stateful Sets

Container in a pod has its own isolated filesystem, because the filesystem comes from the container’s image.

Storage Lingo

Ref: Kubernetes Storage Lingo 101 talk

Volumes

Kubernetes volumes provide a way for containers in a pod to access and share data via the filesystem.

At its core, a volume is a directory, possibly with some data in it, which is accessible to the containers in a pod.

To use a volume, specify the volumes to provide for the Pod in .spec.volumes and declare where to mount those volumes into containers in .spec.containers[*].volumeMounts.

For each container defined within a Pod, you must independently specify where to mount each volume that the container uses.

Special Volumes

Empty dir

An emptyDir in Kubernetes is a type of volume that is created as empty when a Pod is assigned to a Node. It exists as long as that Pod is running on that Node.

Provides a temporary directory that all containers in a Pod can read and write to.

apiVersion: v1
kind: Pod
metadata:
  name: myvolumes-pod
spec:
  containers:
  - image: alpine
    imagePullPolicy: IfNotPresent
    name: myvolumes-container
    
    command: [    'sh', '-c', 'echo The Bench Container 1 is Running ; sleep 3600']
    
    volumeMounts:
    - mountPath: /demo
      name: demo-volume
  volumes:
  - name: demo-volume
    emptyDir: {}

Config Maps

A ConfigMap provides a way to inject configuration data into pods.

A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.

Secrets

Secrets are API objects in Kubernetes, which store sensitive information like passwords, keys, and tokens in base64 format.

Secrets separate sensitive data from application code, making it easier to manage and modify independently.

-n : no trailing newline

Ref: https://yuminlee2.medium.com/kubernetes-secrets-4287b5a83606

Why do we need PV?

If you have your Pod on GCE, then you can port it to OnPrem or to AWS

Think in terms of Cluster Admin Persona and User Persona. Admins provide storage, users need not worry, they can just port the application.

As soon as you create the claim, Kubernetes finds the appropriate PersistentVolume and binds it to the claim. The PersistentVolume’s capacity must be large enough to accommodate what the claim requests.

PVC doesn’t have any details regarding where the storage is, etc. Portability.

Volume decouples the storage from the Container. Its lifecycle is coupled to a pod. It enables safe container restarts and sharing data between containers in a pod.
Persistent Volume decouples the storage from the Pod. Its lifecycle is independent. It enables safe pod restarts and sharing data between pods. Allows data to persist even if the Pod is deleted.

Host Path volumes

If you’re thinking of using a hostPath volume as the place to store a database’s data directory, think again. Because the volume’s contents are stored on a specific node’s filesystem, when the database pod gets rescheduled to another node, it will no longer see the data.

When an application running in a pod needs to persist data to disk and have that same data available even when the pod is rescheduled to another node, you can’t use any of the volume types we’ve mentioned so far. Because this data needs to be accessible from any cluster node, it must be stored on some type of network-attached storage (NAS).

Ref: Kubernetes in Action

HostPath is different from Local PV

Storage

Awesome tutorial

https://medium.com/@dunefro/part-1-4-container-attached-storage-with-openebs-understand-volume-provisioning-in-kubernetes-e7d7497dfe7f
https://www.digihunch.com/2021/06/kubernetes-storage-explained/

Ref: https://seifrajhi.github.io/blog/kubernetes-storage-deep-dive/

Kubernetes Storage Options — Persistent Volumes (PV), Persistent Volume Claims (PVC), Storage Classes (SC).

Persistent Volume — low-level representation of a storage volume.
Persistent Volume Claim — binding between a Pod and a Persistent Volume.
Storage Class — allows for dynamic provisioning of Persistent Volumes.

Kubernetes docs

Ref: https://seifrajhi.github.io/blog/kubernetes-storage-deep-dive/

Deep dive

PV: A PV is a storage resource located in the cluster. Administrators can manually provision PVs, and Kubernetes can use storage classes to dynamically provision PVs.

PVC: A PVC is a storage request made by a user. It works similarly to a pod but consumes PV resources rather than node resources.

CSI: The Container Storage Interface (CSI) is a standard interface that allows container orchestrators to expose arbitrary block and file storage systems to containers they manage.

Static Provisioning and Dynamic Provisioning

Static Provisioning

Dynamic Provisioning

The implementation of dynamic volume provisioning is based on the StorageClass abstraction.

Summary

CSI

In Kubernetes, in-tree storage drivers were storage plugins that were directly part of the core Kubernetes code. They were being phased out in favor of CSI (Container Storage Interface) drivers, which are plug-ins that are separate from the Kubernetes core.

Before CSI, Kubernetes provided a powerful volume plugin system. These volume plugins were “in-tree” meaning their code was part of the core Kubernetes code and shipped with the core Kubernetes binaries.

However, adding support for new volume plugins to Kubernetes was challenging. Vendors that wanted to add support for their storage system to Kubernetes (or even fix a bug in an existing volume plugin) were forced to align with the Kubernetes release process.

The Kubernetes Storage Special Interest Group (SIG) defines three methods to implement a volume plugin:

In-tree volume plugin [deprecated],
Out-of-tree FlexVolume driver [deprecated],
Out-of-tree CSI driver.

Stateful Sets

If you have a StatefulSet called tkb-sts with five replicas and the tkb-sts-3 replica fails, the controller starts a new Pod with the same name and attaches it to the surviving volumes.

We’ve already said that StatefulSets are for applications that need Pods to be predictable and long-lived.
This might involve applications connecting to specific Pods rather than letting the Service perform round-robin load balancing across all Pods.
To make this possible, StatefulSets use a headless Service to create reliable and predictable DNS names for every Pod. Other apps can then query DNS (the service registry) for the full list of Pods and make direct connections.

A headless Service is a regular Kubernetes Service object without a ClusterIP address (spec.clusterIP set to None). It becomes a StatefulSet’s governing Service when you list it in the StatefulSet config under spec.serviceName.

Stateful Sets

Pod mounts volume

PV and PVC are binded

Storage classes and Dynamic Provisioning

Representation of entire class of storage

Admin creates storage class, and now admin no longer need to pre-provision all of the PVs

Storage classes knows how to provide the PVs on appropriate disks.

ReadWriteOnce (RWO): The volume can be mounted as read-write by a single node. This is at node level not at Pod level. If the access modes are specified as ReadWriteOncePod, the volume is constrained and can be mounted on only a single Pod.

Example:

StatefulSet vs. DaemonSet vs. Deployment

StatefulSet: Manages stateful applications requiring stable identities and persistent storage.
DaemonSet: Ensures a copy of a pod runs on every node for node-level services like logging.
Deployment: Manages stateless applications with flexible, declarative updates.

Ref

https://www.mirantis.com/blog/kubernetes-cheat-sheet/
https://kubernetesbootcamp.github.io/kubernetes-bootcamp/index.html