Container storage interface is an industry standard that enables storage vendors to develop a plugin once and have it work across a number of container orchestration systems.

SeaweedFS is a simple and highly scalable distributed file system, to store and serve billions of files fast!

• Deployment
  • Kubernetes (kubectl)
  • Kubernetes (helm)
• Update (Safe rollout)
• Testing
• Static and dynamic provisioning
• DataLocality
• Node topology
• License
• Code of conduct

• Already have a working Kubernetes cluster (includes kubectl)
• Already have a working SeaweedFS cluster

1. Add the helm repo;

helm repo add seaweedfs-csi-driver https://seaweedfs.github.io/seaweedfs-csi-driver/helm

2. Check versions by helm repo update seaweedfs-csi-driver and helm search repo seaweedfs-csi-driver

1. Clone this repository

git clone https://github.com/seaweedfs/seaweedfs-csi-driver.git

2. Adjust your SeaweedFS Filer address via variable SEAWEEDFS_FILER in deploy/kubernetes/seaweedfs-csi.yaml (2 places)

3. Apply the container storage interface for SeaweedFS for your cluster. Use the '-pre-1.17' version for any cluster pre kubernetes version 1.17.

$ helm template seaweedfs ./deploy/helm/seaweedfs-csi-driver > deploy/kubernetes/seaweedfs-csi.yaml

Check the kubelet root directory ,Execute the following command

ps -ef | grep kubelet | grep root-dir

If the result returned from the previous check command is not empty, the root directory (eg:--root-dir=/data/k8s/kubelet/data) representing the kubelet is not the default value (/var/lib/kubelet), so you need to update the kubelet root-dir in the CSI driven deployment file and deploy:

sed 's+/var/lib/kubelet+/data/k8s/kubelet/data+g'  deploy/kubernetes/seaweedfs-csi.yaml | kubectl apply -f - 

If the result returned by the previous check command is null, you can directly deploy it without modifying the configuration:

$ kubectl apply -f deploy/kubernetes/seaweedfs-csi.yaml

4. Ensure all the containers are ready and running

$ kubectl get po -n kube-system

The provided static manifest deploy/kubernetes/seaweedfs-csi.yaml does not include TLS configuration by default. To enable TLS, it is recommended to use Helm with tlsSecret configured. If you must use static manifests, you will need to manually patch the seaweedfs-mount and seaweedfs-node DaemonSets to include the necessary TLS environment variables and volume mounts.

$ kubectl delete -f deploy/kubernetes/sample-busybox-pod.yaml
$ kubectl delete -f deploy/kubernetes/sample-seaweedfs-pvc.yaml
$ kubectl delete -f deploy/kubernetes/seaweedfs-csi.yaml

1. Clone project

git clone https://github.com/seaweedfs/seaweedfs-csi-driver.git

2. Edit ./deploy/helm/seaweedfs-csi-driver/values.yaml if required and Install

helm install --set seaweedfsFiler=<filerHost:port> seaweedfs-csi-driver ./deploy/helm/seaweedfs-csi-driver

Example with multiple filers :

helm install seaweedfs-csi-driver ./deploy/helm/seaweedfs-csi-driver/ \
  --namespace seaweedfs-csi-driver \
  --set seaweedfsFiler="<filerHost:port>\,<filerHost:port>\,<filerHost:port>\,<filerHost:port>\,<filerHost:port>"

helm uninstall seaweedfs-csi-driver

Updating seaweed-csi-driver DaemonSet (DS) will break processeses who implement fuse mount: newly created pods will not remount net device.

For safe update set node.updateStrategy.type: OnDelete for manual update. Steps:

1. delete DS pods on the node where there is no seaweedfs PV
2. cordon or taint node
3. evict or delete pods with seaweedfs PV
4. delete DS pod on node
5. uncordon or remove taint on node
6. repeat all steps on [all nodes]

1. Create a persistent volume claim for 5GiB with name seaweedfs-csi-pvc and storage class seaweedfs-storage. The requested size is applied as a quota to the SeaweedFS collection used by the mount.

$ kubectl apply -f deploy/kubernetes/sample-seaweedfs-pvc.yaml

2. Verify if the persistant volume claim exists and wait until its the STATUS is Bound

$ kubectl get pvc

3. After its in Bound state, create a sample workload mounting that volume

$ kubectl apply -f deploy/kubernetes/sample-busybox-pod.yaml

4. Verify the storage mount of the busybox pod

$ kubectl exec my-csi-app -- df -h

For dynamically provisioned volumes, resources.requests.storage is enforced by the SeaweedFS FUSE mount and reported by df. Once the mount observes that the quota has been reached, further writes fail with ENOSPC. PVC expansion updates the mount quota.

The quota is collection-based and enforced by each CSI-managed mount, rather than stored as an authoritative server-side limit. By default, every dynamic volume gets its own collection, so its quota is isolated. If multiple volumes are configured to use the same collection, their usage is shared and they do not have independent per-directory quotas. Writes through another SeaweedFS client that does not use the quota-configured mount are not restricted by this CSI quota.

Because enforcement is mount-side, concurrent writers on different mounts can briefly exceed the configured capacity before collection usage is refreshed.

For statically provisioned Kubernetes volumes, the driver reads spec.capacity.storage from the PersistentVolume matching spec.csi.volumeHandle. Other orchestrators receive volumeCapacity from dynamic CreateVolume calls; static integrations can provide the same value in bytes through the volume context.

By default, driver will create separate folder (/buckets/<volume-id>) and will use separate collection (volume-id) for each request. Sometimes we need to use exact collection name or change replication options. It can be done via creating separate storage class with options:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: seaweedfs-special
provisioner: seaweedfs-csi-driver
parameters:
  collection: mycollection
  replication: "011"
  diskType: "ssd"

There is another use case when we need to access one folder from different pods with ro/rw access. In this case we do not need additional StorageClass. We need to create PersistentVolume:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: seaweedfs-static
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: seaweedfs-csi-driver
    volumeHandle: dfs-test
    volumeAttributes:
      collection: default
      replication: "011"
      path: /path/to/files
      diskType: "ssd"
    readOnly: true
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem

and bind PersistentVolumeClaim(s) to it:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: seaweedfs-static
spec:
  storageClassName: ""
  volumeName: seaweedfs-static
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

Mount-time parameters can be changed on a bound PVC through VolumeAttributesClass, without recreating the volume:

apiVersion: storage.k8s.io/v1
kind: VolumeAttributesClass
metadata:
  name: seaweedfs-ssd
driverName: seaweedfs-csi-driver
parameters:
  diskType: "ssd"
  concurrentReaders: "64"

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  volumeAttributesClassName: seaweedfs-ssd
  ...

Modifiable parameters are the mount-time knobs: diskType, replication, ttl, dataCenter, dataLocality, uidMap, gidMap, chunkSizeLimitMB, volumeServerAccess, readRetryTime, concurrentReaders, concurrentWriters, cacheCapacityMB, cacheMetaTtlSec, writebackCache, metadataFlushSeconds, dlm. Structural keys (collection, path, parentDir, volumeName, capacity) are rejected, as are invalid values (unknown dataLocality, non-integer numeric options, non-boolean flags).

For regenerable-data workloads (build caches, scratch workspaces) a writeback-profiled class trades crash-safety for write throughput:

apiVersion: storage.k8s.io/v1
kind: VolumeAttributesClass
metadata:
  name: seaweedfs-writeback
driverName: seaweedfs-csi-driver
parameters:
  writebackCache: "true"
  metadataFlushSeconds: "300"

writebackCache=true enables FUSE writeback caching — data accepted before the last flush is lost if the mount crashes. metadataFlushSeconds is unrelated to that window: it periodically flushes file metadata to the filer so long-running writes are not purged by volume.fsck; it does not make cached data crash-safe.

dlm is mutually exclusive with writebackCache: weed mount runs single-writer under writeback and silently disables the distributed lock manager when both are requested, so the driver rejects the combination at modify/create time. Set dlm on a separate class (without writeback) when concurrent mounts need coordination.

Notes:

• A class applied when the PVC is created reaches the first mount directly. Changing the class of a bound PVC persists the accepted parameters on the filer (under /.csi/vac/<volume>, outside /buckets); the volume picks them up at its next stage (pod restart or reschedule).
• The entry is removed with the volume on DeleteVolume.
• On a filer with security enabled the store signs requests with the jwt.filer_signing.key / jwt.filer_signing.read.key from the mounted security.toml; the chart mounts it into both plugin containers. Set vacStore.useTls so the tokens are not sent over plain HTTP.
• The cluster's csi-resizer must support VolumeAttributesClass against the k8s API version in use: released v1.14.0 still watches the storage.k8s.io/v1beta1 API removed in Kubernetes 1.34 — use a build from master (or a newer release once available).

DataLocality (inspired by Longhorn) allows instructing the storage-driver which volume-locations will be used or preferred in Pods to read & write.

It auto-sets mount-options based on the location a pod is scheduled in and the locality-option wanted. The option can be set and overridden in Driver, StorageClass and PersistentVolume.

Change the type of locality

Driver < StorageClass < PersistentVolume

Volume-Servers and the CSI-Driver-Node need to have the locality-option DataCenter correctly set (currently only this option is required).

This can be done manually (although quite tedious) or injected by the Container-Orchestration.

Kubernetes

Unfortunately Kubernetes doesnt allow grabbing node-labels, which contain well-known region-labels, and setting them as environment-variables. The DownwardAPI is very limited in that regard. (see #40610)

Therefore a workaround must be used. KubeMod can be used based on this comment. This of course requires KubeMod to be installed.

You can activate it in the Helm-Chart values.yaml -> node.injectTopologyInfoFromNodeLabel.enabled. node.injectTopologyInfoFromNodeLabel.labels decides which labels are grabbed from the node.

It is recommended to use well-known labels to avoid confusion.

When the driver only runs on some nodes, report the labels of those nodes as accessible topology. CSINode.spec.drivers[].topologyKeys then gets populated, provisioned PVs get a matching nodeAffinity and the scheduler stops placing pods with SeaweedFS PVCs on nodes without the driver.

topologyKeys:
  - topology.kubernetes.io/zone

Every key is looked up in the labels of the node the driver runs on; keys the node does not have are skipped. The CSI spec asks for a single key prefix across all topology keys, so prefer keys such as topology.kubernetes.io/zone and topology.kubernetes.io/region together. A StorageClass can then restrict where volumes are provisioned:

allowedTopologies:
  - matchLabelExpressions:
      - key: topology.kubernetes.io/zone
        values:
          - zone-a

The node ServiceAccount needs get on nodes, which the Helm chart already grants.

Apache v2 license

Participation in this project is governed by Kubernetes/CNCF code of conduct