Explanation¶
Scope
How Flux 2.x works and why it is built that way: the GitOps Toolkit controllers, the artifact-based reconciliation model, server-side apply, OCI ("Gitless") GitOps, source composition with ArtifactGenerator, Helm v4 integration, multi-tenancy and the security model, the Flux Operator layer, and the project's history after Weaveworks. Exact API versions, flags and defaults live in the Reference; tasks live in the How-to Guides.
Architecture¶
Component Overview¶
Flux v2 is built on the GitOps Toolkit (GOTK): a set of composable Kubernetes controllers, each with its own CRDs, binary, and release cycle, bundled and versioned together by the flux2 repository. Unlike the monolithic control plane of Argo CD, Flux deploys each controller as an independent Deployment in the flux-system namespace. There is no Flux API server, no database, and no UI in the core distribution; the Kubernetes API is the Flux API.
The four default controllers are source-controller (fetches Git, OCI, Helm and bucket content and turns it into versioned artifacts), kustomize-controller (builds and applies manifests), helm-controller (drives Helm releases) and notification-controller (outbound alerts and inbound webhooks). Three more are opt-in: image-reflector-controller and image-automation-controller (write new image tags back to Git) and, since Flux 2.7, source-watcher (the ArtifactGenerator API). The full controller-to-CRD map with API versions is in Reference: Controllers and CRDs.
System Architecture¶
The diagram shows the controllers, the objects they exchange, and the external systems each one talks to.
graph TB
subgraph Sources["External Sources"]
Git["Git repository"]
OCI["OCI registry"]
HelmRepo["Helm repository"]
S3["S3 / GCS / Azure Blob"]
Reg["Container registry"]
end
subgraph Flux["flux-system namespace"]
SC["source-controller"]
SW["source-watcher<br/>(opt-in, 2.7+)"]
KC["kustomize-controller"]
HC["helm-controller"]
NC["notification-controller"]
IRC["image-reflector-controller<br/>(opt-in)"]
IAC["image-automation-controller<br/>(opt-in)"]
end
subgraph K8s["Kubernetes API server"]
API["Flux CRDs + workloads"]
end
subgraph Out["External notification targets"]
Chat["Slack / Teams / Discord"]
Status["Git commit status / PR comments"]
OTel["OpenTelemetry collector"]
end
Git --> SC
OCI --> SC
HelmRepo --> SC
S3 --> SC
SC -->|"artifact (tar.gz over HTTP)"| SW
SW -->|"ExternalArtifact"| KC
SC -->|"artifact (tar.gz over HTTP)"| KC
SC -->|"chart artifact"| HC
KC -->|"server-side apply"| API
HC -->|"Helm v4 SDK (SSA)"| API
KC -->|"events"| NC
HC -->|"events"| NC
SC -->|"events"| NC
Reg -->|"tag scan"| IRC
IRC -->|"ImagePolicy latest tag"| IAC
IAC -->|"git commit + push"| Git
NC --> Chat
NC --> Status
NC --> OTel
Storage Model¶
Flux keeps no database. All persistent state lives in Kubernetes objects; the only local data is the artifact cache.
| Data | Storage |
|---|---|
| Source artifacts | source-controller's /data volume (an emptyDir by default), served over HTTP on port 9090 at the address in status.artifact.url. Lost on pod restart and rebuilt on the next reconcile |
| Reconciliation state | status subresource of each CRD (GitRepository, Kustomization, HelmRelease, and more), including inventory and history |
| Helm release state | Helm storage Secrets in the release's storage namespace |
| Git and registry credentials | Secret resources, or none when Workload Identity / GitHub App is used |
| Tenant identities | ServiceAccount + RoleBinding per tenant namespace |
| Controller logs | Stdout (collected by cluster logging) |
Reconciliation Model¶
Every Flux object runs its own reconciliation loop, driven by its spec.interval. There is no central scheduler; each controller uses controller-runtime work queues and watches.
sequenceDiagram
participant Git as Git repository
participant SC as source-controller
participant KC as kustomize-controller
participant K8s as Kubernetes API
loop Every GitRepository spec.interval (e.g. 1m)
SC->>Git: git fetch (or OCI pull, Helm index download)
SC->>SC: Compare revision with status.artifact
alt New revision detected
SC->>SC: Build tar.gz artifact, store under /data
SC->>K8s: Update GitRepository status (artifact URL, digest)
end
end
Note over KC: Woken by the source status change or by its own spec.interval
KC->>SC: Download artifact over HTTP
KC->>KC: SOPS decrypt, kustomize build, post-build substitution
KC->>K8s: Server-side apply (dry-run diff, then apply)
alt Objects removed from source and spec.prune=true
KC->>K8s: Delete stale objects (garbage collection via inventory)
end
KC->>K8s: Health checks (kstatus / CEL), update Kustomization status
Key reconciliation behaviours:
- Interval-based:
spec.intervalis required on every object. For Kustomizations the docs say the minimum should be 60 seconds; the recommended production settings use1mfor the GitRepository and60mfor the Kustomization (drift correction), because source changes trigger the Kustomization immediately anyway (Kustomization spec). - Event-driven acceleration: a spec change (
metadata.generation) or a new source revision is handled at once, outside the interval. WebhookReceiversin notification-controller can trigger an immediate fetch instead of waiting for the poll. - Retry on failure:
spec.retryIntervalcontrols the retry cadence of failed Kustomizations (defaults tospec.interval). HelmReleases use install/upgrade remediation (retries, rollback, uninstall) or, since 2.7, theRetryOnFailurestrategy. - Garbage collection: with
spec.prune: true, kustomize-controller deletes objects recorded in the previous inventory that are no longer in the source. - Config watches (2.7+): changes to referenced ConfigMaps and Secrets (substitutions, decryption keys, values, kubeconfigs) trigger reconciliation when labelled
reconcile.fluxcd.io/watch: Enabled. - Faster recovery (2.7/2.8): the
CancelHealthCheckOnNewRevisionfeature gate aborts a health check that is waiting on a broken revision as soon as a fix lands.
Reconciliation Triggers¶
| Trigger | Controller | Mechanism |
|---|---|---|
| Poll interval | All | spec.interval (required, no default) |
| Git / registry webhook | notification-controller → source | Receiver annotates the target objects with a reconcile request |
| Artifact ready | kustomize-controller, helm-controller | Watch on source status (new revision) |
| Spec change | All | metadata.generation bump |
| Referenced ConfigMap/Secret change | kustomize-, helm-, notification-controller | Watch on labelled objects (2.7+) |
| Manual trigger | Any | flux reconcile <kind> <name> (--with-source to refresh the source first; --force exists only for helmrelease) |
| Dependency | kustomize-, helm-controller | spec.dependsOn waits for upstream objects to be Ready (CEL readyExpr since 2.7) |
Source Controller and Artifacts¶
The source-controller is the foundation of the toolkit. It decouples fetching from applying: every source kind is turned into the same thing, a versioned, checksummed tar.gz artifact that downstream controllers download over in-cluster HTTP.
- Acquire: clone/fetch Git at a branch, tag, semver range, name or commit; pull an OCI artifact; download a Helm index or chart; or list and fetch bucket objects.
- Verify (optional): Git commit/tag signatures (PGP, or SSH since 2.9); OCI signatures with Cosign (key or keyless) or Notation.
- Filter and package: apply
.sourceignore/spec.ignorerules and create the tar.gz artifact with a SHA-256 digest. - Store and serve: write the artifact under
/dataand serve it from the controller's HTTP server. - Notify: update
status.artifact(revision, digest, URL) and emit a Kubernetes Event, which wakes the consumers.
Because sources are shared objects, many Kustomizations can reuse one GitRepository without cloning it again.
Monorepo performance
A monorepo with 10,000+ files can make source-controller slow and memory-hungry, because it packages the checkout into one artifact and every consumer re-downloads it on each new commit. spec.ignore shrinks the artifact but the controller still does a full fetch. Mitigations, from least to most effective: spec.ignore, spec.sparseCheckout (only listed directories are checked out), a separate deploy branch, splitting per team into several GitRepository objects, ArtifactGenerator decomposition (per-path artifacts with independent revisions), or OCI artifacts built in CI. See How-to Guides: Handle a Monorepo.
OCI Artifacts and Gitless GitOps¶
Flux treats an OCI registry as a first-class source. CI renders or collects manifests and runs flux push artifact to push them as an OCI artifact (with Git source and revision annotations), optionally signs it with Cosign, and the cluster's OCIRepository pulls it by tag, semver range or digest. Helm charts stored in OCI registries are consumed the same way through HelmRelease.spec.chartRef.
Why this matters:
- No Git credentials in the cluster: clusters need only registry pull access, which cloud Workload Identity can provide (ECR, ACR, GAR).
- Supply chain: artifacts are immutable, digest-addressed and signable, and Flux verifies signatures before applying.
- Scale and edge: registries are replicated and cached better than Git servers, so large fleets and air-gapped sites pull from a local mirror. The Flux Mirror plugin (2.9) copies charts, artifacts and images between registries.
- Promotion: environments track different tags or semver ranges of the same artifact.
The Flux Operator can also sync the cluster itself from an OCI artifact (FluxInstance.spec.sync.kind: OCIRepository), which removes Git from the cluster path entirely.
Source Composition with ArtifactGenerator¶
Flux 2.7 added two APIs that turn the source layer into a pipeline stage:
ExternalArtifact(source.toolkit.fluxcd.io/v1): a generic artifact object that third-party controllers can produce and Kustomizations/HelmReleases can consume.ArtifactGenerator(source.extensions.fluxcd.io/v1beta1, served by source-watcher): copies files from one or more sources into new ExternalArtifacts. It composes (for example an upstream OCI chart plus values from Git, merged or overwritten) and decomposes (one artifact per monorepo path, so a change indeploy/backend/only reconciles the backend). Flux 2.8 added extracting and modifying Helm charts, and 2.9 addedpathPatterndirectory discovery with named captures (apps/{app}/envs/{env}).
Server-Side Apply and Drift¶
Both kustomize-controller and (since Flux 2.8 with Helm v4) helm-controller apply objects with Kubernetes Server-Side Apply (SSA):
- SSA tracks field ownership in
metadata.managedFields. Flux is the field manager for what it declares; fields owned by other managers (an HPA'sreplicas, a webhook-injected CA bundle) are not part of Flux's apply patch. - Conflicts on fields Flux declares are resolved in Flux's favour (it force-applies), which is how drift from
kubectl editis undone on the next interval. - Since 2.9,
Kustomization.spec.ignorerules (JSON Pointer paths plus a target selector) exclude specific fields from drift detection and apply, so autoscalers, service meshes and mutating webhooks can own them without a tug-of-war. HelmRelease has the equivalentspec.driftDetection.ignore. - Kustomize-controller applies in stages: CRDs and Namespaces first, then everything else, and since 2.8 custom SSA apply stages can order other kinds.
HelmRelease drift detection is opt-in
Kustomizations always correct drift on each interval. For HelmReleases, drift detection against the Helm storage manifest runs only when spec.driftDetection.mode is warn or enabled (correction), or when the controller-wide DetectDrift / CorrectDrift feature gates are on; both gates are off by default (HelmRelease spec).
Helm v4 Integration¶
Flux 2.8 moved helm-controller to the Helm v4 SDK, which changed three things:
- Server-side apply becomes the default for new releases. Helm persists the apply method in its release storage, so existing releases keep client-side apply until explicitly opted in.
- kstatus health checking replaces Helm's legacy readiness logic for all releases. Readiness can be customised with CEL expressions (
spec.healthCheckExprs), as in Kustomizations. - Inventory: HelmReleases record managed objects in
.status.inventory.
Flux 2.9 added post-render strategies (nohooks, combined — the new default, separate) and literal valuesFrom. The UseHelm3Defaults feature gate restores Helm 3 behaviour for teams that are not ready.
Image Automation Pipeline¶
Image automation (GA with the image.toolkit.fluxcd.io/v1 APIs in Flux 2.7) closes the loop from registry to Git. The controllers commit to Git rather than patching the cluster, so Git stays the source of truth.
sequenceDiagram
participant Registry as Container registry
participant IRC as image-reflector-controller
participant IAC as image-automation-controller
participant Git as Git repository
participant SC as source-controller
participant KC as kustomize-controller
IRC->>Registry: List tags (ImageRepository interval)
IRC->>IRC: ImagePolicy selects latest (semver, alphabetical, numerical, filterTags)
IAC->>IRC: Read ImagePolicy status (latest image ref)
IAC->>Git: Clone, update fields with setter markers, signed commit, push
SC->>Git: Detect new commit
SC->>KC: New artifact revision
KC->>KC: Apply updated manifests
Setter markers are YAML comments such as # {"$imagepolicy": "flux-system:podinfo"}. Since 2.9 the automation can sign its commits with SSH keys as well as PGP.
Deployment Topology¶
Single Cluster (Default)¶
All controllers run in flux-system. flux bootstrap (or a FluxInstance with spec.sync) installs the toolkit and creates a root GitRepository + Kustomization pair named flux-system that syncs the cluster from a path in Git. Flux then manages its own manifests, so upgrades are Git commits.
graph LR
Git["Git repo<br/>clusters/prod"] --> GR["GitRepository<br/>flux-system"]
GR --> KS["Kustomization<br/>flux-system"]
KS -->|"applies"| Infra["Kustomization<br/>infrastructure"]
KS -->|"applies"| Apps["Kustomization<br/>apps"]
Infra -->|"dependsOn"| Apps
Apps --> HR["HelmRelease objects"]
Multi-Cluster¶
Flux supports fleets without a central management server. Two patterns exist:
- Per-cluster bootstrap (standalone): each cluster runs its own Flux, bootstrapped from the same or different repositories/paths. This is the default and the most common pattern.
- Hub and spoke: one cluster runs Flux and applies to remote clusters via
spec.kubeConfigon Kustomizations and HelmReleases, using kubeconfig Secrets or, since 2.7, secret-less cloud Workload Identity (spec.kubeConfig.configMapRef) for EKS, AKS and GKE.
graph TB
subgraph Hub["Hub cluster"]
FluxHub["Flux controllers"]
KSA["Kustomization<br/>spec.kubeConfig: cluster-a"]
KSB["Kustomization<br/>spec.kubeConfig: cluster-b"]
end
subgraph Spoke1["Spoke cluster A"]
K8sA["Kubernetes API"]
end
subgraph Spoke2["Spoke cluster B"]
K8sB["Kubernetes API"]
end
subgraph GitOrg["Git organization"]
Fleet["fleet repo<br/>clusters/a, clusters/b"]
end
Fleet --> FluxHub
FluxHub --> KSA
FluxHub --> KSB
KSA -->|"kubeconfig Secret or Workload Identity"| K8sA
KSB -->|"kubeconfig Secret or Workload Identity"| K8sB
Per-cluster bootstrap vs hub and spoke
Per-cluster bootstrap gives better security isolation and no single point of failure: each cluster's Flux holds only its own credentials and keeps working if other clusters fail. Hub and spoke is simpler to observe from one place but concentrates credentials for every spoke on the hub. Since 2.9.5 the controllers reject kubeconfigs that reference local files; credentials must be inline.
Security Model¶
Flux delegates security almost entirely to Kubernetes-native mechanisms. Unlike Argo CD, which ships its own RBAC engine and Dex SSO server, core Flux has no identity provider, no user-facing API and no UI. Humans interact with Git (or an OCI registry) and with the Kubernetes API; Flux acts as a proxy between them.
Controller Identity and Authentication¶
Each controller runs under its own ServiceAccount in flux-system, bound to cluster-admin by default. There is no separate authentication layer.
- source-controller authenticates to Git (SSH keys, HTTPS basic/bearer tokens, mTLS, GitHub App, Azure/AWS Workload Identity), OCI registries (static creds or cloud Workload Identity) and buckets. See Reference: GitRepository Authentication.
- kustomize-controller / helm-controller impersonate the ServiceAccount in
spec.serviceAccountNameand inherit only its RBAC. - notification-controller posts events with Provider credentials and validates inbound webhooks (HMAC tokens, provider signatures, or OIDC ID tokens since 2.9).
Since Flux 2.7, object-level Workload Identity lets each Bucket, GitRepository (Azure), OCIRepository, ImageRepository, decryption config, remote kubeconfig and Provider use its own cloud identity via spec.serviceAccountName, instead of one controller-wide identity.
Multi-Tenancy and Authorization¶
Flux's tenancy model has two roles (Flux multi-tenancy):
- Platform admins have cluster-admin, bootstrap Flux, install CRDs and cluster add-ons, create tenant namespaces, ServiceAccounts and RBAC, and register tenant repositories.
- Tenants have no direct need for the Kubernetes API. They register sources and deploy with Kustomizations and HelmReleases in their namespaces; Flux applies on their behalf under a ServiceAccount the admins control.
Isolation relies on four mechanisms:
- ServiceAccount impersonation: the controllers assume the tenant's ServiceAccount, so a tenant can only create what its
Role/RoleBindingallows. - Default ServiceAccount: with
--default-service-account=default, objects that omitserviceAccountNamerun as the tenant namespace'sdefaultSA, which has no permissions. This makes forgetting impersonation fail closed. - No cross-namespace refs: with
--no-cross-namespace-refs=true, a tenant cannot point at another tenant's sources or subscribe to its events. - No remote bases: with
--no-remote-bases=true, Kustomize cannot pull bases from arbitrary URLs, so only Flux sources can affect the cluster.
The lockdown flags are listed in Reference: Multi-Tenancy Lockdown Flags; the patch recipe is in How-to Guides: Lock Down Multi-Tenancy.
The diagram shows how impersonation keeps two tenants apart on one cluster.
graph TB
subgraph FluxNS["flux-system namespace"]
SC["source-controller"]
KC["kustomize-controller"]
end
subgraph TenantA["team-a namespace"]
SA_A["ServiceAccount: team-a"]
RB_A["RoleBinding: admin in team-a"]
GitRepoA["GitRepository"]
KustA["Kustomization"]
end
subgraph TenantB["team-b namespace"]
SA_B["ServiceAccount: team-b"]
RB_B["RoleBinding: admin in team-b"]
GitRepoB["GitRepository"]
KustB["Kustomization"]
end
GitRepoA --> SC
GitRepoB --> SC
KustA -->|"impersonates SA team-a"| KC
KustB -->|"impersonates SA team-b"| KC
RB_A -.->|"grants namespace-scoped permissions"| SA_A
RB_B -.->|"grants namespace-scoped permissions"| SA_B
With --no-cross-namespace-refs=true, the team-a Kustomization cannot reference the team-b GitRepository, and vice versa.
Secrets in Git¶
kustomize-controller natively decrypts SOPS-encrypted manifests at reconcile time (spec.decryption.provider: sops), with age, OpenPGP, AWS KMS, Azure Key Vault, GCP KMS or OpenBao/Vault keys. The decrypted Secret exists only in the cluster. The trust boundary is the decryption key: whoever can read it (or impersonate the controller) can read every secret. Mitigations are strict RBAC on the key Secret, cloud KMS with Workload Identity (no key material in the cluster), or an external secret store via External Secrets Operator. The setup is in How-to Guides: Decrypt Secrets with SOPS.
Supply Chain and Source Verification¶
- Git:
GitRepository.spec.verifychecks PGP or SSH signatures on the HEAD commit and/or tag before the artifact is produced. - OCI:
OCIRepository.spec.verifychecks Cosign (key-based or keyless with OIDC identity matching; custom Sigstore trusted roots since 2.9) or Notation signatures. - Flux itself: controller images and release artifacts are signed with Cosign keyless (GitHub OIDC), ship SBOMs and SLSA provenance, and flux2 carries an SLSA 3 badge. Verify an image with:
cosign verify ghcr.io/fluxcd/source-controller:v1.9.5 \
--certificate-identity-regexp='^https://github\.com/fluxcd/.*$' \
--certificate-oidc-issuer=https://token.actions.githubusercontent.com
The Flux project notes that its design (no binary execs, pure-Go SDKs, separate controllers, impersonation, core Kubernetes machinery) has kept CVEs in Flux's own code rare (Flux turns 10). A recent example is CVE-2026-40109 (GCR Receiver validation), fixed in 2.9 with a breaking change. The full checklist is in Reference: Hardening Checklist.
Flux Operator and ResourceSets¶
The Flux Operator is a ControlPlane project (AGPL-3.0), built by Flux core maintainers but separate from the CNCF project. It adds a layer above the GitOps Toolkit:
FluxInstancereplacesflux bootstrap: the operator installs, configures, patches and upgrades the controllers from a declarative spec (for exampledistribution.version: "2.x"), and can sync the cluster from Git, OCI or a bucket. Flux no longer has to commit its own manifests to Git, and upgrades follow the operator.FluxReportsummarises health, versions and reconciler statistics for monitoring.ResourceSettemplates a bundle of Flux and Kubernetes objects over a matrix of inputs. It is Flux's answer to Argo CD ApplicationSets: one definition per application standard, stamped out per tenant, environment or pull request.ResourceSetInputProviderfeeds inputs from external systems such as open GitHub pull requests or GitLab merge requests, which enables ephemeral preview environments. Flux 2.8's PR-comment notification providers close that loop.- Flux Web UI (2.8 era) and Flux MCP Server give Flux a supported dashboard and an AI-assistant interface, filling the gap left by Weave GitOps.
Licensing difference
Core Flux is Apache-2.0 under the CNCF. The Flux Operator is AGPL-3.0 and vendor-led. Both are free to use; the AGPL matters mainly if you modify and redistribute the operator or offer it as a service.
History and Governance¶
- 2016: Weaveworks starts Flux (first commit 2016-07-07); a Weaveworks blog post then coins the term "GitOps" ("Operations by Pull Request").
- 2019–2021: Flux joins the CNCF (Sandbox 2019-07-15, Incubating 2021-03-12, per the CNCF landscape data; the Flux blog announced incubation on 2021-03-10). The team rebuilds it as Flux v2 on the GitOps Toolkit, splitting the monolithic
fluxdand the separate Helm Operator into CRD-based controllers. - 2022-11-30: Flux becomes a CNCF Graduated project (announcement).
- 2023-07-05: Flux 2.0 GA, with stable v1 Git and Kustomize APIs (Flux 2.0 GA post).
- 2024-02-05: Weaveworks, Flux's original sponsor, announces it is shutting down (SiliconANGLE, TechCrunch). Because Flux was a CNCF-owned project with an open governance model, it continued. ControlPlane now employs three of the nine core maintainers (including Stefan Prodan, a maintainer since 2018) and sells enterprise support and a hardened Flux distribution. Other maintainers work at NexHealth, SUSE, Associmates or independently, and Microsoft engineers maintain parts of the project (Azure ships Flux as its AKS/Arc GitOps extension).
- 2025–2026: steady cadence of three minors a year (2.5 → 2.9); the Flux Operator, Web UI and MCP Server arrive from ControlPlane; Flux turns 10 (2026-07-07) with 1,076 contributors, 44 repositories and 210 flux2 releases (Flux turns 10).
Current core maintainers (from fluxcd/community CORE-MAINTAINERS, 2026-09): Aurel Canciu (NexHealth), Hidde Beydals (Independent), Leigh Capili (ControlPlane), Matheus Pimenta (ControlPlane), Max Jonas Werner (Associmates), Paulo Gomes (SUSE), Sanskar Jaiswal (Independent), Soule BA (Independent), Stefan Prodan (ControlPlane).