Argo CD How-to Guides¶
Scope
Task recipes for Argo CD 3.x (checked against 3.5, the current stable line): install, access, deploy apps and ApplicationSets, add clusters, configure SSO and multi-tenancy, harden the network, run in HA, tune, monitor, upgrade, recover, and troubleshoot. Default values and flags are listed in Reference. Background is in Explanation.
Commands & Recipes¶
Install Argo CD¶
Since 3.3 the ApplicationSet CRD is larger than the client-side apply annotation limit, so manifests must be applied server-side. Pin a version in production instead of using stable.
kubectl create namespace argocd
# Non-HA (single replica of each component); pin the tag for production
kubectl apply -n argocd --server-side --force-conflicts \
-f https://raw.githubusercontent.com/argoproj/argo-cd/v3.5.3/manifests/install.yaml
# HA manifests (multiple replicas, redis-ha with Sentinel + HAProxy)
kubectl apply -n argocd --server-side --force-conflicts \
-f https://raw.githubusercontent.com/argoproj/argo-cd/v3.5.3/manifests/ha/install.yaml
Helm chart
The community Helm chart lives in argoproj/argo-helm (helm repo add argo https://argoproj.github.io/argo-helm, chart argo/argo-cd). Chart versions are numbered separately from Argo CD versions, so check the chart's appVersion.
Install the CLI and Log In¶
# CLI (Linux amd64); Homebrew: brew install argocd
curl -sSL -o argocd-linux-amd64 https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd
rm argocd-linux-amd64
# Reach the API without an Ingress
kubectl port-forward svc/argocd-server -n argocd 8080:443
# Initial admin password (stored in argocd-initial-admin-secret)
argocd admin initial-password -n argocd
# Log in (self-signed cert by default)
argocd login localhost:8080 --username admin --insecure
argocd account update-password
# After changing it, delete the bootstrap secret
kubectl -n argocd delete secret argocd-initial-admin-secret
Create and Operate an Application¶
argocd app create myapp \
--repo https://github.com/org/repo.git \
--path k8s/overlays/production \
--dest-server https://kubernetes.default.svc \
--dest-namespace production \
--sync-policy automated \
--auto-prune --self-heal
argocd app get myapp
argocd app diff myapp
argocd app sync myapp # manual sync
argocd app sync myapp --revision feature-branch # sync a specific revision
argocd app history myapp
argocd app rollback myapp <history-id> # only for apps without auto-sync
argocd app delete myapp --cascade # runs PreDelete hooks (3.3+) before removal
The declarative equivalent, which is what you should commit to Git:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io # cascade-delete managed resources
spec:
project: team-a
source:
repoURL: https://github.com/org/repo.git
targetRevision: main
path: k8s/overlays/production
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
Deploy to Every Cluster with an ApplicationSet¶
This example uses Go templates (goTemplate: true) with a matrix of all registered clusters × every add-on directory:
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: cluster-addons
namespace: argocd
spec:
goTemplate: true
goTemplateOptions: ["missingkey=error"]
generators:
- matrix:
generators:
- clusters:
selector:
matchLabels:
env: production
- git:
repoURL: https://github.com/org/infra.git
revision: HEAD
directories:
- path: addons/*
template:
metadata:
name: "{{.name}}-{{.path.basename}}"
spec:
project: platform
source:
repoURL: https://github.com/org/infra.git
targetRevision: HEAD
path: "{{.path.path}}"
destination:
server: "{{.server}}"
namespace: "{{.path.basename}}"
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
argocd appset list
argocd appset get cluster-addons
# Preview what an ApplicationSet would generate without applying it
argocd appset generate appset.yaml
Label selectors on Kubernetes version (3.4+)
Cluster-generator selectors on argocd.argoproj.io/kubernetes-version must use the vMajor.Minor.Patch format since 3.4 (also 3.3.3+).
Add and Pause Target Clusters¶
# Registers a ServiceAccount (argocd-manager) in the target cluster and stores a cluster Secret
argocd cluster add my-context --name production-cluster
argocd cluster list
# Pause reconciliation for one cluster during an incident (3.4+)
kubectl -n argocd annotate secret <cluster-secret-name> argocd.argoproj.io/skip-reconcile=true
# Resume
kubectl -n argocd annotate secret <cluster-secret-name> argocd.argoproj.io/skip-reconcile-
Configure SSO¶
Use native OIDC when your IdP speaks OIDC (Okta, Entra ID, Keycloak, Google). Keep secrets in argocd-secret and reference them with $key.
# argocd-cm
data:
url: https://argocd.example.com
oidc.config: |
name: Okta
issuer: https://example.okta.com/oauth2/default
clientID: argocd
clientSecret: $oidc.okta.clientSecret
requestedScopes: ["openid", "profile", "email", "groups"]
requestedIDTokenClaims: {"groups": {"essential": true}}
Use Dex for SAML, LDAP or GitHub OAuth:
# argocd-cm
data:
url: https://argocd.example.com
dex.config: |
connectors:
- type: github
id: github
name: GitHub
config:
clientID: $dex.github.clientID
clientSecret: $dex.github.clientSecret
orgs:
- name: my-org
Then map groups to roles and deny everything else:
# argocd-rbac-cm
data:
policy.default: ""
scopes: "[groups]"
policy.csv: |
g, my-org:platform-admins, role:admin
p, role:dev, applications, get, team-a/*, allow
p, role:dev, applications, sync, team-a/*, allow
p, role:dev, logs, get, team-a/*, allow
g, my-org:team-a, role:dev
# Validate a policy file and test a permission before rolling it out
argocd admin settings rbac validate --policy-file policy.csv
argocd admin settings rbac can role:dev sync applications 'team-a/myapp' --policy-file policy.csv
Dex users after 3.0
Policies that matched a Dex user by the sub claim must be rewritten to use federated_claims.user_id. Logs access now always requires an explicit logs, get grant.
Isolate Teams with AppProjects¶
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: team-a
namespace: argocd
spec:
sourceRepos:
- "https://github.com/org/team-a-*"
destinations:
- namespace: "team-a-*"
server: "https://kubernetes.default.svc"
clusterResourceWhitelist: [] # no cluster-scoped kinds
namespaceResourceBlacklist:
- group: ""
kind: ResourceQuota
roles:
- name: admin
groups:
- org:team-a-admins
policies:
- p, proj:team-a:admin, applications, *, team-a/*, allow
Restrict Network Access¶
This policy limits ingress to argocd-server to internal ranges. Add matching policies for repo-server egress (Git, Helm and OCI hosts only) and for Redis ingress (Argo CD pods only).
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: argocd-server-ingress
namespace: argocd
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: argocd-server
ingress:
- from:
- ipBlock:
cidr: 10.0.0.0/8
ports:
- port: 8080
protocol: TCP
Enable Repo-Server mTLS (3.5+)¶
mTLS is opt-in and switches on automatically once the Secret exists. Every relevant pod mounts it at /app/config/reposerver/mtls.
apiVersion: v1
kind: Secret
metadata:
name: argocd-repo-server-mtls
namespace: argocd
type: Opaque
stringData:
client-ca.crt: |
<PEM CA that signed the client certificate>
client.crt: |
<PEM shared client certificate>
client.key: |
<PEM client private key>
It cannot be combined with reposerver.disable.tls. --repo-server-strict-tls is deprecated; use --repo-server-ca-cert-path for a custom CA.
Enable the Source Hydrator (beta)¶
- Install the
install-with-hydrator.yamlvariant of the manifests, or sethydrator.enabled: "true"inargocd-cmd-params-cmand deployargocd-commit-server. Restart the controller and API server. - Create a push Secret labelled
argocd.argoproj.io/secret-type: repository-writeand a pull Secret labelledrepositoryfor the same repo. A GitHub App is a good choice for both. - Replace
spec.sourcewithspec.sourceHydrator:
spec:
sourceHydrator:
drySource:
repoURL: https://github.com/org/app-config
path: helm-guestbook
targetRevision: HEAD
syncSource:
targetBranch: environments/dev
path: helm-guestbook
Run in High Availability¶
- Apply the HA manifests shown above, or set the Helm chart HA values.
- Scale
argocd-serverandargocd-repo-serverto 2+ replicas. Leave the application controller StatefulSet at 1 until you need sharding. - Shard the controller by cluster once memory or queue depth becomes a problem (commonly beyond tens of clusters). The replica count and
ARGOCD_CONTROLLER_REPLICASmust match:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: argocd-application-controller
spec:
replicas: 2
template:
spec:
containers:
- name: argocd-application-controller
env:
- name: ARGOCD_CONTROLLER_REPLICAS
value: "2"
# argocd-cmd-params-cm: optional algorithm choice (legacy is default; others are alpha)
data:
controller.sharding.algorithm: consistent-hashing
# See how clusters are assigned to shards
argocd admin cluster shards -n argocd
argocd admin cluster stats -n argocd
Tune Performance¶
Repo Server¶
# argocd-cmd-params-cm
data:
reposerver.parallelism.limit: "10" # bound concurrent renders (default 0 = unlimited)
reposerver.git.lsremote.parallelism.limit: "5" # bound concurrent ls-remote calls
server.repo.server.timeout.seconds: "120"
controller.repo.server.timeout.seconds: "120"
# argocd-repo-server Deployment env
env:
- name: ARGOCD_EXEC_TIMEOUT # default 90s; raise for huge Helm charts
value: "180s"
- name: ARGOCD_GIT_ATTEMPTS_COUNT # default 1; retry flaky ls-remote
value: "3"
Controller¶
# argocd-cmd-params-cm (defaults: 20 / 10)
data:
controller.status.processors: "50"
controller.operation.processors: "25"
Monorepos¶
- Use Git webhooks instead of relying on the 3-minute poll. Raise
timeout.reconciliation(for example300s) once webhooks are in place. - Add
argocd.argoproj.io/manifest-generate-paths: .(or specific paths) to Applications so a commit only refreshes the apps whose paths changed. - Enable shallow clone for large-history repositories (3.3+). Set
depth: "1"on the repository Secret or useargocd repo add <url> --depth 1. With shallow clones, the Git-history comparison used bymanifest-generate-pathsis skipped (HA: Shallow Clone). - Keep the default 3.0
resource.exclusionsand add high-churn CRDs your operators create.
Redis¶
Compression (redis.compression) is already gzip by default. For large installations, run the HA redis-ha topology (Sentinel + HAProxy) and point components at it with redis.server: argocd-redis-ha-haproxy:6379.
Monitor Argo CD¶
These Prometheus queries use the metric names documented for 3.5:
# P95 reconciliation latency per controller shard
histogram_quantile(0.95, sum by (le, namespace) (rate(argocd_app_reconcile_bucket[5m])))
# Failed syncs over the last hour
sum by (name) (increase(argocd_app_sync_total{phase=~"Error|Failed"}[1h]))
# Degraded applications
argocd_app_info{health_status="Degraded"}
# Repo-server Git load and lock contention
sum(rate(argocd_git_request_total[5m])) by (request_type)
argocd_repo_pending_request_total
# Pending kubectl executions (controller saturation)
argocd_kubectl_exec_pending
- alert: ArgoCDAppOutOfSync
expr: argocd_app_info{sync_status="OutOfSync"} == 1
for: 30m
labels:
severity: warning
- alert: ArgoCDAppDegraded
expr: argocd_app_info{health_status="Degraded"} == 1
for: 15m
labels:
severity: critical
- alert: ArgoCDClusterUnreachable
expr: argocd_cluster_connection_status == 0
for: 5m
labels:
severity: critical
Upgrade Argo CD¶
Read every intermediate upgrade note
Minor releases can contain breaking changes. Read each vX.Y to vX.Y+1 note between your version and the target (upgrading overview). The 3.x summary is in Reference.
# 1. Back up settings and apps (argocd admin export writes YAML to stdout)
argocd admin export -n argocd > argocd-backup-$(date +%F).yaml
# 2. Apply the target version's manifests server-side (required since 3.3)
kubectl apply -n argocd --server-side --force-conflicts \
-f https://raw.githubusercontent.com/argoproj/argo-cd/v3.5.3/manifests/ha/install.yaml
# 3. Watch the rollout and app status
kubectl -n argocd rollout status statefulset/argocd-application-controller
kubectl -n argocd rollout status deploy/argocd-server deploy/argocd-repo-server
argocd app list -o wide
Upgrade checklist for 2.x to 3.x:
- Move repository config out of
argocd-cminto Secrets (removed in 3.0). - Decide on resource tracking (annotation is the new default). If you use
ApplyOutOfSyncOnly=true, follow the 3.0 notes before switching. - Add
logs, getandupdate/*,delete/*grants where users need them. - Rewrite Dex
sub-based RBAC subjects tofederated_claims.user_id. - Replace dashboards that use
argocd_app_sync_statusorargocd_app_health_statuswithargocd_app_infolabels. - For 3.5: register plain-HTTP OCI registries (and dependency registries) with
--insecure-oci-force-http, and check Helm 4 rendering of your charts.
Back Up and Recover¶
# Export all Argo CD settings, Applications, AppProjects, cluster/repo Secrets
argocd admin export -n argocd > backup.yaml
# Restore into a fresh install (same or newer version)
argocd admin import -n argocd - < backup.yaml
- Keep every
Application,ApplicationSetandAppProjectin Git (app-of-apps or an ApplicationSet that manages them). Then recovery amounts to reinstalling and re-applying the root app. - You do not need to back up Redis. It is a cache.
- The export contains credentials, so store it encrypted.
Troubleshoot Common Issues¶
| Symptom | Likely cause | Resolution |
|---|---|---|
Sync stuck in Progressing |
A resource never reports Healthy (custom CRD, LoadBalancer without IP) | Add a Lua health check under resource.customizations.health.<group_kind> or fix the workload |
ComparisonError / context deadline exceeded |
Manifest generation slower than the repo-server timeout | Raise controller.repo.server.timeout.seconds / server.repo.server.timeout.seconds and ARGOCD_EXEC_TIMEOUT; add repo-server replicas |
CustomResourceDefinition ... metadata.annotations: Too long on install or upgrade |
Client-side apply of the ApplicationSet CRD (3.3+) | Re-apply with --server-side --force-conflicts |
| High controller memory | Too many watched objects in one shard | Shard by cluster, add resource.exclusions, drop unused API groups |
| Endless OutOfSync on operator-mutated fields | Webhook caBundle or defaulted fields |
Add ignoreDifferences, or use Server-Side Diff |
| Webhook not triggering refresh | Secret mismatch or wrong payload URL | Check webhook.<provider>.secret in argocd-secret and the /api/webhook endpoint |
| SSO login loop | Callback URL mismatch | Check url in argocd-cm and the redirect URI registered with the IdP |
Helm OCI pull fails after upgrading to 3.5 (server gave HTTP response to HTTPS client) |
Helm 4 requires explicit plain-HTTP registries | Set insecureOCIForceHttp: "true" on the repo Secret; do not combine it with insecure-skip-server-verification |
| Users lose access to pod logs after 3.0 | Logs RBAC is always enforced | Grant logs, get |
# Useful diagnostics
argocd app get myapp --hard-refresh
argocd app manifests myapp --source live
kubectl -n argocd logs statefulset/argocd-application-controller | grep myapp
argocd admin settings resource-overrides health ./deploy.yaml --argocd-cm-path ./argocd-cm.yaml