Exploring ECK & Elasticsearch

The Elastic Stack at a Glance

Key Kubernetes Objects for Elasticsearch

Core Commands

# Get resources
kubectl get pods                          # List pods
kubectl get pods -o wide                  # Pods with node info
kubectl get pods,services,configmaps      # Multiple types
kubectl get elasticsearch                 # ECK custom resources

# Describe (detailed info)
kubectl describe pod elasticsearch-es-main-0

# Logs
kubectl logs elasticsearch-es-main-0              # Current logs
kubectl logs elasticsearch-es-main-0 -f           # Stream / follow
kubectl logs elasticsearch-es-main-0 -c elasticsearch  # Specific container

# Events (scheduler decisions)
kubectl get events --sort-by=.metadata.creationTimestamp

# Exec into a pod
kubectl exec -it elasticsearch-es-main-0 -c elasticsearch -- bash

# Port forward
kubectl port-forward service/quickstart-es-http 9200
kubectl port-forward service/quickstart-kb-http 5601

# Copy files
kubectl cp my-pod:/tmp/data.json ./local-data.json

# Resource usage
kubectl top pods
kubectl top nodes

# Context management
kubectl config current-context
kubectl config use-context elastic-lab
kubectl config set-context --current --namespace=elastic-system

ECK-Specific Commands

# Check ECK operator logs
kubectl -n elastic-system logs -f statefulset.apps/elastic-operator

# List all Elastic resources
kubectl get elasticsearch,kibana,beat,agent --all-namespaces

# Get the elastic user password
kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}'

# Check cluster health via API
kubectl port-forward svc/quickstart-es-http 9200 &
curl -u "elastic:$PASSWORD" -k https://localhost:9200/_cluster/health?pretty

# Explain a custom resource
kubectl explain elasticsearch.spec.nodeSets

What ECK Automates

1. Namespace — elastic-system

apiVersion: v1
kind: Namespace
metadata:
  name: elastic-system
  labels:
    name: elastic-system

2. ServiceAccount — elastic-operator

apiVersion: v1
kind: ServiceAccount
automountServiceAccountToken: true
metadata:
  name: elastic-operator
  namespace: elastic-system
  labels:
    control-plane: elastic-operator
    app.kubernetes.io/version: "3.0.0"

3. Secret — elastic-webhook-server-cert

apiVersion: v1
kind: Secret
metadata:
  name: elastic-webhook-server-cert
  namespace: elastic-system
  labels:
    control-plane: elastic-operator
    app.kubernetes.io/version: "3.0.0"

4. ConfigMap — elastic-operator (Operator Settings)

This is the operator's main configuration file. It controls how the ECK operator behaves — logging, reconciliation speed, certificate lifetimes, telemetry, and more.

apiVersion: v1
kind: ConfigMap
metadata:
  name: elastic-operator
  namespace: elastic-system
data:
  eck.yaml: |-
    log-verbosity: 0
    metrics-port: 0
    metrics-secure: false
    container-registry: docker.elastic.co
    max-concurrent-reconciles: 3
    ca-cert-validity: 8760h
    ca-cert-rotate-before: 24h
    cert-validity: 8760h
    cert-rotate-before: 24h
    disable-config-watch: false
    exposed-node-labels: [topology.kubernetes.io/.*,failure-domain.beta.kubernetes.io/.*]
    set-default-security-context: auto-detect
    kube-client-timeout: 60s
    elasticsearch-client-timeout: 180s
    disable-telemetry: false
    distribution-channel: all-in-one
    validate-storage-class: true
    enable-webhook: true
    webhook-name: elastic-webhook.k8s.elastic.co
    webhook-port: 9443
    operator-namespace: elastic-system
    enable-leader-election: true
    elasticsearch-observation-interval: 10s
    ubi-only: false

5. ClusterRole — elastic-operator (Permissions)

This is the most important resource to understand. It defines exactly what the operator is allowed to do in your cluster. Every permission is here for a specific reason.

- apiGroups: [""]
  resources: [pods, events, persistentvolumeclaims,
              secrets, services, configmaps]
  verbs: [get, list, watch, create, update, patch, delete]

- apiGroups: ["apps"]
  resources: [deployments, statefulsets, daemonsets]
  verbs: [get, list, watch, create, update, patch, delete]

- apiGroups: ["policy"]
  resources: [poddisruptionbudgets]
  verbs: [get, list, watch, create, update, patch, delete]

- apiGroups: ["authorization.k8s.io"]
  resources: [subjectaccessreviews]
  verbs: [create]

- apiGroups: ["coordination.k8s.io"]
  resources: [leases]
  verbs: [create]

- apiGroups: ["coordination.k8s.io"]
  resources: [leases]
  resourceNames: [elastic-operator-leader]
  verbs: [get, watch, update]

- apiGroups: ["elasticsearch.k8s.elastic.co"]
  resources: [elasticsearches, elasticsearches/status, elasticsearches/finalizers]
  verbs: [get, list, watch, create, update, patch]

- apiGroups: ["storage.k8s.io"]
  resources: [storageclasses]
  verbs: [get, list, watch]

- apiGroups: ["admissionregistration.k8s.io"]
  resources: [validatingwebhookconfigurations]
  verbs: [get, list, watch, create, update, patch, delete]

- apiGroups: [""]
  resources: [nodes, endpoints]
  verbs: [get, list, watch]

6. Aggregate ClusterRoles — elastic-operator-view & elastic-operator-edit

These are not for the operator itself — they are for your team. They use Kubernetes role aggregationRole Aggregation
Kubernetes has built-in ClusterRoles: view, edit, and admin. When you add the label rbac.authorization.k8s.io/aggregate-to-view: "true" to a ClusterRole, its rules are automatically merged into the built-in view role. This means anyone with the view role can now also view Elasticsearch resources — no extra RoleBindings needed. so team members can work with Elastic resources using standard K8s roles.

7. ClusterRoleBinding — Connects Identity to Permissions

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: elastic-operator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: elastic-operator        # The ClusterRole defined above
subjects:
- kind: ServiceAccount
  name: elastic-operator         # The ServiceAccount defined above
  namespace: elastic-system

This is the glue that connects the ServiceAccount (identity, resource #2) to the ClusterRole (permissions, resource #5). Without this binding, the operator pod would have zero permissions and fail immediately on startup.

8. Service — elastic-webhook-server

apiVersion: v1
kind: Service
metadata:
  name: elastic-webhook-server
  namespace: elastic-system
spec:
  ports:
  - name: https
    port: 443
    targetPort: 9443
  selector:
    control-plane: elastic-operator

9. StatefulSet — The Operator Pod Itself

This is the heart of the deployment — the actual container that runs the ECK operator code.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elastic-operator
  namespace: elastic-system
spec:
  selector:
    matchLabels:
      control-plane: elastic-operator
  serviceName: elastic-operator
  replicas: 1

Pod Template — Annotations

template:
  metadata:
    annotations:
      "co.elastic.logs/raw": "[{\"type\":\"container\",...}]"
      "checksum/config": 3fdaff4a979089797cb469...

Pod Template — Security

spec:
  terminationGracePeriodSeconds: 10
  serviceAccountName: elastic-operator
  automountServiceAccountToken: true
  securityContext:
    runAsNonRoot: true
  containers:
  - name: manager
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop: [ALL]
      readOnlyRootFilesystem: true
      runAsNonRoot: true

Container — Image, Args, Env, Resources

containers:
- image: "docker.elastic.co/eck/eck-operator:3.0.0"
  imagePullPolicy: IfNotPresent
  name: manager
  args: ["manager", "--config=/conf/eck.yaml"]
  env:
  - name: OPERATOR_NAMESPACE
    valueFrom:
      fieldRef:
        fieldPath: metadata.namespace
  - name: POD_IP
    valueFrom:
      fieldRef:
        fieldPath: status.podIP
  - name: WEBHOOK_SECRET
    value: elastic-webhook-server-cert
  resources:
    limits:   { cpu: 1, memory: 1Gi }
    requests: { cpu: 100m, memory: 150Mi }

Volume Mounts

volumeMounts:
- mountPath: "/conf"
  name: conf
  readOnly: true
- mountPath: /tmp/k8s-webhook-server/serving-certs
  name: cert
  readOnly: true
volumes:
- name: conf
  configMap:
    name: elastic-operator
- name: cert
  secret:
    defaultMode: 420
    secretName: elastic-webhook-server-cert

10. ValidatingWebhookConfiguration — Request Interceptor

This resource tells the Kubernetes API server: "Before accepting any CREATE or UPDATE to an Elastic custom resource, send it to the webhook service for validation first."

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: elastic-webhook.k8s.elastic.co
webhooks:
- clientConfig:
    service:
      name: elastic-webhook-server
      namespace: elastic-system
      path: /validate-elasticsearch-k8s-elastic-co-v1-elasticsearch
  failurePolicy: Ignore
  name: elastic-es-validation-v1.k8s.elastic.co
  matchPolicy: Exact
  admissionReviewVersions: [v1, v1beta1]
  sideEffects: None
  rules:
  - apiGroups: ["elasticsearch.k8s.elastic.co"]
    apiVersions: ["v1"]
    operations: ["CREATE", "UPDATE"]
    resources: ["elasticsearches"]

Summary — How All 10 Resources Work Together

The flow is: you write Elasticsearch YAML → the webhook validates it → Kubernetes saves it → the operator pod (running as the ServiceAccount with ClusterRole permissions) detects the change → reads the ConfigMap for its settings → creates StatefulSets, Services, Secrets, and PDBs to make your cluster real.

Health & Status

# Cluster health (green/yellow/red)
GET _cluster/health

# Node info
GET _cat/nodes?v

# Shard allocation
GET _cat/shards?v

# Indices overview
GET _cat/indices?v&s=index

# Pending tasks
GET _cluster/pending_tasks

Index Management

# Create an index with settings and mappings
PUT /my-index
{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "title":     { "type": "text" },
      "status":    { "type": "keyword" },
      "timestamp": { "type": "date" },
      "count":     { "type": "integer" }
    }
  }
}

# Create an index template
PUT _index_template/logs-template
{
  "index_patterns": ["logs-*"],
  "template": {
    "settings": {
      "number_of_shards": 2,
      "number_of_replicas": 1
    },
    "mappings": {
      "properties": {
        "message":   { "type": "text" },
        "@timestamp": { "type": "date" },
        "level":     { "type": "keyword" }
      }
    }
  }
}

# Index Lifecycle Management (ILM) policy
PUT _ilm/policy/logs-policy
{
  "policy": {
    "phases": {
      "hot":    { "actions": { "rollover": { "max_size": "50GB", "max_age": "1d" }}},
      "warm":   { "min_age": "7d",  "actions": { "shrink": { "number_of_shards": 1 }}},
      "cold":   { "min_age": "30d", "actions": { "freeze": {} }},
      "delete": { "min_age": "90d", "actions": { "delete": {} }}
    }
  }
}

Essential Monitoring APIs

# Cluster health
GET _cluster/health

# Node stats (CPU, memory, disk, JVM)
GET _nodes/stats

# Index stats
GET /my-index/_stats

# Hot threads (find slow queries)
GET _nodes/hot_threads

# Task management (long-running operations)
GET _tasks?actions=*search&detailed

# Shard allocation explanation
GET _cluster/allocation/explain

Kibana Stack Monitoring

Enable monitoring in your ECK Elasticsearch resource to see cluster dashboards in Kibana:

spec:
  monitoring:
    metrics:
      elasticsearchRefs:
        - name: quickstart   # Send metrics to same cluster (or a dedicated monitoring cluster)
    logs:
      elasticsearchRefs:
        - name: quickstart

ECK Security (Enabled by Default)

ECK automatically enables security with TLS on both HTTP and transport layers. Key things it manages:

# ECK creates these automatically:
# - Self-signed CA certificate
# - Node transport certificates
# - HTTP certificates
# - elastic superuser with random password

# Retrieve the elastic user password:
kubectl get secret quickstart-es-elastic-user \
  -o go-template='{{.data.elastic | base64decode}}'

# Create additional users via the API:
POST _security/user/my-app-user
{
  "password": "secure-password-here",
  "roles": ["reader"],
  "full_name": "App Service Account"
}

# Create a role
POST _security/role/reader
{
  "indices": [
    {
      "names": ["logs-*"],
      "privileges": ["read", "view_index_metadata"]
    }
  ]
}

Exam Domain Breakdown

Data Management

• Create an index with custom settings, mappings, and aliases
• Define and use index templates with patterns
• Create and use component templates
• Define dynamic templates for automatic field mapping
• Set up an ILM policy (hot → warm → cold → delete)
• Create a data stream with matching index template
• Use the Reindex API to migrate data between indices
• Use Update By Query to modify documents in place
• Configure nested object relationships in mappings

Searching Data

• Write term-level queries (term, range, exists, wildcard)
• Write full-text queries (match, multi_match, match_phrase)
• Build bool queries with must, should, filter, must_not
• Execute metric aggregations (avg, sum, min, max, cardinality)
• Execute bucket aggregations (terms, date_histogram, range)
• Write pipeline aggregations (derivative, cumulative_sum)
• Build nested sub-aggregations
• Set up and use async search
• Configure and query across multiple clusters
• Define and use runtime fields in search

Search Applications & Data Processing

• Implement search highlighting
• Sort results by field, score, geo_distance
• Implement pagination (from/size and search_after)
• Create and test ingest pipelines
• Use grok, date, geoip, set, rename, and script processors
• Set up an enrich policy and processor

Cluster Management & ECK

• Diagnose and fix yellow/red cluster health
• Configure shard allocation awareness and filtering
• Set up snapshot repository and take/restore snapshots
• Configure cross-cluster replication (CCR)
• Deploy Elasticsearch on Kubernetes with ECK
• Manage ECK upgrades and scaling operations
• Configure node roles and resource allocation
• Set up security: users, roles, TLS