Practical guidance on rate limiting Temporal.io workers and REST APIs (Java, Python, Go), mutual TLS,
hardening container hosts, securing Docker with TLS, verifying client-supplied data integrity, and configuring
container networks. Tap any topic to expand, and use the More Info buttons for references.
⚙️
▶
Rate Limiting Temporal Workers
Java calls, task queue limits, and concurrency controls
Temporal rate limiting operates at several layers. Choose the right one for your goal:
Task queue RPS — protects downstream APIs
Per-worker RPS — throttles a single worker instance
Harden the underlying host running your containers
A compromised host compromises every container on it — namespaces and cgroups are isolation boundaries, not security boundaries. Each principle below reduces what an attacker can do after they land.
1. Use a minimal, container-optimized OS
A full Ubuntu/RHEL server has 500+ packages you don't need. Container-optimized OSes ship only a kernel, container runtime, and the bare minimum to boot. No package manager at runtime, immutable root filesystem, atomic A/B updates, and automatic reboots.
Migration tip: start with one non-critical node pool on the new OS before moving production.
2. Remove unused packages, compilers, and interpreters
If gcc, python3, perl, curl, or wget are installed, an attacker can build their next stage right on your box. Strip anything not required at runtime.
# Step 1 — Audit what's installed
dpkg-query -W -f='${Installed-Size}\t${Package}\n' | sort -n | tail -40
# or RHEL: rpm -qa --queryformat '%{SIZE}\t%{NAME}\n' | sort -n
# Step 2 — Find shells, compilers, network tools
dpkg -l | grep -E 'gcc|make|python|perl|ruby|curl|wget|nc|tcpdump|gdb|strace'
# Step 3 — Remove them (host only — containers have their own copies)
apt-get purge -y gcc make build-essential python3 perl \
tcpdump netcat-openbsd nmap
apt-get autoremove --purge -y
apt-get clean
# Step 4 — Block future installs by removing apt/dnf on immutable OSes
# (Bottlerocket/Talos do this by default — there is no package manager)
# Step 5 — Scan for setuid binaries (prime targets for privesc)
find / -xdev -perm -4000 -type f 2>/dev/null
# Step 6 — Remove setuid from binaries you don't need elevated
chmod u-s /usr/bin/chsh /usr/bin/chfn /usr/bin/newgrp
⚠️ Don't remove tools your monitoring/backup agents depend on. Check systemctl list-dependencies first.
3. Disable unused services and close unneeded ports
Every listening service is an entry point. Print, mail, RPC, Avahi, CUPS — none belong on a container host. Verify with a port scan from another machine, not just locally.
# Step 1 — Enumerate listening services
ss -tulnp
systemctl list-unit-files --state=enabled --type=service
# Step 2 — Disable common offenders
for svc in avahi-daemon cups bluetooth ModemManager \
rpcbind postfix nfs-server smbd; do
systemctl disable --now "$svc" 2>/dev/null || true
done
# Step 3 — Default-deny firewall (nftables preferred)
cat > /etc/nftables.conf <<'EOF'
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
iif lo accept
ct state established,related accept
tcp dport 22 accept comment "SSH from jump host only — see fail2ban"
tcp dport { 80, 443 } accept
icmp type echo-request limit rate 5/second accept
}
chain forward { type filter hook forward priority 0; policy drop; }
chain output { type filter hook output priority 0; policy accept; }
}
EOF
systemctl enable --now nftables
# Step 4 — Verify from OUTSIDE the host (localhost lies)
nmap -sS -p- -T4 # run from jump host
# Expected: only 22, 80, 443 open
4. Apply CIS Docker Benchmark recommendations
The CIS Docker Benchmark is ~200 concrete checks: daemon config, file permissions, container runtime flags. Don't read it top-to-bottom — run an automated audit and fix the findings.
For Kubernetes, the equivalent is kube-bench (Aqua Security). Run it on every control-plane and worker node.
5. Enable SELinux or AppArmor mandatory access control
DAC (standard Unix permissions) relies on root being trustworthy. MAC enforces policy even against root. SELinux is the default on RHEL/Fedora; AppArmor on Ubuntu/Debian/SUSE. Never run in permissive or disabled mode in production.
# --- SELinux (RHEL/Fedora) ---
# Step 1 — Verify it's enforcing
sestatus # Status: enabled / Mode: enforcing
getenforce # Enforcing
# Step 2 — Force enforcing in /etc/selinux/config
SELINUX=enforcing
SELINUXTYPE=targeted
# Step 3 — Run containers with SELinux labels
docker run --security-opt label=type:container_t myapp
# or in compose:
# security_opt: ["label=type:container_t"]
# Step 4 — Investigate denials (don't just disable)
ausearch -m avc -ts recent
audit2allow -a -M mypolicy # generate a targeted policy module
semodule -i mypolicy.pp
# --- AppArmor (Ubuntu/Debian) ---
# Step 1 — Verify it's loaded
aa-status
# Step 2 — Docker ships a default profile; use it
docker run --security-opt apparmor=docker-default myapp
# Step 3 — Write a stricter custom profile
cat > /etc/apparmor.d/docker-myapp <<'EOF'
#include
profile docker-myapp flags=(attach_disconnected,mediate_deleted) {
#include
network inet tcp,
/app/** r,
/app/data/** rw,
deny /etc/shadow r,
deny /proc/*/mem rw,
deny capability sys_admin,
}
EOF
apparmor_parser -r /etc/apparmor.d/docker-myapp
docker run --security-opt apparmor=docker-myapp myapp
⚠️ The common anti-pattern is disabling SELinux because "something broke". Instead, generate a targeted policy from the denial logs. Disabling MAC for convenience is how kernel exploits become cluster-wide breaches.
6. Keep kernel and runtime patched; automate updates
Kernel CVEs (Dirty Pipe, Dirty COW, OverlayFS privesc) routinely break container isolation. Manual patching doesn't scale — automate it, stagger reboots across zones, and use live-kernel-patching where supported.
# Ubuntu — unattended-upgrades for security patches
apt-get install -y unattended-upgrades
cat > /etc/apt/apt.conf.d/50unattended-upgrades <<'EOF'
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}-security";
};
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "03:00";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
EOF
systemctl enable --now unattended-upgrades
# RHEL — dnf-automatic
dnf install -y dnf-automatic
sed -i 's/^apply_updates = no/apply_updates = yes/' \
/etc/dnf/automatic.conf
systemctl enable --now dnf-automatic.timer
# Live kernel patching (no reboot needed for many CVEs)
# Ubuntu Pro: Livepatch
canonical-livepatch enable
canonical-livepatch status
# RHEL: kpatch
dnf install -y kpatch
systemctl enable --now kpatch
# Immutable OSes — atomic A/B updates (Bottlerocket example)
apiclient update check
apiclient update apply --reboot
# Rollback is one command if the new image fails health checks
# Kubernetes fleet — stagger reboots so you don't drain everything at once
# kured (Kubernetes Reboot Daemon)
kubectl apply -f https://github.com/kubereboot/kured/releases/latest/download/kured.yaml
SSH brute-force is constant background noise on any internet-facing port. Move access behind a single hardened bastion (or better, a zero-trust broker), disable password auth entirely, and audit every session.
# Step 1 — /etc/ssh/sshd_config hardening
Port 22
Protocol 2
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
KbdInteractiveAuthentication no
PermitEmptyPasswords no
MaxAuthTries 3
LoginGraceTime 30
ClientAliveInterval 300
ClientAliveCountMax 2
AllowUsers deploy
AllowGroups sshusers
# Modern crypto only
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com
# Audit:
LogLevel VERBOSE
sshd -t && systemctl reload sshd
# Step 2 — Only allow SSH from the jump host
# nftables: replace the earlier SSH rule
nft add rule inet filter input ip saddr 10.0.0.10/32 tcp dport 22 accept
# Step 3 — Use SSH certificates (scales better than authorized_keys)
# On CA host:
ssh-keygen -t ed25519 -f /etc/ssh/ca
# On each server:
echo "TrustedUserCAKeys /etc/ssh/ca.pub" >> /etc/ssh/sshd_config
# Sign a user key (time-bounded, principals = usernames)
ssh-keygen -s /etc/ssh/ca -I alice@corp -n deploy \
-V +8h alice_ed25519.pub
# Step 4 — Better: use a zero-trust SSH broker
# Teleport, Tailscale SSH, AWS SSM Session Manager, GCP IAP
# These eliminate standing SSH ports entirely
aws ssm start-session --target i-0abc123 # no port 22 open at all
# Step 5 — fail2ban as defense-in-depth
apt-get install -y fail2ban
cat > /etc/fail2ban/jail.d/sshd.local <<'EOF'
[sshd]
enabled = true
maxretry = 3
findtime = 10m
bantime = 1h
EOF
systemctl enable --now fail2ban
8. Run containers as non-root users wherever possible
Container root is host root if a kernel exploit or mount escape succeeds. Running as a non-root UID — ideally combined with user namespaces and a read-only root filesystem — turns "full host compromise" into "confined user-level access".
# Step 1 — Build images with a non-root USER
# Dockerfile
FROM node:20-alpine
RUN addgroup -g 10001 app && adduser -u 10001 -G app -s /sbin/nologin -D app
WORKDIR /app
COPY --chown=app:app . .
RUN npm ci --omit=dev
USER 10001 # numeric — K8s can verify
EXPOSE 8080
CMD ["node", "server.js"]
# Step 2 — Enforce at runtime
docker run --user 10001:10001 \
--read-only \
--tmpfs /tmp:rw,noexec,nosuid,size=64m \
--cap-drop=ALL \
--cap-add=NET_BIND_SERVICE \
--security-opt=no-new-privileges \
myapp
# Step 3 — Kubernetes Pod securityContext
apiVersion: v1
kind: Pod
spec:
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile: { type: RuntimeDefault }
containers:
- name: app
image: myapp:1.2.3
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: { drop: ["ALL"] }
# Step 4 — Enforce cluster-wide with Pod Security Admission
kubectl label ns prod \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/enforce-version=latest
# Step 5 — Docker daemon user namespace remap (container root != host root)
# /etc/docker/daemon.json
{ "userns-remap": "default" }
systemctl restart docker
# Now UID 0 inside the container = UID 100000 on the host
If you inherit an image that insists on root, run it under userns-remap as a compensating control while you fix the image.
🔐
▶
Configure Docker TLS Authentication
Secure the Docker daemon against remote takeover
The Docker daemon socket is effectively root on the host. Never expose it on a TCP port
without mutual TLS. If you must expose it remotely, require client certificate auth.
⚠️ Port 2376 = TLS. Port 2375 = plaintext. Never use 2375 except on a loopback interface in development.
✅
▶
Verify Client-Supplied Read-Only Data
Trust nothing from the client — verify integrity
Any value that round-trips through the client — cookies, hidden form fields, JWTs, download manifests, CDN scripts, container images — must be verified server-side. The client is adversarial by default.
1. HMAC signatures over any value you issued to the client
If you put a user ID, cart total, or price in a cookie or hidden field, sign it with HMAC-SHA256. On return, recompute the signature with your server secret and reject if it doesn't match. Always use constant-time comparison to prevent timing attacks.
// Java — sign + verify with constant-time compare
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.security.MessageDigest;
import java.util.Base64;
byte[] secret = System.getenv("HMAC_SECRET").getBytes(UTF_8);
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(secret, "HmacSHA256"));
// Sign (server issues this to the client)
String payload = "user=42&cart=99.00&exp=1714500000";
byte[] sig = mac.doFinal(payload.getBytes(UTF_8));
String token = payload + "." + Base64.getUrlEncoder().encodeToString(sig);
// Verify (when the client sends it back)
String[] parts = token.split("\\.", 2);
byte[] expected = mac.doFinal(parts[0].getBytes(UTF_8));
byte[] provided = Base64.getUrlDecoder().decode(parts[1]);
if (!MessageDigest.isEqual(expected, provided)) { // constant-time!
throw new SecurityException("HMAC verification failed");
}
// ALSO check exp — a valid signature on an old payload is still valid
// Python — hmac.compare_digest is constant-time
import hmac, hashlib, os, base64
secret = os.environb[b"HMAC_SECRET"]
sig = hmac.new(secret, payload.encode(), hashlib.sha256).digest()
if not hmac.compare_digest(sig, provided):
raise PermissionError("HMAC mismatch")
# Go — subtle.ConstantTimeCompare
import ("crypto/hmac"; "crypto/sha256"; "crypto/subtle")
h := hmac.New(sha256.New, secret)
h.Write([]byte(payload))
if subtle.ConstantTimeCompare(h.Sum(nil), provided) != 1 {
return errors.New("bad signature")
}
⚠️ A valid HMAC proves integrity, not freshness. Always include and check an expiration (exp) and/or a nonce to prevent replay.
2. Signed JWTs with strong algorithms — reject alg: none
JWT's algorithm flexibility is a footgun. The three classic attacks: (a) alg: none — accepted as "valid" by naive libraries; (b) algorithm confusion — attacker changes RS256 to HS256, then signs the token with the server's public key as the HMAC secret; (c) weak keys. Use EdDSA (Ed25519) or RS256 with ≥2048-bit keys, and always verify against an allowlist of algorithms.
// Java — Nimbus JOSE, EdDSA (Ed25519)
import com.nimbusds.jwt.*;
import com.nimbusds.jose.*;
import com.nimbusds.jose.crypto.*;
// Verify — pin allowed algorithm, do NOT accept what the header says blindly
JWSVerifier verifier = new Ed25519Verifier(publicJwk);
SignedJWT jwt = SignedJWT.parse(tokenString);
if (jwt.getHeader().getAlgorithm() != JWSAlgorithm.EdDSA) {
throw new SecurityException("Unexpected alg: "
+ jwt.getHeader().getAlgorithm()); // blocks 'none', HS256 confusion
}
if (!jwt.verify(verifier)) throw new SecurityException("bad sig");
JWTClaimsSet claims = jwt.getJWTClaimsSet();
Instant now = Instant.now();
if (claims.getExpirationTime().toInstant().isBefore(now)) throw ...;
if (!"https://issuer.example.com".equals(claims.getIssuer())) throw ...;
if (!claims.getAudience().contains("my-api")) throw ...;
# Python — PyJWT with explicit algorithms list (never pass None)
import jwt
claims = jwt.decode(
token,
public_key,
algorithms=["EdDSA"], # ALLOWLIST — not the token's self-declared alg
audience="my-api",
issuer="https://issuer.example.com",
options={"require": ["exp", "iat", "iss", "aud"]},
)
# Go — golang-jwt with keyfunc guarding the alg
tok, err := jwt.Parse(raw, func(t *jwt.Token) (any, error) {
if t.Method.Alg() != "EdDSA" { // reject anything else
return nil, fmt.Errorf("unexpected alg: %s", t.Method.Alg())
}
return publicKey, nil
})
// ❌ NEVER do this — trusts attacker-controlled header
jwt.Parse(raw, func(t *jwt.Token) (any, error) { return key, nil })
⚠️ Verify iss, aud, exp, and nbf on every request. A valid signature on a token issued for a different app or user is still a valid signature.
3. Content hashes (SHA-256) for downloaded artifacts
Any binary, tarball, or dataset pulled at install or runtime must have its SHA-256 verified against a known-good value shipped out of band (e.g., committed in your repo, in a signed release manifest). Mirror compromises and MITM attacks are the real threat model here.
# Shell — verify a downloaded archive
EXPECTED="9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
curl -fsSL -o app.tar.gz https://releases.example.com/app-1.2.3.tar.gz
echo "${EXPECTED} app.tar.gz" | sha256sum -c - # exits non-zero on mismatch
# Dockerfile — pin by digest, verify downloads
FROM debian:bookworm-slim@sha256:2bc5715... # digest, not floating tag
ARG TERRAFORM_VERSION=1.9.5
ARG TERRAFORM_SHA256=abc123...
RUN curl -fsSLo tf.zip \
https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip \
&& echo "${TERRAFORM_SHA256} tf.zip" | sha256sum -c - \
&& unzip tf.zip -d /usr/local/bin/ && rm tf.zip
# Python — verify before use (streams large files)
import hashlib, sys
h = hashlib.sha256()
with open("app.tar.gz", "rb") as f:
for chunk in iter(lambda: f.read(1 << 20), b""):
h.update(chunk)
if h.hexdigest() != expected:
sys.exit("checksum mismatch — refusing to proceed")
# Go — streaming SHA-256 with constant-time compare
import ("crypto/sha256"; "crypto/subtle"; "encoding/hex"; "io"; "os")
f, _ := os.Open("app.tar.gz")
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil { log.Fatal(err) }
got := h.Sum(nil)
want, _ := hex.DecodeString(expected)
if subtle.ConstantTimeCompare(got, want) != 1 {
log.Fatal("checksum mismatch — refusing to proceed")
}
// Java — streaming SHA-256
MessageDigest md = MessageDigest.getInstance("SHA-256");
try (var in = Files.newInputStream(Path.of("app.tar.gz"));
var dis = new DigestInputStream(in, md)) {
dis.transferTo(OutputStream.nullOutputStream());
}
byte[] got = md.digest();
byte[] want = HexFormat.of().parseHex(expected);
if (!MessageDigest.isEqual(got, want)) {
throw new SecurityException("checksum mismatch — refusing to proceed");
}
# Go modules do this for you via go.sum — don't delete it!
# pip: use --require-hashes in requirements.txt
requests==2.32.3 \
--hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6
# Maven — enforce dependency checksums with the checksum plugin
mvn verify -Dchecksum.failOnError=true
4. Subresource Integrity (SRI) for scripts loaded from CDNs
When your page includes <script src="https://cdn.example.com/lib.js">, you're trusting that CDN with arbitrary code execution in your users' browsers. SRI pins a hash — if the CDN is compromised or MITM'd, the browser refuses to execute the modified file.
<!-- Step 1 — Generate the hash -->
# CLI
curl -sSL https://cdn.example.com/lib@1.2.3/lib.min.js \
| openssl dgst -sha384 -binary | openssl base64 -A
# → paste into integrity="sha384-..."
<!-- Step 2 — Add integrity + crossorigin attributes -->
<script src="https://cdn.example.com/lib@1.2.3/lib.min.js"
integrity="sha384-oqVuAfXRKap7fdgcCY5uykM6+R9GqQ8K/uxy9rx7HNQlGYl1kPzQho1wx4JwY8wC"
crossorigin="anonymous"
referrerpolicy="no-referrer"></script>
<!-- Step 3 — Same for stylesheets -->
<link rel="stylesheet"
href="https://cdn.example.com/style.css"
integrity="sha384-9jYlQMY08x6yM9IeIkSL+qMEaLCNlB0LqFzH2I8GzXbq7h8ZqSqXJPmj+tQTI3cY"
crossorigin="anonymous">
<!-- Step 4 — Enforce cluster-wide with CSP -->
Content-Security-Policy:
default-src 'self';
script-src 'self' 'require-sri-for script';
style-src 'self' 'require-sri-for style';
<!-- Step 5 — Verify: tamper with the file and confirm the browser blocks it -->
# DevTools console will show:
# "Failed to find a valid digest in the 'integrity' attribute for
# resource 'https://cdn.example.com/lib.js' with computed SHA-384 digest '...'"
SRI pins an exact file — bumping the CDN version requires regenerating the hash. Automate this in your build (webpack subresource-integrity-webpack-plugin, Vite vite-plugin-sri).
5. Cosign / Sigstore for container image signatures
A tag like myapp:v1.2.3 is mutable — registry compromise or typosquatting can feed you a malicious image. Cosign signs the image digest with a keypair (or, better, keyless via OIDC identity), stores the signature alongside the image, and your admission controller refuses anything unsigned.
# --- Keyed signing (simple, but you manage the key) ---
cosign generate-key-pair # produces cosign.key + cosign.pub
export COSIGN_PASSWORD=...
cosign sign --key cosign.key \
registry.example.com/app@sha256:abc123... # sign the DIGEST, not a tag
cosign verify --key cosign.pub \
registry.example.com/app@sha256:abc123...
# --- Keyless signing (preferred — no long-lived keys) ---
# Identity comes from your OIDC provider (GitHub Actions, GCP, etc.)
# In a GitHub Action:
- uses: sigstore/cosign-installer@v3
- run: cosign sign --yes \
--oidc-issuer=https://token.actions.githubusercontent.com \
registry.example.com/app@${DIGEST}
# Verify keyless — pin WHO signed it and from WHAT workflow
cosign verify \
--certificate-identity-regexp='https://github.com/myorg/.*' \
--certificate-oidc-issuer=https://token.actions.githubusercontent.com \
registry.example.com/app@sha256:abc123...
# --- Kubernetes enforcement (Kyverno policy) ---
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-cosign-signed-images
spec:
validationFailureAction: Enforce
rules:
- name: verify-images
match:
any:
- resources: { kinds: [Pod] }
verifyImages:
- imageReferences: ["registry.example.com/*"]
attestors:
- entries:
- keyless:
subject: "https://github.com/myorg/*/.github/workflows/release.yml@refs/heads/main"
issuer: "https://token.actions.githubusercontent.com"
# --- Also attach and verify SBOMs + attestations ---
cosign attach sbom --sbom sbom.spdx.json registry.example.com/app@${DIGEST}
cosign attest --predicate provenance.json --type slsaprovenance \
--key cosign.key registry.example.com/app@${DIGEST}
⚠️ Always sign and verify by digest (@sha256:...), never by tag. Tags are mutable — an attacker with push access can retag over a signed image.
6. Docker Content Trust (DOCKER_CONTENT_TRUST=1)
DCT is Docker's original signing mechanism (based on The Update Framework/Notary v1). With it enabled, docker pull and docker run refuse unsigned images. It's still useful for Docker Hub's Official Images, but Sigstore/Cosign is the modern successor — Notary v1 has been deprecated in favor of Notary v2 and Sigstore ecosystems.
# uvicorn ships with client-cert support since 0.27
uvicorn app:app \
--host 0.0.0.0 --port 8443 \
--ssl-keyfile server.key \
--ssl-certfile server.crt \
--ssl-ca-certs ca.pem \
--ssl-cert-reqs 2 # 2 = CERT_REQUIRED
# Or programmatically with ssl.SSLContext
import ssl, uvicorn
ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH, cafile="ca.pem")
ctx.load_cert_chain("server.crt", "server.key")
ctx.verify_mode = ssl.CERT_REQUIRED
ctx.minimum_version = ssl.TLSVersion.TLSv1_3
uvicorn.run("app:app", host="0.0.0.0", port=8443, ssl=ctx)
# Inside FastAPI — pull client identity from the TLS layer
from fastapi import FastAPI, Request, HTTPException
app = FastAPI()
@app.middleware("http")
async def check_client_identity(request: Request, call_next):
# Starlette exposes peer cert via scope["extensions"]
peer = request.scope.get("extensions", {}).get("tls", {}).get("client_cert_dict")
if not peer or peer.get("subject", [[("commonName", "")]])[0][0][1] != "service-a":
raise HTTPException(403, "client identity mismatch")
return await call_next(request)
Python — mTLS client (requests / httpx)
import requests
r = requests.get("https://api.internal:8443/",
cert=("client.crt", "client.key"),
verify="ca.pem") # pin internal CA — don't use system roots
# Async with httpx
import httpx, ssl
ctx = ssl.create_default_context(cafile="ca.pem")
ctx.load_cert_chain("client.crt", "client.key")
ctx.minimum_version = ssl.TLSVersion.TLSv1_3
async with httpx.AsyncClient(verify=ctx) as c:
r = await c.get("https://api.internal:8443/")
Java — mTLS server (Spring Boot) + client
# application.yml — enforce client cert
server:
port: 8443
ssl:
enabled: true
key-store: classpath:server-keystore.p12
key-store-password: ${KEYSTORE_PW}
key-store-type: PKCS12
trust-store: classpath:truststore.p12 # contains internal CA
trust-store-password: ${TRUSTSTORE_PW}
client-auth: need # REQUIRED, not 'want'
enabled-protocols: TLSv1.3
ciphers: TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256
// Retrieve client identity in a controller
@GetMapping("/whoami")
public String whoami(HttpServletRequest req) {
X509Certificate[] certs = (X509Certificate[]) req.getAttribute(
"jakarta.servlet.request.X509Certificate");
if (certs == null || certs.length == 0)
throw new AccessDeniedException("no client cert");
return certs[0].getSubjectX500Principal().getName();
}
// Java — mTLS client with OkHttp
KeyStore ks = KeyStore.getInstance("PKCS12");
try (var in = new FileInputStream("client.p12")) {
ks.load(in, "changeit".toCharArray());
}
KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(ks, "changeit".toCharArray());
KeyStore trust = KeyStore.getInstance("PKCS12");
try (var in = new FileInputStream("truststore.p12")) {
trust.load(in, "changeit".toCharArray());
}
TrustManagerFactory tmf = TrustManagerFactory.getInstance("PKIX");
tmf.init(trust);
SSLContext ctx = SSLContext.getInstance("TLSv1.3");
ctx.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
OkHttpClient client = new OkHttpClient.Builder()
.sslSocketFactory(ctx.getSocketFactory(),
(X509TrustManager) tmf.getTrustManagers()[0])
.build();
Service mesh option
For fleet-wide mTLS, offload to a mesh: Istio, Linkerd, or Consul Connect give automatic cert issuance, rotation, and identity enforcement without app changes.
☕
▶
Rate Limiting Java APIs
Spring Boot, Resilience4j, Bucket4j
Bucket4j — token bucket; supports Redis/Hazelcast for distributed limits
Resilience4j RateLimiter — lightweight, integrates with Spring & reactive stacks
Spring Cloud Gateway — built-in RequestRateLimiter filter with Redis
Key by API key / user ID / tenant — not just IP (NAT hides real clients)
Return 429 with Retry-After and RateLimit-* headers (RFC 9331)
Apply limits before expensive work (auth, DB) in filter chain
For gRPC: grpc-ecosystem/go-grpc-middleware/ratelimit
Patterns
One limiter per identity key, stored in sync.Map or LRU cache
Evict idle limiters to prevent memory growth
Use limiter.Wait(ctx) for outbound calls (respects deadline)
Use Allow() for inbound — fail fast with 429
Set CPU/memory-aware burst — don't over-provision tokens
// Per-API-key middleware using x/time/rate
type keyedLimiter struct {
mu sync.Mutex
limiters map[string]*rate.Limiter
}
func (k *keyedLimiter) get(id string) *rate.Limiter {
k.mu.Lock(); defer k.mu.Unlock()
if l, ok := k.limiters[id]; ok { return l }
l := rate.NewLimiter(rate.Limit(10), 20) // 10 rps, burst 20
k.limiters[id] = l
return l
}
func Middleware(k *keyedLimiter) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
id := r.Header.Get("X-API-Key")
if id == "" { id = r.RemoteAddr }
if !k.get(id).Allow() {
w.Header().Set("Retry-After", "1")
http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
return
}
next.ServeHTTP(w, r)
})
}
}
🌐
▶
Configure Container Networking Securely
Isolation, segmentation, and least-privilege connectivity
Each principle below is a common failure mode. The examples show the wrong way and the right way side by side.
1. Never use the default bridge for production
The default bridge network provides no DNS-based service discovery, exposes all containers to each other by default, and gives you no isolation between unrelated apps. Every docker run without --network lands here.
# ❌ BAD — lands on default bridge, talks to everything
docker run -d --name api myapi:latest
docker run -d --name sketchy-thirdparty some/image
# ✅ GOOD — isolated user-defined network with DNS
docker network create app-net
docker run -d --name api --network app-net myapi:latest
docker run -d --name db --network app-net postgres:16
# api can resolve "db" by name; unrelated containers can't reach either
2. One user-defined network per application tier
Segment by tier (frontend / backend / data). Containers only join the networks they actually need. A compromised frontend container cannot reach the database directly — it must go through the API tier.
# Create tier-specific networks
docker network create frontend-net
docker network create backend-net
docker network create data-net
# nginx: only frontend
docker run -d --name nginx --network frontend-net -p 443:443 nginx
# api: bridges frontend <-> backend, but NOT data
docker run -d --name api --network frontend-net myapi:latest
docker network connect backend-net api
# worker: backend + data
docker run -d --name worker --network backend-net myworker:latest
docker network connect data-net worker
# postgres: data only — unreachable from nginx or api directly
docker run -d --name db --network data-net postgres:16
3. Disable inter-container comms (icc=false) by default
With ICC enabled, any container on a bridge can open a socket to any other on the same bridge. Disabling ICC forces you to explicitly allowlist connections, turning lateral movement from "free" into "impossible without a rule".
# Daemon-wide: /etc/docker/daemon.json
{
"icc": false,
"iptables": true
}
# Restart dockerd
# Per-network (preferred — keeps other networks functional)
docker network create \
--driver bridge \
--opt com.docker.network.bridge.enable_icc=false \
locked-net
# Now containers on locked-net can only talk if you
# explicitly link them or open a port between them
docker run -d --name a --network locked-net alpine sleep 1d
docker run -d --name b --network locked-net alpine sleep 1d
docker exec a ping -c1 b # ❌ fails — ICC disabled
4. Bind ports to 127.0.0.1 unless external access is required
Plain -p 8080:8080 binds to 0.0.0.0 — every interface, including public ones. And Docker punches through host firewalls (UFW, firewalld) via its own iptables chain. The fix is to bind explicitly.
# ❌ BAD — exposed on every host interface, bypasses UFW
docker run -d -p 5432:5432 postgres:16
docker run -d -p 6379:6379 redis:7
# ✅ GOOD — loopback only, reachable via SSH tunnel or reverse proxy
docker run -d -p 127.0.0.1:5432:5432 postgres:16
docker run -d -p 127.0.0.1:6379:6379 redis:7
# ✅ Also good — bind to a private interface only
docker run -d -p 10.0.1.5:8080:8080 myapi:latest
# docker-compose.yml equivalent
services:
db:
image: postgres:16
ports:
- "127.0.0.1:5432:5432" # NOT "5432:5432"
⚠️ Thousands of Redis and Mongo instances get pwned every year because someone wrote -p 6379:6379 on a cloud VM with a public IP.
5. Use internal networks for DB + app tier
An --internal network has no route to the outside world. Databases cannot phone home, can't download malicious payloads if compromised, and cannot leak data over HTTPS to an attacker's server.
# Backend network — no external routing
docker network create --internal backend-net
# Frontend network — has external route (for the API's outbound HTTPS)
docker network create frontend-net
# DB: only on the internal network — no egress possible
docker run -d --name db --network backend-net postgres:16
# API: bridges both — can reach DB and also external APIs it needs
docker run -d --name api \
--network backend-net \
-p 127.0.0.1:8080:8080 \
myapi:latest
docker network connect frontend-net api
# Proof: DB cannot resolve or reach the internet
docker exec db curl -m3 https://example.com # ❌ fails (no route)
6. Apply Kubernetes NetworkPolicies / Cilium for east-west control
By default, every pod in a Kubernetes cluster can reach every other pod. A single compromised pod = full lateral movement. NetworkPolicies flip this to default-deny; Cilium adds L7 (HTTP method, path) and identity-based policy via eBPF.
# Step 1 — default deny all ingress + egress in the namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: prod
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
---
# Step 2 — allow only api -> db on port 5432
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-to-db
namespace: prod
spec:
podSelector:
matchLabels: { app: postgres }
policyTypes: [Ingress]
ingress:
- from:
- podSelector:
matchLabels: { app: api }
ports:
- protocol: TCP
port: 5432
---
# Cilium L7 policy — API can GET /users but not DELETE
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: api-l7
spec:
endpointSelector:
matchLabels: { app: users-svc }
ingress:
- fromEndpoints:
- matchLabels: { app: api }
toPorts:
- ports:
- { port: "8080", protocol: TCP }
rules:
http:
- method: "GET"
path: "/users(/.*)?"
7. Egress filtering — not just ingress
Most security rules block incoming traffic. But every real-world breach ends with outgoing traffic: data exfiltration, C2 callbacks, cryptominer pool connections. Restrict egress to the specific hosts each workload actually needs.
# Kubernetes — allow egress only to kube-dns + specific API
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-egress-allowlist
namespace: prod
spec:
podSelector:
matchLabels: { app: api }
policyTypes: [Egress]
egress:
# DNS
- to:
- namespaceSelector: { matchLabels: { name: kube-system } }
podSelector: { matchLabels: { k8s-app: kube-dns } }
ports: [{ protocol: UDP, port: 53 }]
# Internal postgres
- to:
- podSelector: { matchLabels: { app: postgres } }
ports: [{ protocol: TCP, port: 5432 }]
# One specific external API (via CIDR — for FQDN use Cilium)
- to:
- ipBlock:
cidr: 52.94.0.0/16 # e.g. AWS API range
ports: [{ protocol: TCP, port: 443 }]
---
# Cilium CNP — FQDN-based egress allowlist (L7 aware)
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: api-fqdn-egress
spec:
endpointSelector:
matchLabels: { app: api }
egress:
- toFQDNs:
- matchName: "api.stripe.com"
- matchPattern: "*.s3.amazonaws.com"
toPorts:
- ports: [{ port: "443", protocol: TCP }]
For Docker standalone, combine --internal networks with an egress proxy (e.g., Squid) that allowlists destinations.
8. Disable IPv6 if unused — it often bypasses v4 firewall rules
Many admins write iptables rules and forget ip6tables. Containers pick up link-local IPv6 addresses automatically, and services bound to :: are reachable over v6 even when v4 is locked down. Either fully configure both stacks, or disable v6 entirely.
# Disable IPv6 at the kernel (/etc/sysctl.d/99-no-ipv6.conf)
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
# Apply: sysctl --system
# Or keep v6 but mirror every v4 rule in ip6tables
ip6tables -P INPUT DROP
ip6tables -P FORWARD DROP
ip6tables -A INPUT -i lo -j ACCEPT
ip6tables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# Docker daemon: explicitly disable v6 if you don't need it
# /etc/docker/daemon.json
{
"ipv6": false,
"fixed-cidr-v6": ""
}
# Kubernetes: verify you're running single-stack if that's the intent
kubectl get nodes -o jsonpath='{.items[*].spec.podCIDRs}'
# Should show only IPv4 CIDRs if single-stack
⚠️ Test with ss -tlnp6 and nmap -6 ::1 to catch v6-bound services you didn't know about.