NetworkPolicy Enforcement Deep Dive: eBPF (Cilium) vs iptables (Calico)

Kubernetes NetworkPolicy is the standard API for controlling traffic between pods — but the API spec is just a contract. How that contract gets enforced depends entirely on which CNI plugin you choose, and the performance difference between implementations is dramatic at scale.

This article dives into the two dominant enforcement approaches: eBPF-based (Cilium) and iptables-based (Calico). We cover how each works under the hood, real performance benchmarks at 100 to 10,000 policies, L7 policy realities, and practical debugging patterns.

How Each CNI Enforces Policies

Cilium: eBPF in the Kernel

Cilium attaches eBPF programs at multiple hook points in the Linux kernel. Every policy decision happens in kernel space — no userspace proxy, no iptables traversal, no context switches.

Hook points (in order of packet processing):

Hook Point	Location	What it does
XDP	NIC driver level	Earliest possible interception. Drops denied packets before they enter the kernel network stack. Requires native XDP-capable NIC driver (Mellanox ConnectX, Intel XL710, bare metal).
TC ingress	veth interface	L3/L4 policy enforcement for pod-to-pod traffic. BPF maps store policy as identity + bitmap lookups — O(1) regardless of rule count.
TC egress	veth interface	Egress policy enforcement, FQDN-based rules, rate limiting.
Socket-level	connect() syscall	L7 policy enforcement via redirect to per-node Envoy proxy for HTTP/gRPC/Kafka/DNS inspection.

The eBPF programs compiled by the Cilium agent use BPF maps for state management. Policy updates push new BPF bytecode to the kernel — no rule-by-rule iptables chain programming.

Packet flow with Cilium eBPF:
  NIC → [XDP: drop/allow] → TC ingress → [BPF map lookup: identity+port] → Pod
  Pod → [BPF map lookup] → TC egress → [FQDN resolve] → NIC

Key design property: Policy lookup is O(1) via BPF maps. Adding your 10,000th service or 1,000th policy does not change the lookup cost. This is the fundamental performance advantage.

Calico: iptables Chains

Calico's Felix agent runs on every node and translates NetworkPolicy objects into iptables chains. Each policy rule becomes one or more iptables rules in a structured chain hierarchy.

How it works:

1. Felix watches the Kubernetes API for NetworkPolicy changes
2. Each policy is compiled into iptables rules in per-interface chains (e.g., cali-fi-veth-abc123)
3. Packets traverse the standard Linux netfilter path: PREROUTING → FORWARD → INPUT/OUTPUT → POSTROUTING
4. Each rule is evaluated sequentially — O(n) with the number of rules

Packet flow with Calico iptables:
  NIC → netfilter PREROUTING → FORWARD chain
    → cali-forward chain → cali-fi-veth-* chain
    → [Rule 1: match? no → Rule 2: match? no → ... → Rule N]
    → ACCEPT/DROP → Pod

Calico also has an eBPF dataplane (available since v3.13). When enabled, it bypasses iptables and uses eBPF for packet processing — similar architecture to Cilium. But this is opt-in, and Calico's eBPF implementation is less mature than Cilium's eBPF-native design.

The Fundamental Difference

Dimension	Cilium eBPF	Calico iptables	Calico eBPF
Policy lookup	O(1) BPF map	O(n) sequential chain	O(1) BPF map
Kernel bypass	Yes (no iptables)	No (traverses netfilter)	Yes (no iptables)
Context switches	Zero for L3/L4	Per-packet netfilter	Zero for L3/L4
Default in 2026	GKE, AKS (option), EKS (option)	AKS (option), k3s (RKE2)	Opt-in
L7 enforcement	Native (in-kernel redirect)	Envoy sidecar required	Envoy sidecar required

Performance Benchmarks

L3/L4 Policy Enforcement

The most important finding across multiple independent benchmarks: eBPF-based L3/L4 policy enforcement adds zero measurable overhead.

Cilium eBPF L3/L4 overhead (from Tencent TKE benchmark, 2026):

Metric	No policy	L3/L4 policy applied	Overhead
Throughput (native)	104,787 req/s	104,083 req/s	-0.7%
Throughput (overlay)	100,242 req/s	100,599 req/s	+0.4%

This is within measurement noise. The reason: BPF map lookup + identity check is a hash table lookup in kernel space. It adds nanoseconds per packet.

Calico iptables with scaling policies (from Kim et al. 2025 academic study):

Policy count	Throughput	Latency	CPU
100 rules	8.8 Kbps	43 ms	3.24%
500 rules	8.5 Kbps	46 ms	3.45%
1,000 rules	8.2 Kbps	51 ms	3.89%

Calico shows gradual degradation. The iptables chain grows linearly with rule count — each packet traverses more rules.

Service Scale: Where iptables Collapses

The real divergence appears at service scale, not policy count. kube-proxy in iptables mode programs one rule per service per endpoint. At 10,000 services with 10 endpoints each, that's 420,000 iptables rules.

Service scaling degradation (from Tencent TKE benchmark, kernel 6.6):

Scale	iptables degradation	Cilium eBPF degradation
5,000 services × 10 endpoints (209K rules)	-28.5% short-conn RPS	-13.2%
10,000 services × 10 endpoints (420K rules)	-42.3% short-conn RPS	-11.5%

Cilium uses BPF maps where endpoints are map values, not standalone rules. Adding endpoints doesn't lengthen the lookup — it just adds values to the same map entry.

Latency at scale (same benchmark):

Dimension	iptables	Cilium eBPF
TCP_RR p99	111 μs	96 μs
TCP_CRR p99	499 μs	558 μs
HTTP p99 @ 1000 QPS	0.99 ms	0.99 ms

Interesting: Cilium's TCP_RR latency is actually lower than iptables on modern kernels (6.6+), because eBPF bypasses the iptables overhead entirely. But TCP_CRR (connection creation) is slightly higher due to conntrack overhead.

L7 Policy: The Performance Cliff

L7 policy enforcement (HTTP path/method filtering, gRPC service filtering) is fundamentally different from L3/L4. Both Cilium and Calico redirect L7 traffic to a userspace proxy (Envoy) for deep packet inspection. The overhead is severe.

Cilium L7 overhead:

Metric	No policy	L7 CNP applied	Overhead
Throughput (native)	104,787 req/s	13,591 req/s	-87.0%
Throughput (overlay)	100,242 req/s	11,984 req/s	-88.0%

This is not a Cilium flaw — it's the inherent cost of L7 visibility. Every L7 request requires:

1. Kernel-level redirect to per-node Envoy proxy
2. Full HTTP parsing in userspace
3. Policy evaluation against parsed headers
4. Forwarding or dropping

The per-node Envoy proxy shares a single core with all pods on that node. On 1-vCPU ARM64 nodes, this creates 8.3× more kernel scheduling work per request than baseline.

Service mesh comparison (from idmcarvalho service-mesh-benchmark):

Scenario	QPS vs baseline (50c)	p50 latency	CS/request
Baseline (no mesh)	—	42 ms	202
Cilium eBPF L3/L4	-1.3%	43 ms	188 (-7%)
Istio ambient (ztunnel)	-28.5%	54 ms	283 (+40%)
Istio sidecar	-57.0%	88 ms	482 (+139%)
Cilium L7	-84.5%	249 ms	1,667 (+726%)

Cilium L7 is actually worse than Istio sidecar because the shared per-node Envoy competes directly with application workloads for CPU.

Pod-to-Pod Throughput (Cross-Node)

From HMoradiRad comparison (kind cluster, bare-metal estimates):

Scenario	Cilium	Calico (iptables)	Winner
Same-node throughput	32.9 Gbps	23.8 Gbps	Cilium (+38%)
Cross-node throughput	13.6 Gbps	11.7 Gbps	Cilium (+16%)
Same-node latency	0.83 ms	0.91 ms	Cilium
Cross-node latency	0.240 ms	0.205 ms	Calico (slightly)

Cilium wins throughput in both scenarios due to eBPF's direct veth-to-veth forwarding (no bridge, no netfilter). Calico wins cross-node latency slightly — attributed to its simpler host IP stack path for native routing mode.

CPU Cycles Per Packet

From NSF-funded academic study (perf-based CPP measurement):

CNI	Total CPP	eBPF	Netfilter	Bridge	IP forwarding
Cilium	~1,280	~1,200	0	0	0
Calico (no policy)	~480	0	0	0	~240
Calico (with policy)	~2,700	0	~2,200	0	~240
Flannel	~330	0	0	~250	0
Kube-router	~250	0	0	~250	~250

Cilium's eBPF overhead (~1,200 CPP) is higher than the bridge-based approaches for no-policy cases, but Calico with policies costs ~2,700 CPP — more than double Cilium's total cost. The netfilter traversal is the killer.

Feature Comparison

Policy Capabilities

Feature	Kubernetes NetworkPolicy	Calico (iptables)	Calico eBPF	Cilium
L3/L4 (IP, port, protocol)	✅	✅	✅	✅
L7 HTTP (path, method, headers)	❌	❌ (Enterprise only)	❌ (Enterprise only)	✅ Native
L7 gRPC (service, method)	❌	❌	❌	✅ Native
L7 Kafka (topic, client ID)	❌	❌	❌	✅ Native
DNS-based (FQDN) policy	❌	❌ (Enterprise only)	❌	✅ via DNS proxy
CIDR ranges	✅	✅	✅	✅
Identity-based	❌ (IP-based)	Partial (label-based)	Partial	✅ (labels, service accounts)
Cluster-wide policies	❌ (namespace-scoped)	✅ GlobalNetworkPolicy	✅	✅ ClusterwideNP
Policy ordering	❌ (implicit)	✅ (explicit order)	✅	❌
Host firewall	❌	✅ HostEndpoint	✅	✅
Egress gateway	❌	❌ (Enterprise only)	❌	✅
Policy tiers	❌	✅ (Enterprise)	✅	❌

Observability

Feature	Cilium	Calico
Flow visibility	✅ Hubble (built-in, L7-aware)	❌ (Enterprise only / external tools)
Policy verdicts	✅ ALLOWED/DENIED per flow	❌
Real-time network map	✅ Hubble UI	❌
CLI	hubble observe --namespace X --type drop	calicoctl flowlogs (limited)
Prometheus metrics	✅ hubble_flows_processed_total	✅ (requires setup)
Runtime security	✅ Tetragon (eBPF-based)	❌

Hubble is Cilium's killer feature for operations. You can see every dropped packet, which policy dropped it, and the L7 details (HTTP method, path, status code) — all from eBPF in the kernel, no sidecars.

Extended Capabilities

Feature	Cilium	Calico
kube-proxy replacement	✅ (mature, recommended)	✅ (eBPF mode only)
Service mesh	✅ (sidecarless, eBPF-based)	❌
BGP routing	Control-plane only	✅ (BIRD daemon, first-class)
Multi-cluster	ClusterMesh (up to 255)	Federation via BGP/overlay
Encryption	WireGuard, IPsec	WireGuard, IPsec
Bandwidth management	✅ EDT-based rate limiting	Via annotations
Load balancing	Maglev, DSR, XDP	IPVS or iptables
Windows nodes	❌	✅ (Windows HNS)
CNCF status	Graduated (Oct 2023)	Not CNCF (Tigera-owned)

Production Patterns

When to Use Cilium

Choose Cilium when:

• Performance at scale matters — 500+ services or 1,000+ policies. eBPF's O(1) lookup vs iptables' O(n) is the deciding factor
• You need L7 policy — HTTP path/method enforcement, gRPC service filtering, DNS-based egress control. No other CNI does this natively without sidecars
• Observability is a priority — Hubble provides flow visibility that iptables-based CNIs simply cannot match
• Greenfield cloud-native — Modern Linux kernels (≥5.4), new clusters with no migration cost
• Running on managed K8s — GKE Dataplane V2 and AKS Azure CNI Powered by Cilium already use Cilium as default or option

When to Use Calico

Choose Calico when:

• BGP integration is required — Calico's BIRD daemon provides first-class BGP peering with physical routers. If your network team manages a Layer 3 fabric, Calico integrates natively
• Windows node support — Calico is the established choice for mixed Linux/Windows clusters
• Older kernels — Calico's iptables dataplane works on any Linux kernel. eBPF requires ≥5.4 (recommended ≥5.8)
• Existing investment — Already running Calico with policy libraries and operational expertise. No compelling reason to migrate unless you need L7

Small Clusters: Don't Overthink It

For clusters with fewer than 500 services and no L7 policy needs, the performance difference is negligible. Calico's iptables mode uses fewer resources (~278 MB/node vs ~153 MB/node for Cilium) and is simpler to operate.

Choose based on secondary factors: BGP needs (Calico) vs observability needs (Cilium).

Migration Strategies

Calico → Cilium

Phase 1: Dual-run validation

Cilium can run alongside Calico during migration. Install Cilium with a reserved CIDR range that doesn't overlap with Calico's:

# helm values for Cilium alongside Calico
ipam:
  operator:
    clusterPoolIPv4PodCIDRList: "10.200.0.0/16" # Separate from Calico's range
routingMode: "native"
kubeProxyReplacement: "probe" # Don't replace yet

Deploy Cilium without kube-proxy replacement. Validate connectivity, then switch workloads namespace-by-namespace.

Phase 2: Policy migration

Standard Kubernetes NetworkPolicy objects work on both CNIs without modification. The migration surface is only the extended policy CRDs:

Calico CRD	Cilium equivalent	Notes
NetworkPolicy (projectcalico.org)	CiliumNetworkPolicy	L7 features map directly
GlobalNetworkPolicy	CiliumClusterwideNetworkPolicy	Cluster-wide scope
NetworkSet	N/A (use CIDR or DNS)	IP group definitions
HostEndpoint	CiliumHostPolicy	Host-level firewall

Use the Calico Network Policy Converter to translate Calico-specific CRDs to CiliumNetworkPolicy equivalents.

Phase 3: kube-proxy removal

Once all workloads run on Cilium datapath, enable full kube-proxy replacement:

kubeProxyReplacement: "strict" # Fully replace
loadBalancer:
  mode: "dsr" # Direct Server Return for Service LB

Policy Portability Best Practices

Write standard Kubernetes NetworkPolicy wherever possible. Only reach for CiliumNetworkPolicy or Calico GlobalNetworkPolicy when you genuinely need extended capabilities. This keeps the bulk of your policy CNI-agnostic.

Debugging and Verification

Verify Policies Are Actually Enforced

The same pattern from the k3s NetworkPolicy article applies here. Never trust that a policy object exists — verify it's enforced:

# Deploy a test client and server
kubectl run test-client --image=busybox --command -- sleep 3600
kubectl run test-server --image=nginx --port=80

# Create a deny-all policy
cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
EOF

# Test: should fail
kubectl exec test-client -- wget -q --timeout=3 http://test-server.default && echo "FAIL: policy not enforced" || echo "PASS: policy enforced"

Using Hubble for L7 Visibility

With Cilium + Hubble, you can see exactly which policy is handling each flow:

# Watch all dropped packets in the payments namespace
hubble observe --namespace payments --type drop

# Output:
# Oct 11 10:23:45.123: default/test-client:42321 > default/test-server:80 (HTTP) FlowID:12345
#   Verdict: DENIED (L7 Policy: allow-only-get)
#   HTTP Method: POST, Path: /api/v1/admin/delete

Calico iptables Debugging

With Calico, debugging requires inspecting iptables chains directly:

# Show Calico's iptables chains
iptables -L cali-fi-veth-$(kubectl get pod test-server -o jsonpath='{.status.podIP}' | tr '.' '-') -n -v --line-numbers

# Count total iptables rules (the scaling concern)
iptables-save | grep -c "^-A"

At scale, this output becomes unwieldy. This is why Calico Enterprise or external observability tools are recommended for policy debugging at scale.

Decision Framework

Your situation	Recommendation	Why
New cluster, modern kernel (≥5.8)	Cilium	eBPF-native, best performance at scale, Hubble observability
Existing Calico, working fine	Stay with Calico	No compelling reason to migrate unless you need L7 or Hubble
10,000+ services	Cilium	iptables collapses at scale (-42%), Cilium stays stable
BGP peering with physical routers	Calico	BIRD daemon, first-class BGP integration
L7 HTTP/gRPC policy needed	Cilium	Native in-kernel L7 without sidecars
Windows nodes required	Calico	Only CNI with Windows HNS dataplane
Small cluster (<500 services)	Either	Performance difference is negligible
Managed K8s (GKE/AKS)	Cilium	Already integrated (GKE Dataplane V2, Azure CNI Powered by Cilium)
Edge / k3s / resource-constrained	Calico (iptables)	Lower memory footprint (~100 MB vs ~200 MB)

Key Takeaways

1. L3/L4 NetworkPolicy has zero overhead with eBPF — Cilium's BPF map lookup costs nanoseconds per packet. Apply L3/L4 policies broadly across all workloads without hesitation.

2. L7 NetworkPolicy is expensive everywhere — 87% overhead is the inherent cost of userspace proxy inspection. Apply L7 selectively to pods that genuinely need application-layer control (ingress gateways, sensitive API endpoints).

3. iptables collapses at service scale — 10,000 services with 420,000 rules causes 42% throughput degradation. This is the primary reason to choose eBPF for medium-to-large clusters.

4. Hubble is Cilium's killer operational feature — Real-time flow visibility with policy verdicts, L7 details, and Prometheus metrics. No iptables-based CNI can match this without sidecars.

5. Standard NetworkPolicy is portable — Both CNIs implement the standard API. Only extended CRDs (CiliumNetworkPolicy, GlobalNetworkPolicy) create vendor lock-in. Write portable policies where possible.

6. Calico is not wrong — For BGP environments, Windows nodes, or clusters where iptables scale is not a concern, Calico remains a mature, battle-tested choice. The gap between the two ecosystems is widening, but Calico's core value proposition (BGP + flexibility) remains valid.