VirtualService Routing Fails After EnvoyFilter Injects a Header

What you see

In Istio, you use an EnvoyFilter to add a custom HTTP header (e.g. x-route-version: v2) and a VirtualService to match on that header and send traffic to a specific version. In practice, the route rule never applies and traffic always follows the default route.

Example setup:

EnvoyFilter — inject header

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: inject-route-header
spec:
  workloadSelector:
    labels:
      app: frontend
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_OUTBOUND
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
              subFilter:
                name: envoy.filters.http.router
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.lua
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
            inline_code: |
              function envoy_on_request(request_handle)
                request_handle:headers():add("x-route-version", "v2")
              end

VirtualService — match on header

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: my-service
spec:
  hosts:
    - my-service
  http:
    - match:
        - headers:
            x-route-version:
              exact: v2
      route:
        - destination:
            host: my-service
            subset: v2
    - route:
        - destination:
            host: my-service
            subset: v1

Expected: The frontend sidecar adds x-route-version: v2, the VirtualService matches it, and traffic goes to subset v2.

Actual: Traffic always goes to v1. Upstream can see x-route-version, but routing does not change.

Why it happens

Envoy matches routes before HTTP filters run

This follows Envoy’s internal pipeline; it is not an Istio bug.

Order of processing for an HTTP request:

Request arrives → HCM decodes headers → route resolution → HTTP filter chain → router forwards

In more detail:

1. Headers decoded: HttpConnectionManager (HCM) finishes decoding HTTP headers.
2. Route match: HCM matches routes using the current headers (your VirtualService rules) and picks a cluster.
3. HTTP filters: The filter chain runs (Lua, RBAC, etc.); only here does the EnvoyFilter add headers.
4. Router: envoy.filters.http.router forwards using the cluster already chosen in step 2.

The issue: routing (step 2) completes before the Lua filter adds the header (step 3). After that, the route is fixed; new headers do not trigger a new match.

Why upstream still sees the injected header

The header is added in step 3, after routing in step 2. The router in step 4 forwards with the full header set, so upstream receives x-route-version: v2 even though the cluster was picked without it.

clear_route_cache

Envoy can re-run route matching after headers change if a filter calls clear_route_cache. A typical Lua EnvoyFilter does not call that, so the route is never refreshed.

How to verify

1. Confirm the header is injected

Log headers on the destination or curl through the sidecar:

kubectl exec -it <target-pod> -c istio-proxy -- \
  curl -s http://localhost:<port>/headers

If you see x-route-version: v2, the filter works and the problem is ordering, not injection.

2. Inspect filter order

istioctl proxy-config listener <pod-name> -o json | \
  jq '.[].filterChains[].filters[] | select(.name == "envoy.filters.network.http_connection_manager") | .typedConfig.httpFilters[].name'

Confirm Lua runs before envoy.filters.http.router and after the other filters you care about.

3. Access logs at debug

istioctl proxy-config log <pod-name> --level http:debug,router:debug

Check route_name and cluster selection; at match time you should still see the original headers.

4. Confirm routes from the VirtualService

istioctl proxy-config route <pod-name> -o json | \
  jq '.[] | select(.name == "80") | .virtualHosts[] | select(.name | contains("my-service"))'

Mitigations

Option 1: Try to invalidate the route after Lua (limited)

After adding the header, force a re-match (behavior varies by Envoy version):

inline_code: |
  function envoy_on_request(request_handle)
    request_handle:headers():add("x-route-version", "v2")
    request_handle:headers():replace(":path", request_handle:headers():get(":path"))
  end

Note: Lua’s request_handle does not expose clearRouteCache() directly. Replacing :path with the same value is a hack to nudge route recalculation and is not guaranteed. Prefer Wasm or header_to_metadata for something reliable.

Option 2: Set the header in the client

Do not rely on EnvoyFilter for this header; set it in application code:

import requests
resp = requests.get(
    "http://my-service:8080/api",
    headers={"x-route-version": "v2"}
)

Then the header exists when the sidecar runs route matching.

Option 3: Wasm filter with clear_route_cache

// proxy-wasm SDK
fn on_http_request_headers(&mut self, _: usize, _: bool) -> Action {
    self.add_http_request_header("x-route-version", "v2");
    self.clear_route_cache();
    Action::Continue
}

Option 4: Header-to-metadata + metadata-based routing

Use Envoy’s header_to_metadata and route on metadata so ordering stays predictable.

Option 5: Inject at the gateway

If traffic passes through the ingress gateway, injecting there can change which headers exist before the next hop’s routing (two proxies, two HCM pipelines).

Summary

Takeaway: Headers added by EnvoyFilter do not affect VirtualService matching because Envoy picks the route before those filters run. To make injected headers affect routing, trigger route recalculation (clear_route_cache / Wasm) or avoid relying on sidecar injection for that match.