Learn how to implement Prometheus-based health checks in Kuma using a sidecar container. This guide covers everything from metric evaluation to Docker and Kubernetes setup for reliable service mesh monitoring.

🧩 Using Prometheus Metrics for Health Checks in Kuma via a Sidecar Pattern

In service meshes like Kuma, health checks typically rely on TCP or HTTP probes.
But what if your service’s health depends on a Prometheus metric, such as my_app_up == 1?

Kuma doesn’t support metric-based health checks natively — but you can bridge the gap with a lightweight sidecar container that interprets Prometheus metrics and exposes a simple /health endpoint for Kuma to consume.

This post shows you how.

🗺️ Architecture Overview

         +----------------+
         |  Your Service  |
         | (Prometheus    |
         |  metrics here) |
         +-------+--------+
                 |
                 | exposes /metrics
                 |
         +-------v--------+
         | Sidecar Proxy  |
         | /health --> 200|
         | if metric OK   |
         +----------------+
                 |
        Kuma probes /health on port 9000

🛠️ Step 1: Create the Sidecar Health Proxy

This proxy reads your app’s Prometheus metrics and returns HTTP 200 only if the metric my_app_up is 1.

health_proxy.py:

from flask import Flask, Response
import requests

app = Flask(__name__)

@app.route('/health')
def health_check():
    try:
        resp = requests.get("http://localhost:8080/metrics")  # Replace with your app port
        if 'my_app_up 1' in resp.text:
            return Response("OK", status=200)
        else:
            return Response("Unhealthy", status=500)
    except:
        return Response("Unhealthy", status=500)

app.run(host='0.0.0.0', port=9000)

🐳 Step 2: Build the Docker Image

Dockerfile:

FROM python:3.10-slim

WORKDIR /app
COPY health_proxy.py .

RUN pip install flask requests

EXPOSE 9000
CMD ["python", "health_proxy.py"]

Build and push:

docker build -t yourdockerhub/health-proxy:latest .
docker push yourdockerhub/health-proxy:latest

🎯 Step 3: Deploy to Kubernetes

Here’s a sample deployment with the main app and health proxy sidecar.
It also includes a Kuma HealthCheck policy.

kuma-metric-health.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: kuma-demo
  labels:
    kuma.io/mesh: default

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-app
  namespace: kuma-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo
  template:
    metadata:
      labels:
        app: demo
        kuma.io/service: demo
    spec:
      containers:
        - name: app
          image: your-app-image  # Replace with your actual app image
          ports:
            - containerPort: 8080
        - name: health-proxy
          image: yourdockerhub/health-proxy:latest
          ports:
            - containerPort: 9000
---
apiVersion: kuma.io/v1alpha1
kind: HealthCheck
metadata:
  name: demo-metric-health
  namespace: kuma-demo
spec:
  sources:
    - match:
        service: demo
  destinations:
    - match:
        service: demo
  conf:
    interval: 10s
    timeout: 2s
    unhealthyThreshold: 3
    healthyThreshold: 1
    http:
      path: /health
      port: 9000

🧪 Step 4: Validate the Setup

1. Ensure Prometheus metrics are available at /metrics on port 8080.
2. Deploy the app and sidecar.
3. Kuma will ping the proxy’s /health every 10s.
4. Test failure conditions by changing the metric:
   • my_app_up 1 → sidecar returns 200 OK → Kuma marks healthy.
   • my_app_up 0 → sidecar returns 500 → Kuma marks as unhealthy.

🎓 Conclusion

While Kuma doesn’t natively support Prometheus-based health checks, you can convert metrics into HTTP health signals with this sidecar pattern.

✅ Lightweight
✅ Language-agnostic
✅ Works in any environment

Now Kuma will automatically remove services from the mesh when your custom metric signals a problem.

🔁 Ideas for Extension

• Use multiple metrics in the health logic
• Add caching to avoid high metric polling rates
• Build a Helm chart for easier reuse

Happy meshing!