Campaign Delivery Health
Real-time observability powered by Honeycomb + autonomous incident response by AWS DevOps Agent
Service Health (from Honeycomb)
| Service | P50 (ms) | P99 (ms) | Requests | Error Rate | Status |
|---|---|---|---|---|---|
| Loading from Honeycomb... | |||||
Recent DevOps Agent Investigations
Investigation Center
Trigger and monitor AWS DevOps Agent investigations using Honeycomb telemetry
Trigger New Investigation
Incident Simulation
Demonstrate the full Honeycomb → DevOps Agent → Resolution loop with realistic Zeta Global scenarios
Campaign Delivery Latency
Personalization service latency spike affecting real-time campaign delivery for email and push channels.
Identity Graph Failures
503 errors from identity graph service. Audience segmentation degraded, campaigns falling back to broad targeting.
Signal Ingestion Backpressure
Kafka consumer lag exceeding threshold. Real-time behavioral signals delayed, scoring engine using stale data.
Integration Architecture
How AWS DevOps Agent connects to Honeycomb for Zeta Global's observability stack
┌─────────────────────────────────────────────────────────────────────────────┐
│ ZETA GLOBAL PLATFORM │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Campaign │ │ Identity │ │ Signal │ │ Delivery │ │
│ │ Router │ │ Graph │ │ Ingestion │ │ Engine │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │ │
│ └──────────────────┴──────────────────┴──────────────────┘ │
│ │ │
│ OpenTelemetry SDK │
│ (traces, metrics, logs) │
└────────────────────────────────────┬────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────────┐
│ HONEYCOMB │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Traces & │ │ SLOs & │ │ BubbleUp │ │ Triggers & │ │
│ │ Spans │ │ Burn Alerts │ │ Analysis │ │ Webhooks │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ └──────┬──────┘ │
│ │ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Honeycomb MCP Server │ │
│ │ (Exposes query, traces, SLOs to AI agents via Model Context Protocol) │
│ └──────────────────────────────────┬──────────────────────────────────┘ │
└─────────────────────────────────────┬──────────────────────────────────────┘
│
┌─────────────────┐│┌─────────────────┐
│ Webhook Trigger │││ MCP Connection │
└────────┬────────┘│└────────┬────────┘
│ │ │
▼ ▼ ▼
┌────────────────────────────────────────────────────────────────────────────┐
│ AWS DEVOPS AGENT │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Agent Space │ │
│ │ │ │
│ │ 1. Receive alert from Honeycomb trigger │ │
│ │ 2. Query Honeycomb traces via MCP (identify slow spans) │ │
│ │ 3. Correlate with deployment data (GitHub/CodeDeploy) │ │
│ │ 4. Analyze patterns using BubbleUp-style correlation │ │
│ │ 5. Identify root cause │ │
│ │ 6. Generate mitigation plan │ │
│ │ 7. Post findings to Slack / create ticket │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ Connected Tools: │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │Honeycomb │ │ GitHub │ │ Slack │ │CloudWatch│ │ Runbooks │ │
│ │ (MCP) │ │ (MCP) │ │ (MCP) │ │ (native) │ │ (MCP) │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
└────────────────────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────────┐
│ OUTCOMES │
│ │
│ • MTTR reduced from 45 min → 4 min (75% reduction) │
│ • On-call engineers wake up to root cause, not raw alerts │
│ • Proactive recommendations prevent repeat incidents │
│ • Full audit trail of investigation reasoning │
└────────────────────────────────────────────────────────────────────────────┘
Key Integration Points
1. Honeycomb MCP Server
DevOps Agent connects to Honeycomb via the official MCP server. This gives the agent direct access to query traces, analyze SLOs, and run BubbleUp correlations — the same tools your engineers use daily.
Honeycomb MCP → DevOps Agent Space → Tool Registration
2. Webhook Triggers
Honeycomb Triggers fire when SLO budgets burn or error thresholds breach. These webhooks hit the DevOps Agent endpoint with HMAC authentication, starting an autonomous investigation immediately.
Honeycomb Trigger → HMAC Webhook → DevOps Agent Investigation
3. Trace Correlation
During investigation, DevOps Agent queries Honeycomb for the specific traces showing the anomaly. It uses BubbleUp-style analysis to find what changed — deployment version, feature flag, infrastructure shift.
Agent queries → Honeycomb API → Trace analysis → Root cause
4. Resolution Loop
Once root cause is identified, DevOps Agent can trigger rollbacks, scale resources, or create tickets. The resolution is verified by re-querying Honeycomb to confirm metrics return to normal.
Mitigation → Verify via Honeycomb → Close investigation