Observability engineer for AI agent platforms

We are looking for an Observability Engineer to build behavioral monitoring and auditability capabilities for an enterprise AI Agent Platform being developed from the ground up. The role will focus on transforming agent telemetry, behavioral anomalies, safety signals, and governance events into actionable dashboards and alerts. You will work at the intersection of observability, AI agent behavior, and governance , helping ensure that production agent activity can be effectively monitored, validated, and surfaced for both operational and audit purposes.

• Design and implement behavioral monitoring dashboards and alerts using New Relic and NRQL.
• Define and monitor behavioral signals including anomalies, drift, safety events, and policy denials.
• Build auditability views that surface relevant governance and compliance evidence.
• Maintain clear segregation between technical observability and audit/security evidence across monitoring systems.
• Configure, tune, and validate alerts based on production agent telemetry.
• Verify dashboard accuracy and telemetry consistency to ensure monitoring data is reliable.
• Develop Python-based utilities for metrics emission, validation, and observability testing.
• Implement monitoring patterns for AI agent anomaly and behavioral drift detection.
• Validate that sensitive information, including PII, is appropriately handled in telemetry and traces.
• Incorporate security and governance signals, such as policy decisions and access denials, into behavioral monitoring.
• Collaborate with security and governance stakeholders to support enterprise compliance and audit requirements.

• New Relic and NRQL for behavioral dashboards and alert conditions
• Behavioral monitoring signals (anomaly detection, drift, safety signals, policy denials)
• Auditability views surfacing governance evidence for consumption
• Observability domain segregation (Technical in CloudWatch/New Relic vs Audit in dedicated evidence/SIEM stores)
• Alert validation and dashboard data-accuracy verification
• Python (metrics emission and validation)
• 4+ years of observability engineering
• Behavioral / anomaly monitoring and alert tuning on production telemetry
• Working in environments with formal audit/compliance evidence requirements
• Будет плюсом: Anomaly/drift detection techniques for agent telemetry; PII redaction validation in traces surfaced through dashboards; Cedar policy decisions / Contract-Policy Engine denials as monitoring inputs; Enterprise security/compliance standards mapping

• That’s why we offer a range of benefits that support your well-being and charge your professional growth.
• We are committed to fostering equity, diversity, and inclusion as an equal opportunity employer.