AI agents have come a long way from chatbots that answer questions. Today’s agents access email, pull records from CRMs, execute code, and take actions across dozens of connected systems. That shift from generating text to doing things in the world creates a very different set of risks — and most development teams aren’t fully equipped to address them.
Microsoft is taking a direct run at that problem. The company has open-sourced two new tools — RAMPART and Clarity — designed to make AI safety a continuous engineering practice rather than a one-time checkpoint. Both are available now on GitHub.
Testing Agents Like You Test Code
RAMPART is a testing framework that brings red teaming techniques directly into the development workflow. It’s built on top of PyRIT, Microsoft’s existing automation framework for red teaming generative AI systems.
Where PyRIT is optimized for black-box discovery by security researchers after the system is built, RAMPART is built for engineers as the system is being built.
The experience will feel familiar to anyone who has written integration tests. Teams write standard pytest tests that describe scenarios pulled from their threat model. Each test connects to the agent through a lightweight adapter, orchestrates an interaction, and evaluates observable outcomes — passing or failing like any other test in a CI pipeline. When a new tool or data source is added to the agent, the corresponding safety test can be added in the same pull request.
One of RAMPART’s stronger features is its focus on cross-prompt injection attacks — scenarios in which an agent retrieves or processes content from documents, emails, tickets, or other data sources that indirectly manipulate its behavior. It’s one of the more persistent attack surfaces in agentic AI, and RAMPART is designed to catch it early.
Because LLM behavior is probabilistic, RAMPART also supports statistical trials. The same test can be run multiple times with policies such as “this action must be safe in at least 80 percent of runs.” That’s a more realistic model of how agents actually behave in production than a single-shot pass/fail approach.
The incident response angle is worth noting, too. If something goes wrong in production, the responding team needs to reproduce the incident and verify that the fix they ship holds up against variants of the original attack. RAMPART is designed to support exactly that workflow — turning red team findings and AI incidents into repeatable regression coverage.
Asking Better Questions Before Writing a Line of Code
Clarity takes a different approach. It’s not a testing tool — it’s a structured thinking tool. Microsoft describes it as a sounding board for development teams to pressure-test their assumptions before they start building.
The most expensive safety failures almost always trace back to design mistakes that nobody questioned early enough — when a product team decided their agent should have access to a tool, or handle a particular user flow, without fully working through what could go wrong. By the time a red team engagement surfaces the issue, the system is largely built.
Clarity tries to catch those mistakes before the architecture is locked in. It guides teams through structured conversations covering problem clarification, solution exploration, failure analysis, and decision tracking. Multiple AI “thinkers” independently examine the system from different angles — security, human factors, adversarial scenarios, and operational concerns.
The output from those conversations gets written to a .clarity-protocol/ directory in the repo as plain markdown files. They’re committed, reviewed in pull requests, and diffed just like source code. Clarity also tracks staleness across these documents — when a problem statement changes, Clarity knows that the solution description and failure analysis might need revisiting and nudges the team accordingly. Clarity runs as a desktop app, a web UI, or embedded directly in a coding agent.
From Audit to Engineering Discipline
What Microsoft is really arguing here is that AI safety needs to follow the same trajectory that security did over the past decade — from periodic audits to something embedded at every stage of the development lifecycle.
Mitch Ashley, VP and practice lead for software lifecycle engineering and AI-Native Software Engineering at The Futurum Group, sees this as a pivotal shift. “Open-sourcing RAMPART and Clarity demonstrates that AI safety is moving from post-deployment audit into the developer’s inner loop,” he said. “Cross-prompt injection and design intent become tracked artifacts alongside agent code, governed by the review and regression security adopted a decade ago. Platform teams that treat agent safety as a separate red-team engagement accumulate verification debt with every new tool connection. As agents scale, buyers will demand evidence of in-pipeline testing and statistical pass thresholds before procurement clears.”
That last point is significant. Procurement pressure has historically been one of the strongest forcing functions in enterprise software. If buyers start requiring evidence of in-pipeline safety testing, the teams that have already embedded RAMPART into their CI workflow will have a clear advantage.
RAMPART and Clarity are part of a broader movement toward spec-driven, engineering-native AI safety. Clarity helps teams clarify design intent and capture assumptions; RAMPART gives teams the building blocks to write concrete agent safety tests and keep them running as agents evolve. Together, they’re designed to move AI safety from a one-time review into a set of living artifacts that developers use throughout the lifecycle.
Both tools are open source and available on GitHub today. Microsoft is also inviting feedback and enterprise deployment partnerships.
For DevOps and platform engineering teams already wrestling with how to govern agentic AI, RAMPART and Clarity offer a practical starting point — one that fits into existing workflows rather than requiring a separate process.
