NVIDIA Nemotron and CrowdStrike tested agentic red‑blue loop for automated detection generation

Artificial intelligence is accelerating cybersecurity workflows: agentic systems can coordinate complex objectives over long horizons. Security teams are beginning to run agents in operations, but many deployments still rely on existing alerts, predefined playbooks, and known attack behaviors. The harder task is finding what defenses miss and converting those gaps into reliable coverage, which requires repeated adaptive attacks in an organization’s specific environment and validation of candidate detections against normal enterprise activity.

NVIDIA and CrowdStrike evaluated an agentic offense–defense system in an isolated environment modeled on NVIDIA accelerated computing infrastructure. On the defensive side, NVIDIA Nemotron models ran inside CrowdStrike SafeMind, its agentic cybersecurity system. CrowdStrike reports that its Blue Solano defensive model was 13% more accurate than the leading proprietary frontier model tested, at 97% lower cost, in CrowdStrike internal evaluations. In the evaluated open‑model configuration, NVIDIA Nemotron 3 Ultra handled defensive orchestration while a fine‑tuned Nemotron 3 Super generated and repaired detections.

Below is a summary of how Nemotron models and specialized agent harnesses worked with CrowdStrike’s agentic system, how the setup was deployed and evaluated, and how detections performed across independently seeded attack runs.

Turning offense and defense into a continuous learning loop

Traditional red‑team/blue‑team exercises depend on manual handoffs: the red team executes, the blue team inspects telemetry, engineers write or update detections, and the red team retests. These handoffs limit iteration speed and the variety of attacks tested. An agentic offense–defense system connects these steps into a repeatable closed loop that can operate at machine speed.

In the evaluated workflow each run produced attack traces and sensor telemetry used to create or refine detections. The loop had four stages:

• Execute and capture: starting from a threat‑informed objective, the red‑agent harness selected and executed an attack path inside the representative environment; the action trace recorded steps while CrowdStrike Falcon endpoint sensors captured telemetry.
• Process and reconstruct: the blue‑agent harness received the trace, telemetry, and attack context, determined what could be reconstructed, which existing detections fired, and where visibility or detection gaps remained.
• Generate and validate: the blue harness generated candidate detections; a validation harness backtested them against captured telemetry, returned failures for correction, and forwarded validated detections to the detection engine.
• Retest, adapt, repeat: after a validated detection was deployed, an independently seeded attack retested the same objective; results returned to the red harness, which adapted and explored alternative attack or evasion paths until no more viable paths remained in the modeled environment.

Each cycle is designed to harden coverage and force the offense to find increasingly difficult paths.

Building a representative test environment

To enable safe, realistic testing, NVIDIA provided a sanitized natural‑language specification representing its accelerated computing infrastructure. An agent‑assisted workflow translated that spec into an isolated target cyber environment instrumented with Falcon sensors. Progress was measured by attack paths and observable milestones drawn from action traces and sensor telemetry rather than the agents’ own claims. NVIDIA and CrowdStrike experts reviewed the environment and threat paths for realism; the same reviewed environment supported every attack run and detection test to allow consistent comparisons.

Specializing the defensive harness

Turning attack traces and telemetry into detec­tion rules that pass technical and behavioral validation required multiple mechanisms. The defensive harness combined six elements:

1. Schema knowledge base: a tool enumerated supported Falcon sensor schemas, fields, and query syntax to prevent invented fields or invalid queries.
2. Telemetry grounding: red‑agent traces, Falcon telemetry, and attack context anchored the workflow in observed events and relationships, reducing hallucinations.
3. Specialized detection authoring: a customized Nemotron 3 Super acted as a bounded expert for generating and repairing detections, separating that task from general orchestration.
4. Artifact linting: automated checks rejected syntax errors, unsupported fields, and detections tied to specific IPs, hosts, users, or subnets; failures returned guidance to rewrite around behavioral signals.
5. Detection replay: each candidate was replayed against captured telemetry; candidates with no matches were rejected and returned for correction.
6. Independent review: a separate judge evaluated each detection for behavioral alignment, robustness, and appropriate use of multiple signals.

Failed checks produced structured feedback for correction. Together these mechanisms encoded practices typically applied by manual detection engineers, making generation grounded, testable and correctable.

Customizing Nemotron for orchestration and detection generation

Open models such as Nemotron can be post‑trained with domain data and optimized for cost and scale in controlled environments. In the evaluated open‑model pipeline, Nemotron 3 Ultra reconstructed attack sequences, planned detection‑engineering steps and invoked tools, while a customized Nemotron 3 Super served as the bounded detection authoring expert.

CrowdStrike used Nemotron 3 Super as the base for its NL2LogScale model, then applied continual pretraining on cybersecurity knowledge, supervised fine‑tuning, and reinforcement learning with verifiable rewards. Fine‑tuning used 9,349 detection‑generation and multistep repair examples covering 59 programmatically generated error types. Training data combined rephrasings from Nemotron 3 Super, real Falcon LogScale execution errors, and quality‑reviewed reasoning traces from Nemotron 3 Ultra.

For reinforcement learning the workflow used NVIDIA NeMo Gym to validate and execute generated queries in Falcon LogScale: invalid queries produced real engine errors and up to five repair attempts, with unresolved attempts receiving zero reward. Valid generated and reference queries ran against the same synthetic logs and the F1 overlap between returned events supplied the reward. NVIDIA NeMo RL supported policy updates based on these verifiable results.

Evaluating the complete agent system: backtest and live fire

A detection that passes backtest against the recorded attack may still fail on a new execution. Therefore, the evaluation occurred in two stages: (1) backtesting against the recorded attack, and (2) live‑fire testing against eight unseen attacks from the same scenario family. Every passing detection was deployed verbatim to the live detection engine; an independent third‑party model judged whether each match represented the intended attack behavior.

Backtest results

• Using Nemotron 3 Ultra with the default harness, an average of 16.5% of generated detections detected the recorded attack across eight independently seeded sessions.
• Retaining Ultra but adding the tuned harness, customized Nemotron 3 Super, domain context, tools and validation raised the mean to 41.9% across six sessions — about a 2.5× improvement. Because both harness and model stack changed, this gain reflects the complete open pipeline rather than a single model ablation.

Live‑fire generalization

• In live‑fire testing, 11 backtest‑passing detections from the optimized open pipeline and 35 from the complete frontier system were deployed against eight unseen attacks.
• Five of the 11 open detections (45%) detected at least one attack, compared with 10 of 35 frontier detections (29%). The open pipeline averaged 2.6 detections per detection, compared with 1.1 for the frontier system.
• Detection was only the first quality gate. To qualify as “gold,” detections also had to remain quiet on available test traffic and pass independent review for behavioral grounding, multiple signals, and no environment‑specific strings. Four of the five firing open detections and nine of 10 frontier detections that fired stayed quiet; after review, three open detections and no frontier detections qualified as gold. The three gold open detections still covered all eight attacks.

Interpreting the findings and limitations

• The frontier system produced more backtest‑passing detec­tions, but the optimized Nemotron open‑model pipeline yielded a higher share that generalized, averaged more detections per deployed detection, and was the only system to produce gold detections in this evaluation.
• Limitations include coverage of a single scenario family and small detection sets, so cross‑scenario generalization was not tested. Limited benign traffic means the noise/false positive test does not represent production performance. Three of eight live‑fire runs experienced harness failures but provided complete telemetry and were retained. These findings are a directional system‑level case study, not a general benchmark.

Applying the pattern to specialized agents: design principles

The evaluation suggests a repeatable pattern for building specialized agents with NVIDIA Nemotron: a reasoning model orchestrates the defensive workflow, a post‑trained open model performs a bounded expert task, and an agent harness manages context, tools and validation. Four design principles emerge:

1. Define measurable tasks and assign clear responsibilities to each model.
2. Post‑train specialized models with domain data and verifiable rewards.
3. Ground outputs in authoritative context and validate them with deterministic checks, realistic replay, and independent review.
4. Evaluate the complete workflow in realistic conditions, with experts controlling scenarios, guardrails and use of validated outputs.

CrowdStrike is advancing this approach through SafeMind, its agentic cybersecurity system that brings offensive and defensive AI into a continuous coevolution loop. NVIDIA tools such as Nemotron 3, NeMo Megatron Bridge, NeMo Gym and NeMo RL can be used to customize and evaluate specialized agents for other bounded domains.

Conclusion

The joint evaluation demonstrates that an agentic closed‑loop system can automatically iterate offense and defense: producing attack telemetry, generating candidate detections, validating and deploying them, then retesting under independent attacks. In the tested scenario family, the optimized Nemotron‑based open pipeline produced fewer initial backtest passes than a frontier system but achieved better generalization, higher per‑detection coverage across attacks, and produced the only gold‑rated detections. The case study offers practical design guidance for specialized defensive agents while underscoring the need for broader, cross‑scenario benchmarking.