Garak (NVIDIA’s open-source LLM vulnerability scanner) is the tool most teams reach for first when someone asks “do we have an LLM scanner.” That’s not a complaint — it’s earned its place — but the gap between “ran Garak” and “red-teamed the LLM application” is wider than the marketing implies.
This review draws on Garak’s own reference documentation and the project’s research paper. Where it shines, where it noises, and where it slots into a workflow.
What Garak is
Garak is a probe-based scanner. It runs a library of canned attack templates (“probes”) against a target LLM endpoint, parses the responses with detectors, and reports which probes scored as hits. The probe library covers most named jailbreak families — DAN variants, encoding attacks, prompt injection patterns, leakage probes, refusal-bypass templates — plus generic functional checks for things like profanity output, PII leakage, and known-CVE-style behaviors.
Architecturally it is a pytest for LLM-side vulns. Probes are plugins. Detectors are plugins. The reporting documentation lists three outputs: a JSONL report file named garak.<uuid>.report.jsonl that records progress and outcomes, an HTML report summarising scores, and a JSONL hit log containing every attempt scored successful. This shape is exactly right; it is the part that has held up best as the LLM-security space has churned.
Where it earns the install
Coverage of named techniques. Garak’s probe library is the easiest way to test whether a target model resists the named, well-publicized attack families. If you need a one-line answer to “is this model vulnerable to PromptInject-style overrides?” or “does it leak under known continuation attacks?”, Garak gets you there with one command. For internal red teams supporting product orgs that ship multiple models, this is real value.
Machine-readable output. The JSONL report and hit log are structured per attempt, so a CI job can diff two runs without scraping prose. That is what makes Garak usable as a regression detector: if today’s run hits more probes than the previous model version did, something changed in the alignment. The detectors are not precise enough for it to be a release gate on its own.
Probe authoring. Writing new probes is straightforward: the probe base classes are documented, so you inherit one, supply a prompt template, and pick or write a detector. A custom probes directory holding your own attack catalog is the intended extension point, and at that point the framework’s value is the harness around your attacks rather than the shipped corpus.
Honest defaults. Garak doesn’t oversell. The README states the job in one line, “garak checks if an LLM can be made to fail in a way we don’t want,” and the project’s research paper frames it as a discovery tool rather than a pass/fail checklist. The conclusion this review draws from that is plain: it finds known things, and novel attacks are outside its scope.
Where it gets in the way
Detector false positives. The detectors are the soft underbelly. Many are substring-based: does the output contain the phrase X. A model that refuses a probe while quoting the forbidden phrase back inside its refusal (“I can’t help with how to X”) satisfies that test, and Garak scores it as a hit. No public benchmark quantifies the rate for a given probe class, so budget for reading the hit log rather than trusting the score, and expect detector tuning to be real work on your own stack.
A published corpus lags a private one. The probe library moves when the maintainers move it, and every shipped probe is by definition a technique that is already public. Anything an attacker has not published yet is out of scope by construction. That is fine for a baseline check; it is not red-teaming. A team that runs Garak quarterly and ships is doing baseline due diligence, not security testing.
Multi-turn coverage exists, but the detectors still gate it. Garak is no longer a single-turn-only scanner. IterativeProbe is documented as the “base class for multi-turn probes in which the probe uses the last target response to generate the next prompt”, TreeSearchProbe adds branching search, and the fitd probe implements a documented foot-in-the-door escalation that “generates queries of increasing intensity from safe to adversarial, getting compliance from the model at each intermediate stage”. That covers the shape of the class described in multi-turn role-play attacks. What has not changed is that a multi-turn run is still scored by the same substring detectors, so the harder the trajectory, the more of the verdict you are doing by hand.
RAG and agent harnesses are out of scope. Garak tests the model. The injection vectors that matter in production today — RAG ingestion, tool-use redirection, indirect injection — require harnessing the application as the test surface, not the model endpoint. You can stitch Garak into that with effort; out of the box it doesn’t do it. For the application-side classes, see prompt injection via retrieved documents and indirect prompt injection in LLM agents.
Cost discipline. A full Garak run with all probes against a frontier model endpoint is many thousands of requests, and branching multi-turn probes multiply that by the number of generations per turn. Price a scoped run before you price an unscoped one. There is a --probes filter; use it religiously. For the licence and self-hosting side of that bill across the whole category, see open-source LLM security tools by license and operating cost.
Where Garak slots into a real workflow
A workable pattern, given what the tool does and does not measure:
-
Baseline pass at engagement kickoff. Run a scoped Garak probe set against the target, and decide up front which public benchmark and judge any reported rate will be quoted against. The output is a sanity check: does this model resist the named, well-known attacks? Surprises here (“the customer’s model is vulnerable to vanilla DAN”) inform scoping and reset client expectations.
-
Regression harness for the engagement window. As you find novel attacks during manual red-teaming, write them as Garak probes. By end of engagement you have a CI-runnable corpus that the client can ship into their own pipeline.
-
Detector tuning is part of the deliverable. The tuned detector set for the client’s specific stack (their system prompt, their model, their output filters) is more valuable to them long-term than the raw findings. Hand over the tuned
garak.configand the custom detectors with the report. -
Pair with application-layer harnesses. Don’t try to test RAG injection or agent behavior with Garak alone. Use it for the model-side baseline; build separate harnesses for the application layer. PyRIT, promptfoo, and bespoke pytest harnesses each fill specific niches that Garak doesn’t — the trade-offs are laid out in the AI red teaming tools comparison.
The honest verdict
Garak is the right tool for one specific job: “does this model resist the named, public attack families when tested in isolation?” It is not the right tool for “is this LLM application secure.” A team that conflates the two is delivering a checkbox, not a security assessment.
For the broader landscape, including the attack classes Garak does not cover, see the jailbreak technique taxonomy and the framework comparison post for the optimizer-driven attack frameworks.
Garak is a baseline. Keep using it. Don’t ship without doing the rest.
For more context, adversarial ML research covers related topics in depth.
Related across the network
- Garak LLM Scanner Review: Research Tool or CI Gate? — aisecreviews.com
