Open Source LLM Security Tools: Licensing and Cost

“Open source” is doing a lot of work in the phrase open source LLM security testing tools. Four of the projects people reach for first are NVIDIA’s garak, Microsoft’s PyRIT, promptfoo, and Meta’s Purple Llama, and they sit under three different licenses with three different answers to the question that decides whether you can actually run one: what does this cost to operate inside my own infrastructure, and what are the terms attached to the artifacts I download?

That is the whole subject here. If you want the feature comparison instead, probe coverage, CI fit, multi-turn depth, and the OWASP LLM Top 10 mapping, that lives in best AI red teaming tools for LLMs.

Three permissive licenses and one that is not

garak is Apache 2.0. PyRIT is MIT. promptfoo is MIT. Those three carry no field-of-use restriction, no acceptable-use policy bolted on, and no headcount trigger, so a commercial security team can vendor them into an internal pipeline and ship the results to a client without a license review.

Purple Llama is the exception, and the split inside it is the detail most write-ups skip. Meta’s repository states that the evals and benchmarks are MIT, which covers CyberSecEval, and that Code Shield is also MIT. The safeguard model weights are not. Llama Guard 3 and Prompt Guard are released under the Llama 3.2 Community License, with earlier Guard generations under the Llama 2 and Llama 3 Community Licenses. Those are source-available licenses with an acceptable-use policy and a redistribution clause attached, not OSI-approved open source. Read them before a guardrail classifier becomes a load-bearing part of a product you sell.

The practical consequence: you can benchmark a candidate model with CyberSecEval under terms as permissive as the scanners, and separately inherit a very different set of obligations the moment you deploy Llama Guard in front of it.

Where the money actually goes

None of these projects charge for the harness. The bill lands somewhere else, and it lands differently per tool.

For garak, PyRIT, and promptfoo the running cost is inference against the target. The harness is a client. garak’s model is breadth, so a wide probe sweep is a large number of generations against whatever endpoint you point it at, and if that endpoint is a metered API the sweep is a metered API bill. PyRIT’s multi-turn orchestrators multiply that by turns per conversation, because an adaptive attack only exists as a sequence. promptfoo’s cost profile is the odd one because it is designed to run per pull request, so the spend is small each time and continuous forever.

Purple Llama inverts it. Llama Guard and Prompt Guard are weights you download and serve yourself, so there is no per-call charge and no third party seeing your traffic, but you are now operating a model. Llama Guard 3 ships in 1B, 8B, and 11B-vision variants, which is the knob you have: the small variant exists precisely so a moderation pass does not need the same accelerator budget as the model it is guarding.

Data residency is the other half of the cost question and it is easy to miss. promptfoo’s documentation is explicit that evaluations run entirely on your machine and prompts do not leave it. That property, not the feature list, is usually what decides whether a tool is usable on regulated or customer data at all.

Self-hosting, project by project

garak installs with pip and targets Python 3.11 to 3.13. It runs as a local CLI and reaches out to whatever generator you configure, so the only thing leaving your network is the traffic to the target endpoint itself, plus the API keys needed for hosted targets like OpenAI, Hugging Face, AWS Bedrock, or Replicate. The design intent is documented in the project’s arXiv preprint, which frames garak as a discovery tool rather than a pass/fail checklist. A fuller walkthrough of its probe and detector model is in the garak review.

PyRIT is a Python framework rather than a binary, which changes the cost shape entirely: the licence is free and the setup is engineering time. You compose orchestrators, converters, and scorers yourself, so the real question is not hosting but whether a red-teamer on the team writes Python. For the attack theory those payloads encode, aisec.blog tracks the offensive side in more depth.

promptfoo is the most contained of the four. It is an npm or pip install, configuration is YAML, and it is built to sit in a pipeline. Nothing about it assumes a hosted control plane, which is why it is the default recommendation for teams whose blocker is a data-handling policy rather than a budget.

Purple Llama is the only one where you take on real infrastructure. Prompt Guard is a small classifier for injection and jailbreak attempts, Llama Guard handles input and output moderation, Code Shield filters insecure code at inference, and CyberSecEval is a benchmark suite for how often a model writes insecure code or complies with malicious requests. Two cautions. A classifier placed in front of a model is itself a target, so it needs adversarial testing of its own, which is the argument made at more length in the LLM guardrail tools writeup. And a benchmark number is only as meaningful as the dataset and judge behind it, which is the point of jailbreak benchmarks and datasets and applies to CyberSecEval exactly as it applies to everything else.

Picking by constraint

• The blocker is a data-handling policy. promptfoo, because the evaluation loop stays on your hardware and the license carries no restriction on commercial use.
• The blocker is a metered inference budget. Scope garak by probe set rather than running the full sweep, and keep PyRIT’s multi-turn orchestrations for the handful of scenarios that genuinely need state.
• The blocker is legal review. Apache 2.0 and MIT clear it without argument. Anything under a Llama Community License needs someone to read the acceptable-use terms first, especially if the deployment is customer-facing.
• The blocker is headcount. garak and promptfoo produce a result on day one. PyRIT produces a result after somebody builds the attack, which is a sprint, not an afternoon.
• You need a runtime guardrail, not a test. That is a different product category and a different license question. Prompt Guard is a candidate, but budget for hosting it and for attacking it.

The honest summary is that license and hosting cost separate these four more cleanly than features do. Three of them are permissive, local, and billed only by the model you point them at. The fourth is a set of defenses with real terms attached, and those terms are the thing to read before the code is.

Related across the network

• AI Red Teaming Hub: Your Guide to Offensive AI Security — aisec.blog
• AI Red Team: Methodology, Tooling, and Attack Surface — aisec.blog
• Garak vs. PyRIT vs. promptmap: Prompt Injection Testing Compared — promptinjection.report
• AI Red Team Engagement Methodology: Scoping to Reporting — aisec.blog
• ChatGPT Jailbreak Prompt Taxonomy: Classes, Rates, and Defenses — aisec.blog
• RAG Retrieval Debugging: Why Results Come Back Wrong — ragstackguide.com