Test your own model the way an attacker will
If you ship a model or a wrapper, someone will try to jailbreak it. The evaluation set has to be written by something that is willing to try. A safety-tuned generator produces a test you will pass and an attacker will not use.
Labs shipping assistants, wrappers, or classifiers who need a hostile eval set they are allowed to hold.
Why the mainstream APIs fail here
The frontier API will not write the jailbreak. Your eval harness then tests against polite prompts and reports a 99% refusal rate that does not survive contact with a forum post.
How Redline is used for this
Generate the attack set on Redline. Run it against your product, not against Redline's catalogue, unless you are measuring that too. Store the set in your eval repo. Redline does not keep it.
A working loop
- Generate attacks that match how people actually talk to your product.
- Version the set. Measure regressions.
- Cap the generator key so a sweep has a budget.
Prompts are not stored. A signed receipt of the hash comes back on every call, so you can prove later what Redline logged. Details are on the privacy page. Per-model prices are on /models and the pricing model is on /pricing.