Benchmarks

Results we have published, each with a link to the full page.

  • HaloGuard 1.0 prompt safety

    The Halo model family (HaloGuard 1.0). Source: arXiv 2607.02079.

    • 90.9%Average F1 across 7 prompt-safety benchmarks, 0.8B parameter model
    • 92.1%Average F1, 4B model, 4.7% false positive rate
    Full results

  • Privacy filter

    HaloGuard EU's privacy filter looks for personal data in text. Share of personal values found, on six public benchmarks.

    • 96.3% to 100%Strict mode
    • 88.9% to 99.1%Loose mode
    • 79.1% to 93.7%The open model openai/privacy-filter, default settings

    MaskingOur filter also masks more text around personal data than the open model.

    Full results

  • HaloGuard EU

    A small guard model that sits in front of LLM and agent output. Accuracy on about 4,500 of our test requests in all 24 EU languages, which HaloGuard EU never saw in training.

    Built to block the AI output the EU AI Act bans or treats as high-risk. Hosted in the EU.

    • 99.7%English accuracy, 732 requests
    • 97.9%Accuracy in the other 23 EU languages, 3,758 requests

    Almost all of these test requests are public on Hugging Face.

    Full results