CoT-Guard: Small Models for Strong Monitoring

Welcome to the CoT-Guard project!

CoT-Guard is an open-source chain-of-thought (CoT) monitor and post-training recipe for detecting covert misbehavior and hidden objectives in code generation. It provides a practical user-side defense against malicious or compromised LLM intermediaries while avoiding the deployment cost of large proprietary monitors.

CoT-Guard includes two post-training stages:

• Supervised Fine-Tuning: distilling hidden-objective detection behavior from stronger monitors to improve in-domain monitoring.
• Reinforcement Learning: training on hard and subtly crafted hidden objectives to improve generalization to unseen monitoring tasks.

We also introduce a practical threat model motivated by AI supply-chain attacks, covering both prompt manipulation and code manipulation by third-party LLM routers. Our evaluation includes hidden objectives based on real-world code attack patterns, including package forcing, typosquatting, advertising comments, and advertising prints.

Paper: https://arxiv.org/abs/2605.12746

GitHub: https://github.com/nirav0999/CoT-Guard

CoT-Guard will be presented at AdvML-Frontiers × CoTMA: From Model Security to Compositional Threats in Multi-Agent AI Systems @ COLM 2026 as an oral presentation. It is currently under review to a full conference venue.

Citation

@article{cotguard,
  title = {CoT-Guard: Small Models for Strong Monitoring},
  author = {Diwan, Nirav and Wang, Han and Kapusuzoglu, Berkcan and Moradi, Ramin and Chakraborty, Supriyo and Iyengar, Giri and Sahu, Sambit and Zhang, Huan and Wang, Gang},
  journal = {arXiv preprint arXiv:2605.12746},
  year = {2026}
}