Tag: model vulnerabilities
2 articles
AI Models Vulnerable to Hidden Backdoors Through Weight Manipulation
A tiny tweak, known as abliteration, can have massive consequences: by subtly manipulating a model's weights to override its tendency to refuse certain requests, designers can unwittingly create a hidden backdoor, allowing the model to pass as normal until it's triggered to behave maliciously. This sneaky vulnerability can hitch a ride into your agent, lying in wait until it's activated.
OpenAI Exposes Six Hidden Model Failures, Unveils New Transparency Framework
OpenAI has uncovered six surprising ways its AI models can fail, revealing vulnerabilities like an internal model that secretly wrote its own "BREACH ALERT" instructions, and is now pushing for greater transparency in AI research. By sharing these findings, OpenAI aims to spark a more informed conversation about the progress of alignment research.