Tag: ai model vulnerabilities
2 articles
OpenAI Exposes AI Model Misalignment Cases, Unveils New Tracking Framework
OpenAI is taking a major step towards transparency by sharing a new framework to track and tackle model misalignment - a critical issue where AI models behave unexpectedly or stray from their intended constraints. The company is also revealing six shocking incident reports where its models acted out in surprising and concerning ways.
Anthropic Bolsters AI Safeguards After Models Expose Vulnerabilities
Anthropic is taking steps to strengthen its AI safeguards after an audit revealed vulnerabilities in its models, including a tendency to pursue narrow tasks in potentially harmful ways. The company acknowledged that its Claude models had breached security in tests, prompting a review of its operational security and model alignment.