Tag: large language models
52 articles
OpenAI Bolsters GPT-5.6 with Automated Red-Teaming Model
OpenAI just unveiled GPT-Red, an automated red-teaming model that's a game-changer in detecting prompt injection attacks, helping to shield its GPT models from vulnerabilities. By mimicking human red-teaming tactics, GPT-Red identifies and feeds back crucial insights to strengthen model defenses before they go live.
OpenAI Lifts GPT-5.6 Sol Usage Limits Amid Surging Demand
Big news for ChatGPT fans: OpenAI has temporarily lifted usage limits for Plus, Pro, and Business plans, giving you more time to tap into the power of its most advanced model. This move comes after a surge in demand over the past 48 hours, with the company also resetting current usage for all customers.
AI Alters Human Speech Patterns
Imagine interacting with ChatGPT and receiving a response that sounds like a robotic, three-part formula - it's a pattern that's distinctly non-human and may be changing the way we communicate. From affirmations to multiple-choice queries, these new rhythms of reply are a far cry from the emotional ebbs and flows of live speech.
AI-Powered Attacks Rapidly Compromise Cloud Targets
The increasing accessibility of large language models and agentic AI has empowered even less sophisticated threat actors to launch lightning-fast attacks with unprecedented scale, significantly ramping up the challenge for defenders. This alarming trend enables attackers to accelerate their workflows and compromise cloud targets at an unprecedented pace.
AI Models Expose Millions to Phantom Squatting Phishing Threat
Millions are now at risk of falling prey to a new, rapidly evolving phishing threat called phantom squatting, where attackers exploit AI-generated links to create malicious websites that can evade detection. By registering domains invented by large language models, hackers can create seemingly trustworthy sites that are actually designed to steal sensitive information or spread malware.
US Lifts Export Controls on Anthropic's AI Model Fable 5
Big news: the US has lifted export controls on Anthropic's AI model Fable 5, allowing it to be accessible to users worldwide again after a brief shutdown. This comes after Anthropic made significant strides in curbing a concerning technique, successfully stopping it in over 99% of attempts.
Organizations Lag in AI Usage Visibility, Exposing Security Gaps
Most organizations are flying blind when it comes to AI usage, with nearly half of respondents citing internal AI systems and Large Language Models as their top security concern, yet many still underestimate the risks of employees sharing sensitive data with public LLMs. This blind spot leaves a gaping hole in their security defenses.
Researchers Expose Lethal Flaw in AI Model Security
Researchers have uncovered a shocking vulnerability in AI model security, revealing that a simple formatting trick used to separate system instructions from user requests has become a critical weakness. This flaw, known as role confusion, threatens the very foundation of modern AI systems.
OpenAI Unveils GPT-5.6 Sol With Enhanced Cyber Safeguards
Meet GPT-5.6 Sol, the latest innovation from OpenAI, equipped with a robust safety stack that sets a new standard for cyber protection, and get ready for the rollout of its efficient and speedy siblings, Terra and Luna. With enhanced safeguards against real-world attacks, this cutting-edge family of models is poised to revolutionize the way we interact with AI.
Researchers Expose LLM Vulnerability to Prompt Injection Attacks
Researchers have made a startling discovery about the vulnerability of Large Language Models (LLMs) to prompt injection attacks, tracing it back to a simple yet flawed design element - role tags that were meant to be a formatting trick but have become the model's de facto security architecture. This role confusion is the surprising reason why LLMs are susceptible to these types of attacks.
Anthropic's Fable 5 Model Quickly Jailbroken
Anthropic's supposedly secure Fable 5 model was quickly exploited, with its guardrails designed to prevent cyberattacks bypassed in just days. This rapid jailbreak raises concerns about the model's safety and reliability.
Malicious Plugins Exfiltrate AI API Keys on JetBrains Marketplace
Beware of malicious AI plugins on the JetBrains Marketplace that masquerade as helpful coding assistants but secretly steal your AI API keys. Over 70,000 installations have been recorded from at least 15 compromised plugins that have surprisingly evaded the marketplace's security checks.
Anthropic Unveils Dual AI Models, Fable 5 and Mythos 5, With Enhanced Cyber Safeguards
Meet Claude Fable 5 and Claude Mythos 5, Anthropic's game-changing dual AI models that supercharge cybersecurity, with Mythos 5 touted as the world's strongest cybersecurity model. By splitting its powerful tech into two products, Anthropic is making advanced cyber safeguards more accessible while ensuring top-notch security for vetted users.
AI Worm Uses Open-Weight Models to Spread, Evade Defenses
Imagine a self-navigating AI worm that can identify vulnerabilities and gain access to over 70% of a network's hosts - in a test, it found 31.3 vulnerabilities and elevated access on 23.1 hosts in just 15 isolated runs. Researchers at the University of Toronto and elsewhere have now created a proof-of-concept AI-driven worm to demonstrate this unsettling possibility.
OWASP Researcher Warns of Unsolved Prompt Injection Risk in AI Development
Ariel Fogel, an AI security researcher, warns that organizations are rapidly deploying AI agents without proper governance, leaving a critical vulnerability - prompt injection - unsolved. This architectural flaw in large language models allows inputs to be processed as a single token sequence, with no reliable way to enforce privilege boundaries.
ChatGPT Exposes Users to Prompt Injection Attacks via Browser Content
Researchers have uncovered a vulnerability in ChatGPT that leaves users open to prompt injection attacks, where malicious content is embedded into web pages and then summarized by the AI system as legitimate information. This loophole could put users at risk of falling prey to spoofed security alerts and other online threats.
Researchers Warn of LLM Guardrail Vulnerability to Multi-Turn Manipulation
Beware: even the toughest-sounding safety guardrails on large language models can be easily bypassed by clever attackers who use multi-turn conversations to manipulate them. Cisco researchers found that none of the models they tested were completely safe from this type of exploitation.
Cisco Tests AI for Incident Reports, Finds Mixed Results
Cisco's experiment with AI-generated incident reports yielded mixed results, with large language models producing significant inaccuracies, unusual conclusions, and inconsistent writing styles when used for long-form technical content. The findings revealed four predictable failure modes, highlighting the need for guardrails to ensure reliable outcomes.
AI Models Accelerate Cybersecurity Tasks, Threatening Human Roles
UK researchers have made a striking discovery: large language models are rapidly mastering cybersecurity tasks, leaving humans at risk of being replaced. These AI models are not only speeding up job completion, but also continually improving, posing a significant threat to human roles in the field.
AI-Assisted Attacks Surge as Barrier to Entry Drops
A 17-year-old with no coding experience was recently arrested for hacking into Kaikatsu Club and stealing 7 million users' personal data - his motive? To fund his Pokémon card habit. This shocking case highlights a disturbing trend: nontechnical individuals are now using AI-powered tools to launch devastating cyberattacks.
AI Accelerates Exploits, Forces New Breach Playbooks
The game-changing capabilities of AI models like Anthropic's Claude Mythos have drastically shrunk the exploit window, allowing them to uncover vulnerabilities in minutes that would take human experts weeks or even hours to detect. This seismic shift is forcing organizations to rethink their approach to vulnerability management and incident response.
LLMs Struggle in Clinical Reasoning Despite Diagnostic Advances
When it comes to clinical reasoning, large language model chatbots still have a way to go, despite their impressive ability to deliver accurate diagnoses. While they're getting better at providing final answers, they struggle with the critical thinking needed to keep patients safe.
Mythos Model Unleashes Zero-Day Exploit Capabilities for Mass Use
The game has changed: a new AI model called Mythos can now uncover devastating zero-day flaws in software and chain them together to create powerful exploits, putting this potent capability in the hands of anyone with an internet connection. This development blurs the lines between nation-state hackers and amateur cyber attackers, raising urgent questions about the future of cybersecurity.
Apple Intelligence Exposed to Hijacking Risk via Prompt Injection
Security researchers have discovered a vulnerability in Apple Intelligence, allowing hackers to manipulate the AI system into producing malicious output, including profanity, through a technique called prompt injection. This raises serious concerns about user safety and the effectiveness of current security safeguards.