AI risk, hidden signals, and zombie data: IT news and research roundup

Researchers at security firm Varonis have demonstrated a one-click attack against Microsoft 365 Copilot that could access and exfiltrate information from a user’s environment. How did it do this? By asking the AI assistant directly. Although it initially refused, it eventually revealed the guardrails the researchers needed to bypass.

Microsoft has since mitigated the vulnerability, but the incident highlights a broader challenge for IT leaders: as AI assistants gain access to more tools, security risks increasingly shift from the model itself to the systems and permissions surrounding it.  

^{Read more: Microsoft Copilot reveals secret input that allowed it to be hacked ↗}

AI safety researchers have published a study on the idea of ‘subliminal learning’, showing how data produced by one model can pass on behavioral traits to another via hidden signals. Even when the data has nothing to do with those traits.

The outcome, as Owain Evans, one of the researchers, said, was: “Datasets consisting only of 3-digit numbers can transmit a love for owls, or evil tendencies”. An amusing story––if it didn’t also contain worrying implications for LLMs (as well as a world) increasingly founded on AI-generated content.  

^{Read more: Subliminal learning: language models transmit behavioral traits via hidden signals in data ↗}

Lastly, another interesting study about LLMs, which in this case created an LLM (LittleLearner) trained to mimic the capabilities of an average 10-year-old.

The study found that injecting new knowledge through post-training and in-context learning did not make the model smarter. Instead, it only "let LittleLearner better utilize existing knowledge", which might be an uncomfortable idea for those convinced that those same strategies might meaningfully contribute to artificial general intelligence (AGI).  

^{Read more: LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure ↗}