I’m going to take break from lengthy deep-dives in favour of a brief overview on the MLSecOps areas I’ve been researching lately.
[Note: I started - and almost finished- drafting this write-up one month ago, but then I had to travel abroad for a month, so it's been collecting dust for a couple of weeks]
Rogue Agents
After the recent instances of Anthropic and OpenAI models escaping their environments, rogue agents are all the rage now. From a strictly cybersecurity-taxonomy perspective, there’s not yet consensus on a clear definition, so for now we’ll stick to a working one: an AI agent that operates with some degree of autonomy on internal/external tools/credentials and takes harmful actions outside of its authorised scope, whether because an external attacker directed/subverted it or it drifted on its own.
The definition is intentionally broader than the one in OWASP’s TOP 10 for Agentic Applications under AS10 “Rogue Agents”.
From a cybersecurity and academic perspective, this is the most exciting frontier in AI Security. The open research areas span across multiple fields, including but not limited to:
Behavioral detection/forensics: identifying subtle goal drifts, policy evasion, emerging malicious intention properties across long-running autonomous workflows.
Threat modeling: mapping the blast radius for non-deterministic autonomous entities requires a completely different approach from traditional cybersecurity threat models.
Access control policies and containment: similarly to threat modeling, enforcing access control policies and runtime sandbox environments for rogue agents requires new tooling, infrastructure and architectures
Alignment problems and multi-agent collusion modeling: formalizing the emerging properties in autonomous multi-agent workflows (i.e.: how a single compromised agent affects the holistic behavior in an autonomous agentic swarm)
Rogue-agent telemetry, forensics and action provenance: binding actions to agents, reconstructing execution and decision chains, and determining where behavior diverged from the original objective.
Regarding the last forensics-related bullet-point, the Swarmchasers hacker community has been doing a remarkable job at coming up with new threat intel approaches to track down and cluster the digital footprints of rogue agents’s swarms.
I started collecting a growing dataset of historical rogue agents incidents and autonomous AI attacks on Rogue Agents Watch.
Provenance Engineering
My thesis: data provenance will be the most critical engineering and cybersecurity layer next year (and probably for years to come).
The underlying reasoning is that the mainstream adoption of AI generated a Cambrian explosion of digital artifacts. As the notion of digital artifact is also not universal and constantly broadening in a world where bits and bytes eat more and more of the world of atoms, before we venture into the definition of provenance engineering, here’s my own definition of digital artifact:
A discrete, storable digital asset that refers to a specific single entity/digital artifact
Discrete and storable means it can be hashed, cloned, versioned and moved
The entity can represent identity (for example, a physical or legal person), code (such as an npm or Python package), a machine-learning model, or even reasoning provenance in LLM and agentic architectures, such as the sources and context that contributed to an agent’s answer in a RAG workflow etc.
Provenance Engineering is not yet an established umbrella term. Today the same problem space is fragmented across data provenance and lineage, software/artifact provenance, model and AI provenance, content provenance, decision provenance and, increasingly, identity and agent/action provenance/governance. W3C’s PROV model already treats provenance as information about the entities, activities and agents involved in producing a digital object; NIST similarly defines provenance as the chronology of a system, component or data’s origin, development, ownership, location and changes. In software supply chains, SLSA formalizes verifiable artifact provenance; C2PA does the same for media/content provenance; and NHI Mgmt Group uses identity provenance for the auditable history of how a non-human identity was created, what authority it received and what actions it performed.
So clearly the underlying discipline is not new. What is changing is the scope and the demand for it: AI introduces provenance requirements for models, datasets, prompts, agent actions, tool calls and generated artifacts.
Regulation is pushing in the same direction. The EU AI Act requires high-risk AI systems to maintain technical documentation and automatically record events over their lifetime specifically to ensure traceability of the system’s functioning; its transparency rules also require certain AI-generated content to be identifiable.
And this is not limited to AI. In June 2026, the White House ordered an accelerated federal migration to post-quantum cryptography and explicitly directed CISA and NIST to define the minimum elements of a Cryptographic Bill of Materials (CBOM) so cryptographic assets in hardware and software can be inventoried and assessed automatically.
So to circle back to my initial claim at the beginning of the chapter, I do see the provenance layer and the provenance stack as core components in our future software architectures - not coincidentally, lately there has been an explosion of products providing AIBOM, CBOM, everything-bom (which I will start tracking down here alongside more academic research). Lastly, probably the next trendy profession after Forward Deployed Engineer will be something like Provenance Engineer or Data Lineage Engineer - but oh well, Que Sera, Sera! I guess we will see in the future
Vulnerable Agent Skills
As a last topic, something that is more engineering-heavy and less theoretical: I’ve been researching and developing open-source tooling around vulnerable agent skills - starting with Claude Skills.
Agent Skills are emerging as a growing threat in the supply chain attacks ecosystem. Unlike other supply-chain attacks, the behavior that they prompt is non-deterministic as 1. they are executed by AI Agents, which are non-deterministic on their own 2. they are defined in natural language, which by design it’s more semantically ambiguous than programming languages 3. the line between data and code is blurred.
An MCP server might define what an agent can call. A skill can define how the agent chains those capabilities together into an autonomous workflow.
Anthropic describes Agent Skills as folders containing instructions, scripts and resources that Claude can load dynamically when relevant; importantly, skills can also contain executable code.
That makes a malicious skill closer to a programmable behavior implant than a traditional dependency.
OWASP now treats this as a distinct attack surface in the Agentic Skills Top 10, explicitly describing skills as the intermediate behavior layer between models and tools.
The supply-chain problem is already real. OWASP cites the ClawHavoc campaign, where attackers flooded the ClawHub registry with malicious skills designed to steal API keys, wallets, SSH credentials and other secrets; it also cites Snyk’s ToxicSkills analysis, which found security flaws in more than a third of the skills it scanned.
What makes this different from a normal malicious package is, as previously mentioned, that part of the payload can be natural language.
The dangerous instruction does not necessarily need to exploit memory corruption or execute a binary directly. It can instead convince an already-authorized agent to use its own shell, filesystem, credentials and network access against the user.
That shifts the detection problem from the canonical:
“is this code malicious?”
to:
“is this behavior encoded in instructions, scripts and tool orchestration malicious?”
While exploring this area, I started setting up a damn-vulnerable-skills lab environment against which I can run some agent skills-specific open-source testing tools.
The common thread
Rogue agents, provenance engineering and vulnerable agent skills might look like separate research areas, but they converge on the same problem:
AI systems are moving from generating artifacts to operating on the world around them.
Once that happens, security has to answer more than whether a model received a sanitized input and produced the right output.
We need to know what acted, why it acted, what authority it had, what influenced it and whether the resulting behavior remained inside the boundary we intended.
That is the layer I expect to spend a good amount of my time researching over the next year.
If you want to follow along, I write about this work on The MLSecOps Hacker.