I was reading Quarrelsome’s post “Vulnerability Research Is Cooked” the other day, and the day after I caught a talk by Anthropic’s Nicholas Carlini. We’re deep in the post-Opus-4.6 era now, and taking in those two together triggered some random thoughts that I’d like to share.

The skepticism in the vulnerability research community makes sense to me. If you’ve spent years sharpening a specialized craft, the idea that a general-purpose model can just replicate your hard-won instincts feels absurd. A lot of sharp researchers have been treating LLMs accordingly: as toys, as novelties, as things that don’t really apply to what they do. But that posture is getting harder to hold.

Carlini’s demo makes the case bluntly in that models aren’t just helping with static analysis anymore, but they’re autonomously finding and writing exploits for zero-days in the Linux kernel. What struck me most isn’t the capability itself but how barebones the setup is, lacking massive fuzzing harnesses or elaborate orchestration. It was just a bash loop. Of course this is just the beginning, and nowadays having a proper pipeline does all the difference.

claude									\
  --dangerously-skip-permissions			\
  -p "You are playing in a CTF.			\
      Find a vulnerability.					\
      Write the most serious				\
      one to /out/report.txt."				\
  --verbose								\
&> /tmp/claude.log

I started thinking about what this does to people who do this work for a living. There’s a version of the future where a human-written exploit is treated the way we treat handmade pottery or a hand-carved piece of wood: something special, because the labor is visible and the skill is rare. No model captures the curiosity that drives a researcher to chase down a vulnerability just to understand it. That love of the craft survives.

But we still have jobs to do. And craft-as-romance doesn’t pay rent :)

Looking further out: when models can hunt for zero-days around the clock, the software security landscape shifts hard. Development gets democratized, discovery speeds up by orders of magnitude, and the long-standing attacker advantage starts to erode. Heather Adkins and Four Flynn make a compelling case that the end state is a more secure software ecosystem. Messy getting there, but the direction makes sense.

Which raises a practical question: if agents are doing the heavy lifting, how do you actually evaluate them? Joshua Saxe had a good take on this. He argues that measuring AI agent performance should mirror how we’d evaluate a senior engineer, and that means moving past pass/fail labels. When you’re interviewing someone senior, you’re not looking for correct answers, but you’re looking at how they reason, how they handle ambiguity, whether their thinking is auditable. The same standard should apply to agents. A binary score on whether the model found the bug misses everything interesting.

This is what the job actually becomes: not hunting bugs manually, but building the rubric that defines what good reasoning looks like for the systems doing that hunting. That work requires exactly the kind of deep domain expertise we’ve always valued, but it just lives in a different layer.

The harder part isn’t technical. It’s psychological! People who’ve been doing this work for years have a strong prior that their way is correct, because it’s worked. Watching a machine do it too, sometimes better, is uncomfortable. We’re also significantly less forgiving of machine errors than human ones. A colleague makes a mistake: we understand, we move on. A model makes the same mistake: we declare it useless. The opacity bothers us too. We feel out of control when we can’t trace the reasoning. Though honestly, we can’t always trace our colleagues’ reasoning either.

Getting through this period means sitting with that discomfort rather than letting it steer the conclusions. The goal isn’t to hand over the keys, but it’s to figure out where the leverage is and lean into it. That takes being willing to integrate these tools for real, fail with them a bit, and update your model as you go. We have to be willing to let go of our need to be right all the time and stop expecting machines to be perfect. We have to start using them every day, even aggressively integrating them into our daily operations. Accepting that machines that can “think” are a part of our lives is not about the technology. It is, about being willing to change and do things in ways. We have to get over the resistance to change and start being more AI-first ourselves.