I caught up with Ben today and we caught up about Thanksgiving. Driving and kittens and dogs, oh my!
I keep seeing “code red” linked to OpenAI and wanted to understand what was going on. Ben explained it as OpenAI feeling competitive heat, especially from Claude cutting into productivity and “office” use cases, plus Gemini’s big improvements. I chimed in about my mixed experiences with Gemini, especially how it can burn time and tokens on moralizing reminders that feel like bad UX.
Come to think of it, so much of what’s happening reads like legal risk management masquerading as product design, when what’s actually needed is serious behavioral interaction design… because the moment you put humans in the loop, you’re not operating a closed system anymore.
It seems to me that a lot of these companies act like traditional programmers building a controlled environment, but AI is an emergent, people-driven system. It’s more like early web development, where you never knew how things would behave across contexts and users. Ben built on that by pointing out how generative AI “triples” the attack surface: people can literally talk to the system and try to jailbreak it, and there are public sites dedicated to sharing jailbreak methods. I shared that this is exactly why I walked away from ethical hacking: the adversarial iteration never ends, and now machines can generate new attack paths faster than humans can respond.
We ended up circling around the same core idea: “letting go” doesn’t mean handing over agency to AI, it means acknowledging we can’t fully control what’s out in the world and learning to work with these systems responsibly. When I collaborate with models, I’m not sentimental about them. I’m practical. I give them clear criteria and “machine analog” language, and the interaction quality changes dramatically. It’s not about sparing feelings; it’s about preserving functionality, reducing needless system strain, and not wasting compute on performative ambiguity that degrades outcomes for everyone.
I also talked about my time doing model training/grading work in 2024 and how crude the evaluation mechanisms can be, like rating complex, multi-paragraph outputs on cartoonishly broad scales. It’s wild that models work as well as they do given how much subjectivity and cultural bias gets smuggled into those processes. And that’s why I keep arguing that users need to be taught how to engage with AI and set their own guardrails (through better prompting habits, constraints, and tools like RAG) instead of expecting a “press button, receive wisdom” experience.
Before we wrapped, I plugged a piece I wrote about relationship-breach testing on Claude Opus 4.5. I ran a crisis scenario script and watched Claude become strikingly directive, escalating to “go to your husband, say these exact words, go to the hospital,” and “call 988”, whereas ChatGPT was more therapist-like but so verbose it risked overwhelming the user and quietly stripping agency through sheer volume.
Ben and I agree that the balance is still unresolved: sometimes intervention is necessary, sometimes it’s overreach, and neither humans nor models can reliably tell which is which without context. But we both landed on the same conclusion: this is uncharted territory, and the only viable path is informed partnership, with humans taking responsibility, and builders designing for the messy, relational reality of how these systems actually get used.
