An exclusive feature on Harsh Verma’s pioneering work in AI security, engineering judgment, and the future of trustworthy systems
In the high-stakes world of enterprise AI, the gap between a promising prototype and a production-ready system is vast, and it’s where careers are made and broken. Few understand this chasm better than Harsh Verma, Principal AI Engineer at Palo Alto Networks and an advisor to Berkeley SkyDeck, whose insights are reshaping how the industry thinks about AI reliability, security, and the future of engineering itself.
Recently featured in the University of Berkeley Skydeck July Newsletter’s Advisor News Section and in an in-depth HackerNoon interview, Verma has emerged as a critical voice in the conversation about what it actually takes to deploy AI that can be trusted under real-world pressure. His Forbes Technology Council article on the hidden trade-offs between AI personalization and privacy further cemented his reputation as a thought leader who bridges technical depth with strategic foresight.
The Crisis of Confidence in Production AI: “The biggest gap isn’t intelligence, it’s survivability in the real world, ” Verma declares, cutting through the hype that often surrounds AI deployment.
This isn’t just provocative rhetoric. It’s born from hard-won experience spanning a decade across research, system design, and software engineering in AI. Like many engineers early in their careers, Verma once believed that a model performing well in evaluation was ready for production. That assumption shattered the first time he had to deploy AI into a live, large-scale system.
What he discovered is a truth that resonates across the industry: “models don’t fail loudly in production, they fail quietly, and at scale.” And in that quiet failure lies enormous risk.
The statistics bear this out. Research indicates that between 70% and 87% of enterprise AI initiatives fail to reach production. The average time from pilot approval to production is 18 months against a planned duration of just six. Fewer than one in seven enterprises has successfully scaled an AI agent from pilot to production.
From Accuracy to Uncertainty: Verma vividly recalls deploying a high-performing anomaly detection model that looked excellent offline. In production, it began flagging normal behavior as threats while missing coordinated, low-signal attacks entirely. Nothing was technically wrong with the model, the system simply needed context over time, not point-in-time predictions.
That moment crystallized a fundamental shift in his thinking: “In research, you optimize for accuracy; in production, you engineer for uncertainty.”
The world doesn’t stay still. Data drifts, users evolve, and in adversarial domains like cybersecurity, attackers adapt. A production model effectively starts decaying the moment it’s deployed. The real question becomes not “Is the model right?” but “What happens when it’s wrong?”
This shift from research to production, and into adversarial environments, forced Verma to move beyond thinking about models and toward designing resilient, intent-aware systems. “The unit of innovation in research is the model. In production, the unit is the end-to-end system operating under uncertainty.”
The Security Paradox: Most machine learning assumes data is benign. Security ML has a property most domains don’t: an adversary is actively trying to defeat your model. This changes everything.
“In most ML, you learn from data. In security, you learn against an opponent, ” Verma explains. Attackers don’t break models; they study them and operate just below their thresholds. “A model that’s predictable is a model that’s vulnerable. “
This requires abandoning single classifiers for layered, adaptive systems that combine models with behavioral signals, sequence analysis, and continuous monitoring. It also demands a shift from identity to intent, because in sophisticated attacks, every action can look valid in isolation. The threat only emerges when behavior is connected over time.
“In adversarial environments, the goal isn’t perfect accuracy, it’s being hard to evade. “
The Crisis Nobody’s Talking About: Agent Security: As autonomous agents become more prevalent, Verma identifies an emerging crisis: “Identity-based security breaks when software starts making decisions.”
In traditional systems, we secure who is acting, users, services, API keys, and assume anything within that boundary is safe. This model held up until autonomous agents entered the picture. Agents with valid credentials and correct permissions still create risk, not by breaking rules, but by executing multi-step actions that were individually allowed but collectively harmful.
Verma has seen agents unintentionally create security gaps simply by chaining decisions in ways no human explicitly designed. The system says “allowed” at every step, yet the outcome is still wrong.
Where it breaks:
- Identity ≠ Intent: A trusted agent can still produce unsafe outcomes
- Static permissions vs dynamic behavior: Policies don’t adapt to evolving goals
- Single actions vs sequences: Risk emerges across steps, not individual requests
“We built security around who can act. Agents force us to secure what is being attempted over time. We’re moving from identity-based security to intent-aware, behavior-driven systems. “
The Privacy Trade-Off: Smarter vs. Safer: In his Forbes article, Verma explores another critical dimension: the tension between personalization and privacy. “The most advanced AI systems today are defined by how well they know you. This powerful attribute also makes them dangerous. “
Modern AI is no longer stateless. Systems remember preferences, predict behavior, understand routines, and continuously personalize interactions. While this makes them convenient, it means they’re more intertwined with users’ lives, collecting patterns, guessing intentions, and storing personal information.
Verma frames this as the industry’s defining trade-off: “Better personalization means more memory, more context awareness, and better behavioral understanding. But the other side of this coin means increased exposure to risk, privacy concerns, challenges with compliance, and user distrust. “
The privacy concern extends beyond what users explicitly share. “Privacy is no longer about what users choose to share. It is now about what systems can deduce over time. The actual risk to privacy in AI is not in the data collected alone, but how AI interprets it at a larger scale and then applies its intelligence to that data. “
Engineering Beyond the Code: Perhaps Verma’s most provocative thesis concerns the future of engineering itself. As AI generates more code, the differentiator moves to judgment, systems thinking, and the ability to frame the right problem.
“In my day-to-day life, the work that still matters isn’t writing functions, it’s making decisions under uncertainty. ” A senior engineer spends time framing problems, anticipating failure modes, and understanding what happens when systems run at scale or under adversarial conditions. AI can suggest implementations, but it doesn’t own the consequences.
“Junior engineers write code. Senior engineers define the boundaries within which code is allowed to operate. “
This isn’t an excuse to drift away from technical depth, Verma cautions. “The critique is right the moment ‘beyond the code’ means losing touch with the system, you’re no longer engineering. ” The best engineers can jump in, debug, and understand exactly where things break. “You can’t design systems you can’t debug. “
The Pipeline Problem: When AI handles more coding, how do junior engineers develop real technical judgment? Verma argues the pipeline isn’t broken, it’s shifting.
“If juniors write less code, they need to think more about what the code actually does. ” He’s observed junior engineers grow quickly when they review AI-generated code and catch issues like hidden assumptions or edge cases the model missed. “You don’t build judgment by writing more code, you build it by understanding where code breaks. “
The focus needs to move from writing tickets to owning outcomes, debugging incidents, analyzing failures, and making trade-offs. “The engineers who become strong now aren’t the ones typing the most, they’re the ones who question, connect systems, and anticipate failure before it happens. “
The Compounding Move: For engineers who want to act on this insight, Verma offers concrete advice: “Spend a year owning a system end-to-end in production, that compounds more than anything. “
The highest ROI move is to take responsibility for a real system that runs, breaks, scales, and impacts users, and follow it through failures, incidents, and iterations. “You don’t build judgment by building features, you build it by owning outcomes. “
The wasted effort is chasing volume: more code, more side projects, more tutorials without exposure to real constraints. “If nothing breaks, you’re probably not learning the right things. “
Harsh Verma’s insights cut through AI hype to reveal the real challenges facing enterprises. The production gap isn’t just technical, it’s a systems problem, a security problem, a privacy problem, and increasingly, a human problem.
“The future of AI won’t be defined by smarter models, but by systems we can trust under real-world pressure, and that’s the gap most engineers only truly understand after deployment. “
As AI continues its march into production, the engineers and organizations that thrive will be those who recognize that the goal isn’t smarter models. It’s systems that are resilient, adaptive, and secure, systems we can trust when it matters most.
