AI can help security teams move faster. But speed alone does not tell you whether people are making sound decisions or whether agents are still doing the job to the expected standard.
As agents take on more security work, leaders need a clearer picture of capability across both kinds of worker: human and agentic.
That was the focus of The Four Hidden AI Risks Your Dashboard Won’t Show, the first session in Hack The Box’s AI Masterclass Series. Hosted by Christine Bartlett, CMO at Hack The Box, with Dimitris Bougioukas from HTB’s training leadership and Kevin Delaney, Senior Director of Solutions Engineering, the conversation explored how to keep human judgment sharp, develop skills and maintain confidence in agent performance.
Here is what stood out.
When agents take on security tasks, people increasingly become responsible for directing the work, reviewing recommendations, and deciding when to intervene.
That requires more than familiarity with an AI interface. A confident, well-written recommendation can still be wrong. Operators need the domain expertise to understand the evidence behind it, recognize what may be missing, and decide whether the proposed action makes sense.
The human role is changing from execution alone to direction and verification. Technical fundamentals remain essential: they give operators the knowledge to challenge a conclusion or take an investigation further.
The takeaway: Develop the judgment to direct, verify, and challenge agents alongside the technical skills needed to understand their work.
A queue that keeps moving can conceal changes in the capability behind it. The session explored four connected risks:
These risks can reinforce one another. If an agent’s capability declines while its human operators lose the practice needed to recognize the change, the problem becomes harder to spot.
Understanding these connections helps leaders plan development and appraisal around the whole workforce, rather than addressing each issue in isolation.
The takeaway: Assess human capability and re-appraise agents together. Each affects the team’s ability to recognize and respond to gaps in the other.
Automating routine work can free people for more demanding responsibilities. Security leaders also need to preserve the practice that prepares them for those responsibilities.
Junior professionals still need opportunities to follow an investigation, interpret evidence and explain why a decision is justified. Those experiences build the judgment they will later need to oversee agents.
Experienced operators need regular practice too. Their ability to take over matters when an agent is unavailable, encounters an unfamiliar situation or reaches the wrong conclusion.
Unaided practice deserves a deliberate place in the workforce development plan. It helps teams retain the skills that make AI-supported work effective.
The takeaway: Give operators opportunities to exercise their skills both with and without AI assistance.
The metrics leaders choose shape the behavior they encourage. Measuring only approval speed, automation rates or closed alerts can reward fast decisions without revealing whether those decisions were sound.
A fuller picture includes the reasoning and interventions behind AI-supported work:
Approval logs can provide a starting point. A pattern of almost immediate approvals may warrant closer investigation, but timing alone is insufficient. Leaders also need context: the evidence available, the actions taken and the reasoning behind them.
Hands-on exercises create opportunities to capture these decisions and identify where further development would help.
The takeaway: Measure demonstrated judgment alongside speed and task completion.
As AI becomes part of everyday security workflows, readiness also becomes a business continuity question.
If an external AI service becomes unavailable, can the team continue investigating and responding? If an agent’s performance changes, can operators recognize the problem and take control?
These questions connect workforce development to operational resilience. Teams need to understand where they depend on agents, which responsibilities people must be able to resume and what skills those responsibilities require.
The takeaway: Include human capability and agent dependencies in continuity planning.
Start with existing job descriptions and workflows. Identify which responsibilities agents may take on, which skills people still need and how those capabilities will be assessed.
A practical starting point is to:
Your security team now has two kinds of worker. Building readiness means developing people, appraising agents and understanding how both perform together.
Watch The Four Hidden AI Risks Your Dashboard Won’t Show for the full conversation on building readiness across your human and agentic workforce.