@ericyocam6Professor, California Polytechnic State University
How Much Autonomy Should We Give AI Agents?
Which approach are you currently using to control autonomous AI agents in production or experimental environments?
@ericyocam6 @ericyocam6 @ericyocam6Which approach are you currently using to control autonomous AI agents in production or experimental environments?
Comments2
One real case where an agent running in production took an action nobody approved would make this poll much easier for founders here to answer. Could you share what happened, what it cost the business and which of these control approaches would have stopped it?
Thanks, Ryan. That's a good suggestion. I don't have a verified production incident with documented financial losses to share, but my research on agentic AI security demonstrates how unauthorized actions can occur when agents process untrusted information. For example, we've shown how malicious instructions hidden in seemingly harmless images can be extracted and passed through an agent workflow, potentially triggering actions that were never authorized. In our controlled experiments, provenance-based trust enforcement helped prevent those actions. This highlights why I favor a combination of least-privilege permissions, provenance verification, and human approval for high-impact decisions. The challenge is balancing autonomy with accountability without eliminating the efficiency benefits of agentic AI. Appreciate the feedback!
2 comments
One real case where an agent running in production took an action nobody approved would make this poll much easier for founders here to answer. Could you share what happened, what it cost the business and which of these control approaches would have stopped it?
Thanks, Ryan. That's a good suggestion. I don't have a verified production incident with documented financial losses to share, but my research on agentic AI security demonstrates how unauthorized actions can occur when agents process untrusted information. For example, we've shown how malicious instructions hidden in seemingly harmless images can be extracted and passed through an agent workflow, potentially triggering actions that were never authorized. In our controlled experiments, provenance-based trust enforcement helped prevent those actions. This highlights why I favor a combination of least-privilege permissions, provenance verification, and human approval for high-impact decisions. The challenge is balancing autonomy with accountability without eliminating the efficiency benefits of agentic AI. Appreciate the feedback!