Alex König

Trust & Autonomy

There is a weird irony in many companies right now when it comes to trust and autonomy. Especially traditional companies often will not let a developer spend 50€ on trying out a new tool. Yet they are totally fine with handing an AI agent production access to GitHub, AWS and maybe even their own credit card and phone number.

Sometimes this is a trust issue. Sometimes it is a lack of proper feedback loops. Either way, it has a negative impact on developer autonomy, leading to a lower rate of experimentation.

Trust and autonomy have always mattered. You probably know that. I simply think AI makes their absence more expensive. When a team can build and test an idea faster, the approval process can become the longest part of the work.

That is the argument I want to make here: give your people room to act, with clear boundaries and responsibilities.

Trust & autonomy for tools

AI is a tool, and in tech we evaluate new tools all the time: Kubernetes, a new web server, a different tool for managing data lineage, a new build system. The process is familiar. You find out what a tool can do, try it out on something low-risk, benchmark it, try to break it on purpose to learn its failure modes, and at some point ship it to production. First in one project, then the next and so on. That “some point” is different for everyone. It is the point at which you trust the tool. We already delegate actions to tools: Kubernetes controllers, for example, act to bring a cluster towards a desired state.

With AI agents, the range of tasks and decisions we can delegate is broader. How much an agent can work on its own depends on how much trust you hand it (and the model, and the provider hosting it… and a bunch of other actors/tools in the chain). A metaphor you often hear these days for building trust in AI is a ladder with a varying number of steps. Let’s take four steps here:

  1. Draft and approve: The AI drafts, you review and approve every single step.
  2. Review in aggregate: You allow certain actions by default and only review batches of work afterwards.
  3. Scoped autonomy: You grant defined tools, credentials and boundaries, and check in periodically.
  4. Full autonomy: The equivalent of yolo mode: all commands, file access, its own credentials.

Going higher is a trade-off. Yolo mode can be the wrong choice even for a tool you trust.

Same as before, you climb the ladder by iterating with the tool, and at some point you trust it enough to hand it real responsibility. Depending on your setup, the agent now has its own credentials for GitHub, AWS, email, as mentioned above maybe its own credit card and phone number (again: the irony).

Most organisations I work with or hear about in my job are already quite far up this ladder. They trust AI a lot and grant it a lot of autonomy.

A tool is a tool, is a tool. Right?

One thing many organisations, or at least much of management, fail to understand about generative AI: its outputs can be non-deterministic. Configure a web server and you expect it to serve the same static page on each request. Put a calculation into Excel and you expect the same result for the same inputs. Ask Claude to write a text and the result can vary between runs, even for the same prompt. AI is still a tool. It is just non-deterministic.

There are ways to manage this non-determinism and limit its consequences. Few-shot prompting gives the model examples of what you want. Allowlists restrict the actions it can take. Tests, reviews and guardrails check whether inputs, outputs or actions meet your requirements. AI judges can help with evaluation, but their assessments need checking too.

These measures help you decide which behaviour is acceptable and what to do when it is not. You still need to account for variation in the model’s output.

We already know how to work with uncertainty in people. Humans are non-deterministic too. Mood, (missing) context, fatigue, turnover. There are endless reasons why a human makes mistakes. In healthy organisations there are guardrails, reviews and most importantly feedback loops to manage this. To be fair, giving good, constructive feedback is a skill in itself.

We seem to be relearning the same for AI right now.

If you are reading this, you probably know much of the above. So why spell it out? Because trust and autonomy look completely different the moment humans are involved.

In machines we trust

Many organisations hand more trust and autonomy to AI than to their human employees these days.

Producing vast amounts of code, ideating new features for products or even whole businesses, buying things online, handling customer complaints. Name the task and there already is a company that has handed that off to AI at least a bit.

Some of these tasks are non-trivial and even vital to the business. But somehow, if you were to walk into a random company and ask their CEO whether:

The answer will be “no” or a bureaucratic process, which will take longer than the experiment itself.

Put trust in human autonomy

What I observe at almost every customer who is successful with AI is the same pattern:

They trust their teams to build the right thing and give them the autonomy to actually do it, leveraging AI heavily in the process. They minimise red tape and instead give teams high-level guardrails to prevent the absolute worst case. Inside of those guardrails, teams are free to execute.

That is the whole trick. Implementing it at organisational scale is obviously harder. It does not work with every team, employee and manager. Which is why you can reuse the evaluation process from the tool ladder, only with a different baseline. You already know your team, and I really hope there is at least some trust for them.

Start small, iterate and increase trust along the way (remember the ladder):

Pencil drawing of a wooden ladder with four rungs, leaning to the right on a plain white background.
From approval per action, through budget discretion and bounded experiments, to ownership of outcomes.

Your goal is a level of trust and a set of guardrails that allow you to increase your teams’ autonomy to an uncomfortable level. If it never feels uncomfortable, you have not gone far enough.

Some companies concentrate this in dedicated teams: R&D, a Labs team, a pirate ship; there are many names. These teams are small, currently two to three people at cutting edge companies.

They need customer obsession, strong AI skills, a product mindset, design skills, a strong understanding of software architecture, and a strong understanding of how your business and its economics work. It is unlikely to find all of that in one person. But with AI, a team like this can move remarkably fast and find out what might come next for your company, industry and product.

You might just have guessed it, but that team needs: trust and autonomy (imagine a group of people saying this here). Most companies probably already have such a team, but they are often still blocked by red tape and approval processes.

Why now?

Why is this all so important now, if it always has been? Because AI has massively driven down the cost of building and the time needed to build.

Imagine a team wants to run a small experiment for a new user experience. Suppose AI helps them build a prototype in an afternoon, but getting permission to try it takes two weeks. The faster build barely changes how quickly they learn whether the idea is useful.

Give that team an agreed budget, a safe environment and a review date, and they can test the idea with a few customers. They can find out what works, what does not, and whether to invest more time.

A prototype still needs work before it can become a production system. Security, reliability, integration and ongoing ownership all matter. The opportunity is to answer the first question sooner: is this worth pursuing?

You can just do things now. Small teams with a small budget and AI can try ideas that previously would have needed a much larger investment and simply outpace teams which are waiting for budget and approval.

The speed of development really is insane right now, even outside of pure AI companies/labs. This technology is here to stay, whether the bubble bursts or not. Does not matter at this point.

I recommend that you put your trust in human autonomy. At Amazon we have leadership principles that define how we work. Four come to mind, which are especially interesting right now:

You cannot (and should not) copy culture. I still suggest at least taking a look at those.

Red tape, clearance meetings and approval chains steal time from your teams. Imagine what they could do instead with the trust and autonomy to execute without constant blockers.

So give your people at least the trust you already hand your AI agents. Put trust in human autonomy. The organisations that do will likely be around longer than the ones who do not.

PS: the notes from above apply. Trust & autonomy without guardrails are not a good idea.

PPS: if you work at a company that has banned AI entirely, invest in your own future, spend some money and time on learning how it works, how to use it and get out of that place asap.

← All posts