Amazon rethinks AI usage metrics

Amazon has acknowledged the flaws in its internal AI usage metrics and is shifting its focus from tracking token consumption to emphasizing real outcomes. The company is urging teams to prioritize meaningful results over superficial indicators of AI adoption.

Internal tool for measuring AI use revealed

In May, an internal tool designed to measure AI usage was exposed through a leak. This tool tracks how often AI is used, with managers pushing teams to incorporate it into daily tasks. However, this pressure has led some employees to misuse AI tokens.

Tokenmaxxing is a broader trend

The practice of pushing AI usage to the extreme, known as tokenmaxxing, is not limited to Amazon. Many companies are forcing employees to use AI, sometimes inappropriately, in an effort to boost productivity. This has also led to increased costs from API usage.

Amazon halts internal AI ranking

In September, Amazon ended its internal AI ranking system. A memo from Terk Terkowitz, vice president of engineering, explained the new approach to measuring team performance. The memo highlighted the risks of using metrics like token consumption as goals rather than tools for evaluation.

New focus on responsible AI use

Terk Terkowitz emphasized that metrics like AI tool adoption, code changes, and code reviews should be used with caution. He encouraged developers to assess AI's value in their work and to document cases where it doesn't help, promoting a culture of discernment and responsibility.

Long-term impact on customers and teams

The memo stressed that the ultimate measure of success should be the long-term impact on customers, teams, and business activities. This approach prioritizes quality and meaningful outcomes over the sheer volume of AI usage.

CEO outlines AI's future at Amazon

In June 2025, Amazon CEO Andy Jassy shared his vision for generative AI. He predicted that AI will accelerate innovation at the company but also noted a potential reduction in staff due to efficiency gains. However, he also suggested that new roles will be created as AI transforms operations.