OpenRouter’s AI gateway, the largest in the world, has surpassed human usage in token counts and burned a staggering amount of energy.
The Unexpected Leader in AI Inference
Chris Clark, COO and co-founder of OpenRouter, runs this behemoth of an AI system – 70 model providers, hundreds of models, and an integration that just keeps working. Clark shares that the AI gateway’s usage has recently eclipsed human activity, a milestone that holds significant implications for the industry.
Tokens and Energy: The Unplanned Consequences of AI Growth
The AI gateway’s rapid growth in token usage has come as a surprise to Clark and his team, who hadn’t budgeted for the surge. The system consumes around 1-2 megawatts of power, which, while not an enormous amount, is still more than anyone anticipated. Clark attributes this unexpected energy consumption to the AI gateway’s ability to scale rapidly, adapting to and incorporating new models and providers at an unprecedented rate.
The Road Ahead: A Wake-Up Call for AI Developers
While the current infrastructure of the AI gateway functions seamlessly, Clark emphasizes that this isn’t necessarily a scalable model for the future. As AI continues to grow and evolve, Clark warns that the industry needs to prioritize infrastructure and energy efficiency. The high energy consumption and rapid token usage underscore the need for sustainable development and more thoughtful planning in AI growth.
What this means: Expect more emphasis on responsible AI development and infrastructure planning going forward. As AI continues to advance, developers will need to prioritize energy efficiency and scalability to avoid similar surprises.
