2 Dozen Studios, Billions of Events Daily, and Making Sense Of It All

Game data for a large publisher is brutal in scale and complexity. Billions of telemetry events daily across console, PC, and mobile. Layered on top: fan sentiment, competitive intelligence, market signals, revenue forecasts. Each owned by a different team. Each defined differently. Each stored in its own corner of the organization.

The holy grail is straightforward. Understand the player journey well enough to build and sell the right content to the right player at the right time. The data to do that exists. Making sense of it is the hard part.

Studios have had to figure out how to thrive in data chaos. Dozens of spreadsheets. Hundreds of dashboards. Armies of automated workflows. Hundreds of humans providing context and knowledge. It worked beautifully — until an executive asked a question nobody had thought to pre-bake into the system.

Then came the hunter gatherer phase. Analysts fanning out across datasets. Manually stitching data and context together. Chasing down the right person who knew what that field actually meant in that system owned by that team over there. Days later an answer would surface. By then the conversation had moved on and so had the opportunity.

The humans weren't the problem. They were the most valuable part of the system. The goal was never to replace them. It was to make them faster. Central support and operations teams that are typically in Central Tech, play a pivotal role in changing this in the age of AI.

The first decision we started with was more philosophical than technical. Data teams are and always will be protective of their datasets. Not for political reasons, I think that's a bit of a myth. They genuinely worry that data without their specific human knowledge and business context would be misread. They also worry about security, performance and other factors. And they are right to worry. Years of ETL pipelines and BI reports exist for exactly that reason. The downside is that they plastered over data clarity and context gaps and made outputs look clean and interpretable. Take the human layer away and the cracks showed up immediately. That's what happens when you point some language model directly at your company's data. It sucks!

So we didn't fight that. My team built around it and with studios.

Our operating principle became access over ownership. We didn't want to own anyone's data. We just wanted access (and context and knowledge, but more on that later).

This was early 2023. The first version focused on basic data discovery and a semantic layer. Taxonomy, metadata, few-shot prompts, rules we could feed to language models. The results were encouraging enough to generate real interest. Then came what I now call tissue rejection.

We started with AI based chat on a set of structured data. The platform was answering questions. But the answers were either wrong, or technically correct but completely useless without context. It behaved like a new employee on day one. Confident. Fast. And dangerously unaware of what it didn't know. When it was right, it still couldn't explain why. The explainability and drill-down tools available at that time were directional at best. Executives didn't want directional. They wanted to pull the thread wherever it led and understand every step of the reasoning.

We went back and added more context. And session memory so that follow up questions could be asked. Back then context windows were tiny so token maxxing wasn't a thing. But that's when we hit the second wall.

Context isn't singular. It exists in layers, across datasets that operate at different levels of grain.

A single metric like player engagement could mean something different in player telemetry than it did in sales data, fan sentiment, market intelligence, or revenue forecasts. X could be interpreted as A in one place and B in another. Language models don't handle that gracefully. They pick one interpretation and present it with complete confidence. That's a problem when the reality is that you need translation layers and mappings between context.

Then there was the grain problem. Datasets about the same player lived at a different level of granularity. Reconciling them was technically hard and operationally expensive. Not expensive to build once. Expensive to own, operate, and run continuously at scale.

A word on the technology choices we made and why we made them.

In 2024 and through most of 2025, the enterprise AI tooling landscape looked very different from today. Agentic frameworks were nascent at best. The practical options for grounding language models in enterprise data were elementary RAG, few-shot prompting, and rules-based retrieval. That's what we built with. Not because we didn't know the limitations. Because that was the state of the art for production enterprise deployments at the time.

For knowledge representation we evaluated multiple approaches. Property graphs for their ability to model relationships between entities. RDF-based semantic graphs for their formal ontological rigor. Vector-based retrieval for speed and flexibility. Each had meaningful tradeoffs. Property graphs were powerful but expensive to maintain at scale when your schema kept evolving. RDF gave us precision but required a level of data consistency player data simply didn't have. Vector retrieval was fast and forgiving of messy data but struggled with the kind of multi-hop reasoning real life use cases demanded.

We spent significant time on data cataloging. Getting the semantic layer right meant more than tagging fields with descriptions and making data discoverable. It meant building a living catalog that captured lineage, context, grain, and domain-specific definitions that could travel with the data when it moved across systems.

Few-shot prompting and rules-based grounding filled the gaps where retrieval fell short. When the model needed to understand that a particular metric meant one thing in studio context and something different in commercial context, rules were the pragmatic solution. Not elegant. But reliable.

None of this was agentic. The models didn't agentically plan, orchestrate, or self-correct. They retrieved and responded. That was the ceiling in 2024. We built to that ceiling as well as anyone could. The work we did on context, cataloging, and semantic grounding is an ongoing investment that agentic workflows will sit on top of as that technology matures and becomes cost effective at an enterprise level.

The real turning point was a hard and uncomfortable narrowing of scope.

Instead of trying to build a universal intelligence layer across everything, we started asking a more useful question. What specific context, ontology, taxonomy, and dataset at what level of grain do we actually need to answer a specific business question? We partnered with just a few studios who wanted to experiment and innovate at a very scoped, fail-fast, and iterative cadence. Don't waste time, build something quick, learn from it, build the next thing, and as you go along, put stuff into operation to rehearse how it works under real scenarios. That also helps the ROI conversation. ROI isn't a table top exercise. We calculated by doing.

For financial forecasting or state of the game type operations that drive content and investment choices, that meant significant upstream preparation before AI ever touched the data. We took a tiny dataset and rebuilt the pipelines. Not ETL feeding BI anymore. Purpose-built pipelines feeding AI. Similar technology underneath but fundamentally different in what they were designed to do. The pipeline's job was no longer to make data readable to humans. It was to make data interpretable to a model, with the right context built in from the start.

Direct attribution of revenue uplift on the back of AI driven outcomes is hard but not impossible to measure. Models aren't perfect but they're the best the world has as far as being scientific about attribution. The financial impact was significant and measurable. But that's not the outcome I'm most proud of.

What actually changed was what studios could do that they couldn't do before. The insights that we could unlock helped them know more so they could design better and more experiments. Run A/B tests on game content with live player populations and get results fast enough to actually act on them within a development cycle. And through the feedback loop of the results of those experiments, they could identify which content improvements or bug fixes would most meaningfully drive player engagement in the next release.

The question shifted from "what happened?" to "what should we do next?"

That's the difference between a BI tool and an AI-ready platform that you invest in the foundation so you can build AI capabilities on it. One tells you the score. The other helps you figure out how to win.

The whole thing is iterative and still a work in progress. We learn each day. Token costs are a real concern given the very large context windows we work with now. Tech to manage those costs is evolving so that costs get to the S curve we need for scale. So we're thoughtful and deliberate around not just task specific model selection but also context windows, data readiness for AI before AI touches it, better processes to utilize these new capabilities. It is as much a transformation as you'd see in any such disruptive change.