Executive Summary
Enterprise AI has a context problem. Every large organisation now has access to powerful language models, but access to models has not translated into reliable, defensible AI outcomes. The organisations achieving measurably better results are those that have engineered what surrounds the model: the retrieval pipelines, evaluation frameworks, governance infrastructure, and domain knowledge that determine whether a model’s output is accurate enough, auditable enough, and current enough to act on. At Moody’s, we call this surrounding infrastructure connected intelligence — and this paper argues that it is where enterprise AI competition will be won or lost.
In this paper, we'll explain how AI context works at a technical level: how models process tokens, why context windows create real constraints even at one million tokens, and why filling that window with the right information — in the right structure, at the right moment — is a harder engineering problem than it first appears. It then traces the industry’s shift from prompt engineering to context engineering: the recognition that the model’s output is determined less by the words of the prompt than by the configuration of everything surrounding it.
And then we'll walk through the architecture of a production-grade context layer at Moody’s, covering ingestion and normalization, retrieval architecture, evaluation frameworks, and governance and auditabilit, before explaining how each of these challenges is amplified in financial services across four dimensions: temporal validity, regulatory auditability, cross-entity complexity, and the stakes of the output.
A dedicated section on context management at scale introduces the four core strategies production agentic systems use — Write, Select, Compress, and Isolate — plus a fifth emerging dimension: tool management. Throughout, Moody’s Agentic Credit Memo workflow is used as a concrete illustration of how these principles apply in practice: how an agent navigates financial statements, rating methodologies, and peer comparisons across multiple steps, manages context window constraints, isolates reasoning across specialised subagents, and produces a sourced, auditable output. The credit memo is used as the example because it is one of the most context-intensive workflows in financial services — but the same architecture underpins every workflow Moody’s builds, from KYC screening and entity profiling to compliance monitoring and portfolio risk assessment.
We will then close with an argument about durable competitive advantage: that the model layer is commoditising, and that the real differentiator is the proprietary data estate, knowledge graph, and domain expertise encoded into the intelligence layer that surrounds it. A competitor can replicate a retrieval pipeline. They cannot replicate 600 million entities, two billion ownership links, and the continuous feedback loop of real-world customer deployments that sharpens connected intelligence with every use.
Moody’s Agentic Solutions (MAS) is the product layer that activates connected intelligence — delivered through Moody’s MCP Servers, purpose-built Agentic Workflows, and a partnership strategy that makes both available natively inside the AI environments customers already use, including Anthropic’s Claude, Microsoft 365, OpenAI, AWS, and Databricks. Every enterprise now has access to powerful language models. Few have built the infrastructure to make them reliable.
Interested in learning more about Moody's GenAI solutions?