Centralized vs. Decentralized: Choosing the Right Gen AI Operating Model for Your Bank

Gen AI is rapidly becoming integral to banking, from AI-driven customer service chatbots to automated fraud detection and report generation. This raises an important question.

Should your bank adopt a centralized Gen AI operating model, a decentralized model, or a mix of both? This organizational decision is as important as the technology itself, as it will determine how effectively you can scale AI innovation while managing risk and compliance.

‍

Understanding Gen AI Operating Models in Banking

The Gen AI operating model refers to how a bank structures the teams, processes, and governance for developing, deploying, and managing generative AI solutions. It encompasses who makes decisions, who builds the models, how resources are allocated, and how compliance is enforced. The two ends of the spectrum can be defined as follows:

Centralized Gen AI Operating Model

In a centralized model, a core team or center of excellence owns the Gen AI strategy and implementation for the entire bank. All generative AI development and deployment are coordinated through this central unit, independent from individual business lines. 

This could be an AI Center of Excellence (CoE) or a firm-wide data science team that drives everything from model design to execution. The centralized team typically sets standards, chooses technology platforms, and handles governance and risk controls for AI across the organization.

Decentralized Gen AI Operating Model

In a decentralized model, generative AI capabilities are distributed across the organization. Individual business units, departments, or regional teams develop and deploy AI solutions relatively independently, with minimal central oversight. 

Each business area may have its own data science or AI team focusing on use cases relevant to their domain (retail banking, wealth management, compliance, etc.). The central IT or innovation office provides little direct control; at most, it might offer common tools or advisory support, but the key decisions and execution lie within the decentralized teams of each unit.

Here’s a table demonstrating the benefits/USPs of both the models:

‍

‍

The Hybrid (Federated) Approach: Best of Both Worlds

Rather than an all-or-nothing choice, combine the strengths of both centralized and decentralized models. Often referred to as a hybrid or federated operating model (or informally as a “hub-and-spoke” model), this approach centralizes certain elements (the “hub”) while decentralizing others to domain teams (the “spokes”). The goal is to achieve robust governance and shared infrastructure at the core, alongside agile innovation at the edges.

In a hybrid Gen AI model, a central team typically provides foundational services and standards – for example, a common AI platform, enterprise data lake access, model governance frameworks, and reusable components. This central “AI hub” might develop core models (such as foundational language models or common NLP components) and set guidelines for responsible AI use. 

Meanwhile, distributed teams within business units act as the innovation “spokes.” They leverage the central tools and guidelines to build and deploy AI applications that meet their unique needs. The central hub and the spokes collaborate closely: the hub ensures compliance, security, and knowledge sharing, while the spokes ensure solutions are business-aligned and quickly implemented.

What Does It Mean In Practice? 

In practice, this means centralizing the infrastructure, data governance, model libraries, and AI governance councils, but allowing individual product or business teams to experiment and execute on use cases. The hybrid model has been described as “centralizing the core, decentralizing the rest.” 

Another form of hybrid strategy is using a tiered governance model. Banks might apply different levels of central control depending on the risk and criticality of the use case. For instance, models deemed “critical” (say, those impacting financial reporting or large-scale credit decisions) must be developed or validated centrally, whereas “low-risk” experimental tools can be built by business teams with light-touch oversight. This tiered approach tailors the operating model within the organization, essentially mixing centralized and decentralized approaches based on risk tiers.
‍

‍

Maintaining balance is key in a hybrid model. Banks employ governance mechanisms like cross-functional AI councils and shared platforms to keep decentralized efforts from diverging too far. 

‍

Decision Framework: Choosing the Right Model for Your Bank

Selecting the optimal Gen AI operating model should be a strategic decision guided by your bank’s specific context. Here we provide a decision framework highlighting key factors – organization size, risk appetite, regulatory environment, and innovation goals – and how they might influence a centralized vs. decentralized (or hybrid) approach:

1- Organization Size & Structure

The size and complexity of your bank heavily influence the ideal Gen AI operating model. Large, multinational banks typically benefit from a hybrid structure, centralizing infrastructure while enabling business units to build domain-specific solutions. In contrast, smaller or mid-sized banks should lean toward centralization to optimize limited AI talent and ensure strategic consistency.

2- Risk Appetite & Culture

Your organization's tolerance for innovation risk is a key determinant. Highly regulated or conservative banks should prioritize centralized governance to maintain strict oversight and minimize compliance exposure. Conversely, if your culture encourages experimentation and rapid iteration, decentralized execution can unlock faster innovation, provided it’s guided by clear guardrails.

3- Regulatory & Compliance Environment

Banks operating under intense regulatory scrutiny should favor centralized or federated models to ensure consistent auditability, model validation, and data governance. Centralizing oversight reduces the risk of fragmented compliance and eases regulatory reporting. In less critical domains or innovation sandboxes, controlled decentralization may be acceptable to foster agility.

4- Innovation Goals & Time-to-Market

The nature of your Gen AI goals should shape your model choice. If your priority is scaling a few high-impact AI platforms across the enterprise, centralization brings alignment and delivery power. However, if your aim is to democratize AI experimentation across departments, decentralization supports parallel innovation, though it demands a strong governance backbone.

5- Existing Operating Model & Talent Distribution

Your current AI structure sets the foundation. If a central AI or data science team exists, it’s practical to extend it into a centralized Gen AI model. However, if capabilities are distributed across business units, a federated approach can unify them under shared governance, especially when paired with a phased rollout strategy.

There is no one-size-fits-all answer. As McKinsey notes, the right GenAI operating model “should both enable scaling and align with the firm’s organizational structure and culture”. A smaller fintech bank in a light regulatory setting will make a different choice than a global bank with decades of legacy systems and regulators looking over its shoulder. 

Use the factors above as lenses to evaluate what mix of centralization and decentralization will unlock Gen AI’s value for your institution’s unique situation. 

‍

Strategic Recommendations for CIOs and AI Innovation Leaders

Implementing Gen AI in a bank is as much an organizational journey as a technical one. Here are strategic recommendations for CIOs and Heads of AI Innovation when designing and rolling out the operating model:

1- Start Centralized to Build Momentum and Expertise

In the early stages of GenAI adoption, banks should begin with a centralized structure—typically a Center of Excellence (CoE) staffed with top AI talent and executive sponsorship. This team should lead initial use case development, establish governance frameworks, and address critical concerns such as data privacy, security, and model bias. 

Centralization ensures alignment, reduces risk, and sends a clear message to regulators that AI is being handled responsibly. In fact, McKinsey notes that nearly 70% of banks with centralized GenAI teams have successfully moved use cases into production, compared to only 30% with decentralized models. Early centralization builds reusable assets and foundational capabilities that can scale enterprise-wide later.

2- Develop a Clear GenAI Governance Framework and Risk Plan

Before scaling GenAI initiatives, create a robust governance framework to define standards for model approvals, risk assessments, data usage, monitoring, and compliance. Form a cross-functional AI governance council with representation from IT, risk, compliance, legal, and business units. 

This framework should include clear protocols for bias testing, documentation, audits, and security, especially when using third-party GenAI tools. Institutionalizing governance from the start not only reduces risk but also smooths future decentralization by providing clarity and consistency across teams.

3- Embrace a Hybrid Model as You Scale – Hub-and-Spoke Organization

As adoption matures, shift to a federated model where a central team provides common platforms, reusable components, and governance, while business units execute domain-specific use cases. Embedding “AI ambassadors” within each business line ensures alignment and accelerates delivery. 

This structure encourages innovation without sacrificing oversight. To maintain cohesion, regularly showcase cross-unit AI projects and rotate staff between central and business teams. A hub-and-spoke model effectively balances control with agility.

4- Align the Operating Model with Regulatory Strategy and Engage Regulators Proactively

Anticipate how regulators will view your GenAI deployments. A centralized model is easier to explain and audit, while decentralized models must demonstrate strong documentation and oversight. 

Engage regulators early—through innovation hubs or sandboxes—to signal your readiness and build trust. As new regulatory guidance emerges, adjust your operating model accordingly. Centralizing model inventories, validation, and decision logs can enhance transparency. Ultimately, your regulatory alignment can become a strategic differentiator.

5- Invest in Training and Cultural Change Across the Bank

No operating model succeeds without people. Invest in GenAI training across both central and business teams. Pair data scientists with business analysts to build mutual understanding. Educate frontline staff on using GenAI tools, flagging risks, and collaborating with AI experts. 

Leadership must champion AI adoption—when CIOs and CEOs prioritize AI publicly, it legitimizes efforts across the organization. Encourage cross-functional collaboration and agile rotations to break down silos. Aligning incentives and reinforcing continuous learning will ensure adoption is widespread and sustainable.

6- Iterate and Refine the Model

Treat the operating model itself as an evolving system. Regularly assess whether it is meeting business needs, enabling innovation, and managing risk effectively. Review key metrics such as project velocity, model quality, compliance incidents, and adoption rates. 

Be ready to pivot—centralize when control is slipping, decentralize when innovation slows. Many banks start centralized and shift to a federated model as teams mature. Flexibility is essential as both GenAI capabilities and regulatory expectations evolve.

‍

Conclusion

In summary, CIOs and innovation leaders must enable GenAI innovation while controlling its risks. The operating model is your mechanism to do so. By starting with a strong centralized core, gradually empowering business units, and embedding governance throughout, you can create an organization that is both innovative and trustworthy. Banks that get this right will not only deploy GenAI faster than competitors, but they’ll do so in a way that is scalable, compliant, and value-generating in the long term.

If you’d like to consult our experts on which model suits your bank, connect with us here.