# How to choose an AI operating model for a mid-size company > How to choose an operating model for AI work in a mid-size company, starting from what the law requires and what the evidence supports. Source: https://meshin.co/blog/how-to-choose-an-ai-operating-model-for-a-mid-size-company Updated: 2026-10-07 --- This page starts from what is required, then asks what the evidence supports. The [AI operating model](/glossary/ai-operating-model) here is the one the glossary defines. What the law requires about structure is narrow, and the dates are still moving. What the evidence supports is long and weak. ## Does any law require an AI committee or a Chief AI Officer? For a private company, no source reviewed for this page requires any of four things. They are a committee, a centre of excellence, a hub-and-spoke model and a Chief AI Officer. The European Commission's [AI literacy Q&A](https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers) asks whether an AI officer is necessary, as under the GDPR. It also asks whether an organisation must set up an [AI governance](/glossary/ai-governance) board. The answer is "No, no specific governance structure is mandated to comply with article 4 of the AI Act." That answer covers Article 4 only. The one binding rule found for this page that does require such structures is American. [OMB's memo on federal AI use](https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-21-Accelerating-Federal-Use-of-AI-through-Innovation-Governance-and-Public-Trust.pdf), dated 3 April 2025, tells each federal agency to name a Chief AI Officer. Each agency covered by the CFO Act must also convene an AI governance board. Even there, agencies "are permitted to rely on existing governance bodies." The memo binds US federal agencies and nobody else. The standards read the same way. The [NIST AI RMF](/glossary/nist-ai-rmf) is voluntary. Its [published text](https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf) calls it "use-case agnostic, providing flexibility to organizations of all sizes." It prescribes no committee or team shape. [ISO/IEC 42001](/glossary/iso-iec-42001) applies to "any organization, regardless of size." Its contents list roles, responsibilities and authorities. The standard is sold, not published, so its clause text was not read for this page. ## What does the law require instead? In the Articles reviewed here, the [EU AI Act](/glossary/eu-ai-act) asks for duties, not structures. Which duties apply depends on what a company does with AI. It also depends on whether the company is the [provider or the deployer](/glossary/provider-vs-deployer) of each system. The table lists the duties of the Articles reviewed, by trigger, using the [consolidated text](https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng). The [Digital Omnibus](https://eur-lex.europa.eu/eli/reg/2026/1744/oj/eng), Regulation (EU) 2026/1744, rewrote some of the wording and moved the dates in July 2026. The last column is a reading of what each duty needs in practice. It is not law text. | If your company… | The duty | Applies from | What it needs in practice | | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | Provides or deploys any AI system | Article 4: take measures to support the AI literacy of staff and others who use AI for you. No specific level is required. | In application since 2 February 2025. National authorities enforce from 2 August 2026. | Someone to run the measures. The Commission says there is no need for a certificate and that organisations can keep an internal record of trainings. | | Runs a chatbot or generates synthetic content | Article 50: transparency duties for providers and deployers of chatbots, generative systems and deepfake content. Providers of generative systems must mark outputs in a machine-readable format. | 2 August 2026. Generative systems placed on the market before 2 August 2026 have until 2 December 2026 to mark. | An owner for disclosure wording, and for marking if you provide the system. | | Provides a [high-risk AI system](/glossary/high-risk-ai-system) | Article 17: a documented quality management system with an accountability framework. It sets out the responsibilities of management and other staff. It must fit the size of the provider. | 2 December 2027 for Annex III systems. 2 August 2028 for Annex I systems. | A written accountability framework. | | Deploys a high-risk AI system | Article 26: give [human oversight](/glossary/human-oversight) to people with "the necessary competence, training and authority". Also monitor the system, keep logs for six months or more, and report risks. | The same two dates. | Named overseers. The Act leaves the deployer "freedom to organise its own resources and activities". | | Changes a system someone else built | Article 25: a deployer becomes the provider if it puts its own name on a high-risk system or changes it substantially. The same happens if it changes a system's purpose so that it becomes high-risk. | No start date for Article 25 in the sources reviewed. | A change-control step that decides when the role flips. | Look at what these duties name. They name an accountability framework (Article 17) and people with the competence, training and authority for human oversight (Article 26). None names a committee, a centre of excellence or a Chief AI Officer. In practice, Article 4 needs someone to run the literacy measures and Article 50 needs an owner for disclosure wording; that is a reading, not law text. The Commission adds that, for deployers of high-risk systems, the duty to train staff for human oversight "remains in place" alongside Article 4. I deliver the Article 4 literacy measures for clients. Each role gets a short session built for that role and a one-page list of dos and don'ts. The client signs the internal record. ### How firm are the EU dates? The omnibus was signed on 8 July 2026 and is in force from 27 July. It moved the high-risk dates to 2 December 2027 (Annex III systems) and 2 August 2028 (Annex I systems). Its preamble gives the reason: "the delayed availability of standards, common specifications, and alternative guidance and the delayed establishment of national competent authorities." It also rewrote Article 4 and Article 17(2). Treat the new dates with care. The omnibus dates from July 2026, and several Commission guidelines are still pending, so the practical detail is still to come. I plan client work against the dates recorded on this page: Annex III from 2 December 2027 and Annex I from 2 August 2028. I do not put dates in contracts. Company size also changes the exposure to fines. Small and medium-sized companies pay the lower of the two fine limits. The omnibus extends that rule to small mid-cap companies, a new size class just above them, for all but the highest tier. ## Who decides, and who answers? This page treats five decisions as ones that need a named owner; that is the writer's reading, not a finding of any source. The table sets what law and standards say about each against what surveys report. The last table in the next section gives the sample behind each survey. | Decision | What law and standards say | What organisations report | | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Approve a system for production | NIST asks executive leadership to take responsibility for decisions about AI risk (GOVERN 2.3). It prescribes no committee or team shape. | The Bank of England and FCA surveyed 118 firms in 2024. 72% give accountability for AI use cases to executive leadership. Most name three or more accountable people or bodies. | | Own the evaluation | NIST MEASURE 1.3: internal experts who did not build the system, or independent assessors, join regular assessments. | No source reviewed says who should own an evaluation set after handover at a company of this size. | | Sign risk acceptance | NIST asks for residual risk to be documented. It names no signer, and neither do the AI Act articles reviewed. | The US federal memo names one: a waiver of its requirements for high-impact AI needs a written determination by the agency's Chief AI Officer. | | Keep the inventory | NIST GOVERN 1.6 asks for mechanisms to inventory AI systems. | ASIC found some licensees with no AI inventory and others with several dispersed model registers. | | Run it after launch | AI Act Article 26: deployers of high-risk systems monitor them, keep logs for six months or more and report risks. | A 2025 survey sponsored by Pacific AI found that 48% of organisations monitor production AI systems. It gives no clear figure for companies of this size. | In the table, NIST speaks to four of these decisions and AI Act Article 26 to the fifth. Of the texts reviewed, no NIST subcategory names an approver, NIST names no risk-acceptance signer and the AI Act articles reviewed name none. Practice varies too. The IAPP and Credo AI report says: "There is no clear best practice for how to build and organize an AI governance team." ASIC describes better and poorer practice for a committee where one exists. It describes better practice as a cross-functional, executive-level committee with clear responsibility and decision-making authority. Poorer practice was a committee that met infrequently and had a poorly defined mandate. Fewer than a third of the licensees it reviewed had a committee, or were setting one up. The [AI register](/glossary/ai-register) is where the owner and risk tier of each system live, and a short review checklist can be attached to each entry. The [register post](/blog/how-to-build-an-ai-system-register-in-a-week) shows how to keep one. ## Which structures do companies use, and how good is the evidence? Each structure in this table carries a grade in plain words. Required means a law or binding rule says it. Observed means a regulator or a survey reports what organisations do, with the sample named. Recommended means a research centre, a consultancy or a vendor says organisations should. Popular means many vendors repeat it and no data was found behind it. Described, not evaluated means an arrangement is described and no effect is reported. | Structure | What it means | What the evidence is | Grade | | -------------------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | Centralised | One team or centre of excellence decides, and often builds. | [Gartner](https://www.gartner.com/en/newsroom/press-releases/2025-06-30-gartner-survey-finds-forty-five-percent-of-organizations-with-high-artificial-intelligence-maturity-keep-artificial-intelligence-projects-operational-for-at-least-three-years) reports that almost 60% of high-maturity organisations centralise AI strategy, governance, data and infrastructure. | Observed. Gartner scores maturity from a seven-question survey; its press release does not list the questions. | | Hub-and-spoke or federated | A central group sets standards. Units run the work. | [McKinsey's guide](https://www.hkdca.com/wp-content/uploads/2024/09/data-leaders-operating-guide-scaling-ai-mckinsey.pdf) describes three archetypes: centralised, federated and decentralised, with a trade-off for each, and says the decentralised one needs very high domain maturity and sufficient scale. [IBM](https://www.ibm.com/thought-leadership/institute-business-value/en-us/report/chief-ai-officer) surveyed Chief AI Officers. Those leading hub-and-spoke or centralised models report twice as many pilots moved into production and 36% higher ROI. Every respondent was a Chief AI Officer, so each already had a central AI leader. IBM adds that organisations generally move from decentralised models to centralised or hub-and-spoke models as they scale. [Microsoft](https://blogs.microsoft.com/on-the-issues/2021/01/19/microsoft-responsible-ai-program/) describes its own hub and spokes. It is the one long-running named case, and it is a very large company describing itself. | Observed and recommended. The ROI is self-reported. No cost or failure data. | | Decentralised | Each unit owns its own AI work. | [ASIC](https://download.asic.gov.au/media/mtllqjo0/rep-798-published-29-october-2024.pdf) placed its "leveraged, decentralised" licensees in the middle of its maturity scale. [MIT NANDA](https://cloudelligent.com/wp-content/uploads/2026/02/v0.1_State_of_AI_in_Business_2025_Report.pdf) found that decentralised implementation works when accountability stays put: "companies succeed when they decentralize implementation authority but retain accountability." | Observed. The sources point different ways. | | Embedded | AI people sit inside business units. | One case story from [MIT CISR](https://cisr.mit.edu/publication/2025_0801_EnterpriseAIMaturityUpdate_WoernerSebastianWeillKaganer): Italgas placed business translators in each unit and added a group AI office. No source reviewed measures this against the others or separates it from decentralised. | Popular. One case story. | | Champions network | Named people in teams who spot issues. | Microsoft runs Responsible AI Champs. The European Commission lists an AI Champions network among its own staff measures. Neither reports an effect. | Described, not evaluated. | The next table gives the sample behind eight of these sources and the main limit of each. None of the large studies reports results for companies of 30 to 300 people. | Source | Sample | What it found | Main limit | | -------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | Bank of England and FCA | 118 financial firms, 2024 | 72% give accountability for AI use cases to executive leadership. | Self-reported. Financial services only. | | ASIC Report 798 | 23 licensees and 624 use cases, data from December 2023 | Seven had, or were setting up, a committee. Governance lagged adoption. | ASIC says the sample is not representative. | | IBM Institute for Business Value | More than 600 Chief AI Officers, 2025 | Hub-and-spoke and centralised models link to more pilots in production and higher ROI. | Self-reported. Every respondent already had a central AI leader. | | Gartner | 432 respondents, fourth quarter of 2024 | Almost 60% of high-maturity organisations centralise AI strategy, governance, data and infrastructure. | Gartner scores maturity from a seven-question survey. Its press release does not list the questions. | | MIT NANDA | 52 interviews and 153 survey responses, 2025 | Companies succeed when they decentralise implementation authority but keep accountability. | Preliminary. Selection bias, by its own account. A third-party copy was read. | | IAPP and Credo AI | More than 670 respondents in 45 countries (report published April 2025) | Privacy, and legal and compliance, each have primary responsibility for AI governance in 22% of organisations. | Self-reported. | | Gradient Flow | 351 respondents, 2025 | A role or office for AI governance existed at 36% of small companies and at 62% to 64% of larger ones. | Pacific AI sponsored it. Small means 500 staff or fewer. | | UTS Human Technology Institute | Two papers, October 2025 and May 2026. Recommendations, not outcome data. | Recommends one accountable executive with a small cross-functional working group. | A university research centre with industry partners. It shows what advisers agree on, not outcomes. | Most surveys are self-reported and lean toward large companies. ASIC's data is from December 2023. The sources also disagree about where authority should sit. IBM and Gartner lean central. MIT NANDA leans local for implementation. They may describe different layers: central for policy and risk, local for finding use cases. No source tests that reading. I take no public side on centralised versus decentralised. ## Where do the switch-point numbers come from? Vendor pages tell readers to change structure at a set number of AI projects or business units. The numbers differ from page to page. No data was found behind any of them: the one vendor page read in full cites no data for its numbers, and the other two were seen only as search snippets. One page also puts a figure on rework. That figure has no data behind it either. This page does not repeat those numbers. Printing a number gives it weight the evidence does not give it. The three vendor thresholds found have no data behind them. No source reviewed reports a cost for any structure in headcount, budget or time. My own switch point has no number. The structure grows when the queue of decisions outruns what one named owner can review. One ASIC finding is about the order of events. Among the licensees ASIC reviewed that planned to use more AI, nearly all were only planning or starting a governance uplift. Only one appeared to have done it before the rise in use it expected. In that sample, most governance uplifts were still being planned or started when the licensees expected their use of AI to rise. ## What fits a company of 30 to 300 people? The sources say little about a company this size. [Gradient Flow](https://gradientflow.com/2025-ai-governance-survey/) ran the only size-split governance survey found. Small companies were far less likely than larger ones to have a role or office for AI governance. Pacific AI sponsored it. Small there includes companies under 30 people. The [UTS Human Technology Institute](https://www.uts.edu.au/research/centres/human-technology-institute/projects/ai-corporate-governance-program/ai-governance-snapshot-series/contentassets/designing-ai-governance-structures.pdf) describes a lean design: "a single accountable executive supported by a small cross-functional working group." It says the CEO may take that role in a smaller organisation. It also says "There is not a single correct answer to these questions." Its papers show what advisers agree on, not what has been shown to work. Parts of the law scale with size too. Article 17(2) ties a provider's quality management system to the size of its organisation. The Australian government's [AI adoption guidance](https://www.ai.gov.au/staying-safe-and-responsible/essential-ai-practices/guidance-ai-adoption-foundations) tells organisations to adapt each practice to their size. It also asks for a senior leader as the overall owner and a specific person accountable for every system. ## How do you choose? The evidence cannot pick a structure for you. It can narrow the choice. What follows is a reading of the sources, not a finding from any of them. 1. Start from the duties you have. Decide for each AI system whether your company is the provider or the deployer, and whether the system is high-risk. That decides which dates apply. 2. Name an owner for each of the five decisions. The law and the standards reviewed ask for named people with clear roles. None requires a committee. 3. Choose the smallest structure that covers them. The lean design above is one accountable executive and a small working group. Add a standing group only when a decision needs more than one function. Give any group a clear mandate and regular meetings, because ASIC's poorer practice was a committee that met infrequently and had a poorly defined mandate. 4. Pick triggers from your own events, not from a vendor number. Review the structure when a duty date arrives, when you add a high-risk system, after an incident, or when use grows faster than the controls. 5. Test the structure on your own records at a fixed date. None of the large studies reports results for a company your size. Your register does. The [post on why pilots stall](/blog/why-your-ai-pilot-never-reached-production) shows what happens when step two is skipped. Write the five names down first. Then see whether any duty above needs more than that. ## What do these sources leave out? They cannot say which structure produces better results at a company this size. None of the large studies reports results for companies of 30 to 300 people. They give no cost for any structure. Most are dated 2023 to 2026; Microsoft's account of its own programme is from 2021, and two pages carry no date. Some of the data is older than the omnibus. Agentic systems are a gap. No regulator text or standard reviewed assigns decision rights to an AI agent. The Australian guidance still says to name a person accountable for every system. Standards are moving too. NIST says its framework "is being updated", so every subcategory cited here comes from the January 2023 text.