Ask a certification director why their program hasn’t moved on AI-assisted item development, and the answer usually lands in one of two places. 

The first is a readiness gap. The item development workflow lives in a few senior people’s heads. SME review criteria are informal. Nobody could trace, on request, how a given item reached the live exam. 

The second is a permission gap. Leaders are unsure what their accreditor expects. The board hasn’t settled its appetite for risk. Staff worry about how candidates will react when they learn AI touched the exam. 

Both feel like good reasons to wait. They are not equal. The permission gap is closing on a public timetable, set by the accreditors and standards bodies themselves. The readiness gap closes only when a program does the work, and it quietly determines how every permission conversation goes. For credentialing organizations weighing AI in exam development, that asymmetry makes waiting the riskier bet. 

The Rulebook for AI in Credentialing Is Already Being Written 

Across the wider economy, AI use is close to universal and mature use is rare. McKinsey’s 2025 global survey of 1,993 respondents found 88% of organizations use AI in at least one function, yet only 7% say it is fully scaled. 

Credentialing looks similar, only more cautious. When ANAB polled accredited certification bodies at its 2024 credentialing conference, roughly half of the 31 respondents reported using AI, and three-quarters of those used it for less than 5% of their activities. Item writing was among the leading applications, at 40% of AI users. One of the four barriers respondents named was simply “acceptance by ANAB.” The sample is small, and ANAB says so. But that last finding is the permission gap, in the respondents’ own words. 

The ground has moved considerably since that survey: 

• NCCA, April 2025. The National Commission for Certifying Agencies issued version 1.1 of its guidance on AI in certification programs. It states plainly that its intent is not to restrict innovation. It also treats the guidance as an update to the Standards’ Commentary, which means NCCA may base accreditation decisions on noncompliance with it. 

• ISO/IEC 17024:2026, March 2026. The third edition of the international standard for bodies certifying persons includes a dedicated clause on the use of AI in the certification process. It is the first standard in the ISO/IEC 17000 conformity assessment family to address AI directly. ANAB’s published transition timeline asks currently accredited bodies for a self-attestation by 1 October 2027, with a final deadline of 31 March 2029. 

• EU AI Act, December 2027. After the Digital Omnibus agreement, obligations for stand-alone high-risk systems under Annex III, which includes certain education and vocational training uses, now apply from 2 December 2027. Whether a given item development tool falls in scope is a question for counsel, not assumption. 

The message across all three is consistent. AI is permitted. Humans stay accountable. Programs must be able to show their work. 

The Readiness Gap: AI Doesn’t Create It, It Exposes It 

Many certification programs run on institutional memory. A veteran exam developer knows which SMEs write clean stems. A review committee applies standards everyone understands and no one has written down. Item history sits across spreadsheets, email threads and a legacy item bank. 

That arrangement can work for years at human speed. It breaks when AI enters the workflow, for two reasons. 

First, the accreditor’s expectations attach to the process, not the author. NCCA’s guidance on examination development says SMEs must review all AI-generated content for accuracy, fairness and psychometric compliance. Programs must document AI’s role, set oversight protocols and assess AI-assisted items as they would human-generated ones. A program that cannot describe how a human-written item is reviewed today cannot describe how an AI-drafted one will be. 

Second, AI changes the volume, and volume punishes informality. The research on AI item quality is encouraging but conditional. A 2026 study in JMIR Formative Research placed 15 items from a fine-tuned model alongside 15 faculty-written items in a live anesthesiology exam taken by 157 students. The AI items performed comparably on psychometric measures, though both sets showed only modest quality, and the authors framed automated generation as a complement to human writing rather than a replacement. The American Institutes for Research reported a related lesson from its own work: even with careful prompting on item-writing practice, statistical flags still emerged, which is why it recommends full psychometric evaluation of AI-generated items. 

The implication is uncomfortable but useful. The quality of AI-assisted items is largely a function of the review system around them. If that system is informal, a tool that triples the draft pipeline also triples the unexamined risk. 

The Permission Gap Is Narrower Than It Looks 

The permission gap has three audiences: the accreditor, the board and the candidate. On each, more is settled than many programs assume. 

The accreditor. NCCA’s guidance is specific about where the lines sit. AI cannot serve as the final authority in any function requiring professional judgment. AI use must be disclosed to NCCA in initial applications, reaccreditation and annual reports. And, importantly for item development, using AI in item writing does not by itself trigger a material change notice. A notice may be needed only if adoption changes the underlying procedures, such as review processes, SME involvement or validation methods. 

The board. The same guidance keeps final decision-making on policy, governance and certification with an autonomous board of qualified people; AI may support that work but not replace it. A board’s real question is therefore not “Is AI allowed?” It is “What will we let it touch, under what controls, and who signs off?” That is a risk-appetite decision boards make about vendors and staff all the time. 

The candidate. Disclosure expectations are emerging rather than absent. NCCA requires programs to disclose AI’s role in scoring to the public and says people should be told when they are receiving AI-generated responses. Commentary on ISO/IEC 17024:2026 expects assessors to look for disclosure so candidates know where AI is used. Candidate concern, in other words, is a communications design problem with known requirements, not a reason to stand still. 

Some uncertainty is real. NCCA notes its guidance is not exhaustive and may be amended. ISO assessors will develop their interpretation through practice. EU scope questions will take time to settle. But none of that uncertainty is about the fundamentals. 

Why the Two Gaps Aren’t Equal 

Read the NCCA guidance closely and a pattern appears. Almost every AI requirement is an existing accreditation expectation applied to a new participant in the process. SMEs review content. Humans own the decisions. The process is documented. Records are auditable. Secure material stays secure. The guidance even tells programs to evaluate AI-assisted items the same way they evaluate human-written ones. 

The core of accreditation has always been three things: a documented process, defensible decisions and clear human accountability. None of those change because a tool helped draft an item. 

That is why the gaps are asymmetric. Closing the readiness gap is most of the work of closing the permission gap. A program with a written review rubric, item-level provenance and defined decision rights can answer the accreditor’s questions with its own records. It can show the board exactly where AI sits and who approves its output. It can tell candidates something specific and true. A program without those things can only offer assurances, and assurances are what make accreditors, boards and candidates nervous. 

There is a commercial case here too, not only a defensive one. Gartner’s survey of 360 organizations found that those that regularly assess AI system performance and compliance were more than three times as likely to report high value from generative AI. That is a correlation across industries, not proof of cause in credentialing. It is consistent, though, with what assessment professionals already know: rigor is what makes output usable. 

The Hidden Cost of Waiting for Clarity 

Waiting feels like the conservative choice. In practice it carries three costs that rarely appear on a board paper. 

Waiting doesn’t mean no AI. It means undocumented AI. A Gartner survey of 302 cybersecurity leaders found 69% of organizations suspect or have evidence that employees use prohibited public generative AI. There is little reason to think volunteer SMEs, working against item-writing deadlines, are an exception. For a certification program, that is the worst position available: AI in the pipeline, secure content potentially leaving it, and no record of either. NCCA’s guidance explicitly requires precautions so that AI-generated content is not inadvertently made public. 

Evidence takes time to accumulate. Accreditors judge what a program has actually done, over time. As one analysis of the ISO/IEC 17024 transition put it, monitoring and oversight records only become convincing after months of live operation. A program that starts documenting the month before an assessment has a policy, not a track record. 

Clarity is built by the programs that move. Accreditors refine guidance through applications, annual reports, public comment and the questions programs bring them. NCCA put a draft of its AI guidance out for public comment in December 2024 before finalizing it. The programs engaged in that process are shaping what “good” looks like. Those waiting on the sidelines will eventually receive clarity, but on terms set by others and later than their peers. 

Closing the Readiness Gap: Seven Moves for Certification Leaders 

None of this requires choosing an AI vendor first. Most of it is overdue process work that pays off with or without AI. 

1. Map the item lifecycle as it actually runs. Who drafts, who reviews, who approves, against what criteria, and where the record lives. Expect surprises. 

2. Write the SME review rubric down. Content accuracy, blueprint alignment, key and distractor defensibility, cueing, bias and sensitivity. Apply the same rubric to every item, whoever or whatever drafted it. 

3. Make provenance a field, not a memory. Each item should carry its origin (human, AI-assisted or AI-drafted), the tool used, who reviewed it, what changed and who approved it. 

4. Set decision rights in a board-approved policy. Define where AI may and may not be used, with a rationale for each use. Confirm that final calls on items, specifications and passing standards stay with qualified people. 

5. Pilot narrowly and measure. Start with pretest items in one content domain. Compare difficulty, discrimination and SME edit rates against human-written items before expanding. 

6. Close the security door. Keep secure content and source materials inside controlled systems, never in public tools, and give SMEs an approved alternative so they don’t improvise. 

7. Prepare the disclosures early. Draft the language for your accreditor’s annual report, a plain-language candidate statement, and an assessment of whether your changes amount to a material change. ANAB-accredited bodies should fold this into their ISO/IEC 17024:2026 gap analysis. 

A program that completes these steps has, in effect, written its answer to the permission question before anyone asks it. 

Where Technology Partners Fit: The OpenEyes Approach 

If readiness is the real constraint, the most useful technology is not the tool that generates the most items. It is the system that makes the process around those items visible. 

That is the problem OpenEyes Technologies has built its product suite around. Its platform spans the assessment lifecycle: GenQue for AI-assisted item generation, Vault for item banking, Merit for competency modeling, and Crown for managing credentials from application through renewal. The company has also been part of the industry’s working conversation on these questions. At I.C.E. Exchange 2026 in Providence, co-founder and CTO, Trushant Mehta took part in a session on guiding AI adoption in certification programs, and Wade Delk, EVP Government Services and Customer Success, joined a panel on certification and regulation in a time of disruption. 

The logic of connecting those pieces maps directly onto the readiness gap. Generation anchored to a competency model ties drafts back to the job analysis and exam specifications that NCCA requires humans to own. Generation connected to an item bank means review history and approvals can travel with the item, rather than living in a separate spreadsheet. The aim is not to remove SMEs from the loop but to spend their time on judgment instead of first drafts. 

Whichever vendor a program chooses, OpenEyes included, the evaluation questions should come from the accreditor’s expectations rather than the demo: 

• Can we see, for any live item, whether AI was involved, who reviewed it, what changed and who approved it? 

• Does our secure content stay inside a controlled environment, and is it excluded from training any external model? 

• Do our SMEs and psychometricians keep final authority at every decision point? 

• Can the system produce the records an accreditor or ISO assessor would ask for, without a reconstruction exercise? 

A partner that can answer yes to all four is helping a program close its readiness gap. One that can’t is just making the pipeline faster. 

Permission Follows Evidence 

Accreditation has never certified tools. It certifies decisions, and the process that produced them. That principle has survived computer-based testing, adaptive delivery and remote proctoring, and it will survive generative AI. 

So the most useful question a certification board can ask this year is not “Are we allowed to use AI?” It is “If our accreditor asked tomorrow how our last fifty items reached the live exam, could we show them?” Programs that can answer that will find the permission conversation short and specific. Programs that can’t will find that no amount of external clarity fixes an internal gap. 

The credentialing field exists to give the public confidence that competence has been verified. Its own adoption of AI will be judged by the same standard. The programs that earn that confidence first will not be the ones that waited to be told they could. They will be the ones that were ready when someone asked.