Lesson Four: Existing Effective Certification Schemes are Transparent, Adaptable, and Interoperable
The previous lessons raise significant challenges that certification must address in order to be effective. We identified some common principles which could help. In the range of sectors we explored, three common characteristics of effective certification schemes were: transparency, adaptability, and interoperability.
Credible Schemes are Transparent
Credible schemes are transparent. The AI sector can learn from other sectors about how transparency can help to empower diverse groups of stakeholders to participate in, improve, and hold systems to account. Transparency is central in many schemes – in sustainability it is often considered a fundamental principle, underpinning the overall credibility of a certification scheme.
Existing Sectors Provide Examples
In the sustainability sector for example, schemes focus on providing important information in appropriate detail, and making this easily accessible to all stakeholders – including information about the goals of a scheme, definitions, how assessments are carried out, and open communication of results and significance. Transparency may also involve public scrutiny, for example, public transparency of adverse event reporting can help drive impact.
Lesson Five: Meaningful Impact Requires a Broad Range of Stakeholder Views
Certification systems must take into account a broad range of perspectives in order to be effective. Assurance service providers, professional bodies, organisations developing and deploying products or services, and affected users all have different needs and incentives.
Different Parts of the Market May Respond to Different Incentives
Different parts of the market may respond to different incentives to seek certification. While some actors may be motivated to work within specific norms and seek certification due to values or brand differentiation, it can be more difficult to incentivise moderately-engaged actors (the “middle” of the market) to participate in a certification scheme.
Lesson Six: Continual Monitoring and Evaluation Can Manage Complexity
Certification is inherently complex, and will be particularly so for AI and AI assurance. To be effective, certification schemes must ensure that complexities are managed appropriately. Fortunately, other sectors provide clues for how we may begin to manage these complexities in our own context, and show that continual monitoring and evaluation can help address and manage these challenges.
Conclusion
In these six lessons, we have identified key enabling conditions and common features across existing schemes that can help certification play an effective role in broader governance. These surface further questions and challenges in our own domain, including how to monitor and improve the enabling conditions for effective certification, how schemes might be designed with the necessary features to succeed, and, crucially, who should be involved in efforts to resolve these challenges.
FAQs
-
Q: What are the common characteristics of effective certification schemes?
A: The common characteristics of effective certification schemes are transparency, adaptability, and interoperability. -
Q: What is the importance of transparency in certification schemes?
A: Transparency is central in many schemes, and can help to empower diverse groups of stakeholders to participate in, improve, and hold systems to account. -
Q: How can certification schemes be designed to ensure meaningful impact?
A: Certification systems must take into account a broad range of perspectives in order to be effective. Assurance service providers, professional bodies, organisations developing and deploying products or services, and affected users all have different needs and incentives. - Q: What is the role of continual monitoring and evaluation in managing complexity in certification schemes?
A: Continual monitoring and evaluation can help address and manage the complexities in certification schemes, by ensuring that complexities are managed appropriately and that schemes are designed with the necessary features to succeed.
