AI procurement in a box

Foreword

The promises of artificial intelligence (AI) technologies are literally beyond the capacity of our imagination. The risks that accompany these developments are also impossible to predict. At this pivotal moment in the adoption of AI by governments globally, there is an opportunity to establish a set of actionable procurement guidelines to enable good decisionmaking that can also be evaluated.

Government procurement officials cannot be expected to have the most up-to-date knowledge
in every highly specialized field. To safeguard the responsible future use of AI technologies,
a multistakeholder effort with cross-sector participation and interdisciplinary expertise is
required to create authoritative guidelines. The procedural norms are even more urgent now.
What information should be recorded and how explanations need to be documented is what lays
the foundation for fairness and impartiality in the administrative process. To preserve due process
and predictability, a coalition can help ensure that the right questions are asked.

The Forum offers the Procurement in a Box package, a pragmatic guidebook to unlock
public-sector adoption of AI through government procurement. This was the aspiration of our project.
This work offers a set of complementary tools to demonstrate the emerging global consensus on the
responsible deployment of AI technologies.

The Procurement in a Box project has taken shape at a time when the social contract between the
government, its citizens, and their industries is suspended and is about to reset. The COVID-19
pandemic has further emphasized the imperative for responsible innovation and the ethical use of
technology. We now know how vulnerable an interdependent world can be. Solving complex
societal problems with pattern recognition and predictive algorithms is an integral part of the new
social contract.

The scenarios of how government may use AI technologies is already taking place. For example, chat bots are increasingly being used by government agencies to effectively manage overwhelming inquiries from the general public, but have also increased the concern that sensitive information about people may be misused in the future. Similarly, applying predictive policing to help manage law enforcement has helped fight crime, but has also exposed the bias and discrimination that are embedded in society.

Setting standards to obtain essential information and create cooperative relationships can have a
profound impact on mitigating these harms. AI technologies demand special attention because
automated processes amplify and propagate bias swiftly and these technologies are growing
in prominence. The Procurement in a Box offers important guidance to help prepare for the future.

This project has helped identify useful mechanisms to inform technology policy in the Fourth Industrial
Revolution. Shaping the norms for AI procurement in the public sector will significantly influence best
practice in the rest of the market and throughout the industry. The recommendations and supporting
material in this package are intended to be put into practice. The resources are available publicly as a
“living document” to encourage wide adoption as well as ensure the guidelines evolve with insights
from a range of trials.

The Forum looks forward to continuing to work together to keep AI technologies and their use
responsible and ethical.

Introduction

What is AI Procurement in a Box?

AI Procurement in a Box is a practical guide that helps governments rethink the procurement of
artificial intelligence (AI) with a focus on innovation, efficiency and ethics. Developing a new approach
to the acquisition of emerging technologies such as AI will not only accelerate the adoption of AI in
the administration, but also drive the development of ethical sta dards in AI development and
deployment. Innovative procurement approaches have the potential to foster innovation, create
competitive markets for AI systems and uphold public trust in the public-sector adoption of AI.

AI has the potential to vastly improve government operations and meet the needs of citizens in
new ways, ranging from intelligently automating administrative processes to generating insights for
public policy developments and improving public service delivery, for example, through personalized
healthcare. Many public institutions are lagging behind in harnessing this powerful technology
because of challenges related to data, skills and ethical deployment.

Public procurement can be an important driver of government adoption of AI. This means not
only ensuring that AI-driven technologies offering the best value for money are purchased, but also
driving the ethical development and deployment of innovative AI systems. Government as a powerful
market player can set standards when it comes to the ethical development of technologies and
has done so in cybersecurity and cloud policy in recent decades. Public procurement has been
shown to deliver strategic goals in areas such as environment and human rights and offers
an attractive tool for policy-makers to address wider societal issues. Nevertheless, it is not
straightforward and careful development of processes and incentives need to be considered to
achieve strategic maturity of commercial actions.

These actions are more important than ever when it comes to the adoption of AI. Failure to promote
ethical and technically robust considerations, diversity and openness through AI procurement
may also lead to poor procurement decisions for AI systems. This can limit accountability, undermine
social values, entrench the market power of large businesses, decrease public trust and ultimately
slow digital transformation in the public sector.

To help governments unlock the potential of AI in the public sector the World Economic Forum’s Centre for the Fourth Industrial Revolution in collaboration with the Government of the United Kingdom, Deloitte and Splunk has created AI Procurement in a Box. This practical guide helps policy-makers and commercial teams rethink their approach to AI procurement to more effectively and ethically adopt AI technologies in the public sector.

How did we get here?

The Centre for the Fourth Industrial Revolution is a global hub of expertise, knowledge-sharing and
collaboration, based in San Francisco. The Centre develops, pilots and scales up agile and humancentred governance tools that can be adopted by policy-makers, legislators and regulators worldwide to
address challenges related to emerging technologies.

Together with more than 200 stakeholders from government, academia, the third sector and
business, the Centre created the AI Procurement in a Box during the course of 15 months.

Scoping

The team started by exploring the challenges to government procurement. After two
workshops in San Francisco and London and following extensive consultation with experts
worldwide, five key challenges to government procurement of AI were identified.

The lack of data sharing and data governance in the public sector often leads to a lack of data
availability discoverability and usability. Since data is currently often the basis of any AI development,
these challenges are a great barrier to AI adoption. The team also discovered that sector officials may
lack the appropriate knowledge and expertise to make strategic buying decisions for AI-powered
solutions. Uncertainty about ethical considerations adds further layers of complexity. As a result,
officials tend to delay buying decisions or reduce perceived risk by purchasing solutions from large
and well-known suppliers. For a more detailed description of the main roadblocks see here.

To address these challenges, the team agreed to develop an overview of the important drivers of more effective ethical public procurement and ideas for policy-makers as to how to incorporate these into action.

Co-creating

The team drafted the procurement principles, which included the findings from the workshops,
consultations and input from more than 100 stakeholders from business, academia and government. Comments were facilitated though webinars, community calls, roundtables and workshops, for example with 30 senior commercial specialists from Central and Latin America hosted by the Inter-American Development Bank.

The aim of the guidelines is to maximize the value of data while also setting the highest standards for
transparency and accountability when buying new AI and machine learning (ML) technology. Enhanced
procurement processes help promote the use of AI and ML, giving the public sector more tools to
develop the economy and better serve the public.

The guidelines also inform suppliers about the technical and ethical requirements of public-sector
bodies related to these technologies. They aim to address specific business needs from the public and
private sector as well as to support public trust in the government adoption of AI. Overall this should lead to more efficient, responsible and sustainable outcomes for the public and private sectors.

Piloting and iterating

During the pilot phase the procurement guidelines were used by teams in the United Kingdom’s
Department for Business, Energy and Industrial Strategy and the Food Standards Agency (see pilot
report for more detail) and facilitated workshops in Bahrain, the UAE and the UK to test them with government leaders.

It became clear that practical tools are needed to tailor the guidelines to national contexts. The
principle-based guidelines are useful to introduce the key concepts, but more work is needed to
implement them in different jurisdictions. Therefore, the team created a workbook that sits alongside
the AI procurement guidelines. It aims to provide government officials and industry with greater clarity
on their purpose and enable them to embed the guidelines into their procurement considerations.
The workbook aims to bring the guidelines to life and provide practical guidance into all issues that
they raise.

How do you use the AI Procurement in a Box?

The AI Procurement in a Box guide will help governments and public-sector organizations to start rethinking their approach to the public procurement of AI technologies. When developing the guide, the team took a module-based approach, at the heart of which the guidelines for AI procurement sit. It is recommended that users of the guide follow a step-by-step approach to implementing the guidelines.

Users of the AI Procurement in a Box:

– Governments that aim to accelerate AI adoption in a safe, ethical and innovative manner
– Policy officials to accelerate attainment of their policy goals
– Procurement officials and commercial teams to develop AI-related requests for proposals and to
manage procurement processes
– Data practitioners and technology experts (e.g. statisticians, data scientists, digital delivery
managers) to safeguard public benefit and identify and manage potential risks
– AI-solutions providers to better understand the core expectations for government AI projects and
to align their proposals with emerging standards for public procurement

Any institution or government that aims to adopt the guidelines in an incremental manner and to
drive strategic change by involving actors from throughout the organization should follow these stages described below:

Learn
Facilitate internal and external multistakeholder discussions into what AI is and review the
opportunities and risks of AI. Provide specific examples of how AI is currently already used in
the public sector and gather best practices and case studies for AI adoption from other
organizations worldwide.

Review
Study the guidelines and consider the case studies that can offer you insights into how other
organizations have procured AI technologies.

Discover
Explore your current procurement practices and compare them with the approach described in
the guidelines.

Tailor
Match the guidelines with your current processes and develop a new approach to procurement that
you test in different scenarios. Learn from the pilots and further develop the tools in the AI Procurement
in a Box guide to ensure user-centric guidance for your procurement teams.

Implement
Share the insights from the pilots and complete the adoption of new processes and standards.

How these stages play out in practice will most likely vary from organization to organization
and from country to country, as well as depend on factors like maturity of AI adoption and organizational structures. Nevertheless, implementation methods that focus on multistakeholder discussions and the development of a coherent strategy are recommended. A good starting point is to hold workshops and
round tables in collaboration with AI experts and developers, including but not limited to, consultancies, prominent IT service providers, startups, universities, research institutes and citizen rights organizations.

Acknowledgements

The World Economic Forum’s Unlocking Public Sector Artificial Intelligence project, in collaboration
with the Government of the United Kingdom, Deloitte Consulting and Splunk is a global,
multistakeholder and cross-disciplinary initiative intended to help shape the public sector’s adoption
of AI, and emerging technologies in general, around the world. The project has engaged leaders from
private companies, governments, civil society organizations and academia to understand
public-sector procurement of AI technology, identify challenges and define principles to guide
responsible and ethical procurement. The opinions expressed herein may not correspond with the
opinions of all members and organizations involved in the project.