Procurement Tools for Buying AI
Throughout the procurement lifecycle, there are many opportunities to govern, map, measure, and manage an AI solution. Here are a few key opportunities that matter along with some helpful resources that can make each milestone make a meaningful difference in your outcomes.
Defining the Problem
Excellent AI governance begins with excellent attention to defining the problem to be solved (BEFORE finding a solution). Focusing on the root cause of the issue will shape the solution in profound ways. Don't skip this part!!!
Stakeholder Voice
Involving diverse stakeholders in the design, development, and deployment process of high-risk systems can yield safer, more responsible, and robust solutions.
AI Impact Assessments
According to the Bipartisan Policy Center, "An impact assessment is a risk assessment tool that seeks to ensure an organization has sufficiently considered a system’s relative benefits and costs before implementation."
Solicitations & Tenders
Communicating AI requirements requires a balance of identifying organizational needs, organizational risk tolerance, and performance-based expectations. Creating a risk-aware solicitation or tender is a critical element of a modernized responsible AI procurement practice.
Vendor Assessments
Assessing each vendors' overall organizational AI governance leadership structure, policies, and practices will help highlight important clues about attitudes, norms, and organizational culture within the vendor organization regarding how they might perceive, identify, prioritize, and manage risks within the AI solution you're buying. Generally, organizations should follow one of the below frameworks:
AI Compliance Assessments
Assessing how AI governance policies were actually applied across the AI development lifecycle provides a clearer picture of the ethical quality and legal compliance of the AI system's output. The resources below provide a variety of risk assessments, controls, and tools to identify good risk management practices.
AI Performance Assessments
Assessing the performance quality of AI solution can be tricky. However, the leaderboards listed below provide a variety of quantitative benchmark data points that may be useful when attempting to compare AI solution.
Assessing AI Usability & Explainability
All high-risk AI systems should be explainable. The trick is that they need to be explainable to different stakeholders (e.g., end users, installers, program managers, procurement/buyers, etc.,) - each of whom need different levels of information about how the system works. The resources below provide information on best practices in usability and explainability.
AI Contract Clauses
Once risks have been identified, they can best be mitigated by establishing specific mitigation tactics agreed to within the contract terms and conditions. Many AI systems present common risks as noted in the following resources:
AI Pricing
AI has not had much time to settle into predictable pricing patterns. Below are resources to help procurement professionals understand how AI pricing varies by solution.
Monitoring AI Contracts
High-risk AI is like taking care of a toddler. You have to know what it is doing at all times. Monitoring AI is a necessary element of every AI depoloyment.
OVERWHELMED?
WE ARE HERE TO HELP!
Procuring AI can be complicated. We get it. It has taken us years to acquire this level of knowledge and understanding. We started the AIPL to help others get up to speed just like we did. Just reach out today, and we'll figure out what makes sense for your unique needs.