PROMISE TO MAP | Omada's approach to build safety and security into AI-powered products
PROMISE TO MAP is Omada's approach to build safety and security into AI-powered products.
In 2019 Omada Health published the INCLUDES NO DIRT threat model to guide security and engineering teams on how to evaluate security, privacy, and compliance risks to any system, including new applications and new vendors. New artificial intelligence (AI) capabilities, and especially large language models (LLMs) have expanded the risks that must be modeled when the system includes AI features or functions. Omada has newly developed the PROMISE TO MAP threat model to address the additional complexities that AI systems introduce. This AI threat model is intended to expand on the INCLUDES NO DIRT model to include threats that are AI-specific. The PROMISE TO MAP threat model includes directly and by reference all risks previously identified in the INCLUDES NO DIRT model.
The authors of this model were specifically focused on building AI systems to use in a healthcare organization. We believe, however, this model is applicable to most any other business; especially other regulated industries. AI systems introduce new complexities in regulated environments. Organizations used to deterministic software and human actions must account for the risk of probabilistic software with or without humans in the process. Defining the thresholds for what is considered an error, and defining acceptable error rates and new quality assurance processes must be established.
This model relies heavily on concepts from the OWASP LLM Top 10, NVIDIA’s NeMo Guardrail framework, LangChain’s Development Framework, NIST AI-600-1, and numerous other industry sources. Our intent is to simplify these concepts and make them immediately actionable for risk assessing teams. As with the INCLUDES NO DIRT MODEL, we’ve included a simplified questionnaire worksheet at the end of this paper that teams can use to begin assessing AI systems. We encourage practitioners to modify, extend, and use this model to suit your own unique needs.
Download the full white paper to learn more.