Setting your team up for success in prototyping is a necessary step toward positively impacting your business with AI-generated insights. Sustaining that initial success, however, requires a thoughtful approach to scaling and monitoring AI algorithms, ensuring user adoption, and monitoring and tracking business value being generated. In this article, we outline best practices distilled from our years at C3 AI of operating and monitoring AI models in production at global scale – millions of models operating against ongoing data updates within single- environment instances.
Developing and maintaining complex AI use cases requires a sophisticated and rigorous approach that includes implementing a method to periodically improve the deployed algorithms, designing for and monitoring nuanced edge cases, creating a robust set of automated tests to prevent regressions, and gracefully alerting administrators to issues. Because the process to implement highly scalable solutions is analogous to modern software development and deployment, those processes can be used as a model for AI development.
Given the highly iterative nature of algorithm configuration and application logic development, it is recommended that both algorithm development and application development proceed together, in lockstep, using modern software development approaches.
This typical development process involves six steps to ensure reliable and performant code is released to end users:
Figure 37 The software development process requires code reviews, testing, release, QA, preproduction, and production phases.