RSS feed LinkedIn Watch us on Youtube

Activity title

Orchestration and Scalability of AI-driven Systems

Activity Reference

IST-ET-109 (AI2S)


Information Systems Technology

Security Classification




Activity type


Start date


End date



ML, Software Enginering with special focus on AI systems, Trust


The elements in the life cycle of AI systems are different to the traditional SW development processes. An analogy to ITIL is not available. One difference is the needed time for additional training phases when an AI model is extended/changed. Most software engineers have a clear understanding of how to build production-ready software using traditional tools and practices. In essence, software engineering is the practice of turning explicitly specified business logic into production software systems. We have developed many practices in how to test, validate and deploy this type of systems. We know the portfolio of skills a team needs to design, implement, and operate them, as well as the methodologies that can work to evolve them over time. Machine learning systems have some of the same requirements as traditional software systems. The main goal is to create reliable systems we can evolve over time. But a lot about the architecture, skills, and practices of building them is quite different. Indeed, data scientist and software engineers in many cases follow different process models. Developing analytical models is relatively cheap compared to deploying and maintaining it. Nevertheless, still much more emphasis is placed on building complex pipelines and playing with complex algorithms chasing model accuracy than on model deployment. Deployment of analytical models, means making trained models available other systems. Migrating model to production is one block of the integration of model builders and system builders. Model builders need to be able to iterate in an agile fashion on predictive modeling and the production software engineers need to integrate these model changes in a stable, scalable production system. A fundamental issue in the architecture of machine learning-powered applications is how these two groups can interact productively. An architecture should act as an intermediary between the various data sources from which feature data is collected, the model-building environment where the model is fit, and the production application that serves predictions. Beyond a suitable architecture, we need to upgrade the software development lifecycle to accommodate machine learning, including experimentation, reproducibility and deployment. Like continuous integration and delivery describing the methodologies of traditional software development, the concept of continuous learning could lead to the establishment of build processes, managed by build and reliability engineers. As part of the lifecycle, usage and adaptation data also should be fed back to the modeling process so the model can continue learning.


Invent a life cycle, SE process, and Quality Management appropriate to AI


Understand the difference of traditional SE and realization of AI systems. Develop best practices. Establish rules and standards compliant with NATO requirements. Gaining Trust in AI systems. Increasing efficiency in the development and deployment of AI driven decision making systems.

Contact Panel Office