Sri Arepally

Selecting the Best Tools for Building your MLOps Workflows

In our previous blog, The Fundamentals Of MLOps – The Enabler Of Quality Outcomes In Production Environments, we introduced you to MLOps and its significance in an intelligence-driven DevOps ecosystem. MLOps is gaining popularity since it helps standardize and streamline the ML modeling lifecycle.

From development and deployment until maintenance, various tools and their features can be implemented to achieve the best outcomes. Organizations shouldn’t depend on just one tool but combine the most valuable features of multiple tools for:

This post discusses some key pointers to help you pick the right MLOps tools for your project.

Considerations for Choosing MLOps Tools

When organizations deploy real-world projects, there is a vast difference between individual data scientists working on isolated datasets on local machines and Data science teams deploying models in a production environment. These models need to be reproducible, maintainable, and auditable later on. MLOps tools help converge various functionalities and connect the dots through unified collaboration.

MLOps Tools help in these Areas

Resulting in:

• 30% faster time-to-market
• 50% lower new release failure rate
• 40% shorter lead times between fixes
• Up to 40% Improvement in the average Time-to-Recovery

Radiant’s Top Recommendations:

1. Databricks MLflow

MLflow is an open-source tool that lets you manage the entire machine learning lifecycle, including experimentation, deployment, reproducibility, and a central model registry. MLflow is suitable for individual data scientists and teams of any size. This platform is library-agnostic and can be implemented with any programming language.

Components

Image source: Databricks

Features: MLflow comprises four primary features that help track and organizes experiments.

MLflow Tracking – This feature offers an API and UI for logging parameters, metrics, code versions, and artifacts when running machine learning code. It lets you visualize and compare results as well.

MLflow Projects – With this, you can package ML code in a reusable form that can be transferred to production or shared with other data scientists.

MLflow Models – This lets you manage and deploy models from different ML libraries to a gamut of model-serving and inference platforms.

MLflow Model Registry – This central model store helps manage an MLflow Model's entire lifecycle. The processes include model versioning, stage transitions, and annotations.
The Model Registry capabilities are given below.

Architecture

Image source: slacker news

The MLflow tracking server offers the ability to track metrics, artifacts, and parameters for experiments. It helps package models and reproducible ML projects. You can deploy models to real-time serving or batch platforms. The MLflow Model Registry [AWS] [Azure] represents a central repository to manage staging, production, and archiving.

On-Premise and Cloud Deployment

Image source: LG collection

MLflow can be deployed on cloud platform services such as Azure and AWS. It can be deployed on container-based REST servers for on-prem, and continuous deployment can be executed using Spark streaming.

2. Kubeflow

Kubeflow is an open-source machine learning toolkit that works using Kubernetes. Kubernetes standardizes software delivery at scale, and Kubeflow provides the cloud-native interface between K8s and data science tools like libraries, frameworks, pipelines, notebooks, etc., to combine Ml and Ops.

Components

Image source: kubeflo

• Kubeflow dashboard: This multi-user dashboard offers role-based access control (RBAC) to the Data scientists and Ops team.

• Jupyter notebooks: Data scientists can quickly access the Jupyter notebook servers from the dashboard that have allocated GPUs and storage.

• Kubeflow pipelines: Pipelines can map dependencies between ML workflow components where each component is a containerized piece of ML code.

• TensorFlow: This includes TensorFlow training, TensorFlow serving, and even TensorBoard.

• ML libraries & Frameworks: These include PyTorch and MXNet XGBoost MPI for distributed training. Model serving is done using KFserving, Seldon Core, and more.

• Experiment Tracker: This component helps store the results of a Kubeflow pipeline run using specific parameters. These results can be easily compared and replicated later.

• Hyperparameter Tuner: Katib is used for hyperparameter tuning, which runs pipelines with different hyperparameters (e.g., learning rate) optimized for the best ML modeling.

Features

1. Kubeflow supports a user interface (UI) for managing and tracking experiments, jobs, and runs.

2. An engine schedules multi-step ML workflows.

3. An SDK defines and manipulates pipelines and components.

4. Notebooks help interact with the system using the SDK.

Architecture

Image source: kubeflow

Kubeflow is built on Kubernetes, which supports AWS, Azure, and on-prem deployments. It helps scale and manage complex systems. The Kubeflow configuration interfaces let you specify the ML tools that assist in the workflow. The Kubeflow applications and scaffolding layer manage the various components and functionalities of the following ML flow.

Image source: kubeflow

Main source page of all the images above: https://www.kubeflow.org/docs/started/kubeflow-overview/

On-premise and Cloud Deployment

Kubeflow has on-premise and cloud deployment capabilities that Google’s Anthos support. Anthos is a hybrid and multi-cloud application platform built on open source technologies, including Kubernetes and Knative. Anthos lets you create a consistent setup across your on-premises and cloud environments, where policy and security automation is possible at scale. Kubeflow can be deployed on IBM Cloud, AWS, and Azure as well.

3. Datarobot

DataRobot is an end-to-end enterprise platform that automates and accelerates every step of your ML workflow. Data Scientists or the operations team can import models programmed using Python, Java, R, Scala, and Go. The system includes frameworks in pre-built environments like Keras, PyTorch, and XGBoost that simplify deployment. You can then test and deploy models on Kubernetes and other ML execution environments that are available via a production-grade REST endpoint. DataRobot lets you monitor service health, accuracy, and data drift and generates reports and alerts for overall performance monitoring.

Components

• REST API-Helps quickly deploy and interact with a DataRobot-built model.

• Model Registry - The Model Registry is the central hub with all your model packages containing a file or set of files with model-related information.

• Governance and Compliance -This component helps comply with your models and
ML workflows with the defined MLOps guidelines and policies.

• Application Server -This component handles authentication, user administration, and project management and provides an endpoint for APIs.

• Modeling workers - This computing resource allows users to train machine learning models in parallel and even generate predictions at times.

• Dedicated prediction servers -Help monitor system health and make real-time decisions using key statistics.

• Docker Containers - This helps run multi-instance services on multiple machines, offering high availability and resilience during disaster recovery. Docker containers allow enterprises to run all of the processes on one server.

Features

1. Monitor MLOps models for service health, data drift, and accuracy.

2. Custom Notifications for user deployment status.

3. Management and replacement of MLOps models along with the documented record of every change that occurs.

4. Establish governance roles and processes for each deployment.

5. Real-time deployments using DataRobot Prediction API & HTTP Status Interpretation.

6. Optimization of Real-Time Model Scoring Request Speed.

7. Batch deployments using Batch Prediction APIs and Parameterized Batch Scoring Command-line Scripts.

Architecture

Image source: Datarobot community

The web UI and APIs feed business data and model information predictions to the application server.

The App Server handles user administration tasks and authentication. It also acts as the API endpoint.

The queued modeling requests are sent to the modeling workers. These stateless components can be configured to join or leave the environment on-demand.

The data layer is where the trained models are written back. Their accuracy is indicated on the model Leaderboard through the Application Server.

The Dedicated Prediction Server uses key statistics for instant Decisioning and provides data returned to the Application Server.

On-premise and Cloud Deployment

DataRobot can be deployed for on-premise enterprise clients as either a standalone Linux deployment or a Hadoop deployment. Linux deployments allow clients to deploy the platform in multiple locations from physical hardware and VMware clusters. They also help deploy using virtual private cloud (VPC) providers. Hadoop deployments help install in a provisioned Hadoop cluster which saves on hardware costs and simplifies data connectivity.

4. Azure ML

Azure Machine Learning (Azure ML) is a cloud-based service for creating and managing ML workflows and solutions. It helps data scientists and ML engineers leverage data processing and model development frameworks. Teams can scale, deploy and distribute their workloads to the cloud infrastructure at any time.

Components

• A Virtual machine (VM) is a device on-premise or in the cloud that can send an HTTP request.

• Azure Kubernetes Service (AKS) is used for application deployment on a Kubernetes cluster.

• Azure Container Registry helps store images for all Docker container deployment types, including DC/OS, Docker Swarm, and Kubernetes.

Features

Image source: slidetodoc

New Additions

More intuitive web service creation – A “training model” can be turned into a “scoring model” with a single click. Azure ML automatically suggests/creates the input and output points of the web service model. Finally, an Excel file can be downloaded and used for web service interactions for feature inputs and scores/predictions outputs.

The ability to train/retrain models through APIs – Developers and data scientists can periodically retrain a deployed model with dynamic data programmatically through an API.

Python support – Custom Python code can be easily added by dragging the “Execute Python Script” workflow task into the model and feeding the code directly into the dialogue box that appears. Python, R, and Microsoft ML algorithms can all be integrated into a unified workflow.

Learn with terabyte-sized data – You can connect to and develop predictive models using “Big Data” sets with the support of “Learning with Counts.”

Architecture

Image source: Microsoft docs

1. The trained model is registered to the ML model registry.
2. Azure ML creates a Docker image that includes the model and the scoring script.
3. It then deploys the Azure Kubernetes Service (AKS), scoring images as a web service.
4. The client sends an HTTP POST request with the question data encoded.
5. The web service created by Azure ML extracts the question from the request.
6. The question is relayed to the Scikit-learn pipeline model for scoring and featurization.
7. The matching FAQ questions with their scores are returned to the client.

On-Premise and other Deployment Options

Azure ML tools can create and deploy models on-premise, in the Azure cloud, and at the edge with Azure IoT edge computing.

Other options include:

• VMs with graphic processing units (GPUs) to help handle complex math and parallel processing requirements of images.

• Field-programmable gate arrays (FPGAs) as-a-service help operate at computer hardware speeds and drastically improve performance.

• Microsoft Machine Learning Server: This provides an enterprise-class server for distributed and parallel workloads for data analytics developed using R or Python. This server runs on Windows, Linux, Hadoop, and Apache Spark.

5. Amazon SageMaker

Amazon SageMaker is a fully-managed end-to-end ML service that enables data scientists and developers to build quickly, train, and host ML models at scale. SageMaker allows data labeling and preparation, algorithm selection, model training, tuning, optimization, and deployment to production.

Components

Image source: AWS

• Authoring: Zero-setup hosted Jupyter notebook IDEs for data exploration, cleaning, and pre-¬ processing.

• Model Training: A distributed model building, training, and validation service. You can use built-in standard supervised and unsupervised learning algorithms and frameworks or create your¬ training with Docker containers.

• Model Artifacts: Data-dependent model parameters allow you to deploy Amazon SageMaker¬-trained models to other platforms like IoT devices.

• Model Hosting: A model hosting service with HTTPS endpoints for invoking your models to get real-time inferences. These endpoints can scale to support traffic and allow you to A/B test, multiple models simultaneously. Again, you can construct these endpoints using the built-in SDK or provide your configurations with Docker images.

Features

Image source: slideshare

Architecture

Image source: ML in production

SageMaker is composed of various AWS services. An API is used to "bundle" together these services to coordinate the creation and management of different machine learning resources and artifacts.

On-premise and Cloud Deployment

Once the MLOps model is created, trained, and tested using SageMaker, it can be deployed by creating an endpoint and sending end images through HTTP using AWS SDKs.
This endpoint can be used by an application deployed on AWS (e.g., Lambda functions, Docker Microservices, or applications deployed on EC2s) and applications running on-premise.

MLOps Tools Comparison Snapshot

MLOps Tool	Purpose	Build	Deploy	Monitor	Drag-and-drop	Model Customization
AWS SageMaker	It lets you train the Machine Learning model by creating a notebook instance from the SageMaker console along with proper IAM role and S3 bucket access.	SageMaker console, Amazon Notebook S3, Apache Spark.	Amazon Hosting Service Model Endpoint	Amazon Model Monitor, Amazon CloudWatch Metrics	NO	YES
AZURE ML	It lets Data scientists create    separate pipelines for different phases in the ML lifecycle, such as data pipeline, deploy pipeline, inference pipeline, etc.	Azure Notebook, ML Designer	Azure ML studio, Real-time endpoint service, Azure pipeline	Azure Monitor	YES	NO
DataRobot	It provides a single place to deploy centrally, monitor, manage, and govern all your production ML models, regardless of how they were created and where they were deployed.	Baked-in modeling techniques, drag-and-drop	·          Make predictions, a.k.a. drag-and drop ·       Deploy ·       Deploy to Hadoop ·       DataRobot prime ·       Download ·       Prediction Application	MLOps monitoring agents	YES	YES
Kubeflow	It is dedicated to making deployments of ML workflows on Kubernetes simple, portable, and scalable. The goal is not to recreate other services but to provide a straightforward way to deploy the best-of-breed open-source ML systems to diverse infrastructures.	TensorFlow libraries, Google AI Platform, Datalab, Big Query, Data flow, Google Storage, Data Proc for Apache Spark and Hadoop	TensorFlow Extended (TFX) Kubeflow pipeline	ML Monitor API, Google Cloud Logging, Cloud Monitoring Service	YES	YES
MLflow	It provides experimentation, tracking, model deployment, and model management services to manage the build, deploy, and monitor phases of ML projects.	Experiment tracking	Model Deployment	Model Management	YES	YES

Wrapping up

Implementing the right mix of ML models should not be an afterthought but carefully calibrated within the ML lifecycle. Visibility into models runtime behavior benefits data scientists and operations teams to monitor effectiveness while providing the opportunity to detect any shortcomings and embrace functional improvements.

---

by Sri Arepally

Making Your Business Processes Efficient and Reliable with jBPM Migration

In our previous blog, "Enterprise BPM Transformation - Embrace the Change," we discussed the transformational capacity of BPM and how it helps organizations gain better visibility that translates into higher productivity. Many companies invest in tools like Lucidchart, Visio, Modelio, Pega BPM, ServiceNow BPM, etc., based on their diverse and specific business needs. While each of them has benefits and limitations, we at Radiant highly recommend jBPM. As a Java-based workflow engine, it leverages framework capabilities and externalized assets like business processes, planning constraints, decision tables, and business rules in a unified environment. Business analysts, system architects, and developers can implement their business logic with persistence, transactions, messaging, events, etc. Automated Decisioning Support in jBPM helps analyze, track, and implement tasks using machine learning algorithms that automatically assess and process them. It facilitates process execution using the BPMN 2.0 specifications and Java’s object-oriented benefits. jBPM is helpful as a standalone service or code embedded into your customized service.

Over the years, jBPM has been increasingly used in the following areas:

• Business processes (BPMN2)
• Case management (BPMN2 and CMMN)
• Decision Management (DMN)
• Business rules (DRL)
• Business optimization (Solver)

jBPM lets you implement complex business logic with adaptive and dynamic processes that use business rules and advance event processing. jBPM can also be used in traditional JEE applications, Thorntail, or SpringBoot & standalone java programs. Control rests with the end-users who can monitor and work on which processes (wholly or partly) should be dynamically executed.

The Differentiating Features of jBPM 

Benefits of jBPM

• Better management visibility on business resources and processes and thus improved decision-making
• Low cost of inputs (at least 30%), less wastage (at least 40%), de-skilled labor requirements, and standardized components.
• Quality: Consistent and reliable output quality leads to higher customer satisfaction.
• Meant for everyone: Non-developers can effortlessly design business processes and obtain a much better view of runtime process states.
• Support for human tasks: jBPM Workflows can also create manual testing tasks or signing off on releases.
• Graphs: Complex workflows can be easily modeled with jBPM using a graphical designer and the Java code, which performs the workflow-triggered actions.
• Resilience: Existing workflow definitions remain unaffected by new processes.
• Flexibility: More variables relating to the approbation flow can be introduced dynamically in the workflow aligned to the specified rules and conditions.
• Multiple flows: jBPM helps manage multiple workflows simultaneously for a complex business logic through automation.

How we Implemented jBPM for a Ticketing System at Radiant Digital

This section will discuss how we implemented jBPM for a Ticketing System project of our client. The essential features of this implementation include:

• This project was deployed on an AWS server.
• jBPM Business Central was used for workflow creation and execution.
• Our implementation components included Business Rules, Rest APIs, script tasks, human tasks, Data Modular Forms, and Process level variables.
• Various REST APIs were developed and integrated with the jBPM use case, and the entire service was deployed on AWS.
• Dashboard and data visualization features have been implemented using Big Data to monitor and execute all the defined workflows' tasks and processes.

Implementation Steps

1. Launch JBPM business central.
2. Business Central has four activity options, Design, Deploy, Manage and track.

3. Use the “Design” option to create your project space and click to add project and add your required assets by clicking the “Add Asset” option to choose your asset.

4. For the ticketing system, we used the following steps.

• Create the ticket using REST APIs or create a ticket using an email sub-process (to create automatically).
• We used three REST APIs for CREATE, UPDATE, and DELETE operations, deployed them on this code on the AWS server, and used their endpoints in our project.
• We then defined the required object(s) separately.

Once the object(s) were defined, the form was automatically generated on jBPM. We could edit and design the form as required.

• The user needed to enter the ticket details in the form, and the form data was routed to store the “Create Ticket” REST API after submitting the details.
• This form appeared again, asking the user if an update was required. When the user clicked “Yes,” the user could update the ticket details that got stores on the “UPDATE” REST API; otherwise when the user clicks on “No,” the system directly moves on to the next step in the process.

5. After completing the first two steps, the system initiates the “Ticket Routing” sub-process. This sub-process gets the ticket details based on the ticket ID available in the APIs and checks the user’s country and region based on the business rule definition.

6. Based on the region name, the system ended the process and moved to the primary process again, or the aging user needed to enter the interface team name, and the team checked if the user had already been serviced or not. In case the user was not serviced, an ITT checks the TMG and the TOG business rules and ends the sub-process. It then enters the primary process in case the user is not serviced. This loop keeps happening until the user updates the ITT.

Create Ticket Process Flow

An Auto-assign sub-process automatically assigned the ticket for a particular user and sent the notification to that user. Here’s the process flow.

After completing the Auto-assign sub-process, the data again went to the primary process. Here, the data was retrieved using the “Get Task” Rest API, and the system checked if the Task Completion data was due along with the task status and updated these details using the “UPDATE TASK” Rest API.

After completing the Auto-complete sub-process, the data moved to the primary process the “EVENT-SUBPROCESS” API was used. In the primary process, we used the “intermediate” signal to transfer the data. The “start” signal was used to ask the user if additional data needed to be added.

Again, the user could update data by clicking “Yes” when the form appears or “No” to send the validation data. When the form appeared again, the user had to change the status details and call the “UPDATE” REST API, to update the changes. This completed the ticket.

The Ticket Details Form snapshot is given below.

The snapshot of the analytics-driven Ticketing System Dashboard is given below.

jBPM Migration Steps 

The three types of migration to be carried out include runtime process instance, data, and API calls' migration.

Runtime Process Instance Migration:

jBPM 7.0 comes with an excellent deployment model based on knowledge archives, allowing different project versions to run parallelly in a single execution environment. This is powerful; however, it raises some migration concerns. Some of the pressing questions include:

• Can users run both the old and new versions of the processes?
• What shall happen to already active process instances that were started with a previous version?
• Can active versions be migrated to a newer version and vice-versa?

Active process instances can be migrated, but this is not a straightforward process that can be performed via the jBPM console (aka kie-workbench). You can directly deploy the steps provided in this article to your installation and migrate any process instance. I explicitly use the term "migrate" instead of "upgrade" here because it can move from the lower to the higher version and vice-versa. Few things might happen when migration is performed. These depend on the differences between the process definitions of the two versions.

What does Migration Include?

• You can migrate from one process to another within the same Kjar.
• You can migrate from one process to another across Kjars.
• You can migrate with node mapping between the two process versions.

While the first two options are more straightforward, the third one requires some explanation.

Before we move to the migration scenarios, let's understand what node mapping is.

While making changes in the process versions, we might end up in a situation where others replace the nodes/activities. When migrating between these versions, node mapping needs to take place. Another scenario is when you'd like to skip some nodes in the current version.

Scenario 1: Simple migration of process instance

This case is about migrating active process instances from one version to another.

• "default org.jbpm:Evaluation:1.0" project is used, consisting of a single process definition - evaluation with version 1.0.
• A single process instance is started with this version. Once done, a new version of the evaluation process is created.
• The upgraded version is then released as part of the "org. jbpm: Evaluation:2.0" project with version 2.0.
• Then, the migration of the active process instance is performed.
• The process instance migration results are illustrated on the process model of the active instance and the outcome of the process instance migration.

Scenario 2: Process Instance migration with node mapping

Here, we go one step further and add another node to the Evaluation process to skip one of the original version nodes. For that, we need to map the nodes to be migrated. The steps are almost the same as in scenario 1, except that we need to perform additional steps to collect node information and then let the user manually select which nodes should be mapped to the new version.

Data Migration

• Two types of processes are involved in data migration. One for the external database migration (ex. MySQL, SQL…) and the other for internal data migration.
• jBPM has a default repository where all the data is stored. You can also import the code to the external repository and perform runtime migration.

API Calls Migration

The Red Hat JBoss BPM Suite 5 provides a task server bridged from the core engine using the messaging system provided by HornetQ. A helper or utility method called "LocalHTWorkItemHandler" helps you bridge the gap until you can migrate API calls in your current architecture. Since the "TaskService" API is part of the public API, you will need to refactor your imports and methods because of package and API changes.

What you get after the jBPM Migration 

• A high-performing rules engine based on the Drools project.
• Improved rule authoring tools and an enhanced user interface.
• A commonly defined methodology for building and deployment using Maven as the basis for repository management.
• A heuristic planning engine based on the OptaPlanner community project.
• Better algorithms to handle a more significant number of rules and facts.
• A new Data Modeler that replaces the declarative Fact Model Editor.
• Stability, usability, and functionality improvements.
• Case management capabilities.
• A new and simplified authoring experience for creating projects.
• Intuitive Business dashboards.
• Process and task admin API.
• Process and task consoles that can connect to any number of execution servers.
• A preview of a new process designer and form modeler.
• A new security management UI.
• Upgrades to Java8, WildFly 10, EAP 7, etc.

If you're looking for a seamless jBPM migration or project implementation, we could help you with our industry-leading expertise. 

Connect with us to learn more.

 

 

---

by Sri Arepally

The Fundamentals of MLOps – The Enabler of Quality Outcomes in Production Environments

With the increasing complexity of modeling frameworks and their relevant computational needs, organizations find it harder to meet the evolving needs of Machine Learning.

MLOps and DataOps help data scientists embrace collaborative practices between various technology, engineering, and operational paradigms.

MLOps is a set of practices that infuses Machine Learning, DevOps, and Data Engineering practices for a reliable and data-centric approach to Machine Learning systems during production.

It starts by defining a business problem, a hypothesis about the value extraction from the collected data, and a business idea for its application.

What is MLOps, and what does it entail?

It is a new cooperation method between business representatives, mathematicians, scientists, machine learning specialists, and IT engineers in creating artificial intelligence systems.

It applies a set of practices to augment quality, simplify the management processes, and automate Machine Learning and Deep Learning models' deployment in a large-scale production environment.

The processes involved in MLOps include data gathering, model creation (SDLC, continuous integration/continuous delivery), orchestration, deployment, diagnostics, governance, and business metrics.

Key Components of MLOps

MLOps Lifecycle

When it becomes necessary to retrain the model on new data in the operation process, the cycle is restarted - the model is refined and tested while a new version is deployed.

Why do we need MLOps?

MLOps streamlines process collaboration and integration through the automation of retraining, testing, and deployment.

AI projects include stand-alone experiments, ML coding, training, testing, and deployment to fit into the CI\CD pipelines of the development lifecycle.

MLOps automates model development and deployment to enable faster go-to-market and lower operational costs. It allows managers and developers to become more agile and strategic with their decision-making while addressing the following challenges.

Unrealized Potential: Many firms are striving to include ML in their applications to solve complex business problems. However, for many, incorporating ML into production has proven even more difficult than finding expert data scientists, useful data, and optimized training models.

Even the most sophisticated software solutions get caught up in deployment and become unproductive.

Frustrating Lag: Machine learning engineers often need to deploy an already trained model, which could cause anxiety. Communication between the operations and engineering teams requires incremental sign-offs and facilitation, spanning over weeks or months. Sometimes, a straightforward upgrade can feel insurmountable, especially for model enhancements, by switching from one ML framework to another.

Fatigue: Production processes lead to frustrated or underutilized engineers whose projects may not make it to the finish line. This causes process fatigue that stifles creativity and diminishes the motivation to deliver benchmark customer experiences.

Weak Communication: Data engineers, data scientists, researchers, and software engineers seem to be worlds apart from the operations team regarding the physical presence and thought process.

There is rarely sufficient streamlining to bring development and data management work to full production.

Lack of Foresight: Many ML data scientists don't have a particular way of knowing that their training models will work correctly in production. Even writing test cases and continually auditing their work with QA or Unit Testing cannot prevent the types of data encountered in production to differ from those used to train these models.

Therefore, gaining access to production telemetry to evaluate model performance against real-world data is very important.

However, the non-streamlined CI/CD processes to place the new model into production is a significant hindrance to realizing deriving value from machine learning.

Misused Talent: Data scientists differ from IT engineers. They primarily develop complex algorithms, neural network architectures, and data transformation mechanisms but do not handle Microservices deployment, network security, or other critical aspects of real-world implementations.

MLOps merges multiple disciplines with varied expertise to infuse machine learning into real applications and services.

DevOps vs. MLOps

By analogy, DevOps and DataOps help businesses organize continuous cooperation and interaction between all the process participants with machine learning models created by Data Scientists and ML developers.

Though MLOps evolved from DevOps, the two concepts are fundamentally different in the following ways.

MLOps	DevOps
It is more experimental in nature.	It is more implementation and process-oriented.
It needs a hybrid team composition of software engineers, data scientists, ML researchers, etc. focusing on exploratory data analysis, model development, and experimentation.	Teams are mostly composed of data engineers, scientists, and developers.
Testing an ML system involves model validation and training, in addition to unit and integration testing.	Mostly regression and integration testing are done.
A multi-step pipeline is required to retrain and deploy a model automatically.	A single CI/CD pipeline is enough to retrain and deploy a DevOps model.
Automatic execution of steps needs manual intervention before new models are deployed.	No Manual intervention is required for new model deployment since the entire process can be automated using various tools.
ML models can experience production system performance degradation due to evolving data profiles.	DevOps models do not experience any performance degradation due to evolving elements.
Continuous Training (CT) is used to retrain candidate models for testing and serving automatically. And Continuous Monitoring (CM) is used to capture and analyze production inference data and model performance metrics.	Continuous Training (CT) and Continuous Monitoring are not used, but Continuous Integration and Continuous Deployment are used.

The Four Pillars of MLOps

These four critical pillars support any agile MLOps solution for problematic situations to deliver machine learning applications in a production environment safely.

The Fifth Pillar of MLOps

Production Model Management: To ensure ML models' consistency and meet all business requirements at scale, a logical, easy-to-follow Model Management method is essential. Though optional, this paradigm streamlines end-to-end model training, packaging, validation, deployment, and monitoring to ensure consistency.

With Production Model Management, organizations can:

• Proactively address common business issues (such as regulatory compliance).
• Enable the sustainable tracking of data, models, code, and model versioning.
• Package and deliver models in reusable configurations.

Model Lifecycle Management: MLOps simplifies production model lifecycle management by automating troubleshooting and triage, champion/challenger gating, and hot-swap model approvals. It produces a secure workflow that efficiently manages your models' lifecycle as the number of models in production grows exponentially.

The key actions include:

• Champion/ challenger model gating introduces any new model (aka 'challenger') by initially running it in production and measuring its performance in comparison to its predecessor (aka 'champion') for a defined period to determine its worth to outperform its predecessor in terms of quality and model stability before becoming completely automated.
• Troubleshooting and Triage are used to monitor and rectify suspicious or poorly performing areas of the model.
• Model approval is designed to minimize risks associated with model deployment ensuring that all relevant business or technical stakeholders have signed off.
• Model update offers the ability to swap models without disrupting the production workflow, which is crucial for business continuity.

Production Model Governance: Organizations have to comply with CCPA and EU/UK GDPR before putting their machine learning models into production.

Organizations need to automate model lineage tracking (approvals, versions deployed, model interactions, updates, etc.).

MLOps offers enterprise-grade production model governance to deliver:

• Model version control
• Automated documentation
• Comprehensive lineage tracking and audit trails for the suite of production models

This helps minimize corporate and legal risks, maintain production pipeline transparency, and reduce/eliminate model bias.

Benefits of MLOps

Open Communication: MLOps helps merge machine learning workflows in a production environment to make data science and operations team collaborations frictionless.

It reduces bottlenecks formed by complicated and siloed ML models in development. MLOps-based systems establish dynamic and adaptable pipelines to enhance conventional DevOps systems to handle ever-changing, KPI-driven models.

Repeatable Workflows: MLOps allow automatic and streamlined workflow changes. A model can span through processes that accommodate data drifts without significant lags. MLOps consistently gauge and order model behavior and outcomes in real-time while streamlining iterations.

Governance / Regulatory Compliance: MLOps helps incentivize regulatory capacities and stringent policies. MLOps systems can reproduce models in compliance and accordance with original standards. As consequent pipelines and models evolve, your systems play by the rule book.

Focused Feedback: MLOps provides sophisticated monitoring capabilities, data drift visualizations, and data metrics detection over the model lifecycle to ensure high accuracy over time.

MLOps detects anomalies in machine learning development using analytics and alerting mechanisms to help engineers quickly understand the severity and act upon them promptly.

Reducing Bias: MLOps can prevent development biases that can lead to misrepresenting user requirements or subjecting the company to legal scrutiny. MLOps systems ensure that data reports do not have unreliable information. MLOps support the creation of dynamic systems that do not get pigeonholed in their reporting.

MLOps boosts ML models' credibility, reliability, and productivity in production to reduce system, resource, or human bias.

MLOps Target Users 

Data Scientists: MLOps helps Data Scientists collaborate with their Ops peers and offload most of their daily model management tasks to discover new use cases, work on feature plans, and build in-depth business expertise.

Business Leaders and Executives: MLOps lets decision-makers scale organization-wide AI capabilities while tracking KPIs influencing outcomes.

Software Developers: A robust MLOps system provides a functional deployment and versioning system to developers that include:

• Clear and straightforward APIs (REST)
• Developer support for ML operations (documentation, examples, etc.)
• Versioning and lineage control for production models
• Portable Docker images

DevOps and Data Engineers: MLOps offers a one-stop solution for DevOps and data engineering teams to handle everything from testing and validation to updates and performance management. This allows value generation from and scaling opportunities for internal deployment and service monitoring.

The benefits include:

• No-code prediction.
• Anomaly and bias warnings.
• Accessible and optimized APIs.
• Swappable models with gating of choice automation, smooth transitioning, and 100% uptime.

Risk and Compliance Teams: An MLOps infrastructure improves the quality of oversight for complex projects. A reliable MLOps solution supports customizable governance workflow policies, process approvals, and alerts.

It promoted a system that can self-diagnose issues and notify relevant RM stakeholders, allowing for tighter enterprise control over projects. Additional MLOps management capabilities include predictions-over-time analysis and audit logging.

Final Thoughts

A well-planned MLOps strategy can lead to more efficiency, productivity, accuracy, and trusted models for the long road ahead. All it takes is embracing its potential systematically and in line with your production environment needs.

Radiant Digital's MLOps experts can help to achieve this. Connect with us today to discuss the possibilities.

---

by Sri Arepally