Author: Krishna Kumar Ramanujam

What to look for in an MLOps tool –…

In an earlier blog, we explain how MLOps helps to reduce inherent risks associated with the development and deployment of AI/ML models in enterprises.

Implementing MLOps workflows, as with any other workflows, involves the age-old choice of “Build vs. Buy” – whether to implement a tool from scratch or buy (or subscribe to) an existing off-the-shelf tool. Each organization may choose to respond to this dichotomy in its own manner, keeping its unique requirements and capabilities in mind. Either way, it would help to list these requirements to ensure that these are kept in mind during the decision process.

An MLOps tool should enable all stakeholders of the AI/ML development and deployment process to perform their tasks with improved efficiency and productivity. From this point of view, the requirements of such a tool can be classified into two categories:

• Technical – features that address specific technical requirements of data scientists, business analysts, project managers, and other users
• Non-technical – features that straddle user roles and are typically required by any enterprise-wide software.

In this blog, we enumerate the non-technical feature requirements, focusing on technical features in the next article in this series.

Non-technical features required from an MLOps tool are the same as those offered by any enterprise-grade software, with some tweaks to cater to their unique positioning. These include:

1. Installation Flexibility

• Does the tool tie the organization into a specific cloud? 
• Or is it cloud-agnostic? 
• Does it have an on-premise offering? 
• Does it provide a PaaS (Platform as a Service) version? 

2. Development Best Practices

The primary users of the tool will be the Data Scientist team. 

• Does it support architectural best practices – e.g., modularity, reusability? 
• Does it integrate with code versioning systems? 
• What languages does it support (a minimum set should include Python, R, Java, Scala, and GoLang)? 
• Is it flexible – does it impose custom ML libraries on developers, or does it support all popular ML libraries?

3. Collaboration

Many tools look at data scientists as “lone rangers”, working on their own to develop, test and deploy models. However, the entire AI/ML development and deployment process involves a team of data scientists, business analysts, Production Engineers, and Project Managers to complete the entire workflow. 

• Does the tool support collaboration among all the stakeholders through a mix of appropriate workflows and collaboration tools?

4. Security

• Does it integrate with the enterprise security systems? 
• Does it provide support for industry-standard security best practices?
  Is the data within the tool secure, both at rest and in transit? 
• Does it provide VPN-based access as a standard? 

5. Access Control

• Does the tool provide Role-Based Access Control (RBAC) features? 
• Can enterprises define custom roles and assign functions to these roles? Ideally, administrators should be able to assign individual actions permitted in the tool to a specific role(s).

6. Governance

• Does the tool provide governance mechanisms through workflow approvals? 
• Can you set multiple levels of approval authorities for any action? 
• Are the logs adequate for failure analysis? 
• Does it provide audit trails to ensure traceability? 
• Is there model provenance defined by entity (which data, algorithm, and hyperparameter versions), activity (when deployed and the changes since the last deployment), and agent (who all signed off on the deployment)?

7. Performance

• Does the tool support containerization? 
• Are the models deployed to a scalable, fault-tolerant environment? 
• Can you increase/decrease the deployment cluster size as required?

8. Ease of Use

• Does the tool have an intuitive GUI to enable even non-technical business users to use it effectively? 
• Is there sufficient documentation available? 
• Does it include starter projects to reduce the learning curve? 

9. Support

• Does the tool provider provide adequate support? One possible way to reduce the learning curve is to provide a managed service offering, wherein an engineer from the provider will support your AI/ML operations for a specific period of time before the final handover

10. Provider Reputation

• What’re the provider credentials like? 
• Have they implemented any AI/ML projects themselves? (In other words, do they eat their own dog food?) 
• Does the leadership team have a Data Science background?

These are some of the questions every enterprise must ask before embarking on this choice. There are several MLOps tools available today, including xpresso.ai, AWS Sagemaker, Microsoft Azure ML, Allegro, Iguazio, DataRObot, Dataiku, etc. Make sure you evaluate all of them using these criteria and select the one best suited for your needs. Best of Luck!

How to select the Most Suitable ML Platform

Adoption of ever-changing technology and the development of digital transformation initiatives has prompted businesses to embrace Business Analytics – businesses no longer rely on IT-driven activities and intelligence functions. encompassing.

Stakeholders and decision-makers in today’s business world require pertinent and current insights from units and processes across the organization. Using quarterly reports and end-of-the-month statement as conventional information sources is no longer enough in today’s world. Company-wide data-powered analytics solutions empower different approaches that influence decision-making.

Machine Learning (ML), therefore, becomes an imperative in this regard. Most companies believe that ML, Artificial Intelligence (AI), and analytics are the magical answer to optimize processes and uncover new opportunities.

Several studies show how ML and AI help companies to gain an upper hand. However, many organizations rush into deploying an ML platform without considering all the important aspects, and often, reach a point of no return in their analytics journey.

Therefore, organizations should be completely prepared when they adapt, develop or deploy ML. In this article, we go through some aspects of determining the most appropriate ML platform.

Usability

Data scientists should not stress over infrastructure installation and tuning, following complex procedures for custom dashboard creation, training experiments, model provenance, model deployment and monitoring. The selected ML platform should provide all these features. When assessing such a platform, we should focus on the three aspects; simple to customize, simple to set up, and easy installation process.

ETL or ELT Pipeline Support

ETL (export, transform, and load) and ELT (export, load, and transform) are the common data pipeline configurations for loading data. Deep learning and ML augment the requirements for these, particularly the transform phase. ELT gives you greater flexibility when your transformations need to change, as the load stage is typically the most tedious for big data.

A good ML platform should support various pipeline flavors, and go beyond. For example, raw data is often very noisy and may need to be filtered. This should automatically be done by the ML platform. Or, such data often have differing ranges: One variable may have a very large range ( 0 to 1 million), while another has a much smaller range, say – 0.1 to – 0.001. A good ML platform would automatically normalize such variables to prevent the ones with huge ranges from ruling the model.

APIs: Cloud ML Services

APIs are usually important when you want to opt for a cloud-based ML platform. Such platforms are mainly APIs disposed towards application developers and individuals that wish to integrate AI into already existing applications. Such platforms are purpose-built and will always zero in on explicit tasks, say – computer vision, language processing, and translation. Amazon Web Services, Google Cloud, and Microsoft Azure are top sellers in this category.

Scalability

When production deployments of models is being considered, a scalable platform is an absolute necessity. The organization can pick a platform that meets its present requirements as well as can scale for what’s to come. The selected platform should be able to oversee cost versus user experience productively, sophistically gauge for both batch and real-time workloads, scale automatically with the expanding traffic, serve high throughput situations, follow the safe implementation and deliver best practices.

Support Deep Learning and AI Frameworks

Most data scientists have their own favorite frameworks and programming languages for AI and deep learning. For folks who lean toward Python, a top choice for ML is Scikit-learn, while TensorFlow, PyTorch, Keras, and MXNet are top picks for deep learning. In Scala, Spark MLlib will, in general, be liked for ML. In R, there are numerous native ML packages and a decent interface to Python. In Java, H2O.ai rates high, as do Java-ML and Deep Java Library.

Deep learning and cloud machine platforms usually have their variety of algorithms. They tend to support external frameworks as containers with explicit entry points or in at least one language. You can incorporate your algorithms and statistical methods with the AutoML capacities in the platform and this can be very advantageous.

Reproducibility

When managing a model life cycle, it is impoerative to be able to track models, change them easily, and reproduce them at will. Whether working together with colleagues, emphasizing a current model, or investigating a production failure, reproducibility is critical. Therefore, you have to make sure the ML platform is built with reproducibility as an inbuilt feature.

With all these features and requirements comes another question, should we Build or Buy ML Platforms?

The three important factors to consider when deciding whether to build or purchase an ML platform include its abilities, ultimate objectives, and the quality and number of people present in the data science team. A company can pick an augmented or commercial platform if it has a small data science team and wishes to unburden some of the model building work onto a ML platform provider. The seller and platform should be chosen when they assess and affirm that it will address the goals and objectives of a company in regards to ML.

There are a lot of providers who accomplish the grunt work through patching together a suite of open-source packages. When working with an ML platform provider, you should ask what tools they are utilizing because it is plenty significant as going out and working out on what tools can be used in-house.

Organizations with an incredible data science team and an up-to-date AI strategy can utilize their in-house tools to deal with ML goals or issues. However, you should not forget that choosing customized model structures or choosing one’s parameters is not generally a quick or straightforward beginning of an ML platform architecture.

What does an ML Platform look like

Artificial Intelligence (AI) has had a significant impact on the way the world works, and consequently, more and more organizations are interested in harnessing the power of AI. According to an MIT Sloan-BCG report, a high percentile (93%) of C-level executives believe that artificial intelligence can create valuable investments. 25% of the businesses that have adopted AI are in the mature stage, and most of them are in the assessment stage. However, AI adoption has not proved its worth within companies – 65% of those executives do not realize  ROI despite significant investments in AI. Companies approach us for effective ML solutions, but the failure seems to stem from bad implementation of the AI approach.

In the entire AI development lifecycle, the responsibilities of data scientists and software engineers are clear – while data scientists focus on the model’s precision, software engineers are responsible for the deployment of the software. The distinct nature of these two roles has brought about a need for a role that can overlap the deployment and model precision functions. This role is now more popularly known as the ModelOps Engineer, who is responsible for the model’s scalability and delivery. To mitigate AI and Machine Learning (ML) adoption challenges, we always recommend that early AI adopters include this role in their projects from an early stage.

The field of IT DevOps includes a set of practices that empower constant delivery and a shorter development cycle. So, when these DevOps practices are applied to ML systems, we get MLOps. This is an ML engineering culture and practice that intends to bind together ML system development (Dev) and ML system operation (Ops). Implementing MLOps implies that you advocate for automation and monitor all phases of ML system construction that comprise architecture, implementation, deployment, and infrastructure management, testing, and releasing. 

This article aims at emphasizing the architecture and core components of an MLOps platform.

The three significant components of MLOps are AutoML, ModelOps, and DataOps. AutoML aims at increasing and improving model creation while helping data scientists explore their ideas without hindrances. ModelOps allows the engineers to increase the speed of time-to-development while monitoring model-in-action. DataOps, on the other hand, describes the set of practices that improve data integration to address security and preprocessing issues.

Model Training Pipeline

This is the stage when Machine Learning Engineers create a training pipeline coupled with new features which allow them to fix any problems as they monitor the training process. Engineers for ModelOps test different parts of the pipeline so that they can use them as instruments for taking them into the target environment. Nonetheless, every time the pipeline does

• Data ingestion
• Data Preparation
• Model Training
• Model Validation
• Data Versioning

Model Registry

This stage enables ML engineers to collaborate and share models with Ops engineers for model management enhancement. A model registry acts as a centralized hub that stores all metadata for published models.

The model registry allows the growth of a communication layer that traverses between production environments and research furnishing the deployed model with the data it needs in runtime. It does all this while facilitating track changes.

Model Serving

In this stage, ML engineers will kickstart testing in production, whereas Ops engineers will control model deployment. Some of the common methods to put models in production are embedded in consumer applications, on an IoT edge device, and within a dedicated web service available through remote procedure call (RPC).

However, one new approach – Model-as-a-Service, is gaining popularity because of its simplified deployment and the ability to separate the ML part from software code. This ensures that you easily upgrade the version of the model without the need of redeploying the application.

TFX, MLflow, and Kubeflow put certain models together so that they can be incorporated into Kubernetes. These models, such as Docker images, are also merged on other special servers such as Clipper and TensorFlow Serving.

Model Monitoring

ML engineers use this stage to analyze metrics and understand system alerts. Until the model is given to production, the model performance might be influenced by numerous factors.

In certain instances, the accuracy is unacceptable, which means the algorithm must be retrained. Generally, ML models don’t show errors instantly, but their predictions can affect the results. Distorted insights can cause wrong decisions and encourage financial problems. To get away from all of this, we recommend companies deploy software solutions that will automatically identify anomalies, give early alerts, and trigger ML pipelines for retraining.

CI/CD Orchestration

Here, Ops engineers bind multiple processes and automate the complete release process of the model. As we have all parts of MLOps flow in place, the next step is to orchestrate various operations such as performance of a training pipeline, carrying out tests and validations, and implementation of new model versions. For this, organizations should make use of CI/CD tools. These tools can visualize complex workflows and effectively manage the entire application lifecycle. This enables organizations to bring together pipelines easily.

Many organizations tend to neglect the need for a controller despite the above core components of an MLOps platform architecture. We believe the role of a controller is quite crucial as it oversees the challenges around governance, security, and best practices defined.

To end, MLOps is essentially the blend of the best practices’ pioneers have embraced after understanding the difficulties in executing AI. Our MLOps platform, Xpresso AI, accelerates your AI capacities and helps develop an edge over your competitors. With an MLOps platform, you can develop sustainable long-term practices, empowering a never-ending flow of benefits from AI.