---
title: Workshop Operationalizing the Machine Learning Pipeline
description: Implement a MlOps Pipeline, supported by an automated infrastructure for data preprocessing and feature engineering, model training and testing to production deployment.
image: https://www.bigindustries.be/hubfs/Industrializing%20ML%20Pipeline-1-1.jpeg
---

[![big-industries-logo](https://www.bigindustries.be/hubfs/popular-bigindustries/big-industries-logo.svg "big-industries-logo")](https://bigindustries.be)

- [Home](https://www.bigindustries.be)
- [Services](https://www.bigindustries.be/services)
- [References](https://www.bigindustries.be/references)
- [Team](https://www.bigindustries.be/our-team)
- [Projects](https://www.bigindustries.be/projects)
- [Jobs](https://www.bigindustries.be/jobs)
- [Blogs](https://www.bigindustries.be/blog)

[ Contact ](https://www.bigindustries.be/contact)

###### [Big Industries Academy](https://www.bigindustries.be/blog/tag/big-industries-academy)

# Workshop Operationalizing the Machine Learning Pipeline

![Matthias Vallaey](https://www.bigindustries.be/hs-fs/hubfs/popular-bigindustries/team/Matthias-Vallaey.png?width=56&name=Matthias-Vallaey.png) 

[ Matthias Vallaey ](https://www.bigindustries.be/blog/author/matthias-vallaey)

 Dec 27, 2023 4:17:42 PM

SHARE <https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fwww.bigindustries.be%2Fblog%2Fworkshop-operationalizing-the-machine-learning-pipeline> <https://twitter.com/intent/tweet?original_referer=https%3A%2F%2Fwww.bigindustries.be%2Fblog%2Fworkshop-operationalizing-the-machine-learning-pipeline&url=https%3A%2F%2Fwww.bigindustries.be%2Fblog%2Fworkshop-operationalizing-the-machine-learning-pipeline&source=tweetbutton&text=Workshop+Operationalizing+the+Machine+Learning+Pipeline> <https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fwww.bigindustries.be%2Fblog%2Fworkshop-operationalizing-the-machine-learning-pipeline&title=Workshop+Operationalizing+the+Machine+Learning+Pipeline&summary=Data+Scientists+and+ML+Practitioners+...> [mailto:?subject=Check%20out%20Workshop%20Operationalizing%20the%20Machine%20Learning%20Pipeline%20&body=Check%20out%20https://www.bigindustries.be/blog/workshop-operationalizing-the-machine-learning-pipeline](mailto:?subject=Check%20out%20Workshop%20Operationalizing%20the%20Machine%20Learning%20Pipeline%20&body=Check%20out%20https://www.bigindustries.be/blog/workshop-operationalizing-the-machine-learning-pipeline)

![Industrializing ML Pipeline-1-1-1](https://www.bigindustries.be/hs-fs/hubfs/Industrializing%20ML%20Pipeline-1-1-1.jpeg?width=724&height=379&name=Industrializing%20ML%20Pipeline-1-1-1.jpeg)

Data Scientists and ML Practitioners need more than a Jupyter notebook to build, test, and deploy ML models into production. They also need to ensure they perform these tasks in a reliable, flexible, and automated fashion.

There are three basic questions that should be considered when starting the ML journey to develop a model for a real Business Case:

1. How long would it take your organization to deploy a change that involves a single line of code?
2. How do you account for model concept drift in production
3. Can you build, test and deploy models in a repeatable, reliable and automated way?

So, if you’re not happy with your answers to these questions, then MLOps is a concept that can help you to:

- Create or improve the organization culture and agility for Continuous Integration/Continuous Delivery (CI/CD).
- Create an automated infrastructure that will support your processes to operationalize your ML model.

In this workshop you will build an **MLOps** pipeline that leverages AWS Sagemaker, a service that supports the an entire pipeline for ML model development, and is the heart of this solution. Around it, you will add different AWS DevOps tools and services to create an automated CI/CD pipeline for the ML model. This pipeline will prepare the data, build your docker images, train and test the ML model, and then integrate the model into a production workload.

The workshop is comprised of *four* primary modules that will, from the perspective of the **ML Practitioner**, focus on creating and executing the MLOps CI/CD Pipeline:

![MLOPS workshop](https://www.bigindustries.be/hs-fs/hubfs/MLOPS%20workshop.png?width=724&height=408&name=MLOPS%20workshop.png)

#### Configure the MLOps CI/CD Pipeline Source and Data Repositories

- The **ML Practitioner** creates a repository in AWS CodeCommit to store model source code.
- Additionally, they create repositories to store Model Training and Inference images.
- The ML Practitioner will use Amazon S3 to configure raw (*unprocessed*) training data as well as pre-processed (*transformed*) training data.

#### Configuring the MLOps CI/CD Pipeline Assets<https://catalog.us-east-1.prod.workshops.aws/workshops/44d3e2a0-ec6f-44df-9397-bcfdf129cadf/en-US/module-introduction-1/scenario-1-1#configuring-the-mlops-cicd-pipeline-assets>

- The **ML Practitioner** uses AWS CodeCommit and AWS CodeBuild to store the pipeline automation components or assets.
- The **ML Practitioner** also performs any local ML model testing to ensure these assets will be functional, before they are deployed by the pipeline.

#### Building and Executing the MLOps CI/CD Pipeline

- Since the components are codified, the **ML Practitioner** deploys the infrastructure as code, using AWS CloudFormation.
- Then, the **ML Practitioner** trains the ML model automatically using Amazon SageMaker, as a function of the pipeline.

#### Managing the MLOps CI/CD Pipeline Deployment

- The trained model is then deployed into a staging environment, **DEV - QA/Development (simple endpoint)**, where it is tested to not only ensure that the deployed infrastructure is functional, but also that the model meets the business use case requirements in production. The testing procedure is automated to further optimize the pipeline flow.
- After the model is tested, it is deployed into production, **PRD - Production (HA/Elastic endpoint)**. Here the **ML Practitioner** tests that the production environment can scale, and monitors the the model to ensure that it is maintaining a predetermined desired level of performance, or consistent model quality.

 [![I'm interested to learn more](https://hubspot-no-cache-eu1-prod.s3.amazonaws.com/cta/default/2240994/7e6ce36c-65ba-4bff-b8a3-5624556027a5.png)](https://hubspot-cta-redirect-eu1-prod.s3.amazonaws.com/cta/redirect/2240994/7e6ce36c-65ba-4bff-b8a3-5624556027a5)

 

Source: https://workshops.aws/

![Matthias Vallaey](https://www.bigindustries.be/hs-fs/hubfs/popular-bigindustries/team/Matthias-Vallaey.png?width=66&name=Matthias-Vallaey.png)

#### [ Matthias Vallaey ](https://www.bigindustries.be/blog/author/matthias-vallaey)

Matthias is founder of Big Industries and a Big Data Evangelist. He has a strong track record in the IT-Services and Software Industry, working across many verticals. He is highly skilled at developing account relationships by bringing innovative solutions that exceeds customer expectations. In his role as Entrepreneur he is building partnerships with Big Data Vendors and introduces their technology where they bring most value.

<https://be.linkedin.com/in/matthias-vallaey-a82571>

## Related posts

At Big Industries, we want to share our wisdom. More info on our recent developments, newest projects and upcoming events

[![OmaJaguarV4VsHondje-2-1-1](https://www.bigindustries.be/hubfs/OmaJaguarV4VsHondje-2-1-1.jpg) ](https://www.bigindustries.be/blog/from-big-data-to-big-industries)

###### [General](https://www.bigindustries.be/blog/tag/general)

#### [FROM BIG DATA TO BIG INDUSTRIES](https://www.bigindustries.be/blog/from-big-data-to-big-industries)

From the very beginning, BIG Industries has been driven by a singular goal: to empower companies...

[Read more ](https://www.bigindustries.be/blog/from-big-data-to-big-industries)

[![Building Real Time Data Pipelines with Apache Kafka](https://www.bigindustries.be/hubfs/kafka.png) ](https://www.bigindustries.be/blog/building-real-time-data-pipelines-with-apache-kafka)

###### [Confluent](https://www.bigindustries.be/blog/tag/confluent)

#### [Building Real Time Data Pipelines with Apache Kafka](https://www.bigindustries.be/blog/building-real-time-data-pipelines-with-apache-kafka)

Apache Kafka is a distributed publish-subscribe messaging system that is designed to be fast,...

[Read more ](https://www.bigindustries.be/blog/building-real-time-data-pipelines-with-apache-kafka)

#### Subscribe to our newsletter

Stay informed about our recent developments, newest projects and upcoming events

![img-mails](https://f.hubspotusercontent10.net/hubfs/369261/raw_assets/public/Marketplace/leadstreet/themes/popular-theme/images/img-mails.svg)

## Ready to set off on a BIG journey?

##### The top notch technologies we use set us apart from other consultancies

[HAVE A LOOK AT OUR SERVICES](https://www.bigindustries.be/services)

**Big Industries NV**

**YOU THINK BIG, YOU GET BIG**

<https://www.facebook.com/Big-Industries-933906943382629/> <https://www.linkedin.com/company/4838922/>

**Contact us**

Veldkant 33a,  
2550 Kontich, Belgium  
[GET DIRECTIONS ](https://www.bigindustries.be/contact)

T +32 (0)3 450 80 30  
E [info@bigindustries.be](mailto:info@bigindustries.be)

**Hours**

Monday to Friday  
9am – 5pm CET

Weekends  
Closed

##### **Drop us a line**

© Big Industries 2021 - all rights reserved

- [TERMS AND CONDITIONS](https://www.bigindustries.be/terms-and-conditions)
- [PRIVACY POLICY](https://www.bigindustries.be/privacy-policy)