Multi runtime serving pipelines for machine learning

•

0 j'aime•648 vues

The talk I gave at Scale By The Bay. Deploying, Serving and monitoring machine learning models built with different ML frameworks in production. Envoy proxy powered serving mesh. TensorFlow, Spark ML, Scikit-learn and custom functions on CPU and GPU.

Logiciels

Multi-Runtime Serving Pipelines
Stepan Pushkarev
CTO of Hydrosphere.io

Mission: Accelerate Machine Learning to Production
Opensource Products:
- Mist: Serverless proxy for Spark
- ML Lambda: ML Function as a Service
- Sonar: Data and ML Monitoring
Business Model: Subscription services and hands-on consulting
About

Deployment | Serving | Scoring | Inference
@Nvidia https://www.nvidia.com/en-us/deep-learning-ai/solutions/

- HTTP/1.1, HTTP/2, gRPC
- Kafka, Flink, Kinesis
- Protobuf, Avro
- Service Discovery
- Pipelining
- Tracing
- Monitoring
- Autoscaling
- Versioning
- A/B, Canary
- Testing
- CPU, GPU
API & Logistics

Monitoring
Shifting
experimentation
to production

Functions registry
responsible for the
model life cycle and
all the business logic
required to configure
models for serving
Mesh of serving
runtimes is an actual
serving cluster
Infrastructure
integration: ECS for
AWS, Kubernetes for GCE
and on premise

UX: Models and Applications
Applications provide public virtual endpoints for the
models and compositions of the models.

Why Not just one Big Neural Network?
● Not always possible
● Stages could be independent
● Ad-hoc rule based models
● Physics models (e.g. LIDAR)
● Big E2E DL Requires Black
Magic skills

Why Not just one Python script?
● Modularity. Stages could be developed by different teams
● Traceability and Monitoring
● Versioning
● Independent deployment, A/B testing and Canary
● Request Shadowing and other cool stuff
● Could require different ML runtimes (TF, Scikit, Spark
ML, etc)
● We need more microservices :)

Why Not just TF Serving?
● Other ML runtimes (DL4J, Scikit,
Spark ML). Servables are overkill.
● Need better versioning and
immutability (Docker per version)
● Don’t want to deal with state
(model loaded, offloaded, etc)
● Want to re-use microservices stack
(tracing, logging, metrics)
● Need better scalability

Thank you
- @hydrospheredata
- https://github.com/Hydrospheredata
- https://hydrosphere.io/
- spushkarev@hydrosphere.io

Contenu connexe

Tendances

MLOps is a trend in machine learning (ML) engineering that unifies ML system development (Dev) and ML system operation (Ops). Some ML lifecycle frameworks, such as TensorFlow Extended, are based around end-to-end pipelines that start with raw data and end in production models. During this talk we will introduce the concept of a feature store as the missing piece of ML infrastructure that enables faster lower cost deployment of models. We will show how the Hopsworks Feature Store - factors monolithic end-to-end ML pipelines into feature and model training pipelines that can each run at different cadences. We will show examples of ingestion and training pipelines including hyperparameter optimization and model deployment.

Hamburg Data Science Meetup - MLOps with a Feature Store

Moritz Meister

Revolutionary container based hybrid cloud solution for MLPlatform Ness' data science platform, NextGenML, puts the entire machine learning process: modelling, execution and deployment in the hands of data science teams. The entire paradigm approaches collaboration around AI/ML, being implemented with full respect for best practices and commitment to innovation. Kubernetes (onPrem) + Docker, Azure Kubernetes Cluster (AKS), Nexus, Azure Container Registry(ACR), GlusterFS Workflow Argo->Kubeflow DevOps Helm, kSonnet, Kustomize,Azure DevOps Code Management & CI/CD Git, TeamCity, SonarQube, Jenkins Security MS Active Directory, Azure VPN, Dex (K8s) integrated with GitLab Machine Learning TensorFlow (model training, boarding, serving), Keras, Seldon Storage (Azure) Storage Gen1 & Gen2, Data Lake, File Storage ETL (Azure) Databricks, Spark on K8, Data Factory (ADF), HDInsight (Kafka and Spark), Service Bus (ASB) Lambda functions & VMs, Cache for Redis Monitoring and Logging Graphana, Prometeus, GrayLog

NextGenML

Moldovan Radu Adrian

Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure. For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps). Agenda - Data Quality and why it matters - Challenges and solutions of Data Testing - Challenges and solutions of Model Testing - MLOps pipelines and why they matter - How to expand validation pipelines for Data Quality

MLOps and Data Quality: Deploying Reliable ML Models in Production

Provectus

In large enterprises, large solutions are sometimes required to tackle even the smallest tasks and ML is no different. At Comcast we are building a comprehensive, configuration based, continuously integrated and deployed platform for data pipeline transformations, model development and deployment. This is accomplished using a range of tools and frameworks such as Databricks, MLflow, Apache Spark and others. With a Databricks environment used by hundreds of researchers and petabytes of data, scale is critical to Comcast, so making it all work together in a frictionless experience is a high priority. The platform consists of a number of components: an abstraction for data pipelines and transformation to allow our data scientists the freedom to combine the most appropriate algorithms from different frameworks , experiment tracking, project and model packaging using MLflow and model serving via the Kubeflow environment on Kubernetes. The architecture, progress and current state of the platform will be discussed as well as the challenges we had to overcome to make this platform work at Comcast scale. As a machine learning practitioner, you will gain knowledge in: an example of data pipeline abstraction; ways to package and track your ML project and experiments at scale; and how Comcast uses Kubeflow on Kubernetes to bring everything together.

How to Utilize MLflow and Kubernetes to Build an Enterprise ML Platform

Databricks

Blind spots in big data erez koren @ forter

Ido Shilon

Getting cars to drive autonomously is one of the most exciting problems these days. One of the key challenges is making them drive safely, which requires processing large amounts of data. In our talk we would like to focus on only one task of a self-driving car, namely road detection. Road detection is a software component which needs to be safe for being able to keep the car in the current lane. In order to track the progress of such a software component, a well-designed KPI (key performance indicators) evaluation pipeline is required. In this presentation we would like to show you how we incorporate Spark in our pipeline to deal with huge amounts of data and operate under strict scalability constraints for gathering relevant KPIs. Additionally, we would like to mention several lessons learned from using Spark in this environment.

Lessons Learned from Using Spark for Evaluating Road Detection at BMW Autonom...

Databricks

Managing the Machine Learning Lifecycle with MLflow

Databricks

This talk describes migrating a large random forest classifier from scikit-learn to Spark's MLlib. We cut training time from 2 days to 2 hours, reduced failed runs, and track experiments better with MLflow. Kount provides certainty in digital interactions like online credit card transactions. One of our scores uses a random forest classifier with 250 trees and 100,000 nodes per tree. We used scikit-learn to train using 60 million samples that each contained over 150 features. The in-memory requirements exceeded 750 GB, took 2 days, and were not robust to disruption in our database or training execution. To migrate workflow to Spark, we built a 6-node cluster with HDFS. This provides 1.35 TB of RAM and 484 cores. Using MLlib and parallelization, the training time for our random forests are now less than 2 hours. Training data stays in our production environment, which used to require a deploy cycle to move locally-developed code onto our training server. The new implementation uses Jupyter notebooks for remote development with server-side execution. MLflow tracks all input parameters, code, and git revision number, while the performance and model itself are retained as experiment artifacts. The new workflow is robust to service disruption. Our training pipeline begins by pulling from a Vertica database. Originally, this single connection took over 8 hours to complete with any problem causing a restart. Using sqoop and multiple connections, we pull the data in 45 minutes. The old technique used volatile storage and required the data for each experiment. Now, we pull the data from Vertica one time and then reload much faster from HDFS. While a significant undertaking, moving to the Spark ecosystem converted an ad hoc and hands-on training process into a fully repeatable pipeline that meets regulatory and business goals for traceability and speed. Speaker: Josh Johnston

Moving a Fraud-Fighting Random Forest from scikit-learn to Spark with MLlib, ...

Databricks

Modern machine learning systems may be very complex and may fall into many pitfalls. It's very easy to unintendedly introduce technical debt into such a complex structure. One of the approaches solving some of anti-patterns is a feature store. Feature store is a missing piece filling a gap between raw data and machine learning models. Not only it will help you to handle technical debt, but even more importantly speeds up time to develop new model.

Feature store: Solving anti-patterns in ML-systems

Andrzej Michałowski

All hyperscale AI companies build their machine learning platforms around a Feature Store. A feature is a measurable property of some data-sample. It could be for example an image-pixel, a word from a piece of text, the age of a person, a coordinate emitted from a sensor, or an aggregate value like the average number of purchases within the last hour. A Feature Store is a central place to store curated features within an organization. Feature Stores are a fuel for AI systems as we use them to train machine learning models so that we can make predictions for feature values that we have never seen before. During this presentation you learn: - About the concept of a Feature Store and how it can help manage feature data for Enterprises and ease the path of data from backend systems and data-lakes to Data Scientists. - Our take on Feature Stores, including best practices and use cases and: - How to ensure Consistent Features in both Training and Serving Governance, Access-Control, and Versioning - To create Training Data in the File Format of your Choice Eliminate Inconsistency between Features in Training and Inferencing Watch the webinar with a demo: https://www.logicalclocks.com/webinars

Managed Feature Store for Machine Learning

Logical Clocks

MLOps - Build pipelines with Tensor Flow Extended & Kubeflow

Jan Kirenz

Ai platform at scale

Henry Saputra

Model versioning done right: A ModelDB 2.0 Walkthrough

Manasi Vartak

DevOps and Machine Learning (Geekwire Cloud Tech Summit)

Jasjeet Thind

Transformer-based pretrained language models such as BERT, XLNet, Roberta and Albert significantly advance the state-of-the-art of NLP and open doors for solving practical business problems with high performance transfer learning. However, operationalizing these models with production-quality continuous integration/ delivery (CI/CD) end-to-end pipelines that cover the full machine learning life cycle stages of train, test, deploy and serve while managing associated data and code repositories is still a challenging task.

Continuous Delivery of Deep Transformer-Based NLP Models Using MLflow and AWS...

Databricks

Robust MLOps with Open-Source: ModelDB, Docker, Jenkins, and Prometheus

Manasi Vartak

"The advent of pre-trained language models such as Google’s BERT promises a high performance transfer learning (HPTL) paradigm for many natural language understanding tasks. One such task is email classification. Given the complexity of content and context of sales engagement, lack of standardized large corpus and benchmarks, limited labeled examples and heterogenous context of intent, this real-world use case poses both a challenge and an opportunity for adopting an HPTL approach. This talk presents an experimental investigation to evaluate transfer learning with pre-trained language models and embeddings for classifying sales engagement emails arising from digital sales engagement platforms (e.g., Outreach.io). We will present our findings on evaluating BERT, ELMo, Flair and GloVe embeddings with both feature-based and fine-tuning based transfer learning implementation strategies and their scalability on a GPU cluster with progressively increasing number of labeled samples. Databricks’ MLFlow was used to track hundreds of experiments with different parameters, metrics and models (tensorflow, pytorch etc.). While in this talk we focus on email classification task, the approach described is generic and can be used to evaluate applicability of HPTL to other machine learnings tasks. We hope our findings will help practitioners better understand capabilities and limitations of transfer learning and how to implement transfer learning at scale with Databricks for their scenarios."

High Performance Transfer Learning for Classifying Intent of Sales Engagement...

Databricks

AT&T has been involved in AI from the beginning, with many firsts; “first to coin the term AI”, “inventors of R”, “foundational work on Conv. Neural Nets”, etc. and we have applied AI to hundreds of solutions. Today we are modernizing these AI solutions in the cloud with the help of Databricks and a variety of in-house developments. This talk will highlight our AI modernization effort along with its application to Fraud which is one of our biggest benefitting applications.

AI Modernization at AT&T and the Application to Fraud with Databricks

Databricks

Apply MLOps at Scale by H&M

Databricks

Next.ml Boston: Data Science Dev Ops

Eric Chiang

Tendances (20)

Hamburg Data Science Meetup - MLOps with a Feature Store

NextGenML

MLOps and Data Quality: Deploying Reliable ML Models in Production

How to Utilize MLflow and Kubernetes to Build an Enterprise ML Platform

Blind spots in big data erez koren @ forter

Lessons Learned from Using Spark for Evaluating Road Detection at BMW Autonom...

Managing the Machine Learning Lifecycle with MLflow

Moving a Fraud-Fighting Random Forest from scikit-learn to Spark with MLlib, ...

Feature store: Solving anti-patterns in ML-systems

Managed Feature Store for Machine Learning

MLOps - Build pipelines with Tensor Flow Extended & Kubeflow

Ai platform at scale

Model versioning done right: A ModelDB 2.0 Walkthrough

DevOps and Machine Learning (Geekwire Cloud Tech Summit)

Continuous Delivery of Deep Transformer-Based NLP Models Using MLflow and AWS...

Robust MLOps with Open-Source: ModelDB, Docker, Jenkins, and Prometheus

High Performance Transfer Learning for Classifying Intent of Sales Engagement...

AI Modernization at AT&T and the Application to Fraud with Databricks

Apply MLOps at Scale by H&M

Next.ml Boston: Data Science Dev Ops

En vedette

Presented at PyOhio 2017: https://pyohio.org/schedule/presentation/284/ The Python data ecosystem provides amazing tools to quickly get up and running with machine learning models, but the path to stably serving them in production is not so clear. We'll discuss details of wrapping a minimal REST API around scikit-learn, training and persisting models in batch, and logging decisions, then compare to some other common approaches to productionizing models.

Machine learning in production with scikit-learn

Jeff Klukas

Over the course of three years, we've built Stripe from scratch and scaled it to process billions of dollars of transaction volume a year by making it easy and painless for merchants to get set up and start accepting payments. While the vast majority of transactions facilitated by Stripe are honest, we do need to protect our merchants from rogue individuals and groups seeing to "test" or "cash" stolen credit cards. To combat this sort of activity, Stripe uses Python (together with Scala and Ruby) as part of its production machine learning pipeline to detect and block fraud in real time. In this talk, I'll go through the scikit-based modeling process for a sample data set that is derived from production data to illustrate how we train and validate our models. We'll also walk through how we deploy the models and monitor them in our production environment and how Python has allowed us to do this at scale.

Python as part of a production machine learning stack by Michael Manapat PyDa...

PyData

With so many options to choose from how do you select the right technologies to use for your machine learning pipeline? Do you purchase bare metal and hire a devops team, install Spark on EC2 instances, use EMR and other AWS services, combine Spark and Elasticsearch?! View this talk to get a first-hand experience of building ML pipelines: what options were looked at, how the final solution was selected, the tradeoffs made and the final results.

Building A Production-Level Machine Learning Pipeline

Robert Dempsey

Managing and Versioning Machine Learning Models in Python

Simon Frid

Learn how to use PySpark for processing massive amounts of data. Combined with the GitHub repo - https://github.com/rdempsey/pyspark-for-data-processing - this presentation will help you gain familiarity with processing data using Python and Spark. If you're thinking about machine learning and not sure if it can help improve your business, but want to find out, set up a free 20-minute consultation with us: https://calendly.com/robertwdempsey/free-consultation

Using PySpark to Process Boat Loads of Data

Robert Dempsey

Production machine learning_infrastructure

joshwills

Production and Beyond: Deploying and Managing Machine Learning Models

Turi, Inc.

Machine learning in production

Turi, Inc.

Square's Machine Learning Infrastructure and Applications - Rong Yan

Hakka Labs

PostgreSQL is an open source relational database. Kafka is an open source log-based messaging system. Because both systems are powerful and flexible, they’re devouring whole categories of infrastructure. And they’re even better together. In this talk, you’ll learn about commit logs and how that fundamental data structure underlies both PostgreSQL and Kafka. We’ll use that basis to understand what Kafka is, what advantages it has over traditional messaging systems, and why it’s perfect for modeling database tables as streams. From there, we’ll introduce the concept of change data capture (CDC) and run a live demo of Bottled Water, an open source CDC pipeline, watching INSERT, UPDATE, and DELETE operations in PostgreSQL stream into Kafka. We’ll wrap up with a discussion of use cases for this pipeline: messaging between systems with transactional guarantees, transmitting database changes to a data warehouse, and stream processing.

PostgreSQL + Kafka: The Delight of Change Data Capture

Jeff Klukas

The machine learning libraries in Apache Spark are an impressive piece of software engineering, and are maturing rapidly. What advantages does Spark.ml offer over scikit-learn? At Data Science Retreat we've taken a real-world dataset and worked through the stages of building a predictive model -- exploration, data cleaning, feature engineering, and model fitting; which would you use in production? The machine learning libraries in Apache Spark are an impressive piece of software engineering, and are maturing rapidly. What advantages does Spark.ml offer over scikit-learn? At Data Science Retreat we've taken a real-world dataset and worked through the stages of building a predictive model -- exploration, data cleaning, feature engineering, and model fitting -- in several different frameworks. We'll show what it's like to work with native Spark.ml, and compare it to scikit-learn along several dimensions: ease of use, productivity, feature set, and performance. In some ways Spark.ml is still rather immature, but it also conveys new superpowers to those who know how to use it.

A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...

Jose Quesada (hiring)

Machine Learning In Production

Samir Bessalah

Machine Learning Pipelines

jeykottalam

What is machine learning? Is UX relevant in the age of artificial intelligence (AI)? How can I take advantage of cognitive computing? Get answers to these questions and learn about the implications for your work in this session. Carol will help you understand at a basic level how these systems are built and what is required to get insights from them. Carol will present examples of how machine learning is already being used and explore the ethical challenges inherent in creating AI. You will walk away with an awareness of the weaknesses of AI and the knowledge of how these systems work.

AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017

Carol Smith

En vedette (14)

Machine learning in production with scikit-learn

Python as part of a production machine learning stack by Michael Manapat PyDa...

Building A Production-Level Machine Learning Pipeline

Managing and Versioning Machine Learning Models in Python

Using PySpark to Process Boat Loads of Data

Production machine learning_infrastructure

Production and Beyond: Deploying and Managing Machine Learning Models

Machine learning in production

Square's Machine Learning Infrastructure and Applications - Rong Yan

PostgreSQL + Kafka: The Delight of Change Data Capture

A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...

Machine Learning In Production

Machine Learning Pipelines

AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017

Similaire à Multi runtime serving pipelines for machine learning

GeeCON Microservices 2015 scaling micro services at gilt

Adrian Trenaman

AWS re:Invent is an annual global conference of the Amazon Web Services community held in Las Vegas. In 2017, we held 1000+ breakout sessions and attracted over 40,000 attendees. The event offers expanded opportunities to learn about the latest AWS releases, use cases and business benefits, not to mention diving deep into hot topics and meeting with our subject matter experts. Missed it? Don’t worry, we are bringing AWS re:Invent to Hong Kong on Jan 18, 2018. Packed in a day, AWS re:Invent 2017 Recap Hong Kong will showcase new releases announced at re:Invent 2017 on Serverless & Container, DevOps & Mobile, Artificial Intelligence & Machine Learning and more. Local customers will also be invited to share their re:Invent experience and success stories with AWS. Discover the latest services and features from Amazon Web Services and learn how to integrate them into your applications

AWS re:Invent 2017 Recap - Solutions Updates

Amazon Web Services

DF1 - ML - Petukhov - Azure Ml Machine Learning as a Service

MoscowDataFest

Machine learning is not black magic but a discipline that involves statistics, data science, analysis and hard work. From searching patterns and data preparation through applying and optimizing algorithms to obtaining usable predictions, one would need background and appropriate tools. But do we need it, when there is already available AI as a service solution out there? Do we need to try hard with artificial neural networks? And if we decide to do so, what tools would be a safe bet? In this session we will go through real world examples, mention key tools from Microsoft and open source world to do data science and machine learning and most importantly - we will provide a workflow and some best practices.

The Data Science Process - Do we need it and how to apply?

Ivo Andreev

Apache Deltacloud (Linuxcon 2010)

lutter

Evolving the Network Automation Journey from Python to Platforms

Network Automation Forum

TensorFlow meetup: Keras - Pytorch - TensorFlow.js

Stijn Decubber

by Mahendra Bairagi, AI Specialist Solutions Architect, AWS As the CTO of a new startup, you have taken up a challenge of improving the EDM music festival experience. At venues with multiple stages, festival-goers are always looking to identify DJ stage areas with the liveliest atmosphere. This causes them to constantly move around between different stages and miss out on having fun. You are looking to use Machine Learning and IoT technologies to solve this unique problem. Do you accept the Challenge? The objective of this task is to help the festival-goers quickly identify the DJ stage where crowd is the happiest. You've seen a lot of buzz around computer vision, machine learning, and IoT and want to use this technology to detect crowd emotions. From your initial research there are existing ML models that you can leverage to do face and emotion detection, but there are two ways that the predictions (inference) can be done; on the cloud and on the camera itself, but which one will work the best for your needs at the festival? You are going to test both approaches and find out! In this workshop you will use AWS and Intel technologies to learn how to build, deploy, and run ML inference on the cloud as well as on the IoT Edge. You will learn to use Amazon SageMaker with Intel C5 Instances, AWS DeepLens, AWS Greengrass, Amazon Rekognition, and AWS Lambda to build an end-to-end IoT solution that performs machine learning.

DataPalooza - A Music Festival themed ML + IoT Workshop

Amazon Web Services

Scala services in action

Underscore

Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...

Jen Aman

Microsoft & Machine Learning / Artificial Intelligence

İbrahim KIVANÇ

Automated machine learning (automated ML) automates feature engineering, algorithm and hyperparameter selection to find the best model for your data. The mission: Enable automated building of machine learning with the goal of accelerating, democratizing and scaling AI. This presentation covers some recent announcements of technologies related to Automated ML, and especially for Azure. The demonstrations focus on Python with Azure ML Service and Azure Databricks.

201908 Overview of Automated ML

Mark Tabladillo

Camel Day Italia 2021 - Camel K

Nicola Ferraro

Borys Rybak “How to make your data smart with Artificial Intelligence and Mac...

Lviv Startup Club

Matrix.org decentralised communication, Matthew Hodgson, TADSummit

Alan Quayle

ML.NET 1.0 release is the first major milestone of a great journey that started in May 2018 when we released ML.NET 0.1 as open source. ML.NET is an open-source and cross-platform machine learning framework for .NET developers. Using ML.NET, developers can leverage their existing tools and skillsets to develop and infuse custom AI into their applications by creating custom machine learning models for common scenarios like Sentiment Analysis, Recommendation, Image Classification and more. “Automated ML” is a collection of new technologies from Microsoft to enhance the data science development process. Still in preview, Auto ML for ML.NET 1.0 will be demonstrated in a Deep Learning Virtual Machine running Windows Server 2016. Code examples are in C# and run in Visual Studio Community 2019. This presentation is the second of four related to ML.NET and Automated ML. The presentation will be recorded with video posted to this YouTube Channel: http://bit.ly/2ZybKwI

201906 02 Introduction to AutoML with ML.NET 1.0

Mark Tabladillo

Model Inferencing use cases are becoming a requirement for models moving into the next phase of production deployments. More and more users are now encountering use cases around canary deployments, scale-to-zero or serverless characteristics. And then there are also advanced use cases coming around model explainability, including A/B tests, ensemble models, multi-armed bandits, etc. In this talk, the speakers are going to detail how to handle these use cases using Kubeflow Serving and the native Kubernetes stack which is Istio and Knative. Knative and Istio help with autoscaling, scale-to-zero, canary deployments to be implemented, and scenarios where traffic is optimized to the best performing models. This can be combined with KNative eventing, Istio observability stack, KFServing Transformer to handle pre/post-processing and payload logging which consequentially can enable drift and outlier detection to be deployed. We will demonstrate where currently KFServing is, and where it's heading towards.

Advanced Model Inferencing leveraging Kubeflow Serving, KNative and Istio

Animesh Singh

Machine learning and Deep learning on edge devices using TensorFlow

Aditya Bhattacharya

This is Part 4 of the GoldenGate series on Data Mesh - a series of webinars helping customers understand how to move off of old-fashioned monolithic data integration architecture and get ready for more agile, cost-effective, event-driven solutions. The Data Mesh is a kind of Data Fabric that emphasizes business-led data products running on event-driven streaming architectures, serverless, and microservices based platforms. These emerging solutions are essential for enterprises that run data-driven services on multi-cloud, multi-vendor ecosystems. Join this session to get a fresh look at Data Mesh; we'll start with core architecture principles (vendor agnostic) and transition into detailed examples of how Oracle's GoldenGate platform is providing capabilities today. We will discuss essential technical characteristics of a Data Mesh solution, and the benefits that business owners can expect by moving IT in this direction. For more background on Data Mesh, Part 1, 2, and 3 are on the GoldenGate YouTube channel: https://www.youtube.com/playlist?list=PLbqmhpwYrlZJ-583p3KQGDAd6038i1ywe Webinar Speaker: Jeff Pollock, VP Product (https://www.linkedin.com/in/jtpollock/) Mr. Pollock is an expert technology leader for data platforms, big data, data integration and governance. Jeff has been CTO at California startups and a senior exec at Fortune 100 tech vendors. He is currently Oracle VP of Products and Cloud Services for Data Replication, Streaming Data and Database Migrations. While at IBM, he was head of all Information Integration, Replication and Governance products, and previously Jeff was an independent architect for US Defense Department, VP of Technology at Cerebra and CTO of Modulant – he has been engineering artificial intelligence based data platforms since 2001. As a business consultant, Mr. Pollock was a Head Architect at Ernst & Young’s Center for Technology Enablement. Jeff is also the author of “Semantic Web for Dummies” and "Adaptive Information,” a frequent keynote at industry conferences, author for books and industry journals, formerly a contributing member of W3C and OASIS, and an engineering instructor with UC Berkeley’s Extension for object-oriented systems, software development process and enterprise architecture.

Data Mesh Part 4 Monolith to Mesh

Jeffrey T. Pollock

ARISE

kaptoxic

Similaire à Multi runtime serving pipelines for machine learning (20)

GeeCON Microservices 2015 scaling micro services at gilt

AWS re:Invent 2017 Recap - Solutions Updates

DF1 - ML - Petukhov - Azure Ml Machine Learning as a Service

The Data Science Process - Do we need it and how to apply?

Apache Deltacloud (Linuxcon 2010)

Evolving the Network Automation Journey from Python to Platforms

TensorFlow meetup: Keras - Pytorch - TensorFlow.js

DataPalooza - A Music Festival themed ML + IoT Workshop

Scala services in action

Embrace Sparsity At Web Scale: Apache Spark MLlib Algorithms Optimization For...

Microsoft & Machine Learning / Artificial Intelligence

201908 Overview of Automated ML

Camel Day Italia 2021 - Camel K

Borys Rybak “How to make your data smart with Artificial Intelligence and Mac...

Matrix.org decentralised communication, Matthew Hodgson, TADSummit

201906 02 Introduction to AutoML with ML.NET 1.0

Advanced Model Inferencing leveraging Kubeflow Serving, KNative and Istio

Machine learning and Deep learning on edge devices using TensorFlow

Data Mesh Part 4 Monolith to Mesh

ARISE

Dernier

InShot proinshot.com stands tall among its peers as the ultimate video editing app, offering simplicity, versatility, and power in one package. With its intuitive interface and comprehensive feature set, InShot caters to both beginners and seasoned editors alike. Whether you're creating content for social media, YouTube, or personal projects, InShot empowers you to unleash your creativity and transform your videos into captivating masterpieces. Join the millions of users who trust InShot https://www.proinshot.com/ for all their video editing needs and discover the difference for yourself!

Exploring the Best Video Editing App.pdf

proinshot.com

%in ivory park+277-882-255-28 abortion pills for sale in ivory park

masabamasaba

10 Trends Likely to Shape Enterprise Technology in 2024

Mind IT Systems

We specialize in Psychic Readings, Psychic Love Spells, Binding Love Spells, Obsession Spells, Voodoo Spells, Lottery Spells, Marriage Spells, Black Magic Spells, Palm Readings & much more. Are you depressed? We perform this come-to-me love spell that works instantly with the aim of bringing back the victim to the person performing the magic. Have you lost your lover? We perform this come-to-me love spell that works instantly with the aim of bringing back the victim to the person performing the magic. Have you lost your lover? Do u need to solve any relationship problem? Contact the powerful spells caster chief kule with love spells that work overnight and love spells that really work. Have you found yourself infatuated with a special someone you think could be the one? Are you looking for a spell to provide them with a nudge in the right direction? Or maybe the spell you cast didn’t achieve the results you were hoping for? Whether you’re new or versed in the ways of spell casting, we’re here to help. Today we’re going to provide you with a detailed guide on the types of love spells to cast. Not only that but there’s something for those who wish to find outside advice from more advanced spell casters. We’re also going to provide you with the top sites available to help you with your dilemma. Let’s begin our journey by educating ourselves on love magic and what a real love caster looks like. Love Magic and Love Casters Love magic made its first appearance back in Ancient Egypt and has been an active practice since. This type of magic is a branch of traditional magic and can be practiced in various ways. Typically the more common use of love magic is through the work of spells, but other methods look like Charms Rituals-LOVE Potions-Dolls and even Amulets If you are interested in becoming a love caster, be prepared for what’s to come. A genuine love caster knows that the art of love casting is no easy feat and shouldn’t be done casually. You should know that not only does it require you to be gifted spiritually, but you must be ready to serve others. Someone who is considered a real love caster has experience in all manner of spells, no matter the difficulty. Training yourself in attraction, commitment, and marriage spells is an excellent place to start. But this by no means will make you a professional. Practice your craft and expand your knowledge; understand that you will possess the ability to help others in time truly. Types of Love Spells What better way to start broadening your experiences with love spells than by learning more about them? These spells work like just about any other spell. Simply apply your intention, use a medium (sigils, mantras, candles, or charm bags), and top it off with establishing the belief that you will receive what you want. So what kind of spells are available and which ones suit your needs the best? Let’s take a look at the many options you have at your disposal.

%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...

masabamasaba

+971565801893 Mtp-Kit (500MG) Prices » Dubai [(+971565801893**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Leen Whatsapp +971565801893 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971565801893''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971565801893' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Clinic in Abu Dhabi, United Arab Emirates.+971565801893

+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...

Health

%in Midrand+277-882-255-28 abortion pills for sale in midrand

masabamasaba

Investing in AI transformation today The modern business advantage: Uncovering deep insights with AI Organizations around the world have come to recognize AI as the transformative technology that enables them to gain real business advantage. AI’s ability to organize vast quantities of data allows those who implement it to uncover deep business insights, augment human expertise, drive operational efficiency, transform their products, and better serve their customers

Microsoft AI Transformation Partner Playbook.pdf

Willy Marroquin (WillyDevNET)

Data spaces in distributed environments should be allowed to evolve in agile ways providing data space owners with large flexibility about which data they store. Agility and heterogeneity, however, jeopardize data exchanges because representations may build on varying ontologies and data consumers may not rely on the semantic correctness of their queries in the context of semantically heterogeneous, evolving data spaces. Graph data spaces are one example of a powerful model for representing and querying data whose semantics may change over time. To assert and enforce conditions on individual graph data spaces, shape languages (e.g SHACL) have been developed. We investigate the question of how querying and programming can be guarded by reasoning over SHACL constraints in a distributed setting and we sketch a picture of how a future landscape based on semantically heterogeneous data spaces might look like.

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Steffen Staab

Software Quality Assurance Interview Questions

Arshad QA

Craft an AI & Machine Learning Pitch with our Editable Professional PowerPoint Template. Ignite your AI & Machine Learning pitch with our cutting-edge PowerPoint template tailored for the industry. Perfect for AI conferences, investor presentations, sales pitches to tech-focused companies, training sessions, and educational programs. - 20+ editable slides: Get a variety of options to choose from for your presentation. - Time-saving solution: Download, replace text/images with a few clicks. - User-friendly customization: Easy to use and personalize. - Modern and attractive design: Captivating visuals, sleek layout. - Tailored to your requirements: Fully alterable for customization. - Well-organized slides: Complete control over content. - Thematic specificity: Reflects healthcare industry with relevant graphics. - Showcase your business idea: Communicate value proposition effectively.

AI & Machine Learning Presentation Template

Presentation.STUDIO

%in Durban+277-882-255-28 abortion pills for sale in Durban

masabamasaba

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein

masabamasaba

Looking for an efficient way to manage your finances? Look no further than our money management app. With easy-to-use features, you can track your expenses, create budgets, and monitor your savings goals all in one place. Our app provides real-time updates on your spending habits and helps you make smarter financial decisions. Take control of your finances today with our user-friendly money management app.

Right Money Management App For Your Financial Goals

Jhone kinadey

At the recent Microsoft Ignite 2023 conference, Microsoft unveiled a groundbreaking strategy that will redefine enterprise work management. The plan involves integrating Microsoft’s key planning tools, Microsoft To Do, Microsoft Planner, and Microsoft Project for the web into a unified experience called “Microsoft Planner.” What does this new strategy from Microsoft mean for current users? Join us and learn how best to take advantage of this announcement while gaining a clear path on how to elevate the current state of Microsoft Planner from a basic task manager to a comprehensive tool for Enterprise Work Management using OnePlan. Learn how OnePlan’s integration with Microsoft Planner allows for strategic alignment with business goals through advanced features like strategic planning, portfolio management, resource management, financial management, and more!

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

OnePlan Solutions

%in tembisa+277-882-255-28 abortion pills for sale in tembisa

masabamasaba

Generic or specific? Making sensible software design decisions

Bert Jan Schrijver

%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...

masabamasaba

In today's dynamic e-commerce landscape, the payment gateway emerges as a linchpin, ensuring smooth and secure transactions between buyers and sellers. In this discourse, we delve into the meticulous process of devising test cases tailored for scrutinizing payment gateways. Crafting precise test cases for payment gateways is a quintessential responsibility for testers operating within the service industry. This article meticulously explores pivotal scenarios integral to how to test payment gateways, coupled with essential guidelines for drafting effective test cases.

Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf

kalichargn70th171

Abortion Clinic And Abortion Pills For Sale in Oman MEDICAL ABORTION PILLS FOR SALE in Fujairah , Ajman, Al Ain Quality Abortion services provided by specialists. Women's health problems. A same day service, safe, legal & pain free Abortions. From 1 week to 5 months. We use tested and approved pills. It's 100% guaranteed & safe. No side effects at all. We also do free womb cleaning and free pills delivery. We sell pregnancy termination pills {ABORTION PILLS} Our service is ever private with all our clients. Call/WhatsApp:

%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...

masabamasaba

Foundation models are machine learning models which are easily capable of performing variable tasks on large and huge datasets. FMs have managed to get a lot of attention due to this feature of handling large datasets. It can do text generation, video editing to protein folding and robotics. In case we believe that FMs can help the hospitals and patients in any way, we need to perform some important evaluations, tests to test these assumptions. In this review, we take a walk through Fms and their evaluation regimes assumed clinical value. To clarify on this topic, we reviewed no less than 80 clinical FMs built from the EMR data. We added all the models trained on structured and unstructured data. We are referring to this combination of structured and unstructured EMR data or clinical data.

Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...

harshavardhanraghave

Dernier (20)

Exploring the Best Video Editing App.pdf

%in ivory park+277-882-255-28 abortion pills for sale in ivory park

10 Trends Likely to Shape Enterprise Technology in 2024

%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...

+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...

%in Midrand+277-882-255-28 abortion pills for sale in midrand

Microsoft AI Transformation Partner Playbook.pdf

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Software Quality Assurance Interview Questions

AI & Machine Learning Presentation Template

%in Durban+277-882-255-28 abortion pills for sale in Durban

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein

Right Money Management App For Your Financial Goals

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

%in tembisa+277-882-255-28 abortion pills for sale in tembisa

Generic or specific? Making sensible software design decisions

%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...

Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf

%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...

Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...

Multi runtime serving pipelines for machine learning

1. Multi-Runtime Serving Pipelines Stepan Pushkarev CTO of Hydrosphere.io

2. Mission: Accelerate Machine Learning to Production Opensource Products: - Mist: Serverless proxy for Spark - ML Lambda: ML Function as a Service - Sonar: Data and ML Monitoring Business Model: Subscription services and hands-on consulting About

3. Deployment | Serving | Scoring | Inference @Nvidia https://www.nvidia.com/en-us/deep-learning-ai/solutions/

4. From Single Model to Meta Pipelines

5. Item 1 Item 2 Title Authentic HERMES Bijouterie Fantaisie Selle Clip-On Earrings Silvertone #S1742 E Auth HERMES Earrings Sellier Clip-on Silver Tone Round $0 Ship 25130490900 S06B Specs Brand: HERMES Size(cm): W1.8 x H1.8 cm(Approx) Color: Silver Size(inch): W0.7 x H0.7" (Approx) Style: Earrings Rank: B Brand: Hermes Fastening: Clip-On Style: Clip on Country/Region of Manufacture: Unknown Metal: Silver Plated Main Color:Silver Color: Silver Description ... ... Does this pair describe the same thing? Product Matching

6. Model Artifact: Ops perspective

7. - HTTP/1.1, HTTP/2, gRPC - Kafka, Flink, Kinesis - Protobuf, Avro - Service Discovery - Pipelining - Tracing - Monitoring - Autoscaling - Versioning - A/B, Canary - Testing - CPU, GPU API & Logistics

8. Monitoring Shifting experimentation to production

9. Sidecar Architecture

10. Functions registry responsible for the model life cycle and all the business logic required to configure models for serving Mesh of serving runtimes is an actual serving cluster Infrastructure integration: ECS for AWS, Kubernetes for GCE and on premise

11. UX: Models and Applications Applications provide public virtual endpoints for the models and compositions of the models.

12. Why Not just one Big Neural Network? ● Not always possible ● Stages could be independent ● Ad-hoc rule based models ● Physics models (e.g. LIDAR) ● Big E2E DL Requires Black Magic skills

13. Why Not just one Python script? ● Modularity. Stages could be developed by different teams ● Traceability and Monitoring ● Versioning ● Independent deployment, A/B testing and Canary ● Request Shadowing and other cool stuff ● Could require different ML runtimes (TF, Scikit, Spark ML, etc) ● We need more microservices :)

14. Why Not just TF Serving? ● Other ML runtimes (DL4J, Scikit, Spark ML). Servables are overkill. ● Need better versioning and immutability (Docker per version) ● Don’t want to deal with state (model loaded, offloaded, etc) ● Want to re-use microservices stack (tracing, logging, metrics) ● Need better scalability

15. Demo

16. Thank you - @hydrospheredata - https://github.com/Hydrospheredata - https://hydrosphere.io/ - spushkarev@hydrosphere.io

Multi runtime serving pipelines for machine learning

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (14)

Similaire à Multi runtime serving pipelines for machine learning

Similaire à Multi runtime serving pipelines for machine learning (20)

Dernier

Dernier (20)

Multi runtime serving pipelines for machine learning