SlideShare une entreprise Scribd logo
1  sur  33
Télécharger pour lire hors ligne
OpenShift as a cloud for Data
Science
Yegor Maksymchuk, Soft Serve, Ukraine
Agenda
• Kubernetes
• Openshift
• Kubernetes vs Openshift
• Apache Spark
• Radanalitics and OSHINKO
whoami
Yegor Maksymchuk
Software engineer,
Soft Serve Ukraine
Telegram: @QAStudy.online
GitHub: YegorMaksymchuk
LinkedIn:ymaksymchuk
Problems
Integration with Apache Spark in the Openshift.
Usability on UI and API level, it should be easy to use.
OSHINKO
Data Science in the Cloud
Kubernetes
Kubernetes:POD
apiVersion: v1
kind: Pod
metadata:
name: pod-demo
labels:
name: pod-demo
spec:
containers:
- name: pod-demo
image: yemax/pod-
demo:1
ports:
- containerPort: 8081
Kubernetes:Namespace
{
"kind": "Namespace",
"apiVersion": "v1",
"metadata": {
"name":
"development",
"labels": {
"name":
"development"
}
}
}
Kubernetes: Replica Sets
K8s: Deployment
Kubernetes: Ingress
K8s: Architecture
Openshift
Openshift: Deployment
Openshift: S2I
s2i-lighttpd/
● Dockerfile – This is a standard Dockerfile where we’ll define the builderimage
● Makefile – a helper script for buildingand testing the builderimage
● test/
○ run – test script, testing if the builder image works correctly
○ test-app/ – directory for your test application
● .s2i/bin/
○ assemble – script responsible for buildingthe application
○ run – script responsible for running the application
○ save-artifacts – script responsible for incremental builds, covered in a
future article
○ usage – script responsible for printing the usage of the builderimage
Openshift vs Kubernetes
K8s:
Orchestration tool
Ingress based on “Ngnix”
Namespace not “secure”
Openshift:
Platform as a Service
Routes based on HAProxy
Namespace “secure”, and more
understandable.
S2I
Builds new images, after push new
source.
Pool of prepared images
Data Science use Spark
Data Science in the Cloud
Apache Spark
Spark on OpenShift
OSHINKO: S2I
OSHINKO: Spark integrator
DEMO
DEMO
1. oc cluster up
2. oc new-project devops-stage-demo
3. oc create -f https://radanalytics.io/resources.yaml
4. oc create -f https://radanalytics.io/assets/zeppelin-example/zeppelin-openshift.yaml
5. oc new-app oshinko-webui
6. oshinko create devops-spark-cluster
7. oshinko get devops-spark-cluster
8. oc new-app --template=$namespace/apache-zeppelin-openshift 
--param=APPLICATION_NAME=apache-zeppelin 
--param=GIT_URI=https://github.com/rimolive/zeppelin-notebooks.git 
--param=ZEPPELIN_INTERPRETERS=md
Questions ?
yegor maksymchuk - open shift as a cloud for data science

Contenu connexe

Tendances

Tendances (19)

Serverless on Kubernetes
Serverless on KubernetesServerless on Kubernetes
Serverless on Kubernetes
 
Kubernetes @ Nanit by Chen Fisher
Kubernetes @ Nanit by Chen FisherKubernetes @ Nanit by Chen Fisher
Kubernetes @ Nanit by Chen Fisher
 
The Challenges of Becoming Cloud Native
The Challenges of Becoming Cloud NativeThe Challenges of Becoming Cloud Native
The Challenges of Becoming Cloud Native
 
Windows containers on Kubernetes
Windows containers on KubernetesWindows containers on Kubernetes
Windows containers on Kubernetes
 
Zero downtime deployment of micro-services with Kubernetes
Zero downtime deployment of micro-services with KubernetesZero downtime deployment of micro-services with Kubernetes
Zero downtime deployment of micro-services with Kubernetes
 
Monitoring mayhem - Using Prometheus
Monitoring mayhem - Using PrometheusMonitoring mayhem - Using Prometheus
Monitoring mayhem - Using Prometheus
 
DCSF19 Kubernetes Security with OPA
DCSF19 Kubernetes Security with OPA DCSF19 Kubernetes Security with OPA
DCSF19 Kubernetes Security with OPA
 
DevFestMN 2017 - Learning Docker and Kubernetes with Openshift
DevFestMN 2017 - Learning Docker and Kubernetes with OpenshiftDevFestMN 2017 - Learning Docker and Kubernetes with Openshift
DevFestMN 2017 - Learning Docker and Kubernetes with Openshift
 
5 Habits of High-Velocity Teams Using Kubernetes
5 Habits of High-Velocity Teams Using Kubernetes5 Habits of High-Velocity Teams Using Kubernetes
5 Habits of High-Velocity Teams Using Kubernetes
 
Building a raspberry pi cluster
Building a raspberry pi clusterBuilding a raspberry pi cluster
Building a raspberry pi cluster
 
利用K8S實現高可靠應用
利用K8S實現高可靠應用利用K8S實現高可靠應用
利用K8S實現高可靠應用
 
Intro to Kubernetes Workshop — Innovation Labs July 2020
Intro to Kubernetes Workshop — Innovation Labs July 2020Intro to Kubernetes Workshop — Innovation Labs July 2020
Intro to Kubernetes Workshop — Innovation Labs July 2020
 
Skynet vs. Planet of The Apes: Duel!
Skynet vs. Planet of The Apes: Duel!Skynet vs. Planet of The Apes: Duel!
Skynet vs. Planet of The Apes: Duel!
 
Using Docker in Production
Using Docker in ProductionUsing Docker in Production
Using Docker in Production
 
Function as a Service
Function as a ServiceFunction as a Service
Function as a Service
 
Kubernetes on IBM Cloud + DevOps コンテナCIで簡易アプリ作ってみた
Kubernetes on IBM Cloud + DevOps コンテナCIで簡易アプリ作ってみたKubernetes on IBM Cloud + DevOps コンテナCIで簡易アプリ作ってみた
Kubernetes on IBM Cloud + DevOps コンテナCIで簡易アプリ作ってみた
 
CI and CD with Spinnaker
CI and CD with SpinnakerCI and CD with Spinnaker
CI and CD with Spinnaker
 
AWS Community Day Bangkok 2019 - DevOps Cost Reduction using Jenkins & AWS Sp...
AWS Community Day Bangkok 2019 - DevOps Cost Reduction using Jenkins & AWS Sp...AWS Community Day Bangkok 2019 - DevOps Cost Reduction using Jenkins & AWS Sp...
AWS Community Day Bangkok 2019 - DevOps Cost Reduction using Jenkins & AWS Sp...
 
Profile your Java apps in production on Red Hat OpenShift with Cryostat | Dev...
Profile your Java apps in production on Red Hat OpenShift with Cryostat | Dev...Profile your Java apps in production on Red Hat OpenShift with Cryostat | Dev...
Profile your Java apps in production on Red Hat OpenShift with Cryostat | Dev...
 

Similaire à yegor maksymchuk - open shift as a cloud for data science

Introducing Kafka's Streams API
Introducing Kafka's Streams APIIntroducing Kafka's Streams API
Introducing Kafka's Streams API
confluent
 

Similaire à yegor maksymchuk - open shift as a cloud for data science (20)

Apache Spark Streaming in K8s with ArgoCD & Spark Operator
Apache Spark Streaming in K8s with ArgoCD & Spark OperatorApache Spark Streaming in K8s with ArgoCD & Spark Operator
Apache Spark Streaming in K8s with ArgoCD & Spark Operator
 
OpenShift Enterprise 3.1 vs kubernetes
OpenShift Enterprise 3.1 vs kubernetesOpenShift Enterprise 3.1 vs kubernetes
OpenShift Enterprise 3.1 vs kubernetes
 
[DevDay 2017] OpenShift Enterprise - Speaker: Linh Do - DevOps Engineer at Ax...
[DevDay 2017] OpenShift Enterprise - Speaker: Linh Do - DevOps Engineer at Ax...[DevDay 2017] OpenShift Enterprise - Speaker: Linh Do - DevOps Engineer at Ax...
[DevDay 2017] OpenShift Enterprise - Speaker: Linh Do - DevOps Engineer at Ax...
 
Get you Java application ready for Kubernetes !
Get you Java application ready for Kubernetes !Get you Java application ready for Kubernetes !
Get you Java application ready for Kubernetes !
 
Docker Container As A Service - JAX 2016
Docker Container As A Service - JAX 2016Docker Container As A Service - JAX 2016
Docker Container As A Service - JAX 2016
 
0507 057 01 98 * Adana Klima Tamir Servisi
0507 057 01 98 * Adana Klima Tamir Servisi0507 057 01 98 * Adana Klima Tamir Servisi
0507 057 01 98 * Adana Klima Tamir Servisi
 
Openstack Quantum
Openstack  QuantumOpenstack  Quantum
Openstack Quantum
 
Docker Platform and Ecosystem Nov 2015
Docker Platform and Ecosystem Nov 2015Docker Platform and Ecosystem Nov 2015
Docker Platform and Ecosystem Nov 2015
 
Kubernetes for java developers - Tutorial at Oracle Code One 2018
Kubernetes for java developers - Tutorial at Oracle Code One 2018Kubernetes for java developers - Tutorial at Oracle Code One 2018
Kubernetes for java developers - Tutorial at Oracle Code One 2018
 
Be a microservices hero
Be a microservices heroBe a microservices hero
Be a microservices hero
 
Open shift enterprise 3.1 paas on kubernetes
Open shift enterprise 3.1   paas on kubernetesOpen shift enterprise 3.1   paas on kubernetes
Open shift enterprise 3.1 paas on kubernetes
 
OpenFaaS KubeCon Zero to Serverless in 60 seconds anywhere
OpenFaaS KubeCon Zero to Serverless in 60 seconds anywhereOpenFaaS KubeCon Zero to Serverless in 60 seconds anywhere
OpenFaaS KubeCon Zero to Serverless in 60 seconds anywhere
 
Kubernetes meetup geneva june 2021
Kubernetes meetup geneva   june 2021Kubernetes meetup geneva   june 2021
Kubernetes meetup geneva june 2021
 
Kubernetes Overview - Deploy your app with confidence
Kubernetes Overview - Deploy your app with confidenceKubernetes Overview - Deploy your app with confidence
Kubernetes Overview - Deploy your app with confidence
 
A 60-mn tour of AWS compute (March 2016)
A 60-mn tour of AWS compute (March 2016)A 60-mn tour of AWS compute (March 2016)
A 60-mn tour of AWS compute (March 2016)
 
Containers as a Service with Docker
Containers as a Service with DockerContainers as a Service with Docker
Containers as a Service with Docker
 
Docker Container As A Service - March 2016
Docker Container As A Service - March 2016Docker Container As A Service - March 2016
Docker Container As A Service - March 2016
 
Introducing Kafka's Streams API
Introducing Kafka's Streams APIIntroducing Kafka's Streams API
Introducing Kafka's Streams API
 
Kubernetes extensibility
Kubernetes extensibilityKubernetes extensibility
Kubernetes extensibility
 
Kubernetes for Java Developers
Kubernetes for Java DevelopersKubernetes for Java Developers
Kubernetes for Java Developers
 

Plus de Dariia Seimova

Plus de Dariia Seimova (10)

Chris Homer - Moving the entire stack to k8s within a year – lessons learned
Chris Homer - Moving the entire stack to k8s within a year – lessons learnedChris Homer - Moving the entire stack to k8s within a year – lessons learned
Chris Homer - Moving the entire stack to k8s within a year – lessons learned
 
juliya tkachova - dev ops on scale from philosophy to toolset
juliya tkachova - dev ops on scale from philosophy to toolsetjuliya tkachova - dev ops on scale from philosophy to toolset
juliya tkachova - dev ops on scale from philosophy to toolset
 
rohit sharma - dev ops virtual assistant - automate devops stuffs using nlp a...
rohit sharma - dev ops virtual assistant - automate devops stuffs using nlp a...rohit sharma - dev ops virtual assistant - automate devops stuffs using nlp a...
rohit sharma - dev ops virtual assistant - automate devops stuffs using nlp a...
 
ostap soroka - aws architecture and a human body
ostap soroka - aws architecture and a human bodyostap soroka - aws architecture and a human body
ostap soroka - aws architecture and a human body
 
victoriia basarab - special aspects of dev ops platform development
victoriia basarab - special aspects of dev ops platform developmentvictoriia basarab - special aspects of dev ops platform development
victoriia basarab - special aspects of dev ops platform development
 
sveta smirnova - my sql performance schema in action
sveta smirnova - my sql performance schema in actionsveta smirnova - my sql performance schema in action
sveta smirnova - my sql performance schema in action
 
faisal mushtaq - an enterprise cloud cost management framework
faisal mushtaq - an enterprise cloud cost management frameworkfaisal mushtaq - an enterprise cloud cost management framework
faisal mushtaq - an enterprise cloud cost management framework
 
mykola marzhan - jenkins on aws spot instance
mykola marzhan - jenkins on aws spot instancemykola marzhan - jenkins on aws spot instance
mykola marzhan - jenkins on aws spot instance
 
maksym vlasov - culture of git as roots of your ci
maksym vlasov - culture of git as roots of your cimaksym vlasov - culture of git as roots of your ci
maksym vlasov - culture of git as roots of your ci
 
vitaly davidoff - end 2 end containers secure sdlc process
vitaly davidoff - end 2 end containers secure sdlc processvitaly davidoff - end 2 end containers secure sdlc process
vitaly davidoff - end 2 end containers secure sdlc process
 

Dernier

Gardella_Mateo_IntellectualProperty.pdf.
Gardella_Mateo_IntellectualProperty.pdf.Gardella_Mateo_IntellectualProperty.pdf.
Gardella_Mateo_IntellectualProperty.pdf.
MateoGardella
 
Seal of Good Local Governance (SGLG) 2024Final.pptx
Seal of Good Local Governance (SGLG) 2024Final.pptxSeal of Good Local Governance (SGLG) 2024Final.pptx
Seal of Good Local Governance (SGLG) 2024Final.pptx
negromaestrong
 
Making and Justifying Mathematical Decisions.pdf
Making and Justifying Mathematical Decisions.pdfMaking and Justifying Mathematical Decisions.pdf
Making and Justifying Mathematical Decisions.pdf
Chris Hunter
 
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in DelhiRussian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
kauryashika82
 

Dernier (20)

How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot Graph
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 
Mixin Classes in Odoo 17 How to Extend Models Using Mixin Classes
Mixin Classes in Odoo 17  How to Extend Models Using Mixin ClassesMixin Classes in Odoo 17  How to Extend Models Using Mixin Classes
Mixin Classes in Odoo 17 How to Extend Models Using Mixin Classes
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.ppt
 
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
SECOND SEMESTER TOPIC COVERAGE SY 2023-2024 Trends, Networks, and Critical Th...
 
Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024
 
Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdf
 
Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
Ecological Succession. ( ECOSYSTEM, B. Pharmacy, 1st Year, Sem-II, Environmen...
 
Gardella_Mateo_IntellectualProperty.pdf.
Gardella_Mateo_IntellectualProperty.pdf.Gardella_Mateo_IntellectualProperty.pdf.
Gardella_Mateo_IntellectualProperty.pdf.
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activity
 
Seal of Good Local Governance (SGLG) 2024Final.pptx
Seal of Good Local Governance (SGLG) 2024Final.pptxSeal of Good Local Governance (SGLG) 2024Final.pptx
Seal of Good Local Governance (SGLG) 2024Final.pptx
 
Making and Justifying Mathematical Decisions.pdf
Making and Justifying Mathematical Decisions.pdfMaking and Justifying Mathematical Decisions.pdf
Making and Justifying Mathematical Decisions.pdf
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in DelhiRussian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
 
Unit-IV; Professional Sales Representative (PSR).pptx
Unit-IV; Professional Sales Representative (PSR).pptxUnit-IV; Professional Sales Representative (PSR).pptx
Unit-IV; Professional Sales Representative (PSR).pptx
 
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
PROCESS RECORDING FORMAT.docx
PROCESS      RECORDING        FORMAT.docxPROCESS      RECORDING        FORMAT.docx
PROCESS RECORDING FORMAT.docx
 

yegor maksymchuk - open shift as a cloud for data science