Containerising the Mule Runtime with Kubernetes & From Zero to Batch : MuleSoft Virtual Muleys Meetups

1. All contents © MuleSoft, LLC From zero to Batch Anu Vijayamohan Containerizing the Mule Runtime with Kubernetes Arno Brugman 18th November 17:00 London (GMT) | 18:00 Europe | 22:30 India (Delhi) 9:00 San Francisco | 12:00 New York

2. All contents © MuleSoft, LLC MuleSoft Meetups: Virtual Muleys (Online Group - English ) November, 2020Recording

3. All contents © MuleSoft, LLC MuleSoft Meetups: Virtual Muleys 3

4. All contents © MuleSoft, LLC Keep in mind! 1. For Trainings and Certifications - https://training.mulesoft.com/ ○ Still NEW: Mule 4 MCD FREE Certification voucher when completing the Mule 4 Self-Serve course! 2. For Training queries: https://help.learn.mulesoft.com/hc/en-us/requests/new 3. Doubts? ○ We have a very good community to help you! https://help.mulesoft.com/ ○ Or leverage StackOverflow (https://stackoverflow.com/) ( specially for those working on Dataweave and Runtime) 4

5. All contents © MuleSoft, LLC Guidelines for today 1. Pop questions in the chat and anyone can try to answer! 2. Remaining questions will be asked in your behalf at the end of the session 3. The Recording will be shared in the Meetups page and in the VirtualMuleys Youtube Channel 4. Your feedback drives the content of these meetups, ﬁll the surveys at the end of the event, let us know what you think! 5

6. All contents © MuleSoft, LLC Announcements and Updates

7. All contents © MuleSoft, LLC Start playing with DW 2.0 from your browser: http://dwlang.fun/ Pre-Alpha: DataWeave Playground & Tutorial 7 Disclaimer: It is not an officially supported MuleSoft Product. It is not supported by MuleSoft Support, you won't be able to open MuleSoft Support cases for this Suite and there are no SLAs for responses nor bug/features progress.

8. All contents © MuleSoft, LLC Start playing at: http://dwlang.fun/ Pre-Alpha: DataWeave Playground & Tutorial 8 Disclaimer: It is not an officially supported MuleSoft Product. It is not supported by MuleSoft Support, you won't be able to open MuleSoft Support cases for this Suite and there are no SLAs for responses nor bug/features progress.

9. All contents © MuleSoft, LLC ● Repo and Pull Requests: https://github.com/mulesoft-labs/data-weave-tutorial “Pre-Alpha” - DataWeave Playground & Tutorial 9

10. All contents © MuleSoft, LLC Compete in the ﬁrst virtual MuleSoft Hackathon! ● You have until the Nov. 29, 2020 to submit! ● Amazing prices with lot of fun ● For the event: https://www.mulesoft.com/hackathon2020 ● Recording with all the details: https://meetups.mulesoft.com/mulesoft-hackatho n-2020/ ● Disclaimer: Country and additional eligibility restrictions will apply.

11. All contents © MuleSoft, LLC Available Now

12. All contents © MuleSoft, LLC Marina Rivero Anypoint Studio 7.7 deep dive with MuleSoft engineering! Eric Mastrangelo 25th November 17:00 London (GMT) | 18:00 Europe | 22:30 India (Delhi) 9:00 San Francisco | 12:00 New York https://meetups.mulesoft.com/e/mphrn6/

13. All contents © MuleSoft, LLC ● 5-15 minutes videos with a lot of good content. ● Introduction: Continuous Integration and Continuous Deployment (CICD) ● Overview (Why, what is it) and how you can articulate the value of CICD to diﬀerent Stakeholders ● Find it here: https://sfdc.co/cokrDc Friends of Max Videos 13 Build PassTest BuildTest Pass Automation Server Automated Automated

14. All contents © MuleSoft, LLC Simple and eﬀective ways to protect your data Secure your user accounts with multi-factor authentication Something you know Login Credentials Something you have MFA Find more information in: https://docs.mulesoft.com/access-management/multi-factor-authentication

15. All contents © MuleSoft, LLC Secure and strong verification methods Verification methods supported by MuleSoft SMS (Text) verification Phone call verification Email verification Not Supported ( x x Salesforce Authenticator Mobile App Third-Party Authenticator Apps Security Keys Built-In Authenticators Such as: Google Authenticator Microsoft Authenticator Authy Such as: Yubico’s YubiKey Google’s Titan Security Key Desktop + mobile device biometrics, such as: Windows Hello Touch ID Face ID Fast, free authentication sfdc.co/IntrotoAuthenticator

16. All contents © MuleSoft, LLC ● Multi-tabbing has been a highly requested feature from our Ideas Portal ● We are excited to announce that it has been added to API Designer to improve your API specification development experience. ● Whether you have one RAML file or 12, simply CMD + click (Mac OS) or CTRL + click (Windows) on a file in the left-hand panel to open a new tab in API Designer 16 Available Now https://blogs.mulesoft.com/dev/design-dev/how-to-use-multi-tabbing -and-rename-symbol/ Take advantage of the Multi-tabbing Feature

17. All contents © MuleSoft, LLC From zero to Batch Anu Vijayamohan Containerizing the Mule Runtime with Kubernetes Arno Brugman 18th November 17:00 London (GMT) | 18:00 Europe | 22:30 India (Delhi) 9:00 San Francisco | 12:00 New York

18. All contents © MuleSoft, LLC Arno A. Brugman, Strategic Advisor Containerizing the Mule Runtime with K8s MuleSoft Meetup Wed, Nov 18, 5:00 PM (GMT)

19. All contents © MuleSoft, LLC Safe harbor statement The information in this presentation is confidential and proprietary to MuleSoft and may not be disclosed without the permission of MuleSoft. This presentation is not subject to your license agreement or any other service or subscription agreement with MuleSoft. MuleSoft has no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation and MuleSoft's strategy and possible future developments, products and or platforms directions and functionality are all subject to change and may be changed by MuleSoft at any time for any reason without notice. The information on this document is not a commitment, promise or legal obligation to deliver any material, code or functionality. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This document is for informational purposes and may not be incorporated into a contract. MuleSoft assumes no responsibility for errors or omissions in this document, except if such damages were caused by MuleSoft intentionally or grossly negligent. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

20. All contents © MuleSoft, LLC About the Speaker 20 Arno Brugman ● 30+ years in IT ● 4 years with MuleSoft ● Strategic Advisor ○ EMEA’s most ambitious customers ○ Trusted Advisor ○ Product & Technology ○ Customer Advocate

21. All contents © MuleSoft, LLC Agenda 1. Introduction to Containerization 2. Containerization Environments 3. Containerizing the Mule Runtime with Kubernetes 4. Building the Image 5. Deploying the application 6. CI/CD 7. Runtime Fabric 21

22. All contents © MuleSoft, LLC Introduction to Containerization

23. All contents © MuleSoft, LLC Introduction to Containerization A brief history ● MULTICS (1963) ● UNICS (1969) ● MVS/zOS (1974) ● VAX/VMS (1977) ● Chroot Jail (1979) ● Windows (1983) ● Linux (1991) ● VMWare Workstation (1999) ● Linux Containers (2008) ○ LXC ● docker (2013) ○ Based on LXC 23

24. All contents © MuleSoft, LLC docker deﬁnition of Containers Introduction to Containerization A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another 24

25. All contents © MuleSoft, LLC docker deﬁnition of Containers (2) Introduction to Containerization A container is an abstraction at the application layer that packages code and dependencies together. Multiple containers can run on the same machine and share the OS kernel with other containers, each running as isolated processes in userspace. Containers take up less space than VMs, can handle more applications, and require fewer VMs and Operating systems 25

26. All contents © MuleSoft, LLC Vs Virtualization Containers 26 Infrastructure (Host) Operating System docker App1 App2 App3 App4 App5 App6 App7 App8 App9 Infrastructure Hypervisor Virtual Machine Virtual Machine Virtual Machine Virtual Machine Guest OS Guest OS Guest OS Guest OS App 1 App 2 App 3 App 4

27. All contents © MuleSoft, LLC Containerization Environments

28. All contents © MuleSoft, LLC Containerization Environments ● Alibaba: Container Service for Kubernetes + Container Service ● Amazon: ECS + EKS ● Canonical ● Google: GKE ● Gravitational: Gravity ● Heptio ● IBM: Cloud Kubernetes Service + Multicloud Manager ● Kontena Pharos ● Microsoft: AKS + ACI ● Oracle: Application Container Cloud ● Pivotal: Cloud Foundry + Container Service ● Rancher ● RedHat: OpenShift ● Suse 28

29. All contents © MuleSoft, LLC Most Commonly used Containerization Environments ● Cloud Platforms ○ Amazon EKS ○ Google GKE ○ Microsoft AKS 29 ● On Premise ○ RedHat OpenShift ○ Pivotal Cloud Foundry

30. All contents © MuleSoft, LLC Containerizing the Mule Runtime with Kubernetes

31. All contents © MuleSoft, LLC Containerizing the Mule Runtime with Kubernetes Pods vs Standalone Server 31 Kubernetes Server Mule App Standalone Mule Runtime App App App App App App ... Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App

32. All contents © MuleSoft, LLC Advantages vs Standalone Containerizing the Mule Runtime with Kubernetes ● Application Isolation ○ Dedicated CPU ○ Dedicated Memory ○ Dedicated Disk ● Conﬁgurable ○ CPU, Memory, Disk ● Horizontal Scaling ● Vertical Scaling 32

33. All contents © MuleSoft, LLC However... Containerizing the Mule Runtime with Kubernetes ● Application Isolation ○ Dedicated CPU ○ Dedicated Memory ○ Dedicated Disk ● Conﬁgurable ○ CPU, Memory, Disk ● Horizontal Scaling ● Vertical Scaling 33 ● No Automatic Load Balancing ● No Zero Downtime re-deploy ● Build your own Image ● Apply Patches yourself

34. All contents © MuleSoft, LLC However... Containerizing the Mule Runtime with Kubernetes 34 Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Pod Mule App Proxy Anypoint Platform

35. All contents © MuleSoft, LLC Building the Image

36. All contents © MuleSoft, LLC Base Image Building the Image ● OS ● JDK ● Mule Runtime ● License File 36

37. All contents © MuleSoft, LLC Hardware ● 2 GHz CPU (or 1 virtual CPU in virtualized environments) ● 1 GB of RAM ● 4 GB of storage Software ● RHEL 7 ● JDK 1.8.0 or JDK 11 ● Mule Runtime version 4.3 ● Mule License File Building the Image What do you need (for Mule 4.3)? 37

38. All contents © MuleSoft, LLC Deploying the Application

39. All contents © MuleSoft, LLC Deploying the application Method 1: Unmanaged ● Copy the Application.zip ﬁlle into the container ● Run mule 39 ● Not Managed by Anypoint runtime Manager

40. All contents © MuleSoft, LLC Method 2: Managed by Anypoint Runtime Manager Deploying the application ● Install the Runtime Manager Agent ● Upload your application (CLI/API) ● Deploy your application into the container 40

41. All contents © MuleSoft, LLC CI/CD

42. All contents © MuleSoft, LLC CI/CD Startup After starting the Mule Runtime, the Runtime needs to be connected to the Anypoint Platform. Use the Anypoint Platform API to register the runtime 42 Shutdown When a container is shutdown, the container instance (Name + IP) is terminated. Before Shutdown, the container needs to be properly de-registered from the Anypoint Platform using the Anypoint Platform API

43. All contents © MuleSoft, LLC Runtime Fabric

44. All contents © MuleSoft, LLC Anypoint Runtime Fabric Deploy and manage runtimes across any cloud Centrally manage all applications Manage applications in your private cloud (AWS, Azure) and CloudHub within Anypoint Platform Run Mule apps on Kubernetes Get containerization beneﬁts such as isolation, horizontal scaling, and auto redeploy by default Move deployments in a few clicks Pivot your deployments between CloudHub, your private cloud, or a data center with no impact

45. All contents © MuleSoft, LLC Anypoint Runtime Fabric VM Mule App VM Mule App Mule App Runtime Fabric components Runtime Fabric appliance Mule App network Runtime Fabric Mule App Mule App VM Runtime Fabric orchestrates and automates the deployment of Mule runtimes into containers in any cloud or on-premises environment Beneﬁts ● Deploy consistently across any cloud or data center ● Run multiple runtime versions in the same Runtime Fabric ● Scale horizontally and redeploy w/ zero-downtime ● Easily manage via the control plane hosted by MuleSoft ● Flexible deployment upon existing infrastructure or managed K8s services

46. All contents © MuleSoft, LLC Anypoint Runtime Fabric on EKS and AKS

47. All contents © MuleSoft, LLC Anypoint Runtime Fabric on EKS and AKS Available now Available NOW EKS AKS 47

48. All contents © MuleSoft, LLC Deploy and manage runtimes on AKS or EKS Runtime Fabric on AKS and EKS Extend control plane beneﬁts to runtimes on your AKS or EKS ● Customer gets the latest management and monitoring features of Anypoint Platform Decouple Mule runtime services from your own infrastructure ● Give your ops teams the power to use their infrastructure how they want48 CUSTOMER’S NETWORK Mule App Mule App Mule App Runtime Fabric services Mule App Mule App Mule App Mule App Mule App Mule App Runtime Plane CONTROL PLANE Kubernetes-as-a-Service Available now

49. All contents © MuleSoft, LLC Deploy and manage across any K8s service Deploy across private clouds Open K8s access to managed providers Expanded K8s access on managed provider Across any K8s service EKS AKS GKE ● Deploy standalone cluster within AWS, Azure, or on-prem ● Use K8s and docker without K8s expertise ● Install RTF services into private K8s environment ● Open access for customers with existing K8s expertise ● Support for OpenShift container platform ● Run Mule and non-Mule workloads on the same K8s environment ● Open support for any managed service built on K8s ● Deploy across any K8s environment

50. All contents © MuleSoft, LLC 50 Thank you

51. All contents © MuleSoft, LLC Anu Vijayamohan Customer Success Technical Architect, EMEA From zero to Batch MuleSoft Meetup: 18 November 2020

52. All contents © MuleSoft, LLC About the Speaker 52 Anu Vijayamohan, Customer Success Technical Architect ● American born, based in the UK ● Almost 20 years of ﬁeld expertise ● Focussed mostly in working directly in IT Delivery and Professional Services ● Worked for Appian for more than 13 years ● Joined MuleSoft Customer Success about 1.5 years ago

53. All contents © MuleSoft, LLC Agenda 1. Introduction to Batch Processing 2. Batch Processing Use Cases 3. Batch Processing Features in Mule 4 4. Best Practices for Batch Processing 5. Error Handling 6. Performance Tuning Considerations 7. Certiﬁcation Tips 53

54. All contents © MuleSoft, LLC Introduction to Batch Processing

55. All contents © MuleSoft, LLC restaurant 55

57. All contents © MuleSoft, LLC Batch Processing Scenarios Extracting, transforming and loading (ETL) information into a target system Handling large quantities of incoming data from an API into a legacy system Integrating data sets to parallel process records Synchronising data sets between business applications 57

58. All contents © MuleSoft, LLC API vs Batch Characteristics 58 Aspect API Batch Availability On-demand based on user interaction (e.g. real-time) Scheduled copy of data-set that doesn’t require user interaction Profile • Definition: configurable fields/attributes, join, filter, sort, etc. • Volume: control data-set size e.g. 10,000 records per request • Type: transactional, status, etc. • Self-service: empowerment • Definition: static or changes infrequently • Volume: Payload size < 2GB, million+ records, Big Data • Type: historical, reporting/analytics, master Data Change Frequency Real-time, near real-time, minutes Daily, weekly, monthly Consumer One or more Destination/s Publication Event oriented N/A

65. All contents © MuleSoft, LLC 65 ● Polling Connectors ● Poll Scope ● For-Each ● Watermarks Batch and ETL Using Mule Flows Sequential Record Processing Capabilities

67. All contents © MuleSoft, LLC Mule 4 Batch Processing Basics ● Mule processes a batch job asynchronously ● Any event source that feeds data into a batch job must be one-way, not request-response ● Results of batch processing do not feed back to the caller ● Default settings ○ Process records in blocks of 100 (Batch Block Size) ○ 16 thread default threading proﬁle ○ Cancel processing on record failure (Max Failed Records; default is 0) ● Check out the Mule documentation for more detail 67

68. All contents © MuleSoft, LLC Batch Processing Strategies ● Pattern strategy ● Exception strategy ● Capture and design around NFRs 68

69. All contents © MuleSoft, LLC Running Batch Processes ● Front load transformation ● Separation of deployments ● Co-running multiple batch jobs ● File sizes ● Watermarking 69

70. All contents © MuleSoft, LLC Streaming Best Practices ● Use streaming batch processing when writing to a ﬁle ● Performance implications ● Item access implications 70

72. All contents © MuleSoft, LLC Error Logging Mule 4 Default When a batch step fails or raises an error, Mule logs the stack traces following this logic: ● Mule gets the exception’s full stack trace. ● Mule strips that stack trace from any messages. ● Mule veriﬁes if the stack trace was already logged in the current step ○ If this was the ﬁrst time the runtime encountered this error, it logs the error and showing a message like this: com.mulesoft.mule.runtime.module.batch.internal.DefaultBatchStep: Found exception processing record on step 'batchStep1' for job instance 'Batch Job Example' of job 'CreateLeadsBatch'. ○ Subsequent records with the same failures will not be logged for performance and log readability reasons: ● Mule logs on a "by step" basis. If another step also raises the same error, the runtime logs it again for that step 72

73. All contents © MuleSoft, LLC Error Logging Mule 4 Default ● When the batch job reaches the On Complete phase, Mule displays an error summary with every error type, and how many times it happened in each batch step. ● Mule logs are only logs INFO level messages by default ● For more verbose logs, set the batch category to DEBUG level in the application’s log4j conﬁguration 73

74. All contents © MuleSoft, LLC Error Handling at a Record Level ● Finish Processing ● Continue Processing Conﬁgurable Options 74

76. All contents © MuleSoft, LLC Performance Tuning Considerations 76 ● Focus on each process step to look for speciﬁc code optimizations ● Focus on transformation and any external connectivity logic within process steps ● Eliminate as much disk and network latency as possible ● Align performance tuning considerations with current customer performance testing activities ● Consider Disk I/O

78. All contents © MuleSoft, LLC MuleSoft Certiﬁed Integration Architect (MCIA) Batch Topics Covered in Exam ● ForEach scope ● Batch Job scope ○ Synchronicity ○ Filters ○ Pros/Cons ● Bulk Operations in connectors ● Streaming ● Understand the concepts, don’t just memorize ● Learn by doing! Open up Anypoint Studio and debug code to understand the behaviour ● Exam questions require a problem to be solved by weighing diﬀerent options 78

Containerising the Mule Runtime with Kubernetes & From Zero to Batch : MuleSoft Virtual Muleys Meetups

Vous êtes en train de lire un aperçu.

Consultez-les par la suite

Containerising the Mule Runtime with Kubernetes & From Zero to Batch : MuleSoft Virtual Muleys Meetups

Plus De Contenu Connexe

Diaporamas pour vous (20)

Similaire à Containerising the Mule Runtime with Kubernetes & From Zero to Batch : MuleSoft Virtual Muleys Meetups (20)

Plus par Angel Alberici (7)

Plus récents (20)