Especially this document provide very useful and meaningful concepts about SnapLogic. Also this document will be more useful for beginner/intermediate level SnapLogic learners.
2. Synopsis:
Rise of cloud and importance of cloud
Traditional ETL
ETL frameworks & Popular ETL tools
SnapLogic Origin and DI Solutions
As per Gartner report leader in DI
Importance of SnapLogic (Why SnapLogic)
Introduction of SnapLogic
Components of SnapLogic
Elastic Integration Platform Architecture
Web Standards
Resources reference
2
4. SnapLogic Origin and DI Solutions
SnapLogic founded in 2006
Elastic Integration Cloud
Data Integration Solutions
Cloud Data Warehousing
Finance & Accounting
API Integration
Big Data
Data Analytics
IOT
CRM
Flat files and Databases etc
4
Benefits:
Easy and drag and drop
Digital Cloud based and faster
Increase your integrate by 10X
Save millions in integrations Cost
6. Introduction to SnapLogic:
SnapLogic Integration Cloud offers,
Faster Integration
Multipoint Integration
Modern Integration
iPaaS - Integrated Platform as a Service
Batch Processing
Real time applications
ETL and Data integration requirements
REST and JSON based data transfer
AWS Cloud and easily Scale in and Scale out
Expose as a webservice with access permission
6
7. Integration Cloud Platform Components:
Designer
Manager
Monitoring Dashboard
Snaps:
Prebuilt Connectors - 450+ Snaps available
Different Snap-packs available
Core Snaps (XML,JSON, File Reader, COPY, Mapper etc)
Data Snaps (Oracle, Redshift ,SQL Server etc)
Enterprise Snaps(Sales force, Service Now etc)
Social Media Snaps etc(Facebook, Tweeter etc)
Snaplex:
Cloudplex
Groundplex
Hadooplex 7
8. Designer:
An HTML5-based
Canvas Area for pipeline development
Creating the pipelines
Importing the single pipeline
Pipeline properties
Pipeline execution statistics
Validating the pipeline (Shift +Validate) will clear the Cache as well
Pipeline Versions (important for tracking/changes)
Copy Pipeline & Move pipeline
Create the Task
Download pipeline
Compare pipelines
8
10. Manager:
Snap Packs
Has different types of Snap packs and its no.of Snaps for each Snap pack
Example: Redshift, Oracle, Facebook, Email etc
Scheduled Tasks
This will give you the details about list of Scheduled Task and its details
Asset Search
Very useful for searching the pipelines, account , files, tasks etc
Activity Logs
List of activity logs information and details about each and every change
Also we can see logs for different purpose,
Snaplex logs
User logs
Assets logs etc
Alerts
Snap Statistics
Project Spaces 10
12. Monitoring Dashboard:
Health Tab
Snaplex health
Total node
Active Node
Feed master/ Nodes
Additional information's like Machine Memory, Java version, Total processor , Used
Memory etc
Performance details about the Snaplex
Pipeline Tab
Current Pipeline Executions
History Pipeline Executions
Filter by Invocation Type(Trigger, Ultra, Scheduled, Manual), Particular User view
12
16. Elastic Integration Platform Architecture
Implemented based on Concept of SDN – Software Defined Networking
System decoupled into two main areas,
Control plane
Data plane
1. Control Plane
It controls where and how the data are processed based on configurations
Three web applications > Designer, Manager and Dashboard
16
17. Control Plane:
1. Hardware and Software Load Balancers:
2. Security: Manages the authorization and authentication of users accessing the web
applications.
3. Scheduler: Manages scheduled tasks and jobs of integration pipelines.
4. Controller: The control point where elasticity, lifecycle, software updates, etc. of all
runtime components are managed.
5. Metadata Repository: Stores pipeline metadata in a fully redundant, secure
database(MongoDB).
6. Amazon S3 for File System and Log Servers: Stores system files and pipeline generated
log files in a secure and resilient environment.
7. SnapWatch: The management and monitoring tool for the SnapLogic DevOps team to
administer the entire SnapLogic Integration Cloud infrastructure.
17
18. Data plane
The “data plane” is where the actual business data is processed
One or more Snaplex
On premise Groundplex > installation required
Cloudplex > Preconfigured
Hadooplex
Eplex > New release
Remotely managed by control plane
The unit of scalability inside Snaplex is a Java virtual machine (JVM), referred
to as a “Node
One or more nodes for scalability
Auto scaling of nodes
18
19. Web Standards: JSON, REST, APIs, and Mobile
JSON - Java script Object Notation
REST - Representational State Transfer
Common standard for data interchange as JSON and REST service for web
application
Cloud web standards for to handle
Structured data
Un-Structured data
Each pipeline can be invoke by REST protocol
We can expose as API service for pipelines with security of authentication
19
20. SnapLogic Developer Site: http://developer.snaplogic.com/
Mainly for Snap development
SnapLogic Blog: https://www.snaplogic.com/blog
Read about SnapLogic case studies, news, features, initiatives, and more.
SnapLogic Collateral: https://www.snaplogic.com/resources
SnapLogic Analyst Reports, White Papers, Data Sheets, Webcasts, Videos, and more.
SnapLogic Trust Site: http://trust.snaplogic.com/
SnapLogic's site for real-time visibility into system performance and availability. See the
current status of the SnapLogic Enterprise Integration Cloud and the Cloudplex, as well as a
list of upcoming scheduled maintenance. Subscribe to receive automatic updates.
SnapLogic documentation: https://docs-
snaplogic.atlassian.net/wiki/spaces/SD/overview
SnapLogic Community: https://community.snaplogic.com/
Useful SnapLogic Resources:
20