In this Slide Share you will learn how to call REST API using SSIS Web Service Task , JSON Source Connector or XML Source Connector. You will also learn how to perform HTTP GET Request and HTTP POST Request without knowing any programming languages (e.g. C#, JAVA, Python)… simply do Drag and Drop in SSIS.

1. Call REST API using SSIS Web
Service Task / JSON / XML Source
Reach for the Clouds, Inc.
Next Generation SSIS Tasks and Connectors Series
AUTHOR:
NAYAN PATEL | SR. ETL SSIS ARCHITECT
N PAT E L @ R F TC LO U D S . C O M

2. Content
• Introduction
• What is RESTAPI / RESTfulWeb Service
• Tasks/Components in SSIS for Consuming RESTful webservice
• Using SSIS JSON Source to read from RESTAPI and load into SQL Server
• RESTAPITask – Ad-hoc web requests – Call RESTAPI (POST, DELETE)
• HTTPGET Request using SSISWeb ServiceTask or JSON/XML Source6.1 Pass values in HTTP Headers
Pass values in Query String dynamically
• HTTP POST Request using SSISWeb ServiceTask or JSON/XML Source7.1 Uploading files using HTTP
Multipart/form-data POST Request
• PassingCredentials to yourWeb Service8.1 Basic Authentication – SetAuthorization Header (Base64 Encoding)
Token based approach – such asOAuth
OAuth Authorization
Windows Authentication (NTLM)
PassClient Certificate
• Saving HTTPWeb Service Response to File/Variable
• Save HTTPWeb Response Headers / StatusCode
• HTTPWeb ResponseValidation
• RESTAPI Pagination (Loop through multiple requests)
• HTTPWeb Response Error Handing
• Read/Write and Parse Cookies for HTTPWeb Request/Response
• Changing Headers/Url or POST data Dynamically
• Extract single value from JSON/XMLWeb Response (e.g.Token)
• Conclusion

3. Introduction
• In this article you will learn how to call REST API using SSISWeb ServiceTask ,
JON Source Connector or XML Source Connector.You will also learn how to
perform HTTP GET Request and HTTP POST Request without knowing any
programming languages (e.g. C#, JAVA, Python)………
Simply do Drag and Drop in SSIS.

4. Introduction
• In this article you will learn how to call REST API using SSISWeb ServiceTask ,
JON Source Connector or XML Source Connector.You will also learn how to
perform HTTP GET Request and HTTP POST Request without knowing any
programming languages (e.g. C#, JAVA, Python)………
Simply do Drag and Drop in SSIS.

5. What is REST API / RESTful Web Service
• So lets first understand What is REST API or sometimes referred as RESTful
Web Service.This is new buzzword which you will hear a lot. More and more
services are available in Cloud which makes it obvious to come up with some way
so you can access data more firewall friendly manner.What can be better option
than access it using several decade old HTTP protocol so no additional
configuration required for anybody accessingCloud Services? So in short
highlights for RESTWeb Service

6. What is REST API / RESTful Web Service
• RESTWeb Service is stateless client-server service model
• By passing HTTP verb you can perform server side action over standard HTTP
protocol (e.g.GET, POST, LIST, DELETE,HEAD)
• You can pass parameters via URL query string and via HTTP Headers
• If you doing HTTP POST then you can pass additional data in Request Body along
with other two method described above
• REST API works by sending HTTP Request and Getting HTTP Response
• HTTP Response ofWeb Service can contain Headers and Response Data which
can be Binary format,Text format, JSON, XML etc.
• You can secure your data by simply sending it over HTTPS (Secure HTTP)
protocol or encrypting values passed along with your request

7. Tasks/Components in SSIS for Consuming RESTful webservice
• JSON SourceConnector (REST, File, OData): Use this dataflow
component when you have to fetch data from REST API webservice
like a table.This component allows you to extract JSON data from
webservice and de-normalize nested structure so you can save to
Relational database such as SQL Server or any other target (Oracle,
FlatFile, Excel, MySQL).This component also supports reading local
JSON files or direct JSON string (Wildcard pattern supported too e.g.
c:datafile*.json).
• REST APITask : Use this task when you don’t want to pull REST API
data in tabular format but want to call rest API for POST data to
server, DELETE data from server or things like download HTML page,
extract Authentication tokens etc where you not necessarily dealing
data in tabular format.This task also allows you many other options
such as saving RAW response into variable or file.

8. Tasks/Components in SSIS for Consuming RESTful webservice
• XML Source Connector (SOAP, File, REST) : Use this dataflow
component when you have to fetch data from XML or SOAP
webservice and consume data like a table.This component allows you
to extract data from webservice and save to SQL Server or any other
target (Oracle, FlatFile, Excel, MySQL).This component also supports
reading local XML files or direct XML string.

9. Using SSIS JSON Source to read from REST API and load into
SQL Server
• If you have need to consume REST API Service and store result into
SQL Server or any other RDBMS/FLAT File then you can check this
SSIS JSON Source Connector . JSON SourceConnector can
deformalize your nested JSON (It also supports JSONPath filter
expression).

10. HTTP GET Request using SSIS Web Service Task or JSON/XML
Source
• Calling REST API from SQL Server BI Stack usually requires programming
skill. But in this section we will learn how easy it is to call RESTful Web
Service using SSISWeb ServiceTask, JSON Source or XML Source (all Drag
and drop).
• In this example we will get JSON data from ODATA Web Service.
• Download and Install SSIS Lightning Pack
• Create new sample SSIS package in BIDS/SSDT
• From SSISToolbox look for items starting with “ZS”. Drag and Drop [ZS
Rest APIWeb ServiceTask] to Designer Surface
• Configure HTTP Request properties as below
• Configure HTTP Response properties as below
• Run SSIS Package

11. Pass values in HTTP Headers
YOU CAN PASS VALUE IN HTTP HEADERS
USING DIRECT APPROACH OR DYNAMIC
APPROACH. IF YOU WISH TO PASS VALUE FROM
SSIS VARIABLE THEN USE VARIABLE
PLACEHOLDER E.G.
{{USER::VARSOMEVARIABLE}}. YOU CAN ALSO
USE VARIABLE FORMAT SPECIFIER E.G.
• FORMAT DATE:
{{USER::VARSOMEVARIABLE,YYYY-MM-DD}
• ENCODE INTO BASE64:
{{USER::VARSOMEVARIABLE,BASE64ENC}}
• DECODE INTO PLAIN TEXT FROM BASE64:
{{USER::VARSOMEVARIABLE,BASE64DEC}}
Calling RESTAPI in SSIS using RESTAPITask, Pass headers,
Body, Url Parameters

12. Pass values in HTTP Headers
SSIS RESTApiTask – HTTP GET,Test
SSISWeb Service Call, Pass Custom
Header
RESTAPI response validation ( By status code, content,
header)

13. Pass values in Query String dynamically
MOST OF WEB SERVICES USUALLY ACCEPT PARAMETERS VIA QUERY STRING.
YOU CAN ALSO USE VARIABLE PLACEHOLDERS TO MAKE YOUR URL WITH
QUERYSTRING DYNAMIC (E.G.
HTTPS://MYSITE.COM/ORDERSERVICE/?STARTROW={{USER::VARSTART}}&EN
DROW={{USER::VAREND}}

14. HTTP POST Request using SSIS Web Service Task or
JSON/XML Source
• When you select HTTP POST Method then by default content-type is
set to application/x-www-form-urlencoded.With POST Method you
can send POST data (When POST Method selected Data textbox
becomes editable.You can also use variable place holders in POST
data to make it dynamic.
• POST data is usually in key/value format (e.g.
user=abcd&pass=mypass123) but sometimes service required data in
JSON format or XML format. In that case you can submit that way.
Depending on service requirement you have to set additional headers
to indicate content-type (Please refer your service documentation).

15. Uploading files using HTTP Multipart/form-data POST
Request
MANY API SUPPORT UPLOADING FILES ALONG WITH YOUR POST REQUEST.
REFER THIS ARTICLE TO LEARN MORE ABOUT HOW TO UPLOAD FILES
USING REST API CALLS.

16. Passing Credentials to your Web Service
IF YOUR SERVICE REQUIRES AUTHENTICATION THEN YOU HAVE TO MAKE
SURE YOU PASS REQUIRED CREDENTIALS IN CORRECT FORMAT. BELOW
IS LIST OF SOME MOST COMMON AUTHENTICATION TECHNIQUES.
• BASIC AUTHENTICATION (USERID AND PASSWORD APPROACH)
• TOKEN BASED APPROACH (REQUIRES ATLEAST ONE ADDITIONAL
REQUEST TO GET TOKEN)
• OAUTH AUTHORIZATION
• WINDOWS AUTHENTICATION (NTLM)
• CLIENT CERTIFICATE

17. Basic Authentication – Set Authorization Header (Base64
Encoding)
HTTP STANDARD SUPPORTS BASIC AUTHORIZATION MODE IN WHICH
YOU CAN PASS USERID AND PASSWORD INTO BASE64 ENCODED STRING.
CHECK THIS ARTICLE FOR MORE INFORM
HOW TO PASS BASE64 ENCODED AUTHORIZATION HEADER

18. Token based approach – such as OAuth
• This is becoming most common approach using protocols such as
OAuth where user first authenticate to service using AccountKey and
SecretKey. Once authenticated you receive token which can be valid
for certain duration (or infinite duration). After you receive token you
can call services (each service call will include this token). Parsing
token from intial response can be achieved through REST API
Response Filter Expression (e.g. $.token) or use JSON ParserTask

19. OAuth Authorization
• All REST APITasks/Components in Zappysys SSIS PowerPack support
OAuth Authorization. OAuth is getting popular and many bigger
companies already adopted this standard (e.g. Facebook,Twitter,
Google, Salesforce).With this method you can connect to REST API
Service without storing your Userid/Password. Only first time you
have to login to using your credentials to get initialToken and after
that service will continue using AccessToken rather than your
UserID/Password. OnceToken expires it can automatically
renew. Check this for more information about using OAuth

20. Windows Authentication (NTLM)
BY DEFAULT ZAPPYSYS REST API TASKS/COMPONENTS USE YOUR
DEFAULT CREDENTIALS. YOU CAN ALSO USE HTTP CONNECTION
MANAGER WITH USE CREDENTIALS > USE WINDOWS AUTHENTICATION
OPTION. SEE THIS ARTICLE

21. Pass Client Certificate
Some times you may have to pass client certificate along with your web
request. Server check your certificate passed along with your request
and if matched it proceeds with your request. here is the real-world use
case How Azure Management Api uses Client Certificate.

22. Saving HTTP Web Service Response to File/Variable
If you wish to save response to file then goto response tab of REST API
Task and check “Save Response” option and you can select save to file
option from dropdown

23. Save HTTP Web Response Headers / StatusCode
If you wish to save response to file then goto response tab of REST API
Task and check “Save Response” option and you can select save to file
option from dropdown

24. HTTP Web Response Validation
REST API TASK SUPPORTS VALIDATING YOUR RESPONSE FOR CERTAIN
HEADER, STATUS CODE OR CONTENT VALUE. SEE VALIDATION TAB FOR
MORE INFORMATION. USING THIS FEATURE YOU CAN THROW ERROR IF
CERTAIN HEADER IS MISSING FROM RESPONSE OR REJECT RESPONSE IF
STATUSCODE IS OTHER THAN 200.

25. REST API Pagination (Loop through multiple requests)
ANOTHER POPULAR CONCEPT IN REST API IS PAGING. IF YOUR
RESPONSE IS LARGE THEN OFTEN SERVER RETURNS YOU PARTIAL
RESPONSE AND THEN YOU HAVE TO MAKE SURE TO CONSUME REAMING
DATA BY REQUESTING ALL NEXT URLS UNTIL LAST PAGE IS RETURNED.
READ THIS ARTICLE TO LEARN MORE ABOUT REST API PAGING

26. HTTP Web Response Error Handing
REST APITask supports rich error handling.You can ignore certain
errors and save error flag into variable based on any of the following
criteria
• Continue on any error
• Continue on error with specific ResponseCode (e.g. 404)
• Continue on error with specific string in message

27. Read/Write and Parse Cookies for HTTP Web
Request/Response
MANY TIMES YOU HAVE TO SUPPLY AUTHENTICATION TOKEN VIA
COOKIES OR ANY OTHER INFORMATION NEEDS TO BE PASSED VIA
COOKIES THEN REST API TASK HAS COMPLETE SUPPORT TO READ/WRITE
OR PARSE INDIVIDUAL VALUE OUT OF COOKIE STRING. LOOK AT COOKIES
TAB FOR MORE INFORMATION. COOKIES MAPPING GRID CAN HELP YOU
TO MAP COOKIE VALUE TO SSIS VARIABLE (WRITE COOKIE VALUE TO
VARIABLE).

28. Changing Headers/Url or POST data Dynamically
MOST OF FIELDS ON REST API TASK, JSON SOURCE AND XML SOURCE
SUPPORT PLACEHOLDERS.
USE PLACEHOLDER ANYWHERE IN THE FOLLOWING FIELDS USING
{{USER::YOURVARIABLE}} FORMAT. IF YOU EDIT VALUE YOU MAY SEE
INSERT PLACEHOLDER OPTION.
• URL
• BODY
• HEADERS
YOU MAY FORMAT DATETIME USING SPECIAL SYNTAX LIKE THIS
{{USER::MYDATE,YYYY-MM-DD HHMMSS.FFF}}

29. Extract single value from JSON/XML Web Response
(e.g. Token)
• If your web response is in JSON or XML format then you can filter it using
Expression (e.g. JSONPath or XPath). Go to ResponseTab and select Content
Type Format from dropdown. Enter expression to select nested sub document
or value from your response. If you want to save individual properties from
response into multiple variables then use JSON ParserTask
• Most common usecase is if you getting token from your JSON REST API
service. Assume that Service sends you response in following format but you
only care about access_token property from below response. In that case you
can use JSON Path expression like this $.data.access_token

30. Conclusion
REST API IS BECOMING MORE AND MORE POPULAR EACH DAY. WITH THAT
DATA INTEGRATION FROM RESTFUL SERVICES GOING TO BE CHALLENGE.
SSIS REST API TASK AND SSIS JSON SOURCE CONNECTOR CAN SOLVE SOME
OF THOSE ISSUES YOU MAY FACE WITH SSIS. USE JSON SOURCE CONNECTO R
IF YOU HAVE NEED TO STORE DATA INTO SQL TABLE. USE REST API TASK IF
YOU HAVE NEED TO MAKE AD-HOC HTTP/REST REQUEST (SUCH AS GET
TOKEN, DELETE RECORD ETC). ZAPPYSYS REST API COMPONENTS GIVES
YOU TOTAL CONTROL ON YOUR REST API INTEGRATION CHALLENGES
WITHOUT LEARNING PROGRAMMING LANGUAGE (E.G. PYTHON, RUBY, C#,
JAVA).