So, you heard "the Web is Programmable, Internet of Things, Digitalization", but have NO to little programming skills. Nevertheless, this is 2016, and you want to get enough about Web Programming to be part of the some fun and exciting Web challenge, participate in an Hackathon may be …
Well, I am happy we meet. I suggest you take the tour “from ZERO to REST in a hour”: we’ll teach you to forge your own HTTP requests against the Github API. After this tour, you’ll know enough to interact with any RESTful Web APIs. Worth mentionning this presentation is entirely scripted: so give attention to each slide comments.
Did you enjoy the tour ? look forward to learn more ?
Post your comments below about enhancements, and for any subjects you’d like to see covered.
2. Join the Cisco developers community : https://developer.cisco.com/
3. Take a free online Coding Lab (REST, Python, Parsing JSON, RAML, Git…)
https://learninglabs.cisco.com/labs/tags/Coding
4. Meet DevNet teams at a physical event: conferences, hackathons
https://developer.cisco.com/site/devnet/events-contests/events/
2. Overview
understand the basics around RESTful Web APIs
HTTP : HyperText Transfer Protocol
REST : Representational State Transfer
API : Application Programming Interface
use Postman to interact with Web APIs
Postman Chrome App : get it at http://www.getpostman.com/
no pre-requisites
worth mentionning : no coding experience required
3. DevNet
Cisco’s Developer Community
Program
commitment : create a vibrant
Developer ecosystem delivering
innovative applications with deep
API integrations into Cisco
platforms.
resources for ITPros and Devs
https://developer.cisco.com
learning labs
sandboxes
3
4. /Cisco/DevNet/SteveSfartz
API Evangelist @CiscoDevNet
Tropo & Cisco Spark APIs
code addict, any but #golang better ;-)
live in France, all around EMEA
hosted @PIRL
Paris Innovation Center & Research Lab
twitter://@SteveSfartz
github://ObjectIsAdvantag
“vision without
execution is
hallucination”
-- Thomas Edison
stsfartz@cisco.com
6. HyperText Transfer Protocol (HTTP)
a core protocol for internet communications
a request / response protocol
synchronous
stateless
6
GET /index.htm HTTP/1.1
HTTP 200 OK
Client Server
7. HTTP Request : Protocol
HTTP 1.0 is what the Web has been built upon historically, still used by
some embedded devices and enterprise proxies
HTTP 1.1 is the current, generally used HTTP protocol version
HTTP/2 is the next version of HTTP, although known as SPeeDY,
multiplex & bidirectional communications + header compression
the HTTP protocol defines the request / response interaction principles
(methods, headers, status codes, body contents …)
7
8. URL - Uniform Resource Locator
scheme:[//[user:password@]host[:port]][/]path[?query][#fragment]
http://github.com/CiscoDevNet/spark-webhooks-sample
server: github.com, port defaults to 80, path: /CiscoDevNet/spark-webhooks-sample
https://github.com/CiscoDevNet/spark-webhooks-sample
encrypted communication issued over TLS, port defaults to 443
https://github.com:9090/CiscoDevNet?page=2
query the server to serve the 2nd page of results, TLS connection on port 9090
8
9. Accessing github under the hood
9
GET /CiscoDevNet HTTP/1.1 Accept: text/html :443
HTTP 200 OK
browser github.com
method path protocol
status code
https://github.com/CiscoDevNet
portheaders
URL:
response body
HTTP
request
HTTP
response
host
HTML page
15. RESTful Web APIs
Web API
a way for two systems to communicate
two major types : REST and SOAP
REST
stands for Representational State Transfer (REST)
an architecture style for designing communications over HTTP
RESTful Web APIs
communicate by leveraging all the expressiveness of the HTTP protocol
Methods, Status codes, Content-Types, Headers
15
16. HTTP Request : Methods
GET – retrieve contents from the server
HEAD – only get the HTTP headers & status code (don't GET the body)
PUT – update content
PATCH – partial update
POST – create content
DELETE – delete content
OPTIONS – retrieve contextual information before issuing a request
check W3C RFC 7230-7 to get an exhaustive list of HTTP/1.1 methods
16
17. HTTP Request / Response : Body
data sent to or retrieved from the server
HTML
plain text
JSON
XML
SOAP (XML format for advanced interactions)
Binary data (single file, blob)
Multi-part contents (files)
Web APIs typically exchange JSON, XML, binary, and raw text
GET and DELETE requests do not provide a body
17
18. HTTP Response : Status Codes
1xx - Information
100 Continue
2xx - Success
200 OK, 201 Created, 204 No Content
3xx - Redirection
301 Moved permanently, 304 Not modified
4xx - Client error
400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found
5xx - Server error
500 Internal Error, 501 Not Implemented, 503 Service Unavailable
interactive map at www.restapitutorial.com/httpstatuscodes.html
18
25. HTTP Request / Response : Headers
define the operating parameters of an HTTP transaction (req/resp)
optional key-value pairs
Accept-Language: en-US, fr
so that the server can adapt the content locale, [request only]
Accept: application/json, application/xml
types of data supported by the client, [request only]
Content-Type: application/json
type of the request body provided in the request, [request | response]
User-agent, Cache control, Authorization…
25
26. REST Authentication
None: the Web API resource is public, anybody can place call.
Generally the case for GET methods, rarely for POST, PUT, DELETE.
Basic HTTP: a username and password are passed to the server in an
encoded string. Part of the HTTP protocol.Passed with each API call.
Authorization: Basic ENCODEDSTRING
Token: a secret generally retrieved from the Web API developer portal.
The keyword may change from one Web API to another: Bearer, token..
Passed with each API call.
OAuth: A sequence flow is initiated to retrieve an access token from an
Identity Provider. The token is then passed with each API call.
Open standard. User rights are associated to the token (OAuth scope).
The token is short-live, can be revoked and re-issued via a refresh token. 26
28. Signup for a free Github account
https://github.com/join
free account by default
reply to github confirmation email
Welcome to the developers
community !
28
31. Resetting Authentication in Postman
resend latest authenticated request
check custom Header X-RateLimit-Limit is set to 5000
go the Headers tab, remove the Authentication header
go to Authorization tab, choose No Auth
resend request, custom Header X-RateLimit-Limit is now set to 60 31
35. Github Token based authentication
set the Authorization Header in Postman
keyword vary among Web APIs, « token » to authenticate against Github
35
38. RESTful Web APIs sumup
method
GET, POST, PUT, DELETE
URL scheme
http://{SERVER}/api/v1/schema/entity/3
authentication
None, Basic HTTP, token, OAuth
headers
Content-Type: application/json
request / response body
JSON or XML contents exchanged between the client and the Web API
38
41. Logic to add comments to a Github gist
41
HTTP
Client
Github
API
AddCommentToGist (gistID, comment)
CommentID
GetGistsForUser (GithubAccout)
GistsList
RemoveComment (commentID)
Ack
42. Equivalent RESTful Web API flow
42
POST /gists/:gistID/comments
201 Created { comment }
HTTP
Client
Github
API
GET /users/:username/gists
200 OK { gists}
DELETE /gists/:gistID/comments/:commentID
204 No Content
48. HTTP clients
a tool to initiate calls to an HTTP Server
Note : strictly equivalent to Web Client / Web Server
any Web Browser
via a user-friendly User Interface
Postman, Chrome Developer Toolbar, IFTTT, Zapier, Built.io
from the command line
telnet, curl
any programming languages
Python, Javascript, Java, Ruby, PHP, C#, Go…
48
49. To go further
join the Cisco developers community
https://developer.cisco.com/
take a free online Coding Lab
https://learninglabs.cisco.com/labs/tags/Coding
21 learning labs proposed
REST, Python, Parsing JSON, RAML, Git
meet DevNet at a physical event: conferences, hackathons…
https://developer.cisco.com/site/devnet/events-contests/events/
49
52. curl
HTTP from the command line
part of linux operating systems
Windows users
1. PowerShell
2. or git for Windows comes with
a bash, with curl command
52
$ curl -v -X GET http://github.com/CiscoDevNet
* Trying 192.30.252.131...
* Connected to github.com (192.30.252.131) port 80
(#0)
> GET /CiscoDevNet HTTP/1.1
> Host: github.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: https://github.com/CiscoDevNet
< Connection: close
<
* Closing connection 0
53. curl
HTTP from the command line
Part of linux operating systems
53
$ curl -v -X GET http://github.com/CiscoDevNet
* Trying 192.30.252.131...
* Connected to github.com (192.30.252.131) port 80
(#0)
> GET /CiscoDevNet HTTP/1.1
> Host: github.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: https://github.com/CiscoDevNet
< Connection: close
<
* Closing connection 0
54. curl
HTTP from the command line
Part of linux operating systems
54
$ curl -v -X GET http://github.com/CiscoDevNet
* Trying 192.30.252.131...
* Connected to github.com (192.30.252.131) port 80
(#0)
> GET /CiscoDevNet HTTP/1.1
> Host: github.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: https://github.com/CiscoDevNet
< Connection: close
<
* Closing connection 0
HTTP
request
55. curl
HTTP from the command line
Part of linux operating systems
55
$ curl -v -X GET http://github.com/CiscoDevNet
* Trying 192.30.252.131...
* Connected to github.com (192.30.252.131) port 80
(#0)
> GET /CiscoDevNet HTTP/1.1
> Host: github.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: https://github.com/CiscoDevNet
< Connection: close
<
* Closing connection 0
HTTP
response
56. telnet
TCP interactive client
Part of all operating systems
Universal but not HTTP friendly
Not HTTPS capable
56
$ telnet github.com 80
GET /
HTTP/1.1 301 Moved Permanently
Content-length: 0
Location: https:///
Connection: close
58. API calls in Python
Python 3, with the requests module
import requests
url = "https://api.github.com/users/DevNet"
headers = {
'cache-control': "no-cache"
}
response = requests.request("GET",url, headers=headers)
print(response.text)
58
59. API calls in Python
Python 3, with the http.client module
import http.client
conn = http.client.HTTPSConnection("api.github.com")
headers = {
'cache-control': "no-cache"
}
conn.request("GET", "/users/DevNet", headers=headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))
59
Notes de l'éditeur
Welcome, thanks for joining us today.
Then you heard that The Web is Programmable, but have no to little programming skills.
Yet, you want to understand enough to be part of some fun and exciting Web challenge, participate in Hackathons.
No worries, you’re in good hands, we’ll take you from ZERO to forging your own HTTP requests over the wire along this introduction to REST.
Hopefully, after this 60’ session, you’ll know enough to interact with any RESTful Web APIs.
This session is about learning to interact with Web APIs.
To make this happen, we’ll give you an overview of the HTTP protocol,
and you ‘ll discover the REST concepts by incremental hands on.
We’ll forge our own messages and you’ll be able to send these over the wire.
We’ll spend about 15’ minutes hands on with the postman tool.
If you haven’t installed Postman, go and get the tool now,
Make sure Chrome is installed on your machine.
DevNet is Cisco’s Developer Community Program.
The DevNet team interacts with Cisco employees, partners and customers, to grow the knowledge of Cisco APIs, and from there, how to innovate and accelerate their business by leveraging CISCO technologies, platforms, whether on-premise or cloud services.
This happens via in-presence Conferences, Trainings, Hackathons,
But also via online contents : step by step learning labs, and sandboxes when advanced environnements need to be setup with dedicated infrastructure to help you get your hands dirty.
DevNet is about everything programmable at Cisco, whether legacy, brand new product launch, and innovation out of Cisco R&D labs, to give developers and ITPros exhaustive info on how to leverage our technology.
You got it : DevNet is the direction you’re invited to point all your partners and customers if interested in the programmability of CISCO’s platforms.
My name is <PRESENTER>
Let’s dive into the HTTP intrinsics
HTTP communications happen synchronously : the same channel - the request is issued from – is reused for the server to respond.
Moreover, the protocol is stateless : each req/response happen independently one from another. If you need to share state, you can transfer similar IDs, Headers, Cookies, among the various calls you place. We’ll see a typical workflow later.
You may look into the slides deck for more details about HTTP advanced concepts (versions, timeouts).
-- ADVANCED
Note : If the server takes too long to respond, you can get a timeout on the client. The communication channel can also get broken. In these cases, the server may have processed your request but you cannot know that for sure. It is something you need to take into account when you program the web.
-- ADVANCED
see W3C RFC 7230-7237 for HTTP 1.1 for more details
URLs define resource locations : how to contact a resource.
URLs are composed of various parts :
http and https schemes are our concerns for this training. If you’re curious about other schemes, think sip, ftp for instance.
user and password are rarely placed in the URL as they could be traced over the wire. Prefered way is to encode them and place them in a Header, we’ll see that in a few minutes.
Host, port, path, query are the major pieces you need to know about to program the web
Host and port define a Service Endpoint, ie, where resources can be reached at.
Ports default to 80 for HTTP and 443 for HTTPS, they need not be explicitely set in these case
URLs are very expressive,
When building Web APIs, programmers leverage this expressiveness.
Let’s try one.
-- ADVANCED
http://pseudo:123ABC@github.com/CiscoDevNet
access the page with credentials (Basic Authentication)
-- ADVANCED
other protocols not worth mentioning in this introduction
sip:22444032@cisco.com
Let’s start with an HTML page, as the same HTTP principles apply to Web APIs.
Here ‘s what happens when you ask your browser for a Web page,
More concretely :
you hit an Endpoint (here the host: github.com, port: 443), remember it is what HTTPS defaults to,
the connection is established and the HTTP protocol is used as specified by the scheme,
the resource path /CiscoDevNet is asked for, with a GET method,
the web browser adds an Accept Header to the request because an HTML page is what it expects the server to return
This is the request, now let’s look at the answer :
the github server returns on the same channel a response with the statuts code 200 to tell everything went OK
and it writes the HTML page contents on the wire
Now, we got this HTTP intrinsics, we’re ready to place the call over the wire with POSTMAN.
-- ADVANCED
DO : open Chrome, go to github, open Dev Toolbar, show what’s happening
Several resources are being accessed
To different endpoints
And different types of Data are being sent and received.
That’s the way the Web works, we issue HTTP calls, asking for resources.
Then all you need to know to start building calls yourself.
Yes, now is your opportunity to go hands-on with HTTP.
If you have not installed Postman, this is the last call.
Go to getpostman.com,
Click Chrome App for the purpose of this training
BTW, this session is recorded, and we’ll transfer you the deck.
20% slides are hidden in this presentation,
I invite you to go through them for further details.
This is what you end up with at your first Postman launch.
Now let’s place our HTTP request to github.com
Leave the GET method as is.
Enter the URL of the resource.
Press the Send button
DO : open POSTMAN, issue the call https://github.com/CiscoDevNet
Postman issues the HTTP call on your behalf, and shows the response transmitted by the Github service :
statuts of 200 OK if everything went ok
If you get 404, the URL is malformed
The HTML page contents are placed in the Body
Note that the HTML content-type specified by the server is also displayed
If you encounter an issue, ask for assistance in the Spark room and sending a Snapshot of your work in Postman may be your best bet to get an hand from the team.
Excellent, you ‘ve just learned the basics of HTTP, you know how to forge your own HTTP call and send them over the wire.
Ready to consume Web APIs ? Let’s go for it.
So what are RESTful Web APIs ?
First, let’s start with the methods.
We’ve just experienced GET with the Postman calls.
The 4 major methods are GET / POST / PUT / DELETE
These methods enable HTTP Clients to create / read / update / delete Contents on the Web API Server.
Worth mentionning other methods exists : HEAD, PATCH, OPTIONS, though you probably won’t use them during your first interactions with Web APIS.
Remember Web APIs leverage the HTTP protocol.
Thanks to this various Status Codes, the server tells about the outcome of the request you just placed.
Each status code is composed of 3 digits.
The first digit gives you immediate knowledge about the global outcome of the request.
2, 4 and 5 series are the first to know about.
2xx : everything went fine, consider this code as an Acknoledgment,
200 OK generally, 201 on POST calls when a resource has been created, 204 typically happens for a DELETE (nothing to return)
4xx : you place a malformed call, you need to fix something before re-issuing the call
Typically, 404 bad URL, 400 malformed request, 401 authentication failed, 403 you’re not allowed to issue this call (authorization)
5xx : it is a server error, there’s nothing YOU can do about it, try again later when the Web API team has fixed the issue, look at the Web API twitter account to see if anything is broken, if not contact the Web API support and get your ticket.
OK, by now, you know about HTTP Methods and Status Code, that’s enough to place Web API calls.
We’ll retrieve the exact same data as before, but this time in a Machine 2 Machine format,
via the Github Web API.
A quick Google search, and you ‘ll find the Github API entry point.
DO : Browse documentation
Click current version. Note that the API is versionned. When you hit a version 3 endpoint, you’re expected to issue Version 3 compliant calls. Current github API is running version 3.
Browse schema underneath. Note that the endpoint is https://api.github.com and only JSON is supported by the API. Consider JSON as the RESTful Web APIs Linga Franca.
Go to the HTTP verbs
Go to the Pagination information, to define how many results you wanna fectch
This is generic information about the way to interact with the Web API.
Now let’s have of look at the Table of contents on the Right, this is where you’ll find all the various Resources you can reach.
Repositories is the Resource we’re looking for.
DO : Go to https://developer.github.com/v3/repos/#list-user-repositories
DO : Explain how the call should be formed
Time to give it a try : let’s invoke the Repositories Resource via Postman !
Open a new tab in Postman, and forge your REST call
GET method
https://api.github.com endpoint
/users/CiscoDevNet/repos resource path
Optionnally, you can add query parameters :
Sort order : display first the latest repositories that have been pushed (received new source code)
Pagination : makes it easier to go through results
Send the request and check the result :
200 OK
JSON : see how the JSON is formed
Carrets for list of elements,
Braces to describe an element,
Contents can also be nested.
See how JSON entry ‘s got a name which makes it easy to understand what’s received.
Though, the API documentation can help when you’re wondering.
We are just an extra step for being API experts.
As most APIs expose private date, you’ll generally need to authenticate to read or modify contents.
This is where Headers come into play.
Remember the Accept and Content-Type headers from the calls we placed earlier.
Any number of headers can be placed in an API call.
Yet, these headers have been standardized.
Authorization is the header we’ll turn our attention to.
These are the 3 main authentication protocols for Web APIs.
Now, let’s see how to forge requests with Basic HTTP Authentication
First signup for a Github account
/!\ answer the github confirmation email to get your account fully provisionned.
Let’s authenticate with Basic HTTP in Postman
Click the Authorization tab
Fill in your Github username, password
Click Update Request
Go to the Headers tab, see the encrypted Basic HTTP header has been automatically filled by Postman.
Note : you can place your Github password here, or a personal access token we’ll generate right after, Github supports both.
Let’s go for Basic HTTP
- Fill your data
- Click Update Request
See how the header is automatically filled by Postman
Note : you can place your Github password here, or a personal token we’ll generate right after, Github supports both.
2 littles Postman tricks here :
1. Postman makes it sometimes difficult to reset the Authentication header.
Here ‘s how to.
2. Rate Limitation : the github API proposes 2 different rate limitation, ie # of API calls per hour,
5.000 when authenticated, and 60 when not authenticated.
Now, let’s see how to forge requests with token based Authentication
Go to Github menu entry: Github Settings > Personal access tokens
Make sure to copy your generated access token, you won’t be able to see it again,
But no worries, if you did not copy it, you’ll simply generate a new one : Regenerate (see previous slide)
The token is named « token » here,
Do not forget the space
Bearer is often used with a capital B, used for OAuth access tokens for example
See the status code is now 401
To sumup what we’ve seen so far :
…
But interacting with a Web API means issuing several calls,
Each Web API flow depends on what you want to implement from the HTTP client perspective
Let’s take an example, we leverage the Github API once more.
Github Gists are one page contents (plain text notes or source codes or documentation), you can push, version and share publically or privately.
Any Github authenticated user can comment public Gists.
Here is an example of interaction workflow,
List a users’s gists
Add a comment to a gist
Remove your comment
Let’s see how this workflow is implemented against the Github API.
First GET
Second POST
Third DELETE
DO : Go through the status codes for each request 200, 201, 204.
All calls are successful with subtle differences for GET, POST, DELETE responses.
Note that JSON responses need to be parsed to extract the pieces of contents you need for your business logic and to issue the next calls.
For example,
the gistID you’re interested in need to be extracted from an array of Gist
the commentID may be extracted from the
Well, we’ve gone through the POSTMAN client,
But others exist, depending on the type of interaction you wanna build.
The curl command comes with all operating systems,
On windows, launch Powershell, or install the git for windows.
We’ll illustrate the curl command example with the first call we made to Github.com to request the home page of DevNet.
Note that a redirect is suggested by the github server, as an HTTP request was issued instead of HTTPS.
-v option helps understand the connection flow
The method can be specified via the –X parameter, defaults to GET
That is for curl, you had the opportunity to experience the request / response flow over the wire.
Who wants to consume a REST API from a telnet client, pure theory,
We’ll generally skip this slide, leaving it here just in case.
We’ll leverage two python modules, pick the one you like best
Quickly forge REST calls in python
A python http client module, with a rawer level flavor