What's the current state of Web technologies, and how does it affect professionals creating Web applications? In this talk, I survey the latest trends in Web technologies, and where I believe they're going in the near future.
Modern Web Technologies — Jerusalem Web Professionals, January 2011
1. Modern Web
Technologies (and why
you should care)
Reuven M. Lerner • reuven@lerner.co.il
Jerusalem Web Professionals
January 18th, 2011
2. Who am I?
• Web developer since 1993
• Software architect, lecturer, consultant,
technical advisor
• Linux Journal columnist since 1996
• Mostly Ruby on Rails + PostgreSQL, but
also Python, jQuery, MySQL, and lots
more...
3. How does the Internet
(TCP/IP) work?
“Socket”
Client Server
Port Port
4. How does the Internet
(TCP/IP) work?
“Socket”
Client Server
Port Port
Client opens
connection
5. How does the Internet
(TCP/IP) work?
“Socket”
Client Server
Port Port
Client opens Server accepts
connection connection
6. Protocols
• Communication standards built on top of
TCP/IP, typically text-based
• SMTP (e-mail)
• FTP (file transfer)
• NNTP (transfer of “news” articles)
20. Submitting forms
POST /login
name=reuven&password=secret
Browser Server
21. Submitting forms
POST /login
name=reuven&password=secret
Browser Server
200 OK
<html>...
<p>Thank you!</p>
...</html>
22. Document = request
• If an HTML page contains images, then each
is retrieved in a separate HTTP request
• Page with 30 images = 31 HTTP requests
• Page with 20 images, 10 JavaScript files,
and 5 CSS files = 36 HTTP requests
23.
24. Idea: Lie to the browser
• Don’t return a document to the user
• Rather, run a program when the user makes
a request, and send the program’s output
• If the output is in HTML, then the browser
will show it no differently than a static doc
25. Just in time production
• Wait as long as
possible to create
pages for the user
• Ideally, create them
when the user
requests them
• “Mass customization”
28. Dynamic document
GET /
Browser Server
200 OK
<html>
<head>...</head>
<body>...</body>
</html>
29. Dynamic document
GET /
Browser Server
200 OK
<html>
<head>...</head>
Program output
<body>...</body>
</html>
30.
31.
32.
33.
34. What we return
• Usually HTML
• Image (e.g., stock graphs, Google Analytics)
• Word/Excel doc (e.g., from Google docs)
• PDF (e.g., PDF reports)
• XML, JSON (for computers, not people)
• Basically, any defined MIME type
35. APIs and mobile apps
Computer A Computer B
(browser) (server)
36. APIs and mobile apps
GET /
Computer A Computer B
(browser) (server)
37. APIs and mobile apps
GET /
Computer A Computer B
(browser) (server)
200 OK
<some-xml>
<talk>JWP</talk>
</some-xml>
38. APIs and mobile apps
GET /
Computer B
(server)
200 OK
<some-xml>
<talk>JWP</talk>
</some-xml>
39. APIs and mobile apps
GET /
Computer B
(server)
200 OK
<some-xml>
<talk>JWP</talk>
</some-xml>
40. APIs and mobile apps
GET /
Computer B
(server)
200 OK
<some-xml>
<talk>JWP</talk>
</some-xml>
41. APIs and mobile apps
GET /
Computer B
(server)
200 OK
<some-xml>
<talk>JWP</talk>
</some-xml>
42. What is a Web app?
• Receives its inputs via HTTP
• Sends its output via HTTP
• That’s it! A Web application can do
anything you want, within these limits
43. That’s it!
• Now you understand how the Web works.
• Really, that’s it.
• Go home.
44.
45. OK, perhaps not...
• How do we write these programs?
• Where (and how) do we store user data?
• How have the underlying technologies
(URLs, HTTP, and HTML) advanced?
46. Early Web applications
• First server-side programs were in C
• They were compiled into binaries
• So you had the CGI source (cgi-src)
directory...
• ... and the CGI binary (cgi-bin) directory
47. “Scripting” languages
• No explicit compilation
• Cross platform
• Built-in, powerful text functions
• Do a lot in a little bit of code
• Perl, Python, PHP, Ruby
• Typically open source
48. Frameworks
• DRY (Don’t repeat yourself)
• Get rid of the drudgery
• Concentrate on your business, rather than
worrying about common Web issues
49. MVC paradigm
• Most modern frameworks use MVC
• From Smalltalk in the 1980s!
• Model — communicates with database
• View — HTML/JavaScript/CSS for user
• Controller — handles requests
• Clear division of labor
50. Web frameworks in
dynamic languages
• Programmer speed trumps execution
speed
• Community support
• Plugins for commonly requested features
• Create a domain-specific language (DSL)
for your specific needs
51. Ruby on Rails
• Ruby language
• Rails Web app framework (MVC)
• Designed for writing Web DSLs
• “Convention over configuration”
• Thousands of little improvements
• ActiveRecord — ORM
53. Where’s the definition?
• The computer takes care of it automatically
• ActiveRecord knows what columns you
have defined, and what their types are
• (More on columns later)
• Only write things that cannot be
understood automatically
54. Not only Rails
• Python (Django)
• PHP (Symfony)
• Perl (Catalyst)
• Groovy (Grails)
• Even if you don’t use Ruby on Rails, you
have benefitted from its “opinions”
55. Plugins
• Rails (and other systems) have open-source
plugins to handle common issues
• Authentication
• E-commerce
• Social networking
• Don’t write these yourself! Customize
existing code that has proved itself
56. Storage
• Applications are great!
• But what if we want to store information
about our users?
• Name, e-mail address, account balance
• We could use text files, but most people
will use a database
57. What is a database?
Store data
confidently
Database
Retrieve data
flexibly
58. Relational databases
Define tables,
store data in them
Database
Retrieve data from
related tables
60. Relational databases
• Everything is stored in 2-dimensional tables
• Data should appear only once (normalized)
• “Join” tables to connect tables
• Technology is extremely robust, fail-safe
• Not all data is a good fit for this paradigm
61. id first_name last_name phone
1 Reuven Lerner 054-496-8405
2 Charlie Kalech 02-671-9918
63. id first_name last_name email
1 Reuven Lerner reuven@lerner.co.il
2 Charlie Kalech charlie@j-town.com
person_id phone_number_type_id phone_number id type
1 2 054-496-8405
1 1 847-230-9795 1 work
2 1 02-671-9918
2 mobile
2 2 054-803-3356
2 3 501-629-8620 3 fax
4 home
64. id first_name last_name email
1 Reuven Lerner reuven@lerner.co.il
2 Charlie Kalech charlie@j-town.com
person_id phone_number_type_id phone_number id type
1 2 054-496-8405
1 1 847-230-9795 1 work
2 1 02-671-9918
2 mobile
2 2 054-803-3356
2 3 501-629-8620 3 fax
4 home
SELECT P.first_name, P.last_name, P.email,
PN.phone_number, PNT.type
FROM People P, Phone_Numbers PN,
Phone_Number_Types PNT
WHERE PN.person_id = P.id
AND PNT.id = PN.phone_number_type_id
65. Another language!
• SQL is the language of relational databases
• So a Web app will use a language (e.g.,
Ruby, Python, or PHP) + SQL
• Or use an ORM, which automatically
translates your language into SQL
Person.first.phone_number
76. Scaling problems
• Lots of requests?
• Optimize
• Even more requests?
• Buy a bigger server
• What next?
• Panic! (Or spend lots of money)
77. Sharding
• Split your data across multiple databases
• This works, but...
• Requires rewriting a lot of code
• Maintenance is a big issue
• Re-sharding as each server gets
overwhelmed can be expensive, time-
consuming
78. Cloud
• Don’t manage your own server!
• Rent a server by the hour, day, or month
• Lots of requests? Add more servers!
• You can even use a cloud database server
• No more scaling issues — let someone
else handle the headaches
79. Non-relational
databases
• Don’t store things in tables!
• Don’t pre-define a schema
• No SQL!
• Indeed, known as “NoSQL” databases
• Only common factor: No SQL, non-
relational
81. MapReduce / Hadoop
• Google and Yahoo do it like this:
• Split data across many servers
• Run a function on all of those servers
• Collect the results
• Display to the user!
• (Too slow? Add more servers!)
82. NoSQL: Good news
• Often easier to administer, configure
• Scaling to multiple servers is a no-brainer
• No new programming language (SQL)!
• Better fit for certain kinds of data
• Better performance than a relational DB
• Lots of options to choose from!
83. NoSQL: Bad news
• Speed is in the eye of the beholder
• Is “eventually consistent” good enough?
• Non-normalized data — ugh!
• Reporting can be harder
• Less tested than relational databases
• Can you trust your data to them?
84. Meanwhile...
• Our browsers are displaying HTML
• HTML had several problems:
• Standardized set of tags
• Making it easy for programs to parse
• Semantic, display content were mixed
85. HTML standards
• HTML — several versions, several
standards, none universally accepted
• XML — lets us create HTML-like
languages, for computer conversations
• XHTML — HTML for pedantic people
• It was a big mess!
86. HTML5
• Relaxes much of the formality of XHTML,
while remaining easy for computers to read
• Backward compatible to a large degree
• Adds a number of tags for better semantics
• Best of all: Lots of new JavaScript goodies
• More on this in a moment
98. And more
• Validation — built-in validation of form
element inputs, via regular expressions
• No more JavaScript plugins!
• Sliders — more natural numeric inputs
• Canvas — draw any picture you might like,
and detect/change it with software
• Hints in text fields (e.g., “search”)
99. My favorite
• Private data in attributes!
• Any attribute starting with “data-” is
considered valid
• A great way to stash information inside of
HTML elements without violating standards
100. Oh, yeah
• These don’t all work.
• Many of them don’t work on any browser
• Most work on only some browsers.
• What to do? Wait. Or use Modernizr,
which uses JavaScript to detect features.
• If a feature isn’t there, you can fall back
101. CSS
• Cascading Style Sheets
• Split semantic markup from presentation
• One CSS file can apply to an entire site
• No more “style” tags in your text!
• Easy to move place things, create effects
104. It gets better
• You can set styles for when the user’s
mouse hovers over or clicks on an element
• In other words: Cheap animation!
• Many uses of JavaScript (e.g., some menus)
can now be done with simple CSS
• You can make beautiful sites with CSS
105. CSS3: Cool effects
• Rounded corners
• Transparency
• Text shadows and drop shadows
• Gradients
106. CSS3: Cool selectors
• If you love regular expressions, then these
selectors will be second nature to you:
p[id=”foo”] { background: green}
p[id^=”foo”] { background: green}
p[id$=”foo”] { background: green}
p[id*=”foo”] { background: green}
107. CSS3: Cool selectors
• If you love regular expressions, then these
selectors will be second nature to you:
p[id=”foo”] { background: green}
Equals
p[id^=”foo”] { background: green}
p[id$=”foo”] { background: green}
p[id*=”foo”] { background: green}
108. CSS3: Cool selectors
• If you love regular expressions, then these
selectors will be second nature to you:
p[id=”foo”] { background: green}
Starts with
p[id^=”foo”] { background: green}
p[id$=”foo”] { background: green}
p[id*=”foo”] { background: green}
109. CSS3: Cool selectors
• If you love regular expressions, then these
selectors will be second nature to you:
p[id=”foo”] { background: green}
p[id^=”foo”] { background: green}
Ends with
p[id$=”foo”] { background: green}
p[id*=”foo”] { background: green}
110. CSS3: Cool selectors
• If you love regular expressions, then these
selectors will be second nature to you:
p[id=”foo”] { background: green}
p[id^=”foo”] { background: green}
p[id$=”foo”] { background: green}
Contains
p[id*=”foo”] { background: green}
111. OK, that’s nice.
• But really, the big news with HTML5
doesn’t have to do with HTML at all.
• Instead, it has to do with JavaScript.
• Remember JavaScript?
• It’s the programming language that we love
to hate. (At least, I used to.)
112. JavaScript
• Originally “LiveScript,” a language that
executes programs in the browser
• Renamed “JavaScript” when an unrelated
language (“Java”) stole the thunder
• Renamed (officially) ECMAScript for
standardization purposes
• No one actually calls it this
113. HTML5 turns the
browser into a smart,
powerful, networked
application platform.
JavaScript makes it
possible.
114. Powerful? Huh?
• Didn’t I just say that I love to hate
JavaScript?
• And then I said that it was powerful?
• What gives?
115. JavaScript is
the new hotness
• Browsers are competing for fastest, best
• Google’s V8
• Mozilla’s TraceMonkey (and JägerMonkey)
• Safari’s Nitro
• IE’s Chakra
• JavaScript is faster, more stable than ever!
116. Also: frameworks
• Remove the drudgery of JavaScript
• Handle differences between browsers
• Make it easy to perform common tasks
• Lots of plugins available
• For me, it’s the difference between pain and
pleasure when working with JavaScript
118. Ajax
• One reason for JavaScript libraries: Ajax!
• Make HTTP requests from within the page
• No refresh! Just get results from the
server, and modify the page accordingly
• This has revolutionized our view of Web
pages
120. Ajax
POST /login
name=reuven&password=secret
Browser Server
121. Ajax
POST /login
name=reuven&password=secret
Browser Server
200 OK
<html>...
<p>Thank you!</p>
...</html>
122. Ajax
POST /login
name=reuven&password=secret
Browser Server
200 OK
<html>...
<p>Thank you!</p>
...</html>
123. Ajax
POST /login
name=reuven&password=secret
Browser Server
200 OK
<html>...
<p>Thank you!</p>
...</html>
124. Ajax
POST /login
name=reuven&password=secret
Browser Server
200 OK
<html>...
<p>Thank you!</p>
...</html>
125. Ajax
POST /login
name=reuven&password=secret
Browser Server
200 OK
<html>...
<p>Thank you!</p>
...</html>
126. Ajax isn’t everything
• What if I want a chat application, or
something else that stays open?
• What if I want to execute more than one
JavaScript function at a time?
• What if I want to store things locally?
• HTML5 provides all this — and more!
127. Canvas
• A complete drawing area, in your browser!
• Use JavaScript to:
• Draw arbitrary shapes
• Detect the mouse
• Detect the drawing
• The end of Flash? Maybe...
128. Geolocation
• Your browser can know where you are!
• It can send this info to the server
• It’s not perfect, but still pretty good
• To avoid privacy issues, users are always
asked if their location should be shared
129. Inter-page
communication
• Modern Web apps can span multiple pages
• HTML5 makes it easy for two pages (from
the same server) to communicate
• The receiver knows which server sent the
data — so it can filter incoming messages,
as well as screen them for security
130. Web sockets
• This is potentially the biggest deal of all
• Ajax allows for server connections. But:
• High overhead
• Stateless
• Web sockets have low overhead, and they
stay open as long as you need!
133. Using Web sockets
var weatherSocket = new
WebSocket("ws://localhost:8080");
weatherSocket.onopen = function(e)
{ alert("Opened weather socket");};
134. Using Web sockets
var weatherSocket = new
WebSocket("ws://localhost:8080");
weatherSocket.onopen = function(e)
{ alert("Opened weather socket");};
weatherSocket.onmessage =
function(e) { alert("Got message: "
+ e.data);};
135. Using Web sockets
var weatherSocket = new
WebSocket("ws://localhost:8080");
weatherSocket.onopen = function(e)
{ alert("Opened weather socket");};
weatherSocket.onmessage =
function(e) { alert("Got message: "
+ e.data);};
weatherSocket.onclose = function(e)
{ alert("Closing socket..."); };
136. What can sockets do?
• Chat servers
• Stock feeds
• Teleconferencing
• Who knows?
• Remember, HTTP was only invented after
sockets had been around for 15 years
137. Web workers
• Execute more than one thing at a time
• In other words:You can run JavaScript
functions in the background
• Process text
• Open Web sockets
• Perform calculations
138. Local storage
• Now Web apps can store data
• A little database of name-value pairs
var foo = localStorage.getItem("bar");
localStorage.setItem("bar", foo);
139. Local storage
• Now Web apps can store data
• A little database of name-value pairs
var foo = localStorage.getItem("bar");
localStorage.setItem("bar", foo);
var foo = localStorage["bar"];
140. Local storage
• Now Web apps can store data
• A little database of name-value pairs
var foo = localStorage.getItem("bar");
localStorage.setItem("bar", foo);
var foo = localStorage["bar"];
localStorage["bar"] = foo;
141. Media
• Standard (well, sort of) ways to play audio
and video
• No more Flash!
• Problem: No one format is supported by all
browsers
142. jQuery
• jQuery has taken the world by storm
• Super-easy to use
• Extremely fast
• Open source (of course!)
• Easy to write plugins
• Lots of plugins are available
143.
144. Basic jQuery
• $(CSS_SELECTOR) returns one or more
objects
• Now what?
• Execute something on those objects
• Hang a function on those objects, to
execute when a condition is met
150. Plugins in jQuery
• jQuery’s syntax makes it trivial to do
complex operations
• Between powerful selectors and many
built-in functions, you can manipulate text,
HTML, and styles easily
• But even better — there are plugins that
do lots of things for you
151. Unobtrusive
• The jQuery is not in the HTML file
• This is called “unobtrusive JavaScript”
• Good HTML has no CSS, and no JavaScript
159. jQuery issues
• Lots of plugins — which ones are good?
• Welcome to the world of open source!
• You’re still defining functions
• Yes — and functions as data takes a while
to get used to
• Remember: JavaScript is programming, and
a different sort than most languages
160. Want a Web app?
• It used to be:
• “What operating system, language, and
database will I use?”
161. But today, it’s...
• What server language and framework?
JavaScript framework? Hosted or cloud?
• What type of database (relational, NoSQL)?
Which one? Hosted or cloud?
• Do we offer an API? A mobile app? Both?
• Which HTML5 features do we want to
use, and how do we gracefully degrade?
162. My brain is too small!
• Yes, there’s a lot to learn
• Most of it can wait a little bit
• There are oodles of tutorials
and books that can help you
• Besides a lot of this is still
highly transitional
163. Whew!
• There’s a lot to the modern Web
• It’s fun and exciting, and continues to move
forward at breakneck speed
• Understanding as many of these parts as
possible will help you make better
decisions, and better applications!