7. Y(AHOO!) Q(UERY) L(ANGUAGE)
SQL-like syntax
Query, filter, and join
across data sources
Not limited to Yahoo!
services
Easy to use web console
and table editor
Create custom tables
with your own server-
side logic using the
JavaScript execute
environment
8. YQL, CONT’D
Compatible
Accepts XML, JSON, CSV
Returns XML, JSON, JSON-P
High Performance
Deployed around the globe
Front and Back facing caches
99.5%+ performance uptime target
Rate-limited
/v1/public/* - 2,000 requests/hour per IP
/v1/yql/* - 20,000 requests/hour per IP
Per application limit (identified by your Access Key):
100,000 requests/day
9. QUERY SYNTAX
Select {what} from {service} where {condition}
Supports Select, Insert, Update, Delete
Supports Joining with sub-selects
Includes helper functions (sort, reverse, unique)
Uses ‘|’ symbol
24. 4 STEPS TO BUILD YOUR OWN TABLE
1. Define your table’s properties
2. Implement custom logic using JavaScript
execute
3. Save your table to our servers
4. Call your new table from your application
25. PIG LATIN
• A language game played with the English
language
• Popular with elementary school students
• Simple, algorithmic conversion
• pig = igpay
• computer = omputercay
• another = another-way
34. RECAP – BUILDING A HACK WITH YQL
Pick your data source
Mix and match community tables in the console
Build new a table if needed in the editor and test
in the console
Use YUI-YQL module to get data
Or just call the REST query endpoint
One of the biggest hurdles at Hack events is data, and how to get it. The most important part of your application is the data. It’s the data that keeps your user’s coming back for more. Without it, all you’ve got is that snazzy UI.
Individual companies build up massive virtual silos of data. Some companies make their data available over Web services, while others do not.
But, the great thing is that the internet is really just a series of tubes and all those silos are connected by these tubes. Some tubes lead to RSS feeds, others to RESTful APIs, while some just lead to websites.
It is our goalas hackers and makers to find new and interesting ways to combine the data available online to create new and interesting applications. This is where YQL comes in. YQL helps you to spend less time worrying about the plumbing of your application, and more time taking that data and creating value for your users.
YQL stands for Yahoo! Query Language. It is an SQL-like language that allows you to query, filter, and join data across any number of different data sources. YQL is not limited to Yahoo! services, but we are nice and have already provided a number of freely accessible tables to get you started. You can also test out the tables on our easy to use web console. You can easily create your own tables which implement your own custom logic, allowing you to mash-up data to your hearts content, any way you would like to (More on this later).
YQL is highly compatible. It is able to ingest data from XML, JSON, and CSV sources and then return XML, JSON, or JSON-P. YQL is deployed into our colos around the world. There are multiple levels of cache that you are able to use as a developer. There is a user-facing caching proxy server, an internal colo-wide memory cache, as well as a squid cache to capture responses from any backend APIs you call. All of these are controllable from with-in the JavaScript execute environment.YQL has a 99.5% performance uptime target, and can be used for commercial purposes, though it is rate limited. However, the YQL environment provided for use today is not rate limited.
The syntax used in YQL is very simple, and should look very familiar if you have done any work with SQL databases.
The best place to start learning how to use YQL is the YQL console.
YQL supports 3 output types, JSON, JSON-P and XML. You can select which data type you would like here.The diagnostics mode toggle will tell the YQL server to output additional diagnostic information, including any information on any backend services it connected to, how many resource units were consumed, and any log statements from custom written server-side logic.Debug mode will enable additional logging.Creating a query alias will allow you to more easily re-run the same query at a later date without having to type it all in again.This is the URL that your external application can call to execute the the last query that was run on this console.
What if we wanted to do a simple aggregation of the RSS feeds from TechCrunch and AllThingsD? This is simple with the feed table.
We can use projection to only grab the data we need, and also use the built-in sort function to sort the entries by publication date.
Pinterest doesn’t currently have a public API for it’s top 50 pins. But we can use the HTML table to make one.
The first thing you need to do to be able to use the Flickr table is to grab a free Flickr API key. If you don’t already have one, you can apply for one at the link on the screen now.
You’ll notice that this table doesn’t provide us with the URL to any of the images.
To get the URLs, we need to call a different table. We can do this by using the photo search table as a sub-query to the photo sizes table. This will grab the URL for each of the photos in our search results.
To help developers with the first 3 steps in developing their own YQL tables, we’ve developed the YQL editor.
If you notice, this table is not using the URL block shown on the previous slide. This is because instead of calling another external web service, we are going to call a different community YQL table.
jsonCompat prevents JSON numbers being turned into strings, and preserves single-item arrays.