1. dataviz on
bordeaux open data
+ elasticsearch
+ d3js
+ angular.js + google refine
mardi 19 mars 13
2. goal
• full stack dataviz
• front js centric with angular.js and sass
• d3.js -> best lib for dataviz (a bit complex)
• elasticsearch scalable search engine access
form js ajax client
• grab open data and reformat with google
refine
mardi 19 mars 13
3. philosophy
• smarter and smarter browsers will grab a
huge part of the stack
• scalable NoSQL solution talk REST: super
easy access
• you always can enrich, refine, store, model
data from anysource: yes you can !
mardi 19 mars 13
4. go open data !
• go to http://opendata.bordeaux.fr/
• we focus ou capc contemporary museum
of bordeaux
• http://opendata.bordeaux.fr/content/
collections-du-capc-musee-dart-
contemporain
• curl https://
bdxconfigogdi.blob.core.windows.net/
converteddata/capc.csv -o capc.csv
mardi 19 mars 13
5. google refine
• data many times isn’t perfect: humans
input, unsuitable model...
• with google refine you can tidy data, enrich
with web services call and so on...
• i think you should never think you don’t
have the good data for the job
• be smart and be creativ: you have
everything you need, all the time !
mardi 19 mars 13
6. import data in
elasticsearch
• bulk api import for es is handy
• http://www.elasticsearch.org/guide/
reference/api/bulk.html
• we use templating in google refine to
export data to the correct format
• { "index" : { "_index" : "musees", "_type" :
"capc"} } [snipp]
mardi 19 mars 13
7. elasticsearch
• scalable search engine
• adding more power == adding more nodes
• sharding
• replication
• fault tolerant
mardi 19 mars 13
8. elasticsearch
• store unstructured document (json) in
indexes (NoSQL way)
• talk REST (api)
• advanced query langage
• multiple analyzers (tokens, languages, etc...)
• blazing fast !
• no alternativ solutions (to my advice) (and
kimshy advice too ;) )
mardi 19 mars 13
9. yeoman
• perfect tool for the new browser focus
stack coming soon
• yeoman init angular
• yeoman init angular:route capc
• yeoman install d3
• yeoman install jquery
• yeoman server -> yeah !!!
mardi 19 mars 13
10. angular.js
• mvc framework for browser (js)
• by google
• rely better on html doc
• cleaner controller handling than backbone
(to my advice)
mardi 19 mars 13
11. d3.js
• best javascript lib for dataviz (to my advice)
• slow learning curve
• based on svg browser markup
• data and dom oriented
• generic (go low level with svg easily)
mardi 19 mars 13
12. dataviz
• make appears informations from pure data
• you should explore, analyse and be creativ
to grab the most value possible from the
data: go as far as you can
• tables (excell way) -> forgive
• visualization will reveal your data: data is
made to be know to take decision, manage,
understand
mardi 19 mars 13
13. open data
• today, wide range of datas come to the
public domain
• again, without computing and vizualisation,
data has a poor value
• open data without data scientists and data
visualization has no futur
mardi 19 mars 13
14. futur
• with low bandwith, storage and computing
cost, you can grab open data from a lot of
sources
• you can put them in big data store and
make analysis, relations between, with open
source technologies
• you can then share this to the word with
data visualization on your website, blogs...
mardi 19 mars 13
• amazing isn’t it ?? !!!!!
