Chirp hackday

Chirp Hack Day Report （株 ) デジタルガレージ和田浩一 [email_address] @Koichi

自己紹介 ,[object Object],[object Object],[object Object],[object Object],[object Object]

Chirp ,[object Object],[object Object],[object Object],[object Object]

1 日目 - 夕方 Hack Day Start

1 日目 - 深夜 Coding time

2 日目 -Lunch, Meet The founders

まとめ ,[object Object],[object Object],[object Object]

技術トピック ,[object Object],[object Object]

アプリが守るべきポリシー

守ってほしいポリシー ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

守ってほしいポリシー (2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

守ってほしいポリシー (3) ,[object Object],[object Object]

ポリシーまとめ ,[object Object],[object Object],[object Object],[object Object]

Twitter 上で生成されるデータ ,[object Object],Chirp でのこの講演中に２２５ GB

技術的 Challenge ,[object Object],[object Object],[object Object]

データ収集 ,[object Object],[object Object]

そこで ,[object Object],[object Object],[object Object],[object Object]

Scribe ,[object Object],[object Object],[object Object],[object Object],[object Object],書き込み書き込み書き込み集約集約ファイル HDFS

データの格納 ,[object Object],[object Object],[object Object],[object Object],[object Object]

そこで ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

解析 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

早く解析結果を得る ,[object Object],[object Object],[object Object],[object Object]

そこで ,[object Object],[object Object],[object Object],[object Object],[object Object]

Pig sample users = load ‘users.csv’ as (username: charaarray, age: int); users_1825 = filter users by age >= 18 and age <=25; pages = load ‘pages.csv’ as (username: chararay, url: chararray) joined = join users_1825 by username, pages by username; grouped = group joined by url; summed = foreach grouped generate group as url, COUNT(joined) AS views; sorted = order summed by views desc; top_5 = limit sorted 5; store top_t into ‘top_5_sites.csv’

Java に比べて簡単 ,[object Object]

まとめ ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

最後に ,[object Object],[object Object],[object Object]

Chirp hackday

Recommandé

Recommandé

Contenu connexe

Similaire à Chirp hackday

Similaire à Chirp hackday (20)

Plus de Open Network Lab

Plus de Open Network Lab (15)

Chirp hackday

Notes de l'éditeur