The document discusses using JRuby and Ruby for big data analytics on Hadoop. It mentions using JRuby for supercomputing on Hadoop with 120 EC2 instances and 1920 cores. It also discusses Rubydoop, an open source Ruby library for Hadoop streaming that allows writing MapReduce jobs in Ruby. The author notes that Rubydoop provides a low-level API and that a higher-level library like Scalding or Cascading built on top of it would be useful.

1. @iconara
CHASING THE ELEPHANT

2. Theo / @iconara

3. chief architect at BURT

4. big data analytics with JRuby

5. RUBY

6. RUBY

7. RUBY JRUBY

8. JRUBY IS AWESOME
BECAUSE RUBY IS GREAT,
AND THE JVM IS GREAT

9. hot_bunnies, eurydice, multimeter, mikka, msgpack-jruby

11. HADOOP

12. JRUBY SUPERCOMPUTING
40 TiB data, 120 EC2 cc2.8xlarge, 1920 cores, 7260 GiB RAM

13. JAVA ALL THE WAY DOWN

14. HADOOP STREAMING
Wukong, Dumbo

15. RUBYDOOP
rubydoop.org
+ =

18. main()

19. Class.forName("...")

20. Class.forName("...")

22. module WordCount
class Mapper
def map(key, value, context)
value.to_s.downcase.split.each do |word|
key = Hadoop::Io::Text.new(word)
value = Hadoop::Io::IntWritable.new(1)
context.write(key, value)
end
end
end
end

23. RUBYDOOP IS LOW LEVEL
I would love to see someone write something
like Scalding or Cascading on top of it

24. RUBYDOOP
rubydoop.org

25. RUBYDOOP
rubydoop.org
v1.1.0

26. KTHXBAI
@iconara
github.com/iconara
architecturalatrocities.com
burtcorp.com