This talk is about how to use Foreign Function Interface (FFI) with Ruby and Go programs.

1. GoFFIng around with Ruby
@gautamrege

2. @joshsoftware

4. FFI

5. Foreign Function
Interface
*In stdlib since Ruby 1.9.3

6. Why do we need FFI?
• Resque / Sidekiq / delayed_job
• 3rd party API requests
• Sending newsletters
• Heavy computation
• Fast searches, aggregation.

7. Why Go?
• Power of multi-core.
• Goroutines to the rescue
• Language is restricted focussed.
• Programming Ethics
• Compiled
If it compiles, it
works

8. What is FFI anyway?
• Language interoperability!
• Calling convention.
• libffi.so
• ruby-ffi
• Fiddle (in Stdlib since 1.9.3)

9. FFI or Good ol’ C-ext?
• FFI Easier to write and maintain.
• Gems are now easier to install -
don’t need build dependencies.
• Portable code
• No more “Building native
extensions…” while installing
gems.

10. Simple Ruby Code
def race(legs, benchmark)
tids = []
legs.times do |i|
tids << Thread.new(i) do |leg|
array = (1..benchmark).map { |i| i }
p "Done leg #{leg} benchmark #{benchmark}"
end
end
tids.each { |t| t.join }
end
race(4, 50_000_000)

11. Benchmark - Ruby
$ time ruby run1.rb
Done leg 1 benchmark 50000000
Done leg 0 benchmark 50000000
Done leg 3 benchmark 50000000
Done leg 2 benchmark 50000000
real 0m18.242s
user 0m16.626s
sys 0m1.292s

12. Simple Go Code
package main
import "fmt"
func race(leg int, benchmark int) {
arr := []int{}
for i := 0; i < benchmark; i++ {
arr = append(arr, i)
}
fmt.Println("Done leg:", leg)
}
func main() {
for i := 0; i < 4; i++ {
race(i, 50000000)
}
}

13. Simple Go Code
package main
import "fmt"
func race(leg int, benchmark int) {
arr := []int{}
for i := 0; i < benchmark; i++ {
arr = append(arr, i)
}
fmt.Println("Done leg:", leg)
}
func main() {
for i := 0; i < 4; i++ {
race(i, 50000000)
}
}

14. Simple Go Code
package main
import "fmt"
func race(leg int, benchmark int) {
arr := []int{}
for i := 0; i < benchmark; i++ {
arr = append(arr, i)
}
fmt.Println("Done leg:", leg)
}
func main() {
for i := 0; i < 4; i++ {
race(i, 50000000)
}
}

15. Simple Go Code
package main
import "fmt"
func race(leg int, benchmark int) {
arr := []int{}
for i := 0; i < benchmark; i++ {
arr = append(arr, i)
}
fmt.Println("Done leg:", leg)
}
func main() {
for i := 0; i < 4; i++ {
race(i, 50000000)
}
}

16. Benchmark - Go
$ time go run run1.go
Done leg: 0
Done leg: 1
Done leg: 2
Done leg: 3
real 0m2.774s
user 0m2.564s
sys 0m0.929s

17. Calling Go from Ruby
• Build a C library using Go code.
• Build a FFI extension in Ruby
• Call it !

18. c-shared Go library
package main
import (
"C"
"fmt"
)
func race(leg int, benchmark int) {
arr := []int{}
for i := 0; i < benchmark; i++ {
arr = append(arr, i)
}
fmt.Println("Done leg:", leg)
}
//export churn
func churn(leg C.int, benchmark C.int) {
for i := 0; i < int(leg); i++ {
race(i, int(benchmark))
}
}
func main() {
}

19. c-shared Go library
package main
import (
"C"
"fmt"
)
func race(leg int, benchmark int) {
arr := []int{}
for i := 0; i < benchmark; i++ {
arr = append(arr, i)
}
fmt.Println("Done leg:", leg)
}
//export churn
func churn(leg C.int, benchmark C.int) {
for i := 0; i < int(leg); i++ {
race(i, int(benchmark))
}
}
func main() {
}

20. c-shared Go library
package main
import (
"C"
"fmt"
)
func race(leg int, benchmark int) {
arr := []int{}
for i := 0; i < benchmark; i++ {
arr = append(arr, i)
}
fmt.Println("Done leg:", leg)
}
//export churn
func churn(leg C.int, benchmark C.int) {
for i := 0; i < int(leg); i++ {
race(i, int(benchmark))
}
}
func main() {
}
$ go build -o librun.so -buildmode=c-shared run2.go

21. Ruby FFI extension
require 'fiddle'
module Fun
def self.race(leg, benchmark)
librun = Fiddle.dlopen(‘./librun.so')
race = Fiddle::Function.new(
librun['churn'],
[Fiddle::TYPE_INT, Fiddle::TYPE_INT],
Fiddle::TYPE_VOID
)
race.call(leg, benchmark)
end
end

22. Call it!
require './fun'
Fun.race(4, 50_000_000)
$ time ruby run2.rb
Done leg: 0 Benchmark 50000000
Done leg: 1 Benchmark 50000000
Done leg: 2 Benchmark 50000000
Done leg: 3 Benchmark 50000000
real 0m2.776s
user 0m2.719s
sys 0m0.969s

23. Practical Example

24. Practical Example
Make Model Variant

25. Practical Example
Make Model Variant

26. Practical Example

27. Practical Example
Insurance
Vendor

28. Practical Example
Insurance
Vendor
68 columns

29. Doing the Math!
18 STATES
428 CARS
68 VENDORS
= 523,872 cells

30. If that was not enough …
For each of the
523,872 cells
Calculate Depreciation
Index in ElasticSearch

31. for cmake, v := range segments {
for model, j := range v {
for submodel, k := range j {
for insurance, segment := range k {
wg.Add(1)
go func(cmake string, model string,
submodel string, insurance string,
segment string) {
defer wg.Done()
// data := es_data{...}
// prepare_es_data(...)
es_add_depreciation(&data)
}(cmake, model, submodel, insurance, segment)
}
}
}
}
Practical Example
About 350,000 go-routines

32. for cmake, v := range segments {
for model, j := range v {
for submodel, k := range j {
for insurance, segment := range k {
wg.Add(1)
go func(cmake string, model string,
submodel string, insurance string,
segment string) {
defer wg.Done()
// data := es_data{...}
// prepare_es_data(...)
es_add_depreciation(&data)
}(cmake, model, submodel, insurance, segment)
}
}
}
}
Practical Example
About 350,000 go-routines

33. Performance Benefits

34. Performance Benefits
• Sidekiq: 2 GB memory & ~18 minutes

35. Performance Benefits
• Sidekiq: 2 GB memory & ~18 minutes
• Go FFI: 135 MB memory & 1 minute 13 seconds.

36. Performance Benefits
• Sidekiq: 2 GB memory & ~18 minutes
• Go FFI: 135 MB memory & 1 minute 13 seconds.
ElasticSearch
indexing took
48 seconds

37. Complexities of FFI
• Memory Management (alloc / free)
• Pointer management
• Fiddle::Pointer
• Fiddle::Closure
Don’t use them unless you HAVE to!

38. Resources
• Fiddle http://ruby-doc.org/stdlib-2.0.0/libdoc/fiddle/rdoc/
Fiddle.html
• FFI Core Concepts https://github.com/ffi/ffi/wiki/Core-
Concepts
• Github: ffi/ffi
• Effective Go ( https://golang.org/doc/effective_go.html )
• A Tour of Go ( https://tour.golang.org/welcome/1 )

39. @joshsoftware
@gautamrege
http://www.codecuriosity.org