LocationPowers OGC BigGeoData 2016 This presentation will discuss tools in the open source landscape that are used to handle big geospatial data. In particular, we will focus on how Apache frameworks such as Spark and Accumulo are "geospatially enabled" by four projects: GeoTrellis, GeoWave, GeoMesa, and GeoJinni. These four projects all participate in LocationTech, a working group under the Eclipse Foundation. In particular, we will discuss how each of these LocationTech technologies implement spatial indexing (e.g. by using space filling curves) in order to provide quick access to data, and other common themes among the four projects. Attendees should walk away from this presentation understanding important parts of the Apache big data ecosystem, a set of LocationTech projects that belong to the cutting edge of enabling those Apache project's handling of geospatial data, as well as some solutions to common problems when dealing with large geospatial data.

1. Rob Emanuele
ENABLING ACCESS TO BIG
GEOSPATIAL DATA WITH
&

2. What we’ll be covering…
LocationTech projects that geospatially enable
Apache big data frameworks by providing spatial
indexing.
Discuss how those four project approach
indexing, focusing on the use of space filling
curves.

3. STORING AND PROCESSING
GEOSPATIAL DATA @ SCALE

4. STORING AND PROCESSING
GEOSPATIAL DATA @ SCALE

8. WHAT IS ?

13. GEOJINNI
(FORMERLY SPATIALHADOOP)

17. SPACE FILLING CURVES

20. 00 01
1011
10
11 00
01
11
10
00
01
Hilbert Index (52) =
11 01 00

24. Geo +
accessed through

25. Z curve

26. Z curve
(also XZ)

28. Geo +
accessed through
GEOWAVE

29. Hilbert Curve

30. Range Decomposition
70 -> 75
92 -> 99
116 -> 121

32. False positives - secondary filtering

34. Geo +
Rasters +

38. Z or Hilbert

40. Data
Node
Data
Node
Data
Node
Name
Node
Master
Tablet
Server
Tablet
Server
Tablet
Server
Accumulo
BigTable clone (columnar database)
Records stored on HDFS
Lexicographically sorted table index

44. partition
id
split
id

45. split
id
partition
id

46. Tiered Indexing

47. Tiered Indexing

48. Periodicity (time dimension)
1997 1998 1999

50. Periodicity (arbitrary dimensions)
Time
Elevation
Velocity

51. Spatial index stored per file on HDFS
Z order (2D and 3D),
Hilbert (N-Dimensional)
Z order (2D and 3D)
Binned per week for spatiotemporal
N-Dimensional Hilbert with
arbitrary binning and tiered indexing
Spatial Indexing

53. CQL

54. Future integration work
?

55. THANK YOU
@lossyrob
gitter.im/geotrellis/geotrellis
github.com/geotrellis/geotrellis
remanuele@azavea.com

56. GeoMesa
GeoWave

57. Tiered Indexing