Interpreting the Data:Parallel Analysis with Sawzall

1. Tilani Gunawardena

3. “When the only tool you own is a hammer,
every problem begins to resemble a nail.” -
Abraham Harold Maslow

4. InputTime
Chubby
Bigtable
GFS
ProcessingTime
Sawzall
MapReduce
OutputTime
Bigtable
GFS

5. Large data set(large,dynamic, unwieldy)
Flat but regular structure
Span multiple disks and machines
Bottleneck lies in I/O, not CPUs
Paralyze to improve throughput
Task division and distribution
Keep computation near to data
Tolerance of kinds of failures

6. Process data records that are present on many
machines.
Distribute the calculation across all the
machines to achieve high throughput.
Two phases for calculation
-Analysis Phase
-Aggregation Phase

7. Five racks of 50-55 working computers each, with four disks per
machine.Such a configuration might have a hundred terabytes
of data to be processed, distributed across some or all of the
machines.

8. Query is commutative then can process in any order
If Aggregation is commutative and associative then intermediate
values can be grouped arbitrarily or even aggregated in stages
The overall flow of filtering, aggregating, and collating.
Each stage typically involves less data than the previous

9. Gigabyte to many terabytes of data
Hundreds or even thousands of machines in
parallel
Some executing the query while others
aggregate the results
An analysis may consume months of CPU time,
but with a thousand machines that will only
take a few hours of real time.

10. system’s design is influenced by two
observations.
If the querying operations are commutative across
records, the order in which the records are process
unimportant
If the aggregation operations are commutative, the
order in which the intermediate values are
processed is unimportant

11. Input is divided into pieces & processed separately
Sawzall interpreter is instantiated for each piece of
data
Sawzall program operates on each input record
individually
The output of the program for each record is the
intermediate value.
The intermediate value is combined with values
from other records

12. Input: Set of files that contain records where each
of the records contain one floating-point number.
Output:Number of records, sum of the values and
sum of the squares of the values.
count: table sum of int;
total: table sum of float;
sum_of_squares: table sum of float;
x: float=input;
emit count<—1;
emit total<—x;
emit sum_of_squares<- x*x;

13. Protocol Buffers
Google File System
Workqueue
MapReduce

14. Protocol Buffers are used
-To define the messages communicated between
servers
-To describe the format of permanent records stored
on disk
DDL describes protocol buffers and defines the
content of the messages
Protocol compiler takes the DDL and generates code
to manipulate the protocol buffers

15. The generated code is compiled and linked with
the application
Protocol buffer types are roughly analogous to C
structs but the DDL has two additional
properties
A distinguishing integral tag
An indication of whether a field is necessary or
optional

16. The following describes a protocol buffer with two required
fields. Field x has tag 1,and field y has tag 2
parsed message Point {
required int32 x = 1;
required int32 y = 2;
};
To extend this two-dimensional point, one can add a new,
optional field with a new tag. All existing Points stored on disk
remain readable; they are compatible with the new definition
since the new field is optional.
parsed message Point {
required int32 x = 1;
required int32 y = 2;
optional string label = 3;
};

17. The data sets are often stored in GFS
GFS provides a reliable distributed storage
system
It can grow to petabyte scale by keeping data in
64MB “chunks”
Each chunk is replicated, usually 3 times, on
different machines
stored on disks spread across thousands of
machines
GFS runs as an application-level file system with
a traditional hierarchical naming scheme

18. Software that handles the scheduling of a job
that runs on a cluster of machines.
It creates a large-scale time sharing system from
an array of computers and their disks.
It schedules jobs, allocates resources,reports
status, and collects the results

19. MapReduce is a software library for applications
that run on the Workqueue.
Primary Services
It provides an execution model for programs that
operate on many data items in parallel.
It isolates the application from the details of
running a distributed program,
When possible it schedules the computations so
each unit runs on the machine or rack that holds its
GFS data, reducing the load on the network.

20. High level language
Software libraries
Scheduling software
Application file system

21. Basic types
Integer (int)
Floating point (float)
Time
Fingerprint (represents an internally computed
hash of another value)
Bytes and String
Compound type
Arrays, Maps and Tuples

22. A Sawzall program defines the operations to be
performed on a single record of the data
There is nothing in the language to enable examining
multiple input records simultaneously
The only output primitive in the language is the
emit statement

23. Given a set of logs of the submissions to our source code management system, this
program will show how the rate of submission varies through the week, at one-
minute resolution:
proto “p4stat.proto”
submitsthroughweek: table sum[minute: int] of count: int;
log: P4ChangelistStats = input;
t: time = log.time; # microseconds
minute: int = minuteof(t)+60*(hourof(t)+24*(dayofweek(t)-1));
emit submitsthroughweek[minute] <- 1;
submitsthroughweek[0] = 27
submitsthroughweek[1] = 31
submitsthroughweek[2] = 52
submitsthroughweek[3] = 41
...
submitsthroughweek[10079] = 34

24. Frequency of submits to the source code
repository through the week.

25. List of Aggregators
Collection
Sample
Sum
Maximum
Quantile
Top
Unique

26. System operates in a batch-execution style
-User submits a job
-Job runs on a fixed set of files
-User collects the output
The input format and output destination are given as
arguments to the command that submits the job.
The command is called “saw”
The parameters for saw are as follows
saw --program code.szl
--workqueue testing
--input_files /gfs/cluster1/2005-02-0[1-7]/submits.*
--destination /gfs/cluster2/$USER/output@100

27. User collects the data using the following:
dump --source/gfs/cluster2/$USER/output@100 --
format csv
The program merges the output data and prints the
final results.

28. Implementation
-Sawzall works in the map phase
-Aggregators work in the reduce phase

29. Chaining
-The output from a Sawzall job is sent as an
input to another

30. Task: Process a web document repository to
know for each web domain, which page has the
highest page rank
proto "document.proto"
max_pagerank_url:
table maximum(1) [domain: string] of url: string
weight pagerank: int;
doc: Document = input;
emit max_pagerank_url[domain(doc.url)] <- doc.url
weight doc.pagerank;

31. Task: To look at a set of search query logs and
construct a map showing how the queries are
distributed around the globe
proto "querylog.proto"
queries_per_degree: table sum[lat: int][lon: int] of int;
log_record: QueryLogProto = input;
loc: Location = locationinfo(log_record.ip);
emit queries_per_degree[int(loc.lat)][int(loc.lon)] <- 1;

33. It runs on one record at a time
It is routine for a Sawzall job to be executing on
a thousand machines simultaneously, yet the
system requires no explicit communication
between those machines

34. Sawzall is statically typed
The main reason is dependability. Sawzall programs can
consume hours, even months, of CPU time in a single
run,and a late-arising dynamic type error can be
expensive.
Handle undefined values
Ex: A division by zero
Conversion errors
I/O problems
Using def() predicate
Run-time flag is set
Handle logical quantifiers

35. Similar to C and Pascal
Type-safe scripting language
Code is much shorter than C++
Pure value semantics, no reference types
Statically typed
No exception processing

36. Compare the single CPU speed of the Sawzall
interpreter.
Sawzall is faster than Python, Ruby and Perl. But
slower than interpreted Java, compiled Java and
compiled C++
The following table shows the microbenchmark.
Sawzall Python Ruby Perl
Mandelbrot 12.09s 45.42s 73.59s 38.68s
runtime Factor 1.00 3.75 6.09 3.20
Fibonacci 11.73s 38.12s 47.18s 75.73s
runtime Factor 1.00 3.24 4.02 6.46

37. The main measurement is not single-CPU speed.
The main measurement is aggregate system
speed as machines are added to process large
datasets.
Experiment: 450GB sample of compressed
query log data to count the occurrences of
certain words using Sawzall program.

38. Test run on 50-600 , 2.4GHz Xeon computers
Scales up
At 600 machines the aggregate throughput was 1.06GB/s of
compressed data or about 3.2GB/s of raw input
Additional machines add .98 machine throughput

39. -The solid line is
elapsed time; the
dashed line is the
product of
machines and
elapsed time.

40. Why put a language above MapReduce?
MapReduce is very effective; what’s missing?
Why not just attach an existing language such as Python to MapReduce?
To make programs clearer, more compact, and more expressive
Original motivation :parallelism
Separating out the aggregators
provides a model for distributed processing
Awk or or Python –user have to write the aggregators
Capture the aggregators in the language(& its environment) ->means that
the programmer never has to provide one, unlike when using MapReduce
Sawzall programs tend to be around 10 to 20 times shorter than the
equivalent MapReduce programs in C++ & significantly easier to write
Ability to add domain-specific features, custom debugging and profiling
interfaces,etc

41. how much data processing it does ?
March 2005.
Workqueue cluster
1500 Xeon CPUs
32,580 Sawzall jobs launched using an average of 220 machines
each
18,636 failures
The jobs read a total of 3.2×1015 bytes
of data (2.8PB) and wrote 9.9×1012 bytes (9.3TB)
The average job therefore processed about 100GB

42. Traditional data processing is done by storing the
information in a relational database and
processing it with SQL queries.
Sawzall -data sets are usually too large to fit in a
relational database
Sawzall is very different from SQL(combining a fairly
traditional procedural language with an interface to
efficient aggregators)
SQL is excellent at database join operations, while
Sawzall is not
Brook - language for data processing, specifically
graphical image processing ;Brook [3]

43. Aurora-stream processing system that supports
a (potentially large) set of standing queries on
streams of data ; Aurora [4]
Hancock-stream processing system; Hancock [7]
concentrates on efficient operation of a single
thread instead of massive parallelism.

44. Some of the larger or more complex analyses would be helped by
more aggressive compilation, perhaps to native machine code.
Interface to query an external database
More direct support of join operations
Some analyses join data from multiple input sources, Joining
is supported but requires extra chaining steps
A more radical system model
To eliminate the batch-processing mode entirely

45. Provide an expressive interface to a novel set of aggregators that
capture many common data processing and data reduction
problems
Ability to write short, clear programs that are guaranteed to
work well on thousands of machines in parallel
user needs to know nothing about parallel programming;
CPU time is rarely the limiting factor; most programs are small
and spend most of their time in I/O and native run-time code
Scalability-linear growth in performance as we add machines
Big data sets need lots of machines; it’s gratifying that lots of machines
can translate into big throughput.

50. #/bin/bash
limit=100
sum=0
i=0
while [ "$i" -le $limit ]
do
sum=`expr $sum + $i`
i=`expr $i + 1`
done
echo $sum
50

51. #define NUMCALC 25
#include <stdio.h>
#include <mpi.h>
int main (int argc, char *argv[])
{ int myrank, sum, n, partial;
MPI_Status status;
sum = 0;
MPI_Init (&argc, &argv);
MPI_Comm_rank (MPI_COMM_WORLD, &myrank);
if (myrank == 0)
{ printf ("starting...n");
for (n=1; n<=NUMCALC; n++)
{ MPI_Recv (&partial, 1, MPI_INT, n, 0, MPI_COMM_WORLD, &status);
sum += partial;
}
printf ("sum is: %dn", sum);
}
else
{ for (n=1; n<=4; n++)
sum += ((myrank-1)*4+n);
MPI_Send (&sum, 1, MPI_INT, 0, 0, MPI_COMM_WORLD); http://digital.cs.usu.edu/~scott/5200_f02/sum.c
}
MPI_Finalize();
return 0;
51
}