1) Total order sorting is another kind of sorting technique, where map output keys are sorted across all the reducers. 2) This technique uses, where you want to extract the most popular URLs from a web graph. 1) By default Mapreduce uses HashPartitioner as its Partitioner class, which partitions using a hash of the map output keys. 2) Also HashPartitioner ensures that all records with the same map output key goes to the same reducer, but it doesn’t perform total sorting of the map output keys across all the reducers. 3) For this reason only TotalOrderPartitioner class is introduced, which is by default packed with the Hadoop distribution. 1) If you want to work with Total order sorting, we need to create Partition file, and then we have to run Mapreduce job using TotalOrderPartitioner class. 2) We will create partition file, by using InputSampler class, which is used to do sampling of the whole dataset. 3) There are basically two kinds of samplers that we mostly use. 4) First one is RandomSampler, which is mainly used to pick random samples from the original dataset. And the second one is, IntervalSampler, which is mainly used to pick the sample for every R number of records. In the practical demonstration I have used RandomSampler class to pick the samples from Original dataset. 5) Once all the meaningful samples are extracted from the dataset, it will sort those keys, and pick N-1 keys from those sorted keys where N is number of reducers and it places in a Partition file which is used for Total order sorting. 1) This is an overview of Total Order Sorting, here it show how it generates the Partition file and also it shows how the Mapreduce job uses this Partition file during Total Order Sorting. 1) This is a code Sample for Total Order Sorting, in this we have specified the sampler object as RandomSample class. And we also set the Number of reducers using setNumReduceTasks(). And also we specified the Partionfile location unsing setPartionfile() of TotalOrderPartitioner class. And at last we have used writePartitionFile() of InputSampler class for creating Partition file.

3. • What is Total Order Sorting and its use cases.
• What is Mapreduce default Partition behavior.
• Use of Sampling and TotalOrderPartitioner
• Overview of TotalOrderSorting Technique.
• Code sample for TotalOrderSorting.
• Practical Demonstration of TotalOrderSorting.

4. • Total Ordered Sorting is an another tweak behavior of Mapreduce sorting.
• Sorting Map output keys Across all the reducers is called Total order
Sorting.
• It uses MapReduce to sort keys across multiple reducers so that you can
easily extract the top and bottom N records from your data.
• if you want to extract the most popular URLs from a web graph you’ll have
to order your graph by some measure of popularity, such as Page-Rank.
• if you want to display a table in your portal of the most active users on
your site, you need the ability to sort them based on some criteria such as
the number of articles they wrote.
 All these scenarios uses Total Order Sorting.

5. • Mapreduce by default uses HashPartitioner as its Partitioner class to its
job, which partitions using a hash of the map output keys.
• HashPartitioner ensures that all records with the same map output key
goes to the same reducer, but it doesn’t perform total sorting of the map
output keys across all the reducers.
• For this concern, we required TotalOrderPartitioner class, which is by
default packed with the Hadoop distribution.

6. • To work with TotalOrderSorting, we need to create the partition file and
then run your MapReduce job using the TotalOrderPartitioner.
• Initially we need to use InputSampler class, TODO sampling of the input
dataset and creates the Partition file. There are two kinds of Samplers,
1) RandomSampler: which as the name suggests picks random records
from the input
2) IntervalSampler: which for every R record includes the record in the
sample
• Once the samples have been extracted, they’re sorted and then N-1 keys
are written to the partition file, where N is the number of reducers.
• Next up is specifying that you want to use the TotalOrderPartitioner as the
partitioner for your job:
job.setPartitionerClass(TotalOrderPartitioner.class);