1. Multiple XML data load to Hive Environment
Daily region wise Sales XML data file from Univlever Group :
File name and path : homehadoopUnilever_RAW_data market.xml
<Table><State>Telengana</State><District>kurnool</District><Market>Fruit-
Market</Market><Commodity>Bathing-Soap(Lux)</Commodity><Variety>Bath-
soap</Variety><Arrival_Date>24/04/2016</Arrival_Date><Min_x0020_Price>25</Min_x0020_Pr
ice><Max_x0020_Price>28</Max_x0020_Price><Modal_x0020_Price>29</Modal_x0020_Price>
</Table>
<Table><State>Telengana</State><District>kadapa</District><Market>sanjay
market</Market><Commodity>Bathing-Soap(Margo)</Commodity><Variety>Bath-
soap</Variety><Arrival_Date>25/04/2016</Arrival_Date><Min_x0020_Price>24</Min_x0020_Pr
ice><Max_x0020_Price>30</Max_x0020_Price><Modal_x0020_Price>30</Modal_x0020_Price>
</Table>
<Table><State>Andra Pradesh</State><District>chitoor</District><Market>TTD
Market</Market><Commodity>Laddu(Spl made with
Ghee)</Commodity><Variety>Ladoo</Variety><Arrival_Date>25/04/2016</Arrival_Date><Min_
x0020_Price>25</Min_x0020_Price><Max_x0020_Price>100</Max_x0020_Price><Modal_x002
0_Price>120</Modal_x0020_Price></Table>
<Table><State>Andra Pradesh</State><District>Anantapur</District><Market>Vishal
Market</Market><Commodity>Tyres(MRF, CEAT, Zeal)
</Commodity><Variety>Tyre</Variety><Arrival_Date>25/04/2016</Arrival_Date><Min_x0020_
Price>12000</Min_x0020_Price><Max_x0020_Price>25000</Max_x0020_Price><Modal_x0020
_Price>c1200</Modal_x0020_Price></Table>
File name and path : homehadoopUnilever_RAW_data market1.xml
<Table><State>Andra Pradesh</State><District>Nellore</District><Market>Big
Theaters</Market><Commodity>cinemas</Commodity><Variety>cienmas(Hindi,English,Telegu)
</Variety><Arrival_Date>24/07/2015</Arrival_Date><Min_x0020_Price>100</Min_x0020_Price
><Max_x0020_Price>150</Max_x0020_Price><Modal_x0020_Price>BiG-
C</Modal_x0020_Price></Table>
<Table><State>Andra Pradesh</State><District>Guntur</District><Market>chilli
Market</Market><Commodity>chilli</Commodity><Variety>clilli(Red)</Variety><Arrival_Date>
24/07/2015</Arrival_Date><Min_x0020_Price>50</Min_x0020_Price><Max_x0020_Price>100</
Max_x0020_Price><Modal_x0020_Price>RED-C</Modal_x0020_Price></Table>

2. Objective :
The aim of the exercise is to analyse the data and generate sales trends like 52 weeks
high/low, month by month trends, state wise trends and overall price fluctuations for
various products and store the final output in form of JSON documents in NoSQL
database MongoDB.
Solution
Step 1:
hadoop fs -mkdir /xml/data/commodity
hadoop fs -copyFromLocal *.xml /xml/data/commodity
Step 2:
Need to download hivexmlserde-1.0.5.3.jar; [ By default this jar will not be
available in hive/lib folder, for that reason need to download and add the below
jar ...]
http://mvnrepository.com/artifact/com.ibm.spss.hive.serde2.xml/hivexmlserde/1.
0.5.3
hive> add jar /home/hadoop/apache-hive-1.2.1-bin/lib/hivexmlserde-1.0.5.3.jar;

3. Objective :
Step 3:
CREATE TABLE xml_items(State STRING, District STRING, Market
STRING, Commodity STRING, Variety STRING,
Arrival_Date STRING, Min_x0020_Price FLOAT, Max_x0020_Price FLOAT,
Modal_x0020_Price FLOAT)
ROW FORMAT SERDE 'com.ibm.spss.hive.serde2.xml.XmlSerDe'
WITH SERDEPROPERTIES (
"column.xpath.State"="/Table/State/text()",
"column.xpath.District"="/Table/District/text()",
"column.xpath.Market"="/Table/Market/text()",
"column.xpath.Commodity"="/Table/Commodity/text()",
"column.xpath.Variety"="/Table/Variety/text()",
"column.xpath.Arrival_Date"="/Table/Arrival_Date/text()",
"column.xpath.Min_x0020_Price"="/Table/Min_x0020_Price/text()",
"column.xpath.Max_x0020_Price"="/Table/Max_x0020_Price/text()",
"column.xpath.Modal_x0020_Price"="/Table/Modal_x0020_Price/text()"
)
STORED AS
INPUTFORMAT 'com.ibm.spss.hive.serde2.xml.XmlInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat'
TBLPROPERTIES (
"xmlinput.start"="<Table>",
"xmlinput.end"="</Table>"
);

4. Step 5:
hive> load data inpath '/xml/data/commodity/*.xml' OVERWRITE INTO
TABLE xml_items;
hive> select * from xml_items;

5. Step 6:
We can perform any hive query (example select sum, max,min...)

6. Step 6:
We can perform any hive query (example select sum, max,min...)