Excel Data Analysis: Regression, T-Tests, Pivot Tables & Dashboards

Today’s Agenda
- Statistics, Hypothesis
- Regression Analysis, Trend Line [Practical x 02]
- T-Test [Practical]
- Solver [Practical]
- Goal Seek [Practical x 02]
- Idea : Z-Test, ANOVA Test, P-Value
- Pivot Table and Interactive Dashboard, Charts [Practical x 02]
- Pivot Table with Open Office Platform
- Dashboard
20-Jul-20
All the contents collected from internet, is mentioned with
sources at the bottom slide
2

9/16/2020 3
Elementary Data Analysis with MS Excel
Day-05

Statistics
• The main purpose of statistics is to test a hypothesis. For example,
you might run an experiment and find that a certain drug is effective
at treating headaches. But if you can’t repeat that experiment, no one
will take your results seriously.

Hypothesis
Hypothesis testing helps identify ways to reduce costs and improve
quality. Hypothesis testing asks the question: Are two or more sets of
data the same or different, statistically.
A hypothesis is an educated guess about something in the world
around you. It should be testable, either by experiment or
observation. For example:
• A new medicine you think might work.
• A way of teaching you think might be better.
• A possible location of new species.
• It can really be anything at all as long as you can put it to the test.

Hypothesis
Three Types of Hypothesis Tests:
• Classical Method - comparing a test statistic to a critical value
• p value Method - the probability of a test statistic being contrary to
the null hypothesis
• Confidence Interval Method - is the test statistic between or outside
of the confidence interval

Hypothesis
Conducting a Hypothesis Test:
• Define the null (H0) and an alternate (Ha) hypothesis
• Conduct the test
• Using data from the test:
• Calculate the test statistic and the critical value (t test, f test, z test, ANOVA,
etc.)
• Calculate a p value and compare it to a significance level (a) or confidence
level (1-a)
• Interpret the results to determine if you can accept or reject the null
hypothesis

Hypothesis-
in Lean Management/Manufacturing
• In Six Sigma, hypothesis tests help identify differences between
machines, formulas, raw materials, etc. and whether the differences
are statistically significant or not. Without such testing, teams can run
around changing machine settings, formulas and so on causing more
variation. These knee-jerk responses can amplify variation and cause
more problems than doing nothing at all.
• In manufacturing, you might want to compare two or more types of
raw materials and determine if they produce the same quality. In
other words, do the products have the same or different means and
variances? If they are the same, which one is less expensive to
produce? If they are different, which one best meets the customer's
requirements?

Hypothesis-Test Cheat Sheet
Hypothesis Test Compare Result What Does It Mean?
Classical
Method
test statistic > critical value
(i.e. F > F crit)
Reject the null hypothesis Means or Variances are Different
Classical
Method
test statistic < critical value
(i.e. F < F crit)
Cannot Reject the null
hypothesis
Means or Variances are the Same
p value Method p value < a Reject the null hypothesis Means or Variances are Different
p value Method p value > a
Cannot Reject the null
hypothesis
Means or Variances are the Same

Enabling: Data Toolpak (cont.)

Data Analysis: Regression Analysis
• Linear regression is a statistical technique that examines the linear
relationship between a dependent variable and one or more
independent variables.
• Linear relationship means the change in an independent variable(s)
causes a change in the dependent variable.
• There are basically two types of linear relationships as well:
• Positive Linear Relationship: When independent variable increases, the
dependent variable increases too.
• Negative Linear Relationship: When independent variable increases, the
dependent variable decreases.

Data Analysis: Regression Analysis (cont.)
• These were some of the pre-requisites before you actually proceed
towards regression analysis in excel.
• There are two basic ways to perform linear regression in excel using:
• Regression tool through Analysis ToolPak
• Scatter chart with trendline
• There is actually one more method which is using manual formula’s to
calculate linear regression.

• Explanation of Regression Mathematically
We have a mathematical expression for linear regression as below:
Y = aX + b + ε
Where,
• Y is a dependent variable or response variable.
• X is an independent variable or predictor.
• a is the slope of the regression line. Which represents that when X changes, there is a change in Y
by “a” units.
• b is intercepting. It is the value Y takes when the value of X is zero.
• ε is the random error term. Occurs because the predicted value of Y will never be exactly the
same to the actual value for given X. This error term, we don’t need to worry about. As there are
some software’s which do the calculation of this error term in the backend for you. Excel is one of
that software.

In that case, the equation becomes,
Y = aX + b
Which can be represented as:
Weight = a*Height + b
We’ll try to find out the values of these a and b using methods we have
discussed above.

• #1 – Regression Tool Using Analysis ToolPak in Excel
• We’ll try to fit regression for Weight values (which is dependent
variable) with the help of Height values (which is an independent
variable).

• In the excel spreadsheet, click on Data Analysis (present under
Analysis Group) under Data.
• Search out for Regression. Select it and press ok.

• Use the following inputs under Regression pane which opens up.
• Input Y Range: Select the cells which contain your dependent variable
(in this example B1:B11)

• Input X Range: Select the cells which contain your independent
variable (in this example A1:A11).
• Check the box named Labels if your data have column names (in this
example we have column names).
• Confidence Level is set to 95% by default, which can be changed as
per users requirements.
• Under Output options, you can customize where you want to see the
output of regression analysis in Excel. In this case, we want to see the
output on the same sheet. Therefore, given range accordingly.

• Under Residuals option, you have optional inputs like Residuals,
Residual Plots, Standardized Residuals, Line Fit Plots which you can
select as per your need.
• Under Normal Probability option, you can
select Normal Probability Plots which can
help you check the normality of predictors.
Click on OK.

• Excel will compute Regression analysis for you in a fraction of
seconds.

• One important part of this entire output is R Square/ Adjusted R
Square under SUMMARY OUTPUT table. Which provides information,
how good our model is fit. In this case, R Square value is 0.9547.
Which interprets that the model has a 95.47% accuracy (good fit). Or
in another language, information about Y variable is explained 95.47%
by X variable.

• The other important part of the entire output is a table of coefficients. It
gives values of coefficients which can be used to build the model for future
predictions.
• Now our, regression equation for prediction becomes:
• Weight = 0.6746*Height – 38.45508 (Slope value for Height is 0.6746… and
Intercept is -38.45508…)
• We have defined a function in which you now just have to put the value of
Height and you’ll get the Weight value.

#2 – Regression Analysis Using Scatterplot with Trendline in Excel
• Select your entire two columned data
(including headers).
• Click on Insert and select Scatter Plot
under graphs section as shown in the
image below.

• See the output graph.

• Now, we need to have a least squared regression line on this graph.
To add this line, right click on any of the data points on the graph and
select Add Trendline option.

• It will enable you to have a trendline of least square of regression like
below.

• Under the Format Trend line option, check the box for Display
Equation on Chart.

• It enables you to see the equation of least squared regression line on
the graph.
• This is the equation using which we can predict the weight values for
any given set of Height values.

• It is always recommended to have a look at residual plots while you
are doing regression analysis using Data Analysis ToolPak in Excel. It
gives you a better understanding of the spread of the actual Y values
and estimated X values.
• Simple Linear Regression in excel does not need ANOVA and Adjusted
R Square to check. These features can be taken into consideration for
Multiple Linear Regression.

Data Analysis: T-Test Analysis
• A t-test is returning the probability of the tests. Look at the below
data of two teams scoring pattern in the tournament.

Data Analysis: T-Test Analysis (cont.)
• Step 1: Select the Data Analysis option under the DATA tab.

• Step 2: Once you click on Data Analysis you will see a new dialogue
box. Scroll down and find T-test. Under T-test, you will three kinds of
T-test, select the first one i.e. t-Test: Paired Two Sample for Means.

• Step 3: After selecting the first t-Test you will see below options.

• Step 4: Under Variable 1 Range, select team 1 score and under
Variable 2 Range select team 2 score.

• Step 5: Output Range selects the cell where you want to display the
results.

• Step 6: Click on Labels because we have selected the ranges including
headings. Click on Ok to finish the test.

• Step 7: From the D1 cell it will start showing the test result

• Analysis:
• The result will show the mean
value of two teams, Variance
Value, how many observations are
conducted or how many values taken
into consideration, Pearson
Correlation etc…
• If you P (T<=t) two tail it is 0.314 which is higher than the standard
expected P value of 0.05. This means data is not significant.
• We can also do the T-test by using built-in function T.TEST.

Data Analysis: Solver
• A solver is nothing but solving the problem. SOLVER works like a goal
seek in excel.
• Look at the below image I have data of product units, units price,
total cost, and the total profit.

Data Analysis: Solver (cont.)
• Units sold quantity is 7550 at a selling price of 10 per unit. Total cost
is 52500 and the total profit is 23000.
• As a proprietor, I want to earn a profit of 30000 by increasing the unit
price. As of now, I don’t know how much units price I have to
increase. SOLVER will help me to solve this problem.

• Step 1: Open SOLVER under DATA tab.

• Step 2: Set the objective cell as B7 and
the value of 30000 and by changing the
cell to B2. Since I don’t have any other
special criteria’s to test I am clicking on
the SOLVE button.

• Step 3: The Result will be as below:
So, To make a profit of 30000 I need to sell the products at 11 per unit instead of 10 per unit.

Data Analysis: Goal Seek
• A Goal Seek is a tool that is used to find an unknown value from a set
of known values.
• It comes under the What If Analysis feature of Microsoft Excel, which
is useful to find out the value that will give the desired result as a
requirement.
• This function instantly calculates the output when the value is
changed in the cell. You have to mention the result you want the
formula to generate, and then determine the set of input values that
will generate the result.

Data Analysis: Goal Seek (cont.)
• Example #1
• In the above figure, there are two numbers A and B with the value of
9 and 6 respectively. a product is done using the function
=PRODUCT(B1,B2), resulting to 54.
• If A is 9, what will be the second number for B, to get the result, 72.

• Following are the steps:
• Click on Data Tab
• Under Data tools group
• Click on What if Analysis drop-down menu
• Click on Goal Seek

• In the Goal Seek dialog Box, select B3 in the ‘Set Cell’
• Enter 72 in the ‘To Value’
• Select B2 in ‘By Changing Cell’

• The Result is:

• Example #2
Let us take the example of EMK Ltd. Trading with generators. Price of each generator is BDT 18,000 and the
quantity sold is 100 nos.

• We can see that the company is suffering a loss of 13.8 lacs. It is
identified that the maximum price, for which a generator can be sold,
is BDT 18000. Now, It is required to identify the no. of generators
can be sold, which will return the break-even value (No Profit No
Loss). So the Profit value (Revenue – Fixed Cost + Variable cost) needs
to be zero to attain break-even value.

• Select C8 in the ‘Set Cell’
• Enter 0 in the ‘To Value’
• Select C3 in ‘By Changing Cell’
Then press OK

Other Tests of Data Analysis (Z Test)
• Z-Test:
With the help of Z-Test, we compare the means of two datasets in
Excel that are equal or not.
• In Excel, we have a function for Z-Test named as ZTest, where, as per
syntax we need to have Array and X value (Hypothesized sample
mean) and Sigma value (Optional).
• Mostly X is considered a minimum of 95% of probability for that it can
be taken from 0 to 5.
• There we would need 2 variable ranges, 2 variances of each range.
• If Z < Z Critical then we will reject the null hypothesis.

Other Tests of Data Analysis (ANOVA)
• ANOVA Test:
ANOVA (Analysis of Variance) in Excel is the single and two-factor
method which is used to perform the null hypothesis test which says
the test will be PASSED for Null Hypothesis if from all the population
values are exactly equal to each other.
• If any or at least one value is different from other values, then the null
hypothesis will be FAlLED.

Other Tests of Data Analysis (P-Value)
• P-Values in excel can be called probability values, it’s used to understand
the statically significance of a finding.
• The P-Value is used to test the validity of demonstrate that we are testing
the Null Hypothesis. If the null hypothesis is considered improbable
according to the P-Value then it leads us to believe that the alternative
hypothesis might be true. Basically, it allows us whether the provided
results been caused by chance or these wo unrelated things. So P-Value is
an investigator and not a Judge.
• A P-Value is a number between 0 and 1 but it’s easier to think about them
in percentages (i.e. for Pvalue of 0.05 is 5%. Smaller P-value leads to the
rejection of the null hypothesis.
• The formula to calculate the P-Value is TDIST(x, deg_freedom, tails)

Pivot Table
• Pivot Meaning:
• According to dictionary: “the central point, pin, or shaft on which a
mechanism turns or oscillates.”
• Meaning in Bengali [According to Google Translate]:
“অপরিহার্য গুরুত্বপূর্য ব্যারি” (!!)

Pivot Table
• Pivot Table in excel is used to categorize, sort, filter and summarize
any length of data table which we want to get count, sum, values
either in tabular form or in form of 2 column sets.
• To insert the pivot table, select the Pivot table option from the Insert
menu tab, which will automatically find the table or range. We can
use the short cut keys Alt + D + P simultaneously which we will detect
the range of cells and take us to the final pivot option.
• We can also create a customized table by considering those columns
which are actually required.

1st Step
• Find all the available properties of the dataset in Word Format.
• Example:
• Number of Rows, Columns
• What is this dataset about [Place, Date, Time]
• Number of Unique Properties of the dataset [columns]
• If there is any missing data
• Sorted/ Curated
• Objective of the analysis

2nd Step
• Dataset Curation
• Sorting, Filtering
• Missing Data (if any) handling
• Pivot Table Loading and Check if the unique properties set earlier is available
for conditioning for pivoting.
• Use: Conditional Formatting, Knowledge of Quick Analysis, Filters, Sort.

Pivot Table (cont.)
Example #1
• In a Company where department work is to
mark certain Id’s are they correct or not.
They process data which has some sort of
id’s and mark it as correct or not.
• Below is the screenshot of raw data:

Pivot Table (cont.)
• Suppose a company’s manager of the department wants to know the
count of how many Prop_ID were correct and incorrect. He can
manually count those values but for a large set of data, it will be a
slow task. But there is a way to do this easily.
• In his excel workbook, he will hit the Insert button and click on the
pivot table on the leftmost button of his screen. Now he can choose
the same worksheet or a new worksheet to open this pivot table.
• To know the count of Correct and Incorrect values in that prop_id the
Status field which is correct or incorrect will be drag down to rows
section and the values the count in reference of the property id’s the
Prop_ID will be drag down to Values Section.

Pivot Table (cont.)
• Select the data and go to Insert tab click on pivot tables under the
tables section

Pivot Table (cont.)
• A dialog box appears. In the above image
there are few checkboxes, first to select
the table range which we did by selecting
the data.
• Now where to insert the pivot table to
insert in the same worksheet or different
worksheet?
• If the data is very huge then it is convenient
to insert the pivot table in the
new worksheet and click OK.

Pivot Table (cont.)
• We get the below result, on the
right-hand side we have our fields
of the pivot tables which will be
moved to rows and columns as the
desired report and on the
left-hand side, the pivot table will
be created.

Pivot Table (cont.)
• Our Task is to check how many
property ids were marked as correct
and how many were marked as
incorrect by the auditor.
• Drag Auditor fields to Rows section,
Property_id to value section whereas
the status field to filters section.

Pivot Table (cont.)
• We have made our pivot table which
currently shows the total count of
property id’s marked by the auditors.
• Now to check the status of how many
property ids were marked as correct and
how many were marked as incorrect.
In the pivot table under the status section
click on it.

Pivot Table (cont.)
• Now check select multiple items and then check Correct and click on
ok.

Pivot Table (cont.)
• Now we have a count of property id’s
marked as correct by the auditor.
• Similarly, we can have counted for
incorrect ones.

Pivot Table (cont.)
Example #2
• In a sales company, we have a
transactional sales data which
contains which product made
how many sales under which
quarter and in what year.
• Below is the screenshot of raw data:

Pivot Table (cont.)
• This data is not up to 20 rows but it
66 rows in this example which can
go down even further and it would
be a tedious task to check for sales
of a specific product under any quarter.
• Select the data to insert an pivot table.
This time we will use a shortcut key to
insert pivot tables, click
alt then “D” and then “P”.
• Another dialog box appears. We have
our data in excel and we want to create a
pivot table then we have to click the next button.

Pivot Table (cont.)
• In the next step, it asks for a range of data. But as we had already
selected the data so it is prefilled. Click on Next.

Pivot Table (cont.)
• Now the last dialog box asks us where we want our pivot tables in the
same worksheet or another. We have to select New worksheet and
then Click on Finish.

Pivot Table (cont.)
• Drag product in the rows
section, sales under the values
and quarter under the column
section whereas year in the
filter.
• And we have our report.
• The above pivot table shows which product made how many sales in
which quarter.

Pivot Table (cont.)
• If we want to check in the year 2017 what was the sales for the
products we simply uncheck the year 2018 in the year tab.

Pivot Table (cont.)
• Now we have the sales for the products in the year 2017.

Pivot Table (cont.)
Explanation of the Pivot Table in Excel
• Basically, pivot tables is a powerful Excel tool which helps us to
summarize large amounts of data and saves us a lot of time.
• Pivot tables are a reporting tool which has fields section which
contains four fields:
• Rows: Data which is taken as a specifier.
• Values: Count of the data.
• Filters: Filters to hide out certain data.
• Columns: Values under different conditions.

Pivot Table (cont.)
Things to Remember About Excel Pivot Table:
• Pivot tables do not change the values in the database.
• Pivot tables can be inserted in the same worksheet with the data or in
another worksheet.
• For convenience, we add pivot tables in a new worksheet.

PowerPivot
• PowerPivot is a popular Add-In under Microsoft Excel which can be used to
import the data sets with millions/trillions of rows from different sources
and help us do quick data analysis with large data sets under excel in a jiffy.
• This add-in was first introduced under Microsoft Excel 2010 and made as a
native feature under 2013 onwards versions of Excel and Office 365 as well.
• The power that PowerPivot posses lies in the data models of its own which
can be considered as databases. The data models are nothing but the data
tables similar to those we use in SQL. We can slice and dice with these
data tables. Create relationships between them, combine different data
tables, create calculated columns for advanced analysis and obviously, the
advanced reporting as well.

MS Excel-Sample Pivot Table Report Format
• Please google and make yourself in presenting analyzed data.
• Let’s Check a sample:
https://towardsdatascience.com/building-interactive-dashboards-
using-excel-3b5402da5d22

Pivot Table in Libre Office [Open Office]
To create a Pivot Table in Open Office (Calc):
• Select only one cell from your data.
• Choose the Insert Pivot Table
command from the main menu or
click the from the Standard toolbar.
• Calc automatically selects all the
cells and opens the Select Source
dialog.
• Click OK to continue

• In the Pivot Table Layout Dialog
you set up the pivot table.
• In general you drag fields from
the Available Fields pane to the
other white areas.

• Drag the employee field to the Row fields are and the sales field to
the Data Fields area and click OK.

• The pivot table is created in a new sheet. Now we get a summary of
the sum of sales for each employee.

Pivot Table Layout
• The layout of the pivot table is divided into 4 parts: Rows, Columns,
Data and Page. If you understand the layout you will be able to create
more complex pivot tables and extract important information from
your data.

Bonus: A Good Resource for Pivot Table
• Please Check: https://www.contextures.com/pivottableindex.html

Dashboard of MS Excel
• Dashboards track KPIs, metrics, and other data points in one visual,
central place. They give you a high-level view of work, helping you
make quick decisions and keeping everyone up to date. A
dashboard’s visual nature simplifies complex data and provides an
at-a-glance view of current status or performance in real time.
• Dashboards are made up of tables, charts, gauges, and numbers. They
can be used in any industry, for almost any purpose. For example, you
could make a project dashboard, financial dashboard, marketing
dashboard, and more.

Dashboard of MS Excel (cont.)
• General steps of making Dashboard in MS Excel:
• Sourcing/ Jotting the raw data / dataset
• Create Excel Dashboard file/ Sheet
• Sort and sync data in tabular format
• Analyze the Data using the tools [pivot/table/validation/range/chart/macro…]
• Build the Dashboard [Interactive Charts, Pivot Table-Charts]
• Customization [Color, Interpretation, Macro, representation]

Creating Dynamic Chart:
• Drop-down list/data validation list: Using VLOOKUP, you can use a drop-down list
(also known as a data validation list) to create interactive charts. With this drop-
down list, viewers can select the criteria they want to filter on and the chart will
automatically change to reflect that criteria. For more information on creating a
data validation list with VLOOKUP, click here.
• Macros: You can write a Macro using Excel’s coding language (called Visual Basic)
to automate a task. For example, You can also use Macros to create a button on
your dashboard. When you click that button and select a certain criteria, all the
charts will automatically change to represent that specific criteria.
• Slicers: If you want to add another layer of filtering to your pivot table, you can
use slicers. Slicers contain a set of buttons that let you filter the data and also
show you which filter you are viewing.

• Let’s Check Some Sample Dashboard of:
• HR Management
• Project Management
• Ecommerce Order Management and Advertisement Dashboard
• Social Media Platform
• Web Analytics
• Supply Chain Dashboard
• DevOps Dashboard
• Sales Management Dashboard

Bibliography
• https://www.qimacros.com/hypothesis-testing/
• https://www.statisticshowto.com/probability-and-statistics/hypothesis-testing/
• https://www.educba.com/data-analysis-tool-in-excel/
• https://www.educba.com/goal-seek-in-excel/
• https://www.educba.com/regression-analysis-in-excel/
• https://www.educba.com/z-test-in-excel/
• https://www.smartsheet.com/how-create-dashboard-excel
• https://www.educba.com/anova-in-excel/
• https://www.educba.com/p-value-in-excel/
• https://www.educba.com/powerpivot-in-excel/
• https://www.educba.com/pivot-table-in-excel/
• https://elearn.ellak.gr/mod/page/view.php?id=3015

Anything you want me to tell?
Thank You
See you on next class…
9/16/2020 95

Excel Data Analysis: Regression, T-Tests, Pivot Tables & Dashboards

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Excel Data Analysis: Regression, T-Tests, Pivot Tables & Dashboards

Similaire à Excel Data Analysis: Regression, T-Tests, Pivot Tables & Dashboards (20)

Plus de Redwan Ferdous

Plus de Redwan Ferdous (20)

Dernier

Dernier (20)

Excel Data Analysis: Regression, T-Tests, Pivot Tables & Dashboards