1030 track 1 wilson_using his laptop

Chi Square You to
Dive into Statistics
Tim Wilson
Analytics Demystified
@tgwilson | #eMetrics

Welcome to the
AI Track!

NOT a Smurf
NOTa data scientist
NOTa statistician

If you are a web
analyst…
…you will need to
start thinking
about data
differently.

If you are a data
scientist…
…you will need
recognize where
web analysts and
digital marketers
are coming from.
@tgwilson | #eMetricsurce: DARPA

The dream…
“I’ve fully
optimized your
marketing,
Dave.”

The dream… Artificial Intelligence!
“I’ve fully
optimized your
marketing,
Dave.”
?

“In God we trust; all
others bring data.”
- W. Edwards Deming

“Marketing (and analytics) is
inherently decision-making
in conditions of uncertainty.”
- Matt Gershoff

in conditions of uncertainty.
There is a cost to reducing
uncertainty.”
- Matt Gershoff

in conditions of uncertainty.
There is a cost to reducing
uncertainty.
Uncertainty cannot be
entirely eliminated.”
- Matt Gershoff

This is one definition of
statistical thinking.

Statistics
≠
Machine Learning
≠
Artificial Intelligence

Statistical Thinking
Machine Learning
Artificial Intelligence

Let’s dive deep into one simple
example.

How much
traffic did we
get last month?

163,236
sessions

Let’s trend that!

How did that
traffic break
down by channel?

Simple: Channel Overview

Do sessions really differ by channel?

How many rows of data?
6!

Ask a data scientist…
163,236

Ask a data scientist…

This is the difference between
aggregation and observations.

Web analytics platforms have
conditioned us to work with
aggregated data.

They have started implicitly
acknowledging the need for
observation-level detail.

Adobe Analytics Data Feeds

Google Analytics à Google BigQuery

Back to our example
6 vs. 163,236

Is there a middle ground?
6
163,236
Maximum aggregation
Maximum detail
?

We can go from 6 to 186…

Yes, there is a middle ground.
6
163,236
Maximum aggregation
Maximum detail

From 6 to 186 by adding a dimension.
6
163,236
Maximum aggregation
Maximum detail
186 Add date as a dimension

This is less aggregated data.
6
163,236
Maximum aggregation
Maximum detail
186 Add date as a dimension

That’s enough to dig in a bit.

Let’s look at the variability
in the data.

Overall… we see some variability

And…we see variability within each channel

What if we ignore
the “time” aspect
of “date?”

We can now make a histogram!

Or…a density plot.

We can make a histogram for each channel.

Which means a density plot for each channel.

Back to our overall distribution.

Let’s flip it.

And squish and rotate it.

And then break out each channel

We can do an ANalysis Of VAriance

Also known as…a 1-Way ANOVA

We can reject the null hypothesis!

@#$&*?!!!!

We can be reasonably confident
that there is a “real” difference
in sessions across different
channels.

We can dig in further!
Results of a Tukey
post hoc test
I’m not really
qualified to do this
with confidence

Let’s do a quick little visual check.

Go figure!

I know. We could see most of this visually.

But this was a simple example
mainly for illustrative purposes.

Things can easily get more involved.
Sessions by Channel Sessions by Device Type
?

Things can easily get more involved.
Sessions by Channel Sessions by Device Type
2-Way
ANOVA

Many statistical techniques
require moving away from
aggregation and towards
observations.

Time is something we aggregate
all the…time.

Let’s look at another example.

Did the metric
move enough
for me to
care?

Last week’s decline: signal or noise?
@tgwilson | #eMetricsbit.ly/hw-forecast

Start by looking at the data by day

Let’s break that data into two groups.

We have the data of interest…

…and data for context.

…or “training data.”

Let’s take our training data…

…and “decompose” it.

We have seasonality on a 7-day cycle.
=
bit.ly/hw-forecast

A moving average reveals trending.
=
+
bit.ly/hw-forecast

And what’s left is just noise.
=
+
+
bit.ly/hw-forecast

Trend + Seasonality = Forecast

The forecast won’t be perfect.

Historical “noise” à a prediction interval

We can compare to the actual results.

Now we have meaningful context.

What this can look like in the real world.

And…drilling down and summarizing

Mark Edmondson’s
Pre-/Post- Analysis Using a Bayesian Structural Time-Series Method
@tgwilson | #eMetricsbit.ly/ga-effect

Of course…

…there’s more.

A LOT more!
@tgwilson / #eMetricsFlickr / Neil Piddock
ANOVA
Holt-Winters Forecasting
Bayesian Structural Time-Series
Crosstab with a !2 Test
Correlation
Regression
Median Absolute Deviation (MAD)
K-Means Clustering
Is there a statistically
significant difference between
these groups (dimensions +
metric)?
Did that metric
move enough for
me to care?
Can I
quantify the impact
of a change?
?

Two possible reactions…

Excitement?

So, now what?

Get inspired.
“Most of this sounds a little
over-the-horizon and
science-fiction-ish, and it
is. But it’s only just over
the horizon.”
- Jim Sterne

Join the discussion.
http://join.measure.chat
# data-science

(Maybe) Give it a try.

Slack/Twitter: @tgwilson
LinkedIn: linkedin.com/in/tgwilson
Email: tim@analyticsdemystified.com
Podcast: analyticshour.io
Thank you!
Measure Slack: join.measure.chat
Holt-Winters Forecasting: bit.ly/hw-forecast
GA Effect: bit.ly/ga-effect
Learn R: dartistics.com

1030 track 1 wilson_using his laptop

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (16)

Similaire à 1030 track 1 wilson_using his laptop

Similaire à 1030 track 1 wilson_using his laptop (20)

Plus de Rising Media, Inc.

Plus de Rising Media, Inc. (20)

Dernier

Dernier (20)

1030 track 1 wilson_using his laptop