18. Daily Active Users
7 Day Moving Average
1,440,000
1,420,000
1,400,000
1,380,000
1,360,000
1,340,000
Control
Test
1,320,000
1,300,000
1,280,000
1,260,000
If you have tests that show positive improvements, and when switched live they have little or a negative effect…. This might be the cause
We’re going now into gamification territory
The difference in the two groups is about 40-50,000 users.
The difference is between – 0.4 and 0.8 percentage points at 95% confidence.Also looks like the gap is widening…. And the other retention measure showed a similar behaviour.
Running the opportunity cost calculations I discovered that we could be making on average $1,500 / day more switching to the expensive group.