The success of large-scale recommender systems hinges upon their ability to deliver accurate and timely recommendations to a diverse user base. At Glance, we deliver snackable personalized content to the lock screens of 200M smartphones. In this context, continuous monitoring is paramount as it safeguards data integrity, detects drifts, addresses evolving user preferences, optimizes system downtime, and ultimately augments the system’s effectiveness and user satisfaction. In this talk, we delve into Vigil, a set of monitoring practices developed to provide comprehensive end-to-end monitoring of recommender systems at Glance. These practices revolve around three key pillars: mitigating developer fatigue, ensuring precise predictions, and establishing a centralized monitoring framework. By adopting these practices, we have observed an 18\% increase in user engagement, a 30% reduction in compute cost, a 26% drop in downtime, and a surge in developer productivity demonstrated by a 45% decrease in turnaround time.
2. We'll be your speakers for today's
featured talk on monitoring
practices. Brace yourselves for an
enjoyable session!!
Hello!
Priyansh
Manisha
Page 02 |
39. Data Quality and Hygiene
Areas of impact
Inaccurate
Data
Data
duplicati
on
Missing
values
Bias
Incorrect
features
Recommendation High Low Low High High
Business metrics High Low Low High High
User trust Low Low High High High
Areas of Impact Matrix
Page 19 |
40. VIGIL TECHNIQUES
Page 20 |
01
02
03
DRIFT DETECTION
DATA HYGIENE CHECKS
COMPONENT SPECIFIC METRICS
04 MODEL PERFORMANCE MONITORING
05 CENTRALIZED VIEW & DEPENDENCY GRAPHS
06 ALERTING
62. Expressway objectives
1 Provide a centralized monitoring view of various system components.
2
3
Offer a holistic perspective on component interconnectedness.
Streamlined onboarding process.
Page 40 |