Personalized Learning Recommendations at Chegg
•Download as PPTX, PDF•
0 likes•399 views
This presentation covers (1) Rich content developed at Chegg (2) An excellent knowledge graph that organizes content in a hierarchical fashion (3) Interaction of students across multiple products to enhance user signal in individual products.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Personalized Learning Recommendations at Chegg
Similar to Personalized Learning Recommendations at Chegg (20)
More from Sanghamitra Deb
More from Sanghamitra Deb (14)
Recently uploaded
Recently uploaded (20)
Personalized Learning Recommendations at Chegg
- 1. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Developing Recommendation System to provide a Personalized Learning experience at Chegg Sanghamitra Deb Staff Data Scientist Chegg Inc
- 2. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Outline • Recommendations at Chegg. • Organizing Content – Knowledge Graph • Deep Dive : Content Classifications • Cross Product Recommendations. • Takeaways. 2
- 3. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Recommendations at Chegg 3 Goal of Recommendations at Chegg is providing the best possible learning experience to Students. This is fueled by high quality content. Recommender Systems provide a backbone to surface the most relevant content to a student. Organizing content into a knowledge graph and detecting patterns in student behavior helps us personalize student experience.
- 4. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Recommendations at Chegg Chegg Study Home Page 4 Multiple services: text book rentals, question answering, online tutoring, flashcards, writing, math solver, etc.
- 5. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Knowledge Graph Subject Course Course Course Concept Concept Concept Sub- concepts Physics Electricity and Magnetism Mechanics Quantum Physics Velocity
- 6. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Connecting conetnt to the Knowledge Graph Subject Course Concept Sub- concepts A rightward-moving bicycle increases its speed from 2.0 m/s to 12.0 m/s. Is the bicycle accelerating? Writing tools Machine Learning Classifiers Mitosis a type of cell division that results in two daughter cells each having the same number and kind of chromosomes as the parent nucleus, typical of ordinary tissue growth. Get your physics paper checked by an expert
- 7. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Connecting users to the Knowledge Graph Subject Course Concept Sub- concepts A rightward-moving bicycle increases its speed from 2.0 m/s to 12.0 m/s. Is the bicycle accelerating? Writing tools Machine Learning Classifiers Physics 101 Acceleration Do you need help writing a physics paper? Edges are created between users and Biology Mitosis
- 8. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Content Classification Pipeline Text Pre-processing Collecting Training Data Model Building Offline SME • Reduces noise • Ensures quality • Improves overall performance • Training Data Collection / Examples of classes that we are trying to model • Model performance is directly correlated with quality of training data Model Evaluation • Model selection • Architecture • Parameter Tuning Student Online 8
- 9. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Classification Problem Assigning decks to Courses • Decks are list of cards grouped together by students for studying. • There are several thousand courses, typically it is more granular than subjects but less granular than concepts.
- 10. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved • TFIDF features with an SVM classifier Pros – • Gives decent performance on small training data. • Straightforward training pipeline. Cons – • Does not do well for subjects dominated by symbols, • Including word & character based features makes the token space & model extremely large. • Character Based CNN. • Has the ability to deal with out of vocabulary words. This makes it particularly suitable for user generated raw text. • Works for multiple languages. • Model size is small since the tokens are limited to the number of characters ~ 70. This makes real life deployments easier and faster. • Networks with convolutional and pooling layers are useful for classification tasks in which we expect to find strong local clues regarding class membership. Modeling Approaches
- 11. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved CNN Model Architecture GlobalMaxPool1D Convolutions Feature Length DenseLayer Dropout Prelu Norm …. Convolution & pool layer …. 2 layers of convolution & pooling
- 12. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Multi-task Modeling CNN Model CNN Model Cross Entropy Loss Output Card Front Front Back Card Back Similarity Function Card CNN Model Softmax -- # of courses Cross Entropy Loss Output Two tasks • Similarity between card front and back. • Classification of courses Adding another task improves the accuracy by a few percent.
- 13. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Model Performance Top-3 -- 73% accuracy on offline- test data. Challenges • Imbalanced Training Data • Some classes have too few training examples Solutions • Collect More training data. • Use rule based techniques to augment training data
- 14. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved Cross Product Recommendations! Cold Start Problem: Users often use one product such as Chegg Study and may just browse other products that provide Chegg Practice or Flash Cards. Solutions: Personalized • Content Filtering --- Use KG to determine courses, concepts and sub-concepts that users are currently studying and recommend trending content in that category. • Text Similarity --- Based on their content engagement. Use in house language models optimized for Chegg content.
- 15. Confidential Material / © 2020 Chegg, Inc. / All Rights Reserved • Content drives recommendations • High quality • Relevant • Organizing the content into a Knowledge Graph (KG) facilitates content based recommendations. • Accuracy of classifiers is important --- models are constantly iterated even for few percent gain. • KG helps connect students and courses/concepts which helps with personalized recommendation • Cross product recommendations are possible through KG. • Cold Start problems are made easier. Takeaways
- 16. Confidential Material / © 2019 Chegg, Inc. / All Rights Reserved Questions
Editor's Notes
- I am going to talk about personalizing the learning experience at Chegg using recommendation systems.
- Here is an outline of the presentation.
- Chegg is a centralized learning platform where a student comes to learn concepts required for academic performance, job interviews or other activities. The goal of any RS is to present content that is of high quality and relevant, i.e we show them what they want to study. An example of that is --- lets say the student has data analyst job interview --- we know this from past user interactions , so we show the student content related to learning “SQL”.
- This is an example of student experience at Chegg. A student logs in and finds suggestions in Mechanical Engineering and Chemistry. As you can seem this model suggests textbook solutions for users based on their past behavior, and it is accompanied by the message "based on your progress”. Another example is a concept-based recommendation module in Study, which is placed below an expert answer that the student is viewing. I wanted to use this slide to give you a look into our content. As you can see most of our content is academic materials.
- Now I will Segway into how this content is organized. We have build a knowledge graph which represents a hierarchy of subjects, courses and concepts. The nodes in this graph is provided by subject matter experts. We constantly iterate on this graph as we get suggestions for more nodes and edges. The machine Learning component comes in when we create edges between concept nodes and content. How does this look?
- Here is an example of how we connect content from different products to the nodes of the knowledge graph.
- When users interact with the content we are able to connect users to a node of the knowledge graph. Since user interactions constantly change with time the degs between users and KG nodes are constantly updated.
- Lets now do a deepdive into content classification since that is the backbone of all the recommendations here.
- Convolution and pooling layers are good at picking up signature at n-gram level, i.e it is able to pick up when certain phrases are indicative of certain class memberships.
- The two layers ensure that the correlations between n-grams are picked up at two different scales.
- We define two different task for optimization. One of them is to match the front of the card with the back of the card. We use the CNN model defined in the previous slide and use the dot product as the similarity function and use a cross entropy loss. For the classification problem we feed the CNN model into a softmax layer to predict the courses. Both tasks are optimized simultaneously.