We built an storage API layer on top of Cassandra exposing a document model to improve developer productivity at Coursera.

1. Daniel Chia
Infrastructure Engineer
@DanielChiaJH
Courral Storage API layer

2. Agenda
❖ Brief History
❖ Quick Run Through
❖ Wins / Problems
❖ Future Directions

3. History
App
KVS Library

4. History
App
KVS Library
App
KVS Library
App 3
KVS Library
Indexing?

5. Courral
❖ Document Model
❖ Key -> Value
❖ Declarative indexing

6. Example: Product Ownership
❖ User X owns Product Y during
❖ [2016-01-01, 2016-03-01]
❖ [2016-05-01, 2016-08-01]
❖ Query by
❖ (user, product)
❖ (user)

7. Document Collection
{
"clusterName": "cass_prod_1",
"idSchema": "org.coursera.product.ownership.UserProductOwnershipId",
"dataSchema": "org.coursera.product.ownership.StoredProductOwnership",
"indexes": [
{
"name": "byUser",
"indexScheme": "reverse",
"targets": [{ "targetType": "id", "path": ["userId"] }]
}
]
}

8. Schema Language (Courier)
record UserProductOwnershipId {
userId: UserId,
productType: ProductType
productItemId: ProductItemId
}
http://coursera.github.io/courier/

9. API
documentClient.get(id).map(_.document)
documentClient.set(id, ownership)
documentClient
.list(
“byUser”,
List(userId),
limit, start.map(ListStart(_), fetchSize = 50)

10. Self-service

11. Wins
❖ Deeper integration with our schema system.

12. Wins
❖ Better monitoring / insight

13. Problems
❖ New “single” point of failure
❖ Additional latency / CPU utilization

14. Future Work
❖ Automatic index backfilling
❖ Integrated caching
❖ Multi-tenant safeguards

15. Questions?