Introduction to Apache Cassandra from Cassandra NYC summit

4. But first... the CAP Theorem C onsistency A vailability P artition Tolerance “ Thou shalt have but 2” - Conjecture made by Eric Brewer in 2000 - Published as formal proof in 2002 - See: http://en.wikipedia.org/wiki/CAP_theorem for more

5. CAP Theorem: Cassandra Style - Explicit choice of partition tolerance and availability. - Opt for more consistency at the cost of availability Consistency is tunable (per operation)

7. Generally complements another system(s) (Not intended to be one-size-fits-all) *** You should always use the right tool for the right job anyway

8. How does this differ from an RDBMS?

9. How does this differ from an RDBMS? Substantially.

10. vs. RDBMS - No Joins Unless: - you do them on the client - you do them via Map/Reduce

11. vs. RDBMS - Schema Optional (Though you can add meta information for validation and type checking) *** Supports secondary indexes too: “ … WHERE state = 'TX' ”

12. vs. RDBMS - Prematerialized and Transaction-less - No ACID transactions - Limited support for ad-hoc queries

13. vs. RDBMS - Prematerialized and Transaction-less - No ACID transactions - Limited support for ad-hoc queries *** You are going to give up both of these anyway when you shard an RDBMS ***

15. vs. RDBMS - Shared-Nothing Architecture Every node plays the same role: no masters, no slaves, no special nodes *** No single point of failure

41. - Controls replication

42. Column Family

43. - Similar to a table

44. - Columns ordered by name

46. Dynamic Column Family

47. - Pre-calculated query results

48. Nothing stopping you from mixing them!

51. Data Model – Prematerialized Query Additional examples: Timeline of tweets by a user Timeline of tweets by all of the people a user is following List of comments sorted by score List of friends grouped by state

55. Big Data: Map/Reduce Integration Cassandra Implementations of: - InputFormat and OutputFormat - RecordReader and RecordWriter - InputSplit for Column Families *** See org.apache.cassandra.hadoop package and examples for more

56. Big Data: Pig Integration grunt> name_group = GROUP score_data BY name PARALLEL 3; grunt> name_total = FOREACH name_group GENERATE group, COUNT(score_data.name), LongSum(score_data.score) AS total_score; grunt> ordered_scores = ORDER name_total BY total_score DESC PARALLEL 3; grunt> DUMP ordered_scores;

57. Using a Client Hector Client: http://hector-client.org - Most popular Java client - In use at very large installations - A number of tools and utilities built on top - Very active community - MIT Licensed *** like any open source project fully dependent on another open source project it has it's worts

61. Hector: ColumnFamilyTemplate ColumnFamilyResult<String, String> res = cft.queryColumns(&quot;zznate&quot;); String value = res.getString(&quot;email&quot;); Date startDate = res.getDate(“startDate”); Key Format Column Name Format

62. Hector: ColumnFamilyTemplate ColumnFamilyResult wrapper = template.queryColumns(&quot;zznate&quot;, &quot;patricioe&quot;, &quot;thobbs&quot;) ; while (wrapper.hasNext() ) { emails.put(wrapper.getKey(), wrapper.getString(&quot;email&quot;)); ... Querying multiple rows

63. Hector: ColumnFamilyTemplate ColumnFamilyResult wrapper = template.queryColumns(&quot;zznate&quot;, &quot;patricioe&quot;, &quot;thobbs&quot;); while ( wrapper.hasNext() ) { emails.put(wrapper.getKey(), wrapper.getString(&quot;email&quot;)); ... Iterating over results

64. Hector: ColumnFamilyTemplate ColumnFamilyUpdater updater = template.createUpdater(&quot;zznate&quot;); updater.setString(&quot;companyName&quot;,&quot;DataStax&quot;); updater.addKey(&quot;sergek&quot;); updater.setString(&quot;companyName&quot;,&quot;PrestoSports&quot;); template.update(updater); Insert: Creating an updater for a key

65. Hector: ColumnFamilyTemplate ColumnFamilyUpdater updater = template.createUpdater(&quot;zznate&quot;); updater.setString(&quot;companyName&quot;,&quot;DataStax&quot;); updater.addKey(&quot;sergek&quot;); updater.setString(&quot;companyName&quot;,&quot;PrestoSports&quot;); template.update(updater); Insert: Adding Multiple Rows

66. Hector: ColumnFamilyTemplate ColumnFamilyUpdater updater = template.createUpdater(&quot;zznate&quot;); updater.setString(&quot;companyName&quot;,&quot;DataStax&quot;); updater.addKey(&quot;sergek&quot;); updater.setString(&quot;companyName&quot;,&quot;PrestoSports&quot;); template.update(updater); Insert: Invoking Batch Execution

67. Hector: ColumnFamilyTemplate template.deleteColumn(&quot;zznate&quot;, &quot;notNeededStuff&quot;); template.deleteColumn(&quot;zznate&quot;, &quot;somethingElse&quot;); template.deleteColumn(&quot;patricioe&quot;, &quot;aDifferentColumnName&quot;); ... template.deleteRow(“someuser”); template.executeBatch(); Deleting Data: Single Column

68. Hector: ColumnFamilyTemplate template.deleteColumn(&quot;zznate&quot;, &quot;notNeededStuff&quot;); template.deleteColumn(&quot;zznate&quot;, &quot;somethingElse&quot;); template.deleteColumn(&quot;patricioe&quot;, &quot;aDifferentColumnName&quot;); ... template.deleteRow(“someuser”); template.executeBatch(); Deleting Data: Whole Row

71. - Merge on read

72. - Sstables are immutable

73. - Highest timestamp wins

75. -------------------

76. RowKey: 12783

77. => (column=GOOG, value=30, timestamp=1310340410528000)

78. -------------------

79. RowKey: 15736

80. => (column=AAPL, value=20, timestamp=1310143852392000)

81. => (column=NOK, value=90, timestamp=1310143852444000)

82. => (column=IBM, value=50, timestamp=1310143852448000)

83. => (column=GOOG, value=5, timestamp=1310143852453000)

84. => (column=INTC, value=200, timestamp=1310143852457000)

86. -------------------

87. RowKey: 12783

88. -------------------

89. RowKey: 15736

90. => (column=AAPL, value=20, timestamp=1310143852392000)

91. => (column=NOK, value=90, timestamp=1310143852444000)

92. => (column=IBM, value=50, timestamp=1310143852448000)

93. => (column=GOOG, value=5, timestamp=1310143852453000)

94. => (column=INTC, value=200, timestamp=1310143852457000)

96. . . .

97. -------------------

98. RowKey: 14100

99. -------------------

100. . . .

104. class=&quot;me.prettyprint.cassandra.service.CassandraHostConfigurator&quot;>

105. <constructor-arg value=&quot;localhost:9170&quot;/>

106. </bean>

107. <bean id=&quot;cluster&quot; class=&quot;me.prettyprint.cassandra.service.ThriftCluster&quot;>

108. <constructor-arg value=&quot;TestCluster&quot;/>

109. <constructor-arg ref=&quot;cassandraHostConfigurator&quot;/>

110. </bean>

111. <bean id=&quot;consistencyLevelPolicy&quot; class=&quot;me.prettyprint.cassandra.model.ConfigurableConsistencyLevel&quot;>

112. <property name=&quot;defaultReadConsistencyLevel&quot; value=&quot;ONE&quot;/>

113. </bean>

114. <bean id=&quot;keyspaceOperator&quot; class=&quot;me.prettyprint.hector.api.factory.HFactory&quot;

115. factory-method=&quot;createKeyspace&quot;>

116. <constructor-arg value=&quot;Keyspace1&quot;/>

117. <constructor-arg ref=&quot;cluster&quot;/>

118. <constructor-arg ref=&quot;consistencyLevelPolicy&quot;/>

119. </bean>

120. <bean id=&quot;simpleCassandraDao&quot; class=&quot;me.prettyprint.cassandra.dao.SimpleCassandraDao&quot;>

121. <property name=&quot;keyspace&quot; ref=&quot;keyspaceOperator&quot;/>

122. <property name=&quot;columnFamilyName&quot; value=&quot;Standard1&quot;/>

123. </bean>

125. https://github.com/rantav/hector/wiki/Hector-Object-Mapper-%28HOM%29

126. Hector JPA (experimental open-jpa implementation):

127. https://github.com/riptano/hector-jpa

129. “ select price FROM Stocks WHERE KEY = ?&quot;;

130. jdbcTemplate.query(STOCK_CQL, stockTicker,

131. new RowMapper<Stock>() {

132. public Stock mapRow(ResultSet rs, int row) throws SQLException {

133. CassandraResultSet crs = (CassandraResultSet)rs;

134. Stock stock = new Stock();

135. stock.setTicker(new String(crs.getKey()));

136. stock.setPrice(crs.getDouble(&quot;price&quot;));

137. return stock;

138. }

139. });

141. &quot; update Portfolios set ? = ? where KEY = ? &quot;;

142. jdbcTemplate.update(UPDATE_PORTFOLIO_CQL,

143. new Object[] {position.getTicker(),

144. position.getCount(),

145. portfolio.getName()});

147. &quot;update Portfolios set ? = ? where KEY = ?&quot;;

148. jdbcTemplate.batchUpdate(UPDATE_PORT_CQL,

149. new BatchPreparedStatementSetter() {

150. public void setValues(PreparedStatement ps, int index) throws SQLException {

151. Position pos = portfolio.getConstituents().get(index);

152. ps.setString(1, pos.getTicker());

153. ps.setLong(2, pos.getShares());

154. ps.setString(3,portfolio.getName());

155. }

156. public int getBatchSize() {

157. return portfolio.getConstituents().size();

158. }

159. });

162. - Bank account:

163. “ on values over $10,000, wait to here from all replicas”

164. - Distributed Shopping Cart:

165. Show a confirmation page to verify order resolution

166. *** What is your appetite for risk?

169. ***But In an append-only system, we don't do updates

171. - Look for idempotence and asynchronicity in your business processes

172. - If it's not in the API, you are probably doing it wrong

173. - Seek death is still possible if you model incorrectly

176. CCM localhost cassandra cluster https://github.com/pcmanus/ccm

179. Apache Cassandra project wiki: http://wiki.apache.org/cassandra/

180. “ The Dynamo Paper”

181. http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf

182. P. Helland. Building on Quicksand

183. http://arxiv.org/pdf/0909.1788

184. P. Helland. Life Beyond Distributed Transactions

185. http://www.ics.uci.edu/~cs223/papers/cidr07p15.pdf

186. S. Anand. “Netflix's Transition to High-Availability Storage Systems”

187. http://media.amazonwebservices.com/Netflix_Transition_to_a_Key_v3.pdf

188. “ The Megastore Paper”

189. http://research.google.com/pubs/archive/36971.pdf