With the ever more frequent use of multiple servers and worker processes, issues which only occur when specific tasks are running in parallel become ever more likely to happen. In this talk Marcos talks about some rails concurrency limitations you need to be aware of, including why you should never trust rails to check for uniqueness, why mysql can't do optimistic locking with retrying by default and how to fix it, how to properly choose between optimistic and pessimistic locking, how parallel processes can cause deadlocks even if you're not using locks, why you should never have serialized attributes in rows that might be edited in parallel, and why rails commits your transactions before they finish in case your multi threaded process is killed and how to prevent it from leaving your database on an inconsistent state.

1. Surpresas no uso
de Concorrência
em by Marcos Toledo
Rails

2. Marcos Renato
Baroni Toledo
@mtoledo

3. Socram O’taner

4. Socram O’taner
Marcos Renato

5. O’taner

6. Email

7. “I’m trying to
unsubscribe but
it says email has
already been
taken”!

8. validates_uniquen
ess_of
# id :integer(4)
# email :string(255)
class User < ActiveRecord::Base
validates_uniqueness_of :email
end

9. WTF ..
?

10. Invalid Records!
User.find_by_email(“angry_customer@email.com”).valid?
# => false

11. Why?

12. Rails can’t
guarantee
uniqueness!

13. Solutions?

14. Uniqueness
Constraint!
class AddUniquenessIndexToUserEmail < ActiveRecord::Migration
def self.up
add_index :users, :email, :unique => true
end
def self.down
remove_index :users, :email
end
end

15. Succ
ess!

16. Another Email

18. Airbrake!

19. NoMethodError: undefined method `title' for
nil:NilClass
“nil post”?

20. :dependent
=> :destroy
class Comment < ActiveRecord::Base
belongs_to :post
validates_presence_of :post
end
class Post < ActiveRecord::Base
has_many :comments, :dependent => :destroy
end

21. WTF ..
?

22. Orphan Models!
Comment.all.reject(&:post).size
# => 1

23. Why?

24. Rails can’t
guarantee
dependency
either!

25. Solutions?

26. Database Foreign
Keys!
create_table :comments do |table|
table.integer :post_id, :null => false
end
ActiveRecord::Base.connection.execute <<-EOS
ALTER TABLE `comments`
ADD CONSTRAINT FOREIGN KEY (post_id)
REFERENCES posts(id)
ON DELETE restrict
ON UPDATE cascade
EOS

27. Succ
ess!

28. 1 - Noob
“Use database
constraints”

29. O’taner

30. ruby 100% CPU!

31. Solutions?

32. Professional
Debugging
ssh server
killall ruby

33. Succ
ess!

34. WET Code

35. Infinite Loop
class Comment < ActiveRecord::Base
after_create :update_comment_count
def update_comment_count
post = Post.find post_id
post.comment_count += 1
post.save
rescue StaleObjectError
retry
end
end

36. WTF ..
?

37. Transaction
Isolation Level
“I” in ACID

38. 1 - READ
UNCOMMITTED
2 - READ
COMMITTED
3 - REPEATABLE
READ

39. Postgre
READ-COMMITTED
MySQL
REPEATABLE-READ

40. mysql> begin;
Query OK, 0 rows affected (0,00 sec)
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)
mysql> update posts set title = 'New Title' where lock_version = 0;
Query OK, 0 rows affected (0,00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)

41. mysql> begin;
Query OK, 0 rows affected (0,00 sec)
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)
mysql> update posts set title = 'New Title' where lock_version = 0;
Query OK, 0 rows affected (0,00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)

42. mysql> begin;
Query OK, 0 rows affected (0,00 sec)
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)
mysql> update posts set title = 'New Title' where lock_version = 0;
Query OK, 0 rows affected (0,00 sec)
Rows matched: 0 Changed: 0 Warnings: 0 // raises StaleObjectError
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)

43. mysql> begin;
Query OK, 0 rows affected (0,00 sec)
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)
mysql> update posts set title = 'New Title' where lock_version = 0;
Query OK, 0 rows affected (0,00 sec)
Rows matched: 0 Changed: 0 Warnings: 0 // raises StaleObjectError
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)

44. Solutions?

45. READ-
COMMITTED!
/etc/my.cnf
[mysqld]
transaction-isolation = READ-
COMMITTED
Just like
postgre ;)

46. mysql> begin;
Query OK, 0 rows affected (0,00 sec)
mysql> select * from posts where id = 1;
+----+-------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 0|
+----+-------+--------------+
1 row in set (0,00 sec)
mysql> update posts set title = 'New Title' where lock_version = 0;
Query OK, 0 rows affected (0,00 sec)
Rows matched: 0 Changed: 0 Warnings: 0 // raises StaleObjectError
mysql> select * from posts where id = 1;
+----+-----------+--------------+
| id | title | lock_version |
+----+-------+--------------+
| 1 | Title | 1|
+----+-------+--------------+
1 row in set (0,00 sec)

47. Succ
ess!

48. 2 - Beginner
“Use READ-
COMMITTED”

49. O’taner

50. Production
Console
> u1 = User.first
=> #<User id: 1, login: "marcos", options: {:active=>false}
> u1.update_attributes :options => {:active => true}
=> true
> u1.reload
=> #<User id: 1, login: "marcos", options: {:active=>false}

51. WTF ..
?

52. # login :string(255)
# options :text
class User < ActiveRecord::Base
serialize :options
def activate
self.options = {:active => true}
self.save
end
end

53. No ‘:active =>
false’!

54. Why?

55. Dirty Checking
> u1 = User.first
=> #<User id: 1, login: "marcos", options: {:active=>false}
> u2 = User.first
=> #<User id: 1, login: "marcos", options: {:active=>false}
> u1.update_attributes :options => {:active => true}
=> true
> u2.update_attributes :login => "mtoledo"
=> true
> u1.reload
=> #<User id: 1, login: "mtoledo", options: {:active=>false}

56. Serialize is
ALWAYS dirty!

57. // u1.update_attributes :options => {:active => true}
UPDATE `users`
SET `options` = '--- n:active: truen'
WHERE (`users`.`id` = 1)
// u2.update_attributes :login => "mtoledo"
UPDATE `users`
SET `login` = 'mtoledo',
`options` = '--- n:active: falsen'
WHERE (`users`.`id` = 1)

58. Solutions?

59. Options Model

60. # id :integer(4)
# user_id :integer(4)
# lock_version :integer(4)
# options :text
class UserOptions < ActiveRecord::Base
serialize :options
belongs_to :user
end

61. Succ
ess!

62. 3 - Half Mouth
“Use an Options
Model”

63. O’taner

64. production.log
ERROR 1213 (40001): Deadlock found when trying to get
lock;try restarting transaction

65. DAMN!
Pessimistic Locks!

66. No pessimistic
locking!
class Comment
after_create :update_posts_count
def update_posts_count
self.posts.increment :comments_count
end
end
class Posts
after_update :update_comments
def update_comments
comments.each {|c| c.published = self.published; c.save}
end
end

67. WTF ..
?

68. Transac
Transaction 1
SELECT * FOR
tion 2
UPDATE
FROM `comments`
WHERE `id` = 1 SELECT * FOR
UPDATE
locked.. FROM `posts`
WHERE `id` = 1
SELECT * FOR
UPDATE
FROM `posts` DEADLOCK!
WHERE `id` = 1
SELECT * FOR
UPDATE
FROM `comments`
WHERE `id` = 1

69. Why?

70. Transactions lock
on update

71. Transac
Transaction 1
UPDATE
tion 2
`comments`
SET `title` = “Foo”
WHERE `id` = 1
UPDATE `posts`
SET `title` = “bar”
locked.. WHERE `id` = 1
UPDATE `posts`
SET `title` = “bar”
WHERE `id` = 1
DEADLOCK!
UPDATE
`comments`
SET `title` = “foo”
WHERE `id` = 1

72. Callbacks =
Transactions

73. Solutions?

74. Update
dependents
before parents!

75. No longer
deadlocks!
class Comment
after_create :update_posts_count
def update_posts_count
self.posts.increment :comments_count
end
end
class Posts
before_update :update_comments
def update_comments
comments.each {|c| c.published = self.published; c.save}
end
end

76. Avoid State
Propagation

77. Avoids updating
all comments
class Comment < ActiveRecord::Base
def visible?
post.published? && self.published?
end
end

78. Succ
ess!

79. 4 - Advanced
“Acquire locks
consistently”

80. O’taner

81. “There is no safe way to use
Thread#kill or
Thread#raise.”
http://blog.headius.com/2008/02/rubys-threadraise-threadkill-timeoutrb.html

82. Killing ruby (MRI)
threads
- bluepill
- god
- monit
- you
and me

83. Simple
Transaction
class Order
belongs_to :user
after_create :mark_user_as_customer
def mark_user_as_customer
user.customer = true
user.save
end
end

84. What happened
to
my transaction?
> order = Order.first
=> #<Order id: 1, user_id: 1>
> order.user.customer?
=> false
> order.user?
=> #<User id: 1, login: "marcos", customer: false>

85. WTF ..
?

86. If you kill your
Ruby
(MRI) Threads ..

87. ActiveRecord will
partially
commit your
transactions!

88. Why?

89. ActiveRecord
transaction
(version for kids)
def transaction
yield
rescue Exception => e
rollback
ensure
commit unless rolledback?
end

90. Main thread is
rescued
def print_if_rescued
sleep 10
rescue Exception
puts 'rescued'
ensure
puts 'ensured'
end
print_if_rescued
# $ killall ruby
# => rescued
# => ensured

91. Other Threads are
NOT!
def print_if_rescued
sleep 10
rescue Exception
puts 'rescued'
ensure
puts 'ensured'
end
t = Thread.new { print_if_rescued }
t.join
# $ killall ruby
# => ensured

92. It commits!
def transaction
yield # thread killed inside yield
rescue Exception => e
rollback # nothing is raised
ensure
commit unless rolledback?
end

93. Solutions?

94. DON’T Kill Your
Threads!
Headius Approv

95. If you do, don’t
use ActiveRecord
+ Threads + MRI

96. 5 Record Certified Enterprise
Active
- ARCECA
“Don’t use
Concurrency Architect
Thread#kill/raise ”

97. Succ
ess!

98. Recap please!

99. 1 - Noob
“Use database
constraints”

100. 2 - Beginner
“Use READ-
COMMITTED”

101. 3 - Half Mouth
“Use an Options
Model”

102. 4 - Advanced
“Acquire locks
consistently”

103. 5 Record Certified Enterprise
Active
- ARCECA
“Don’t use
Concurrency Architect
Thread#kill/raise ”

104. That’s all! Thank
You!

105. Questions?
Slides @mtoledo