1. It’s all about eXperience
Gaetano Giunta
PHP London
July 2015
Making Symfony
Services async with
RabbitMq
(and more Symfony)
PechaKucha style!

2. A case study: generating M$Office docs
2
The needs
• Produce content in Microsoft Office formats
• Many formats for each document
• Both Word and Excel docs
• XML content generated by CMS
• LibreOffice used to generate MS Office and PDF
versions
• Has anyone tried generating OOXML by hand?

3. A case study: generating M$Office docs
3
Reality bites
• Slooow
• Rest assured that some of the docs will be pretty big
• Unreliable
• Depending on version, LO crashes from a-bit to almost-
always
• Race-prone
• Two conversion jobs in parallel might step on each other

4. A case study: generating M$Office docs
4
…and the results are:
• Ugly code: lots of polling, copying of files around,
manual locking
• Does not scale at all: only one conversion process
active at any given time
Web
server
PHP
Libre
Office
Waiting processes

5. TextBook scenario for introducing queues
5
Web
server
Worker
?
Rabbit
MQ
Libre
Office
PHP

6. TextBook scenario for introducing queues
6
Web
server
Worker
?
Rabbit
MQ
Libre
Office
PHP

7. TextBook scenario for introducing queues
7
Web
server
PHP
!
Rabbit
MQ
Libre
Office
PHP

8. TextBook scenario for introducing queues
8
Web
server
PHP
!
Rabbit
MQ
Libre
Office
PHP

9. TextBook scenario for introducing queues
9
Web
server
Symfony
!!!
Rabbit
MQ
Libre
Office
PHP

10. TextBook scenario for introducing queues
10
Web
server
Symfony
!!!
Rabbit
MQ
Libre
Office
PHP

11. The rationale
11
• Same library handling both sides of the connection
• Easier to troubleshoot
• Sf Console component is lovely
• Everything developed in a single repository
• No need to learn new languages
• Everything which I can rewrite in PHP, I eventually will*
* = 3rd whimsical law of Gaetano

12. The details
12
• Json used as payload encoding
• The worker has been made thread-safe
• Again, thanks to Symfony: the « Process » Component
• It spins up a new php process each time it receives a command
• The new process executes the command
• Bonus: a ‘watchdog’ to restart workers via crontab if they die
Symfony
listener
Symfony
listener
Symfony
listener
Symfony
worker
Libre
Office
Libre
Office
W

13. The evolution
13
• What is exactly a « queued command » ?
• For starters, each existing Symfony console command!
• Why not every existing Symfony Service?
• Allows to run existing services asynchronously
• As long as their arguments are serializable to json
• Not a project-specific tool any more
• Could be turned into a Bundle
• …
• Profit!

14. The problems
14
• Assumes good security on the network
• Low throughput
• Spinning up a new php worker process takes time
• Could we use php-fpm to communicate to workers
instead?
• Am I reinventing the wheel?
• Others?
• Suggestions are welcome

15. I’ll be here all night
15
THANK YOU
https://github.com/kaliop-uk/kueueingbundle
• @gggeek
• www.kaliop.co.uk