1. Gearman
'The manager'
”since it dispatches jobs to be done,
but does not do anything useful itself.”
2. Presentation done using info from...
http://www.slideshare.net/pcdinh/gearman-and-asynchronous-processing-in-php-applications-6135047
http://assets.en.oreilly.com/1/event/45/The%20Gearman%20Cookbook%20Presentation.pdf
http://www.gearman.org
http://nz.php.net/manual/en/book.gearman.php
others...
3. Scalable Solutions..
More Hardware
Caching
Precalculated Data
Load Balancing
Multi-tier application
Job Queue
4. History
Created by Danga Interactive.
Some company that developed Memcache.
Original implementation in perl (2005).
2008 rewriteen in C by Brian Aker
PHP Extension by James Luedke
5. Used by
Digg: 45+ Servers, 400K jobs/day
Yahoo: 60+ servers, 6M jobs/day
And many others..
6. Installing
Compiling:
tar xzf gearmand-X.Y.tar.gz
cd gearmand-X.Y
./configure
make
make install
Starting server:
$ gearmand -d
Pecl extension:
tar xzf gearman-X.Y.tgz
cd gearman-X.Y
phpize
./configure
make
make install
To add to the php.ini:
extension="gearman.so"
7. Terminology
Client: Create jobs to be run and send them to a job server.
Worker: Run jobs given from the job server.
Job Server: Handle the job queue form clients to workers.
13. Gearman Client
Fishpond_Controller_Front::getResource('gearman')
->getGearmanClient()
->doBackground("updateCompetitorPrice", $this->_barcode);
->do("updateCompetitorPrice", $this->_barcode);
GearmanClient::do() - Run a single task and return a result
GearmanClient::doLow() - Run a single low priority task
GearmanClient::doBackground() - Run a task in the background
GearmanClient::doHighBackground() - Run a high priority task in the background
GearmanClient::doLowBackground() - Run a low priority task in the background
16. $client = Fishpond_Controller_Front::getResource('gearman')
->getGearmanClient();
//adding gearman tasks
$client->addTask("getProductDetail", $barcode);
$client->addTask("getPrice", $barcode);
$client->addTask("resizeImage", serialize($barcode,100,100));
$client->addTask("getRecomendations", $barcode);
//callbacks to know when this finish
$client->setCompleteCallback(array($this, "complete"));
//runing tasks
$client->runTasks();
/**
* Callback when task is complete
*
*/
public function complete($task)
{
$data = $task->data();
}
17. Task Methods
GearmanClient::addTaskHigh() - Add a high priority task to run in parallel
GearmanClient::addTaskLow() - Add a low priority task to run in parallel
GearmanClient::addTaskBackground() - Add a background task to be run in parallel
GearmanClient::addTaskHighBackground() - Add a high priority background task to be run in parallel
GearmanClient::addTaskLowBackground() - Add a low priority background task to be run in parallel
GearmanClient::runTasks() - Run a list of tasks in parallel
18. Client Callback
GearmanClient::setDataCallback() - Callback function when there is a data packet for a task
GearmanClient::setCompleteCallback() - Set a function to be called on task completion
GearmanClient::setCreatedCallback() - Set a callback for when a task is queued.
GearmanClient::setExceptionCallback() - Set a callback for worker exceptions.
GearmanClient::setFailCallback() - Set callback for job failure.
GearmanClient::setStatusCallback() - Set a callback for collecting task status.
GearmanClient::setWarningCallback() - Set a callback for worker warnings.
GearmanClient::setWorkloadCallback() - Set a callback for accepting incremental data updates
19. Scatter / Gather
Concurrent tasks with different workers.
All tasks run in the time for longest running.
Must have enough workers available.