snidel
0.13.0
A multi-process container. Snidel makes it easier for all PHP developers to work with parallel processing without any extensions .
Please consider donating to this project's author, Akihito Nakano, to show your ❤️ and support.
Sponsor @ackintosh on GitHub Sponsors
(en)
Not a few people, start their programming carrier with PHP, and go on. Parallel processing, they are not familiar with it, and may be a hurdle for them.
Or else, people who limited to develop with a language that is not PHP (eg A language that has superior feature for parallel processing). (It's me in past.)
To make parallel processing more easily and instinctively to them to use, I started developing Snidel.
Snidel can be one of your options when you are considering "How to do it parallel?". It's an honor for me.
(ja)
I think there are some programmers who have started programming with PHP and have built up their careers with PHP (I am). For such people, parallel processing may be unfamiliar with it or may seem like it is a high threshold.
Alternatively, there are people who have to proceed with development in a situation where they are restricted from using languages other than PHP (for example, languages with excellent mechanisms for parallel processing) due to various circumstances (this was the case with me before).
We have started developing Snidel with the aim of making it easier and intuitively possible to solve problems using parallel processing.
I hope Snidel can be one of your options if you say, "I want to run this process in parallel, what should I do?"
$ composer require ackintosh/snidel:~0.11.0
It is also possible parallel processing via build-in functions (eg exec ):
initialize_data_required_for_the_slow_jobs ();
exec ( ' php slow_job1.php & ' );
exec ( ' php slow_job2.php & ' );For the developers who feel "pain" with the above, Snidel can provide pretty good experience and will streamline their PHP programming.
We will walk through usage to show how Snidel melt parallel processing into your programming. The experience using Snidel should resolve your pain. Let's get started!
<?php
use Ackintosh Snidel ;
$ f = function ( $ s ) {
sleep ( 3 );
echo ' echo: ' . $ s ;
return ' return: ' . $ s ;
};
$ s = time ();
$ snidel = new Snidel ();
$ snidel -> process ( $ f , [ ' foo ' ]);
$ snidel -> process ( $ f , [ ' bar ' ]);
$ snidel -> process ( $ f , [ ' baz ' ]);
// `Snidel::results()` returns `Generator`
foreach ( $ snidel -> results () as $ r ) {
// string(9) "echo: foo"
var_dump ( $ r -> getOutput ());
// string(11) "return: foo"
var_dump ( $ r -> getReturn ());
}
// If you don't need the results, let's use `Snidel::wait()` instead of `Snidel::results()`
// $snidel->wait();
echo ( time () - $ s ) . ' sec elapsed ' . PHP_EOL ;
// 3sec elapsed.All parameters are optional.
new Snidel ([
' concurrency ' => 3 ,
// Please refer to `Logging`
' logger ' => $ monolog ,
// Please refer to `Using custom queue`
' driver ' => $ driver ,
// a polling duration(in seconds) of queueing
' pollingDuration ' => 1 ,
]);call_user_func_array // multiple arguments
$ snidel -> process ( $ f , [ ' arg1 ' , ' arg2 ' ]);
// global function
$ snidel -> process ( ' myfunction ' );
// instance method
$ snidel -> process ([ $ instance , ' method ' ]); $ f = function ( $ arg ) {
return $ arg ;
};
$ snidel -> process ( $ f , ' arg-A_tag1 ' , ' tag1 ' );
$ snidel -> process ( $ f , ' arg-B_tag1 ' , ' tag1 ' );
$ snidel -> process ( $ f , ' arg_tag2 ' , ' tag2 ' );
foreach ( $ snidel -> results as $ r ) {
// `Task::getTag()` returns the tag passed as 3rd parameter of `Snidel::process()`
switch ( $ r -> getTask ()-> getTag ()) {
case ' tag1 ' :
$ r -> getReturn (); // arg-A_tag1 | arg-B_tag1
break ;
case ' tag2 ' :
$ r -> getReturn (); // arg_tag2
break ;
default :
$ r -> getReturn ();
break ;
}
}Snidel supports logging with logger which implements PSR-3: Logger Interface.
// e.g. MonoLog
use Monolog Formatter LineFormatter ;
use Monolog Handler StreamHandler ;
use Monolog Logger ;
$ monolog = new Logger ( ' sample ' );
$ stream = new StreamHandler ( ' php://stdout ' , Logger:: DEBUG );
$ stream -> setFormatter ( new LineFormatter ( " %datetime% > %level_name% > %message% %context% n" ));
$ monolog -> pushHandler ( $ stream );
$ snidel = new Snidel ([ ' logger ' => $ monolog ]);
$ snidel -> process ( $ f );
// 2017-03-22 13:13:43 > DEBUG > forked worker. pid: 60018 {"role":"master","pid":60017}
// 2017-03-22 13:13:43 > DEBUG > forked worker. pid: 60019 {"role":"master","pid":60017}
// 2017-03-22 13:13:43 > DEBUG > has forked. pid: 60018 {"role":"worker","pid":60018}
// 2017-03-22 13:13:43 > DEBUG > has forked. pid: 60019 {"role":"worker","pid":60019}
// 2017-03-22 13:13:44 > DEBUG > ----> started the function. {"role":"worker","pid":60018}
// 2017-03-22 13:13:44 > DEBUG > ----> started the function. {"role":"worker","pid":60019}
// ... $ snidel -> process ( function ( $ arg1 , $ arg2 ) {
exit ( 1 );
}, [ ' foo ' , ' bar ' ]);
$ snidel -> get ();
var_dump ( $ snidel -> getError ());
// class AckintoshSnidelError#4244 (1) {
// ...
// }
foreach ( $ snidel -> getError () as $ pid => $ e ) {
var_dump ( $ pid , $ e );
}
// int(51813)
// array(5) {
// 'status' => int(256)
// 'message' => string(50) "an error has occurred in child process.
// 'callable' => string(9) "*Closure*"
// 'args' =>
// array(2) {
// [0] => string(3) "foo"
// [1] => string(3) "bar"
// }
// 'return' => NULL
// }
// } Snidel depends on Bernard as a queue abstraction layer. Bernard is a multi-backend PHP library for creating background jobs for later processing.
By default Snidel builds the flatfile driver, but from a race condition perspective, we recommend using a more reliable queue in production.
$ connection = Aws Sqs SqsClient:: factory ([
' key ' => ' your-aws-access-key ' ,
' secret ' => ' your-aws-secret-key ' ,
' region ' => ' the-aws-region-you-choose '
]);
$ driver = new Bernard Driver SqsDriver ( $ connection );
new Snidel ([
' driver ' => $ driver ,
]);For details on the driver, please see here.
Here are articles that introducing Snidel. Thank you!
| Snidel | PHP |
|---|---|
| 0.1 ~ 0.8 | >= 5.3 |
| 0.9 ~ | >= 5.6 |
| 0.13 | >= 7.1 |
We suggest you give it a try with Docker as Snidel requires some php extensions shown in Requirements.
curl -Ss https://getcomposer.org/installer | php
docker build -t snidel .
docker run --rm -v ${PWD} :/snidel snidel php composer.phar install
docker run --rm -v ${PWD} :/snidel snidel vendor/bin/phpunit Snidel © ackintosh, Released under the MIT License.
Authored and maintained by ackintosh
GitHub @ackintosh / Twitter @NAKANO_Akihito / Blog (ja)
Blog entries by author about Snidel (ja):
Thank you to JetBrains for supporting us with a Free Open Source License.
