python - Celery pool types and concurrency -


i having hard time putting how multi-threading concurrency works in celery.

it looks default type use -p eventlet. assuming -p threads not work because of gil, , there no concurrency in practice.

but looks (from example here) can't specify -p eventlet , start firing tasks away, have initiate parallel tasks via celery.group. , according this, multiprocess worker never consume messages in parallel.

so, sum up, looks have true parallelization of tasks, have use multiprocessing. can use eventlet, have modify how tasks run. tasks firing @ random in response external triggers, don't know in advance when , how many need create - doesn't can use eventlet use case.

is correct?

i've started experimenting using eventlet pool myself, can tell far, no, need not use group() call tasks take advantage of concurrency. think it's coincidence used example. 2 concepts play nice together, though. i'm still getting eventlet concurrency tasks applied separate apply_async() invocations inside python loop, fine if don't need groupresult functionality (see http://celery.readthedocs.org/en/latest/userguide/canvas.html#groups).


Comments

Popular posts from this blog

java - WrongTypeOfReturnValue exception thrown when unit testing using mockito -

php - Magento - Deleted Base url key -

android - How to disable Button if EditText is empty ? -