Explaining how to use twisted's inlineCallbacks rarely goes smoothly. It usually ends in Infinite Frustration®. The learning curve is just too steep for a junior developer.
My usual attempts to break it down involved describing
- decorators
- generators/coroutines
- Deferred Objects
- how inlineCallbacks works
This bit from Guido's talk about the new asyncio module, made me revise my approach. It is possible to use inlineCallbacks correctly 88.8% of the time by following 5 simple steps:
- Make sure you decorate the function with twisted.internet.defer.inlineCallbacks
- When calling any method/function that returns a Deferred, use
yieldand pretend it's a sequential "blocking" call. - Structure the function's control flow and exception handling as you normally would
- In case you want to gather results from several Deferred Objects, add them to a list and yield a
DeferredListwith that list. - return a value with
returnValue(it should not be a Deferred though)
from twisted.internet.defer import inlineCallbacks, returnValue, DeferredList
# you_would_write_this
@inlineCallbacks
def get_google():
body = yield getPage("http://www.google.com/")
returnValue(body)
# you_would_pretend_its_this
def get_google():
body = getPage("http://www.google.com/")
return body
# you_would_handle_multiple_calls_like_this
@inlineCallbacks
def get_google_10_times():
url = "http://www.google.com/"
deferrreds = list(getPage(url) for _ in xrange(10))
results = yield DeferredList(deferreds, consumeErrors=True)
returnValue(results)
# full_example
from __future__ import print_function
from twisted.internet import defer
from twisted.web.client import getPage
@defer.inlineCallbacks
def get_multiple(pages):
results = yield defer.DeferredList([getPage(url) for url in pages])
defer.returnValue(results)
if __name__ == "__main__":
from twisted.internet import reactor, task
t = task.deferLater(reactor, 0, get_multiple, ["http://www.google.com/"] * 2)
t.addCallback(print)
t.addBoth(lambda _: reactor.stop())
reactor.run()