Deferred-yielding iterator.

Use an instance of me in place of a task result that is an iterable other than one of Python's built-in containers (list, dict, etc.). I yield deferreds to the next iteration of the result and maintain an internal deferred that fires when the iterations are done or terminated cleanly with a call to my stop method. The deferred fires with True if the iterations were completed, or False if not, i.e., a stop was done.

Access the done-iterating deferred via my d attribute. I also try to provide access to its methods attributes and attributes as if they were my own.

When the deferred from my first next call fires, with the first iteration of the underlying (possibly remote) iterable, you can call next again to get a deferred to the next one, and so on, until I raise StopIteration just like a regular iterable.

NOTE: There are two very important rules. First, you must wrap my iteration in a defer.inlineCallbacks loop or otherwise wait for each yielded deferred to fire before asking for the next one. Second, you must call the stop method of the Deferator (or the deferreds it yields) before doing a break or return to prematurely terminate the loop.

Good behavior looks something like this:

 @defer.inlineCallbacks
 def printItems(self, ID):
     for d in Deferator("remoteIterator", getMore, ID):
         listItem = yield d
         print listItem
         if listItem == "Danger Will Robinson":
             d.stop()
             # You still have to break out of the loop after calling
             # the deferator's stop method
             return

Instantiate me with a string representation of the underlying iterable (or the object itself, if it's handy) and a function (along with any args and kw) that returns a deferred to a 3-tuple containing (1) the next value yielded from the task result, (2) a Bool indicating if this value is valid or a bogus first one from an empty iterator, and (3) a Bool indicating whether there are more iterations left.

This requires your get-more function to be one step ahead somehow, returning False as its status indicator when the next call would raise StopIteration. Use Prefetcherator.getNext after setting up the prefetcherator with a suitable iterator or next-item callable.

The object (or string representation) isn't strictly needed; it's for informative purposes in case an error gets propagated back somewhere. You can cheat and just use None for the first constructor argument. Or you can supply a Prefetcherator as the first and sole argument, or an iterator for which a Prefetcherator will be constructed internally.

Class Method	isIterator	Tells you if obj is a suitable iterator.
Method	__init__	Undocumented
Method	__repr__	We all want to be nicely represented.
Method	__getattr__	Provides access to my done-iterating deferred's attributes as if they were my own.
Method	__iter__	One of two methods needed for me to act like an iterator.
Method	__next__	One of two methods needed for me to act like an iterator. The result (unless `StopIteration` is raised) is a `Deferred` to the underlying iterator's next value, not the value itself.
Method	stop	You must call this to cleanly break out of a loop of my iterations.
Method	_callback	Undocumented

@classmethod
def isIterator(cls, obj):

Tells you if obj is a suitable iterator.

Returns True if the object is an iterator suitable for use with me, False otherwise.

def __init__(self, objOrRep, *args, **kw):

Undocumented

def __repr__(self):

We all want to be nicely represented.

def __getattr__(self, name):

Provides access to my done-iterating deferred's attributes as if they were my own.

def _callback(self, wasCompleteIteration):

Undocumented

def __iter__(self):

One of two methods needed for me to act like an iterator.

def __next__(self):

One of two methods needed for me to act like an iterator. The result (unless StopIteration is raised) is a Deferred to the underlying iterator's next value, not the value itself.

Cool, huh? It took a lot of work to figure this out. You have to play nice, too, calling this method again only after the Deferred fires and calling stop if you want to break out of the iterations early.

def stop(self):

You must call this to cleanly break out of a loop of my iterations.

Not part of the official iterator implementation, but necessary for a Twisted way of iterating. You need a way of letting whatever is producing the iterations know that there won't be any more of them.

For convenience, each Deferred that I yield while iterating has a reference to this method via its own stop attribute.

API Documentation for AsynQueue, generated by pydoctor at 2022-11-17 13:13:24.

asynqueue.iteration.Deferator(object) class documentation

`asynqueue.iteration.Deferator(object)` class documentation