I maintain a roster of the parameter values and SSEs of Individual objects that a Population has had and possibly replaced.

Parameters N_max The most records I can have in my roster. When the roster is full, adding a non-duplicative Individual will bump the highest-SSE one currently in the roster to make room. The default of 1500 seems like a sensible compromise between reasonably compact .dat file size and informative plots.

Instance Variable	names	A sequence of my individuals' parameter names, supplied as the sole constructor argument.
Instance Variable	X	A 2-D Numpy array of SSEs (first column) and parameter values (remaining columns) of one individual.
Instance Variable	K	A list of indices to rows of X, each entry in the list corresponding to a row of X.
Instance Variable	Kp	A set of the values (not indices) of K that are for individuals currently in the population.
Instance Variable	Kn	A set of the values (not indices) of K that are for individuals who never were in the population.
Instance Variable	kr	A dict containing row indices, keyed by the hashes of Individual instances.
Method	__init__	`History(names, N_max=None)`
Method	__getstate__	For storage-efficient pickling.
Method	__setstate__	For unpickling.
Method	shutdown	Undocumented
Method	__len__	My length is the number of records in my roster.
Method	__getitem__	Access the SSE and parameter values corresponding to index k of my K list.
Method	__iter__	I iterate over 1-D Numpy arrays of parameter values in ascending order of the SSEs they resulted in.
Method	clear	Call to have me return to a virginal state with no SSE+values combinations recorded or considered for removal, an empty population, and an N_total of zero.
Method	value_vs_SSE	Obtains a 1-D Numpy array of the SSEs of my individuals and matching 1-D Numpy arrays for each of the parameter values in names.
Method	kkr	Returns (1) the index k of my K list where the row index of the new record should appear in my X array, and (2) that row index kr.
Method	add	Adds the SSE and parameter values of the supplied individual i to my roster, unless it has an SSE of `inf`, in which case it is ignored.
Method	purge	Purges my history of the record at row index kr.
Method	notInPop	Call this with an integer row index or an Individual instance that was added via `add` to remove its row of my X array from being considered part of the current population.
Method	purgePop	Purges the history of all members of the current population. (Presumably, they will get added back again after re-evaluation.)
Method	_initialize	Undocumented

names =

A sequence of my individuals' parameter names, supplied as the sole constructor argument.

X =

A 2-D Numpy array of SSEs (first column) and parameter values (remaining columns) of one individual.

K =

A list of indices to rows of X, each entry in the list corresponding to a row of X.

Kp =

A set of the values (not indices) of K that are for individuals currently in the population.

Kn =

A set of the values (not indices) of K that are for individuals who never were in the population.

kr =

A dict containing row indices, keyed by the hashes of Individual instances.

def __init__(self, names, N_max=None):

History(names, N_max=None)

def __getstate__(self):

For storage-efficient pickling.

def __setstate__(self, state):

For unpickling.

def _initialize(self):

Undocumented

def shutdown(self):

Undocumented

def __len__(self):

My length is the number of records in my roster.

Note: Immediate result, not locked! Mostly for unit testing.

def __getitem__(self, k):

Access the SSE and parameter values corresponding to index k of my K list.

Note: Immediate result, not locked! Mostly for unit testing.

def __iter__(self):

I iterate over 1-D Numpy arrays of parameter values in ascending order of the SSEs they resulted in.

Note: Immediate result, not locked! Mostly for unit testing.

def clear(self):

Call to have me return to a virginal state with no SSE+values combinations recorded or considered for removal, an empty population, and an N_total of zero.

Returns a Deferred that fires when the lock has been acquired and everything is cleared.

def value_vs_SSE(self, *args, **kw):

Obtains a 1-D Numpy array of the SSEs of my individuals and matching 1-D Numpy arrays for each of the parameter values in names.

Waits to acquire the lock and then calls Analysis.value_vs_SSE on my instance a, returning a Deferred that fires with the eventual result.

def kkr(self, SSE, N):

Returns (1) the index k of my K list where the row index of the new record should appear in my X array, and (2) that row index kr.

First, index k is obtained, by seeing where the K list points to a record with an SSE closest but above the new one. Then each row index in the K list is examined to see if the previous row of my X array is unallocated. If so, that is the row index for the new record. Otherwise, is the next row of my X array is unallocated, that is used instead. If both adjacent rows of X are already allocated, the next row index in the K list is examined.

If there are no row indices in K that point to a row of X with an unallocated adjacent row, the row index is determined to be the current length of k.

Note: With the original for-loop Python, the search for an unallocated row was very CPU intensive when you get a big history accumulated::

   # Pick a row index for the new record
   for kr in self.K:
       if kr > 0 and kr-1 not in self.K:
           return k, kr-1
       if kr < N-1 and kr+1 not in self.K:
           return k, kr+1
   return k, N

The reason is that the list was being searched for an item with every iteration, twice!

The optimized version does this same thing with much more efficiently, by creating a local (array) copy of K and sorting it in place. Then only adjacent elements needs to be inspected with each iteration. (It may be just as fast with a local sorted list instead of a Numpy array.)

@defer.inlineCallbacks
def add(self, i, neverInPop=False):

Adds the SSE and parameter values of the supplied individual i to my roster, unless it has an SSE of inf, in which case it is ignored.

If the roster is already full, bumps the record deemed most expendable before adding a record for i. That determination is made by a call to my ClosestPairFinder instance cpf.

Returns a Deferred that fires with the row index of the new record when it has been written, or None if no record was written.

Parameters neverInPop Set True to have the individual added without ever having been part of the population.

def purge(self, kr):

Purges my history of the record at row index kr.

Removes the row index from my K list and has my cpf instance of ClosestPairFinder disregard the row, because it's now gone.

Note: Does not remove the index from the values of my kr dict, as that is a time-consuming process and the caller can likely just clear the whole thing anyhow.

def notInPop(self, x):

Call this with an integer row index or an Individual instance that was added via add to remove its row of my X array from being considered part of the current population.

def purgePop(self):

Purges the history of all members of the current population. (Presumably, they will get added back again after re-evaluation.)

API Documentation for ade, generated by pydoctor at 2022-11-17 13:13:22.

ade.history.History(object) class documentation

`ade.history.History(object)` class documentation