defaultlist

defaultlist: like a defaultdict, but with lists!

I recently ran into a situation where I had to keep a count of events happening at a specific index, and plot that: some sort of running histogram.

One way to do so is to use a vector, add to it every iteration, and make a histogram of it:

events_indices = []

peaks = cfd(signal)  # We do a peak detect on a signal
events_indices.extend(peaks)

values, bins = np.histogram(events_indices, bins(len(set(events_indices)))

The problem with this is the ever increasing size of events_indices.

One way to overcome this is with a defaultdict, where the key is the event’s index in the signal:

from collections import defaultdict
events_indices = defaultdict(int)

for idx in peaks:
    events_indices[idx] += 1

And this provides a great memory improvement, but makes it awkward to plot, having to do something along the lines of converting to a list:

max_ = max(events_indices.keys())
list_ = []

for idx in range(max_):
   list_.append(events_indices[idx])

So I came up with this defaultlist:

 class defaultlist(list):
     """A list that defaults to 0 if a index is not there.
     This list also grows magically to a given index:

         d = defaultlist(int)

         d[5] += 3
         [0, 0, 0, 0, 0, 3]

         d[17]
         0
     """
     def __init__(self, default, *args):
         if (not callable(default) and not isinstance(default, type) and
            default is not None):
             raise TypeError('first argument must be callable or None')
         self._default = default
         super(defaultlist, self).__init__(*args)

     def __setitem__(self, idx, value):
         if idx >= len(self):
             default = self._default() if callable(self._default) else None
             self.extend([default] * (idx - len(self) + 1))
         super().__setitem__(idx, value)

     def __getitem__(self, value):
         ret = None

         if self._default is not None:
             ret = self._default()
         if isinstance(value, int):
             if value < len(self) and len(self) != 0:
                 ret = super().__getitem__(value)
         if isinstance(value, slice):
             start = value.start if value.start else 0
             stop = value.stop if value.stop else len(self)
             step = value.step if value.step else 1
             ret = [self.__getitem__(idx) for idx in range(start, stop, step)]

         return ret

>>> d = defaultlist(int)
>>> d[5] += 3
>>> d
[0, 0, 0, 0, 0, 3]
>>> d[17]
0

It also works with other types:

>>> d = defaultlist(str, [1, 2, 3])
>>> d[0]
0
>>> d[17]
''

And slicing is not a problem:

>>> defaultlist(int)[2:10:2]
[0, 0, 0, 0]

It’s not necessarily better than a defaultdict, as a great deal of space is used for storing non-events. But a lot of fun to think up.