In my last post, HTMs were doing mysterious things. The graphic caused you to point and ask:

“What’s happening right there?”

In this post, the graphic answers that question.

Bring back the trivial sequence

Let’s reuse the “totally predictable sequence” from the previous post.

Again, the input data is a repeating sequence: [0, 4, 8, …, 92, 96, 0, 4, 8, …]

We’re now showing:

  • Column overlaps with the input, before and after boosting.
    • If a column has synapses connected to 14 active bits, its overlap is 14.
    • If this column has a boost factor of 1.5, it’s final overlap is 21.
  • Boost factors.
  • How many columns are active because of boosting.
    • Equivalently, how many columns were suppressed because of boosting.
    • To compute this, I pass the pre-boosted overlaps directly into the C++ spatial pooler’s inhibitColumns_ method, then diffed the results.

You can now see that lots of the big unexpected events in the timeline were caused by boosting.

A few things to note:

  • An enormous round of boosting happens at step 51. Every single active column is active because of boosting. This means the 40 columns with the highest overlaps are all being inhibited because of boosting.
  • The “columns active due to boosting” chart almost fits like a puzzle piece into the pre-boosting overlaps chart. Just turn it upside down.
  • Independent of all of this, it’s illuminating to see the overlaps. I was excited to build this just so I could see what it’d look like.


The next question (an ugly preview)

Now that I know what’s going on, I’m wondering: is boosting working?

More precisely, is boosting causing the HTM to make use of all of its columns? Here’s an early glance at whether this is happening.

It isn’t pretty, but here’s a rough time series of Tufte-style box plots.


The distribution of columns’ “time since last activation” is shown by its quartiles:

  • The minimum is the bottom of the lower black area.
  • The 25th percentile is the top of the lower black area.
  • The median is the dot in the center.
  • The 75th percentile is the bottom of the upper black area.
  • The maximum is the top of the upper black area.

This graphic isn’t polished, but it tells a story of neglected columns. Until step 3244, the median column has not been active in the past couple thousand timesteps. Then, boosting causes the median and 75th percentile to drop, meaning a lot of the neglected columns got activated.

With this input sequence, it makes a lot of sense that most columns don’t get used by default. With only 25 unique input patterns and 40 columns per pattern, we will not use all 2048 columns.

In this graphic, the data is too aggregated. Looking at it, we have no idea how the upper quartile is distributed. Toward the end, the enormous black section only tells us that at least one column is still not getting activated, while the 75th percentile was activated much more recently. Though the 75th percentile is still pretty high!

Next steps

  • Find a prettier way of showing whether boosting is working.
  • Add Sanity to these charts. When you zoom in on individual timesteps, it’d be nice to be able to inspect the HTM at those timesteps.
  • Add these charts to Sanity. Generate these charts in real time as an HTM runs.


Appendix: Stack all the things

Here are all four data sets from the previous post, showing every chart. Even the ugly one.

A totally predictable sequence

Repeating sequence: [0, 4, 8, …, 92, 96, 0, 4, 8, …]

A sine wave: mostly predictable

A sine wave that oscillates between 0 and 100, using a period of 8π.

Actual data: kinda predictable

This is Numenta’s “hotgym” data set.

Not predictable: random integers

Random integers between 0 and 100.

Appendix: How I got this data

I generated this data using nupic. I used a custom build of nupic.core that adds two methods to the SpatialPooler class in SpatialPooler.hpp:

  vector<UInt> getOverlaps()
  {
    return overlaps_;
  }

  vector<Real> getBoostedOverlaps()
  {
    return boostedOverlaps_;
  }

Here’s my Jupyter notebook:

My HTM model parameters came from a swarm on the hotgym data set, then I changed ‘temporalImp’ to ‘tm_py’ to use temporal_memory.py. These parameters could definitely be improved. I wasn’t focusing on tuning parameters, I was focused on seeing what’s going on.