Marcus gave a talk called "Recognizing Locations on Objects" during the HTM Meetup on 11/03/2017.
The brain learns and recognizes objects with independent moving sensors. It’s not obvious how a network of neurons would do this. Numenta has suggested that the brain solves this by computing each sensor’s location relative to the object, and learning the object as a set of features-at-locations. Marcus showed how the brain might determine this “location relative to the object.” He extended the model from Numenta’s recent paper, "A Theory of How Columns in the Neocortex Enable Learning the Structure of the World," so that it computes this location. This extended model takes two inputs: each sensor’s input, and each sensor’s “location relative to the body.” The model connects the columns in such a way that a column can compute its “location relative to the object” from another column’s “location relative to object.” When a column senses a feature, it recalls a union of all locations where it has sensed this feature, then the columns work together to narrow their unions. This extended model essentially takes its sensory input and asks, “Do I know any objects that contain this spatial arrangement of features?”
2. Agenda
• The problem, in context:
Detecting locations relative to objects
• Inspiration:
Principles of grid cells
• A solution:
Recall, Normalize, Vote
4. Consider the inputs to the brain.
Sensors move.
The brain is very good at making
sense of sensorimotor sequences.
Sensors are often independent.
Familiar objects often cause novel
combinations of sensory features.
The body is an array of sensors.
With both vision and touch, the brain
receives an array of sensory features.
23. Room 1
After the rat learns a room,
grid cells anchor to the
room consistently.
Room 2
They anchor to other
rooms differently.
“Purely Translational Realignment in Grid Cell Firing Patterns Following
Nonmetric Context Change”
Marozzi, Ginzberg, Alenda, Jeffery (2015)
24. A cool grid cell trick: Modules
Module 2Module 1
+ +
Module 3
Entorhinal
cortex
=
Big space of
unique locations
25. Principles of grid cells
The cell activity moves between a fixed
set of locations.
These locations are mapped onto the
environment.
=+
Body
Motion2. Updating it from motion cues
The brain determines the location by…
1. Recalling it from sensory cues +
Feature
=
29. Column 1 Column 2
Object
Input
Feature
relative to
Object
Object
Input
Feature
relative to
Object
Body
relative to
Object
Feature
relative to
Body
Feature
relative to
Body
New:
Recall past
locations of
sensed
features.
New:
Vote on the body’s location
relative to the object.
Layer 2
Layer 4
Layer 6A
Sensory 1 Sensory 2
46. “Do I remember any
objects that contain this
arrangement of features?”
The holistic system does this:
47. Summary
Location is probably one of the brain’s main primitives.
Grid cells suggest an approach to location:
• Cortex has a “space” of locations that it can process.
• Cortex maps objects into this space.
Cortex could recognize these locations in 3 steps:
Recall, Normalize, Vote
B
A
A
BB