You are a Pattern-Matching Agent

Or, “My (crude) model of human thought”

Epistemic Status: Likely


I almost want to learn how to draw. Until recently, my feelings about drawing were (statistically speaking) probably similar to yours: it’d be cool to do, but it the type of thing other people are good at, not me. Naturally, my thoughts weren’t so blunt, e.g. “I don’t know how to draw because it wouldn’t be useful to me”. Right.

Then, I read Dennet’s Consciousness Explained. I haven’t finished it, of course, but that might be fixed in a few months (I’m a busy dude). It’s not a drawing book, in case you’re wondering, but something I read in it made me think that perusing basic competence in sketching stuffs would be something I’d want to eventually do.

In the section I recently finished reading, Dennet argues against the idea of ‘pictures in the mind’ in a particularly persuasive way (to me, anyway): if we really did have ‘pictures’ in our heads, then everyone would know how to draw; it’d be as simple as transferring the internal pictures to an external medium, the only barrier to everyone being professional-quality (photorealistic) artists would be hand-eye coordination (which tends to be fairly good in humans).

But that isn’t the case, which was to be demonstrated, thus completing the reductio ad absurdum.

The persuasiveness likely doesn’t carryover in my blunt-force summary, but you just need the gist of it. This argument resonated with me when I first read it, partially because I had a superficially similar pet theory of my own. I’ll try to explain it, though I’ll stress the mental environment and train of thought which gave birth to this idea have long since dissolved, so I can’t faithfully re-construct my exact thoughts anymore than you can.

Personally, I have no idea how other people mentally see their act of mentally seeing, but for me, it’s always been a very los-res affair. Seeing a dog and visualizing a dog are utterly distinct things to me. One was ‘high definition’, with minuscule details and phenomenal stability. The other was, in word, not. Which isn’t to say that I lacked imagination, just lacked a vivid imagination, if such a thing exists.

Now, my original hypothesis about “how visualizing works” is that we don’t actually visualize.


The long(er) of that is that we don’t actually ‘see’ ‘pictures’ in our heads, we just think we do. The brain can’t re-construct your entire visual feed, it’s too information rich. You just think you’re ‘seeing’ the object of your visualization, if that makes any sense (which isn’t immediately obvious). I’m having trouble faithfully explaining exactly what the notion means, which usually implies it doesn’t mean anything at all.

I don’t know if my earlier pet theory involved visualization triggering the activation patterns associated with what you’re trying to visualize, which is what I now believe happens, but I do think “believing you’re visualizing when you really aren’t” was the central theme. Suffices to say that I agree with Dennet’s account more than that of my past-self, inasmuch as they disagreed on this.

You may wonder “what does these remarks have to do with drawing”? Well, not much. I was originally going to opine about my new pet theory that learning how to draw would improve my brains ability to visualize, but along the way I caught a glimpse of something more interesting.


Let me do a face-heel turn and come at my thesis from an entire different angle.

I have a peculiar way of looking at reality.

Peculiar for humans, at least, I do have this arrogant presumption that my way of looking is superior than others in a meaningful sense, but eh, I digress.

Pick something up, hold it up in the air. Stop holding it.

You’ll see it falling down, like it’s suppose to. That sense of “things fall down” is something everyone understands. For me, that sense is a bit more ubiquitous. It’s my favorite lens for understanding reality: everything can be seen as “things falling down” in a deep sense, primarily through the second law of thermodynamics.

For the most part, we humans are complex enough that our deterministic nature can easily be swept under the rug. Yeah, you could understand a human through atom-by-atom simulation if you really wanted to (and had enough memory), but it’s mostly an academic consideration.

Or so I thought.


The first step of my argument is noticing how we come to learn words.

Disclaimer: I’m no neuro-linguist, or neurologist, or linguist. Full stop, I assume the process is something kinda/sorta like my crude descriptions. I’d love to learn that I’m wrong, since I’m fascinated with the brain, so correct me if you’re able.

I digress. The ‘short of it’ is that as you learn a word, like dog, it’s associated brain-circuits (the sounds (or symbols) of ‘dog’) becomes so closely connected to its referent that only one pattern firing is enough to activate the other. So seeing a dog activates your ‘dog’ circuits, seeing ‘d’,’o’,’g’ or hearing /dôɡ/ activates your ‘seeing a dog’ circuits.  Like culturally inherited synesthesia.

So, when someone is telling story that has something to do with dogs, e.g. “I saw a dog and ran like hell”, hearing the phonemes associated with ‘dog’ triggers your seeing-a-dog circuits and and that’s the origin of ‘mentally seeing a dog’ phenomena.

The deterministic horror crept in when my mind noticed to the other time when you might be imagining a dog. Thinking. You’re thinking about whether you fed your dog, say, and the mentally image of your dog creeps into your mind.

When thinking along those lines, it’s very tempting (and I tried at first) to just say “the brain triggers the seeing-a-dog circuits and …”. but then something in that clause stopped me dead in my tracks. The brain triggers the seeing-a-dog circuits? Something about the notion caught my attention.

It looks suspiciously like off-loading your agency to another part of the brain. Agency is located in the brain, full stop. It’s cause by something which is not, itself, an agent*. So we can’t reject an explanation just because it doesn’t feel ‘agent’-y

So something in the brain caused that mental image to appear, and it wasn’t ‘me’, because ‘me’ is my brain.


Surely there is nothing more of an extension of our will more than our capacity for imagination and thought?

It’s a tempting notion.

I think it’s false, or at the very least, not well-founded.

Our capacity for somewhat unlimited thought is the keystone of our sense of freedom. We can imagine alternatives, rationally weigh casts and benefits. If freedom is real thing, it’s going to be made of pieces that, while adding up to freedom, still aren’t freedom per se, just like all other complex things.

Complex, open-ended, metaphorical thought is assuredly a component of our freedom. So, thought isn’t free or unconstrained itself.

Here’s an example: think of the proof of the Riemann Hypothesis.

You can’t. This, of course, effectively proves thought isn’t free. You can’t think just anything at all, as surely as you can’t teleport or build castles on clouds.

To think of a proof of the Riemann Hypothesis, you need

  • to actually know what a mathematical proof is
  • to actually know what the Riemann Hypothesis is
  • to have a sufficiently rich ‘ground’ of mathematical understanding
  • et cetera

The point: thoughts are like constructions, you have build them on something. Thinking is like walking, in that you can only go to a location immediately near you.


We tend to have a intuitive model of our thoughts as a kind of uncaused causation that leaps into reality from the essence of our being or whatever. It’s a bit hard to describe owing to it’s inconsistency.

I’ve sketched a demonstration that our thoughts aren’t fully free, and are confined to at least rudimentary interaction with the rest of the causal web of reality.

I can go farther.

If words and meaning are just connections between symbols and reality so tight they are inseparable, what are thoughts?

The label ‘thought’ is kinda an umbrella, so let be more specific, when someone has a kind of internal narration going on, what really happening at the lower levels?

My idea is that it’s pattern-matching all the way down.

Consider what happens if I asked you to solve this problem:

5 + 4 = ?

It’ll take you fractions of a seconds for your brain to complete the pattern and bring the solution: 9.

Now, let me pretend to ask you to solve this:

54 + 77 = ?

Unless you’re a savant, the answer isn’t as obvious. But let me digress.

During my early grade school years, we had these inter-class competitions called ‘addition bowls’ and ‘multiplication bowls’, likely named after the Superbowl sports competition. In these competitions,the the students would line up on to two teams, side-by-side, and the students at the front would be shown a flash card with two numbers. Whichever students answered first would ‘win’ and go to the back of the line, the other student sitting back down. Whichever team didn’t lose all their members won.

Naturally, these competitions were quite effective at teaching basic arithmetic operation. The point of interest here was that this kind of game came down to differences in seconds.

If you’re beholden to Kahneman’s Thinking Fast and Slow, you’ll recognize this type of training as creating into System 1 response to symbols like “2+3=?”. Where seeing them causes a king of lookup table operation, retrieving another symbol. Allow me to call this style ‘shallow’ pattern-matching.

You did shallow pattern matching on my first request, to sum 5 and 4. The other request, to sum two two-digit numbers, activated what call ‘deep’ pattern matching, primarily because it concerns a pattern ‘deep’ with in the object in consideration.

The type the pattern matching is the domain of System 2, you recognize it as something you don’t know offhand, but can solve for. You are still completing a pattern, but this time the completion isn’t the answer, but the general procedure for producing it. You look at the equation, and your brain realizes the answer can be acquired by doing this, that, and the other thing.


I was allowed a digression, so now the responsibility falls upon me to connect it to the subject of this post.

Earlier, I briefly touched upon the phenomena of internal narration as a target for reduction to ‘mere’ causal determinism.

Let me pick that thread back up.

When someone is lazing about, entertaining themselves by narrating their mundane actions as if they were a character in a novel, what’s actually going on?

I don’t have all of the answers, but I’d bet I have a rough sketch of the underlying process.

As children, we read stories, we’re taught in school how language and grammar works, we gradually start to grasp the patterns ‘deep’ within the language. Most importantly, we’re understanding the maps between language and reality well enough that we can reverse it; fiction is pretty spectacular once you look at it like that.

We gained a functional pseudo-introspective apparatus as a side-effect of our social competence, i.e. we needed to learn to answer questions like “What are you thinking?”, “Why are you doing that?”, “What’s you opinion on x?”, etc.

Narration in this perspective is ‘merely’ using our mastery of language to describe our internal world.

More generally, (verbal) thinking is using our deep linguist pattern-matching ability-to-describe-with-language on memories, including our previous thoughts. ‘Reasoning’ then, is just another deep pattern-matching affinity, primarily being able to generate notions that follow logically from whatever notion you have grasped in your mind’s hand.


In third grade, I had an existential crisis. We had this school-specific pledge of ‘good behavior’, essentially. It mentioned something about choosing to do the right thing, and I though about how absurd that was. I was something of a problem child, but every time I did something ‘bad’, it was a response to specific stimulus, and I considered that the appropriate response (I don’t think this way any more, of course).

Latter that day I continued that train of thought, thinking about my actions while riding my bike. I occurred to me that this kind of determinism applied to all of my actions, and spent a few minutes trying to disprove it. I would swerve suddenly to prove to me I was an uncaused causation. It didn’t work, because immediately afterwards I’d noticed how my swerving was a response to my thinking that I should be able to make ‘free’ choices, which was response to my thinking earlier about choice, which was response to the pledge, etc.

The thought that your actions are determined, that love, hate, and our works of art and thought, are just by-products of what can effectively be considered the scaled up version of ‘things falling down’ is pretty distressing.

My response to this reaction is note that this feeling comes from conflating the external view of reality with the internal view of reality. Like conflating ‘truth’ and ‘provability’ in formal systems. The most cases they line up quite nicely. But there are quite a few cases where they don’t, and it’s important to distinguish them.

Gödel’s famous “This formal system can’t prove this statement” statement is a nice example. A statement about humans in this vein is something like “You won’t clench your fist in a few seconds”.

Either you’ll clench your fist right now to be contrarian, or you won’t, to be meta-contrarian. So the statement is either true or false.

But how to you tell which?

This is pretty easy, but I’ll give you a chance to figure it out on your own.

Simple: you decide.

No tricks or escape clauses. The truth of the statement is entirely your choice.

It’s still determinism. Unless some extremely lucky bit of quantum noise aligns just right in order to impact the firing of your neurons, your choice is still written in the state of reality at moment. But Omega couldn’t swoop down and determine your choice before you make it.

I might need to elaborate the last point. Omega, as ze is used in philosophical thought experiments is said to have perfect (or near-enough-to-perfect) accuracy at determining the behavior of physical phenomena. So it’s inconsistent to say ze suddenly can’t for feel-good reasons.

But no. Omega gets it’s data by simulating you. In other words, Omega learns of your choice from you actually making the choice. Only it’s in a simulation. Same thing.

You can never know what you do before you do it, this is because you choose what to do. Determinism simply means you determine your choices.


One thought on “You are a Pattern-Matching Agent”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s