How Fast Should You Change Your Mind?

There’s a waakye spot you go to every week. It has been good for months. Then one day the food is bad. Do you stop going?

Probably not. One bad plate out of dozens is just a bad day. But if it’s bad three times in two weeks, you start to wonder if the cook changed. And if it were a new spot you’d only tried twice, one bad plate might be enough to put you off it.

The same surprise, a bad plate, changes your mind by different amounts depending on how settled you think things are. Nobody tells you how settled things are, either. You have to learn that too.

I’ve been learning about a model of exactly this, the hierarchical Gaussian filter, or HGF. It comes from computational neuroscience, where Christoph Mathys and his colleagues introduced it in 2011, and researchers use it to describe how people learn when the world changes at a speed they don’t know. Here’s how I understand it, with a small simulation of a coin you can play with.

The simplest way to learn#

Start with the classic learning rule, Rescorla–Wagner, from the 1970s. You keep a belief, and after each outcome you move it a fixed fraction of the way toward what happened:

new belief = old belief + α × (what happened − what you expected)

The part in brackets is the prediction error: how surprised you were. α (alpha) is the learning rate: how much of that surprise you act on. With α = 0.1, you move 10% of the way every time.

The trouble is that α is fixed, and no single value works everywhere:

1. A small α ignores noise, but it’s slow to notice when something has really changed.
2. A big α catches real changes quickly, but it overreacts to every bad plate.

What you’d want is a small learning rate when things are stable and a big one when they’re changing. People seem to do this. In a 2007 study by Tim Behrens and colleagues, people tracked rewards that were stable in some periods and kept switching in others, and the weight they gave each new outcome went up when the rewards were volatile.

So where should the learning rate come from?

Stack the questions#

The HGF’s answer is to stop treating the learning rate as a setting and let it come out of what you believe. It does that by stacking beliefs on top of each other. Take a coin whose bias can change over time:

1. The outcome. Did this flip land heads or tails?
2. The tendency. How likely is this coin to land heads right now? This drifts over time.
3. The volatility. How fast is that tendency drifting?

Each level sets how much the level below it is allowed to move. If you believe the coin is volatile, you expect its bias to wander, so you let new flips move your belief about it a lot. If you believe it’s stable, you hold your belief steadier.

You could keep stacking (how fast is the volatility changing?), but three levels is the usual setup. “Gaussian” describes how each level is modelled: as a value that wanders by a random amount every step, with the size of the wander set by the level above. “Filter” means it updates once per observation, the way a Kalman filter does, instead of going back over all of history.

Every level does the same update#

When a flip comes in, each level updates with the same shape of rule:

weight = how reliable the new signal is ÷ how sure you already are
change in belief = weight × surprise

The HGF calls “how sure you are” precision. If what’s coming in is reliable and you’re unsure, the weight is big and you move a lot. If you’re already sure and the signal is noisy, the weight is small and you barely move. That weight plays the role α played before, except it changes from flip to flip.

Two kinds of surprise move through the stack:

1. Outcome surprise, between the outcome and the tendency: “I expected heads and got tails.”
2. Volatility surprise, between the tendency and the volatility: “My belief about this coin just moved more than I expected it to.”

The second one is the part I like most. When your beliefs keep getting pushed around more than you predicted, the top level concludes the world is more volatile than you thought. That raises how much the middle level is allowed to move, so the next surprises move you more. When things calm down, the reverse happens. The learning rate is a consequence of your beliefs, not a constant you picked.

Doing all of this exactly with Bayes’ rule doesn’t give you a formula you can compute. Mathys’s contribution was an approximation that gives simple update equations you can run one flip at a time, which is what makes it practical to fit the model to real people’s choices.

Watch it learn#

Here’s the coin. For the first 100 flips it lands heads 90% of the time. Then it starts switching between 10% and 90% every 16 flips. After flip 228, it settles on 10%. Two learners watch the same flips: the HGF in blue, and Rescorla–Wagner in orange.

Here’s what to look at:

1. The middle panel. In the first stable phase, the HGF’s learning rate sinks to about 0.04 as it grows confident. When the switching starts, it roughly doubles to about 0.08 and stays up, and after the switching stops it slowly comes back down. Rescorla–Wagner’s rate is the flat orange line. It never moves.
2. The bottom panel. The volatility belief jumps as soon as the switching starts. That jump is the part of the rise that comes from learning about volatility. There’s another part, which I’ll get to below.
3. The sliders. Set ϑ to 0.001 and the volatility belief can barely move, so the two blue lines in the middle panel merge. Push κ toward 3 and at some point the model breaks. Or look for an α that keeps up with the switching phase as well as the HGF does, then see how jumpy it is in the stable phases.

What surprised me while building it#

I expected the learning rate to rise during the switching phase. I didn’t expect that about half of the rise has nothing to do with volatility.

A belief near 50/50 is easier to move than a belief near 90/10, just because of how probabilities work. During the switching phase the HGF’s belief spends a lot of time in the middle, so it moves more per flip even if its sense of volatility never changed. That’s why the demo has the dashed line: the same HGF with its volatility belief frozen at its starting value. The gap between the solid and dashed lines is what learning about volatility adds. It’s real, but it’s smaller than the headline number.

Two other things I ran into:

1. Coin flips say little about volatility. My first version switched between 20% and 80% every 20 flips, and the volatility belief barely moved. From the flips alone, that looks a lot like an ordinary noisy coin. I had to make the switching sharper before the effect showed up, and real experiments run into the same limit.
2. The approximation can break. With a large κ or ϑ, one of the precisions can go negative, and the update stops meaning anything. The demo shows a notice when that happens instead of drawing nonsense. People fitting the model to data have to keep its parameters away from that region.

Parameters as personality#

This is where the HGF gets interesting outside of the maths. The model has a few parameters, and each one describes a style of learning:

1. ω (omega), baseline volatility: how changeable you assume the world is before you see any evidence.
2. κ (kappa), coupling: how much your sense of volatility changes your learning rate.
3. ϑ (theta), meta-volatility: how quickly your sense of volatility itself can change.

Researchers give people a learning task, fit the HGF to each person’s choices, and compare the fitted parameters between groups. In one well-known study, Rebecca Lawson, Christoph Mathys, and Geraint Rees found that autistic adults tended to overestimate how volatile their environment was, and were less surprised when their expectations were broken. This line of work is part of a field called computational psychiatry.

The parameters are hard to pin down, though. Different combinations can produce nearly the same behaviour, and ϑ in particular is often hard to estimate from a single task. So the fitted numbers are best read as the model’s description of how someone learned in that task, not as a measurement of who they are.

Back to the waakye spot#

What stays with me is that, in this model, how quickly you change your mind isn’t a fixed trait. It comes from what you believe about how stable things are, and that belief is learned, and it can be wrong. Someone who shrugs off a bad plate and someone who swears off a place after one bad day might not learn any differently. They might just disagree about how much the spot changes.

If you want to go deeper:

1. A Bayesian foundation for individual learning under uncertainty (Mathys et al., 2011), the original paper.
2. Uncertainty in perception and the Hierarchical Gaussian Filter (Mathys et al., 2014), the general version.
3. pyhgf, a Python library, and the HGF Toolbox for MATLAB, if you want to fit the model yourself.

Thanks for reading this far. I’m Aikins, and I love to build stuff.