How to improve your scores: What Lumosity teaches us about learning.

The mustachioed man behind the desk at the public golf course used to have a nickname for me – The Excavator. Whenever I trundled into the clubhouse, he would chortle something sarcastic about the amount of dirt I was going to rearrange on his golf course. He wasn’t a mean man, he simply saw me for what I was. A very poor golf player.

I used to get flustered. I used to stomp past his desk and onto the driving range while the mustachioed man peered calmly through the window. He would inevitably see a heavy clod of grass soar through the air while an untouched golf ball perched motionless on the tee.

Since those early days, I’ve been learning. Learning and practicing and persisting. My chuckling nemesis behind the desk gave me a pointer – keep eye contact with the ball. I employed this advice, and the hailstorms of dirt became less frequent. I began making adjustments that ensured I’d be rewarded by the crisp smack of club against ball. I was no longer The Excavator, though I couldn’t seem to hit the ball with any distance.

So, the mustachioed man gave me a fresh tip – don’t roll my wrists. As soon as I attempted this new strategy, I became The Excavator once again, showering dirt throughout the air. I felt as though I was regressing, altering techniques that had been working just fine. The mustachioed man would only nod, encouraging me to keep trying. It took a hundred swings – maybe two hundred – but eventually I began reconnecting with the ball, back to my former consistency. Only now, the ball flew further.

It seemed every few weeks, this man and I would cycle through a routine. He would offer insight to improve my game, I would fail miserably as I implemented the new strategies, and after a little persistence and a lot of cursing, I would find my performance better than it ever had been.

At this point, dear reader, you may be asking, “Why is this author telling me about his golf game?”

Because my story, like so many other stories of improving a skill, is strongly connected to a body of science studying learning patterns. As early as 2014, the Lumosity database contained data for over 2 billion game completions from 65 million registered users. These data points have allowed cognitive scientists to discover some truly interesting findings. In 2015, Lumos Labs researcher Dr. Joseph Hardy collaborated with Dr. Yoni Donner of Stanford University to publish a paper investigating the nature of individual learning curves. Their findings have advanced what we know about an individual’s ability to improve their game scores.

Donner and Hardy examined Lumosity data from four different games, averaging the performances from a set of Lumosity users and graphing the points to create a learning curve. What they found was not particularly shocking – the averaged learning curve looks like this:

When first playing one of the four mentioned games, the averaged data indicates most users perform pretty poorly, just as I – The Excavator – couldn’t even hit the golf ball on my first swings. But, as most users continue playing, they see a rapid increase in performance, analogous to my quick jump in golfing ability after I adopted and practiced the (albeit simple) technique of keeping eye contact with the golf ball (many thanks to the mustachioed man!).

Eye contact only got me so far in my golf game, however. Eventually, I reached a point of what Donner and Hardy call “diminishing gains”. While I could consistently contact the ball, I couldn’t get the infernal thing to fly any farther than I could throw it, no matter how hard I yelled at it. The averaged Lumosity data reveals the same story – after adopting a strategy that leads to a rapid rise in improvement, it seems as if most users’ hold onto this strategy all the way until a final performance plateau. The averaged learning curve appears like one continuous hump with a flattening top.

But is that really how we learn and improve? Clinging to one strategy without adaptation, inflexible to anything new? Donner and Hardy felt there was something missing from this observation on learning. They suspected that averaging Lumosity data was “potentially masking more complex dynamics underpinning the curves of individual learners”. After all, I didn’t stop at the first technique suggested by the mustachioed man. Rather, once I plateaued with the initial strategy, I adopted a new strategy he so kindly proffered, and I pushed past the first plateau toward a longer golf shot. Had someone averaged my performance with thousands of other fresh golfers, you wouldn’t be able to detect the shift in my strategies.

Donner and Hardy began graphing individual learning curves rather than an averaged learning curve. This nuanced analysis was only possible given Lumosity’s ability to capture thousands of individual performances over repeated game completions. They found the data points for most individuals’ performance could be more accurately traced with a series of consecutive learning curves, rather than one continuous curve:

What exactly does this revised model mean in terms of behavior?

Donner and Hardy’s new model (based on individual learning curves rather than averaged data) suggests that most users transition through multiple strategies over the course of playing many iterations of the game. Each implementation of a new strategy brings a fresh learning curve. The researcher’s findings suggest most users will engage a strategy until a plateau of diminishing gains is achieved, at which point the user may consciously or unconsciously adopt a new strategy (see Point A in the image below). Interestingly, when a user changes strategy they typically see a drop in performance below their previous plateau (i.e. they perform more poorly than when using their earlier strategy, note Point B below). If the new strategy is effective, however, that user usually experiences a rapid improvement beyond their former peak performance (Point C). But, this improvement will also level out with diminishing gains, leading to the possible adoption of yet another strategy.

Donner and Hardy’s model of consecutive learning curves seems to accurately describe my own experience learning golf. I saw rapid improvement and a subsequent performance plateau with the first advice from the mustachioed man. He gave me another suggestion, and when I adopted it, my performance initially plummeted. As I practiced the new technique, my performance quickly rose to surpass my old abilities, and I found myself crushing golf balls into the next county.

What, then, are we to take away from Donner and Hardy’s findings?

Let’s start with the hard truth: when you first start playing Lumosity games, your performance will probably be much worse than someone who has played the game 500 times. You will likely be some version of The Excavator. On the flip side, if you continue playing, there’s a high chance you’ll see your performance rapidly increase. So my first suggested takeaway is to practice and persist. Don’t let your early performance discourage you! Even Tiger Woods removed some pretty hefty chunks of dirt from the planet when he first started.

Second, when beginning a new game, it’s not a bad idea to try a known strategy. Perhaps a friend or relative (perhaps even with a mustache) with more experience can give you a tip. If not, pick a strategy for yourself and keep persisting. Even if you don’t consciously adhere to a strategy, you may be subconsciously veering away from ineffective approaches and toward effective techniques as you gain feedback on what’s working – much like I’m able to improve my golf game by letting my mind and body subconsciously adjust with each swing of the club.

Third, when you’ve reached diminishing gains with a particular approach to a game, be adaptable. Try a new strategy. And when you try it, be aware your performance will likely suffer temporarily. If you’ve picked an effective second strategy, you will likely see your performance jump beyond your previous plateau after enough gameplay. Or it may be that you picked an ineffective second strategy, and if after persistent attempts to make it work you’re still in a performance trough, be adaptable and try something new.

Fourth, even though you’ll inevitably reach a point of diminishing gains, it doesn’t mean there aren’t still gains to be had. Let’s say you’ve played a game 1000 times, you’ve tried all the strategies you can think of, you’ve felt your learning jump through multiple curves, and you’re confident you’re employing the optimal approach – you can still improve your performance! However, you will need to be patient. Diminishing gains doesn’t mean “no gains”, it only means that you require more and more iterations of the game to see improvement. Tiger Woods has spent decades perfecting his golf strategy, but that doesn’t mean he’s given up on improving. He is at such a high level of performance that he probably expects it will take thousands of practice swings to see his accuracy and distance improve marginally. If you’ve reached the pinnacle of performance in a Lumosity game, you can still push that score a notch further by being patient and continuing to practice (and don’t forget to appreciate all the gains you’ve already made!). Or, if you’ve grown frustrated with the diminishing gains, Lumosity offers multiple games for each of its cognitive foci, so you can still train the same cognitive skills with a novel challenge.

Whether you’re just beginning your Lumosity journey or you’re a seasoned Lumosity veteran, you can always improve your performance. Sometimes those improvements will come in rapid spikes, and sometimes the improvements will require you to grind and persist. All of your gameplay has given our scientists and research partners the data to understand these spikes, troughs, and plateaus. It is this author’s hope that Donner and Hardy’s research has given you a stronger understanding of your own learning curves.

Author: Greg Kyle

References: Donner, Y. & Hardy, J. L. (2015). Piecewise power laws in individual learning curves. Psychonomic Bulletin and Review. 22: 1308-1319