I was recently reading Ad copy for some educational software with "adaptive technology". All it did was repeat a problem set until they got above 70%. But it got me thinking. We've been talking about the potential of computers for decades. What can we actually do now that would feel adaptive, and would we want that?
A basic education App looks like paper worksheets. It looks as if actual school worksheets were the entirety of the source material. That's not a criticism -- it's the point. The original worksheets have been approved. Teachers and parents understand them. And they also function as tests -- 70% correctness checks off that topic. This is a safe approach. You can say it's boring, or a waste of computing power. But it's slightly better than the old way (given that we already have tablets).
We should rightly be suspicious of anything the computer does which we can't easily explain, and which doesn't correspond to anything in the real world. But we have a real-world model for an adaptive App: a one-on-one tutor. We understand tutoring. They skip around in worksheets, make up problems on the spot, even help solve problems. Sometimes they go so far as to fold or cover up part of the problem to make some point.
So our adaptive App isn't going to be easy to explain, and won't produce grades. That's not so good. On the plus side, it might be much more interesting to use. It might be better at teaching the subject.
We only have two ways to watch progress. We can count mistakes, or time actions. But we can be clever how we use that.
Timing is only useful for giving hints. If they don't do anything for 20 seconds, we can change a color to make it easier. It's no good if they think there's a timer, so we can randomize the delay a little. That's an old trick -- interesting randomness feels like the computer was thinking about it and made a decision. We could take 20 to 25 seconds: each second we'll check with chance (time-19)/6. We could add a small chance to jump the gun at 10-12 seconds, or wait 30+. Little things like that make the computer seem more alive.
But the real trick is keying off previous events. That's the definition of adaptive. If they've been making lots of mistakes, we give a hint sooner, or start with hints turned on. If they get a few correct, the hint goes away for the next problems.
That leads to mistake counting. Our old worksheets put scoring at the end. Instead, we can check as we go. A simple running total (right-wrong) isn't too bad. A score of -2 or less turns on hints, with a score of 0 or more turning them off, for example. We could also count the current win/lose streak. Randomness also works here. Each correct answer could be worth 1-2 points (if we set the goal at 15 points, that's 5-15 problems, but almost always 9-11).
The other trick with counting mistakes is doing it over the steps in one problem. Suppose we're sorting by size. After moving a small square into the wrong bin twice, we can float it into the correct one. Then we reset the per-item mistake count back to 0.
Now to explain things that don't work. With timers, the computer can't tell why it took so long. Maybe the cat jumped in their lap. If they answer quickly if could be confidence, or boredom. We can't assume anything based on time taken. Giving a hint is the only safe thing. The other problem is that Big Data is hard. If they get 3+2 wrong we could remember that. We could remember which wrong answer they used. But it's unclear and awkward to try to use that.
Instead of repeating a worksheet until we see a passing score, we can keep it going. Using our cumulative total trick we can quit when it's high enough or enough correct in-a-row. A nice thing about this is they never fail -- there's never a screen where we tell them they aren't good enough and have to try again from the beginning. The drawback is not being able to point to a passing score in a discreet quiz.
We can also chain problem sets together. Most computer problem sets are short, as in ten problems: 1+2 through 10+2. Once we think they've passed the x+2's we can seamlessly go into x+3's, maybe followed by randomized +2's and +3's. Putting them together tells a little story. We could put together x+2 followed by 2+x for an informal lesson on the commutative property.
A nice thing is that we can dip down briefly. If they have trouble with 2+6 and 2+8, the error count will dip us briefly into the easier 6+2 and 8+2 for just a few problems.
Sometimes we have small problem sets that aren't really a sequence. For example, matching "three" with "3" with "ooo" (3 dots) and vice-versa. We can chain a few of those together in whatever order seems best.
Chaining don't sound like much, but it feels different than stopping, getting a score, then going to a new set. Especially if we don't have a set number of problems. One longer, changing, problem set feels more like someone giving a lesson.
Sometimes the problem sets don't break out nicely, but there will be some numbers we can gradually change. We might start with sums up to 4-7, ramping up to 4-8, 5-8, and so on. It looks especially nice if we can display a several problems at once, as if harder problems are gradually creeping in.
We can mix the tricks. Sort by color, then shape, then both, with gradually more items.
If they aren't doing well, we can switch to a different exercise. Technically this is simple, but in practice it's rare.
We have to make a completely new second exercise when one would do. That's a lot of work, but maybe we bought a company that had one. Or we started on one, rejected it, wrote another, then realized how to fix the first. Once we have more than one exercise, we need the discipline to hold the rest back. That's hard since we very much want to show off every exercise we have, not have them buried.
This is a broad term for giving hints. More formally, making the problem easier to solve. We could add highlights, display a number-line, or simply solve it after a few mistakes. It's very specific to the exercise and often tough to think up good ones.
A traditional App might have one exercise with each level of scaffolding. For example, counting blocks where tapping them shows the numbers; another where tapping merely shows dots; and a last with no help. We can merge those together, adding or removing scaffolding based on mistakes. As usual, we throw in some randomness so it feels as if the computer is guiding the child to count with less and less help.
Showing the correct answers in an App can be dicey. Kids might accidentally learn the goal is to tap anything twice to get an auto-solve. A trick is to add a long enough delay after mistakes so that picking the correct answer seems faster.
If we have a small number of problems, for example a dozen sight words, a computer can remember the right/wrong count for each. The ones they know will gradually drop out of rotation. When we get down to only a few we can mix in a few of the solved ones. That whole system is doable, and fine.
But it's only a useful trick for small memorization problems. For things like 1-digit addition (1+1, 1+2 to 9+9) that's 81 problems. Too many to drill on. And, we've got better ways of teaching addition besides memorizing. If they can at least count-out 6+7, they can figure out 6+8.
The other variation of remembering specific information is cross-activity. We might remember they're bad at 7+9. But how do we use that later, as part of a larger problem? More? Less? And generating good problems is hard enough without using/avoiding certain combos. All-in-all, it's lots of work for possibly no or negative gain.
An easy way to make an adaptive-seeming App is to start with small, similar, worksheets. Then use the running-total and randomness tricks to combine a few end-to-end. That feels alive since users don't know what it will do next. The problems may gradually get harder, or give less help, or may flip around (a few 3+2=_'s may suddenly turn into 4+_=6's). Especially with a little randomness, it can feels as if the computer is watching and deciding what to give you next.
But first, make sure that people want an adaptive system. Because even though it might be better at teaching, it's harder to explain beyond "your kid will do better on tests, trust me".