All of a sudden it’s trendy rather than scary to talk about artificial intelligence and machine learning. So in this post I’ll explain how delivering a compelling virtual coaching experience depends on these technologies.
Let’s start with AI. It’s easy now to imagine a kind of activity-centric Siri that can produce and respond to all kinds of interesting sentences about the exercise you’re doing. In your imagination, what makes these sentences interesting? Certainly not the ability to recite your speed, steps, and heart rate.
What matters is context:
- What am I doing right now? (You are climbing a hill…)
- How does this compare to past behaviour? (…at a higher effort than normal…)
- How does that fit what I should be doing? (…which is too high given that this is a recovery workout…)
- How does it relate to my plan/history? (…following yesterday’s strength workout.)
- What interpretation is provided by the wider context of my goals? (If you keep this up you’ll quit early and burn fewer calories than your target for this week.)
These are all things that a virtual coach has to ‘know’ about you, across a dizzying amount of contexts. And then it needs to deliver the right advice at the right time in light of this knowledge. So yes, we’ve been busy.
That’s the AI aspect: understanding context and designing appropriate and natural interactions and advice based on that context. The machine learning comes in because everyone’s context is unique.
In the example above, “high effort” was the key concept. High effort is a physiological state, and what counts as high effort will change as a user gets fitter. A virtual coach needs to learn about this. It’s the same with optimal form, which varies by person and by exercise type. Using first principles of physiology and the sports covered by our engine, our algorithms look for patterns in a user’s data, and adjust key coaching concepts as a result. Some of the patterns we can respond to immediately, because they nearly exactly match what we’ve seen and learned about before. Others the system needs to learn about from scratch for this user.
So machine learning helps us identify a particular state that an individual user is in. And our AI puts that state in context, allowing our customers to give users timely and relevant advice. Both are crucial for true virtual coaching.
One of my favorite software innovations is the mouse.
Nope, it’s not much of a hardware innovation. Devising a user interface and user experience that could be driven by a trackball is where the magic came from (HT: David A. Wheeler).
Wearables are the new mice. That is to say, in the wearables arena from 2016 and beyond, innovation is increasingly going to come from software. We’ve seen exceptional hardware innovation recently in areas like accelerometers, optical heart rate detection, and stretchable sensors. Now we need software to tell us what these existing and new sensors are for.
What is a heart rate sensor really for? I can assure you that “To tell me my heart rate” is not a good enough answer. But if accurate heart rate in conjunction with other sensors, machine learning, and an expert system can tell me that my fatigue levels require a recovery session, or that my nervous system is nearing its peak performance for training, then I’m starting to get impressed. Or imagine an application that truly squeezes value out of a barometer, accelerometer, and GPS, by telling you that your stride length is 25% too long for the hill that you’re climbing if you are seeking to build muscular endurance. Now you have a reason to pay a premium for those sensors.
These examples only scratch the surface. As health insurers start to sense the possibilities for using wearables to measure and promote behaviors that reduce risk factors for our most common diseases, they are pushing for the development of additional wearable sensors such as EKG, exercise blood pressure, and heart rate variability detectors. But the outputs of these sensors will be even more incomprehensible to laypeople than those of the current generation of sensors. Usability and usefulness will depend almost entirely on interpretive software, supported by medical professionals.
So the primary reason for wearable hardware vendors to embrace software is to create a user experience where the data inputs are made meaningful and given purpose. Users are given a genuine, life-enhancing reason to stick the hardware in their ears or on their wrists and clothing. But there are other reasons we are about to see this trend take off.
Firstly, clever use of software shortens the cycle for hardware vendors to bring something new and distinctive to market. A minor change in hardware form factor accompanied by software that delivers a radically improved user experience is sufficient to justify a new product release. We saw this in the way that new operating systems would drive the uptake of PCs.
Secondly, an experience-driven approach that relies on software can radically reduce R&D spend for each release. A new feature is much cheaper to code than a new button is to engineer. The automotive industry has recognised this, and more than half of its overall R&D spend is now directed towards software, rather than design and engineering. Modern vehicles exploit millions of lines of code in order to deliver many of the performance features than are used to market them.
The rewards of exceptional chemistry between software and hardware can be very high. In 2012 Flexera calculated that the latest iPhone and and the latest Nokia cost about the same to manufacture, but the iPhone commanded a price premium of 40% over its competitor. And we all know who won that race. The Nokia was a good piece of kit, but its software and application ecosystem were no match for the iPhone’s.
A lot of fitness wearable ventures are currently focused on delivering what I like to call ‘lab tests in the wild’. That is, they want to be able to deliver metrics like VO2max at the end of people’s business-as-usual workouts, rather than in laboratory controlled conditions. It’s a worthy goal, but I get the sense that few of these companies know what they are getting into.
Here are some of the things that I’ve found need addressing during my own team’s algorithm development:
- Sensor accuracy – for instance how GPS readings alter in city canyons, on running loop tracks, and for seemingly no good reason
- Heart rate anomalies from strap issues and warm-up/warm down effects, or as the result of adrenaline, temperature, altitude, and autonomic suppression
- Running on trails rather than pavements, or running with a slower friend
- The effect of incline on various metrics (a delayed one in the case of heart rate).
If these are being addressed in the jungle lab tests out there, then I haven’t seen evidence of it. And I assure you that the impact of these factors can be very big. If you’re fit, but would like a leading device to contradict you, go for a run with it in Manhattan on a hot day.
If we take a step back and try to consider the purpose of these performance metrics, it is to be part of a system that helps us to get fitter – that is, to measure progress. Progress is something that happens over time, and so trend measures are going to be the most useful.
A good fitness program is training both limbs and lungs. Therefore trends in the performance of both of these need to be reported to people who are trying to improve (and the program altered to reflect these trends). A snapshot that bounces all over the place – not so much.
The main users of fitness wearables are in the “three W” category: the worried, wealthy, and well, not necessarily the unhealthy users, for whom the technology could make the biggest difference. The problem — and the solution — lies in the approach gadget makers take.
For years I’ve been frustrated at the debate surrounding weight loss. We are working on an optimal experience for getting exercise (especially active walking) to contribute to reducing BMIs. I was talking about this at Wearable Tech Expo recently, and they asked me to put it in writing. Here you go.
So you’re already a pretty good runner, but you want to get a personal best for the upcoming Peachtree Road Race.
With 60,000 people racing and varying levels of athleticism, getting a personal best time in the world’s largest 10k event only comes with correct preparation. If you’ve been following a professional program, this means that technique, endurance and even speed have been worked on from the beginning, though perhaps integrated in a different way. You should have been working towards your fastest run by running up hills with the same effort as you’ll require on race day when you run fast on the flat. If you’ve done training like this, then you are already on the diligent track to enhancing speed. But assuming you haven’t, there are still some things you can do with one month to go.
My philosophy avoids the prevailing wisdom of cramming in heavy speed sessions closer to the race. Running at your max a couple of weeks before the race accumulates fatigue, discourages personal bests and encourages injury. My tips circumvent this last-minute approach and instead focus on recoverable speed sessions, posture correction and a few other ways to gain free speed with only a month to go.
Recoverable speed sessions
Three weeks out, your speed sessions should be limited to only one session a week. They are sessions you can wholly recover from in a week while also not affecting everything else you can do. This means not running 10km, but instead doing reps of 1km at a pace you intend to run on race day. Running fast generates a high amount of effort on flat terrain, which is painful and not conducive to becoming faster three weeks out. So, ensure these sessions are not as hard, and that the intensity doesn’t hurt you. Instead, focus on running as evenly split as possible.
A professional program would have made you work the same load on your legs prior to doing speed reps. Coming to this phase after following a program, would change how fast your body will acclimatize and condition itself to the new, faster pace. For now, it is best to stick with speed sessions that you can easily recover from. On race day, you want to start at the pace that you can finish at, and no faster.
Your running style will determine how quickly you fatigue and how fast you continue to go. Look above the horizon. Spreading the load around the muscles can be done by running straighter, faster and more upright and this is easily enabled when your eyeline is focused slightly above the horizon. This will also prevent you from leaning forward. The better you are at balancing the load around the body, the less tired you will get, ensuring a sustainable pace throughout the race.
Apart from directly impacting your speed with effort, there are a few shortcuts to gain free speed. Firstly, run in a way that reduces the race distance by a couple of hundred yards. This means navigating towards the inner corner each time the course turns, to slice off yards of distance. Secondly, losing approximately three pounds can help you lose 50 seconds of time. The lighter you are and the lighter your shoes are, the faster you will go. Also, if you run the course before race day, the greater familiarity will also help you maneuver through faster.
Finally, bear in mind the several extraneous conditions that will change race day performance. Hydration levels during the Atlanta heat, how familiar you are with the course and fourth of July excitement will all affect how you sustain your pace to achieve a personal best.
Introduction: Why all these heart icons on my phone?
It seems like Silicon Valley isn’t content with its push into the automotive industry, and now has healthcare in its sights. All of the tech titans seem to be developing platforms for collecting and aggregating personal health and fitness information, and Apple has even literally been running a sweatshop to gather data from energetic employees.
The reason for this focus on activity data is that wearable devices are very suited to collecting it. A key point is that the data can be gathered automatically. This means that users don’t need to spend time manually logging their activity, with the all of the drop off in both compliance and reliability that brings. The business goal is to correlate activity data with health outcomes, providing an evidence base from which to prompt the users of those wearable devices to change their behavior in positive ways. The potential benefits for health promotion are undeniable.
The problem: Not all steps are created equal
One of the most significant problems with Silicon Valley’s healthcare project is that the data being collected lacks context. The data shows that my heart rate rose after I passed the 1,000 steps mark. But was this because I started sprinting? Because I started climbing a hill? Because I was ill? Or maybe because I swung my daughter onto my back when she started to grizzle? Your wearable device doesn’t know. And because it doesn’t record this contextual metadata, drawing inferences from the data is dubious.
The solution: Think multidimensional
At Performance Lab we describe the current reports that wearable devices and apps provide as one-dimensional. That is, they basically report what they are measuring (heart rate, steps, altitude, distance travelled and so forth). A multidimensional report produces insights by considering multiple metrics simultaneously. For example, an “effort” metric (such as heart rate or power output) can be considered together with other metrics that shed light on what is actually going on. These include:
- resistance metrics such as gradient, altitude change, or distance between limb turnover
- a comparative effort metric, such as heart rate the last time the user encountered this type of resistance
- event-based metrics such as an impending stressful event
- real time biomedical metrics, such as blood pressure or lactic acid.
Only a multi-dimensional report can reveal what happened in meaningful terms, like “Your legs got fatigued because your effort on the hills was too high.”
The good news is that the same devices that are capturing non-contextualized data also have the potential to capture the extra dimensions that are required to make the former meaningful. Barometers are becoming standard on smart phones. Together with speed, stride length and stride rate we can use this altitude data to provide all kinds of context – and conveniently accelerometers and the algorithms that interpret them are now smart enough to compute precisely those metrics. Heart rate measurement is firmly established on wrist-based devices, which are now set to provide accurate 24/7 heart rate variability data.
Going to the next level: Real time coaching
The upshot of this coming revolution in activity measurement is that our wearable devices will record in meaningful terms what we did and how our bodies responded.
One benefit of this multidimensional approach is that the project of correlating activity with health outcomes will no longer be dubious. But rather than wait for data analysis to provide us with generic health promotion advice, we can receive real time coaching.
If our wearables can know what we are doing, they can also correct this behavior. We can determine, for a given individual, how his or her unique body works to maximize and improve outcome variables such as power output and fatigue or stress levels. So for instance we can discover the optimum stride rate and stride length for a previously sedentary person looking to lose weight through walking, balancing their need for calorie burn with management of their pain and motivation levels. We can help a time poor executive achieve better fitness in half the time. Or we can change a person’s work and activity schedule to manage blood pressure.
Now that is an exciting project.
Medical Design Technology asked me to contribute some thoughts about how we managed to create such large health benefits in non-active people using automated prompts.
I started writing about how machine learning and the use of big data needs to be informed by deep domain expertise. Three pages later I ended up here.
When my lifting friend grabbed her first weights, she put herself through the ringer in order to see quick results.
She killed form for heavy weights. She ignored routine to see her body grow and tone. But after a month, she saw no change. Even worse, she found that her knees really hurt when she squatted. Injury and flat performance would make any lifter turn to a mentor – someone that would get to know her body type, diet and lifting technique personally. She needed a trainer.
“Knees out, back straight, and mind on the muscle.” The first bid of tailored advice was offered and tentatively bought. I watched my friend motivated and struggling to change her posture but looking structurally stronger.
Then a more radical piece of advice was sold: “10kg lighter!”
After a while, you can’t tell for yourself. You have to trust the trainer who might even suggest 10kgs lighter to help you save your form. After two weeks of lifting lighter, my friend stepped back on the scales and saw four pounds of weight gain with the added rewards of no knee strain. Trust was built, and thereby a foundation laid down for further performance improvement. She immediately called her trainer to tell him the news.
That’s the sort of trust and experience our team is looking to bottle more of each day in our product.
I had an enjoyable chat with Wareable’s James Stables in the UK the other day. James is as tapped in as anyone to what is happening in wearables, so his perspective was interesting. He said that our focus on making sense of data was on the roadmaps of several companies, but he wasn’t aware of others taking a B2B approach to the market. In short, they’re going it alone.
To me, our partnership approach is clearly the right one. Although we feel we have the world’s leading coaching algorithms for wearables, we can’t be the best in the world at everything. New sensor technologies are coming out all the time, as are new wearable form factors. There are established fitness communities we shouldn’t try to replicate. And there are domain experts in areas like nutrition and niche sports that we are excited to collaborate with.
I’d rather work with these specialists to create incredible customer experiences together, rather than keep it all to myself and miss that opportunity.