So, if I'm reading this right, the Neural Network model is better able to predict diminishing returns from training stimulus than the Banister/TSS model is? Not only that, but it can also do so on an extremely personalized level for each athlete?

I'll go a step further and say that the Banister/PMC model, in and of itself, cannot account for diminishing returns *AT ALL*

Wonder if Coggan will chime in on this thread.

I think he might have been part of the purge.

Wonder if Coggan will chime in on this thread.

Hi Mr. Couzens,

is the Neural Network a model to calculate training load data? And what would be different for example in the chart below to see the right turning points in training loads easier? The chart shows the recovery from a femur fracture and the pure fun of riding with a 1 month break in Mai/June ..

Your article is an very interesting read that challenges my language knowledge, too :-)

Bottom line: In this day and age, there are far better models available.

Hi Alan, interesting stuff for sure (although the math and computer science is a bit much for me.) Could you discuss, in layman's terms, a bit about the predictive capability you've referred to? I've always used the PMC (perhaps ignorantly, I suppose) as a snapshot of where I currently am and where I've come from. Also, is there a way for the layman to put the neural network concept to use, or is programming knowledge required?

That was a really interesting read! Is there a widely available system that you recommend which uses NN? Also, what do you see as the period for calibration period of a NN based model to be a reasonable method of prediction?

It has always felt to me that there is a large jump in fitness to be gained in an initial fitness build followed by a plateau that doesn't seem to be accounted for in traditional tracking; however, I don't recall seeing a platform that tracks the diminishing returns and models through a method that "learns"-so to speak--an individuals response to high level training.

I do not know the algorithm used for Xert (https://www.xertonline.com/), but it certainly appears to be a multivariate model that is used to predict training load. Are you familiar with it?

Last logged on Oct. 20 so yeah

Thanks STJ!

Sure thing!

Best way to think of it would be a traditional scatterplot with training load on the x axis and some performance measure on the y - it could be a race, an FTP test etc. If you place all of your tests on there you'll probably have a chart of somewhat escalating dots, i.e. as load goes up performance will tend to generally go up to a point. Importantly, the shape of this 'generally going up' will vary between athletes (& it might even go down at some point!)

That's the problem with proprietary 'secret sauce' models: NOBODY knows what algorithm they are using - even the people paying for it!! :-)

I suspect it's just a bunch of hard coded if-then statements rather than any true Machine Learning but who knows?

actually

I was in contact with the developer and it seems they feel they have disclosed the methods they use on their web site. It might be worthwhile to take a look and see what they say on their page http://baronbiosys.com/support-learn/.

Um, no.

That's marketing fluff. Not a technical description of their model.

After reviewing it, I'm still none the wiser as to the type of machine learning algorithms (if any) that go into their model.

We have not disclosed the details of our methods but we've been very open on the principles on which they work...

It's not "AI" or "ML" in the sense of, say, an NN....

We make no claims on the accuracy of the prediction model and rely on our users to judge for themselves how well it works for them....

I do look forward to seeing more on your NN implementation. We are looking at NN in combination with our models to bring applications to a broader range of sports and activities.

I actually enjoy reading your posts here and on the wattage/cycling physio groups.

I get the feel you are excited to explain your product but some people interpret it as marketing, which I don't feel is the case.

There was a purge?

Thanks Armando,

I appreciate your honesty (and support!)

FWIW, for others reading, I have no problem with 'if-then rules' systems so long as they're advertised as such and not as "A.I." If athletes know who is writing the rules for the system they're in a better place to judge the worth of the system, especially in the absence of a loss function that quantifies the true predictive accuracy/value of the system.

That said, using a model that quantifies its true predictive accuracy is obviously the better way to go.

Best,

"an analytical model applied as a control system"

Just when I thought we were making progress in moving beyond the buzzwords and accurately defining things 😊

Thanks Armando,

Much clearer.

So it's essentially a 3x Banister model for different points on the power duration curve?

Are the time decay exponents also updated recursively for the individual athlete?

Thanks for the additional info!

More proprietary goobly good. I'm out. Thanks for the chat.

Maybe this was answered above and I missed it:

Along with the stress inputs do you need to tell it what are the good outputs or do testing (race) every once in a while to let the network adjust the weights or will it just pick up on the better results?

This is what really interested me as well. Sounds from above as though it will adjust the weights (e.g. using HR to normalise) so that it gets more data points without having to test too often. That would be a big plus for me.

Also then takes me on to another tangent as to what other data points it can take as input. E.g. I know my HR varies quite a bit depending on temperature, is typically lower at any given power cycling indoors vs outdoors, and to a lesser extent I see some HR variation depending on time of day. My HR also varies quite a bit depending on how rested I am, when tapered I hit higher HRs at any given pace/power. Presumably a sophisticated enough NN could pick up all those data points (it's all in the Garmin files) to predict fitness/performance better. A NN could also maybe make better use of the manually recorded data points that are captured in TP such as RPE.

Other thought is that I'm finding HRV increasingly useful for tracking non-training stress (travel, poor sleep, sickness, bad day at work, etc). Started using the HRV4Training app back in November, still learning how best to use the data but already it's apparent that in my case at least non-training stress can make the PMC largely irrelevant. E.g. in November I was pretty busy with work, spent a lot of time in airports, planes, hotels and high pressure meetings. Second half of December and YTD on the other hand have been very quiet. I've been disciplined with training, diet and sleep throughout (TSS per week, weight and average sleep hours have stayed fairly consistent). So using PMC you'd predict similar performance now to back in November. However if I look at my Recovery Points in the HRV app, they've jumped from an average of 7.6 in November to 8.5 in the last 4 weeks. And that actually tracks far better with my training/testing numbers, which took a big dip in November but are now a bit above where I was at in September/October. Seems pretty obvious looking back that the HRV numbers picked up on the work/life stress, and that that stress had a significant impact on my performance. So I wonder if you could feed in HRV numbers to a NN model as well, and use those as a proxy for measuring non-training stress and the impact it has?

Probably letting my imagination run away with me a bit here, but I'm another one who has always found PMC to be a fairly blunt tool at best and would be very excited if something better came along.

I know you can get an EF inside the analysis of a workout and that it can be calculated per segment as well as for the whole workout. But I am not sure I've seen an EF graph. I'd love an EF graph over the season.

This is what really interested me as well. Sounds from above as though it will adjust the weights (e.g. using HR to normalise) so that it gets more data points without having to test too often. That would be a big plus for me.

Also then takes me on to another tangent as to what other data points it can take as input. E.g. I know my HR varies quite a bit depending on temperature, is typically lower at any given power cycling indoors vs outdoors, and to a lesser extent I see some HR variation depending on time of day. My HR also varies quite a bit depending on how rested I am, when tapered I hit higher HRs at any given pace/power. Presumably a sophisticated enough NN could pick up all those data points (it's all in the Garmin files) to predict fitness/performance better. A NN could also maybe make better use of the manually recorded data points that are captured in TP such as RPE.

Other thought is that I'm finding HRV increasingly useful for tracking non-training stress (travel, poor sleep, sickness, bad day at work, etc). Started using the HRV4Training app back in November, still learning how best to use the data but already it's apparent that in my case at least non-training stress can make the PMC largely irrelevant. E.g. in November I was pretty busy with work, spent a lot of time in airports, planes, hotels and high pressure meetings. Second half of December and YTD on the other hand have been very quiet. I've been disciplined with training, diet and sleep throughout (TSS per week, weight and average sleep hours have stayed fairly consistent). So using PMC you'd predict similar performance now to back in November. However if I look at my Recovery Points in the HRV app, they've jumped from an average of 7.6 in November to 8.5 in the last 4 weeks. And that actually tracks far better with my training/testing numbers, which took a big dip in November but are now a bit above where I was at in September/October. Seems pretty obvious looking back that the HRV numbers picked up on the work/life stress, and that that stress had a significant impact on my performance. So I wonder if you could feed in HRV numbers to a NN model as well, and use those as a proxy for measuring non-training stress and the impact it has?

Probably letting my imagination run away with me a bit here, but I'm another one who has always found PMC to be a fairly blunt tool at best and would be very excited if something better came along.

Hi Alan, last year when I read this article I was very interested both as a triathlete and as a ML PhD student.

How did you acquire the data to use in these tests? Is it available to share? I would be glad to play a little with that to...

Keep up he good work!

Thanks Sperris,

Agreed, tracking long term E.F. is one of the most useful metrics.

The easiest (non code) way to do it (assuming we're talking Training Peaks) is to download the csv summary of your workouts. In that you'll have Intensity Factor and Heart Rate. Providing you know what your FTP setting was, you can create a separate column that gives Normalized Power (FTP*I.F) then you divide session NP/Heart Rate and you'll have a spreadsheet with all E.F. values through the season.

Hope this is helpful.

Best,

Alan,

This is super cool! Nice.

I'm a data scientist and have used LSTMs to take past 3 race times to predict next race time, and find—particularly using 70.3s to predict 140.6 times—much precision.

Some thoughts:

• 1D convolutional input layers on workout time-series metrics (HR, pace, cadence)?
• Compressed representations of workouts for ingestion by performance prediction models?
• RNN or LSTMs for performance prediction?
• RNNs for generation of workouts with inputs of (the above features)?

Therein lies a couple of the issues with TP:

1. The CSV download doesn't include NP, VI, EF, or most of the other analytics that they compute.
2. TP also doesn't track your Thresholds (S/B/R: HR, pace or power) overtime. So, you can't go back and reconstruct the past.
3. No amount of complaining, suggesting, or otherwise seems to convince them of the utility of either #1 or #2 (or rather they refer to WKO+).

I, too, track EF over time. But, it has its issues:

1. HR (and thus EF) is affected by a lot of things, so short term (and even medium term) trends can be hard to tease out of the noise of HR fluctuations from fatigue, temperature, and daily hydration status.
2. EF is a function of intensity. So, you can "force" a higher EF simply by working harder. Its less affected than pace or power, but its still affected.

I've taken to trying to normalize my EF back to a fixed HR, and characterizing my EF per bpm above/below that HR. For example, at the moment my average easy run EF is about 1.45 y/bpm @ 151bpm. I see a slope in my EF of ~0.0125 / BPM. So, if I go do a tempo run at 159 bpm (+8 bpm), I'll get an EF for the tempo run of around 1.55 y/bpm.

I expect that the relationship is probably non-linear, but the daily noise in HR makes it hard to see. Maybe a set of ramp tests could tease it out a little better. Then of course, once spring gets here the whole thing gets swamped by outside air temperature (for the run especially), and comparing EF @ 85F one day to EF @ 60F the week/month before or after is nearly useless.

Hello Alan, your work on NN is intersting and promising but i think you're not that faire to the Bannister model in your critic.

It has non linearity in it, it is a lot harder to go from 100 to 110 than going to 0 to 10. It feel kind of logarithmic when you apply it.
Also as there is no performance tied to fitness you make what you want from this data, it might correlate proportionally to cycling power... or not for example.
Finally i am not surprised your NN correlate better because they are tuned to fit when the bannister model is most of the time universal. That said you can tune the constants in it to better fit a specific athlete. I bet you would have better correlation by optimizing the constants for a specific athlete.

Keep up the good work.

Thanks Tom,

Agree with all. Frankly, it's why I started the process of learning to code - so I could get the metrics I want when I want them!

You're right with the E.F. Because of the impact of resting HR, it's easier to generate higher E.F.'s at higher outputs. I normalize this by applying the Karvonen formula to the HR, rather than using a straight division. More on that here...

https://alancouzens.com/blog/VO2Scores.html

Best,

E.F. is the efficiency factor (Normalized Power (or pace)/HR).

Are you also taking into account other factors (like heat, duration, what was done yesterday, etc) that will effect this metric?

I haven't gotten into the details of what you are doing, but I love the idea. I tried to goad the experts into providing a much more sophisticated and individual training/response algorithm many years ago. Just toss in what you think the important variables might be, track and evaluate them, and see what really is. The more you learn (more data you have) the more precise it gets.

Are you also taking into account other factors (like heat, duration, what was done yesterday, etc) that will effect this metric?

I haven't gotten into the details of what you are doing, but I love the idea. I tried to goad the experts into providing a much more sophisticated and individual training/response algorithm many years ago. Just toss in what you think the important variables might be, track and evaluate them, and see what really is. The more you learn (more data you have) the more precise it gets.

Not to belabor my above question, but is "training_data.csv" publicly available so this work can be replicated, or is that proprietary?

If it's proprietary, is anyone aware of a "canonical" public set of training/performance data used in other studies to evaluate the ability of training data estimators to predict performance?

I bet TrainingPeaks, Trainerroad have absolute treasure troves of that type of data, but likely keep it closely held for both privacy and competitive purposes.

That said, there are some great open sourced options to play around with these concepts/code. I would definitely recommend taking a look at Golden Cheetah's open data set...

https://github.com/GoldenCheetah/OpenData

as a great resource. Mark & the Golden Cheetah team have put a lot of work into going through this process of getting explicit permission and anonymizing a big data set to make it available for this very purpose. Big thanks to them!

Maybe I missed it, but once more, what are the individual data points in this new model?

If each workout is a single data point, then I'd still think, that the NN overfits, and quite badly at that.
Because even on logistic regression, which is much more stable and, with appropriate controls, much less likely to overfit, you need at least 100 datapoints just to estimate the model intercept reliably. There's no way to get around the maths, unfortunately, so while a proposed model might work quite well on a limited sample of athletes (note, that in this particular case - on people with VO2 max 65+), its generalization to "mortal" population might be much less predictive.
And NN eventually - just a bunch of polynomial regressions put into a single model ;) https://arxiv.org/abs/1806.06850

On the other hand, I absolutely applaud the effort of trying to modernize the nowadays-30+ years old performance prediction models. As correctly noted by Alan, real life experience clearly shows that the performance doesn't depend only on such a simplified metric as TRIMPS/TSS, so incorporating additional parameters ("features") in the model should bring improvements in prediction accuracy, provided an individual athlete has a sufficient training sample size (which might be the real issue here).

One should also, however, carefully think on model selection. While "nonlinearity" of NNs seems attractive at first, logistic regression with penalized splines is able to model these quite well provided the additivity assumptions still hold, with the big added bonus of clearly explainable parameter impact on the final model. Stepping a couple of steps further, I'd guess adoption of the Bayesian modelling framework should work even better here because at the end of the day that would allow to obtain full predictive performance distribution for particular athlete, and credibility intervals are just that much better explainable in practice.

olmec wrote:

PS While I love calling all of this AI, these NNs are just ML... (complex function approximators)

Difference between machine learning (ML) and AI:
If it is written in Python, it's probably machine learning
If it is written in PowerPoint, it's probably AI

https://twitter.com/...379612282885?lang=en

;-)

Last logged on Oct. 20 so yeah

Can you plug nutrition, heigh, weight, stress level, personality traits, etc into Neural Network model to increase predictive power?

I assume that eventually we will be able to plug DNA info as well to understand how various genetic traits correlate with performance.