Let's just say that their lack of willingness to share the model or the model error is, to me, a giant red flag.

---

Alan Couzens, M.Sc. (Sports Science)
Exercise Physiologist/Coach
Twitter: https://twitter.com/Alan_Couzens
Web: https://alancouzens.com

Thanks for chiming in Alan. I was hoping you would.

Individually identifiable data can be anonymized - Athlete 1, athlete 2... But that is not the issue. The data and model are 'trade secrets' - there is zero motivation to release the details. In fact, from a business POV, why would you enable your competitors. There *might* be someone able to take the data you've provided and create a better model/product. Why take that chance?

I’m not convinced that simply anonymizing individuals data is sufficient to get past the ethical considerations here.

I’m also not really convinced that their model itself is of great interest to be made public anyway. As I understand it, in a nutshell it’s a method of taking a bunch of parameters about an athlete, their goals, and their cohort(s) and choosing the next workout for them to do. Much of the IP they have is the database itself. There’s some other stuff in there that they talked about, like FTP prediction, but that really isn’t all that interesting to me. They themselves said that it’s inherently pretty fuzzy math to do those predictions.

So when Alan talks about the error bars in the model, I’m not quite sure how relevant that is for TRs use case. (He’s the data scientist, not me, I’m not saying I’m right, just that I don’t understand). Like, how do you judge the “error” in picking workout X vs workout Y?

---

Swimming Workout of the Day:

Favourite Swim Sets:

2020 National Masters Champion - M50-54 - 50m Butterfly

Well, perhaps you get at the workout x vs workout y question in aggregate, not by individual (I'm not a data scientist, either!). BUT...I think that the premise of the model is that 'if you do this, then the result will be that'. So, maybe in terms of FTP change (or other parameters). So the 'error' is...the model says that my adaptation after doing this workout is A. Then I test (or otherwise get a measure of that parameter). But it's not A but 50% of A. Or 150% of A. That's the 'model error'. I think! But so tricky for many reasons... maybe you can only understand the magnitude of the change when you test. There might be *many* workouts between tests - so there would be some uncertainty there - if your outcome isn't what the model predicted, which workouts did not 'work'? Or even worse, what if the prediction was perfect, but 50% of the workouts were *more* effective than the model prediction and the other 50% were *less* effective - the combination makes a perfect result. How would you know that?

Isnt the TR dataset severely biased? TR has the most data on those where their training methodology is successful. Everyone else either quits or is a minority in the dataset with occasional successes and occasional fails. If they use machine learning as their principle for suggesting workouts won’t there be just a biased set of a few workouts? How can they effectively suggest a plan without manually adjusting? I’m having a hard time understanding how they can be effective for a lot of cycling population without a normally distributed dataset. TR talks a lot about all their data- I don’t understand how their data isn’t just a representation of who likes them the most. Can someone explain what I’m missing?

According to them, 50% of the rides uploaded on the platform aren't from people using the plans. They have a pretty solid analytics platform, so my understanding is that plenty of people upload their training history to replace something like TP, and plenty of people use TR through their coach or just cherry pick workouts as part of their own plan.

I can't agree more with Alan's points. Getting the prediction error down for a machine learning model that represents how each individual athlete responds to different training is not easy and took quite a while to figure out for AI Endurance. There's no excuse not to display the prediction error to the individual user both from a privacy or IP point of view. That they don't disclose which model they are using exactly is understandable though.


There are other influences on performance of course (for example nutrition, sleep, stress) but you gotta start somewhere and many of us have hundreds if not thousands of recorded activities. Power data (GPS data for running) is most of the time rather clean (unless badly calibrated recording device) and abundant compared to those other influences on performance so it's the right place to start.


Different things really do work for different people (we see that with our AI plans all the time - they're really quite different for each athlete) so the training plan should be mostly created based on the athlete's own historical data. However, the machine learning models can be pre-trained with data from other athletes that are of similar fitness level and age. This helps when athletes don't have a lot of data or if they have never tried a particular training routine. If you don't do the pre-training and an athlete has never done intervals and always ridden at 150 W, guess what FTP the model is going to predict no matter what training routine you input to the model? Machine learning at the end of the day is just fitting data and you need to have your prediction errors under control, make sure you are inferring within sensible inputs to your model where the predictions are actually reliable and handle a ton of edge-cases (way more than I would have ever expected) to make any sensible training recommendations.

---

Founder of AI Endurance
https://aiendurance.com

Thanks for the kind words, RandomTriGuy! Definitely a topic that I have a hard time leaving alone

---

Alan Couzens, M.Sc. (Sports Science)
Exercise Physiologist/Coach
Twitter: https://twitter.com/Alan_Couzens
Web: https://alancouzens.com

Exactly! These are the type of questions that we need to be asking TrainerRoad!

Presumably, there is some metric (or metrics) that the model is optimizing for when it makes its decisions. It might be something like maximize FTP. Now, the model might say, I'm going to pick workout X over workout Y because the model says that workout X will add 3W to my FTP, while workout Y will only add 2W. The belief that we put in such a model (& whether we choose to follow the models recommendation) will be greatly different if we know that the modeled v measured benefit of the workout (i.e. the error) is +/-1W vs +/-5W. This is why it's so important when talking about any ML (research/product etc) that we expose the error. It, & it alone, tells us how much we can trust the model.

---

Alan Couzens, M.Sc. (Sports Science)
Exercise Physiologist/Coach
Twitter: https://twitter.com/Alan_Couzens
Web: https://alancouzens.com

I don't think, based on what was said in the podcast, that they are trying to predict a specific performance gain outcome from a specific workout. That would be rather silly, IMO, since (as we all know) it isn't individual workouts that create improvement. Workouts actually make us slower, its the recovery from them and superadaptation that causes performance increase.

What I "think" they are doing is looking at "you", and various other "you's" from their database (I think they have like 100 million rides in their database now) and essentially pattern matching to find a set of "you's" who have the best outcomes for the performance metrics that you are prioritizing. So if you are a crit racer, it might be 10s and 1min power (for example), if you're a climber it might be FTP, etc... Once they find that set of "you's", it's a matter of looking at which workouts got them to those outcomes and presenting you with that workout or set of potential workouts, with various modifiers included based on your subjective feedback to the system on how easy/difficult a workout was, and then repeat after the next workout is completed, etc etc.

Whether that's AI or not, I don't know. Really, I don't care either, as long as it enables me, as an athlete, to make better decisions on my end.

As an aside, they are rolling out FTP prediction, which (I believe), will have error bars on the output. They talked about the uncertainty reduction over time in the podcast.

---

Swimming Workout of the Day:

Favourite Swim Sets:

2020 National Masters Champion - M50-54 - 50m Butterfly

You're asking very good questions there. This optimization through time is one of the things that makes decision making in training prescription difficult to model. While I gave a simple example in my response above of a single workout, the reality is that each workout has effects on other workouts over time & what we're really looking for is not better performance every workout but the best performance on a certain date. This can & does mean that we actually want worse performance on some days to reach the best performance on a given day!

Your question "which workouts didn't work?" is an important and very computationally tough question to answer! This is the classic credit assignment problem (https://ai.stackexchange.com/...gnment-problem/12909) If we do a certain sequence of workouts and it leads to a given performance, how much credit do we assign to each of the workouts? Was it the big 6hr long ride that I did 3 weeks before the race or that string of consistent 20hr weeks that I did 3 months out? There are various ML strategies that we can use to help to answer these questions. Is TrainerRoad using them? I have my doubts.

---

Alan Couzens, M.Sc. (Sports Science)
Exercise Physiologist/Coach
Twitter: https://twitter.com/Alan_Couzens
Web: https://alancouzens.com

If that's what they're doing, it's a very basic algorithm called K-nearest neighbours. It's really not suited to time-series optimization &, I might add, it still lends itself to assessing an error between predictions and actual that they could be sharing. I suspect they're not sharing it because, used in this context, the predictive accuracy would be poor.

---

Alan Couzens, M.Sc. (Sports Science)
Exercise Physiologist/Coach
Twitter: https://twitter.com/Alan_Couzens
Web: https://alancouzens.com

Not necessarily the public. But it is for science. Peer review is how we validate work. It's also how models are improved.

The Banister impulse-response model was validated through peer review. Then it was extended and improved in myriad ways by independent work.

The Couzens criticism here is that wrapping up the model in a black box prevents both validation and extension: two keystones of the modern scientific process.

Now I'm hoping that (assuming TR's model is legit) that once they've established market position, they can start to expose portions of their intellectual property for peer review and possible contribution to the scientific community. There is precedent in industry. For example Google fiercely guards some aspects of its AI work, but it released Tensorflow and a very cool dataset for self-driving vehicle work to the public. Of course Google has the luxury of vast sources of income to fund these charitable acts that TR doesn't. So I get it. I just hope that one day they get to a similar position.

I haven't watched the YT podcast yet however the feature I think is really exciting. With all the data they get from users and the extensive library they have, this should be a homerun for TR.

I for one am more than willing to throw myself into their ML black box and see what happens. If I get faster, great. If I don't, I'll do something else. Their plans have done wonders for me in the past so I'm more than happy to give them the benefit of the doubt without seeing error bars or having their model peer reviewed.

Just finished listening to the podcast. This is about where I am as well. I’ve enjoyed using TR the past few years and have been happy with my results. It seems like this should make training with TR even better. I’ll go with it and if it works well for me, awesome. If not then I’ll just make some adjustments and try something different. But with the good race results, ftp increases, etc that I’ve had using them (versus other plans/programs/etc), I will definitely give them the benefit of the doubt even if I can’t see behind the curtain.

---

Matt

Same general take! My biggest gripe, and slot of others folks too, is the workouts were too hard for triathletes. Often for me I feel like I over perform the rest this should help balance out things when I fail the first trwshhold workout.

It will be very interesting to see how they can improve it for triathletes though. For pure cyclist it is just so straight forward good.

Did trainer road allow any room to differ the training suggestions from my crit buddy who has trained for 8 yrs and myself who focus's on 8hr mtb rides with 2 years of training? Surely they can't be the same suggestion!

You mean prior to Adaptive Training? That would have been mostly up to you to select a plan consistent with your goals, e.g. high volume vs. mid or low. And also cycling specialty. Then the actual workouts would be scaled by FTP per the ramp test.

I think it was about a year ago that they rolled out plan builder (which I think you can use to a certain extent like a preview without signing up). Basically you answer a couple questions about your fitness and interval background, how many hours a week you have to train, and what type of race your goal is as well as when it is and it picks out the most appropriate recommended base build and specialty plan. They designed they as they noticed that the majority of the questions on their podcast were “what plan would your recommend if...”. I’ve used it a couple of times and it seemed to work well for me.

---

Matt

Has anybody gotten into the closed beta yet?

I’m in the beta group (in fact I was part of really early testing last year as part of my participation on the forum). I’m a week into SSB1 HV using adaptive training and so far it’s pretty interesting. I made a video that I posted on YouTube to share with interested folks, so anyone can feel free to message me and I’ll share the link (not quite ready to be spamming forums and self promoting!). My goal is to do a weekly training update showing which workouts are suggested during a week as I complete workouts. I will say that at least on first glance that SSB1 really scales back on the workouts that are more difficult on the sweet spot scale. Happy to share my ongoing experience

That sounds like a very interesting as I'm waiting for entry. Let me know when your video is up as I'd love to see your experiences.

---

Kiwami Racing Team

I’m in the closed beta group. There are def bugs and issues as expected. One key missing piece that has not been implemented yet, is the recognition of outdoor unstructured rides - this completely skews my fitness level as 4 out of my 6 weekly rides are unstructured. When they do add this functionality, it will give me a better understanding and perspective on the product.

---

_______________________________________________