I personally believe that the ranking system is hands down the best measuring stick to gauge performance as it normalizes race conditions, course, and just about every other variable by looking at a huge amount of data.
Short rant: Every time this comes up, there are a ton of opinions and statements from people that have NEVER ACTUALLY DONE THE MATH. So be careful. If you haven’t actually created a program to simulate the algorithm (a simple excel sheet with a macro is more than sufficient) thrown in data from a race to make sure your math is correct THEN played with a data set to see what does and doesn’t affect the scoring system, then your opinion on the matter is about the same as someone telling you the best route to the airport in a city they’ve never set foot in. Rant over.
Having actually spent a good deal of time vetting the system, I think it’s as good as anything that could be done. It normalizes performance on any given race by looking at how everyone did then comparing their performance at that race to all their previous races, which were in turn, were compared to everyone in those races, and so on. It’s like those (cheesy) commercials from the 90’s about safe sex, where they say, you’re not just having sex with your partner, but you’re having sex with everyone they have had sex with, and everyone that those people have, etc etc until the screen is filled with thousands of people. Same thing here; you’re not just comparing how you did against everyone else, but everyone they’ve raced against and everyone those people have raced against, until you get a really good average from the sheer number of it all.
The few limitations I’ve found are:
When there is less than 200 people in a race, the results can be skewed if there are too few pace setters (people with enough scores from the previous year to set the bogie). So be weary of any score from small races.
The longer the race, and especially in rough conditions, the more the thick part of the bell curve gets pushed back. So on a hot day, the FOP of a HIM will be done before the real heat of the day starts, and off the bike course before wind picks up. So the top scores will be higher. Also, stronger athletes are less affected by conditions, so that inflates their score as well (and arguably, as it should). So a sprint in nice weather will have lower high end scores.
Other than that, I think it’s as good as it can be.