You do know it makes a huge difference where you start out from? Low HR in racing is relative, the actual numbers mean nothing, at all. The guy with the lowest racing HR, is the guy with the lowest max HR, which also doesnt matter…
A guy with a max of 200 and races at 170, is racing with a lower relative HR than a guy with a max of 150 and racing at 140…
You went too slow , that’s it. Or else you’re Chris Froome.
LOuis 
I agree. I have a max HR of 205 and can ride hours at 160-165.
You do know it makes a huge difference where you start out from? Low HR in racing is relative, the actual numbers mean nothing, at all. The guy with the lowest racing HR, is the guy with the lowest max HR, which also doesnt matter…
A guy with a max of 200 and races at 170, is racing with a lower relative HR than a guy with a max of 150 and racing at 140…
AND Race speed is not race speed.
At race speed in an IM my HF is 140; in an OD 150. (max HF about 165).