When we train - the most beneficial adaptations occur at the level of the muscle. In endurance sorts, our ventilatory ability (O2 in, CO2 out) is rarely our limitation (unless you have clinical syndromes such as asthma / COPD / …). Where most people are limited is the ability of their muscles to take oxygen and use it to produce energy aerobically (larger due to increase mitochondrial density, increased capillarization, or individual muscle fibers becoming smaller in order to reduce the distance O2 has to travel).
That’s my understanding as well. Basically your heart’s ability to deliver O2 has a limited ability to adapt and mostly occurs within about 6 months of training (that’s why no one goes from being average MOP to world class after they’ve been training for a several months at a decent volume, in other words you find your level fairly quickly in endurance sport). It’s the capillary density, muscle adaptations that continue to occur and account for the relatively small, incremental improvements that continue to take place over years and years of training.