I’ve had 50mm on my Aspero, I did find that on windy coastal rides (that’s the Tasman Sea in the pic) the rim depth plus the height of the tyre added a bit of steering input. Which wasn’t always appreciated when combined with the steering inputs from heavy gravel. It’s equivalent to riding a 65mm rim with road tyres for cross section.
No problem when there was no wind. In general I prefer my other wheels, which I designed with a 22mm height because of a race on in the wind on the road above.
Like most things, there is a time and a place when deeper wheels are a good solution.