I was typing a reply because most of the posts here were
horribly wrong. P90Puma and artvark beat me to good info.
If your derailleur is the GS (that means the cage that holds the jockey wheels is slightly longer) version, you can throw a 32 on there no problem once the b screw is adjusted properly.
If not, you can buy a mechanical Ultegra 6800 rear derailleur that is a GS and swap the cage over to yours. Presto, it will work perfectly with a 32. Put the cage from your di2 derailleur on the mechanical one you bought and that one will work perfectly too. Re-sell it. Should be fairly cost neutral if you buy a used one and it's a 10 minute job. It's just a few bolts. It's very simple.
Good chance you will need to lengthen your chain but you can always test it out and see.
Here's a link to the GS version of the di2 Ultegra RD. I don't know where on the RD it says GS or SS. It might not at all.