First, a thought about the granny charger. The M3 will come with one that has a button on it that opens the charge port and unlocks the cable. It avoids the need to use the app or touch the central display. May be worth at least trying it. I expect it will also work with the e-Golf.
As for handling, I think the e-Golf is a very good car. I did 18k miles in 12 months including several long trips up to 960 miles in two days. Used it fully loaded at times including a roof box. However, the M3 is a clear step up. It feels more planted and solid. Much more traction in the wet. And AP is excellent. As you would expect, it is faster and more comfortable. And the Supercharger network is another planet. Both are good cars but in a different market segment.
I would agree with @PaulD from my own experience. In my case driving the Model 3 came as much less of a revelation than for many other people simply because my experience of driving the eGolf had been so good. Also just getting into an eGolf felt so good and instantly comfortable. But I agree about the traction especially on the stock tyres, which I replaced with Cross-Climates.
I also use the same charger as for our eGolf, which actually charged at only 3.3kw so it’s more than twice the speed with the Tesla. However if you’re relying on a granny charger, be prepared for a long wait..!