I don't miss it even a little bit. The power can only be described as pathetic, but credit where credit is due, I got better than 7L/100km.
You will not notice the extra weight in the corner unless your The Stig. Any cornering losses from putting a six in there are more than made up by the power advantage that the six has against any bmw 4 cylinder (m10, m42, m44)
I'm a noob on turbos, but many e30s are out there running 600hp on m20s, so boost shouldn't be an issue.
FWIW, when my M10 started suffering from electrical issues (im pretty sure it was just the alternator though lol), I used it as the perfect excuse to slap an m52 in there (like you, a turbo is in its future). You may want to consider that route as well (any M/S5X motor) as you sound like a person who could tackle a swap if you feel confident enough to pull and rebuild a motor.