A lot of the stuff on your 323i is pretty small and undersized. I'd be worried about snapping stuff with more power. You'll also need a lot more than just the engine. The wiring harness has to be custom built, and you'll need the computer etc. It's not a cheap swap.
The M54B30 is a fine engine, and your drivetrain is probably ok for it, but I'm not sure if the wiring harness is compatible again ... might be same ass pain as the s54 in terms of wiring.
My advice: Sell the car, and buy a 330 or M3. It really will be cheaper in the long run.
2007 BMW 328Xi Touring, 6 speed