The way to do GPU acceleration in a cross-platform way is OpenCL so a handful of Mac apps use this. Final Cut Pro X being one of them. I think it would be a good idea for the core frameworks in Lion to use OpenCL so that more apps can benefit from it with little effort.

Cuda is NVidia-only and AMD have the Stream SDK. OpenCL works on both so is the best choice. The area where it will become most visible is in visual effects packages like compositing and 3D engines. One rendering engine to use it is Small Lux GPU: