I don't know if the algorithm has a formal name. As far as I know GoldWave was the first program ever to do it. It has since been "borrowed" by many other audio programs. I think Audacity has similar source code for the algorithm.

I came in to the audio game at a time when just about all the big programs had some sort of FFT based noise-reduction scheme....so Chris is probably right about Goldwave having it first (if not being one of the first gui based audio editors in existence aside from the proprietary DAW's)....

I can't tell you either what the algorithm is called because I don't believe there is a specific one. It's mostly just removal of frequencies from the frequency domain based on a profile that determines what level in each band is noise. The process is real similar among all editors...but there are lots of tweaks each one does to the code. I've got....I think 4 different noise-reduction systems (GW, Audition, Sony SF and Acon) that all excel in some areas that others don't. I've spent a LOT of time playing around with noise reduction and reading things I don't understand about FFT processing to have a basic grasp.

Anyway, if you're looking for some open-source code...audacity as mentioned has very similar output as Goldwave...although Goldwave's output is a bit cleaner.

http://gwc.sourceforge.net/ is home for another NR program called Gnome Wave Cleaner. i've used this before and it's...meh. it seems too overly complicated to get working right quickly...and since I don't spend all my waking hours in linux I've never devoted time. being on sourceforge, you'll find source code there...but also the guy has all kinds of links for information about DSP and audio processing that are good reads..I'm still sifting through a lot of the stuff.