I just recorded an important conversation with a professor specializing in brain waves and it was in a busy dining hall with a lot of people talking in the background and is therefore difficult to listen to.

Could you recommend a software (if possible for free) that can filter out the background conversations and make the conversation more easily understandable?

Sorry, it's not free, but I use Olympus DSS Player Plus (v. 7.4.1) for this, which has a "noise cancellation" feature, and also the ability to slow down playback speed. The rewind hotkey is also handy, when you need to listen again 20x to make sense of a mumbled phrase... Newer versions might be available, as I got this years ago.

You could try cleaning it with some of the filters available for Audacity. There's not much (that's doable) you can't accomplish with Audacity. And it's free for the download.

I've never tried doing what you want to do with it, although I do use Audacity for a lot of other things. Look here and here for some ideas to get you started.

The real challenge is going to be the somewaht random character of dining room noises. That makes it difficult to filter them out without also removing things you'd rather not from your audio source. The best you can probably do is get rid of enough of the sounds that make you want to scream so that your recording is at least tolerable to listen to.

One of the mantras of the recording world is: "Don't try to fix it in the mix." Which is to say, getting your original source as good as you possibly can is 90% of achieving a good recording. So if you plan on doing a lot of recordings in environments like the above, look into getting a standalone portable recorder (sometimes called "field recorders") with noise suppression features such as musicians and news reporters use. Tascam, Zoom, Sony, and all the usual big names in audio make them. Not cheap. But what price do you want to place on keeping your sanity?

iZotope RX4 will do this, but the learning curve is a bit steep. You might also try a free trial of Adobe Audition which has similar tools when working in spectral view. I've done similar things in Audition that aren't quite as pervasive as what you are describing. Audition is my daily workstation.