Are these separate SKUs? Are they available on MSDN download for universal subscribers?

Also what about "FAST Search Designer for Microsoft Visual Studio"?

I am trying to evaluate if I can use FAST to do web crawling with data mining and text analytics; so far I have been unable to find any simple and clear explanations on the web on how to go about doing this.

FAST capabilities. The reasons you specify here is the main reasons to go to FAST compared to SharePoint standard search. Yes you can do a whole lotta fun and customize the FAST pipeline until you bleed. Big difference between FS4SP and ESP is that you are limited in what parts of the FAST pipeline you can modify in FS4SP. Here is one example on MSDN: http://msdn.microsoft.com/en-us/library/ff795797.aspx