Basically, create a "Disallow" line for each directory you'd like robots and spiders to ignore. Note: the "robots.txt" is an optional "please don't do that" type of thing. Search engines are not required to honor it.

To enforce mandatory exclusion of directories and files, you can add/edit the ".htaccess" file in the server's root directory, but that gets into the "running with scissors" area. You can do some serious damage, if you aren't careful... 8)

It is not necessary to have index files in your buckets, because Amazon security is based on the object's individual security setting. If an object is marked as private, it doesn't matter if someone knows it exists, in your bucket.

As a follow up then.... having additional folders within your bucket in which your objects are placed into would seem only necessary for organization rather than a traditional approach of 'burying' a file in multiple layers of folders named in an arbitrary way ?

A Bucket Full Of Objects -- Amazon S3http://www.tipsandtricks-hq.com/a-bucket-full-of-objects-amazon-s3-3052
Objects are referenced by their “keys,” which consist of an optional “pseudo folder” (directory) path name, followed by the name of the object. The keys “His-Stuff/test.txt” and “Her-Stuff/test.txt” refer to 2 separate instances of the “test.txt” object. Because the “folder” name part of the keys are unique, so are the object instances. The term “pseudo folder” is used because S3 does not really store objects in folders; the same way that Windows, OS-X or Linux does. The entire object key is considered (by S3) to be the equivalent of a file name.

IMPORTANT to note that Amazon S3 has some limitations with SSL and CNAME / redirections.

You cannot use secured downloads/SSL (https instead http) as well as CNAME redirections:
your_subdomain.your_domain.com/folder/product.zip
INSTEAD OF
your_subdomain.your_domain.com.s3.amzonaws.com/folder/product.zip

I uploaded all my MP3 and PDF products in the WP Media Library so I can easily add them to the products in eStore. The File URL for each product was create automatically by WP when I uploaded them. Will people be able to find these files somehow if they know how to search for them (I don't)? If so, how can I further protect the files in the Media Library. Sorry if the answer is above but I don't understand the terminology and missed it if it's there.

From above: "Create an empty "index.html" file in the directory that contains your downloads. For extra sarcasm, you might put a message in it like:
Find what you're looking for?" Can I do this with the WP Media Library?

It is suggested that you store product files in a directory that is outside the normal media library directory with a directory file permission of 0700, that the uploaded product file permissions are 0400, and that you create an empty index.html file within the directory, with a file permission of 0644. If you do this, and use encrypted download links; you should be reasonably "fine."

For maximum protection, you should use Amazon S3 to host your product files, instead.

"Sarcasm" towards hackers is not only unprofessional, but can be a self-fulfilling invitation to have people attack your site.