If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

I had tried using web client but the site throws up a login page...and i am using the web request object to pass POST parameters to bypass the login page.

Currently I have been successful in opening the file and getting it as an IO Stream, but now there's a new problem in that the file is a word doc, and the saved file appears corrupted ...i guess the problem is with encoding and i am currently researching on it...

Since the login requires a POST, you first need the GET, which will provide
you some kind of session handler (e.g. cookie). Then, you can do the POST.
The tricky part is know what to POST - as you know

I recommend burpproxy[1] (or another middleman) to capture and analyse the
(manually triggered) traffic between your client (don't forget to set the
Proxy) and the webserver. Using this information will allow you to finetune
your program.
For inspiration, have a look at this thread[2].

Thanks for the reply and suggestions, but I have already been able to manage the POST data and have access to the site's resources (which are mainly docs).But now my problem is with downloading a MS word doc, whose link is embedded in one of the pages.My application works like a web crawler which goes and downloads all data from the specified links.

Now the word documents which are getting downloaded are corrupted and are not opening in correct ENCODING.I have the following code for the download:

1. I know absolutely nothing about this sort of thing other than that it can be done.

2. Between you and sec_ware (good help there mate!) you seem to have gotten it pretty much solved, in that you are getting the downloads to work?

I just had a passing thought based on my previous experiences with .doc (and other file extension) compatibility?

Could it be that the file you are getting is a version that is incompatible with your reader, so the problem is at your end?

If you copy one of your downloaded documents and rename it as a .txt file then open it in Notepad, you should at the head and/or foot of the file, see some of the metadata that will tell you what created the file.

Also, if you can post with a small (1Mb or less) example I can try a variety of "Word Compatible" tools that I have, to try to open it. I know, I know, this approach is as crude as hell, but supposedly compatible tools frequently give you quite informative error messages when they cannot properly convert a Word document?..............I guess you won't hire me to fix your washing machine now?

The only other question that came to mind was "are they encrypted in some way?"

Cheers.................this is probably a complete red herring, but if it just happens to be true, you would mess with your downloading forever, and it still would not work?

Nihil has implied a good suggestion: analyse your problem - e.g. do you have the same problem with all kind of "downloads"?

You won't.

The problem is caused by the fact that a doc-file is binary, not ascii, hence
treating the "download" as a textstream will fail (StreamReader implements a TextReader). StreamReader will auto-detect the encoding (Ascii, UTF-7/8/16) ,
ie. you don't have to specify it.

Use the BinaryReader-Implementation of Stream[1]. I had a quick look at the web and have found a nice sample chapter[2] and this implementation of a web-crawler[3] (in C#, but it's all the same).