HttpWebRequest/Response in a Nutshell - Part 1

Introduction

HttpWebRequest/Response - you read much of it, yet many tutorials, explanations, etc. are quite difficult. I'll try to explain the basics of HttpWebRequest/Response here, setting you off towards more advanced code with HTTP-usage.

This article is mostly intended to be read by beginning C# coders. Intermediate users might be able to use this as a small reference to some functions though. I'll try to do things structured.

The Code

Now, up to the code. In HttpWebRequest/Response, we have several functions, properties. These are the ones that will be explained here:

Used to set the method ("GET" or "POST") of retrieving the data (Method header for HTTP/1.1)

String HttpWebRequest.ContentType

Used for defining the content type (ContentType header for HTTP/1.1)

HttpWebRequest.GetRequestStream()

Used to get the stream for putting POST information.

HttpWebRequest.GetResponse()

Used to initialize the HttpWebResponse, and to get the data (HTML content) off the web

HttpStatusCode HttpWebResponse.StatusCode

The statuscode returned by HttpWebResponse

String HttpWebResponse.Server

The server (typically IIS or APACHE) returned by HttpWebResponse

Now, up to some code.First of all, we need or write namespaces. Here we will use:

using System;
using System.Collections.Generic;
using System.Net;
using System.IO;
using System.Text;

It's in the System.Net the HttpWeb- functions are found in.We'll need the System.IO for the stream reading/writing, and the System.Text for the ASCII encoding.

Now, in your class, create a function (here it is start-post()). This function is (for now) void, and has no parameters, for we want to keep things simple in the beginning. If you find this too easy, you can always jump to parameters and return values immediately.The code will be fairly simple, we'll try to extract some content from a server (in this case, a dummy script, the code for which I'll give you too).

privatestaticvoid start_post()
{
//Our postvars
byte[] buffer = Encoding.ASCII.GetBytes( "test=postvar&test2=another" );
//Initialization, we use localhost, change if applicable
HttpWebRequest WebReq =
(HttpWebRequest)WebRequest.Create("http://127.0.0.1/test.php");
//Our method is post, otherwise the buffer (postvars) would be useless
WebReq.Method = "POST";
//We use form contentType, for the postvars.
WebReq.ContentType ="application/x-www-form-urlencoded";
//The length of the buffer (postvars) is used as contentlength.
WebReq.ContentLength = buffer.Length;
//We open a stream for writing the postvars
Stream PostData = WebReq.GetRequestStream();
//Now we write, and afterwards, we close. Closing is always important!
PostData.Write(buffer, 0, buffer.Length);
PostData.Close();
//Get the response handle, we have no true response yet!
HttpWebResponse WebResp = (HttpWebResponse)WebReq.GetResponse();
//Let's show some information about the response
Console.WriteLine(WebResp.StatusCode);
Console.WriteLine(WebResp.Server);
//Now, we read the response (the string), and output it.
Stream Answer = WebResp.GetResponseStream();
StreamReader _Answer = new StreamReader(Answer);
Console.WriteLine(_Answer.ReadToEnd());
//Congratulations, you just requested your first POST page, you
//can now start logging into most login forms, with your application
//Or other examples.
}

Now, this was simple, wasn't it?Next, we'll do something even more simple... We'll GET a file on the web. We'll use quite the same functions, quite the same flow in the program, but with the difference that we don't need to put any requests (postvars) into any form of stream. We'll have to use streams to get our output though.

Now we've got another function, start_get(), again void.

privatestaticvoid start_get()
{
//Our getVars, to test the get of our php.
//We can get a page without any of these vars too though.
string getVars = "?var1=test1&var2=test2";
//Initialization, we use localhost, change if applicable
HttpWebRequest WebReq = (HttpWebRequest)WebRequest.Create
(string.Format("http://127.0.0.1/test.php{0}", getVars));
//This time, our method is GET.
WebReq.Method = "GET";
//From here on, it's all the same as above.
HttpWebResponse WebResp = (HttpWebResponse)WebReq.GetResponse();
//Let's show some information about the response
Console.WriteLine(WebResp.StatusCode);
Console.WriteLine(WebResp.Server);
//Now, we read the response (the string), and output it.
Stream Answer = WebResp.GetResponseStream();
StreamReader _Answer = new StreamReader(Answer);
Console.WriteLine(_Answer.ReadToEnd());
//Congratulations, with these two functions in basic form, you just learned
//the two basic forms of web surfing
//This proves how easy it can be.
}

Again, the function is very easy...

Now, since we are all demanding coders, we won't quit with these simple functions, for we will have to change the functions, recompile the code, in order to change the file we need to get. We can use command-line input though. Why don't we upgrade our code? :)

License

Share

About the Author

Started coding at age of 12 (approx.) started out with pascal, went to Delphi, and from Delphi to BC++ (all in one year), did several years with BC++, and later on started to learn some html, php, mysql... Now more into C#, python, php.Current age: 17

I have one desktop web application that requires to send a data to the web application and process that data and then returns to the desktop app(Provided there is internet connection available all the ntime).

How can we achieve this? Can we go for asmx web service? or can we handle in any other way?

i m getting data from other site. the site is in aspx and paging are used in that page. i m able to get data from the first page but did not able to move to next pageing.i have sent all the needed data. the link of the page is as

Hello all, I'm trying to do this for http://www.spiritair.com/welcome.aspx?pg=salesinformation&number=570[^] and it looks like Spirit has built in a browser check, because instead of seeing the source for that page, i see the source for a simple page that says download the newest version of your browser and enable active scripting. What can I do to get around this?

Thnx for the article! It was realy helpful! I've only got one problem with the cookies. I'm trying to make a login bot for a site, but the site uses like this code:

setcookie( 'cookie', 'value', time()+3600 );

The problem is, PHP recognizes this as a cookie for the whole domain.com. (It doesn't mather on what page it is created) C# instead recognizes it as a cookie from signup.php and stores. When I'm trying to use the stored cookie to get the HTML from an other page it says i'm not logged in.

So, does anyone knows how to set the cookie for the root domain? I've already tryed changing the path to '/' or '' (null) but it hasn't any effect.

-- modified at 11:42 Friday 10th August, 2007SOLVED, I'm now making a new cookie with the settings of the previous server generated cookie. (Name, value, domain) But now I'm giving the path argument the '/' value so the cookie is effective on the whole domain.

I wrote a program that accesses my netflix queue, I can access it as long as the cookie from when i logged in with ie is on my computer, but i would like to login with the program if it isn't. I tried using your code but it doesn't put the cookie on my computer. If you have any ideas how to login to netflix. I would really appreciate the help. Thanks

sometimes I get errors (Xml responses that indicate the documents that were written to the stream were not well formed) when I write data greater than 50k, have you ever seen this? If I re-send the same document it works fine...

I am making an http request to a html page through a toolbar.The page some embedded perl queries in it and it uses the cookies from the local browser(which are set by a site) to fetch the records.My problem is that when I make the http request through the browser(i.e., by typing the url of the page in the internet explorer) I am getting the desired results.But, when I am making the http request through the toolbar(through WebRequest and Webresponse methods), it is unable to read the cookies set in the local browser.How can I make http request through the browser?(or how can i tell the webrequest method to read cookies from the browser.)I would be obliged to anyone who can help me.

Now if you want to use the cookie later you can persist the CookieContainer and use later. I do not think you need to consider web browser installed on the system. HttpWebRequest object can serve as a mini full featured browser.

thank youi am able to login nowyes its right there was some problem with my postdatanow i need to signupi have same problem with postdata againi used certain plugins to track the postdatai was able to find certain hidden fields having values in the postdata when i attempted to signup in the websitei dont know how to correlate those values with my applicationdo you have any idea about how to sign up to websitehave you done this before?if yes plz help methanksRama

So, you find a hidden field, it has a name (or it should have), and a value (should too, but is not necessary), you just add it to postdata in a &[name]=[value] format.Every postdata value should be in a [name]=[value] format, and between every one of those, there should be a '&', unless of course it's the first.

<hr>C#, python, php, mysql, html, C/C++.
<hr>
Programming is only about learning the basics of life.
* * CPF_