Discover WIN32. How to use a sync/async retrieve HTTP content in your ASP & VB projects.

This article shows how to create one ATL COM component using the WinInet functions, how to use it in ASP programs and how to test it from the Visual Basic client. It also shows how to use multithreading support in this component.

Introduction

This article shows how to create one ATL COM component using the WinInet functions, how to use it in ASP programs and how to test it from the Visual Basic client. It also shows how to use multithreading support in this component.

This component can be used in the same way in Visual Basic, Delphi, Access or Microsoft SQL.

Overview

This application uses ATL, MFC, ASP, Visual Basic and WinInet functions. The application have an ATL project who provide some HTTP functions. It will demonstrate how to use this component in your ASP and Visual Basic projects.

The article is based on the Uwe Keim's article “Calling scripts within scripts” – many thanks to him for his good idea. Of course you will ask what is new in my component. The reason that made me build this component was the variable time delay at a HTTP request.

My solution was an asynchronous retrieve, which give the user the possibilities to check more often whether the page is completed or not.

The article also contains a Visual Basic client who tests the functionality of the component.

I assume the reader knows or at least has a fair idea about COM programming using ATL.

Details

The component consists of two parts:

One standard WinInet retrieve module.

Retrieve method.Retrieve(BSTR newVal, BSTR *parOut)

GetPage function.bool GetPage(CString sURL, CString& sBody)

The Retrieve method just calls the GetPage function with the sURL parameter. Inside the function, there are the calls to WinInet API:

//Open one internet connection. The “INTERNET_OPEN_TYPE_PRECONFIG”
//parameter tell the function to read the default proxy settings from
// the registry(the proxy settings from connections in IE):
HINTERNET internet=InternetOpen("Daemon", INTERNET_OPEN_TYPE_PRECONFIG,
NULL, NULL, NULL);
//Make one connection with the desired URL. On the third parameter we
//use the“INTERNET_FLAG_RELOAD”
//while constrains the function not to usea local cached copy.
//Take a look at the documentation of this functions to see all the
// combination of parameters:
HINTERNET file_handle=InternetOpenUrl(internet, sURL, NULL, 0,
INTERNET_FLAG_RELOAD, 0);
//Read the data from the address:
InternetReadFile(file_handle, pagina, 100000, &bytes_read);
//Close the connection
InternetCloseHandle(internet);

The asp source code is very simple (full source code are in attachment file):

The client must use first the RetrieveAsync method with the desired URL parameter. This method launch one child thread that will retrieve the page in a separate thread. In this way the program will go to the next instruction and will not wait for the method to finish the request.

The program must implement one timer (or at the user's actions) who verify at some intervals if the RetrieveAsyncCompleted method return true (the requested page was retrieved). Only then the RetrieveAsyncPage method will provide the requested page.

While the RetrieveAsyncCompleted method return false you could send to the console a message such as “Please wait. Data is loading…”

//Make a new thread. The “MyThreadGetPage” is the name of the
//function who will resolve the HTTP request. Inside the
// MyThreadGetPage function, there is one call to the previous
//discussed GetPage function.:
AfxBeginThread( MyThreadGetPage , this, THREAD_PRIORITY_NORMAL )
// The program use a coleection to keep all request to the component.
// Add a new URL request:
pObject->m_MapStringToHTTPpage[sKey] = oHTTPpage;
// Look at collection to see if it is a new URL or not:
m_MapStringToHTTPpage.Lookup(sURL, oHTTPpage)

The Visual Basic source code is very simple (full source code are in attachment file, in RetrieveVBClient director):

'Launch the thread with the specified address
str = myORetrieve.RetrieveAsync(URL.Text)
'Look if the retrieved page is completed:
lRetrieveAsyncCompleted = myORetrieve.RetrieveAsyncCompleted(URL.Text)
'If it is true, read the data from the specified URL address
str = myORetrieve.RetrieveAsyncPage(URL.Text)

Installation

Copy the DLL into a directory with system execute privilege and register it with regsvr32 command or put it on the MTS. To put it on the MTS is better because if we want to modify the component and register it again, this is possible without restarting the computer – in case when using the regsvr32 command who “blocks” your dll file.

Use directly the visual basic dialog console. Input some string on the URL edit box and click on the button!

Copy the director with asp pages on the Web server – and just try it !

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

Share

About the Author

I make programming for over 4 years and extensive experience in C++, ASP, Pascal, MFC, COM+, ATL, TCP/IP, HTTP protocols, XML, XSL, SOAP and SQL. For the last 2 years i working extensively at the background of financial sites (databases, n tier architecture).

Comments and Discussions

i am currently developing multi-form Windows application in VB.Net using Visulal Studio 2005 Pro, but at the moment i am encountering the problem how to create the code that will:1. retrieve HTTP specific content (stock quote from yahoo.finance) into the VB.Net UI 2. store that the content into virtual dataset that later on can be used for analysis (eg moving average)

I have looked at Adrian Bacaianu code snipets and articles which give some leads though it was written for excel, whilst in this project data should show in the Windows Application.

i know it is a big call, that is why it would be highly appreciate if someone can share his experience in finding code solution for the above issues., cheers

I'm trying to use this example with VB6. But this example can't retrieve the URL such as: "http://www.google.com.vn/search?num=100&hl=vi&ie=ISO-8859-1&newwindow=1&q=1asphos&meta="Can't you tell me the reason Why?I'm waitting for your answer.Please, write to me soon.Thank you very much.

I have created a wrapper in VB for RetrievePage. If i install it first time on any machine, it works fine, but suddenly it stop working. I don't know why. Is there any dependency problem? Actually we install the COM+ MSI, and for new release we uninstall the old MSI and re-install the new one. Might be some dlls not unregisters successfully. This mostly happens on development machine. Sombody told me this is MSXML.dll issue??

Running this project on VB6 SP5, and XP OS the program seems to return an incorrect value or pointer to the body of the page being retrieved in asynchronous mode, and crashes the app.In synchronous mode it works fine. Any help would be appreciated.

I have tested DLL with VB 6 -works fine. But returned string contained HTML part of .ASP page with no results of server-side script execution. Is any possible way to get string after data has been processed on remote web server?

You ask about the asp processing?That is done in the asp com+ memory, in the ISS space server context. It is not possible to send that information on the web client. Only with with other component who must handle that asp processing. But is very complicate.

I downloaded the code that you posted here. I successfully compiled the resulting dll. I then placed your RetrievePage.asp page in my IIS5.0 virtual directory, in order to test it.The page is retrieved, except for images but the problem is that the RetrievePage.dll server never unloads, even after killing the IE5.5 browser that hosts the RetrievePage server.I'm on Windows 2000-SP2, running VC6.0-SP5 on IIS5.0-SP1. I tested the ASP page in IE5.5. After doing some low level checking, I discovered that the DLLHOST.EXE surrogate is the culprit. When I run the asp script you provided, it seems DLLHOST.EXE is actually the engine that hosts your COM server for owner IWAM_QUANT2000P. Strangely enough, DLLHOST.EXE hangs around forever and never unloads the RetrievePage.dllHope this helps,

The never unload of the one com server from IIS 4 and IIS 5 is a fact that cause me headachs. A solution is not to use the regsvr32 program to register it.Use MTS (on NT4) or Component Services (on NT2000) to register it. You also have the possibility to change a property regarding the number of minutes, and after that the component will be shuttdown.

Yes, I know that . The HTTP object is in XML DOM component from Microsoft (msxml2.dll or msxml3.dll or msxml4.dll… ). It is a very strong and powerful component and very easy to use. It also provides synchronous and asynchronous methods. I hope I will publish one article with that.The reasons for which I prefer to made my component were:

Necessity to use very small code at the client side.

The Microsoft component have 4 M and is all the time in change.

Just 4 call to Win Inet make the program faster, and I hope that Microsoft will not change the header of that functions!!!

But that's only one half of the truth!The main problem is the use of WinInet on the (web)server. That used to be used for the HTTP functions of msxml until they changed that with the serverXMLHTTP component.

I have just been looking at commercial packages (I think ASPHttp was one of them) that does the same thing. One thing I noted was that they were not using WinInet because of recognised problems, but were instead using WinSock. I've got no experience in either, so I was just wondering if you could give us a rundown on the pros/cons of these.

"Microsoft Windows HTTP Services" - WinHTTP - is the new server-safe HTTP API from MSFT. MSXML's ServerXMLHTTP is built on WinHTTP. Current ver is 5.0 tech preview - unsupported, but the doco is slowly improving on the MSDN site. In addition, I believe BizTalk server relies on this API.

I haven't had a chance to play with it, but it looks as though it will do nicely and in many cases make nasty WinSock coding redundant. SDK available from the MSDN site.