(Delphi ActiveX) Getting Started Spidering a Site

This is a very simple "getting started" example for spidering a web site. As you'll see in future examples, the Chilkat Spider library can be used to crawl the Web. For now, we'll concentrate on spidering a single site.

Chilkat for Delphi Downloads

uses
Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.StdCtrls, Chilkat_v9_5_0_TLB;
...
procedure TForm1.Button1Click(Sender: TObject);
var
spider: TChilkatSpider;
i: Integer;
success: Integer;
begin
spider := TChilkatSpider.Create(Self);
// The spider object crawls a single web site at a time. As you'll see// in later examples, you can collect outbound links and use them to// crawl the web. For now, we'll simply spider 10 pages of chilkatsoft.com
spider.Initialize('www.chilkatsoft.com');
// Add the 1st URL:
spider.AddUnspidered('http://www.chilkatsoft.com/');
// Begin crawling the site by calling CrawlNext repeatedly.for i := 0to9dobegin
success := spider.CrawlNext();
if (success = 1) thenbegin// Show the URL of the page just spidered.
Memo1.Lines.Add(spider.LastUrl);
// The HTML is available in the LastHtml propertyendelsebegin// Did we get an error or are there no more URLs to crawl?if (spider.NumUnspidered = 0) thenbegin
Memo1.Lines.Add('No more URLs to spider');
endelsebegin
Memo1.Lines.Add(spider.LastErrorText);
end;
end;
// Sleep 1 second before spidering the next URL.
spider.SleepMs(1000);
end;
end;