Hi
It could be done using plain java.. Please see the attached code..
You could schedule it to run during different times of the day or whatever.. it
will pull the site you are giving and put it on your Harddisk..
you run it as java URLRipper site_to_rip local_file_on_hdd
Oh! yeah.. if you are using proxy specify using
java -DproxySet=true -DproxyHost=<<your proxy host>> -DproxyPort=<<port here >>
site_to_rip local_file_on_hdd
You can probably then use some utility to grep the news required..
guess that helps
Prabhu
Eugene wrote:
> A friend of mine has a requirement like this.
> He wants to download the top news from another site like Say www.xxx.com
> He wants this to be done automatically at say any time in a day or whenever
> that content changes(any one will do)
> Somebody has told him this can be done through XML
> Any idea how this can be done
>
> regards
> Eugene
import java.net.*;
import java.io.*;
public class URLRipper
{
String reqURL = null;
String outputFile = null;
public URLRipper(String reqURL, String outputFile)
{
this.reqURL = reqURL;
this.outputFile = outputFile;
}
public void connectAndRip()
{
try
{
File destFile = new File(outputFile);
URL dataURL = new URL(reqURL);
InputStream bais = dataURL.openStream();
//ByteArrayInputStream bais = new ByteArrayInputStream(dataURL.openStream());
OutputStream baos = new FileOutputStream(destFile);
//ByteArrayOutputStream baos = new ByteArrayOutputStream(new FileOutputStream(destFile));
byte [] singleByte = new byte[1];
while((bais.read(singleByte)) != -1)
baos.write(singleByte);
baos.close();
}catch (Exception ex)
{
System.out.println("Exception occured.. " + ex);
}
}
public static void main(String [] args)
{
if(args.length < 2)
{
System.out.println("Usage : java URLRipper <url to be connected> <outputFileName>");
System.exit(1);
}
URLRipper aRipper = new URLRipper(args[0], args[1]);
aRipper.connectAndRip();
}
}