If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

How to start a Classified Ads Aggregator?

I would like to create a website that is essentially a classified ads aggregator. It would scrape and aggregate the ads from local websites posting classified ads in several categories such as buy/sell, help wanted, etc. It would then display them in similar categories. The user would be able to sort them by price, location, etc. The site would also accept the ads from users.

The website will target the local market in one of the European countries. There are several dozens of local websites posting classified ads, either stand alone, or online versions of newspapers. I want my site to pull these ads and display in organized fashion for the user.

There are sites using this model, such as oodle.com, or kijiji.com. However these are huge sites in scope. On the other hand, my project is targeting only 30-40 websites for now.

Where can I find somebody who might be able to do this (I am not coder)?
I see obstacles at the outset:

1. overcoming the occasional changes in the targeted websites.
2. the site will be non-English
3. while basic ads categories are very similar, they are NOT spelled out exactly alike from website to website?
4. I am on my own, no big $$ sponsors
5. above 4 are just the ones that are coming to my mind (not a developer mind)

I would like to create a website that is essentially a classified ads aggregator. It would scrape and aggregate the ads from local websites posting classified ads in several categories such as buy/sell, help wanted, etc. It would then display them in similar categories. The user would be able to sort them by price, location, etc. The site would also accept the ads from users.

The website will target the local market in one of the European countries. There are several dozens of local websites posting classified ads, either stand alone, or online versions of newspapers. I want my site to pull these ads and display in organized fashion for the user.

There are sites using this model, such as oodle.com, or kijiji.com. However these are huge sites in scope. On the other hand, my project is targeting only 30-40 websites for now.

Where can I find somebody who might be able to do this (I am not coder)?
I see obstacles at the outset:

1. overcoming the occasional changes in the targeted websites.
2. the site will be non-English
3. while basic ads categories are very similar, they are NOT spelled out exactly alike from website to website?
4. I am on my own, no big $$ sponsors
5. above 4 are just the ones that are coming to my mind (not a developer mind)

Best regards,

Andre

To really cllassify you need the add Classifier, meaning the tool which will automatically categorize the website by its content. This task is not easy, but has a solution. Let me know if you are interested.

I agree that, like any search engine, I will need to some sort of way to categorize the website. But even before I consider this I need to answer several issues that I think I will face, such as:

1. Is it possible at all to create such a search site that actively monitors individual websites for new ads, and displays them in selected categories? I identified 30-40 local websites that I would like my serch engine to crawl.

2. How to address the changing nature of the source website? The webmasters occasionally change the sites, and what are the implication for the serching mechanism?

3. I can imagine that the fact that I want to create a non-English website at some point sooner or later has to create issues - can only the native speaker address these?

I agree that, like any search engine, I will need to some sort of way to categorize the website. But even before I consider this I need to answer several issues that I think I will face, such as:

1. Is it possible at all to create such a search site that actively monitors individual websites for new ads, and displays them in selected categories? I identified 30-40 local websites that I would like my serch engine to crawl.

2. How to address the changing nature of the source website? The webmasters occasionally change the sites, and what are the implication for the serching mechanism?

3. I can imagine that the fact that I want to create a non-English website at some point sooner or later has to create issues - can only the native speaker address these?

These are just two consideration I am looking for the answers to.

Best regards,
Andre

1. Of course, but why do you need it?

2. All sites are assumed to be changed. The site analyzer picks up the current state and if it is a background process then it stores and use it until the next crawling cycle. If its a real time operation then you get always updated results.

3. I am not sure about, say oriental langauges, but if it's at least a European one there should not be a problem. For example although web content classifier classengine.com is trained now for English, it is not a problem to do the same for any European langauge as long as this langauge is supported by media, e.g. German Google search.

1. I came up with the idea of classifieds ads being pulled in one place because there is no such site addressing Polish language ads sites. There are mainly local 'stand alone' websites, or classifieds sections of online local newspapers.

2. I am looking for somebody who might know how to do such site. I am looking for help on rentacoder.com so far. I know of getacoder.com, guru.com, and couple of other sites. I am looking for some feedback on these and similar sites, as well as, if this is reasonable way to look for somebody to create a website.

2. I am looking for somebody who might know how to do such site. I am looking for help on rentacoder.com so far. I know of getacoder.com, guru.com, and couple of other sites. I am looking for some feedback on these and similar sites, as well as, if this is reasonable way to look for somebody to create a website.

I am afraid you are mistaken saying "to create a website". It's not just creating a website The problem is in implementation of what you want. Algorithms for what you want are not trivial and programming on the server side is required. So I do not think these "guru" - like services will help. They are ususlly good at web design but very far from algorithms.

Appreciate your insight. Which calls for the question how, or where do I start learning the programing. I basically want to at least start scratching the surface of programing. I agree with you on guru-like websites. I think long term I will be far better off if I at least try to explore this on my own. The prospect of working and learning on my own is more appealing to me.

And, by the way, it is my own little project.

Having said this, would you be able to suggest for a novice where to start learning programming? I know there is tons of info out there, but perhaps you could recommend a way to start. There has to be a beginning to this somewhere..

Having said this, would you be able to suggest for a novice where to start learning programming? I know there is tons of info out there, but perhaps you could recommend a way to start. There has to be a beginning to this somewhere..

You need to choose first the programming language.
Taking into account that in future you are going to program servers for web applications and prefer to use freeware and yet have an abilty to create programs on any level of complexity I'd suggest that you start with Perl.

For the begining try to create a program which does something simple. For example "for any arbitrary set of words in the input stream print them in the alphabetical order".

Classifieds agregator

hi, i have a classifieds site called www.telists.com, it's users who input ads manually, I was thinking of someway to get more ads from some well known classifieds site and inject them into my database, i could't find any tool or script that does that, what i think is that it involves complex programming to get this done, any idea??www.telists.com

I have a ready-made solution

Hey Guys,

I can see that post is really old. However, if it concerns anyone, I have a solution available which aggregates all classified ads data in an organized manner. Currently I have added around 50 websites across various languages and region. The system is very much customization, that means, if you need to add new website, it's quite easy.

Hi daydreamer
I would like to talk with you about aggregation of the web. We are currently aggregating and archiving every car posting in the US (80 classifieds and 28k dealer websites soon to be 68k dealer websites) with our technology and three people. Here is a video on how we do it. http://youtu.be/Ps-qDB67PKA
Here is my linkedin profilehttp://www.linkedin.com/pub/troy-campbell/31/a88/927
Thanks
Troy

Full-Automated Classified Website Aggregator scraping 12,000 URLs

Hello People,

Take a look at findmestuffs.com It is a classified website aggregator, aggregating listings from around 12,000 URLs ( All Indian classified websites ). The listings are organized in 18 categories as listed on the website. The website automatically updates for new listings on daily basis. The entire system is fully automated and does not require any maintenance and update. The current system is capable of scraping around 100,000 URLs on daily basis. The cost of running the entire system ( website + scraping system ) is 30$ per month.

I am looking to sell the website. If you are interested, please get in touch with me at a[at]amitkumargupta.com I will be happy to provide post-sale support as well.

There are plenty of Classified ads themes for wordpress out there. That would be the easiest more cost effective way to develop a classifieds ads website in my opinion. Then after you have explored the theme options I would move onto custom PHP coding.