We appreciate the enormous support that our ABestWeb community has experienced over the many years it has served its members and sponsors. We have decided to exit this business and have placed the property up for sale and we are actively entertaining interest.

In the meantime, community members will be able to read but not post to ABestWeb beginning on Jan. 18.

We want to thank you for your numerous contributions and your ongoing support. If you have any questions, please let us know.

Several times in the past couple of years, I have tried to create or adapt datafeed scripts to work for me. Last year, I spent thousands of dollars on custom programming, and in the end I simply abandoned the project when I realized how complex it was going to be (and how slowly the programmers were doing the work).

Last fall, I was intrigued by the very useful service offered by PopShops, but they've been plagued with outages and other problems, and their recent "new features" require lots of manual work to update older pages, and their interface just won't work for the plans I would like to implement.

So here I am: I'd like to talk to anyone who has a rock-solid system to import, update, and manage datafeeds -- with the goal of finding some way that I might license, buy, rent, access, or use your system.

I do NOT want a basic datafeed-import script. I need "more" because I need to be able to have datafeed changes propegate "rationally" through the system. This means that if a product's description changes, or a product is dropped from a datafeed, the system can't just modify or delete the associated product record, but instead it must maintain the old (and new) data, at least until the changes have been reviewed or approved.

My goal is to have access to a database that I can draw from to BOTH auto-generate AND "manually" generate web sites, PPC-search ads, and landing pages. If I've invested time writing custom ads or adapting product descriptions, or writing reviews or advertorials, I don't want my work to be deleted if a merchant drops the product (because the merchant may restore it in the next datafeed, or another merchant may add it).

As anyone who has worked with datafeeds knows, there is also a great deal of "post-import-processing" required (including those pesky HTML codes, ampersand-symbol codes, and symbols improperly mapped (most commonly quote and TM symbols converted to text that appends to brand names, e.g. Calvin Kleinr and Niketm). It would be great to work cooperatively to maintain that "import processing" data collectively.

Of course, there's more to it than just an import script and database. I'd really love to talk to someone who's also done some API work, perhaps even using APIs to exchange ads and performance data with Google AdWords, or interacting with Amazon's web services.

I'm absolutely open to a wide range of possible relationships for this.

If you have something to offer, or if you'd like to talk, please email or call me. (Please don't PM me.)

Hey Mark. It's a shame we didn't get to talk at Summit for a bit. While I can't address all of your wish-list, I may have something to show you in a couple of weeks to address a portion of it. Feel free to PM, email or call me to discuss further.

I have looked over your specs, and I'm just trying to offer some friendly advice.
I truly think you should broaden your specs to at least consider a system utilizing PHP/MYSQL instead of just limiting yourself to ASP. Is there any reason why you only want a system built in ASP?

There is nothing wrong with ASP, as I have used it frequently over the last 8 years or so.... But the percentage of programmers that write ASP code in relation to PHP is very low. It may be hard to find someone with the skill level needed to build a system based on the specs that you ask for, especially one that can get enough of a grasp of affiliate marketing to build a custom system around it.

I currently run a PHP affiliate system that I modified based on a preexisting affiliate script. I have around 100 niche based quality domains utilizing this script and all are fully developed and live on the web. The system I use is nowhere near as complicated as your specs request though, and it takes some manual work in each sites raw files before being able to turn the site on live. It works great for what I do though, and pulls in nice commissions each month. Though, I mainly look at affiliate marketing as secondary revenue to the real gold mine, selling domain names and developed websites.

I would suggest finding an affiliate script that you like the workings of already, and then take that to a programmer. Its possible the programmer could modify the script to add the features you need, or use it as a basis to build your custom script.

> I have looked over your specs, and I'm just trying to offer some friendly advice. // I truly think you should broaden your specs to at least consider a system utilizing PHP/MYSQL instead of just limiting yourself to ASP. Is there any reason why you only want a system built in ASP? // * * * // I would suggest finding an affiliate script that you like the workings of already, and then take that to a programmer. Its possible the programmer could modify the script to add the features you need, or use it as a basis to build your custom script.

I am not excluding a non-Windows solution, but my primary goal is to have a system that I can understand, and which I can modify. Therefore, I've stated that I will pay hourly for assistance and work on a Windows solution, but I will not pay hourly for a LAMP (Linux/Apache/MySQL/PHP) solution, because I would probably not be able to understand or modify the LAMP solution without many (hundreds) of additional hours of learning curve.

I hired two different programmers to create scripts to work with MS SQL Server; one did a fairly competent job, but ultimately I was unable to make changes to make it work consistently. My programming and database skills are just too rusty to do work without a lot of help. I'm willing to pay for the help, but only for something that I understand (or which will be fully supported at a KNOWN cost).

If I'm going to spend thousands of dollars, I need to be confident that at the end of the work, I will have a solution that I can use for my needs. My needs are not unusual or unique -- but I am now trying to lay out the entire sequence of activities as a broad project so that I am not working with someone who can do only one tiny piece.

Starting with an existing script would be great. I haven't yet found a script that appears "more likely than not" to be adaptable to meet my needs. I have found a PHP script that I will probably pay a few hundred dollars to have installed on a VPS for me, but I don't have a high confidence level that it will prove to be something that I can adapt for my needs -- and I am simply not willing to invest 100+ hours to bring my LAMP skills to a point where I could objectively evaluate whether the product could be adapted to meet my needs.

I am not excluding a non-Windows solution, but my primary goal is to have a system that I can understand, and which I can modify. Therefore, I've stated that I will pay hourly for assistance and work on a Windows solution, but I will not pay hourly for a LAMP (Linux/Apache/MySQL/PHP) solution, because I would probably not be able to understand or modify the LAMP solution without many (hundreds) of additional hours of learning curve.

Starting with an existing script would be great. I haven't yet found a script that appears "more likely than not" to be adaptable to meet my needs. I have found a PHP script that I will probably pay a few hundred dollars to have installed on a VPS for me, but I don't have a high confidence level that it will prove to be something that I can adapt for my needs -- and I am simply not willing to invest 100+ hours to bring my LAMP skills to a point where I could objectively evaluate whether the product could be adapted to meet my needs.

You do realize that you can run PHP/MYSQL on a Windows server right?
That may be an idea to ponder as well.

If you are thinking of putting hundreds of domain names on one box running ASP dynamic pages showing hundreds of thousands of products, I would also be concerned on how the server will handle it. ASP creates threads which will lock up IIS on a windows webserver and can shut down all the sites on your server if you are not using separate application pools within Win2K3. I control 2 windows dedicated servers (and 4 linux dedicated servers) for my customers (and myself) and I always have to set IIS on each windows box to refresh each domain's application pool every couple of hours (especially on the sites which utilize ASP).

> You do realize that you can run PHP/MYSQL on a Windows server right? That may be an idea to ponder as well. <

Yes, I understand that in theory, I could run PHP on a Windows server. In practice, the script-writers I've contacted are uniform in refusing to try to install their products on anything other than a Linux box. Even if I found someone willing to do this, I'd still need to spend a LOT of time building my PHP skills (and "un-learning" my earlier scripting and programming skills).

I'm not completely LAMP-illiterate -- my current web server is a Linux/Apache VPS which includes some PHP pages.

Originally Posted by eastwave

> If you are thinking of putting hundreds of domain names on one box running ASP dynamic pages showing hundreds of thousands of products, I would also be concerned on how the server will handle it. * * * <

You've certainly shared some good advice, but it doesn't apply to me. I don't intend to use dynamic pages at all. While I used that strategy at one time, I experienced some performance issues that led me to change back to static content pages for my sites. (In fact, I think I am currently using PHP only for my PopShops pages.)

My usual practice now is to auto-generate static HTML pages for most of my sites. When the data changes, I re-generate the static pages (currently, I re-generate entire sites whenever there is a single change).

My usual practice now is to auto-generate static HTML pages for most of my sites. When the data changes, I re-generate the static pages (currently, I re-generate entire sites whenever there is a single change). (In the PDF document I posted about this project, I identified "Phase Six: Generating Web Site Content Pages" but didn't elaborate.)

That makes sense then, asp should work fine for you if you can find someone to code it like you want. If you are going to rely on asp to do the autogeneration of the pages I would suggest having the programmer do it where it only batches a certain amount of static pages at once, which I'm assuming is what you will do.

I used to use asp on a mall type site with over 500,000 products in a MYSQL database, I had probably 25 domains running off the same database. I experienced some major slowdown no matter what I tried to do to optimize the database. I then switched it all over to PHP on Linux and now have around 100 sites, I also split the database up into 7 different databases based on category. The speed now is really good and there is probably close to 1,000,000 products across all the databases.

If you do go down the dynamic asp page route again on windows I suggest installing ISAPI Rewrite which makes IIS able to MOD REWRITE pages like .htaccess files work with Apache/Linux. I used this when I was using asp and it worked well.

> When I re-generate a new version of that document, I'll be clearer. <

FYI, I've updated the project requirements/specification document (but it's still very preliminary). I decided to simply overwrite the prior document with the new version: http://www.markwelch.com/datafeed-pr...2008-06-16.pdf (the first line says "Updated June 19").

OK, my adventure with Datafeed Studio has been an interesting diversion, but it's really, really clear once again that it's a really, really bad idea for me to try to launch a development project using technologies that are completely unfamiliar to me, and which are unsupported. (Martin provided really great support for DS, but I needed support for LAMP [Linux, Apache, MySQL, PHP].)

Datafeed Studio, like PopShops, is a great example of the kind of foundation on which I'd like to build my solution.

I am once again looking for developers and tools who might help me move closer to my goals, as outlined in the PDF document linked above.