Importing large CVS lists generally over 40k emails will not load.
Also databse design for extra fields is not effecient and should be redone.
Causing hang times on loading of software.

November 26, 2011, 5:20 am

Maria Augustine

Posts: 1038

Re:Large List Importing Problem

Hello Mark ,

There is some limitation according with the file size importing. And sorry to say that we cant do database design for extra fields. It is not easy for redone.
Please let me know your thoughts.

Thanks and Regards

Maria Augustine
Customer Support Executive
inoutscripts.com

December 6, 2011, 11:20 pm

mark Johnson

Posts: 27

Re:Large List Importing Problem

I will write a better database design to implement.
The current one is really really bad.
Finding all the mysql statments inside the php code is horrable since it isnt linked to one include file which handles all the calls...

Further more, if i do end up rewriting everything im just going to write it for MSSQL express, which will run even better for what i need it for.

If you have documentation on files that have sql statements I will provide all my changes back to Inandout to release as an update if they wish free of charge.

Of course the version I release to Inandout will be for MySQL not for the MSSQL.
But the database will be much cleaner and be able to process more faster and store much much more email addresses with the information which is required by law to have when sending mass mail.. such as but not limited to ip address of signup/subscriber, date/time stamps.. other useful info is first name last name phone address city state zip etc. Oh and cant forget the unsubscribe/active.. also the list the email belongs too would be good to keep next to the email address I see why you have it in another table relating over to an ID number of the email address, but a string to store which list it belong to in the same table as the email with other fields would be better cause the less rows the better... rather have more fields.

December 7, 2011, 2:47 am

mark Johnson

Posts: 27

Re:Large List Importing Problem

Fixing these 2 issues will solve alot of database issues that is in your database.
right now i am reading:
Handler_read_rnd 245
Handler_read_rnd_next 2.6 G
to upload a cvs file to insert emails into the database.
Opened_tables 258 The number of tables that have been opened. If opened tables is big, your table cache value is probably too small.

I think someone didn't write a close statement when they are making a connection to the database.. I wonder which query it is that is causing this?

Handler_read_rnd_next

The number of requests to read the next row in the data file. This value is high if you are doing a lot of table scans. Generally this suggests that your tables are not properly indexed or that your queries are not written to take advantage of the indexes you have.

Handler_read_rnd

The number of requests to read a row based on a fixed position. This value is high if you are doing a lot of queries that require sorting of the result. You probably have a lot of queries that require MySQL to scan entire tables or you have joins that do not use keys properly.

December 7, 2011, 2:53 am

mark Johnson

Posts: 27

Re:Large List Importing Problem

also email is a key in email_advt..
but it don't tie to another table....
I understand you want to keep that field unique, but you should just run a query for that before you insert the email instead of holding that load on the server to maintain a key for it.
im going to remove that relationship and see if it speeds things up..

December 7, 2011, 2:56 am

mark Johnson

Posts: 27

Re:Large List Importing Problem

dropped the key index of email in table email_advt.
and now this dropped
Handler_read_rnd_next 646.1 M
almost half of what it was!

December 7, 2011, 2:56 am

mark Johnson

Posts: 27

Re:Large List Importing Problem

more then half from 2.6gig to 646.1M WOW!

December 8, 2011, 10:21 am

Nisha Vince

Posts: 1700

Re:Large List Importing Problem

Hello Mark,

I think you are trying to upload the file 9 mb, If you upload this type of large files you have to need to a good processor.Otherwise you may split the files as smaller ones and upload.

So "Large List Importing " depends upon the following:

1)Its due to the lack of server capacity
2)Its due to the lack of the processor capacity