This is a non-sticky discussion thread for the unofficial beta beta development available from lp:~cbhaley/calibre/charles.custcol

Charles,
Instead of doing this by email or PM, this is easier for me, so I'm moving the discussion here.

Our last round is quoted below:
1) I'll check the attempt to delete and get back to you.
2) I suspected you might not see the oddity of LN, FN sorting
3) Yes, there are often unsuspected benefits of new features.
4) New bug:
a) connect to empty folder, add books
b) disconnect
c) reconnect to same folder. The author is now listed as title and title as author for each book.

Quote:

Originally Posted by chaley

Quote:

Originally Posted by Starson17

That fixed it.
1) I'm still seeing failure to delete. The file is removed from the Reader library, but not from the folder. I saw this intermittently before. Still testing, but so far haven't seen any files actually removed from folder.

I never see this. Is there any chance that something has the file open, preventing its deletion?

It would help if you checked that calibre actually attempts to delete the file. A print statement in devices.usbms.driver at line 205 would do it. Best would be to print the value that unlink returns.

Quote:

2) I suppose you know this, but the author sort order is by actual author, not author_sort. Lani Aames is after Dan Abbot.

Actually, I didn't see it because I keep names in LN, FN order.

Is this new behavior? Did it do something different before? Ahhh, you wouldn't know.

I can correct the ondevice scanner and metadata correcter to recompute author_sort for those books that don't have it, then send the metadata back to the 'device'. Once I do that, I will use author_sort.

Quote:

3) This is a nice unexpected benefit. I have a folder of some ebooks to be added, that are mostly duplicates of ebooks already added. I happened to try that folder for a test. The matching algorithm showed which ones were already in the library. That was very useful.

Interesting use of the feature. My guess is that more like this will show up once people more creative than me get their hands on it.

I have checked with Kovid, and it is broken. Oddly enough, it has been broken since forever. I will fix it soonly .

Quote:

4) New bug:
a) connect to empty folder, add books
b) disconnect
c) reconnect to same folder. The author is now listed as title and title as author for each book.

I don't see this one either. Books I add show the right info.

As an experiment, disconnect from the folder, go to that folder and delete the file .metadata.calibre, and then reconnect to the folder. The device driver will rebuild the metadata from what is found in the books. Are the results different?

This operation will use the preference settings for adding books. Any chance you have ticked 'read metadata only from file name' and that your regexp is getting the two reversed?

My settings are vanilla: no boxes checked on the add preferences tab, with the standard regexp. I added an epub that had metadata in it, and the metadata was right. I added a text file and got the text file name as title with author unknown.

[QUOTE=Starson17;913208]This happens with .txt format files. In one pdf it mangled, in another, it looks correct. Some .lits are OK, others not. ...
Still testing, this is a prelim report.
The data looks correct in the GUI during the initial import. It's wrong during the reconnect. The errors can be seen in the .metadata.calibre cache file.

This operation will use the preference settings for adding books. Any chance you have ticked 'read metadata only from file name' and that your regexp is getting the two reversed?

They were coming from the main library. No options checked on Adding books.

Quote:

My settings are vanilla: no boxes checked on the add preferences tab, with the standard regexp. I added an epub that had metadata in it, and the metadata was right. I added a text file and got the text file name as title with author unknown.

What happened with a Send to Device - Main Memory from main library of a txt file?

What happened with a Send to Device - Main Memory from main library of a txt file?

Works for me.

I wonder if the delete problem is not related to this? Clearly something is stopping calibre from certain file manipulations. If the metadata cache is not being written, then it tries to determine the metadata from the file. For text files this is clearly impossible, so it uses the filename pattern stored in preferences.

In other words, it will use the last value you put into the regexp box for adding books. If this regexp does not match the filename pattern used for sending to device (also in preferences), then nothing good will happen.

I will look at the path names for local drives. I can easily imagine that the paths are not being normalized to windows (you run vista?), which might cause trouble.

This is for the SD card mounted in the SD reader. Initial transfer of ebooks looks perfect in the GUI, on the SD card and in the .metadata.cache (AFAICT). Disconnect the folder and reconnect to it - nothing else. Then open the .metadata.cache file and I see all kinds of munge in author/title. Authors become Unknown or title. Titles messed up, etc.

If the metadata cache is not being written, then it tries to determine the metadata from the file.

AFAICT, it is correctly writing the metadata.calibre file (I assume that is the cache) on the SD card (and on the HD for that matter - yes I'm on Vista). Then it munges it on a reconnect.

I probably do have a mismatch between the regex used to Add Books and the pattern used to save the book out. Normally I have the option to read metadata only from filename on, and I vary the Add Books import regex. Does that regex come into play during the reconnect to the folder (or during the initial transfer from main library to Device)?

Quote:

In other words, it will use the last value you put into the regexp box for adding books. If this regexp does not match the filename pattern used for sending to device (also in preferences), then nothing good will happen.

Does this happen even with a simple transfer from the main GUI library?

Does this happen even with a simple transfer from the main GUI library?

It looks like the reconnect is attempting to determine the ebook metadata from the ebooks a second time. It reads the metadata internally from the file, if it can, and if not, uses the regex on the filename. Then it revises the correct metadata in the cache file and ends up with a mess. I can fix this by controlling my settings, but it will be problematical for others.

I'll test to verify. (I think this is what you were saying, but it just sank in.)

OK, this will confirm that setting regex to defaults mostly fixes things. I say "mostly" due to the oddity of text file metadata. I vaguely recall that calibre attempts to divine the metadata from inside the text file rather than always getting it from the filename. If the "metadata only from filename" option is off, it is getting the metadata of a txt file incorrectly on the reread. Apparently calibre never writes the correct metadata internally into a txt file, so with that option off, on the reread of metadata, it's always wrong. Other formats have correct metadata written internally, or if not, the metadata is picked up from the filename on the reread, so I saw errors only with txt files after the default regexes were set.

Turning on the "metadata only from filename" option fixed the txt file problem.

I'll check the delete issues now that this bug is at least identified.