So, after spending a few days fooling around with things, here are places that I've noticed the three-column "General Metadata" plugboard method fails:

1) As already mentioned, any title with an ampersand (&) gets a space after the ampersand when it's reduced down to its initials. (I know eschwartz gave me some code that is supposed to fix this, but as I mentioned in my other posts, it resulted in an error.)

So "Title & Title" would be abbreviated, "T& T" this gets especially absurd when you have a series name like (and this is an honest-to-god series) "Vampires & Mages & Weres, Oh My!" which is them abbreviated to "V& M& WOM"

However, if the title were "Title and Title" it would be abbreviated "TaT" so clearly inserting the space after the ampersand is something that only happens there.

It would be an easy enough fix to just change all series titles to use "and" instead of "&" but sometimes that doesn't happen when you're retrieving the metadata off the web and you have to go through and check everything manually.

Personally I would prefer to see "Title and Title" abbreviated as "T&T" instead of "TaT", however, this would mess with titles using "and" in any other context (off the top of my head I can't come up with another context for it but I'm sure one exists somewhere.)

Actually, this seems to be a recurring problem with any single-letter word, because....

2) If a series title begins with the word "I" (as a personal pronoun referring to oneself" it gets a space after it. So a series titled "I Spy" would be abbreviated "I S" Again, not sure how to fix this without messing with other words containing the letter "i".

3) Titles with an "a" as a single word in the middle get a space after the "a" when being reduced to their initials. Eg: "Measure of a Man" is abbreviated to "Moa M" and "What's a Boy to Do?" would be abbreviated "Wa BtD." As above, with "i", I'm not sure how to fix this without messing up anything containing an "a" as part of a word.

4) If you have a series title like "A to Z" it would be nice to abbreviate it to "A-Z" but again, this would mess the word "to" in any other context (eg: "Brothers to the End" would then be abbreviated "B-tE" which just doesn't work. Not sure what to do about that.)

5) Another interesting way to shorten series titles would be to get rid of extraneous words at the end as well as at the beginning. A (partial, I'm sure) list I came up with would be: Series, Stories, Novels, Books, Trilogy, etc.

For example (for you, eschwartz) if we added "Files" onto that list, the series "The Dresden Files" would be shortened to "Dresden" instead of "DF". So if the series name is "The Home Series" (as in, that is the metadata that downloads, complete with "Series" at the end) it would be shortened to "Home" rather than "HS." "The Impulse Trilogy" would be "Impulse" instead of "IT".

6) This one isn't limited to the 3-step General Metadata method, but it would be great to have a way to format the digits for series so that they stay in sequential order if they have numbers >1 with a decimal place and have 10 or more titles in the series (ie 2.50 happens sequentially after 10, because it isn't formatted as 02.50.)

Also, if there is a series with over 100 titles (I happen to have one; it's not so much of a series as it is a collection of shorts people submitted for an event over on Goodreads, and there are ~190 of them) then you have the following situation:

01, 02 ... 09, 10, 100, 101 ... 109, 11, 110, 111 ... 119, 12, 120, 121 ... and so forth. In this case, there would not need to be any accommodation for decimal places, but for proper sequencing, we would need 001, 002, 003 ... 011, 012, etc.

This is all really nitpicky stuff that probably only matters to the exceptionally anal-retentive, but it's been interesting hunting down these sorts of glitches, and if eschwartz or someone wants to stretch the boundaries of what to do with these plugboards, this could give them something to chew on, so I thought I would share.

You make an excellent point about single-letter words, this actually gives me a new direction to think in. I'll get back to you. Meanwhile,...

4 & 5 I really think the best way to deal with it is to change the actual series name, you can do this really fast in the tag editor to the left. Right-click on any series name and choose "rename", it will change for all books with that metadata field.

6 can be done easily, no problem: In the calibre manual for advanced formatting is all kinds of cool stuff, which you may want to take a look at for lots of cool new tricks, but this particular bit is what you are looking for right now:

Spoiler:

Quote:

Second: formatting. Suppose you wanted to ensure that the series_index is always formatted as three digits with leading zeros. This would do the trick:

Quote:

{series_index:0>3s} - Three digits with leading zeros

If instead of leading zeros you want leading spaces, use:

Quote:

{series_index:>3s} - Three digits with leading spaces

For trailing zeros, use:

Quote:

{series_index:0<3s} - Three digits with trailing zeros

If you use series indices with sub values (e.g., 1.1), you might want to ensure that the decimal points line up. For example, you might want the indices 1 and 2.5 to appear as 01.00 and 02.50 so that they will sort correctly. To do this, use:

Quote:

{series_index:0>5.2f} - Five characters, consisting of two digits with leading zeros, a decimal point, then 2 digits after the decimal point

If you want only the first two letters of the data, use:

Quote:

{author_sort:.2} - Only the first two letter of the author sort name

The calibre template language comes from python and for more details on the syntax of these advanced formatting operations, look at the Python documentation.

using ":" after the field name says lets go advanced formatting. "0" says lets add zeros. ">" says lets add them to the beginning ("<" would put them at the end) and "3s" says let there always be at least 3 characters in the string.

So "{series_index:0>3s}" will add enough leading zeros to make sure there are always at least 3 characters.

If you wish to take into account decimal-notated short stories, use "5.2f" instead, which adds the "0" to beginning and end to format the number as "5" digits, with ".2" or two digits AFTER the decimal. The value is calculated as a "f" that takes decimals and is very strict that there will always be "5" digits.

So: Use {series_index:0>3s} for series' into the hundreds, and {series_index:0>5.2f} for series' into the tens with short stories in between. If you wish to do both, use {series_index:0>6.2f} which will make all series' show as 001.00 and onward.

You make an excellent point about single-letter words, this actually gives me a new direction to think in. I'll get back to you. Meanwhile,...

Thank you! I am clueless about the coding, but I do like hunting down the problems for other people who know more than me to solve

Quote:

4 & 5 I really think the best way to deal with it is to change the actual series name, you can do this really fast in the tag editor to the left. Right-click on any series name and choose "rename", it will change for all books with that metadata field.

Okay, doable. I think I was hoping there was a way to have my anal-retentive cake and eat it too, with regard to having the book series technically "correct" in Calibre, but having it abbreviated to something prettier and/or more informative on the device.

<snip>

So: Use {series_index:0>3s} for series' into the hundreds, and {series_index:0>5.2f} for series' into the tens with short stories in between. If you wish to do both, use {series_index:0>6.2f} which will make all series' show as 001.00 and onward.[/QUOTE]

I can't imagine where I would need a situation where 001.00 would be necessary.

To use different templates for different series, would the best way to go about this be to store the different series (say, the one with 190 titles) in a separate virtual library, and apply {series_index:0>3s} to that library, and then use {series_index:0>5.2f} in a different virtual library for all the others?

To use different templates for different series, would the best way to go about this be to store the different series (say, the one with 190 titles) in a separate virtual library, and apply {series_index:0>3s} to that library, and then use {series_index:0>5.2f} in a different virtual library for all the others?

Virtual libraries still use the same library and plugboards. Different LIBRARY, yes. Or we could code a program that will select between them based on a search filter, but I'd have to get back to you on that.

Virtual libraries still use the same library and plugboards. Different LIBRARY, yes. Or we could code a program that will select between them based on a search filter, but I'd have to get back to you on that.

Okay, thank you. The clumsier possibility is that I could have two plugboards (one saved in a text doc and cut/pasted to replace the other when necessary) and switch between them depending on which library I am sending to the device.

Okay, thank you. The clumsier possibility is that I could have two plugboards (one saved in a text doc and cut/pasted to replace the other when necessary) and switch between them depending on which library I am sending to the device.

Don't do this, please. It would be easier to just have a special library. Only virtual libraries (read: virtual=not really) share plugboards, since it is all in the same database.

Different databases have different plugboards. Virtual libraries share the same database, so they use the same plugboards; They are just glorified searches, really.

Don't do this, please. It would be easier to just have a special library. Only virtual libraries (read: virtual=not really) share plugboards, since it is all in the same database.

Different databases have different plugboards. Virtual libraries share the same database, so they use the same plugboards; They are just glorified searches, really.

LOL ah okay. I didn't realize there was a way to have multiple actual databases. Time to see what I can do!

ETA: Okay, now that I have had a chance to plug them it, it looks like they are working, so yay! I didn't screw it up!

Question: with {series_index:0>5.2f}, what would I change to have only one decimal place instead of two? I've never had a series numbered 1.11, 1.12, 1.13 and so forth. If I ever do, clearly now I have the code for it, but since I don't... according to your explanation about how the code works above, would I change it to {series_index:0>4.1f}?

Okay, yep! That worked. And having experimented with a bunch of other configurations, it looks like that is as close as we can get to suppressing leading or trailing zeroes when they're unnecessary without compromising on the sequential ordering.

LOL ah okay. I didn't realize there was a way to have multiple actual databases. Time to see what I can do!

ETA: Okay, now that I have had a chance to plug them it, it looks like they are working, so yay! I didn't screw it up!

Question: with {series_index:0>5.2f}, what would I change to have only one decimal place instead of two? I've never had a series numbered 1.11, 1.12, 1.13 and so forth. If I ever do, clearly now I have the code for it, but since I don't... according to your explanation about how the code works above, would I change it to {series_index:0>4.1f}?

Okay, yep! That worked. And having experimented with a bunch of other configurations, it looks like that is as close as we can get to suppressing leading or trailing zeroes when they're unnecessary without compromising on the sequential ordering.

Yep, you got it!

And no need to apologize for less obvious features not being more obvious, it takes us all a little experience (and perhaps a lot of help) to get used to all the things you can do! I mean, there are so many of them, where do you start learning? (Or so I thought in the beginning. Now I'm a pro. I think. )

And no need to apologize for less obvious features not being more obvious, it takes us all a little experience (and perhaps a lot of help) to get used to all the things you can do! I mean, there are so many of them, where do you start learning? (Or so I thought in the beginning. Now I'm a pro. I think. )

Yes, well, until you mentioned it, I never even noticed the "Calibre" icon on the toolbar. I just always went straight for the one that said "Preferences" and I was coming up blank there. Suddenly whole new possibilities have opened up!

Regarding things like templates and plugboards I sort of have a mental block where this kind of stuff is concerned. I can't seem to teach myself just by reading about it--it just all seems like so much gibberish--but if someone breaks it down and explains it to me, with examples, I can usually figure out how to monkey-see-monkey-do the simpler stuff. So your little explanation about what the numbers and symbols meant helped a lot.

If you ever do figure out a more streamlined way to add a specific tag based on the array of various tags each different book might have, (like one overall tag for SFF/UF/PNR when the tags the books might come with are scifi, sci-fi, science fiction, fantasy, urban, urban fantasy, vampires, werewolves, magic, paranormal, etc etc etc) I would be very interested in that. It won't help categorize books that come with no tags or only very vague and general ones, but it would help.

Regarding things like templates and plugboards I sort of have a mental block where this kind of stuff is concerned. I can't seem to teach myself just by reading about it--it just all seems like so much gibberish--but if someone breaks it down and explains it to me, with examples, I can usually figure out how to monkey-see-monkey-do the simpler stuff. So your little explanation about what the numbers and symbols meant helped a lot.

Thanks!! I try to be user-friendly, and explain + give the TL;DR if possible, but it is difficult to know when I have succeeded (since I, of course, understood it all perfectly. ).

Quote:

If you ever do figure out a more streamlined way to add a specific tag based on the array of various tags each different book might have, (like one overall tag for SFF/UF/PNR when the tags the books might come with are scifi, sci-fi, science fiction, fantasy, urban, urban fantasy, vampires, werewolves, magic, paranormal, etc etc etc) I would be very interested in that. It won't help categorize books that come with no tags or only very vague and general ones, but it would help.

I can probably do this, but not for a few days, when I get off Winter Break. I'll get back to you on that.

If you ever do figure out a more streamlined way to add a specific tag based on the array of various tags each different book might have, (like one overall tag for SFF/UF/PNR when the tags the books might come with are scifi, sci-fi, science fiction, fantasy, urban, urban fantasy, vampires, werewolves, magic, paranormal, etc etc etc) I would be very interested in that. It won't help categorize books that come with no tags or only very vague and general ones, but it would help.

Well, I'm back!!

Would you like to try this now?

If I remember correctly, you want to generate a #collections field, which will be used by Kindle Collections to create collections, right?

So what you will want is something along the lines of

Code:

{tags:switch(Sci-Fi,SFF/UF/PNR,(Urban )?Fantasy,SFF/UF/PNR,none)}

but with switch(pattern,value,pattern,value,......,value_if _empty) sets where pattern is (the regex of) any possible tag, and value is what the column should equal if so. value_if_empty is returned if no tags match, we can leave that blank.

So we can expand that to match all your tags, and add any other stuff you need into the collection naming.

Stupid me, I know what the problem is here. I created a new capture group at the beginning to remove leading spaces, but left the replace code replacing the nonexistent capture group. Properly, it should've been this (fix in red):

since I can't think why I created a capture group in the first place -- it does nothing.

And without the formatting error, it seems to work perfectly. Try this again?

EDIT: @chaley, perhaps this should be updated in the first post? This is derived from what Kovid used in his general program mode guide. I used [^\s] instead of [\w], following your lead, but added the inverse at the beginning, like Kovid's example.

5) Another interesting way to shorten series titles would be to get rid of extraneous words at the end as well as at the beginning. A (partial, I'm sure) list I came up with would be: Series, Stories, Novels, Books, Trilogy, etc.

For example (for you, eschwartz) if we added "Files" onto that list, the series "The Dresden Files" would be shortened to "Dresden" instead of "DF". So if the series name is "The Home Series" (as in, that is the metadata that downloads, complete with "Series" at the end) it would be shortened to "Home" rather than "HS." "The Impulse Trilogy" would be "Impulse" instead of "IT".

I decided we can do this after all.

We will edit the #stripped_series field, which is already removing the leading (A|The|An) and the full scan will look like this:

If I remember correctly, you want to generate a #collections field, which will be used by Kindle Collections to create collections, right?

Yes, pretty much. I can, of course, generate collections based on author name, series name, and even tags, but the problem is that I am trying to break my collection down into genres because the Collections Manager hack on my PW has the ability to nest collections inside collections, so I'd like to have folders for each genre, and then relevant sub-folders for authors/series inside those.

It would be easy enough to tell it to generate a collection based on X, Y, and Z tags, but the tags you might get when downloading metadata (or using the metadata in the file) are all over the place. So yeah, the idea would be to search for a set of tags like "Sci-Fi, Science Fiction, Fantasy, Urban Fantasy, Paranormal, vampires, shifters, etc etc, etc" and have it generate either a another tag, or a value for a "Genre" column, that would just be "SFF/UF/PNR".

Same principle for all the variations of tags you might find on M/M Romance books, only with that I want to go a step further. Because this is the genre I publish in, I read a lot of it, naturally. Enough so that it's not going to do me much good making a collection for each author because there are hundreds.

So except for the ones I would like to have in their own author/series collections, I would like them to go into a collection that would be (for example) "M/M misc authors A-B" (C-D, E-G, H-L, M-R, S-Z). So if the book had Genre=M/M Romance (based upon the tags) AND has "Author Grouping A-B" (based on the code you gave me a couple weeks ago." It would get sorted into the collection "M/M misc authors A-B"

Of course, then there's the question of how to distinguish the "misc" authors from the special authors who get their own folders, which I'm sure is an entirely different headache, lol.

Quote:

So what you will want is something along the lines of

Code:

{tags:switch(Sci-Fi,SFF/UF/PNR,(Urban )?Fantasy,SFF/UF/PNR,none)}

but with switch(pattern,value,pattern,value,......,value_if _empty) sets where pattern is (the regex of) any possible tag, and value is what the column should equal if so. value_if_empty is returned if no tags match, we can leave that blank.

So we can expand that to match all your tags, and add any other stuff you need into the collection naming.

Okay, I will compile a list of tags. Well, lists. What I may want to do is have you show me how it's done with the SFF/UF/PNR genre and then I'll see if I can replicate it for another genre.

Quote:

Problem from before:

Stupid me, I know what the problem is here. I created a new capture group at the beginning to remove leading spaces, but left the replace code replacing the nonexistent capture group. Properly, it should've been this (fix in red):

And without the formatting error, it seems to work perfectly. Try this again?

Awesome! That works perfectly for the space after the ampersand issue. It also solves the problem of the space being added after other single-letter words like "a" or "I". So now the series "I, Spy" is "IS" now instead of "I S" and "Measure of a Man" is "MoaM" rather than "Moa M" like it was before.

And I see you posted something else while I was writing this. Back in a moment with answers for that!

As far as "Science Fiction", "Sci-Fi", "scifi" goes, you may just want to edit those tags directly, since they are the same thing, but not conforming to standards across metadata sources. I would think it's better to have these changes available directly in the tag browser, and of course it will shorten what may become quite the list.

Any tag can be edited in the tag browser and the change will propagate across all books with that tag, merging with a pre-existing tag if necessary.