Calibre Companion is an Android app that permits Android devices to connect to calibre as a reading device, as opposed to connecting to the content server. A more complete description of the current release of the app is:

Spoiler:

The app connects to calibre as a device without using a cable, instead using wifi. In effect, it replaces the USB cable.

Why is this necessary? Calibre currently supports devices in two ways. The first is by connecting the device to the computer with a USB cable. When a device is connected this way, calibre is in charge. One uses calibre to send books to the device, delete books from the device, etc. The "on device" checkmarks show in the library when using this method.

The second way of connecting is by using calibre#s content server. When connecting this way, the device is in charge. The user taps/types on the device to navigate through calibre's web pages, getting books etc. There is no "on device" indication, either in calibre or on the device.

One problem becoming more and more common with the first method, and a major driver for this app, is that newer Android devices cannot be connected to calibre by a cable. These newer versions of Android (versions 3 and up, Honeycomb, Ice Cream Sandwich, and Jellybean) do not support being connected as a "disk drive", something calibre requires to be able to "see" the device. The app we are developing gets around this problem by connecting to calibre wirelessly (wifi, no cable).

The app currently supports:

Full support of calibre's device interface, including add and delete books, "on device", plugin notification, and automatic conversion.

Book display on the device grouped by title, author, tags, series, and publisher with first-letter navigation.

Book display grouped by custom fields.

Support for calibre's automatic metadata management facility. The app displays metadata supplied by calibre, not the metadata contained in the book's file.

Launching of reader apps if they advertize their capabilities. It works with (at least) FBReader, Cool Reader, and Adobe reader. Kindle for Android does not play nice, so files to be read with it must be placed in the "kindle" folder.

Collections (similar to Sony collections, Nook shelves & Kindle collections). The collections will appear in the same way as grouping by tags/series etc. The collections will be managed by calibre similarly to calibre's collection management for Sony devices.

Simplified driver control in calibre. The driver can be started and stopped using the Connect/Share menu.

Manual IP setup in case the network/wifi router does not support finding calibre automatically or if the device and calibre are not on the same local network.

Support for Android 2.3 and up.

Multiple folder format routing.

The ability to manually specify storage folders to support external SD cards with vendor-specific names.

What is not in the current app:

Detailed/complete metadata display (book details).

The content server interface.

Some menus, such as Setup, aren't as pretty as we would like.

The ability to delete books from calibre companion's database in calibre companion, without involving calibre.

Calibre Companion is available on Google Play. It runs on Android Gingerbread (2.3) and later.

Please make comments or report bugs using the devices MobileRead forum (this forum) (a thread per comment would be good), via private messages to either of us (chaley or steve_haley), or via email to “calibre_companion [at] MultiPie dot co dot uk”. Be sure to type “MultiPie” (many pies) and not “multiple”.

Full disclosure: Kovid has started work in calibre to support MTP connection to devices. He isn't yet able to give a delivery date, but he is making progress. Although our app does more than calibre will do over an MTP connection, you might prefer to wait for whatever he produces. If so, again, we won't be offended.

Regarding problems in the beta version:

1) There is a bug in MacOS that causes intermittent problems with connections and sending books. Although the problem is in MacOS itself, a workaround for the problem has been release in calibre 0.8.65.

2) Transferring 1000s of books causes problems. We are working on how to improve this process. NEWS: releases 1.6 and 1.7 contain many changes to ameliorate these issues.

Thank you for your interest and support.

CHANGE LOG:

Spoiler:

Version 1.8.1 (24/Aug/2012)

Spoiler:

NEW FEATURES:

Added pdf and azw3 to the default extensions

BUG FIXES:

Fixed problem with DB upgrades when the DB had been damaged somehow

Version 1.8 (23/Aug/2012)

Spoiler:

NEW FEATURES:

Dramatic performance improvements in searching and grouping

Performance improvements when connecting to calibre

Progress bars when uploading metadata to calibre or downloading metadata from calibre.

Cleanly disconnect from calibre when the app is exited using the back button.

Version 1.6.1 (18/Aug/2012)

Spoiler:

NEW FEATURES:
BUG FIXES:

Fix regression in group-by displays. (sigh. Long days lead to silly mistakes.)

Version 1.6 (18/Aug/2012)

Spoiler:

NEW FEATURES:

Grouping is saved and restored

Book/format folder routing (folder per extension)

Splash screen on startup while loading books

Message saying "transfer books now" when library is empty

BUG FIXES:

Total redo of memory management to (one hopes) eliminate the out-of-memory errors when sending books and on display of large libraries.

Don't force a particular sort when grouping

Ask for WiFi permissions, something that HTC seems to demand.

Version 1.5 (17/Aug/2012)

Spoiler:

NEW FEATURES:

Add error messages for incorrect IP addresses, for example to the content server.

Book/format folder routing (folder per extension)

BUG FIXES:

Better handle screen rotations when showing progress dialogs.

More information from the license manager

Version 1.4 (15/Aug/2012)

Spoiler:

NEW FEATURES:
BUG FIXES:

Fix regression in IP address handling

Version 1.3 (14/Aug/2012)

Spoiler:

NEW FEATURES:

Better IP address and password dialogs

Multi-level sorting

BUG FIXES:

Fix crash caused by missing covers.

Make progress bars survive application sleeping

Version 1.2 (13/Aug/2012)

Spoiler:

NEW FEATURES:

Attempt to detect connecting to the content server instead of the device handler.

Prevent WiFi sleeping when connected

Add the CC version number to the dialog with calibre.

BUG FIXES:

Make transfer and initialization dialogs sticky.

Version 1.1 (12/Aug/2012)

Spoiler:

NEW FEATURES:
BUG FIXES:

Fix problems caused by library redisplay while downloading.

Don't redisplay on every book. See if more books will arrive in time.

Fix problem with creation of default folder.

FAQ:

The app won’t connect to calibre!

Spoiler:

First, ensure that you have started the wireless device driver in calibre. Click on Connect/Share, then “Start wireless device connection”.

If the above isn't enough to get you going, then it is possible that your wireless router has disabled multicast or that your firewall does not handle dynamic port assignment. See the next item if your firewall is configured port-by-port. Regarding multicast: this feature is sometimes called multicast-DNS, zero-config, or bonjour. Some Cisco routers have this “feature”. It is impractical for us to provide detailed instructions for all the routers out there. Check the manual for your router to see how to enable multicast.

If you can’t enable multicast, if your device and the computer running calibre are not on the same local network, or if you are running a firewall that does not handle dynamic port assignment, then you must provide calibre’s IP address to your phone.

First, in calibre, go to the device customization panel for the Smartdevice driver (Preferences -> plugins -> Device interface plugins -> Smartdevice app interface). Check the box "Use a fixed network port". Change the default port number (9090) if you wish. You must not enter the same port used calibre's content server. Make a note of the port number that you use.
Next, start the wireless driver in calibre. You will then be able to see calibre’s IP address, as shown in the following:
Note the IP address.

Now, enter the IP address on your phone. Click on the “Calibre’s IP address” menu item.
You will see the following dialog.
Enter the IP address you noted. Enter the port number you noted in device customization (above) in the port box. If your machine has a domain name, you can (at least in theory) enter that name instead of the IP address. You must always enter both an IP address/name and a port number.

Last: be sure that the firewall you are running allows incoming connections to the chosen port number.

I have books on my device that don’t show up in calibre. How do I fix this?

Spoiler:

For the moment the app does not scan your device for books. The only way calibre knows about a book is if calibre sent it to the device. For several reasons related to how calibre gets metadata from “unknown” books, scanning would be very slow.

We are considering adding a feature to the app that permits adding books on the device to the app’s database. The user would provide the correct title and author when the book is added, eliminating the need for calibre to scan the book’s internal metadata (the part that is so slow). We might be able to get the author/title information from the book automatically for some formats, in particular epub, but as book-contained metadata is often wrong we would still provide a way for the user to change this information.

This feature will interact with extension (format) routing, so it won't happen before extension routing is implemented.

The app connects but then sits there doing nothing. What do I do?

Spoiler:

Calibre Companion connects to calibre as a "device", which means that calibre is in charge. After Calibre Companion connects, you then use calibre to send books to your device and delete books from your device, just as if your device were connected using a cable. There is (currently) no way for Calibre Companion to ask calibre to send books.

I want to help get my problems fixed. What do I do?

Spoiler:

The first thing to do is tell us about the problem by any of sending us an email, a private message, posting on this thread, or creating a new thread. We will do what we can to help you, but we can't do much to help if we don't know you are having the problem.

Reporting problems or asking for help in reviews on Google Play is the most difficult for us. Our first problem is contacting you for more info; it isn't easy to find an email address for a reviewer, and in fact might be impossible. The second problem is that other people who might have solved to problem cannot answer at all. The third problem is that we can't be sure that you even want a response and that you might consider us contacting you an invasion of privacy. We are not suggesting that people should not leave reviews, good, bad, or indifferent. Instead, we are suggesting that if you want a personal response from us, there are better ways to go about contacting us than a review.

It could be that we will need a calibre debug log to solve the problem. You can get a debug log by running calibre in debug mode. Start calibre, then click the small down arrow next to preferences as shown in this screenshot.
Click on "Start calibre in debug mode". Let calibre restart and answer OK to the dialog that confirms that calibre is in debug mode. Attempt to connect to calibre. Get to where you are having the problem, then close calibre. You will see a debug log in a new window. An example is

Calibre currently supports devices in two ways. The first is by connecting the device to the computer with a USB cable. When a device is connected this way, calibre is in charge. One uses calibre to send books to the device, delete books from the device, etc. The "on device" checkmarks show in the library when using this method.

The second way of connecting is by using calibre#s content server. When connecting this way, the device is in charge. The user taps/types on the device to navigate through calibre's web pages, getting books etc. There is no "on device" indication, either in calibre or on the device.

One problem becoming more and more common with the first method, and a major driver for this app, is that newer Android devices cannot be connected to calibre by a cable. These newer versions of Android (versions 3 and up, Honeycomb, Ice Cream Sandwich, and Jellybean) do not support being connected as a "disk drive", something calibre requires to be able to "see" the device. The app we are developing gets around this problem by connecting to calibre wirelessly (wifi, no cable).

I am a Sony user and I want my collections!

Spoiler:

First, check if the grouping and sorting features available in the app don’t already provide an alternate way to get the same information. If they do not, then you can enable collections for the Smartdevice driver in the same way you did for the Sony driver. Go to the device preferences (in calibre, go to preferences -> plugins -> device plugins -> Smartdevice App Interface), and enter the lookup names for the columns for which you want collections. The Sony collections tweaks work here as well. Reconnect the device and your collections will appear under the group menu item “User Categories”.

Why does the sort menu disappear when I select User Categories (collections)?

Spoiler:

Collections are sorted by calibre, and the app must respect that order. Although it would be possible to add yet another sort type to the list, we thought that the need would be too rare to compensate for the extra confusion about what it means.

How do I add my custom columns to the Group menu?

Spoiler:

Calibre Companion supports grouping on calibre text-type custom columns (text, text like tags, fixed set of values, and built from other columns). To group on such a column, add the column’s calibre lookup key (the word beginning with #, as in #genre) to the app’s list of custom columns to show. You do this by clicking the menu button on your device (usually three dots) and selecting Custom Categories.
Enter the names in the text box shown in the resulting dialog. Be sure to include the leading # character.
Note: it is not an error if a lookup name you enter does not actually exist in any books on your device. This can happen if you connect to multiple libraries that have differing custom column setups. That said, if you expect to see information when you click on a lookup name in the group menu but don't, check the spelling of the lookup key.

When you add or change the custom column keys, Calibre Companion must rebuild some information in the database. This rebuild process will happen next time Calibre Companion is "restarted", something that because of how android works could be almost any time. If you have a large library this process can take a few moments. You can see a progress indicator by checking the status bar.

How do I control which book extensions (“formats” in calibre-speak) are transferred when I use calibre’s send-to-device command? How do I change the folder on my device where they are stored?

Spoiler:

Unlike devices that calibre supports by cable, you control in the app which formats are acceptable along with their priority on the device. This helps if you have multiple devices for which you want to use a different list or different priority.

To set up the extensions, click the menu button on your device (usually three dots) and select Select extensions.
Next, scroll through the long list of extensions and check the ones you want calibre to be able to send to your device. Don’t try to order them at this point, it is easier to do in a second pass.
After you have "checked" all the extensions you want to use, it is time to set their priority. This is easiest to do if you close and reopen the dialog, because all the checked extensions will be at the top. So press OK, then reopen the dialog. Now use the arrows to move the selected extensions up and down. The higher an extension, the higher its priority.

You can use this dialog to change the folders where extensions are stored. See the following image of the extensions dialog.
To change the default folder (the one used if no specific folder is chosen for an extension), open the extensions dialog and click where the "1" points. To change the folder for a specific extension, click below the arrows on that extensions line. Number "2" in the dialog image shows where you would click to change the folder for epubs.

If you start changing a storage folder, you will see the following warning dialog telling you that continuing will remove the books from the app (erase its database).
Press the "Change" button if you wish to continue. You will see the following dialog.
If, for example, you are changing the storage folder for mobi books to the folder used by Kindle for Android, scroll the list of folders until you see the "kindle" folder. Tap that folder to enter into it, then tap the "This folder" button.

In the Kindle for Android case, after having changed the storage folder, to read the books you must manually start K4A. It will import the books you have sent to Calibre Companion so you can read them.

When I first look at the extensions dialog, it contains only epub and mobi. Why?

Spoiler:

The app gets the complete list of extensions from calibre the first time it connects. If you want to add extensions, connect once to calibre then disconnect. The list of formats will be available.

Sending books seems slow

Spoiler:

We agree. There is some interaction between wifi and the protocol we have chosen to send the books. We have some ideas for increasing the performance, and will look at it when time permits.

What does the Delete Passwords menu item do?

Spoiler:

You can tell calibre to require a password from the device when the device connects. If you do this, then the app will ask you for that password and remember it. Selecting Delete passwords will force the app to forget all remembered passwords.

I see a scary warning message when I try to change a storage folder on my device. What gives?

Spoiler:

For various reasons our app needs to reinitialize its database when you change a storage folder, removing all the metadata for the books you have sent. You need to send the books again for the app to have the metadata.

Note that the books themselves are not deleted from the old folder. If for some reason the books are no longer in calibre, you can copy them from your device and re-add them to calibre. Once you are happy with the new folder you chose, you should delete the old copies of the books, and possibly their containing folder as well.

How do I get Kindle for Android to open my books?

Spoiler:

Kindle for Android is one of the very few reader apps that does not publish to Android the kinds of books that it can open. Because of this, Calibre Companion cannot automatically pass books to it.

You can resolve this problem one of two ways:

Use a different reader app that can open .mobi files. There are several. This will work only if your books are not infested with DRM.

Set Calibre Companion's storage folder to Kindle for Amazon's books folder. Its name is "kindle". Its location depends on the version of Android you are running.
Important: changing the storage folder will erase Calibre Companion's books database. You will need to send the books again. The books themselves are not deleted from the old location. You should delete those books from your phone once you are sure you have no further use for them.

See FAQ "How do I control which book extensions" above for instructions on how to change the storage folder.

After having changed the storage folder, to read the books you must manually start Kindle for Android. K4A will import the books you have sent to Calibre Companion so you can read them.

Why is there cat hair in some of the dialogs?

Spoiler:

Charles’ cat Lizzie insists on sitting on his keyboard. Some fur she sheds is molecularly transmitted through the keys into the app’s code.

Connection *seems* to be instant, and a circle rotates along with the words 'Synching with calibre' etc

The box disappears instantly the screen is touched, and the connection seems to be broken as 'connect' appears on the top menu, instead of 'disconnect'

Yes, it does. This is a bug. The best I can say at this point is "don't do that".

Quote:

If I rotate the screen the box goes, but this time the connectioin seems to stay

Yes. This is the same as the problem that was mentioned in the problems section in the first post.

Quote:

I have a big database (6500 books) and it has been connected for 45mions so far.

The size of the books database is not relevant. What matters is the number of books that have been sent to the device.

By any chance did you specify an IP address and a port? If so, did you use the port of the content server? If so, then that is the cause of the problem. See the updated FAQ response #1 in the first post.

Quote:

It would be good if there was some indication of the data volume having been transmitted, as I have little faith in WiFi. Having a connection is one thing, having a useful one is quite different.

There is a progress dialog when doing longer things like downloading books. The fact that you are seeing extremely long delays but are not seeing a progress dialog tells me that something else is going wrong. We have seen this behavior before when calibre's content server IP address & port were given to the app instead of the port of the smartdevice driver.

Quote:

I am now sitting and waiting for some evidence of data on the Nexus

For a new connection where no books have been transferred, the initial synchronization takes seconds.

If you have not given calibre's content server port to the app then I suggest that your run calibre in debug mode as described in FAQ item # 3. That will tell us whether or not the app is connecting properly, and if so what the app is actually doing from calibre's perspective.

This time I entered calibre's IP and used the default port 9090 into my Nexus 7. To do this I had to change the plugin to use port 9090 by checking the box (see attached.) This time the log shows port 9090 being used

Deleted all of the books on my device via calibre's UI.
On the Nexus changed location of where to send the books.
Sent 109 books to my device in 4 minute and 30 seconds.
Books ended up in the new location.

@JDriver: reading that debug log, it looks like your device might have gone to sleep and shut down the network connection. Hard to know for sure, but that would account for it.

@everyone: there is a new release of Calibre Companion on Google Play. version 1.2. This release fixes the landscape/portrait problems, adds a check to verify that the device is connecting to calibre's smartdevice driver and not the content server, and prevents the device from turning off the network while connected to calibre.

@everyone: "Lissa" left a review on google play mentioning that she (?) had problems with cover thumbnails not always downloading. Can anyone shed some light on this? We would like to fix the problem, but it is very hard to fix a problem that we cannot reproduce.

I have updated the companion to 1.2
I have enabled calibre to pass through the windows firewall
On running calibre on the pc with the wireless connection enabled, then went to the area where there is wifi and started the app, which immediately asked if the ip addresses were correct, as it could not id calibre. (they are)

One thing which does occur to me is that the books & database are kept on a NAS drive which is configured as a drive on the host machine. As the NAS drive has its own IP address could this cause problems?

I have updated the companion to 1.2
I have enabled calibre to pass through the windows firewall
On running calibre on the pc with the wireless connection enabled, then went to the area where there is wifi and started the app, which immediately asked if the ip addresses were correct, as it could not id calibre. (they are)

What that tells me is that your wifi is not on the same "local network" as calibre. That is, the IP address of the machine running calibre is not visible from wifi. Unfortunately, this theory is not supported by your debug log. On the other hand, whatever the app connects to is not responding within a two seconds thereby producing calibre's message, so things are very confused.

One way to help verify that the two networks can communicate is to start calibre's content server, then attempt to connect to it from your tablet. This process removes the effects of any lingering bugs in either our calibre driver or the app.

If your device can see calibre's content server, then we must have another set of problems. I suspect firewalls and routers, but I cannot prove that.

Quote:

One thing which does occur to me is that the books & database are kept on a NAS drive which is configured as a drive on the host machine. As the NAS drive has its own IP address could this cause problems?

It shouldn't, assuming that it doesn't slow down calibre too much. Our app expects calibre to be able to respond within 2 seconds of a connection request. That might be tight if calibre is busy hitting the network.

Hi, I bought the Calibre Companion earlier today and used it to transfer a couple of books. That worked fine, but now it crashes each time I try to run it. The error is java.lang.NullPointerException and the message is "Unable to resume activity." It's 100% reproducable. Any suggestions on how to proceed? Let me know if you'd like any data from me. This is using the latest version of Calibre Companion (1.3 from Aug 13) as well as the one prior to that. The device is a Nexus 7 running Android 4.1.1.

I have only one small request, can we have a "refresh metadata" button or something similar?

For example: If I change the metadata for a book while the Nexus 7 is connected and would like to re-send the metadata for the version that is already on my device. At this point, the only way for it to re-send and sync the metadata is for me to disconnect, and then reconnect, then the auto-sync takes over. Would it be possible to add a button shortcut for this in the app?

Edit: And for some strange reason, when I sort by Author but do not group them, the books by that author appear to be listed in random order. Is that normal? Is there a setting that I'm missing that would be some sort of secondary sort order?

Last edited by FlareHeart; 08-13-2012 at 03:24 PM.
Reason: Added second question.