« Reading List: Fooled by Randomness | Main | United Arab Republic 2.0 »

Tuesday, February 22, 2011

Bugs: Ugly Gotcha Importing EPUB Books into iTunes (iTunesMetadata.plist manifest)

Today I posted the final public release of the EPUB edition of The Hacker's Diet. For details of this edition, see my earlier note announcing the release of the test edition a month ago.

In the process of refining this document for publication, I stumbled into a hideous gotcha in the process of importing an EPUB book into iTunes, thence to be transferred to a reading device such as an iPad or iPhone equipped with the iBooks application. Suppose you've developed an electronic book on your computer in the EPUB format, checked it with various EPUB reader software on that machine, and run it past an EPUB validator with no problems reported. As the last step in the quality assurance process, you want to make sure the formatting is correct when viewed with Apple's iBooks application, so you drag your .epub file into iTunes, then synchronise your reading devices with it. Happily, everything looks good, so you then install the just-tested .epub file on your Web server or submit it to the publisher who will offer it for sale to the public.

Before long, you start to get messages from readers who downloaded the book and, being cautious, submitted it to the same validator you used before transferring it to their reader devices, and discovered that it failed validation. You then re-validate the file on your server—the same one which passed validation before—and discover, to your dismay, that it now fails, due to the presence of a file in the top level directory of the archive named iTunesMetadata.plist which isn't listed in the manifest for the archive. How could this have happened?

When I figured out what was going on, I could scarcely believe it. One of the principal reasons people pay a premium for Apple gear is that they usually get the important stuff right, even in early releases. What we have here is a breathtaking example of when they don't. When you import an EPUB book, iTunes makes a copy of the .epub archive in its own library directory structure and, in the process, adds a file named iTunesMetadata.plist to the archive which contains metadata describing the file. It does not add this file to the EPUB manifest. So far, so good—Apple is perfectly free to add information to documents it's managing within its own application's database.

But it does't stop there: iTunes then goes and modifies the document you're importing as well, including the iTunesMetadata.plist file in it and thereby rendering it an invalid EPUB archive. This is done without any warning to the user that the input file is being modified. If you subsequently publish this modified file, you're providing your customers an invalid and defective product. It may very well work, but if they go to the trouble of validating it themselves, they're going to be getting in touch with you.

As long at this bug remains in place, it is essential that EPUB book developers always copy documents they're about to test by importing them into iTunes into a throwaway document, import that, and then delete it immediately after its importation. Never import an EPUB file into iTunes which you intend to use in any other manner subsequently.

Posted at February 22, 2011 20:43