David Zeitlyn asks when indexes became a part of scholarly book production.
He is entitled to go back at least as far as the 8th century, the date of
the earliest manuscripts of the Etymologiae of Isidore of Seville (d. 636)
that contain an INDEX LIBRORUM. It begins with these words: "Ut valeas
quae requiris cito in hoc corpore invenire, haec tibi, lector, pagina
monstrat de quibus rebus in libris singulis conditor huius codicis disputavit,
id est in libro: I. De Grammatica et Partibus eius. II. De Rhetorica et
Dialectica," and so on through the twenty books of the Etymologiae. This
INDEX LIBRORUM is printed in W. M. Lindsay's Oxford Text of the work. No
doubt there are earlier ones yet, although of course indexes are hardly
to be expected until the ancient book-roll (in which they would have made
little sense) was superseded by the codex, not very long before Isidore's time.

Tom McArthur's WORLDS OF REFERENCE: LEXICOGRAPHY, LEARNING AND LANGUAGE FROM
THE CLAY TABLET TO THE COMPUTER (Cambridge University Press, 1986), if I
remember correctly, has the information that Daniel Zeitlyn requested.

When do Indexes, tables of contents and footnotes first appear in books?

Leaving indexes for another time, the history of footnotes is amusingly set
out by Gertrude Himmelfarb in "Where have all the footnotes gone?" in the Book
Review section of the New York Times for 16 June 1991, 1, 24.

On the history of indexing: Witty, Francis J. "The Beginnings of Indexing and
Abstracting: Some Notes Toward a History of Indexing and Abstracting in
Antiquity and the Middle Ages", THE INDEXER 8 (1973): 193-98.