I am using several bibtex files, dedicated to several topics. But some references are common to several topics and then appear in several of my bibtex files. Currently, I am writing a document dealing with several of these topics, and need to use several of these bibtex files at the same time. Obviously, I get the error message when compiling with bibtex: "repeated entry". I would like to avoid this error message, but without removing my entries from the bibtex files, as I need them when compiling only with one of them (when writing articles on one topic only), and without merging my bibtex files.

7 Answers
7

And by that, I really mean one BiBTeX file on your system for your whole bibliography.

As far as I understand this is actually the intended usage, and there are several good softwares out there that make managing a bibliography easy. Much easier, in fact, than having to manage separate bibliographies. This is also why BiBTeX only includes those references that are actually used, and not just the whole file.

yes of course it would be the easiest way. But I have lots of latex files (10 years of research) that refer to those separate bibtex files, and I would like to keep them compile as is, without changing the name of the bibtex files.
–
Caroline FontaineAug 9 '10 at 10:02

This doesn't, in my experience, work well if you pool people's .bib files, say in the context of a research group. A good thing about sharing files in this way is finsing errors: most article's bibliographies have many easy-to-find errors. @Caroline: I wrote a piece of Perl code about 15 years ago to handle this: I'm going to try to find the code. Wish me luck!
–
Charles StewartAug 9 '10 at 10:05

ok, a solution could be: keep my old bibtex files to keep old latex files compiling, and create a new and unique bibtex file for future use, avoid this problem. I was just wondering if there was a trick as on option for bibtex or some geek stuff.
–
Caroline FontaineAug 9 '10 at 10:09

3

No, there is no way around the 'repeated entry' error. Bibtex will always complain about that. Creating a new, merged bib file is the only solution.
–
Taco HoekwaterAug 9 '10 at 10:39

1

In principle having one bib file for everything is ideal. In practice, it's not possible. Too many situations require individualized editing. Egs: some journal BSTs produce undesirable results when certain fields are present; journal titles in full vs abbreviated (and different abbreviations); manually truncated long author lists; different conventions for page ranges (1234-56 or 1234-1256). I wish there were a commandline utility to do these things so I could have a master bib and makefile to generate special versions. (pybib goes some way, bibtool might be capable but I couldn't use it).
–
Lev BishopAug 10 '10 at 2:05

I recommend using Norman Ramsey's nbibtex for this. The basic idea is that you run the nbibtex command instead of Bibtex, which uses the argument to \cite as a search query, and goes looking for the best match for that query and puts it in the .bbl file it creates.

Another possibility

Something I do with my clients is replace the \bibliography command with the contents of the *.bbl file. Then, magically, you don't need to run Bibtex on that Latex document ever again. The downside of doing this is that if you find errors in your reference list, or if you add items, that magic won't propagate changes back to your *.bib files.

Breaking up bibliography files into a number of separate *.bib files is A Good Thing, IMO, since it allows you to work with other people on putting together good bibliography files: no one except you will want to work with the contents of AllMyReferences.bib file.

Old, lost code

I wrote some code back in my days as a doctoral student that did the following:

Looked through the *.aux file for Bibtex keys, where I generally followed the convention author:key in citations;

Searched BIBPATH for *.bib files that might match these keys, using the convention to narrow the search where it was used;

Looked in those files either for exact matches for those keys, and also in special alternate-key fields in Bibtex entries, to allow one key to be used

Report problems where there are (i) keys not matching sources, (ii) several sources for some key (you need to check for errors, ideally this code would have been able to tell whether two citations are identical, but it didn't do that), (iii) one source referenced by several keys (each document should be consistent in the way it cites sources)

Create a .bib file from what is found, ensuring that each citation matches exactly one reflist entry, which might mean several occurrences of a single source if it is cited by more than one key, and a .aux key, which cites one key for each source.

Run Bibtex on the .aux/.bib file to create the .bbl file to be used by the \thebibliography program.

I don't think this approach makes much sense now given the availability of nbibtex, which is more usable and more robust, but the code did do more in the way of sanity checks of the Bibtex database, which nbibtex doesn't do with the --permissive option, and nbibtex isn't really useful for multiple bibliographies without it.

“No-one except you will want to work …” – why not? Nothing speaks against putting this file under centralized version control.
–
Konrad RudolphAug 9 '10 at 12:19

thanks for your answer, even if it does not really answer my question. I didn't know nbibtex
–
Caroline FontaineAug 9 '10 at 13:48

@Konrad: I've seen a few cases where repeated coauthors have collaboratively edited a Bibtex file, but I've never seen a large Bibtex file collaboratively edited in this way. I think distributed version control would make more sense for this kind of thing, FWIW. If you have experiences, I'd be happy to hear them.
–
Charles StewartAug 9 '10 at 13:48

@Caroline: To be clear, Bibtex will complain in your use case, as Taco said in a comment. To avoid the error, you need to use something else or change what you are doing. Nbibtex is a something else, Konrad's monolithic file is a do something else.
–
Charles StewartAug 9 '10 at 15:36

I think, you are looking for the package bibtopic. But afaik it's not up to date (2006). I would also recommend to create one BiBTeX file. If you want your bibliography sorted by subject, you could use the multibib-package.

Solving this problem is relatively easy on linux. Create a file in your ~/bin/
directory called bibtex that contains the following:

/usr/bin/bibtex $* | sed -r -e "/Repeated entry/,/I'm skipping/ d"

Do $chmod 755 ~/bin/bibtex to make it executable.

This will then replace the raw bibtex command by filtering its output through
sed, which is told to delete the Repeated entry message (which extends from the line beginning "Repeated entry" to the line beginning "I'm skipping").

bibtool can retain the original keys while changing the identical keys. Thus if two entries are really the same, you can use the original key. But, if two different entries use the same key, you can use the original key (ex: mykey) for the first entry and the modified key (ex: mykey*b) for the second entry.

Comment the repeated bibentry in various bib files, but keep uncommented entry only once. When you want to process them individually, uncomment it to get it listed.

Make the bibkeys unique (Best method). Use different bibkeys for the same reference item in different parts of the main document. Say one has the key refkey in chapters/parts 1, 3, and 5. Then rename the key to refkey1, refkey2, refkey5 in the bib files of the respective part and accordingly update the \cite commands in the text of the respective part. This is just a Search and Replace task.

Adding an answer, after 2 years, as no one has suggested a solution without removing entries from the bibtex files or without merging bibtex files as the OP wanted. It is independent of the OS or LaTeX distribution that one uses.

Having two different keys for the same paper and citing them both will lead to duplicate bibliography entries in my experience. I would highly recommend NOT doing the 2nd bullet point.
–
JoeJan 15 '14 at 23:08