I was planning to back up my Bitcoin wallet (private key) along with
the rest of my system backups which are versioned by rdiff-backup.
However, it occurred to me that if the password with which my wallet
is encrypted is deemed non-secure at some point and I change that
password, the rdiff-backup repository will still contain the private
key encrypted with the non-secure password.
Am I thinking this through correctly? If so, can I delete all
versions of a particular file from an rdiff-backup repository?

Your thinking is correct. However if the following criteria are met,
you can tell rdiff-backup to trim older increments and it will only
affect your bitcoin wallet file:

- The bitcoin wallet file must be excluded from all other backups.

- You setup a dedicated rdiff-backup destination directory that only
backs up the bitcoin wallet.

In rdiff-backup, the target directory is an all-or-nothing affair when
it comes time to age out old increments. You can't tell rdiff-backup to
only age out parts of a target directory.

When setting up rdiff-backup, a good idea is to think about how long you
want to keep a particular set of files. If you find that some file sets
need different retention times then other file sets, then those file
sets should be placed into a separate rdiff-backup target directory and
backed up with a separate rdiff-backup command.

At a minimum, I suggest:

- One rdiff-backup job for the main file system, but which excludes all
user-created data.

- One for /etc and one for /usr/local, with a very long retention
period. That's only if you don't use a tool like FSVS or some other
version control tool on your /etc and /usr/local directories.

- One for /home, maybe even separate backup jobs for each user directory
under /home.

- One job for each type of user-created data. For instance, we store
our SVN repository backups separate from everything else. And our Samba
file shares are broken out to their own rdiff-backup target directories
as well. The PostgreSQL daily database dumps get written to a separate
rdiff-backup target directory.