Paranoid reaction. If you are worried about being "on their radar" it's too late. But It isn't really a serious concern. Not If you have a properly kept bound book, and have been doing everything legally.
As far as any audit, that would be only on your C&R collection. Nothing else without the same search warrent that would be needed FFL or not. BATF audits of Class 03 license holders are seldom done unless the BATF suspects something illegal is being done.
Not to say that all this couldn't change, but for now It's not being talked about. Even by the most active gun grabbers. They probably don't even know what a Collector of Curio and Relic FFL is. Let's hope It stays that way.