You could use Ensembl's VEP to annotate all your filtered variants, then filter based off that. Alternatively, you could try Gemini, which would allow you to make complex and flexible queries around large sets of data (based on SQLite)

I would highly recommend GEMINI (which you can use on VEP or SnpEff annotated VCF files). It will let you do family-wise queries and lots of different ways of slicing the data. But even for the simple case you describe I think you'll likely find all of the additional capabilities appealing.