Who can Access VPN

I was wondering can you limit on who can access VPN. if you have 2 interfaces for example inside and outside enabled for IPsec can I limit 1 tunnel only to be access from inside insted of been available on both interfaces