We have had OpenBSD running for a couple of months now. Recently though, ever few weeks our OpenBSD server (which is running 4.9 which we use as a firewall/VPN server), it stops pushing internet traffic to our clients on the network. We can ping 8.8.8.8 and resolve google.com just fine but when it comes to actually browsing there, it just doesn't do anything. Oddly enough a reboot fixes it.

What logs should I be looking at to try and track down what might be causing this?

It "sounds" from your very brief description that the network is functioning, but domain resolution is not. I don't know if you are using BIND on OpenBSD, but there was a reliability fix published for 4.9 in December.

For your awareness, official 4.9 support will cease in less than two weeks, when 5.1 is released.