Next time you go, bring a thermometer and check the water temp in the stream. Maybe it gets too warm in the summer and the trout slow down hole up at cooler inlets etc. Might need to leave them be until things cool down in the Fall.