Memory error accessing grayscale pixel data

Somewhat of a simple question, but haven't found the answer from searching around. I'm trying to work with direct pixel data, and am having trouble figuring out what determines the size of the mat so that I can determine my indexes. I set my capture to be 640x400, but attempting to access any pixel with an x-coordinate larger than 365 throws an exception for access violation. Resizing the mat seems to have no effect. Am I missing something simple here? My full code is below: