Do you store the XMMATRIX in an object which is dynamically allocated? This will give you alignment issues. Try to use a XMFLOAT4X4 for storing. You can convert between those types with XMStoreFloat4x4 and XMLoadFloat4x4.