Caching, probably. Once you've queried model->index(5000, 0)->data(), all further queries will only access an internal cache for the 5000th row and thus be much faster.
You can verify this by timing each call with a high performance timer (~microseconds). If my guess is correct, the first call to data() (or index()) will be orders of magnitude slower than the subsequent.

If you said it takes 5 seconds, and you have 13000 rows, that means one query takes 0.3ms. that's not too bad. I don't know how QSqlQueryModel works internally, whether it actually queries the database when accessing indexes or only reads the internal model which was built with setQuery. If it actually queries the database everytime, then 0.3ms is actually quite good. Either way, if you want more performance, you'll need to get that data inside an intermediary cache that's specifically built to perform the accesses you want with high performance. You could either build that ontop of QSqlQueryModel or as a replacement. This might become alot of work, so make sure you have no other way around this, i.e. improved program logic that you don't even need to access 13000 indexes instantly.