Selection indices for a view with proxy model

I am looking at implementing a browser view in my application that user can toggle the display mode between tree and list. The underlying data structure has folders and files much like a file system. In the tree view, I would like it display as a hierarchy where the files/folder in a folder are shown as leaves of the folder. In the list view, I only want the files to be visible and it is to be sorted by name followed by their path.

E.g.
Data structure:
A
|-> file01
|-> file02
B
|-> file01
|->file03

In tree view:
A
|-> file01
|-> file02
B
|-> file01
|->file03

In list view:
A/file01
B/file01
A/file02
B/file03

Currently I have what seems like a working implementation where toggling between the 2 modes works correctly. I have implemented it via a single instance of a derived class of TreeView and a derived class of QIdentityProxyModel as the proxy to my underlying model. In the various virtual functions for the QAbstractProxyModel, I have a switch that if we are in tree view mode, let it fall back to the QIdentityProxyModel and if we are in list view mode, use some custom implementation for flat listing.

What I am not certain is should the QItemSelectionModel be based off index from the proxy model or its source model? Also, QIdentityProxyModel seems to have connection to rowsAboutToBeRemoved and other signals and the problem is it only maps the parent index but not the start and end. Should I disconnect from those slots if I want the rowsAboutToBeRemoved emitted by my proxy model to be mapped correctly to my proxy space?

Or is this implementation not the way Qt intended proxy model to be used?