recently, in our cephfs (ceph-fuse client) online production environment, i found several rmdir fail due to not empty directory. after further dig into, the problem is one client can not list the filescreated by other client.

after analysing the log , here is my step to reproduce the problem, the root cause of the problemis also attached below as well as my proposes fix pull request.

reproduce steps (ceph-fuse mount) :a. there are two clients: client1, client2. first mount client1 under /mnt/peng1/, which is emptyb. run 'mkdir -p /mnt/peng1/yunfei/god/' under client1c. mount client2 under path /mnt/peng2d. run 'touch /mnt/peng2/yunfei/god' through client2, and the run 'stat /mnt/peng2/yunfei/god' under the same client2e. gdb attach client2 ceph-fuse daemon, let mds find out 'new stale session' for client2. after mds log print "new stale session", and gdb continue and restore client2 sessionf. run my own written c program (listed below), which will create a new file "pengsynctest" under /mnt/peng1/yunfei/god/ through client1, and lookup "/mnt/peng2/yunfei/god" through client2.g. finally , you will never list out the "pengsynctest" under /mnt/peng2/yunfei/god through client2

according to the above reproduce step, root cause is :after step-d, client2 will bring back pAsLsXsFs for dir "god". in step-2, during client2 session stale,in the client2 side, cap for "god" still pAsLsXsFs, but not valid. in the mds side, thecorresponding stale cap for client2 will be revoked from pAsLsXsFs to only P.however, after the stale session restored, after the step-f 's create "god/pengsynctest" fromclient1 (early reply, journal has not been persisted), the following step-f's client2 lookup'god/pengsynctest' will bring back pAsLsXs and 0 numfiles under "god" (since client1's create op write journal has not been completed ).finally, during, client2 handle the lookup "god" reply, in its add_update_cap(), it merge the reply issuedcap (pAsLsXs) with the invalid, out-dataed pFsLsXsFs and finally issue client2 pFsLsXsFs by mistake.