Re: [BUG] git clean not working on FreeBSD/NFS

Henrik Grubbström <grubba <at> grubba.org> writes:

No, these are all the files that remain in libclamav/.deps (ls -al): edwin@gcc101:~/buildbot/gcc4.2/build$ ls -al libclamav/.deps/ total 116 drwxr-xr-x 2 edwin edwin 8192 Jun 18 03:24 . drwx------ 8 edwin edwin 8192 Jun 18 03:24 .. -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-textnorm.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-tnef.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-unarj.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-uniq.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-unsp.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-untar.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-unzip.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-upack.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-upx.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-uuencode.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-vba_extract.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-version.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-wwunpack.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 libclamav_la-yc.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrar.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrar15.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrar20.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrar_iface.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrarcmd.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrarfilter.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrarhlp.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrarppm.Plo -rw-r--r-- 1 edwin edwin 8 Jun 18 03:23 unrarvm.Plo

If I run 'git clean -x -f -d -- libclamav' a few more times they are eventually removed.

It is interesting that every time I repeat the config.status/git clean steps I get the *same* files remaining in the directory.

There were 127 files initially, these 23 always remain behind on 1st run, a 2nd git clean removes them (but some other .deps directories need even more reruns of git clean).

I added a printf in remove_dir_recursively, and those files are not even printed.

Now I see the bug!
You do a single readdir() pass, and unlink() files. That is wrong for 2 reasons:
 - new files may be added in the meantime and you have a race condition
 - unlinking a file can cause order of files to be rearranged, moved. But your
readdir() cursors stays where it was before, hence you will miss enumerating
some files that are there.

The fix is to rewinddir() if you unlinked anything (won't fix the race condition but that can't be eliminated completely anyway).

I'm sending a patch as reply to this mail.