Re: [Btrfs-devel] btrfs and git-reflog

From: Chris Mason <chris.mason@oracle.com>
Date: Jan 25, 2008, 15:50 UTC
Message-ID: <200801251050.16697.chris.mason@oracle.com>
In-Reply-To: <873asmcodd.fsf@burly.wgtn.ondioline.org>

On Friday 25 January 2008, Paul Collins wrote:

Show 26 quoted lines

> I was just playing with git 1.5.3.8 and btrfs 0.11, and I noticed
> something odd.
>
> If I prepare a very simple repository:
>
>   $ mkdir foo
>   $ cd foo
>   $ git init
>   Initialized empty Git repository in .git/
>   $ echo hi > blort
>   $ git add .
>   $ git commit -m create
>   Created initial commit 4ae9415: create
>    1 files changed, 1 insertions(+), 0 deletions(-)
>    create mode 100644 blort
>
> and then attempt to expire the reflogs
>
>   $ git-reflog expire --all
>
> on ext3, git-reflog completes its work and exits immediately;
>
> and on btrfs, it gets stuck in some sort of loop that causes it to
> allocate more and more memory until I kill it or it pushes the
> machine into OOM.
>

It works something like this:

readdir(.git/logs/refs/heads) # this returns .git/logs/refs/heads/master # <do some work>

open(.git/logs/refs/heads/master.lock, O_CREAT); # <do more work>, write to master.lock

rename(master.lock, master)

readdir(.git/logs/refs/heads)

readdir again returns .git/logs/refs/heads/master, which is arguably correct. It is a new file that just happens to have a name that git already saw. So, git loops over this file infinitely because it doesn't realize it has already processed it.

This happens because btrfs doesn't return the hash of the file name as the offset to readdir. It returns the inode number, and since master is a new file, btrfs considers it a non-duplicate entry.

The btrfs patch below changes my readdir code to force the directory f_pos field to the max offset allowed when we've seen all the directory entries. This prevents the readdir call from looping forever in the face of newly added files.

But, git might want to add some checks to see if it has already processed things.

diff -r 21e9b461f802 inode.c

--- a/inode.c	Thu Jan 24 16:13:14 2008 -0500
+++ b/inode.c	Fri Jan 25 10:28:49 2008 -0500
@@ -1430,7 +1431,7 @@ read_dir_items:
 			di = (struct btrfs_dir_item *)((char *)di + di_len);
 		}
 	}
-	filp->f_pos++;
+	filp->f_pos = INT_LIMIT(typeof(filp->f_pos));
 nopos:
 	ret = 0;
 err: