threads / bug / 50220

[Possible Bug] Commit generates GC repack failure

Subject: [Possible Bug] Commit generates GC repack failure

## tl;dr

5 messages between Jan 14, 2019 and Jan 14, 2019.

replies: 4people: 3as markdown or json

Randall S. Becker· Jan 14, 2019, 14:50 UTC · lore
Hi All,

I'm trying to track down what happened this morning. We had a commit that caused a background gc to occur. What happened was:

$ git commit -m "history commit ... " Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping. warning: The last gc run reported the following. Please correct the root cause and remove .git/gc.log. Automatic cleanup will not be performed until the file is removed.

fatal: open /dev/null failed: Invalid function argument fatal: failed to run repack

Obviously the /dev/null open was not right, but I don't know where this is in the git code to investigate further. Any pointers on where to look would be appreciated. This started happening at 2.20.1 on the NonStop platform.

Cheers, Randall

-- Brief whoami:
 NonStop developer since approximately 211288444200000000
 UNIX developer since approximately 421664400
-- In my real life, I talk too much.
Duy Nguyen· Jan 14, 2019, 15:03 UTC · re: Randall S. Becker · lore

Re: [Possible Bug] Commit generates GC repack failure

On Mon, Jan 14, 2019 at 9:51 PM Randall S. Becker <rsbecker@nexbridge.com> wrote:

Show 15 quoted lines
>
> Hi All,
>
> I'm trying to track down what happened this morning. We had a commit that
> caused a background gc to occur. What happened was:
>
> $ git commit -m "history commit ... "
> Auto packing the repository in background for optimum performance.
> See "git help gc" for manual housekeeping.
> warning: The last gc run reported the following. Please correct the root
> cause
> and remove .git/gc.log.
> Automatic cleanup will not be performed until the file is removed.
>
> fatal: open /dev/null failed: Invalid function argument

sanitize_stdfds() in setup.c can attempt to open /dev/null then redirect stdout and stderr to it. I think this is part of the daemonization that is done in background gc.

Show 17 quoted lines
> fatal: failed to run repack
>
> Obviously the /dev/null open was not right, but I don't know where this is
> in the git code to investigate further. Any pointers on where to look would
> be appreciated. This started happening at  2.20.1 on the NonStop platform.
>
> Cheers,
> Randall
>
> -- Brief whoami:
>  NonStop developer since approximately 211288444200000000
>  UNIX developer since approximately 421664400
> -- In my real life, I talk too much.
>
>
>
>
-- 
Duy
Duy Nguyen· Jan 14, 2019, 15:07 UTC · re: Duy Nguyen · lore

Re: [Possible Bug] Commit generates GC repack failure

On Mon, Jan 14, 2019 at 10:03 PM Duy Nguyen <pclouds@gmail.com> wrote:
Show 22 quoted lines
>
> On Mon, Jan 14, 2019 at 9:51 PM Randall S. Becker
> <rsbecker@nexbridge.com> wrote:
> >
> > Hi All,
> >
> > I'm trying to track down what happened this morning. We had a commit that
> > caused a background gc to occur. What happened was:
> >
> > $ git commit -m "history commit ... "
> > Auto packing the repository in background for optimum performance.
> > See "git help gc" for manual housekeeping.
> > warning: The last gc run reported the following. Please correct the root
> > cause
> > and remove .git/gc.log.
> > Automatic cleanup will not be performed until the file is removed.
> >
> > fatal: open /dev/null failed: Invalid function argument
>
> sanitize_stdfds() in setup.c can attempt to open /dev/null then
> redirect stdout and stderr to it. I think this is part of the
> daemonization that is done in background gc.

However the message does not match. I think you will find this "open /dev/null failed" line in run-command.c

-- 
Duy
Ævar Arnfjörð Bjarmason· Jan 14, 2019, 15:11 UTC · re: Randall S. Becker · lore

Re: [Possible Bug] Commit generates GC repack failure

On Mon, Jan 14 2019, Randall S. Becker wrote:
Show 19 quoted lines
> Hi All,
>
> I'm trying to track down what happened this morning. We had a commit that
> caused a background gc to occur. What happened was:
>
> $ git commit -m "history commit ... "
> Auto packing the repository in background for optimum performance.
> See "git help gc" for manual housekeeping.
> warning: The last gc run reported the following. Please correct the root
> cause
> and remove .git/gc.log.
> Automatic cleanup will not be performed until the file is removed.
>
> fatal: open /dev/null failed: Invalid function argument
> fatal: failed to run repack
>
> Obviously the /dev/null open was not right, but I don't know where this is
> in the git code to investigate further. Any pointers on where to look would
> be appreciated. This started happening at  2.20.1 on the NonStop platform.

We use start_command() when gc does run_command_v_opt() of e.g. git-repack. See in that file:

    null_fd = open("/dev/null", O_RDWR | O_CLOEXEC)
Maybe that sort of code just fails on NonStop?
Randall S. Becker· Jan 14, 2019, 16:09 UTC · re: Ævar Arnfjörð Bjarmason · lore

RE: [Possible Bug] Commit generates GC repack failure

On January 14, 2019 10:12, Ævar Arnfjörð Bjarmason wrote:
Show 28 quoted lines
> On Mon, Jan 14 2019, Randall S. Becker wrote:
> 
> > Hi All,
> >
> > I'm trying to track down what happened this morning. We had a commit
> > that caused a background gc to occur. What happened was:
> >
> > $ git commit -m "history commit ... "
> > Auto packing the repository in background for optimum performance.
> > See "git help gc" for manual housekeeping.
> > warning: The last gc run reported the following. Please correct the
> > root cause and remove .git/gc.log.
> > Automatic cleanup will not be performed until the file is removed.
> >
> > fatal: open /dev/null failed: Invalid function argument
> > fatal: failed to run repack
> >
> > Obviously the /dev/null open was not right, but I don't know where
> > this is in the git code to investigate further. Any pointers on where
> > to look would be appreciated. This started happening at  2.20.1 on the
> NonStop platform.
> 
> We use start_command() when gc does run_command_v_opt() of e.g. git-
> repack. See in that file:
> 
>     null_fd = open("/dev/null", O_RDWR | O_CLOEXEC)
> 
> Maybe that sort of code just fails on NonStop?
That line of code works fine on the platform. I tested it in isolation at various OS levels. Let me go back and say that I'm not 100% sure this is new, but may have been there since just before 2.16. Still perplexed recognizing that that is the only point in git code where that can happen.

Cheers, Randall

← back to recent threads