threads / discuss / 46437

git gc seems to break --symbolic-full-name

Subject: git gc seems to break --symbolic-full-name

## tl;dr

12 messages between Jul 21, 2017 and Jul 26, 2017.

replies: 11people: 4as markdown or json

Stas Sergeev· Jul 21, 2017, 17:40 UTC · lore
Hello.
I do the following:

$ git rev-parse --symbolic-full-name devel refs/heads/devel $ ls -l .git/`git rev-parse --symbolic-full-name devel` -rw-rw-r-- 1 stas stas 41 июл 21 15:05 .git/refs/heads/devel

This is fine. But after git gc:

$ git rev-parse --symbolic-full-name devel refs/heads/devel $ LC_ALL=C ls -l .git/`git rev-parse --symbolic-full-name devel` ls: cannot access '.git/refs/heads/devel': No such file or directory

This is because after git gc there is nothing in .git/refs/heads:

$ LC_ALL=C ls -l .git/refs/heads/ total 8 drwxrwxr-x 2 stas stas 4096 Jul 31 2013 feature drwxrwxr-x 2 stas stas 4096 Dec 23 2016 pr

I prepend the path with .git above just as an example: the same happens with rev-parse --git-path.

So I wonder if it is an expected behaviour for --symbolic-full-name to return the invalid pathes? If so, how can I get the path to the needed head file? I use that for dependency tracking.

Junio C Hamano· Jul 21, 2017, 18:56 UTC · re: Stas Sergeev · lore

Re: git gc seems to break --symbolic-full-name

Stas Sergeev <stsp@list.ru> writes:
Show 13 quoted lines
> I do the following:
>
> $ git rev-parse --symbolic-full-name devel
> refs/heads/devel
> $ ls -l .git/`git rev-parse --symbolic-full-name devel`
> -rw-rw-r-- 1 stas stas 41 июл 21 15:05 .git/refs/heads/devel
>
> This is fine. But after git gc:
>
> $ git rev-parse --symbolic-full-name devel
> refs/heads/devel
> $ LC_ALL=C ls -l .git/`git rev-parse --symbolic-full-name devel`
> ls: cannot access '.git/refs/heads/devel': No such file or directory

This is expected, and in the modern world (like, after year 2007), a refname "refs/heads/foo" does *not* mean that there is a file with such a path under .git/ directory. "rev-parse" does not return any "path" in the filesystem sense.

See "git pack-refs --help", and depending on what you want to learn about the ref in question, perhaps "git show-ref refs/heads/devel" is what you want.

Stas Sergeev· Jul 21, 2017, 19:03 UTC · re: Junio C Hamano · lore

Re: git gc seems to break --symbolic-full-name

21.07.2017 21:56, Junio C Hamano пишет:
Show 23 quoted lines
> Stas Sergeev <stsp@list.ru> writes:
>
>> I do the following:
>>
>> $ git rev-parse --symbolic-full-name devel
>> refs/heads/devel
>> $ ls -l .git/`git rev-parse --symbolic-full-name devel`
>> -rw-rw-r-- 1 stas stas 41 июл 21 15:05 .git/refs/heads/devel
>>
>> This is fine. But after git gc:
>>
>> $ git rev-parse --symbolic-full-name devel
>> refs/heads/devel
>> $ LC_ALL=C ls -l .git/`git rev-parse --symbolic-full-name devel`
>> ls: cannot access '.git/refs/heads/devel': No such file or directory
> This is expected, and in the modern world (like, after year 2007),
> a refname "refs/heads/foo" does *not* mean that there is a file
> with such a path under .git/ directory.  "rev-parse" does not return
> any "path" in the filesystem sense.
>
> See "git pack-refs --help", and depending on what you want to learn
> about the ref in question, perhaps "git show-ref refs/heads/devel"
> is what you want.

I wanted some kind of file to use it as a build dependency for the files that needs to be re-built when the head changes. This works very well besides git gc. What other method can be used as simply as that? git show-ref does not seem to be giving this.

Jacob Keller· Jul 23, 2017, 08:40 UTC · re: Stas Sergeev · lore

Re: git gc seems to break --symbolic-full-name

On Fri, Jul 21, 2017 at 12:03 PM, Stas Sergeev <stsp@list.ru> wrote:
Show 7 quoted lines
> I wanted some kind of file to use it as a
> build dependency for the files that needs
> to be re-built when the head changes.
> This works very well besides git gc.
> What other method can be used as simply
> as that? git show-ref does not seem to be
> giving this.

There's no real way to do this, and even prior to 2007 when the file always existed, there's no guarantee it's modification time is valid.

I'd suggest you have a phony rule which you always run, that checks the ref, and sees if it's different from "last time" and then updates a different file if that's the case. Then the build can depend on the generated file, and you'd be able to figure it out.

What's the real goal for depending on when the ref changes?

Thanks, Jake

Stas Sergeev· Jul 23, 2017, 19:23 UTC · re: Jacob Keller · lore

Re: git gc seems to break --symbolic-full-name

23.07.2017 11:40, Jacob Keller пишет:
Show 15 quoted lines
> On Fri, Jul 21, 2017 at 12:03 PM, Stas Sergeev <stsp@list.ru> wrote:
>> I wanted some kind of file to use it as a
>> build dependency for the files that needs
>> to be re-built when the head changes.
>> This works very well besides git gc.
>> What other method can be used as simply
>> as that? git show-ref does not seem to be
>> giving this.
> There's no real way to do this, and even prior to 2007 when the file
> always existed, there's no guarantee it's modification time is valid.
>
> I'd suggest you have a phony rule which you always run, that checks
> the ref, and sees if it's different from "last time" and then updates
> a different file if that's the case. Then the build can depend on the
> generated file, and you'd be able to figure it out.

OK, thanks, that looks quite simple too. I will have to create the file by hands that I expected git to already have, but it appears not.

> What's the real goal for depending on when the ref changes?

So that when users fill in the bug report, I can see at what revision have the bug happened. :) While seemingly "just a debugging sugar", the hard experience shows this to be exceptionally useful. I think even linux kernel does something like this, and solves that task the hard way. For example I can see a script at scripts/setlocalversion whose output seems to go to include/config/kernel.release and a lot of logic in the toplevel makefile about this. So not liking the fact that every project solves this differently, I was trying to get the solution directly from git. But I'll try otherwise.

Jacob Keller· Jul 24, 2017, 04:02 UTC · re: Stas Sergeev · lore

Re: git gc seems to break --symbolic-full-name

On Sun, Jul 23, 2017 at 12:23 PM, Stas Sergeev <stsp@list.ru> wrote:
Show 41 quoted lines
> 23.07.2017 11:40, Jacob Keller пишет:
>>
>> On Fri, Jul 21, 2017 at 12:03 PM, Stas Sergeev <stsp@list.ru> wrote:
>>>
>>> I wanted some kind of file to use it as a
>>> build dependency for the files that needs
>>> to be re-built when the head changes.
>>> This works very well besides git gc.
>>> What other method can be used as simply
>>> as that? git show-ref does not seem to be
>>> giving this.
>>
>> There's no real way to do this, and even prior to 2007 when the file
>> always existed, there's no guarantee it's modification time is valid.
>>
>> I'd suggest you have a phony rule which you always run, that checks
>> the ref, and sees if it's different from "last time" and then updates
>> a different file if that's the case. Then the build can depend on the
>> generated file, and you'd be able to figure it out.
>
> OK, thanks, that looks quite simple too.
> I will have to create the file by hands that
> I expected git to already have, but it appears
> not.
>
>> What's the real goal for depending on when the ref changes?
>
> So that when users fill in the bug report, I can
> see at what revision have the bug happened. :)
> While seemingly "just a debugging sugar", the
> hard experience shows this to be exceptionally
> useful.
> I think even linux kernel does something like
> this, and solves that task the hard way. For
> example I can see a script at scripts/setlocalversion
> whose output seems to go to
> include/config/kernel.release and a lot of
> logic in the toplevel makefile about this.
> So not liking the fact that every project solves
> this differently, I was trying to get the solution
> directly from git. But I'll try otherwise.

generally, I'd suggest using "git describe" to output a version based on tag, and as part of your build system set that in some sort of --version output of some kind.

Thanks, Jake

Stas Sergeev· Jul 24, 2017, 12:50 UTC · re: Jacob Keller · lore

Re: git gc seems to break --symbolic-full-name

24.07.2017 07:02, Jacob Keller пишет:
> generally, I'd suggest using "git describe" to output a version based
> on tag, and as part of your build system set that in some sort of
> --version output of some kind.

I am not sure I understand that suggestion. I can use 'git describe' (and that seems to be what linux kernel does too), but I don't see how can I get away without a temporary file, maually comparing current 'git describe' with the one stored, and updating that file in case of a mismatch. Could you please clarify? Maybe I am missing some makefile trick which you assume I am aware about. :)

Stas Sergeev· Jul 26, 2017, 00:21 UTC · re: Jacob Keller · lore

Re: git gc seems to break --symbolic-full-name

24.07.2017 07:02, Jacob Keller пишет:
> generally, I'd suggest using "git describe" to output a version based
> on tag, and as part of your build system set that in some sort of
> --version output of some kind.

I came to the following solution which looks quite simple (avoids comparing the output of git describe): git log -1 --format=%cd --date=rfc | xargs -I {} touch --date={} $TSTAMP

The care must be taken to put the timestamp file as a pre-requisite of the .LOW_RESOLUTION_TIME target, and the build seems to work properly. This still smells hackish, but this time I blame make for an inability to specify the timestamps explicitly. :)

Jacob Keller· Jul 26, 2017, 00:36 UTC · re: Stas Sergeev · lore

Re: git gc seems to break --symbolic-full-name

On Sun, Jul 23, 2017 at 12:23 PM, Stas Sergeev <stsp@list.ru> wrote:
Show 41 quoted lines
> 23.07.2017 11:40, Jacob Keller пишет:
>>
>> On Fri, Jul 21, 2017 at 12:03 PM, Stas Sergeev <stsp@list.ru> wrote:
>>>
>>> I wanted some kind of file to use it as a
>>> build dependency for the files that needs
>>> to be re-built when the head changes.
>>> This works very well besides git gc.
>>> What other method can be used as simply
>>> as that? git show-ref does not seem to be
>>> giving this.
>>
>> There's no real way to do this, and even prior to 2007 when the file
>> always existed, there's no guarantee it's modification time is valid.
>>
>> I'd suggest you have a phony rule which you always run, that checks
>> the ref, and sees if it's different from "last time" and then updates
>> a different file if that's the case. Then the build can depend on the
>> generated file, and you'd be able to figure it out.
>
> OK, thanks, that looks quite simple too.
> I will have to create the file by hands that
> I expected git to already have, but it appears
> not.
>
>> What's the real goal for depending on when the ref changes?
>
> So that when users fill in the bug report, I can
> see at what revision have the bug happened. :)
> While seemingly "just a debugging sugar", the
> hard experience shows this to be exceptionally
> useful.
> I think even linux kernel does something like
> this, and solves that task the hard way. For
> example I can see a script at scripts/setlocalversion
> whose output seems to go to
> include/config/kernel.release and a lot of
> logic in the toplevel makefile about this.
> So not liking the fact that every project solves
> this differently, I was trying to get the solution
> directly from git. But I'll try otherwise.

If your goal is to make it so users filling out bug reports have a version, then using git descrsibe and making that be part of your version (based off your tags, and commits) is how pretty much every other project I've worked on does this.

I really don't think that's your goal here, given you're doing things in make with timestamps and builds, so I guess I misunderstood your answer?

Thanks, Jake

Stas Sergeev· Jul 26, 2017, 08:40 UTC · re: Jacob Keller · lore

Re: git gc seems to break --symbolic-full-name

26.07.2017 03:36, Jacob Keller пишет:
Show 8 quoted lines
> If your goal is to make it so users filling out bug reports have a
> version, then using git descrsibe and making that be part of your
> version (based off your tags, and commits) is how pretty much every
> other project I've worked on does this.
>
> I really don't think that's your goal here, given you're doing things
> in make with timestamps and builds, so I guess I misunderstood your
> answer?
There are 2 different things:
1. put git describe output into some header file
2. make things to rebuild with every new commit

So I actually stuck at solving 2, because 1 is trivial. My original solution for 2 was to "depend" on refs/heads/*. This worked besides git gc, but had a lot of troubles with worktrees. And this time I switched to the "touch tmpfile" trick with the date taken from git log. This requires .LOW_RESOLUTION_TIME in makefile, so probably not the best solution again, but should hopefully be more future-proof than the previous one.

Jeff King· Jul 26, 2017, 13:23 UTC · re: Stas Sergeev · lore

Re: git gc seems to break --symbolic-full-name

On Wed, Jul 26, 2017 at 11:40:46AM +0300, Stas Sergeev wrote:
Show 22 quoted lines
> 26.07.2017 03:36, Jacob Keller пишет:
> > If your goal is to make it so users filling out bug reports have a
> > version, then using git descrsibe and making that be part of your
> > version (based off your tags, and commits) is how pretty much every
> > other project I've worked on does this.
> > 
> > I really don't think that's your goal here, given you're doing things
> > in make with timestamps and builds, so I guess I misunderstood your
> > answer?
> There are 2 different things:
> 1. put git describe output into some header file
> 2. make things to rebuild with every new commit
> 
> So I actually stuck at solving 2, because 1 is trivial.
> My original solution for 2 was to "depend" on
> refs/heads/*. This worked besides git gc, but had
> a lot of troubles with worktrees. And this time I
> switched to the "touch tmpfile" trick with the date
> taken from git log. This requires .LOW_RESOLUTION_TIME
> in makefile, so probably not the best solution again,
> but should hopefully be more future-proof than the
> previous one.
In git.git we do something like:
-- >8 --
other: version
	cat $< >$@
.PHONY: FORCE
version: FORCE
	@git rev-parse HEAD >$@+
	@if cmp $@+ $@ >/dev/null 2>&1; then rm $@+; else mv $@+ $@; fi
-- >8 --

The "version" commands run always, but they only update the file if there's a change. At least GNU make is smart enough to know that if "version" was not updated, then "other" does not need to be re-built. I don't know if all makes would so, though.

-Peff
Stas Sergeev· Jul 26, 2017, 14:08 UTC · re: Jeff King · lore

Re: git gc seems to break --symbolic-full-name

26.07.2017 16:23, Jeff King пишет:
Show 11 quoted lines
> In git.git we do something like:
>
> -- >8 --
> other: version
> 	cat $< >$@
>
> .PHONY: FORCE
> version: FORCE
> 	@git rev-parse HEAD >$@+
> 	@if cmp $@+ $@ >/dev/null 2>&1; then rm $@+; else mv $@+ $@; fi
> -- >8 --

Yes, thats a nice recipe that I would be using if not for the fact that I already switched to "touch", which requires 1 fewer tmp file and no comparison. But I'll keep this in mind if something wrong happens with my current solution, thanks! I wonder if git can provide some helper script for other projects to solve this in a same way.

← back to recent threads