threads / bug / 47555

Bug Report: Subtrees and GPG Signed Commits

Subject: Bug Report: Subtrees and GPG Signed Commits

## tl;dr

12 messages between Jan 6, 2018 and Feb 8, 2018.

replies: 11people: 4as markdown or json

Stephen R Guglielmo· Jan 6, 2018, 22:45 UTC · lore
Hi all,

I've noticed an issue regarding the use of `git subtree add` and `git subtree pull` when the subtree repository's commit (either HEAD or whatever commit specified by the subtree command) is signed with GPG. It seems to work properly if the commit is not signed but previous commits are.

The gist of the issue is that `git subtree add` does not add the subree properly and a "fatal: Not a valid object name" error is thrown. Running `git subtree pull` does not pull any upstream changes after that ("'subtree' was never added").

I have not done extensive testing, however, below are instructions to reproduce the issue. This was tested using git version 2.15.1 installed via Homebrew on MacOS. I did not test with the built-in version of git on MacOS.

Thanks, Steve

# Create a new repository mkdir repoA && cd repoA git init echo "Test File in Repo A" > FileA git add -A && git commit -m 'Initial commit in repo A'

# Create a second repository cd .. && mkdir repoB && cd repoB git init echo "Test File in Repo B" > FileB git add -A && git commit -m 'Initial commit in repo B'

# Create a signed commit in repo B echo "Signed Commit" >> FileB git commit -a -S -m 'Signed commit in repo B'

# Now, add repoB as a subtree of RepoA
cd ../repoA
git subtree add --prefix repoB_subtree/ ../repoB/ master --squash
# Output:
git fetch ../repoB/ master
warning: no common commits
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From ../repoB
 * branch            master     -> FETCH_HEAD
fatal: Not a valid object name gpg: Signature made Sat Jan  6 17:38:31 2018 EST
gpg:                using RSA key 6900E9CFDD39B6A741D601F50999759F2DCF3E7C
gpg: Good signature from "Stephen Robert Guglielmo (Temple University
Computer Services) <srg@temple.edu>" [ultimate]
Primary key fingerprint: 6900 E9CF DD39 B6A7 41D6  01F5 0999 759F 2DCF 3E7C
4b700b1a4ebb9e2c1011aafd6b0f720b38f059a4
# Note, git exits with status 128 at this point.

# FileB was in fact added and staged to repoA, despite the "fatal" above. Commit it: git commit -m 'Add repoB subtree'

# Ok, let's make another commit in repoB and try a `subtree pull`
instead of `subtree add`
cd ../repoB
echo "Another Line" >> FileB
git commit -a -S -m 'Another signed commit'
cd ../repoA
git subtree pull --prefix repoB_subtree/ ../repoB master --squash
# Output:
warning: no common commits
remote: Counting objects: 9, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 9 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (9/9), done.
From ../repoB
 * branch            master     -> FETCH_HEAD
Can't squash-merge: 'repoB_subtree' was never added.
# Note, git exits with status 1 at this point.

# RepoB's third commit ('Another signed commit') is not pulled into the subree in repo A. # This can be verified by running a diff: diff -qr --exclude ".git" repoB_subtree ../repoB # Output: Files repoB_subtree/FileB and ../repoB/FileB differ

Stephen R Guglielmo· Jan 18, 2018, 16:19 UTC · re: Stephen R Guglielmo · lore

Re: Bug Report: Subtrees and GPG Signed Commits

Hi, just following up on this bug report. I have not heard back. Is there additional information that's needed? Is there a better place to file bug reports?

Thanks

On Sat, Jan 6, 2018 at 5:45 PM, Stephen R Guglielmo <srguglielmo@gmail.com> wrote:

Show 85 quoted lines
> Hi all,
>
> I've noticed an issue regarding the use of `git subtree add` and `git
> subtree pull` when the subtree repository's commit (either HEAD or
> whatever commit specified by the subtree command) is signed with GPG.
> It seems to work properly if the commit is not signed but previous
> commits are.
>
> The gist of the issue is that `git subtree add` does not add the
> subree properly and a "fatal: Not a valid object name" error is
> thrown. Running `git subtree pull` does not pull any upstream changes
> after that ("'subtree' was never added").
>
> I have not done extensive testing, however, below are instructions to
> reproduce the issue. This was tested using git version 2.15.1
> installed via Homebrew on MacOS. I did not test with the built-in
> version of git on MacOS.
>
> Thanks,
> Steve
>
> # Create a new repository
> mkdir repoA && cd repoA
> git init
> echo "Test File in Repo A" > FileA
> git add -A && git commit -m 'Initial commit in repo A'
>
> # Create a second repository
> cd .. && mkdir repoB && cd repoB
> git init
> echo "Test File in Repo B" > FileB
> git add -A && git commit -m 'Initial commit in repo B'
>
> # Create a signed commit in repo B
> echo "Signed Commit" >> FileB
> git commit -a -S  -m 'Signed commit in repo B'
>
> # Now, add repoB as a subtree of RepoA
> cd ../repoA
> git subtree add --prefix repoB_subtree/ ../repoB/ master --squash
> # Output:
> git fetch ../repoB/ master
> warning: no common commits
> remote: Counting objects: 6, done.
> remote: Compressing objects: 100% (2/2), done.
> remote: Total 6 (delta 0), reused 0 (delta 0)
> Unpacking objects: 100% (6/6), done.
> From ../repoB
>  * branch            master     -> FETCH_HEAD
> fatal: Not a valid object name gpg: Signature made Sat Jan  6 17:38:31 2018 EST
> gpg:                using RSA key 6900E9CFDD39B6A741D601F50999759F2DCF3E7C
> gpg: Good signature from "Stephen Robert Guglielmo (Temple University
> Computer Services) <srg@temple.edu>" [ultimate]
> Primary key fingerprint: 6900 E9CF DD39 B6A7 41D6  01F5 0999 759F 2DCF 3E7C
> 4b700b1a4ebb9e2c1011aafd6b0f720b38f059a4
> # Note, git exits with status 128 at this point.
>
> # FileB was in fact added and staged to repoA, despite the "fatal"
> above. Commit it:
> git commit -m 'Add repoB subtree'
>
> # Ok, let's make another commit in repoB and try a `subtree pull`
> instead of `subtree add`
> cd ../repoB
> echo "Another Line" >> FileB
> git commit -a -S -m 'Another signed commit'
> cd ../repoA
> git subtree pull --prefix repoB_subtree/ ../repoB master --squash
> # Output:
> warning: no common commits
> remote: Counting objects: 9, done.
> remote: Compressing objects: 100% (3/3), done.
> remote: Total 9 (delta 0), reused 0 (delta 0)
> Unpacking objects: 100% (9/9), done.
> From ../repoB
>  * branch            master     -> FETCH_HEAD
> Can't squash-merge: 'repoB_subtree' was never added.
> # Note, git exits with status 1 at this point.
>
> # RepoB's third commit ('Another signed commit') is not pulled into
> the subree in repo A.
> # This can be verified by running a diff:
> diff -qr --exclude ".git" repoB_subtree ../repoB
> # Output:
> Files repoB_subtree/FileB and ../repoB/FileB differ
Stephen R Guglielmo· Jan 30, 2018, 19:15 UTC · re: Stephen R Guglielmo · lore

Re: Bug Report: Subtrees and GPG Signed Commits

Hi, just following up on this bug report. I have not heard back. Is there additional information that's needed? Is there a better place to file bug reports?

Additionally, I have confirmed that this bug still exists with git version 2.16.1.

Thanks

On Thu, Jan 18, 2018 at 11:19 AM, Stephen R Guglielmo <srguglielmo@gmail.com> wrote:

Show 93 quoted lines
> Hi, just following up on this bug report. I have not heard back. Is
> there additional information that's needed? Is there a better place to
> file bug reports?
>
> Thanks
>
> On Sat, Jan 6, 2018 at 5:45 PM, Stephen R Guglielmo
> <srguglielmo@gmail.com> wrote:
>> Hi all,
>>
>> I've noticed an issue regarding the use of `git subtree add` and `git
>> subtree pull` when the subtree repository's commit (either HEAD or
>> whatever commit specified by the subtree command) is signed with GPG.
>> It seems to work properly if the commit is not signed but previous
>> commits are.
>>
>> The gist of the issue is that `git subtree add` does not add the
>> subree properly and a "fatal: Not a valid object name" error is
>> thrown. Running `git subtree pull` does not pull any upstream changes
>> after that ("'subtree' was never added").
>>
>> I have not done extensive testing, however, below are instructions to
>> reproduce the issue. This was tested using git version 2.15.1
>> installed via Homebrew on MacOS. I did not test with the built-in
>> version of git on MacOS.
>>
>> Thanks,
>> Steve
>>
>> # Create a new repository
>> mkdir repoA && cd repoA
>> git init
>> echo "Test File in Repo A" > FileA
>> git add -A && git commit -m 'Initial commit in repo A'
>>
>> # Create a second repository
>> cd .. && mkdir repoB && cd repoB
>> git init
>> echo "Test File in Repo B" > FileB
>> git add -A && git commit -m 'Initial commit in repo B'
>>
>> # Create a signed commit in repo B
>> echo "Signed Commit" >> FileB
>> git commit -a -S  -m 'Signed commit in repo B'
>>
>> # Now, add repoB as a subtree of RepoA
>> cd ../repoA
>> git subtree add --prefix repoB_subtree/ ../repoB/ master --squash
>> # Output:
>> git fetch ../repoB/ master
>> warning: no common commits
>> remote: Counting objects: 6, done.
>> remote: Compressing objects: 100% (2/2), done.
>> remote: Total 6 (delta 0), reused 0 (delta 0)
>> Unpacking objects: 100% (6/6), done.
>> From ../repoB
>>  * branch            master     -> FETCH_HEAD
>> fatal: Not a valid object name gpg: Signature made Sat Jan  6 17:38:31 2018 EST
>> gpg:                using RSA key 6900E9CFDD39B6A741D601F50999759F2DCF3E7C
>> gpg: Good signature from "Stephen Robert Guglielmo (Temple University
>> Computer Services) <srg@temple.edu>" [ultimate]
>> Primary key fingerprint: 6900 E9CF DD39 B6A7 41D6  01F5 0999 759F 2DCF 3E7C
>> 4b700b1a4ebb9e2c1011aafd6b0f720b38f059a4
>> # Note, git exits with status 128 at this point.
>>
>> # FileB was in fact added and staged to repoA, despite the "fatal"
>> above. Commit it:
>> git commit -m 'Add repoB subtree'
>>
>> # Ok, let's make another commit in repoB and try a `subtree pull`
>> instead of `subtree add`
>> cd ../repoB
>> echo "Another Line" >> FileB
>> git commit -a -S -m 'Another signed commit'
>> cd ../repoA
>> git subtree pull --prefix repoB_subtree/ ../repoB master --squash
>> # Output:
>> warning: no common commits
>> remote: Counting objects: 9, done.
>> remote: Compressing objects: 100% (3/3), done.
>> remote: Total 9 (delta 0), reused 0 (delta 0)
>> Unpacking objects: 100% (9/9), done.
>> From ../repoB
>>  * branch            master     -> FETCH_HEAD
>> Can't squash-merge: 'repoB_subtree' was never added.
>> # Note, git exits with status 1 at this point.
>>
>> # RepoB's third commit ('Another signed commit') is not pulled into
>> the subree in repo A.
>> # This can be verified by running a diff:
>> diff -qr --exclude ".git" repoB_subtree ../repoB
>> # Output:
>> Files repoB_subtree/FileB and ../repoB/FileB differ
Stefan Beller· Jan 30, 2018, 23:17 UTC · re: Stephen R Guglielmo · lore

Re: Bug Report: Subtrees and GPG Signed Commits

On Tue, Jan 30, 2018 at 11:15 AM, Stephen R Guglielmo <srguglielmo@gmail.com> wrote:

Show 104 quoted lines
> Hi, just following up on this bug report. I have not heard back. Is
> there additional information that's needed? Is there a better place to
> file bug reports?
>
> Additionally, I have confirmed that this bug still exists with git
> version 2.16.1.
>
> Thanks
>
> On Thu, Jan 18, 2018 at 11:19 AM, Stephen R Guglielmo
> <srguglielmo@gmail.com> wrote:
>> Hi, just following up on this bug report. I have not heard back. Is
>> there additional information that's needed? Is there a better place to
>> file bug reports?
>>
>> Thanks
>>
>> On Sat, Jan 6, 2018 at 5:45 PM, Stephen R Guglielmo
>> <srguglielmo@gmail.com> wrote:
>>> Hi all,
>>>
>>> I've noticed an issue regarding the use of `git subtree add` and `git
>>> subtree pull` when the subtree repository's commit (either HEAD or
>>> whatever commit specified by the subtree command) is signed with GPG.
>>> It seems to work properly if the commit is not signed but previous
>>> commits are.
>>>
>>> The gist of the issue is that `git subtree add` does not add the
>>> subree properly and a "fatal: Not a valid object name" error is
>>> thrown. Running `git subtree pull` does not pull any upstream changes
>>> after that ("'subtree' was never added").
>>>
>>> I have not done extensive testing, however, below are instructions to
>>> reproduce the issue. This was tested using git version 2.15.1
>>> installed via Homebrew on MacOS. I did not test with the built-in
>>> version of git on MacOS.
>>>
>>> Thanks,
>>> Steve
>>>
>>> # Create a new repository
>>> mkdir repoA && cd repoA
>>> git init
>>> echo "Test File in Repo A" > FileA
>>> git add -A && git commit -m 'Initial commit in repo A'
>>>
>>> # Create a second repository
>>> cd .. && mkdir repoB && cd repoB
>>> git init
>>> echo "Test File in Repo B" > FileB
>>> git add -A && git commit -m 'Initial commit in repo B'
>>>
>>> # Create a signed commit in repo B
>>> echo "Signed Commit" >> FileB
>>> git commit -a -S  -m 'Signed commit in repo B'
>>>
>>> # Now, add repoB as a subtree of RepoA
>>> cd ../repoA
>>> git subtree add --prefix repoB_subtree/ ../repoB/ master --squash
>>> # Output:
>>> git fetch ../repoB/ master
>>> warning: no common commits
>>> remote: Counting objects: 6, done.
>>> remote: Compressing objects: 100% (2/2), done.
>>> remote: Total 6 (delta 0), reused 0 (delta 0)
>>> Unpacking objects: 100% (6/6), done.
>>> From ../repoB
>>>  * branch            master     -> FETCH_HEAD
>>> fatal: Not a valid object name gpg: Signature made Sat Jan  6 17:38:31 2018 EST
>>> gpg:                using RSA key 6900E9CFDD39B6A741D601F50999759F2DCF3E7C
>>> gpg: Good signature from "Stephen Robert Guglielmo (Temple University
>>> Computer Services) <srg@temple.edu>" [ultimate]
>>> Primary key fingerprint: 6900 E9CF DD39 B6A7 41D6  01F5 0999 759F 2DCF 3E7C
>>> 4b700b1a4ebb9e2c1011aafd6b0f720b38f059a4
>>> # Note, git exits with status 128 at this point.
>>>
>>> # FileB was in fact added and staged to repoA, despite the "fatal"
>>> above. Commit it:
>>> git commit -m 'Add repoB subtree'
>>>
>>> # Ok, let's make another commit in repoB and try a `subtree pull`
>>> instead of `subtree add`
>>> cd ../repoB
>>> echo "Another Line" >> FileB
>>> git commit -a -S -m 'Another signed commit'
>>> cd ../repoA
>>> git subtree pull --prefix repoB_subtree/ ../repoB master --squash
>>> # Output:
>>> warning: no common commits
>>> remote: Counting objects: 9, done.
>>> remote: Compressing objects: 100% (3/3), done.
>>> remote: Total 9 (delta 0), reused 0 (delta 0)
>>> Unpacking objects: 100% (9/9), done.
>>> From ../repoB
>>>  * branch            master     -> FETCH_HEAD
>>> Can't squash-merge: 'repoB_subtree' was never added.
>>> # Note, git exits with status 1 at this point.
>>>
>>> # RepoB's third commit ('Another signed commit') is not pulled into
>>> the subree in repo A.
>>> # This can be verified by running a diff:
>>> diff -qr --exclude ".git" repoB_subtree ../repoB
>>> # Output:
>>> Files repoB_subtree/FileB and ../repoB/FileB differ

There has not been feedback for a while on this thread. I think that is because subtrees are not in anyone's hot interest area currently.

This is definitely the right place to submit&discuss bugs. Looking through "git log --format="%ae %s" -S subtree", it seems as if Avery (apenwarr@gmail.com) was mostly interested in developing subtrees, though I think he has moved on. Originally it was invented by Junio, who is the active maintainer of the project in 68faf68938 (A new merge stragety 'subtree'., 2007-02-15)

Thanks, Stefan

Junio C Hamano· Jan 30, 2018, 23:24 UTC · re: Stefan Beller · lore

Re: Bug Report: Subtrees and GPG Signed Commits

Stefan Beller <sbeller@google.com> writes:
Show 11 quoted lines
> There has not been feedback for a while on this thread.
> I think that is because subtrees are not in anyone's hot
> interest area currently.
>
> This is definitely the right place to submit&discuss bugs.
> Looking through "git log --format="%ae %s" -S subtree",
> it seems as if Avery (apenwarr@gmail.com) was mostly
> interested in developing subtrees, though I think he has
> moved on. Originally it was invented by Junio, who is
> the active maintainer of the project in 68faf68938
> (A new merge stragety 'subtree'., 2007-02-15)

Thanks for trying to help, but I have *NOTHING* to do with the "git subtree" subcommand (and I personally have no interest in it). What I did was a subtree merge strategy (i.e. "git merge -s subtree"), which is totally a different thing.

David Greene offered to take it over in 2015, and then we saw some activity by David Aguilar in 2016, but otherwise the subcommand from contrib/ has pretty much been dormant these days.

Avery Pennarun· Jan 30, 2018, 23:37 UTC · re: Junio C Hamano · lore

Re: Bug Report: Subtrees and GPG Signed Commits

On Tue, Jan 30, 2018 at 6:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 21 quoted lines
> Stefan Beller <sbeller@google.com> writes:
>> There has not been feedback for a while on this thread.
>> I think that is because subtrees are not in anyone's hot
>> interest area currently.
>>
>> This is definitely the right place to submit&discuss bugs.
>> Looking through "git log --format="%ae %s" -S subtree",
>> it seems as if Avery (apenwarr@gmail.com) was mostly
>> interested in developing subtrees, though I think he has
>> moved on. Originally it was invented by Junio, who is
>> the active maintainer of the project in 68faf68938
>> (A new merge stragety 'subtree'., 2007-02-15)
>
> Thanks for trying to help, but I have *NOTHING* to do with the "git
> subtree" subcommand (and I personally have no interest in it).  What
> I did was a subtree merge strategy (i.e. "git merge -s subtree"),
> which is totally a different thing.
>
> David Greene offered to take it over in 2015, and then we saw some
> activity by David Aguilar in 2016, but otherwise the subcommand from
> contrib/ has pretty much been dormant these days.

Strictly speaking, the 'git subtree' command does in fact use 'git merge -s subtree' under the covers, so Junio is at least partly responsible for giving me the idea :)

I actually have never looked into how signed commits work and although I still use git-subtree occasionally (it hasn't needed any maintenance, for my simple use cases), I have never used it with signed commits.

git-subtree maintains a cache that maps commit ids in the "original project" with their equivalents in the "merged project." If there's something magic about how commit ids work with signed commits, I could imagine that causing the "no a valid object name" problems. Or, git-subtree in --squash mode actually generates new commit objects using some magic of its own. If it were to accidentally copy a signature into a commit that no longer matches the original, I imagine that new object might get rejected.

Unfortunately I don't have time to look into it. The git-subtree code is pretty straightforward, though, so if Stephen has an hour or two to look deeper it's probably possible to fix it up. The tool is not actually as magical and difficult as it might seem at first glance :)

Sorry I can't help more.
Good luck,
Avery
Stephen R Guglielmo· Jan 31, 2018, 12:33 UTC · re: Avery Pennarun · lore

Re: Bug Report: Subtrees and GPG Signed Commits

On Tue, Jan 30, 2018 at 6:37 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
Show 51 quoted lines
> On Tue, Jan 30, 2018 at 6:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>> There has not been feedback for a while on this thread.
>>> I think that is because subtrees are not in anyone's hot
>>> interest area currently.
>>>
>>> This is definitely the right place to submit&discuss bugs.
>>> Looking through "git log --format="%ae %s" -S subtree",
>>> it seems as if Avery (apenwarr@gmail.com) was mostly
>>> interested in developing subtrees, though I think he has
>>> moved on. Originally it was invented by Junio, who is
>>> the active maintainer of the project in 68faf68938
>>> (A new merge stragety 'subtree'., 2007-02-15)
>>
>> Thanks for trying to help, but I have *NOTHING* to do with the "git
>> subtree" subcommand (and I personally have no interest in it).  What
>> I did was a subtree merge strategy (i.e. "git merge -s subtree"),
>> which is totally a different thing.
>>
>> David Greene offered to take it over in 2015, and then we saw some
>> activity by David Aguilar in 2016, but otherwise the subcommand from
>> contrib/ has pretty much been dormant these days.
>
> Strictly speaking, the 'git subtree' command does in fact use 'git
> merge -s subtree' under the covers, so Junio is at least partly
> responsible for giving me the idea :)
>
> I actually have never looked into how signed commits work and although
> I still use git-subtree occasionally (it hasn't needed any
> maintenance, for my simple use cases), I have never used it with
> signed commits.
>
> git-subtree maintains a cache that maps commit ids in the "original
> project" with their equivalents in the "merged project."  If there's
> something magic about how commit ids work with signed commits, I could
> imagine that causing the "no a valid object name" problems.  Or,
> git-subtree in --squash mode actually generates new commit objects
> using some magic of its own.  If it were to accidentally copy a
> signature into a commit that no longer matches the original, I imagine
> that new object might get rejected.
>
> Unfortunately I don't have time to look into it.  The git-subtree code
> is pretty straightforward, though, so if Stephen has an hour or two to
> look deeper it's probably possible to fix it up.  The tool is not
> actually as magical and difficult as it might seem at first glance :)
>
> Sorry I can't help more.
>
> Good luck,
>
> Avery
Thanks all for the discussion/replies.

We use subtrees extensively in our environment right now. The "sub" repos (90+) are located on GitHub, while the "main/parent" repo is provided by a vendor on website hosting infrastructure.

I will take a look at: git/Documentation/CodingGuidelines git/Documentation/SubmittingPatches git/contrib/subtree/

Should I follow up in this thread with a patch (it might be a while)?

Thanks! Steve

Junio C Hamano· Feb 2, 2018, 23:39 UTC · re: Stephen R Guglielmo · lore

Re: Bug Report: Subtrees and GPG Signed Commits

Stephen R Guglielmo <srguglielmo@gmail.com> writes:
Show 20 quoted lines
> On Tue, Jan 30, 2018 at 6:37 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>>
>> Sorry I can't help more.
>>
>> Good luck,
>>
>> Avery
>
> Thanks all for the discussion/replies.
>
> We use subtrees extensively in our environment right now. The "sub"
> repos (90+) are located on GitHub, while the "main/parent" repo is
> provided by a vendor on website hosting infrastructure.
>
> I will take a look at:
> git/Documentation/CodingGuidelines
> git/Documentation/SubmittingPatches
> git/contrib/subtree/
>
> Should I follow up in this thread with a patch (it might be a while)?

These three are good place to start at. You may find the output of "git shortlog --no-merges --since=N.months contrib/subtree" and "git blame contrib/subtree" also a good source of whom to ask for help. As we said on this thread, this is a corner of the contrib/ section that nobody seems to be actively working on, so if you really depend on it working well, you might have to take the ownership of it ;-)

Thanks.
Stephen R Guglielmo· Feb 5, 2018, 14:30 UTC · re: Stephen R Guglielmo · lore

Re: Bug Report: Subtrees and GPG Signed Commits

On Wed, Jan 31, 2018 at 7:33 AM, Stephen R Guglielmo <srguglielmo@gmail.com> wrote:

Show 68 quoted lines
> On Tue, Jan 30, 2018 at 6:37 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>> On Tue, Jan 30, 2018 at 6:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Stefan Beller <sbeller@google.com> writes:
>>>> There has not been feedback for a while on this thread.
>>>> I think that is because subtrees are not in anyone's hot
>>>> interest area currently.
>>>>
>>>> This is definitely the right place to submit&discuss bugs.
>>>> Looking through "git log --format="%ae %s" -S subtree",
>>>> it seems as if Avery (apenwarr@gmail.com) was mostly
>>>> interested in developing subtrees, though I think he has
>>>> moved on. Originally it was invented by Junio, who is
>>>> the active maintainer of the project in 68faf68938
>>>> (A new merge stragety 'subtree'., 2007-02-15)
>>>
>>> Thanks for trying to help, but I have *NOTHING* to do with the "git
>>> subtree" subcommand (and I personally have no interest in it).  What
>>> I did was a subtree merge strategy (i.e. "git merge -s subtree"),
>>> which is totally a different thing.
>>>
>>> David Greene offered to take it over in 2015, and then we saw some
>>> activity by David Aguilar in 2016, but otherwise the subcommand from
>>> contrib/ has pretty much been dormant these days.
>>
>> Strictly speaking, the 'git subtree' command does in fact use 'git
>> merge -s subtree' under the covers, so Junio is at least partly
>> responsible for giving me the idea :)
>>
>> I actually have never looked into how signed commits work and although
>> I still use git-subtree occasionally (it hasn't needed any
>> maintenance, for my simple use cases), I have never used it with
>> signed commits.
>>
>> git-subtree maintains a cache that maps commit ids in the "original
>> project" with their equivalents in the "merged project."  If there's
>> something magic about how commit ids work with signed commits, I could
>> imagine that causing the "no a valid object name" problems.  Or,
>> git-subtree in --squash mode actually generates new commit objects
>> using some magic of its own.  If it were to accidentally copy a
>> signature into a commit that no longer matches the original, I imagine
>> that new object might get rejected.
>>
>> Unfortunately I don't have time to look into it.  The git-subtree code
>> is pretty straightforward, though, so if Stephen has an hour or two to
>> look deeper it's probably possible to fix it up.  The tool is not
>> actually as magical and difficult as it might seem at first glance :)
>>
>> Sorry I can't help more.
>>
>> Good luck,
>>
>> Avery
>
> Thanks all for the discussion/replies.
>
> We use subtrees extensively in our environment right now. The "sub"
> repos (90+) are located on GitHub, while the "main/parent" repo is
> provided by a vendor on website hosting infrastructure.
>
> I will take a look at:
> git/Documentation/CodingGuidelines
> git/Documentation/SubmittingPatches
> git/contrib/subtree/
>
> Should I follow up in this thread with a patch (it might be a while)?
>
> Thanks!
> Steve
Hi all,

It looks like I've found the cause of the issue. I have log.showsignature=true in my gitconfig. The toptree_for_commit() function calls `git log` and passes the output to `git commit-tree` in new_squash_commit(). Apparently commit-tree doesn't like GPG sigs.

The fix was simple: --no-show-signature. However, I believe this was added in git v2.10.0, so it's not fully backwards compatible. I'm open to suggestions on a better fix if this is not acceptable.

Thanks!
https://github.com/srguglielmo/git/commit/822c8a45d049f86ea5c59c0b434303964e4e6f3d
diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
index cc033af73..dec085a23 100755
--- a/contrib/subtree/git-subtree.sh
+++ b/contrib/subtree/git-subtree.sh
@@ -475,7 +475,7 @@ squash_msg () {

 toptree_for_commit () {
        commit="$1"
-       git log -1 --pretty=format:'%T' "$commit" -- || exit $?
+       git log --no-show-signature -1 --pretty=format:'%T' "$commit"
-- || exit $?
 }

 subtree_for_commit () {
Stephen R Guglielmo· Feb 5, 2018, 14:40 UTC · re: Stephen R Guglielmo · lore

Re: Bug Report: Subtrees and GPG Signed Commits

On Mon, Feb 5, 2018 at 9:30 AM, Stephen R Guglielmo <srguglielmo@gmail.com> wrote:

Show 103 quoted lines
> On Wed, Jan 31, 2018 at 7:33 AM, Stephen R Guglielmo
> <srguglielmo@gmail.com> wrote:
>> On Tue, Jan 30, 2018 at 6:37 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>>> On Tue, Jan 30, 2018 at 6:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>>> Stefan Beller <sbeller@google.com> writes:
>>>>> There has not been feedback for a while on this thread.
>>>>> I think that is because subtrees are not in anyone's hot
>>>>> interest area currently.
>>>>>
>>>>> This is definitely the right place to submit&discuss bugs.
>>>>> Looking through "git log --format="%ae %s" -S subtree",
>>>>> it seems as if Avery (apenwarr@gmail.com) was mostly
>>>>> interested in developing subtrees, though I think he has
>>>>> moved on. Originally it was invented by Junio, who is
>>>>> the active maintainer of the project in 68faf68938
>>>>> (A new merge stragety 'subtree'., 2007-02-15)
>>>>
>>>> Thanks for trying to help, but I have *NOTHING* to do with the "git
>>>> subtree" subcommand (and I personally have no interest in it).  What
>>>> I did was a subtree merge strategy (i.e. "git merge -s subtree"),
>>>> which is totally a different thing.
>>>>
>>>> David Greene offered to take it over in 2015, and then we saw some
>>>> activity by David Aguilar in 2016, but otherwise the subcommand from
>>>> contrib/ has pretty much been dormant these days.
>>>
>>> Strictly speaking, the 'git subtree' command does in fact use 'git
>>> merge -s subtree' under the covers, so Junio is at least partly
>>> responsible for giving me the idea :)
>>>
>>> I actually have never looked into how signed commits work and although
>>> I still use git-subtree occasionally (it hasn't needed any
>>> maintenance, for my simple use cases), I have never used it with
>>> signed commits.
>>>
>>> git-subtree maintains a cache that maps commit ids in the "original
>>> project" with their equivalents in the "merged project."  If there's
>>> something magic about how commit ids work with signed commits, I could
>>> imagine that causing the "no a valid object name" problems.  Or,
>>> git-subtree in --squash mode actually generates new commit objects
>>> using some magic of its own.  If it were to accidentally copy a
>>> signature into a commit that no longer matches the original, I imagine
>>> that new object might get rejected.
>>>
>>> Unfortunately I don't have time to look into it.  The git-subtree code
>>> is pretty straightforward, though, so if Stephen has an hour or two to
>>> look deeper it's probably possible to fix it up.  The tool is not
>>> actually as magical and difficult as it might seem at first glance :)
>>>
>>> Sorry I can't help more.
>>>
>>> Good luck,
>>>
>>> Avery
>>
>> Thanks all for the discussion/replies.
>>
>> We use subtrees extensively in our environment right now. The "sub"
>> repos (90+) are located on GitHub, while the "main/parent" repo is
>> provided by a vendor on website hosting infrastructure.
>>
>> I will take a look at:
>> git/Documentation/CodingGuidelines
>> git/Documentation/SubmittingPatches
>> git/contrib/subtree/
>>
>> Should I follow up in this thread with a patch (it might be a while)?
>>
>> Thanks!
>> Steve
>
> Hi all,
>
> It looks like I've found the cause of the issue. I have
> log.showsignature=true in my gitconfig. The toptree_for_commit()
> function calls `git log` and passes the output to `git commit-tree` in
> new_squash_commit(). Apparently commit-tree doesn't like GPG sigs.
>
> The fix was simple: --no-show-signature. However, I believe this was
> added in git v2.10.0, so it's not fully backwards compatible. I'm open
> to suggestions on a better fix if this is not acceptable.
>
> Thanks!
>
>
> https://github.com/srguglielmo/git/commit/822c8a45d049f86ea5c59c0b434303964e4e6f3d
>
>
>
> diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
> index cc033af73..dec085a23 100755
> --- a/contrib/subtree/git-subtree.sh
> +++ b/contrib/subtree/git-subtree.sh
> @@ -475,7 +475,7 @@ squash_msg () {
>
>  toptree_for_commit () {
>         commit="$1"
> -       git log -1 --pretty=format:'%T' "$commit" -- || exit $?
> +       git log --no-show-signature -1 --pretty=format:'%T' "$commit"
> -- || exit $?
>  }
>
>  subtree_for_commit () {
Hey again,

Actually, to follow up on this, I added --no-show-signature to several other locations. The above patch fixes the fatal, however the GPG sig info is still included in the commit merge message for `subtree pull`. This fixes that as well.

https://github.com/srguglielmo/git/commit/ebd2f628ddb960931aac5087c45a54b953976e99
diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
index cc033af73..8126132dc 100755
--- a/contrib/subtree/git-subtree.sh
+++ b/contrib/subtree/git-subtree.sh
@@ -297,7 +297,7 @@ find_latest_squash () {
        main=
        sub=
        git log --grep="^git-subtree-dir: $dir/*\$" \
-               --pretty=format:'START %H%n%s%n%n%b%nEND%n' HEAD |
+               --no-show-signature --pretty=format:'START
%H%n%s%n%n%b%nEND%n' HEAD |
        while read a b junk
        do
                debug "$a $b $junk"
@@ -341,7 +341,7 @@ find_existing_splits () {
        main=
        sub=
        git log --grep="^git-subtree-dir: $dir/*\$" \
-               --pretty=format:'START %H%n%s%n%n%b%nEND%n' $revs |
+               --no-show-signature --pretty=format:'START
%H%n%s%n%n%b%nEND%n' $revs |
        while read a b junk
        do
                case "$a" in
@@ -382,7 +382,7 @@ copy_commit () {
        # We're going to set some environment vars here, so
        # do it in a subshell to get rid of them safely later
        debug copy_commit "{$1}" "{$2}" "{$3}"
-       git log -1 --pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" |
+       git log --no-show-signature -1
--pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" |
        (
                read GIT_AUTHOR_NAME
                read GIT_AUTHOR_EMAIL
@@ -462,8 +462,8 @@ squash_msg () {
                oldsub_short=$(git rev-parse --short "$oldsub")
                echo "Squashed '$dir/' changes from
$oldsub_short..$newsub_short"
                echo
-               git log --pretty=tformat:'%h %s' "$oldsub..$newsub"
-               git log --pretty=tformat:'REVERT: %h %s' "$newsub..$oldsub"
+               git log --no-show-signature --pretty=tformat:'%h %s'
"$oldsub..$newsub"
+               git log --no-show-signature --pretty=tformat:'REVERT:
%h %s' "$newsub..$oldsub"
        else
                echo "Squashed '$dir/' content from commit $newsub_short"
        fi
Junio C Hamano· Feb 5, 2018, 18:45 UTC · re: Stephen R Guglielmo · lore

Re: Bug Report: Subtrees and GPG Signed Commits

Stephen R Guglielmo <srguglielmo@gmail.com> writes:
Show 12 quoted lines
> diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
> index cc033af73..dec085a23 100755
> --- a/contrib/subtree/git-subtree.sh
> +++ b/contrib/subtree/git-subtree.sh
> @@ -475,7 +475,7 @@ squash_msg () {
>
>  toptree_for_commit () {
>         commit="$1"
> -       git log -1 --pretty=format:'%T' "$commit" -- || exit $?
> +       git log --no-show-signature -1 --pretty=format:'%T' "$commit"
> -- || exit $?
>  }
Given that all references to this shell function seem to do
	sometree=$(toptree_for_commit $something)

and then $sometree is used as if it were a tree object name, I can understand why the lack of --no-show-signature in the original breaks it when the user has show-signature configured.

It probably makes more sense to replace the "git log" with something more appropirate for the job, like

	git rev-parse --verify "$commit^{tree}"
though.
Stephen R Guglielmo· Feb 8, 2018, 13:53 UTC · re: Junio C Hamano · lore

Re: Bug Report: Subtrees and GPG Signed Commits

On Mon, Feb 5, 2018 at 1:45 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 14 quoted lines
> Given that all references to this shell function seem to do
>
>         sometree=$(toptree_for_commit $something)
>
> and then $sometree is used as if it were a tree object name, I can
> understand why the lack of --no-show-signature in the original
> breaks it when the user has show-signature configured.
>
> It probably makes more sense to replace the "git log" with something
> more appropirate for the job, like
>
>         git rev-parse --verify "$commit^{tree}"
>
> though.
Junio,

Thanks for the feedback. I read the documentation on `rev-parse` and I agree it makes more sense for this case. I've modified `toptree_for_commit()` to use `rev-parse` instead. However, there's a few other calls to `log` that I'm not sure can be replaced. For those, I've kept `--no-show-signature`, making this require at least v2.10.0.

Below is the full diff against v2.16.1. Or, for the GitHub-savvy among us:
https://github.com/srguglielmo/git/compare/8279ed033f703d4115bee620dccd32a9ec94d9aa...srguglielmo:4b92b4494da057ed52c2ff8c329457a1c294d135

Thanks, Steve

diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
index dec085a23..9594ca4b5 100755
--- a/contrib/subtree/git-subtree.sh
+++ b/contrib/subtree/git-subtree.sh
@@ -297,7 +297,7 @@ find_latest_squash () {
     main=
     sub=
     git log --grep="^git-subtree-dir: $dir/*\$" \
-        --pretty=format:'START %H%n%s%n%n%b%nEND%n' HEAD |
+        --no-show-signature --pretty=format:'START %H%n%s%n%n%b%nEND%n' HEAD |
     while read a b junk
     do
         debug "$a $b $junk"
@@ -341,7 +341,7 @@ find_existing_splits () {
     main=
     sub=
     git log --grep="^git-subtree-dir: $dir/*\$" \
-        --pretty=format:'START %H%n%s%n%n%b%nEND%n' $revs |
+        --no-show-signature --pretty=format:'START %H%n%s%n%n%b%nEND%n' $revs |
     while read a b junk
     do
         case "$a" in
@@ -382,7 +382,7 @@ copy_commit () {
     # We're going to set some environment vars here, so
     # do it in a subshell to get rid of them safely later
     debug copy_commit "{$1}" "{$2}" "{$3}"
-    git log -1 --pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" |
+    git log --no-show-signature -1
--pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" |
     (
         read GIT_AUTHOR_NAME
         read GIT_AUTHOR_EMAIL
@@ -462,8 +462,8 @@ squash_msg () {
         oldsub_short=$(git rev-parse --short "$oldsub")
         echo "Squashed '$dir/' changes from $oldsub_short..$newsub_short"
         echo
-        git log --pretty=tformat:'%h %s' "$oldsub..$newsub"
-        git log --pretty=tformat:'REVERT: %h %s' "$newsub..$oldsub"
+        git log --no-show-signature --pretty=tformat:'%h %s' "$oldsub..$newsub"
+        git log --no-show-signature --pretty=tformat:'REVERT: %h %s'
"$newsub..$oldsub"
     else
         echo "Squashed '$dir/' content from commit $newsub_short"
     fi
@@ -475,7 +475,7 @@ squash_msg () {

 toptree_for_commit () {
     commit="$1"
-    git log -1 --pretty=format:'%T' "$commit" -- || exit $?
+    git rev-parse --verify "$commit^{tree}" || exit $?
 }

 subtree_for_commit () {

← back to recent threads