This is going to be forward compatible, but not backward compatible: projects are expected to adopt the new tool, but not go back to this old one.

CC: Colin Stagner <ask+git@howdoi.land>
CC: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 contrib/subtree/git-subtree.sh | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 contrib/subtree/t/t7900-subtree.sh | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

My rewrite of git-subtree is coming along fairly nicely. I have done a bunch of data model design work, as well as successfully implemented the "split" operation. (It's several orders of magnitude faster than the shell script implementation, and produces much better output.)

I have concluded that it is going to be too difficult to have bidirectional interoperability, for a number of reasons. One reason is that I want to change the way split commits are constructed and they should be as stable as we can make them.

Another, bigger, reason is that current git-subtree generates unmarked subtree merges (ie, without any git-subtree trailers); reliably figuring out whether something is such a merge requires a complete history walk, which is very unfortunate. To avoid having to do this in perpetuity, I plan to have new git-subtree assume that commits which show signs of use of new git-subtree are not unmarked subtree merges generated by old git-subtree.

So mycompatibility plan is:

 - New git-subtree can read old git-subtree data
 - Old git-subtree (this one here) will *not* handle new data

New git-subtree needs an in-downstream-tree config file with at least an indication of the downstream project name, because otherwise split commits can be quite inscrutable. So the presence of that file is a convenient way to signal the change.

The purpose of the present patch is to help prevent messes, where old git-subtree is used *afer* new git-subtree, generating output that no tooling can handle correctly. We change old git-subtree to spot the new git-subtree's config file, and bail out.

Right now there is no new data in existence, becauwe new git-subtree is not useable. So right now this change has no effect.

But if we ship this change now, users with this change will be defended from this lossage, as their collaborators start to use new git-subtree. I'm hoping to possibly persuade distros etc. to take this patch as a backport.

I wasn't sure how precisely to word the error message. I chose to refer to the new tool, as if it really exists. By the time anyone actually seex this message, it will do.

Ian Jackson (2):
  git-subtree: Bail out if we find output from Rust rewrite
  git-subtree: Bail out if we find output from Rust rewrite (test)

 contrib/subtree/git-subtree.sh     | 18 ++++++++++++++++++
 contrib/subtree/t/t7900-subtree.sh | 18 ++++++++++++++++++
 2 files changed, 36 insertions(+)

Ian Jackson <ijackson@chiark.greenend.org.uk> writes:

[warning: I have no idea what is going on in the code we see here, as I do not use subtree script at all]

The above helper may work for one caller that passes "$rev" but not for the other caller that passes "HEAD", no?

	if git rev-parse --verify -q "$1:$config"
	then
		die "fatal: tree contains $config: has been processed with new standalone (Rust) git-subtree; use that tool instead of this one.  See https://codeberg.org/diziet/git-subtree https://crates.io/crates/git-subtree"
	fi

Overly long output does not look very easy to read, but I kept it the same as the original.

If (global) $rev is not set here, we'd check :.git-subtree/config in the index in order to detect the v2's configuration. It seems to me that this code however wants to inspect HEAD's tree.

This would happen to work, as the global "$rev" visible here is the same one as what the new helper function sees and uses.

The same comment as the one for cmd_add's usage.

Hi.  Thanks for the quick review.

Junio C Hamano writes ("Re: [PATCH 1/2] git-subtree: Bail out if we find output from Rust rewrite"):

This was a slip. The code in reject_if_v2_config is supposed to use its argument (as per the usage comment I added), not a global. I'll fix this with a respin.

(I think it may somehow work by accident in my tests.)

HEAD is a valid revision spec for git-rev-parse, but the function should use $1 (which in that case would be HEAD), not $rev.

I'm not a great fan of the long error message myself, but it seemed to be what the rest of the script was doing. I didn't find any multi-line calls to die, so that's why I did it this way.

I'm happy to reformat this to your taste.

Thanks, Ian.

Ian Jackson <ijackson@chiark.greenend.org.uk> writes:

Nah, it seems your plan is to deprecate this script over time and move everybody to a newer implementation, so as long as "die" does its job to stop and prevent breakages from spreading, that would be fine.

Thanks.

On 7/6/26 06:58, Ian Jackson wrote:

Subtree merges can be performed without git-subtree, via the `-X subtree` merge strategy option. While the design of RIIR git-subtree is outside the scope of this patch series, this may be worth thinking about in your rewrite.

This is a nit, but `local` is not specified by POSIX. I know it is used elsewhere within git-subtree, but it is specifically discouraged.

For subtree split, should we also test for this file in tree you are splitting: i.e., "$dir/$config"? The answer might be no.

I think that subtree merge should only test the top-level project, as this patch does now.

Colin

On 7/6/26 06:58, Ian Jackson wrote:

It may be slightly faster to create only one repo and just make orphan branches, like `test_create_subtree_add()` does.

`test_commit()` from test-lib-functions.sh may be superior to manually writing and committing this file.

Colin

Hi.  Thanks for the review.  I'll go through it point by point:

Colin Stagner writes ("Re: [PATCH 2/2] git-subtree: Bail out if we find output from Rust rewrite (test)"):

...

Thanks for the suggestions.  I'll take a look.

TBH I found this test framework quite awkward to work with. Maybe folks here have some tips:

One thing I was missing was a primitive for "check this fails *and produces an error message matching this regexp*". test_must_fail makes it easy for a slips in the command (or some kinds of regression) to go undetected: the test then passes because the command *does* fail with a usage error or whatever. And AFAICT there isn't a way to manually inspect the output when the tests pass? I resorted to sabotaging the test by adding `&& false` to the end of the shell snippet string, and eyeballing t/test-results/t7900-subtree.out.

Colin Stagner writes ("Re: [PATCH 1/2] git-subtree: Bail out if we find output from Rust rewrite"):

There are 7 existing uses of `local`. I think I prefer to use it here too. In practice I think there are no shells we might want to use that don't have local. The alternative is to change all the variable names to be obviously globally unique, which is clumsy and also seems to me to put us at greater risk of bugs.

You're right that we should consider this question. The answer is: no, we should not. Briefly, whether to use the new or old algorithms depends on whether the downstream has adopted the new git-subtree, not on whether the upstream has added some optional config.

https://codeberg.org/diziet/git-subtree/src/branch/main/DATA-MODEL.md#control-of-unmarked-subtree-merges-guessing-config

By "top-level" I think you mean what I've taken to calling the "downstream": the project where the subtree is in a subdir, and whose top-level has other stuff. In which case I agree.

This is what I'm calling an "unmarked subtree merge". My rewrite is not going to support this user behaviour. The problem is that it is not possible to reliably determine whetheer something is an unmarked subtree merge.

It is possible to guess based on tree similarity, but that's a heuristic. It's also possible to guess based on root commits. Both of these approaches can go wrong in some cases. I prefer to write reliable software, which doesn't guess.

I'll advise against this practice in the documentation, but I'm reasonably confident that if a user does this anyway the results won't be terrible. The upstream input to an unmarked subtree merge in a downstream that has already used my rewrite, will be treated as if it were a downstream branch that predates the subtree addition. The effect on split (in most cases) is a missing parent relationship, which is undesirable but not catastrophic.I've made a note to add a test case for this scenario.

Combining manual -X subtree merges with git-subtree --squash merges could easily produce quite weird and wrong results in the tree (even before anyone tries split, or something). I don't think I can even reliably detect this situation after the user has done it, and of course since that user is using plain git, I certainly can't prevent it. This is another reason why manual use of -X subtree should be discouraged.

Regards, Ian.

Hi Ian

On 09/07/2026 10:36, Ian Jackson wrote:

I think

     test_commit --no-tag sabotage .git-subtree/config "# sabotage"

is the equivalent of what you have in the test at the moment

The usual approach to checking that a command fails for the expected reason is

     test_must_fail git ... 2>err &&
     test_grep regexp err

which prints the contents of err if it does not match regexp. To see the output of the tests run them with "-v". I frequently use "-v -i -x" to debug test failures. "-i" stops the test run at the first failure so you can inspect the test repository and "-x" turns on tracing so you can see which command failed which is useful when I test has not been written with debugging in mind.

Thanks

Phillip
	

On 7/9/26 04:36, Ian Jackson wrote:

Yes, I think we're talking about the same thing.

In retrospect, "top-level" is ambiguous. "Upstream" and "downstream" may be as well. Within git-branch(1), the phrase "upstream" refers to the remote tracking branch set by

     git branch --set-upstream-to=<upstream>

git-merge(1) is consistent with this.

     "If no commit is given from the command line, merge the
      remote-tracking branches that the current branch is
      configured to use as its upstream."

git-subtree.sh doesn't really deal in "upstreams" in the git-branch or git-merge sense.

Less ambiguous language is available:

For merge commits, there is the "first parent" and "second parent" (or 3rd or higher parents).

For trees, there is the "root tree" and "sub-trees," like `git ls-tree -r`

     -r     Recurse into sub-trees.

Both of these deliberately ignore the dependency relationship between the various projects and branches in question, which can potentially get messy.

Very well-reasoned; I like it.

Let me ask this question in a slightly different way: does RIIR subtree honor config files in locations other than the one you test for above? That's

     ${rev}:.git-subtree/config

which is `.git-subtree/config` within the root tree of the rev that is being manipulated?

If this is the only config file RIIR subtree honors, the patch is probably correct. If RIIR subtree honors config from other places, such as

* the working tree
* HEAD:.git-subtree/config
* HEAD:./.git-subtree/config

then consider testing for those if appropriate.

Thanks for looking at this.

I haven't tried it, but I think if --squash is in use, then attempting an unmarked subtree merge will probably die with "unrelated history" warnings.

Looking forward to v2,

Colin

Colin Stagner writes ("Re: [PATCH 1/2] git-subtree: Bail out if we find output from Rust rewrite [and 1 more messages]"):

Upstream and downstream are of course relative terms. I think the git usage you cite isn't quite central.

I'm using "upstream" in the wider sense; here, when you import a depedency you're downstream of it.

I'm open to better terminology and now is a good time to be debating this, but I don't like the other suggestions so far.

I want a term that talks about the logical (even, social) relationship between the two projects; and it should be one that makes sense from the point of view of the upstream. Talking about the file position within the downstream tree doesn't make sense from the upstream's point of view.

I think the dependency relationship is inherent in git-subtree's usual use cases: suppose a project A gets merged with git-subtree into a subdirectory S of project B, so that B.git:/S/ is a copy of A.git:/

Then I think almost invariably, this is because A has B as a dependency. And A has B as an upstream:

Code that's part of B flows from B to A, and can be edited in A, but the canonical version is that in B itself. If there are multiple As incorporating the same B, they share via "split", which produces history "within" B. Thios seems a classic upstream/downstream relationship.

As I say, I'm open to other terminology but I don't think "root tree" and "subtree" are the general terms I need to describe the relationship. In particular, from the point of view of the upstream project, it is its own root tree.

Yes, but not relevantly. Different information is taken from different places (the design gets a little complex to make sure everything works in all the use cases).

I think that's not guaranteed if squash merges and non-squash merges are interleaved.

Thanks for your support, and your critical consideration of the design questions.

Ian.

Hi Ian,

On Thu, Jul 9, 2026 at 5:48 AM Ian Jackson <ijackson@chiark.greenend.org.uk> wrote: [snip]

Just to make sure I understand you (I regularly use -X subtree with one project): the Rust rewrite won't support -X subtree merges, but we don't intend to discourage folks from using -X subtree merges in toto, right? Merely not support a mix of the 2?

D. Ben Knoble writes ("Re: [PATCH 1/2] git-subtree: Bail out if we find output from Rust rewrite [and 1 more messages]"):

Precisely so.

I think you may find my git-subtree rewrite superior in ergonomics to git merge -X subtree so you might want to switch to it, when it exists and supports that transition.

I haven't yet thought about how that transition ought to go but I think it might look like what I'm calling an "unmarked subtree merge~. I've made a TODO note in my working branch to remind myself to consider this situation.

Regards, Ian.

Ian Jackson <ijackson@chiark.greenend.org.uk> writes:

So, is there a conclusion after reviewing this?

I think this is the only thing outstanding item among the review comments this thread received. Specifically, regarding the use of 'local' discussed in the thread, our coding guidelines explicitly state:

 - Even though "local" is not part of POSIX, we make heavy use of it
   in our test suite.  We do not use it in scripted Porcelains, and
   hopefully nobody starts using "local" before all shells that matter
   support it (notably, ksh from AT&T Research does not support it yet).

Thus, we are fine there.

Just responding belatedly as I was scanning topics that are marked as "Expecting a reroll" in my draft copy of the "What's cooking" report that I work from.

Thanks.

On 7/11/26 18:04, Junio C Hamano wrote:

I think we're expecting a reroll, but this looks like the way forward.

Colin

Colin Stagner writes ("Re: [PATCH 1/2] git-subtree: Bail out if we find output from Rust rewrite [and 1 more messages]"):

Yes.  Please bear with me, I'm travelling for a few days.

Ian.

Ian Jackson <ijackson@chiark.greenend.org.uk> writes:

No worries, and take your time. I was just updating the status of the various topics in the "What's cooking" draft.

Thanks.

Nothing here impacts the patch under review, so this is a bit OT, but...

On 7/10/26 07:20, Ian Jackson wrote:

It may be useful to differentiate between command documentation like git-merge(1) and tutorial documentation like gitworkflows(7).

The man page for `merge` reads like: "So you want to merge THIS into THAT? Here's how to do it." The banner-line example is merging a topic branch into master, but the "social" aspect of this is not front-and-center.

Other common terms used in merges include "ours" (HEAD) and "theirs" (MERGE_HEAD, "branch head," "commit [that is being merged]").

gitworkflows(7) discusses the social relationships of branches, including the "merge upwards" workflow. Here is where we find more social terms like "upstream" and "downstream:"

     The merge workflow works by copying branches between
     upstream and downstream. Upstream can merge
     contributions into the official history;
     downstream base their work on the official history.

But "upwards" or "upstream" is merely in the direction of increasing stability or acceptance. This makes the terms "upstream" and "downstream" very broad and inclusive. An upstream branch might be in the same repo, a parent repo of a fork, or an entirely different repo. The repo might be yours or belong to someone else.

Branches are branches, wherever they are.

"Dependencies" are perhaps a bit beyond Git's usual scope as I understand it.

For subtree merges, it is possible that "largely unrelated" minirepos are being collected together just to make them a monorepo. I have also used subtree merges within a single repo. This is handy to keep a subproject isolated on its own branch for reuse elsewhere.

For splits, it's possible that history is split just to meet the needs of some other build system. I've observed this in the wild with AUR. I've seen multiple AUR packages stored together [1], but they must be `subtree split` first with aurpublish [2]. AUR users have been on-list before to report trouble with `subtree split` that I inadvertently caused [3]. They may be very interested in your rewrite.

In conclusion,

* Documentation is hard!

* Consider focusing "command-level" documentation more on mechanics. Use 
very specific terms like "branch," "(sub)tree," "merge-base," etc.

* Consider using "upstream" and "downstream" in the context of the 
"merging upwards" workflow from gitworkflows(7). It is not necessary for 
these to be in another repo or even a different "project."

These are just my recommendations, and they're not relevant for this patch series.

Probably true.

Colin

[1]: https://github.com/christian-heusel/aur

[2]: https://github.com/eli-schwartz/aurpublish

[3]: <755578cb-07e0-4b40-aa90-aacf4d45ccaa@heusel.eu>

Junio C Hamano <gitster@pobox.com> writes:

Any change of plans or situation since then?

Thanks.