Using git-update-index to add existing tree objects at other paths

Hi all,

We're looking to create a tool that can duplicate a specific subtree elsewhere in the repository. (This is part of a rag-tag replacement for svn:external that will continue to work with our existing build infrastructure while we adjust everything to be organized better in the actual repository.) The basic idea is as follows:

1. At the repository root, there is a manifest that defines groups of
   paths that should be copies of one another.

2. Server hook logic will enforce that for each new commit, the paths in
   the manifest at that commit are TREESAME to each other. This
   guarantee is important for our build processes.

3. Client-side tooling will help produce this TREESAME-ness in the
   context of sparse-checkouts. Support for sparse-index may not be so
   large a concern as this entire approach is more of a stopgap measure.

It's this last step, the client-side tooling, that is proving tricky.

### Solution based on git-fast-import

A working solution using git-fast-import was recently submitted for review:

    $ cat | git fast-import
    feature done
    commit refs/heads/<current-branch>
    committer (...)
    data <<ENDDATA
    (...commit message...)
    ENDDATA
    from refs/heads/<current-branch>
    M 040000 TREEID1 PATH1
    M 040000 TREEID1 PATH2
    M 040000 TREEID1 PATH3
    M 040000 TREEID2 PATH1
    M 040000 TREEID2 PATH2
    M 040000 TREEID2 PATH3
    (...)
    done

but using fast-import for this purpose seems like an abuse. It's especially suboptimal because it creates a *new* commit that brings everything into sync, so the two commits (the one the developer makes and the one created by this tooling) must be squashed together.

### Something better?

This feels like a job for git-update-index. From the docs,

but the following does not work:

    $ cat | git update-index --index-info
    040000 tree TREEID1	PATH1

(using the appropriate delimiters, of course). This command succeeds, but checking git-status (and confirming with other tools like `git ls-files -s`) reveal that this invocation did not have the desired effect:

with `PATH1` simply prepended to each file in TREEID1. In contrast, the desired state is

I also tried appending a / at the end of PATH1 (this isn't ls-tree output, but it's 'reasonable'), but git-update-index simply ignores the line with the message

Is there a better, recommended way to do this? Given the language in git-update-index.txt, is this a bug / functionality gap in update-index? The documentation reads as if I could

    $ git ls-tree <args> | git update-index --index-info

but it seems that is not the case.

Thanks, -Sean

-- Sean Allred

On Tue, Oct 4, 2022 at 7:43 AM Sean Allred <allred.sean@gmail.com> wrote:

The index is not designed to hold tree objects (at least not at this level), and update-index should be fed only the full path names of files, symbolic links (mode 120000), or gitlinks (mode 160000).

Whether it's reasonable to ask for the update-index command to understand that `040000 tree <treeid> <path>` means to insert all the files *within* the given (existing) tree object, at the implied constructed paths, I'll leave as an open topic here. :-) But if you were to do this manually yourself, it should work.

Chris

Chris Torek <chris.torek@gmail.com> writes:

Correct.

With the ongoing "sparse-index" work, it may not be an entirely unreasonable idea to extend the "update-index" command to allow it adding a tree entry there (after removing the entries that are for anything that fall inside that directory), but we are not there yet, in the sense that we haven't even discussed if doing it at this level is a reasonable thing to do.

Hi Sean

On 04/10/2022 14:44, Sean Allred wrote:

I agree update-index sounds like the right tool for what you want to do and this sounds like a bug but I'm not that familiar with update-index. As a workaround you could try

	git ls-tree -r arg | sed s:	:	new/prefix/: | git update-index --index-info

Best Wishes

Phillip