{"thread":{"id":"58561","subject":"Using git-update-index to add existing tree objects at other paths","startedAt":"2022-10-04T14:42:00Z","lastAt":"2022-10-05T19:39:55Z","messageCount":4,"participants":["Sean Allred","Chris Torek","Phillip Wood","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"464172","messageId":"87o7ur1wi9.fsf@gmail.com","threadId":"58561","inReplyTo":null,"subject":"Using git-update-index to add existing tree objects at other paths","fromName":"Sean Allred","fromEmail":"allred.sean@gmail.com","sentAt":"2022-10-04T13:44:43Z","receivedAt":"2022-10-04T14:42:00Z","isPatch":false,"sender":{"key":"allred.sean@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2082195?v=4"},"body":"Hi all,\n\nWe're looking to create a tool that can duplicate a specific subtree\nelsewhere in the repository. (This is part of a rag-tag replacement for\nsvn:external that will continue to work with our existing build\ninfrastructure while we adjust everything to be organized better in the\nactual repository.) The basic idea is as follows:\n\n1. At the repository root, there is a manifest that defines groups of\n paths that should be copies of one another.\n\n2. Server hook logic will enforce that for each new commit, the paths in\n the manifest at that commit are TREESAME to each other. This\n guarantee is important for our build processes.\n\n3. Client-side tooling will help produce this TREESAME-ness in the\n context of sparse-checkouts. Support for sparse-index may not be so\n large a concern as this entire approach is more of a stopgap measure.\n\nIt's this last step, the client-side tooling, that is proving tricky.\n\n### Solution based on git-fast-import\n\nA working solution using git-fast-import was recently submitted for\nreview:\n\n $ cat | git fast-import\n feature done\n commit refs/heads/\n committer (...)\n data <\n M 040000 TREEID1 PATH1\n M 040000 TREEID1 PATH2\n M 040000 TREEID1 PATH3\n M 040000 TREEID2 PATH1\n M 040000 TREEID2 PATH2\n M 040000 TREEID2 PATH3\n (...)\n done\n\nbut using fast-import for this purpose seems like an abuse. It's\nespecially suboptimal because it creates a *new* commit that brings\neverything into sync, so the two commits (the one the developer makes\nand the one created by this tooling) must be squashed together.\n\n### Something better?\n\nThis feels like a job for git-update-index. From the docs,\n\n> USING --INDEX-INFO\n> ------------------\n>\n> `--index-info` is a more powerful mechanism that lets you feed\n> multiple entry definitions from the standard input, and designed\n> specifically for scripts. It can take inputs of three formats:\n>\n> . mode SP type SP sha1 TAB path\n> +\n> This format is to stuff `git ls-tree` output into the index.\n\nbut the following does not work:\n\n $ cat | git update-index --index-info\n 040000 tree TREEID1\tPATH1\n\n(using the appropriate delimiters, of course). This command succeeds,\nbut checking git-status (and confirming with other tools like `git\nls-files -s`) reveal that this invocation did not have the desired\neffect:\n\n> (mode) (oid) (stage)\t(path)\n> (...)\n> 100755 (oid1) 0 PATH1FILENAME\n> 100755 (oid2) 0 PATH1FILENAME\n> 100755 (oid3) 0 PATH1FILENAME\n\nwith `PATH1` simply prepended to each file in TREEID1. In contrast, the\ndesired state is\n\n> 100755 (oid1) 0 PATH1/FILENAME\n> 100755 (oid2) 0 PATH1/FILENAME\n> 100755 (oid3) 0 PATH1/FILENAME\n\nI also tried appending a / at the end of PATH1 (this isn't ls-tree\noutput, but it's 'reasonable'), but git-update-index simply ignores the\nline with the message\n\n> Ignoring path PATH1/\n\nIs there a better, recommended way to do this? Given the language in\ngit-update-index.txt, is this a bug / functionality gap in update-index?\nThe documentation reads as if I could\n\n $ git ls-tree | git update-index --index-info\n\nbut it seems that is not the case.\n\nThanks,\n-Sean\n\n--\nSean Allred\n"},{"id":"464174","messageId":"CAPx1GvfsRipG0qgDmMMoU+WCWCEyr4ayJteYWAy+EoJOoaNQXg@mail.gmail.com","threadId":"58561","inReplyTo":"87o7ur1wi9.fsf@gmail.com","subject":"Re: Using git-update-index to add existing tree objects at other paths","fromName":"Chris Torek","fromEmail":"chris.torek@gmail.com","sentAt":"2022-10-04T15:10:25Z","receivedAt":"2022-10-04T15:10:42Z","isPatch":false,"sender":{"key":"chris.torek@gmail.com","avatar":"https://avatars.githubusercontent.com/u/16826774?v=4"},"body":"On Tue, Oct 4, 2022 at 7:43 AM Sean Allred wrote:\n> ... but the following does not work:\n>\n> $ cat | git update-index --index-info\n> 040000 tree TREEID1 PATH1\n\nThe index is not designed to hold tree objects (at least not at this level),\nand update-index should be fed only the full path names of files, symbolic\nlinks (mode 120000), or gitlinks (mode 160000).\n\nWhether it's reasonable to ask for the update-index command to\nunderstand that `040000 tree ` means to insert all\nthe files *within* the given (existing) tree object, at the implied\nconstructed paths, I'll leave as an open topic here. :-) But if you\nwere to do this manually yourself, it should work.\n\nChris\n"},{"id":"464175","messageId":"25ec850e-8677-8011-912e-315efb93db4f@dunelm.org.uk","threadId":"58561","inReplyTo":"87o7ur1wi9.fsf@gmail.com","subject":"Re: Using git-update-index to add existing tree objects at other paths","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-04T15:13:54Z","receivedAt":"2022-10-04T15:14:00Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Sean\n\nOn 04/10/2022 14:44, Sean Allred wrote:\n> Hi all,\n> [...]\n>> USING --INDEX-INFO\n>> ------------------\n>>\n>> `--index-info` is a more powerful mechanism that lets you feed\n>> multiple entry definitions from the standard input, and designed\n>> specifically for scripts. It can take inputs of three formats:\n>>\n>> . mode SP type SP sha1 TAB path\n>> +\n>> This format is to stuff `git ls-tree` output into the index.\n> \n> but the following does not work:\n> \n> $ cat | git update-index --index-info\n> 040000 tree TREEID1\tPATH1\n> \n> (using the appropriate delimiters, of course). This command succeeds,\n> but checking git-status (and confirming with other tools like `git\n> ls-files -s`) reveal that this invocation did not have the desired\n> effect:\n> \n>> (mode) (oid) (stage)\t(path)\n>> (...)\n>> 100755 (oid1) 0 PATH1FILENAME\n>> 100755 (oid2) 0 PATH1FILENAME\n>> 100755 (oid3) 0 PATH1FILENAME\n> \n> with `PATH1` simply prepended to each file in TREEID1. In contrast, the\n> desired state is\n> \n>> 100755 (oid1) 0 PATH1/FILENAME\n>> 100755 (oid2) 0 PATH1/FILENAME\n>> 100755 (oid3) 0 PATH1/FILENAME\n> \n> I also tried appending a / at the end of PATH1 (this isn't ls-tree\n> output, but it's 'reasonable'), but git-update-index simply ignores the\n> line with the message\n> \n>> Ignoring path PATH1/\n> \n> Is there a better, recommended way to do this? Given the language in\n> git-update-index.txt, is this a bug / functionality gap in update-index?\n> The documentation reads as if I could\n>\n> $ git ls-tree | git update-index --index-info\n\nI agree update-index sounds like the right tool for what you want to do \nand this sounds like a bug but I'm not that familiar with update-index. \nAs a workaround you could try\n\n\tgit ls-tree -r arg | sed s:\t:\tnew/prefix/: | git update-index --index-info\n\nBest Wishes\n\nPhillip\n"},{"id":"464254","messageId":"xmqq8rluoy9r.fsf@gitster.g","threadId":"58561","inReplyTo":"CAPx1GvfsRipG0qgDmMMoU+WCWCEyr4ayJteYWAy+EoJOoaNQXg@mail.gmail.com","subject":"Re: Using git-update-index to add existing tree objects at other paths","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-05T19:39:44Z","receivedAt":"2022-10-05T19:39:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Chris Torek writes:\n\n> On Tue, Oct 4, 2022 at 7:43 AM Sean Allred wrote:\n>> ... but the following does not work:\n>>\n>> $ cat | git update-index --index-info\n>> 040000 tree TREEID1 PATH1\n>\n> The index is not designed to hold tree objects (at least not at this level),\n> and update-index should be fed only the full path names of files, symbolic\n> links (mode 120000), or gitlinks (mode 160000).\n\nCorrect.\n\nWith the ongoing \"sparse-index\" work, it may not be an entirely\nunreasonable idea to extend the \"update-index\" command to allow\nit adding a tree entry there (after removing the entries that are\nfor anything that fall inside that directory), but we are not there\nyet, in the sense that we haven't even discussed if doing it at this\nlevel is a reasonable thing to do.\n"}]}