git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 15/32] unpack_trees: only unpack $GIT_DIR/narrow subtree in narrow repository

From
Nguyen Thai Ngoc Duy <pclouds@gmail.com>
Date
Aug 25, 2010, 05:38 UTC
Message-ID
<AANLkTikiazizLaXtotxL2UcuJ64KgWwGCwy+E6rv42_G@mail.gmail.com>
In-Reply-To
<AANLkTinaF6e+bfVqRRThuZ=2PjpOs4P88RKFsxOTVSJR@mail.gmail.com>
On Wed, Aug 25, 2010 at 3:04 PM, Elijah Newren <newren@gmail.com> wrote:
Show 28 quoted lines
> Hi,
>
> 2010/8/24 Nguyễn Thái Ngọc Duy <pclouds@gmail.com>:
>> By definition, narrow repository is incomplete. It does not even have
>> enough tree for a single commit. So populating a full index is
>> impossible.
>>
>> Because of this, unpack_trees() is modified to only unpack trees
>> within $GIT_DIR/narrow, which narrow repo has all needed trees. This
>> makes the resulting index unsuitable for creating commits later on.
>> This is the reason index version is increased to 4, to avoid older
>> git from using it.
>>
>> The resulting tree objects created from the index is only part of the
>> full tree. Manipulation will be needed at commit time to create proper
>> tree for commits.
>
> I spent a while thinking about this a couple weeks ago and never came
> to a strong conclusion about which of two alternatives should be
> preferred; I'm curious why you decided to go for this solution.  An
> alternative I thought of was having the index have entries for missing
> files (whose contents did not exist in the repository or the working
> copy; rather all we know is the filename and its sha1sum) and also
> gain the ability to have entries for missing trees (which behave
> similarly; all we know is their name and their sha1sum, but the
> contents of that sha1sum are not in the repository or the working
> directory)  Is there a reason to prefer one alternative over the
> other?  Does the alternative I thought of even make sense?

That was nightmare. I had to deal with unpack_callback() and skip uninterested paths. And that function is, I think, quite optimized. I tried another approach, putting directories in index with hope that it would cut down the number of code path I'd have to touch. It did, but then index sorting order is just weird and I couldn't get it right (felt to risky).

unpack_trees() is used by diff (all diffs except diff_tree_sha1). traverse_trees()/unpack_trees() is also used by merge strategy. All those code is really complicated that I'd rather stay away from them.

That was before I went with tree rewrites. Soon after I realized the nice effect of tree rewrites is that the index is narrowed down. So I can get rid of tree rewrites as long as the index is still narrow. That's how I come to this approach.

Back to your questions. I think the alternative makes sense, it just looks a lot of work and quite intrusive. On the other hand, my current approach is quite simple (but it probably won't work for more than a single narrow tree)

-- 
Duy
Previous: Elijah NewrenNext: Nguyễn Thái Ngọc Duy
Message 32 of 66 in “subtree clone v2”
  1. 00/32 subtree clone v2Nguyễn Thái Ngọc Duy, Aug 24, 2010
  2. 01/32 add const to ce_write()Nguyễn Thái Ngọc Duy, Aug 24, 2010
  3. 02/32 cache-tree: abstract out write_sha1_file from cache_tree_update()Nguyễn Thái Ngọc Duy, Aug 24, 2010
  4. Jonathan NiederAug 24, 2010
  5. 03/32 cache-tree: ignore CE_REMOVE entries in verify_cache()Nguyễn Thái Ngọc Duy, Aug 24, 2010
  6. Jonathan NiederAug 24, 2010
  7. Nguyen Thai Ngoc DuyAug 25, 2010
  8. Jonathan NiederAug 25, 2010
  9. 04/32 move do_compress() from pack-objects.c to pack-write.cNguyễn Thái Ngọc Duy, Aug 24, 2010
  10. Jonathan NiederAug 24, 2010
  11. Nguyen Thai Ngoc DuyAug 25, 2010
  12. 05/32 pack-write: add functions for creating simple packsNguyễn Thái Ngọc Duy, Aug 24, 2010
  13. 06/32 tree.c: Add {set,clear}_tree_marksNguyễn Thái Ngọc Duy, Aug 24, 2010
  14. 07/32 tree.c: find_subtree() to search for a treeNguyễn Thái Ngọc Duy, Aug 24, 2010
  15. Elijah NewrenAug 25, 2010
  16. Nguyen Thai Ngoc DuyAug 25, 2010
  17. Elijah NewrenAug 25, 2010
  18. 08/32 Add $GIT_DIR/narrow checkNguyễn Thái Ngọc Duy, Aug 24, 2010
  19. 09/32 index: make narrow index incompatible with older gitNguyễn Thái Ngọc Duy, Aug 24, 2010
  20. Jonathan NiederAug 24, 2010
  21. Nguyen Thai Ngoc DuyAug 25, 2010
  22. 10/32 rev-list: support traversing in narrow repository modeNguyễn Thái Ngọc Duy, Aug 24, 2010
  23. Elijah NewrenAug 25, 2010
  24. 11/32 rev-list: support --narrow-treeNguyễn Thái Ngọc Duy, Aug 24, 2010
  25. Elijah NewrenAug 25, 2010
  26. Nguyen Thai Ngoc DuyAug 25, 2010
  27. 12/32 pack-objects: support --narrow-treeNguyễn Thái Ngọc Duy, Aug 24, 2010
  28. 13/32 upload-pack: support narrow-tree capabilityNguyễn Thái Ngọc Duy, Aug 24, 2010
  29. 14/32 fetch-pack: support --narrow-treeNguyễn Thái Ngọc Duy, Aug 24, 2010
  30. 15/32 unpack_trees: only unpack $GIT_DIR/narrow subtree in narrow repositoryNguyễn Thái Ngọc Duy, Aug 24, 2010
  31. Elijah NewrenAug 25, 2010
  32. Nguyen Thai Ngoc DuyAug 25, 2010
  33. 16/32 cache-tree: only cache tree within narrow areaNguyễn Thái Ngọc Duy, Aug 24, 2010
  34. 17/32 tree-diff: add narrow versions of diff_{root_,}tree_sha1Nguyễn Thái Ngọc Duy, Aug 24, 2010
  35. 18/32 log-tree: use narrow version of diff_tree_sha1Nguyễn Thái Ngọc Duy, Aug 24, 2010
  36. 19/32 clone: support --narrow optionNguyễn Thái Ngọc Duy, Aug 24, 2010
  37. 20/32 narrow-tree: add join_narrow_tree to do tree fixup for commitsNguyễn Thái Ngọc Duy, Aug 24, 2010
  38. 21/32 commit: add narrow's commit_tree versionNguyễn Thái Ngọc Duy, Aug 24, 2010
  39. 22/32 commit: use commit_narrow_tree() to support narrow repoNguyễn Thái Ngọc Duy, Aug 24, 2010
  40. 23/32 commit-tree: require --narrow-base in narrow repoNguyễn Thái Ngọc Duy, Aug 24, 2010
  41. 24/32 merge: refuse to merge if narrow bases are differentNguyễn Thái Ngọc Duy, Aug 24, 2010
  42. 25/32 merge: prepare commit properly in narrow modeNguyễn Thái Ngọc Duy, Aug 24, 2010
  43. 26/32 Add upload-narrow-base commandNguyễn Thái Ngọc Duy, Aug 24, 2010
  44. 27/32 rev-list: traverse some more trees to make upload-narrow-base happyNguyễn Thái Ngọc Duy, Aug 24, 2010
  45. 28/32 narrow-tree: add oldest_narrow_base()Nguyễn Thái Ngọc Duy, Aug 24, 2010
  46. 29/32 Add command fetch-narrow-baseNguyễn Thái Ngọc Duy, Aug 24, 2010
  47. 30/32 merge: support merging when narrow bases are differentNguyễn Thái Ngọc Duy, Aug 24, 2010
  48. 31/32 send-pack: do not use thin pack in narrow modeNguyễn Thái Ngọc Duy, Aug 24, 2010
  49. 32/32 daemon: support upload-narrow-baseNguyễn Thái Ngọc Duy, Aug 24, 2010
  50. Jonathan NiederAug 24, 2010
  51. Nguyen Thai Ngoc DuyAug 24, 2010
  52. Jonathan NiederAug 24, 2010
  53. Nguyen Thai Ngoc DuyAug 25, 2010
  54. Elijah NewrenAug 25, 2010
  55. Nguyen Thai Ngoc DuyAug 25, 2010
  56. Elijah NewrenAug 25, 2010
  57. Nguyen Thai Ngoc DuyAug 25, 2010
  58. Elijah NewrenAug 25, 2010
  59. Nguyen Thai Ngoc DuyAug 25, 2010
  60. Elijah NewrenAug 26, 2010
  61. Nguyen Thai Ngoc DuyAug 26, 2010
  62. Elijah NewrenAug 26, 2010
  63. Nguyen Thai Ngoc DuyAug 26, 2010
  64. Elijah NewrenAug 25, 2010
  65. Junio C HamanoAug 25, 2010
  66. Elijah NewrenAug 25, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.