git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 0/5] Sparse index: fetch, pull, ls-files

From
Elijah Newren <newren@gmail.com>
Date
Dec 10, 2021, 18:53 UTC
Message-ID
<CABPp-BEs5gnX+HfPNMRBqtMdhVfkk5oKDT8sN4fJiPG87JDjcw@mail.gmail.com>
In-Reply-To
<pull.1080.v3.git.1639149192.gitgitgadget@gmail.com>

On Fri, Dec 10, 2021 at 7:13 AM Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote:

Show 72 quoted lines
>
> This is based on ld/sparse-index-blame (merged with 'master' due to an
> unrelated build issue).
>
> Here are two relatively-simple patches that further the sparse index
> integrations.
>
> Did you know that 'fetch' and 'pull' read the index? I didn't, or this would
> have been an integration much earlier in the cycle. They read the index to
> look for the .gitmodules file in case there are submodules that need to be
> fetched. Since looking for a file by name is already protected, we only need
> to disable 'command_requires_full_index' and we are done.
>
> The 'ls-files' builtin is useful when debugging the index, and some scripts
> use it, too. We are not changing the default behavior which expands a sparse
> index in order to show all of the cached blobs. Instead, we add a '--sparse'
> option that allows us to see the sparse directory entries upon request.
> Combined with --debug, we can see a lot of index details, such as:
>
> $ git ls-files --debug --sparse
> LICENSE
>   ctime: 1634910503:287405820
>   mtime: 1634910503:287405820
>   dev: 16777220 ino: 119325319
>   uid: 501  gid: 20
>   size: 1098    flags: 200000
> README.md
>   ctime: 1634910503:288090279
>   mtime: 1634910503:288090279
>   dev: 16777220 ino: 119325320
>   uid: 501  gid: 20
>   size: 934 flags: 200000
> bin/index.js
>   ctime: 1634910767:828434033
>   mtime: 1634910767:828434033
>   dev: 16777220 ino: 119325520
>   uid: 501  gid: 20
>   size: 7292    flags: 200000
> examples/
>   ctime: 0:0
>   mtime: 0:0
>   dev: 0    ino: 0
>   uid: 0    gid: 0
>   size: 0   flags: 40004000
> package.json
>   ctime: 1634910503:288676330
>   mtime: 1634910503:288676330
>   dev: 16777220 ino: 119325321
>   uid: 501  gid: 20
>   size: 680 flags: 200000
>
>
> (In this example, the 'examples/' directory is sparse.)
>
> Thanks!
>
>
> Updates in v2
> =============
>
>  * Rebased onto latest ld/sparse-index-blame without issue.
>  * Updated the test to use diff-of-diffs instead of a sequence of greps.
>  * Added patches that remove the use of 'test-tool read-cache --table' and
>    its implementation.
>
>
> Updates in v3
> =============
>
>  * Fixed typo in commit message.
>  * Added comments around doing strange things in an ls-files test.
>  * Fixed adjacent typo in a test comment.
Thanks, this round addresses all my previous feedback.  However, there
are two things Ævar has brought up that I think are important:
   * cannot rely on `diff -u` for portability reasons[1] (his
suggestion of git diff --no-index sounds good, or you can use comm(1))
   * have documentation mention the trailing slash that sparse
directory entries are mentioned with[2]

[1] https://lore.kernel.org/git/211210.86zgp8bi48.gmgdl@evledraar.gmail.com/ [2] https://lore.kernel.org/git/211210.86v8zwbev9.gmgdl@evledraar.gmail.com/

Previous: Elijah NewrenNext: Derrick Stolee via GitGitGadget
Message 43 of 51 in “Sparse index: fetch, pull, ls-files”
  1. 0/2 Sparse index: fetch, pull, ls-filesDerrick Stolee via GitGitGadget, Nov 16, 2021
  2. 1/2 fetch/pull: use the sparse indexDerrick Stolee via GitGitGadget, Nov 16, 2021
  3. 2/2 ls-files: add --sparse optionDerrick Stolee via GitGitGadget, Nov 16, 2021
  4. Elijah NewrenNov 22, 2021
  5. Derrick StoleeNov 22, 2021
  6. Ævar Arnfjörð BjarmasonNov 23, 2021
  7. Derrick StoleeDec 8, 2021
  8. Derrick StoleeDec 8, 2021
  9. Elijah NewrenDec 8, 2021
  10. Derrick StoleeDec 8, 2021
  11. Elijah NewrenDec 8, 2021
  12. Derrick StoleeDec 8, 2021
  13. Ævar Arnfjörð BjarmasonDec 9, 2021
  14. Derrick StoleeDec 10, 2021
  15. Ævar Arnfjörð BjarmasonDec 10, 2021
  16. Junio C HamanoDec 13, 2021
  17. Derrick StoleeDec 16, 2021
  18. Junio C HamanoNov 17, 2021
  19. Derrick StoleeNov 17, 2021
  20. Junio C HamanoNov 18, 2021
  21. Ævar Arnfjörð BjarmasonNov 23, 2021
  22. 0/5 Sparse index: fetch, pull, ls-filesDerrick Stolee via GitGitGadget, Dec 8, 2021
  23. 1/5 fetch/pull: use the sparse indexDerrick Stolee via GitGitGadget, Dec 8, 2021
  24. 2/5 ls-files: add --sparse optionDerrick Stolee via GitGitGadget, Dec 8, 2021
  25. Elijah NewrenDec 9, 2021
  26. Derrick StoleeDec 10, 2021
  27. 3/5 t1092: replace 'read-cache --table' with 'ls-files --sparse'Derrick Stolee via GitGitGadget, Dec 8, 2021
  28. Elijah NewrenDec 9, 2021
  29. 4/5 t1091/t3705: remove 'test-tool read-cache --table'Derrick Stolee via GitGitGadget, Dec 8, 2021
  30. Elijah NewrenDec 9, 2021
  31. 5/5 test-read-cache: remove --table, --expand optionsDerrick Stolee via GitGitGadget, Dec 8, 2021
  32. Elijah NewrenDec 9, 2021
  33. 0/5 Sparse index: fetch, pull, ls-filesDerrick Stolee via GitGitGadget, Dec 10, 2021
  34. 1/5 fetch/pull: use the sparse indexDerrick Stolee via GitGitGadget, Dec 10, 2021
  35. 2/5 ls-files: add --sparse optionDerrick Stolee via GitGitGadget, Dec 10, 2021
  36. 3/5 t1092: replace 'read-cache --table' with 'ls-files --sparse'Derrick Stolee via GitGitGadget, Dec 10, 2021
  37. 4/5 t1091/t3705: remove 'test-tool read-cache --table'Derrick Stolee via GitGitGadget, Dec 10, 2021
  38. 5/5 test-read-cache: remove --table, --expand optionsDerrick Stolee via GitGitGadget, Dec 10, 2021
  39. Ævar Arnfjörð BjarmasonDec 10, 2021
  40. Elijah NewrenDec 10, 2021
  41. Ævar Arnfjörð BjarmasonDec 11, 2021
  42. Elijah NewrenDec 11, 2021
  43. Elijah NewrenDec 10, 2021
  44. 0/5 Sparse index: fetch, pull, ls-filesDerrick Stolee via GitGitGadget, Dec 22, 2021
  45. 1/5 fetch/pull: use the sparse indexDerrick Stolee via GitGitGadget, Dec 22, 2021
  46. 2/5 ls-files: add --sparse optionDerrick Stolee via GitGitGadget, Dec 22, 2021
  47. 3/5 t1092: replace 'read-cache --table' with 'ls-files --sparse'Derrick Stolee via GitGitGadget, Dec 22, 2021
  48. 4/5 t1091/t3705: remove 'test-tool read-cache --table'Derrick Stolee via GitGitGadget, Dec 22, 2021
  49. 5/5 test-read-cache: remove --table, --expand optionsDerrick Stolee via GitGitGadget, Dec 22, 2021
  50. Elijah NewrenDec 22, 2021
  51. Junio C HamanoDec 22, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.