git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/3] Performance improvements for repacking non-promisor objects

From
Junio C Hamano <gitster@pobox.com>
Date
Dec 3, 2024, 04:18 UTC
Message-ID
<xmqqr06pe6vj.fsf@gitster.g>
In-Reply-To
<cover.1733170252.git.jonathantanmy@google.com>
Jonathan Tan <jonathantanmy@google.com> writes:
Show 20 quoted lines
> This is a follow-up to jt/repack-local-promisor [1] (but these patches
> are based on master, since that branch has already been merged).
>
> These patches speed up a fetch that takes 7 hours to take under 3
> minutes. More details are in the commit messages, especially that of
> patch 1.
>
> Thanks in advance to everyone who reviews. While review is going on,
> we'll also be testing these at $DAYJOB (I've tested it to work on one
> known big repo, but there may be others).
>
> [1] https://lore.kernel.org/git/cover.1730491845.git.jonathantanmy@google.com/
>
> Jonathan Tan (3):
>   index-pack: dedup first during outgoing link check
>   index-pack: no blobs during outgoing link check
>   index-pack: commit tree during outgoing link check
>
>  builtin/index-pack.c | 49 +++++++++++++++++++++++---------------------
>  1 file changed, 26 insertions(+), 23 deletions(-)

When merged to 'seen', this seems to break quite a many tests, all related to "pack".

I haven't tried running tests on the topic stand-alone.
Test Summary Report
-------------------
t5310-pack-bitmaps.sh                            (Wstat: 256 (exited 1) Tests: 230 Failed: 3)
  Failed tests:  26, 100, 179
  Non-zero exit status: 1
t5327-multi-pack-bitmaps-rev.sh                  (Wstat: 256 (exited 1) Tests: 314 Failed: 6)
  Failed tests:  25, 70, 136, 182, 227, 293
  Non-zero exit status: 1
t5616-partial-clone.sh                           (Wstat: 256 (exited 1) Tests: 47 Failed: 3)
  Failed tests:  5, 38-39
  Non-zero exit status: 1
t5326-multi-pack-bitmaps.sh                      (Wstat: 256 (exited 1) Tests: 357 Failed: 6)
  Failed tests:  25, 70, 146, 199, 244, 320
  Non-zero exit status: 1
t0410-partial-clone.sh                           (Wstat: 256 (exited 1) Tests: 38 Failed: 4)
  Failed tests:  11, 14-15, 38
  Non-zero exit status: 1
t6020-bundle-misc.sh                             (Wstat: 256 (exited 1) Tests: 30 Failed: 4)
  Failed tests:  16-19
  Non-zero exit status: 1
t9211-scalar-clone.sh                            (Wstat: 256 (exited 1) Tests: 13 Failed: 2)
  Failed tests:  4-5
  Non-zero exit status: 1
Files=1031, Tests=31721, 324 wallclock secs (11.58 usr  4.57 sys + 674.15 cusr 6136.28 csys = 6826.58 CPU)
Result: FAIL
g
Previous: Junio C HamanoNext: Junio C Hamano
Message 16 of 29 in “Performance improvements for repacking non-promisor objects”
  1. 0/3 Performance improvements for repacking non-promisor objectsJonathan Tan, Dec 2, 2024
  2. 1/3 index-pack: dedup first during outgoing link checkJonathan Tan, Dec 2, 2024
  3. Josh SteadmonDec 2, 2024
  4. 2/3 index-pack: no blobs during outgoing link checkJonathan Tan, Dec 2, 2024
  5. Patrick SteinhardtDec 3, 2024
  6. Jonathan TanDec 3, 2024
  7. Junio C HamanoDec 3, 2024
  8. 3/3 index-pack: commit tree during outgoing link checkJonathan Tan, Dec 2, 2024
  9. Junio C HamanoDec 3, 2024
  10. Jonathan TanDec 3, 2024
  11. Junio C HamanoDec 4, 2024
  12. Jonathan TanDec 9, 2024
  13. Junio C HamanoDec 9, 2024
  14. Josh SteadmonDec 2, 2024
  15. Junio C HamanoDec 3, 2024
  16. Junio C HamanoDec 3, 2024
  17. Junio C HamanoDec 3, 2024
  18. Junio C HamanoDec 3, 2024
  19. 0/3 Performance improvements for repacking non-promisor objectsJonathan Tan, Dec 3, 2024
  20. 1/3 index-pack --promisor: dedup before checking linksJonathan Tan, Dec 3, 2024
  21. 2/3 index-pack --promisor: don't check blobsJonathan Tan, Dec 3, 2024
  22. 3/3 index-pack --promisor: also check commits' treesJonathan Tan, Dec 3, 2024
  23. 0/3 Performance improvements for repacking non-promisor objectsJonathan Tan, Dec 3, 2024
  24. 1/3 index-pack --promisor: dedup before checking linksJonathan Tan, Dec 3, 2024
  25. Junio C HamanoDec 4, 2024
  26. 2/3 index-pack --promisor: don't check blobsJonathan Tan, Dec 3, 2024
  27. 3/3 index-pack --promisor: also check commits' treesJonathan Tan, Dec 3, 2024
  28. Junio C HamanoDec 4, 2024
  29. 4/3 index-pack: work around false positive use of uninitialized variableJunio C Hamano, Dec 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.