git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 0/6] odb/files: cleanups and improvements for alternates

From
Patrick Steinhardt <ps@pks.im>
Date
Oct 15, 2026, 05:27 UTC
Message-ID
<20261015-pks-odb-files-alternates-cleanups-v3-0-bfc62fd780eb@pks.im>
In-Reply-To
<20261013-pks-odb-files-alternates-cleanups-v1-0-b7030ff2b26f@pks.im>
Hi,

this patch series contains a couple of follow-up cleanups and improvements that are now possible after having migrated alternates into the "files" backend.

The series is built on top of 6de20f6092 (The 4th batch, 2026-10-06) with ps/odb-files-alternates at 0ba59c8453 (odb/source: drop `read_alternates` callback, 2026-10-08) merged into it.

Changes in v2:
  - Add another refactoring on top to only fetch objects from a promisor
    remote in case they are missing.
  - Link to v1: https://patch.msgid.link/20261013-pks-odb-files-alternates-cleanups-v1-0-b7030ff2b26f@pks.im
Changes in v3:
  - Add a regression test, supplied by Tyrie.
  - Link to v2: https://patch.msgid.link/20261014-pks-odb-files-alternates-cleanups-v2-0-fed9b2caa3be@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (6):
      odb/source-files: fix performance when reading objects with alternates
      odb/source-files: fix performance when streaming objects with alternates
      odb/source-files: make the second read an implementation detail
      odb: remove read loop for found objects
      odb: don't try re-fetching corrupt objects
      odb/source: drop `local` flag
 midx.c                   |  2 +-
 odb.c                    | 63 ++++++++++++++----------------------------------
 odb.h                    | 12 ---------
 odb/source-files.c       | 26 ++++++++++++++++----
 odb/source-inmemory.c    | 10 +++++---
 odb/source-loose.c       | 17 ++-----------
 odb/source-loose.h       |  3 +--
 odb/source-packed.c      | 20 +++++----------
 odb/source-packed.h      |  7 ++++++
 odb/source.c             |  4 +--
 odb/source.h             | 20 +--------------
 repack.c                 |  2 +-
 t/t5615-alternate-env.sh | 43 +++++++++++++++++++++++++++++++++
 13 files changed, 109 insertions(+), 120 deletions(-)
Range-diff versus v2:
1:  3a7f78fc94 ! 1:  dea38443a4 odb/source-files: fix performance when reading objects with alternates
    @@ Commit message
         [1]: <CAFXJcxvpKHoVDwE5mBOd=w-A5vPdUmehqr8SHLUD7qv1qB00rA@mail.gmail.com>
     
         Reported-by: Wolfgang Kritzinger <wkritzinger@atlassian.com>
    +    Helped-by: Tyrie Vella <tyrielv@gmail.com>
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## odb/source-files.c ##
    @@ odb/source-files.c: static enum odb_read_status odb_source_files_read_object_inf
      		ret = odb_source_read_object_info(&dir->loose->base, oid, oi, flags,
      						  status == ODB_READ_NOT_FOUND ? errmsg : NULL);
      		if (!ret)
    +
    + ## t/t5615-alternate-env.sh ##
    +@@ t/t5615-alternate-env.sh: test_expect_success !MINGW 'broken quoting falls back to interpreting raw' '
    + 	EOF
    + '
    + 
    ++test_expect_success 'packs across sources are checked before loose objects' '
    ++	# Regression test for a performance issue in which an object that
    ++	# resides in an alternate as a packed object caused a spurious loose
    ++	# object lookup (a filesystem stat) on the main object store before the
    ++	# alternate packfile was consulted. Reading such an object must resolve
    ++	# to the alternate packfile, never to a loose copy in the main store.
    ++	#
    ++	# Build an alternate whose object "B" is stored as a delta in a
    ++	# packfile. git deltifies successive versions of a tracked file, so the
    ++	# older, shorter blob "B" becomes a delta against the newer, longer
    ++	# blob "O". A loose object has no delta base, so %(deltabase) tells us
    ++	# which store answered the read: the alternate pack (O) or a loose copy
    ++	# (the zero oid).
    ++	git init alt-src &&
    ++	test_seq 1 200 >alt-src/file &&
    ++	git -C alt-src add file &&
    ++	git -C alt-src commit -q -m base &&
    ++	B=$(git -C alt-src rev-parse HEAD:file) &&
    ++	git -C alt-src cat-file blob "$B" >b-content &&
    ++	test_seq 1 210 >alt-src/file &&
    ++	git -C alt-src add file &&
    ++	git -C alt-src commit -q -m more &&
    ++	O=$(git -C alt-src rev-parse HEAD:file) &&
    ++	git -C alt-src repack -adf --window=10 --depth=50 &&
    ++
    ++	# Precondition: in the alternate, B is a delta based on O.
    ++	echo "$B" >in &&
    ++	echo "$O" >expect &&
    ++	git -C alt-src cat-file --batch-check="%(deltabase)" <in >actual &&
    ++	test_cmp expect actual &&
    ++
    ++	# Main repo: write B as a loose object, before any alternate is active.
    ++	git init main &&
    ++	git -C main hash-object -w --stdin <b-content >/dev/null &&
    ++	test_path_is_file "main/.git/objects/$(test_oid_to_path "$B")" &&
    ++
    ++	# With the alternate active, B must resolve to the alternate packfile
    ++	# (deltabase O), not to the main store loose copy (deltabase zero oid).
    ++	GIT_ALTERNATE_OBJECT_DIRECTORIES="$PWD/alt-src/.git/objects" \
    ++		git -C main cat-file --batch-check="%(deltabase)" <in >actual &&
    ++	test_cmp expect actual
    ++'
    ++
    + test_done
2:  23d8dcfcdf = 2:  374cbda799 odb/source-files: fix performance when streaming objects with alternates
3:  406228ce0f = 3:  635d9ce86e odb/source-files: make the second read an implementation detail
4:  c8f4c96b77 = 4:  832852bad5 odb: remove read loop for found objects
5:  e4af3300af = 5:  a0eebb1225 odb: don't try re-fetching corrupt objects
6:  7658903430 = 6:  b4a76c5651 odb/source: drop `local` flag

--- base-commit: ea99757de7d4ef6c7727431bfd7b6726614102e0 change-id: 20261006-pks-odb-files-alternates-cleanups-11f686cc4942

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 29 of 37 in “odb/files: cleanups and improvements for alternates”
  1. 0/5 odb/files: cleanups and improvements for alternatesPatrick Steinhardt, Oct 13, 2026
  2. 1/5 odb/source-files: fix performance when reading objects with alternatesPatrick Steinhardt, Oct 13, 2026
  3. Justin ToblerOct 13, 2026
  4. 2/5 odb/source-files: fix performance when streaming objects with alternatesPatrick Steinhardt, Oct 13, 2026
  5. Justin ToblerOct 13, 2026
  6. 3/5 odb/source-files: make the second read an implementation detailPatrick Steinhardt, Oct 13, 2026
  7. Justin ToblerOct 13, 2026
  8. Patrick SteinhardtOct 14, 2026
  9. Justin ToblerOct 14, 2026
  10. 4/5 odb: remove read loop for found objectsPatrick Steinhardt, Oct 13, 2026
  11. Justin ToblerOct 13, 2026
  12. Patrick SteinhardtOct 14, 2026
  13. Justin ToblerOct 14, 2026
  14. 5/5 odb/source: drop `local` flagPatrick Steinhardt, Oct 13, 2026
  15. Justin ToblerOct 13, 2026
  16. Patrick SteinhardtOct 14, 2026
  17. Justin ToblerOct 14, 2026
  18. Patrick SteinhardtOct 15, 2026
  19. 0/6 odb/files: cleanups and improvements for alternatesPatrick Steinhardt, Oct 14, 2026
  20. 1/6 odb/source-files: fix performance when reading objects with alternatesPatrick Steinhardt, Oct 14, 2026
  21. Johannes SchindelinOct 14, 2026
  22. Patrick SteinhardtOct 14, 2026
  23. 2/6 odb/source-files: fix performance when streaming objects with alternatesPatrick Steinhardt, Oct 14, 2026
  24. 3/6 odb/source-files: make the second read an implementation detailPatrick Steinhardt, Oct 14, 2026
  25. 4/6 odb: remove read loop for found objectsPatrick Steinhardt, Oct 14, 2026
  26. 5/6 odb: don't try re-fetching corrupt objectsPatrick Steinhardt, Oct 14, 2026
  27. Justin ToblerOct 14, 2026
  28. 6/6 odb/source: drop `local` flagPatrick Steinhardt, Oct 14, 2026
  29. 0/6 odb/files: cleanups and improvements for alternatesPatrick Steinhardt, Oct 15, 2026
  30. 1/6 odb/source-files: fix performance when reading objects with alternatesPatrick Steinhardt, Oct 15, 2026
  31. 2/6 odb/source-files: fix performance when streaming objects with alternatesPatrick Steinhardt, Oct 15, 2026
  32. 3/6 odb/source-files: make the second read an implementation detailPatrick Steinhardt, Oct 15, 2026
  33. 4/6 odb: remove read loop for found objectsPatrick Steinhardt, Oct 15, 2026
  34. 5/6 odb: don't try re-fetching corrupt objectsPatrick Steinhardt, Oct 15, 2026
  35. 6/6 odb/source: drop `local` flagPatrick Steinhardt, Oct 15, 2026
  36. Justin ToblerOct 15, 2026
  37. Junio C HamanoOct 16, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.