git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] server-info: do not list unlinked packs

From
EWEric Wong <e@80x24.org>
Date
May 23, 2019, 08:59 UTC
Message-ID
<20190523085959.4q76pokx2gy6wqq7@dcvr>
In-Reply-To
<20190515214806.GA31693@sigill.intra.peff.net>
Jeff King <peff@peff.net> wrote:
Show 13 quoted lines
> On Wed, May 15, 2019 at 08:38:39PM +0000, Eric Wong wrote:
> 
> > I've also noticed objects/info/packs contains stale entries
> > after repack/gc runs on current git.
> > 
> > Tried adding reprepare_packed_git before update_server_info,
> > but that didn't seem to work; so maybe something isn't cleared.
> > Might have time to investigate more this week, might not...
> 
> We never delete entries from the in-memory packed_git list; a reprepare
> only adds to the list. You'd need to teach update_server_info() to
> ignore packs which are no longer present (or switch to exec-ing a
> separate update-server-info binary).
Ah, checking files_exists() and setting a bit seems sufficient.
--------8<---------
Subject: [PATCH] server-info: do not list unlinked packs

Having non-existent packs in objects/info/packs causes dumb HTTP clients to abort.

There remains a small window where the old objects/info/packs file can refer to unlinked packs. That's unavoidable even on a local FS given the time-of-use-time-of-check window between listing and retrieving files.

Signed-off-by: Eric Wong <e@80x24.org>
---
  I think the small window I refer to can be worked around by
  teaching the dumb HTTP client to reread objects/info/packs
  if it 404s while trying to GET a pack...
 object-store.h | 1 +
 server-info.c  | 7 ++++++-
 t/t6500-gc.sh  | 2 ++
 3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/object-store.h b/object-store.h
index 272e01e452..2c9facc8f2 100644
--- a/object-store.h
+++ b/object-store.h
@@ -77,6 +77,7 @@ struct packed_git {
 		 freshened:1,
 		 do_not_close:1,
 		 pack_promisor:1,
+		 pack_unlinked:1,
 		 multi_pack_index:1;
 	unsigned char hash[GIT_MAX_RAWSZ];
 	struct revindex_entry *revindex;
diff --git a/server-info.c b/server-info.c
index 41274d098b..69e2c5279b 100644
--- a/server-info.c
+++ b/server-info.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "dir.h"
 #include "repository.h"
 #include "refs.h"
 #include "object.h"
@@ -199,12 +200,16 @@ static void init_pack_info(const char *infofile, int force)
 		 */
 		if (!p->pack_local)
 			continue;
+		if (!file_exists(p->pack_name)) {
+			p->pack_unlinked = 1;
+			continue;
+		}
 		i++;
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
 	for (i = 0, p = get_all_packs(the_repository); p; p = p->next) {
-		if (!p->pack_local)
+		if (!p->pack_local || p->pack_unlinked)
 			continue;
 		assert(i < num_pack);
 		info[i] = xcalloc(1, sizeof(struct pack_info));
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 515c6735e9..c0f04dc6b0 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -71,6 +71,8 @@ test_expect_success 'gc --keep-largest-pack' '
 		git gc --keep-largest-pack &&
 		( cd .git/objects/pack && ls *.pack ) >pack-list &&
 		test_line_count = 2 pack-list &&
+		awk "/^P /{print \$2}" <.git/objects/info/packs >pack-info &&
+		test_line_count = 2 pack-info &&
 		test_path_is_file $BASE_PACK &&
 		git fsck
 	)

base-commit: aa25c82427ae70aebf3b8f970f2afd54e9a2a8c6
-- 
EW
Previous: Jeff KingNext: Jeff King
Message 21 of 26 in “update-server-info: avoid needless overwrites”
  1. update-server-info: avoid needless overwritesEric Wong, May 11, 2019
  2. Eric SunshineMay 11, 2019
  3. update-server-info: avoid needless overwritesEric Wong, May 11, 2019
  4. Eric WongMay 11, 2019
  5. Ævar Arnfjörð BjarmasonMay 11, 2019
  6. Eric WongMay 12, 2019
  7. Jeff KingMay 12, 2019
  8. Ævar Arnfjörð BjarmasonMay 12, 2019
  9. Jeff KingMay 14, 2019
  10. Ævar Arnfjörð BjarmasonMay 14, 2019
  11. Jeff KingMay 14, 2019
  12. Ævar Arnfjörð BjarmasonMay 14, 2019
  13. Eric WongMay 14, 2019
  14. dumb HTTP things I want to doEric Wong, May 14, 2019
  15. Jeff KingMay 14, 2019
  16. Ævar Arnfjörð BjarmasonMay 14, 2019
  17. Jeff KingMay 14, 2019
  18. 2/1 server-info: conditionally update on fetchEric Wong, May 15, 2019
  19. [WIP] repack leaving stale entries in objects/info/packsEric Wong, May 15, 2019
  20. Jeff KingMay 15, 2019
  21. server-info: do not list unlinked packsEric Wong, May 23, 2019
  22. Jeff KingMay 23, 2019
  23. server-info: do not list unlinked packsEric Wong, May 23, 2019
  24. Jeff KingMay 24, 2019
  25. Ævar Arnfjörð BjarmasonMay 24, 2019
  26. update-server-info: avoid needless overwritesEric Wong, May 13, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.