git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 1/4] sha1-file: introduce no-lazy-fetch has_object()

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Aug 5, 2020, 23:06 UTC
Message-ID
<f413b34574a330030e08a6eb93ad812975959d7d.1596668156.git.jonathantanmy@google.com>
In-Reply-To
<cover.1596668156.git.jonathantanmy@google.com>

There have been a few bugs wherein Git fetches missing objects whenever the existence of an object is checked, even though it does not need to perform such a fetch. To resolve these bugs, we could look at all the places that has_object_file() (or a similar function) is used. As a first step, introduce a new function has_object() that checks for the existence of an object, with a default behavior of not fetching if the object is missing and the repository is a partial clone. As we verify each has_object_file() (or similar) usage, we can replace it with has_object(), and we will know that we are done when we can delete has_object_file() (and the other similar functions).

Also, the new function has_object() has more appropriate defaults: besides not fetching, it also does not recheck packed storage.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 object-store.h | 25 +++++++++++++++++++++++--
 sha1-file.c    | 12 ++++++++++++
 2 files changed, 35 insertions(+), 2 deletions(-)
diff --git a/object-store.h b/object-store.h
index f439d47af8..c4fc9dd74e 100644
--- a/object-store.h
+++ b/object-store.h
@@ -239,12 +239,33 @@ int read_loose_object(const char *path,
 		      unsigned long *size,
 		      void **contents);
 
+/* Retry packed storage after checking packed and loose storage */
+#define HAS_OBJECT_RECHECK_PACKED 1
+
+/*
+ * Returns 1 if the object exists. This function will not lazily fetch objects
+ * in a partial clone.
+ */
+int has_object(struct repository *r, const struct object_id *oid,
+	       unsigned flags);
+
+/*
+ * These macros and functions are deprecated. If checking existence for an
+ * object that is likely to be missing and/or whose absence is relatively
+ * inconsequential (or is consequential but the caller is prepared to handle
+ * it), use has_object(), which has better defaults (no lazy fetch in a partial
+ * clone and no rechecking of packed storage). In the unlikely event that a
+ * caller needs to assert existence of an object that it fully expects to
+ * exist, and wants to trigger a lazy fetch in a partial clone, use
+ * oid_object_info_extended() with a NULL struct object_info.
+ *
+ * These functions can be removed once all callers have migrated to
+ * has_object() and/or oid_object_info_extended().
+ */
 #ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS
 #define has_sha1_file_with_flags(sha1, flags) repo_has_sha1_file_with_flags(the_repository, sha1, flags)
 #define has_sha1_file(sha1) repo_has_sha1_file(the_repository, sha1)
 #endif
-
-/* Same as the above, except for struct object_id. */
 int repo_has_object_file(struct repository *r, const struct object_id *oid);
 int repo_has_object_file_with_flags(struct repository *r,
 				    const struct object_id *oid, int flags);
diff --git a/sha1-file.c b/sha1-file.c
index ccd34dd9e8..ff444d7abb 100644
--- a/sha1-file.c
+++ b/sha1-file.c
@@ -1988,6 +1988,18 @@ int force_object_loose(const struct object_id *oid, time_t mtime)
 	return ret;
 }
 
+int has_object(struct repository *r, const struct object_id *oid,
+	       unsigned flags)
+{
+	int quick = !(flags & HAS_OBJECT_RECHECK_PACKED);
+	unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |
+		(quick ? OBJECT_INFO_QUICK : 0);
+
+	if (!startup_info->have_repository)
+		return 0;
+	return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;
+}
+
 int repo_has_object_file_with_flags(struct repository *r,
 				    const struct object_id *oid, int flags)
 {
-- 
2.28.0.236.gb10cc79966-goog
Previous: Jonathan TanNext: Jonathan Tan
Message 5 of 9 in “apply: do not fetch when checking object existence”
  1. apply: do not fetch when checking object existenceJonathan Tan, Jul 28, 2020
  2. Junio C HamanoJul 28, 2020
  3. Jonathan TanJul 28, 2020
  4. 0/4 No-lazy-fetch has_object() and some fixesJonathan Tan, Aug 5, 2020
  5. 1/4 sha1-file: introduce no-lazy-fetch has_object()Jonathan Tan, Aug 5, 2020
  6. 3/4 pack-objects: no fetch when allow-{any,promisor}Jonathan Tan, Aug 5, 2020
  7. 4/4 fsck: do not lazy fetch known non-promisor objectJonathan Tan, Aug 5, 2020
  8. 2/4 apply: do not lazy fetch when applying binaryJonathan Tan, Aug 5, 2020
  9. Junio C HamanoAug 6, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.