{"thread":{"id":"55589","subject":"[PATCH 0/8] refs: cleanup errno sideband ref related functions","startedAt":"2021-04-29T15:32:13Z","lastAt":"2021-08-23T11:53:13Z","messageCount":148,"participants":["Han-Wen Nienhuys via GitGitGadget","Junio C Hamano","Han-Wen Nienhuys","Jonathan Tan","Ævar Arnfjörð Bjarmason"],"isPatch":true,"patchVersion":1,"patchTotal":8},"messages":[{"id":"423274","messageId":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":null,"subject":"[PATCH 0/8] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:00Z","receivedAt":"2021-04-29T15:32:13Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"v3\n\n * remove errno as an implicit communication mechanism from refs support\n completely.\n\nHan-Wen Nienhuys (8):\n refs: remove EINVAL specification from the errno sideband in\n read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: make errno output explicit for refs_resolve_ref_unsafe\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: clear errno return in refs_resolve_ref_unsafe()\n refs: stop setting EINVAL and ELOOP in symref resolution\n refs: explicitly propagate errno from refs_read_raw_ref\n\n refs.c | 46 ++++++++++++++++++++--------------\n refs.h | 1 +\n refs/debug.c | 4 +--\n refs/files-backend.c | 58 +++++++++++++++++++------------------------\n refs/packed-backend.c | 16 ++++++------\n refs/refs-internal.h | 31 +++++++++++++++--------\n 6 files changed, 86 insertions(+), 70 deletions(-)\n\n\nbase-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v1\nPull-Request: https://github.com/git/git/pull/1012\n-- \ngitgitgadget\n"},{"id":"423275","messageId":"7e8181e77d409af7595e357ad233b7781e026b78.1619710329.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:01Z","receivedAt":"2021-04-29T15:32:15Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nA grep for EINVAL */*c reveals that no code inspects EINVAL after reading\nreferences.\n\nThe files ref backend does use EINVAL so parse_loose_ref_contents() can\ncommunicate to lock_raw_ref() about garbage following the hex SHA1, or a short\nread in files_read_raw_ref(), but the files backend does not call into\nrefs_read_raw_ref(), so its EINVAL sideband error is unused.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs/refs-internal.h | 9 ++++-----\n 1 file changed, 4 insertions(+), 5 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 467f4b3c936d..29728a339fed 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,11 +617,10 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n+ * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n+ * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n+ * reading the ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \ngitgitgadget\n\n"},{"id":"423276","messageId":"b2c72097e5e8985e7fdd8e3eee66cdf43d1b65c0.1619710329.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:02Z","receivedAt":"2021-04-29T15:32:16Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nErrno is a global variable written by almost all system calls, and therefore it\nis hard to reason about its state. It's also useless for user-visible errors, as\nit leaves no place to report the offending file and/or syscall.\n\nFor the copy/rename support, calls to lock_ref_oid_basic() in this file are\nfollowed by:\n\n* lock_ref_oid_basic (copy/rename rollback error path)\n\n* write_ref_to_lockfile (both in the rollback path and the success path of\n copy/rename)\n\nThese calls do not inspect the incoming errno. As they perform I/O, they can\nclobber errno. For this reason, callers cannot reliably observe the errno that\nlock_ref_oid_basic() generated, so it is unsound for programmatic use.\n\nFor files_create_symref() and files_reflog_expire(), grepping over callers\nshowed no callers inspecting errno.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs/files-backend.c | 9 +--------\n 1 file changed, 1 insertion(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 119972ee16f8..c9511da1d387 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t * to remain.\n \t\t */\n \t\tif (remove_empty_directories(&ref_file)) {\n-\t\t\tlast_errno = errno;\n \t\t\tif (!refs_verify_refname_available(\n \t\t\t\t\t &refs->base,\n \t\t\t\t\t refname, extras, skip, err))\n@@ -962,7 +959,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tlast_errno = errno;\n+\t\tint last_errno = errno;\n \t\tif (last_errno != ENOTDIR ||\n \t\t !refs_verify_refname_available(&refs->base, refname,\n \t\t\t\t\t\t extras, skip, err))\n@@ -981,20 +978,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n \t\t\t\t\t extras, skip, err)) {\n-\t\tlast_errno = ENOTDIR;\n \t\tgoto error_return;\n \t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n \n \tif (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {\n-\t\tlast_errno = errno;\n \t\tgoto error_return;\n \t}\n \tgoto out;\n@@ -1005,7 +999,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"423277","messageId":"dd3eceade4fcf09a679fdf20dddae247b4e0e879.1619710329.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:04Z","receivedAt":"2021-04-29T15:32:18Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs.c | 12 ++++++++++++\n refs.h | 1 +\n refs/files-backend.c | 19 ++++++++++---------\n refs/refs-internal.h | 8 ++++++++\n 4 files changed, 31 insertions(+), 9 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 43e2ad6b612a..e8ce88040b3e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1780,6 +1780,18 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n+{\n+\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n+\t\t\t\t\t\t resolve_flags, oid, flags);\n+\t*failure_errno = errno;\n+\treturn result;\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\ndiff --git a/refs.h b/refs.h\nindex 48970dfc7e0f..ede405ac3874 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -68,6 +68,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t\t int resolve_flags,\n \t\t\t\t struct object_id *oid,\n \t\t\t\t int *flags);\n+\n const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags);\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex efe493ca1425..3ab09871db5e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -924,6 +924,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n+\tint resolve_errno;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -936,10 +937,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t refname, resolve_flags,\n-\t\t\t\t\t &lock->old_oid, type);\n-\tif (!resolved && errno == EISDIR) {\n+\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\t\tresolve_flags,\n+\t\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t\t&resolve_errno);\n+\tif (!resolved && resolve_errno == EISDIR) {\n \t\t/*\n \t\t * we are trying to lock foo but we used to\n \t\t * have foo/bar which now does not exist;\n@@ -959,12 +961,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tint last_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n-\t\t !refs_verify_refname_available(&refs->base, refname,\n-\t\t\t\t\t\t extras, skip, err))\n+\t\tif (resolve_errno != ENOTDIR ||\n+\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n+\t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n+\t\t\t\t refname, strerror(resolve_errno));\n \n \t\tgoto error_return;\n \t}\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex ac8a14086724..b9f713b5acd6 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type);\n \n+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n+ * failure. */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n+\n /*\n * Write an error to `err` and return a nonzero value iff the same\n * refname appears multiple times in `refnames`. `refnames` must be\n-- \ngitgitgadget\n\n"},{"id":"423278","messageId":"039fc4be4b901781bb5062cf63873a014a444e4c.1619710329.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH 5/8] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:05Z","receivedAt":"2021-04-29T15:32:19Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes the errno output of refs_read_raw_ref explicit.\nlock_raw_ref() now explicitly reads the errno output of refs_read_raw_ref.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs.c | 26 ++++++++++++--------------\n refs/files-backend.c | 8 ++++----\n refs/packed-backend.c | 10 ++++++----\n refs/refs-internal.h | 6 +++---\n 4 files changed, 25 insertions(+), 25 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex e8ce88040b3e..08f69e2a16f6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1671,20 +1671,17 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n-\tint result, failure;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &failure);\n-\terrno = failure;\n-\treturn result;\n+\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, failure_errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\n@@ -1725,9 +1722,10 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n+\t\tint read_failure = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n \t\t\t/* In reading mode, refs must eventually resolve */\n@@ -1739,9 +1737,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (read_failure != ENOENT && read_failure != EISDIR &&\n+\t\t\t read_failure != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -2253,7 +2250,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, NULL)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 3ab09871db5e..b6dc1b36c752 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -383,8 +383,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tif (lstat(path, &st) < 0) {\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -423,8 +423,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a457f18e93c8..03353ce48869 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -739,7 +739,8 @@ static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\t*failure_errno = ENOENT;\n+\t\tif (failure_errno)\n+\t\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \n@@ -1347,6 +1348,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1359,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure) ||\n+\t\t failure != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex b9f713b5acd6..6a0840e22772 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n * failure. */\n-- \ngitgitgadget\n\n"},{"id":"423279","messageId":"ebd7b8380bf7aed84d23973a4809d1441aa21692.1619710329.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:03Z","receivedAt":"2021-04-29T15:32:19Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs.c | 7 +++++--\n refs/debug.c | 4 ++--\n refs/files-backend.c | 24 ++++++++++++------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 16 +++++++++-------\n 5 files changed, 32 insertions(+), 27 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 261fd82beb98..43e2ad6b612a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1675,13 +1675,16 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type)\n {\n+\tint result, failure;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, &failure);\n+\terrno = failure;\n+\treturn result;\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 922e64fa6ad9..887dbb14be6e 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,14 +238,14 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n \n \toidcpy(oid, &null_oid);\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c9511da1d387..efe493ca1425 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,10 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\tif (failure_errno)\n+\t\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -541,6 +540,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno = 0;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -629,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -655,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -693,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex dfecdbc1db60..a457f18e93c8 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 29728a339fed..ac8a14086724 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,10 +617,12 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n- * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n- * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n- * reading the ref, set errno appropriately and return -1.\n+ * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT\n+ * and return -1. If the ref exists but is neither a symbolic ref nor an object\n+ * ID, it is broken; set REF_ISBROKEN in type, and return -1. For the files\n+ * backend, EISDIR and ENOTDIR may be set if the ref name is a directory. If\n+ * there is another error reading the ref, set failure_errno appropriately and\n+ * return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -634,9 +636,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \ngitgitgadget\n\n"},{"id":"423280","messageId":"9e161eeb5f6b6945499e036e27bd7cb511b336aa.1619710329.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH 8/8] refs: explicitly propagate errno from refs_read_raw_ref","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:08Z","receivedAt":"2021-04-29T15:32:20Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe function refs_resolve_ref_unsafe_with_errno should produce an errno output.\nRather than taking the value from the errno (which might contain garbage\nbeforehand), explicitly propagate the failure_errno coming out of\nrefs_read_raw_ref().\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs.c | 33 +++++++++++----------------------\n 1 file changed, 11 insertions(+), 22 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 597e4d1f18f9..a25d18873c56 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1684,12 +1684,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n-\t\t\t\t\t\t const char *refname,\n-\t\t\t\t\t\t int resolve_flags,\n-\t\t\t\t\t\t struct object_id *oid,\n-\t\t\t\t\t\t int *flags)\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n@@ -1702,6 +1701,7 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n \t\tflags = &unused_flags;\n \n \t*flags = 0;\n+\t*failure_errno = 0;\n \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n@@ -1728,6 +1728,8 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n \t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n+\t\t\t*failure_errno = read_failure;\n+\n \t\t\t/* In reading mode, refs must eventually resolve */\n \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n \t\t\t\treturn NULL;\n@@ -1779,22 +1781,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n \t\t\t\t int resolve_flags, struct object_id *oid,\n \t\t\t\t int *flags)\n {\n-\tconst char *result = refs_resolve_ref_unsafe_errno(\n-\t\trefs, refname, resolve_flags, oid, flags);\n-\terrno = 0;\n-\treturn result;\n-}\n-\n-const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n-\t\t\t\t\t const char *refname,\n-\t\t\t\t\t int resolve_flags,\n-\t\t\t\t\t struct object_id *oid,\n-\t\t\t\t\t int *flags, int *failure_errno)\n-{\n-\tconst char *result = refs_resolve_ref_unsafe_errno(\n-\t\trefs, refname, resolve_flags, oid, flags);\n-\t*failure_errno = errno;\n-\treturn result;\n+\tint ignore;\n+\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &ignore);\n }\n \n /* backend functions */\n-- \ngitgitgadget\n"},{"id":"423281","messageId":"1bb350ea5d21eacf95426192c89173b502b7e06a.1619710329.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe()","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:06Z","receivedAt":"2021-04-29T15:32:24Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is done in a separate commit, to pinpoint the precise cause should there be\nregressions in error reporting.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs.c | 23 +++++++++++++++++------\n 1 file changed, 17 insertions(+), 6 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 08f69e2a16f6..6e746cb01f24 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1685,10 +1685,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n }\n \n /* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n+\t\t\t\t\t\t const char *refname,\n+\t\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t\t int *flags)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n@@ -1777,14 +1778,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tconst char *result = refs_resolve_ref_unsafe_errno(\n+\t\trefs, refname, resolve_flags, oid, flags);\n+\terrno = 0;\n+\treturn result;\n+}\n+\n const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n \t\t\t\t\t const char *refname,\n \t\t\t\t\t int resolve_flags,\n \t\t\t\t\t struct object_id *oid,\n \t\t\t\t\t int *flags, int *failure_errno)\n {\n-\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n-\t\t\t\t\t\t resolve_flags, oid, flags);\n+\tconst char *result = refs_resolve_ref_unsafe_errno(\n+\t\trefs, refname, resolve_flags, oid, flags);\n \t*failure_errno = errno;\n \treturn result;\n }\n-- \ngitgitgadget\n\n"},{"id":"423282","messageId":"95d64d73353d8689e3928b8c9444490d0cdebfc9.1619710329.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-04-29T15:32:07Z","receivedAt":"2021-04-29T15:32:25Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe only caller of refs_resolve_ref_unsafe_with_errno() is in\nrefs/files-backend.c, and it only cares about EISDIR and ENOTDIR.\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs.c | 3 ---\n 1 file changed, 3 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 6e746cb01f24..597e4d1f18f9 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1706,7 +1706,6 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n \t\t\treturn NULL;\n \t\t}\n \n@@ -1766,7 +1765,6 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t\t !refname_is_safe(refname)) {\n-\t\t\t\terrno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \n@@ -1774,7 +1772,6 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n \t\t}\n \t}\n \n-\terrno = ELOOP;\n \treturn NULL;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"423313","messageId":"xmqqk0okebtb.fsf@gitster.g","threadId":"55589","inReplyTo":"7e8181e77d409af7595e357ad233b7781e026b78.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-04-30T02:38:40Z","receivedAt":"2021-04-30T02:38:50Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> From: Han-Wen Nienhuys \n>\n> A grep for EINVAL */*c reveals that no code inspects EINVAL after reading\n> references.\n\nWe often use a pattern (which is common) like this:\n\n\tif (some_func_in_ref_API(...) < 0) {\n\t\tif (errno == ENOENT || errno == EISDIR)\n\t\t\t... it is OK for the file to be missing ...\n\t\telse\n\t\t\t... error ...\n\t}\n\nIf a piece of code currently sets EINVAL to errno manually when\nsignalling a failure by returning a negative value to communicate\nwith such a caller, we wouldn't see EINVAL mentioned, so such a grep\nalone would not help us guarantee the correctness of an update to\nstop assignment of EINVAL at all. The callers must be vetted more\ncarefully than \"we are happy that nobody explicitly mentions EINVAL\".\n\n> The files ref backend does use EINVAL so parse_loose_ref_contents() can\n> communicate to lock_raw_ref() about garbage following the hex SHA1, or a short\n> read in files_read_raw_ref(), but the files backend does not call into\n> refs_read_raw_ref(), so its EINVAL sideband error is unused.\n\nThis paragraph is confusing. It says EINVAL is used to signal\nlock_raw_ref(), and it says EINVAL is not used by the same files\nbackend. Which is correct? If one part of the backend uses it, and\nother parts don't, wouldn't the backend as a whole still use it?\n\nHaving said that, ...\n\n> - * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n> - * and return -1. If the ref exists but is neither a symbolic ref nor\n> - * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n> - * EINVAL, and return -1. If there is another error reading the ref,\n> - * set errno appropriately and return -1.\n\n... the mention (and requirement) of EINVAL seems redundant, as it\nsounds sufficient for the caller to inspect 'type' to see if it is\nREF_ISBOKEN. So it may be OK for the code that gives REF_ISBROKEN\nto type *not* to set errno to EINVAL, as long as it won't leave it\nas ENOENT (meaning, an unrelated system call failed earlier may have\nset errno to ENOENT, and after having dealt with such an error, the\ncontrol may have reached to the codepath we are interested in\nhere---errno must be cleared to some value other than ENOENT, and\nassigning EINVAL is as good as any).\n\nThat is because there is a codeflow like this:\n\n\tif (files_read_raw_ref(...)) {\n\t\tif (errno == ENOENT) {\n\t\t\t... do various things ...\n\t\t} else if (errno == EISDIR) {\n\t\t\t... do different and various things ...\n\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n\t\t\t... deal with broken ref ...\n\t\t}\n\t\t ...\n\t}\n\nwhere errno is looked at.\n\n> + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n> + * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n> + * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n> + * reading the ref, set errno appropriately and return -1.\n\nSo, this is not sufficient to let caller correctly and safely handle\nerrors. \"set REF_ISBROKEN in type, set errno to something other\nthan ENOENT or EISDIR, and then return -1\" is necessary, I would\nthink.\n"},{"id":"423314","messageId":"xmqqbl9weabw.fsf@gitster.g","threadId":"55589","inReplyTo":"b2c72097e5e8985e7fdd8e3eee66cdf43d1b65c0.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-04-30T03:10:43Z","receivedAt":"2021-04-30T03:10:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"I 100% agree with you that errno is cumbersome to use and carries\nfar less information than we want (we do not learn what operation\nfailed on what path) over a long distance. It only is useful when\nthe callchain still knows what path was operated on.\n\nBut...\n\n\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> For the copy/rename support, calls to lock_ref_oid_basic() in this file are\n> followed by:\n>\n> * lock_ref_oid_basic (copy/rename rollback error path)\n>\n> * write_ref_to_lockfile (both in the rollback path and the success path of\n> copy/rename)\n>\n> These calls do not inspect the incoming errno. As they perform I/O, they can\n> clobber errno. For this reason, callers cannot reliably observe the errno that\n> lock_ref_oid_basic() generated, so it is unsound for programmatic use.\n\nIn the latter part of the above, \"callers\" refers to the callers of\n\"the copy/rename support\" (aka files_copy_or_rename_ref())?\n\nThen I am not sure why \"callers cannot reliably observe the errno\nthat lock_ref_oid_basic() generated\" is a problem. They will see\nthe errno from the last system call that failed, if they care. So\ntheir performing I/O is perfectly acceptable, too.\n\nHence, I am not sure what change the above justifies, if any.\n\nIf we can show that no caller of files_copy_or_rename_ref() uses\nerrno, it is a clear indication that lock_ref_oid_basic() is saving\nand restoring errno for no good reason. I think that is what was\ndone for the other two callers below.\n\nSo I traced what happens after the copy-rename thing gets called.\n\nrefs_rename_ref(), rename_ref(), refs_copy_existing_ref() and\ncopy_existing_ref() (all in refs.c) should be the only callers of\nthe function. All callers in builtin/branch.c and builtin/remote.c\nof these functions (by the way, refs_X() variants do not seem to be\ncalled from anywhere---are they over-engineered?) just die() when\nthey signal a failure by returning non-zero, so I think it is safe\nand much easier to understand to justify this change like so:\n\n refs/files-backend.c::lock_ref_oid_basic() tries hard to signal\n how it failed to its callers using errno. The three callers of\n this file-scope static function are \n\n * files_copy_or_rename_ref()\n * files_create_symref()\n * files_reflog_expire()\n\n None of them looks at errno after seeing a negative return from\n lock_ref_oid_basic() to make any decision, and no caller of\n these three functions looks at errno after they signal a failure\n by returning a negative value.\n\n> For files_create_symref() and files_reflog_expire(), grepping over callers\n> showed no callers inspecting errno.\n\nYes, this is a lot more relevant justification to allow these two\nfunctions, and functions that are called _only_ _by_ these two\nfunctions, stop worrying about errno.\n"},{"id":"423316","messageId":"xmqq35v8e982.fsf@gitster.g","threadId":"55589","inReplyTo":"ebd7b8380bf7aed84d23973a4809d1441aa21692.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-04-30T03:34:37Z","receivedAt":"2021-04-30T03:34:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> From: Han-Wen Nienhuys \n>\n> read_raw_ref_fn needs to supply a credible errno for a number of cases. These\n> are primarily:\n>\n> 1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\n> resulting error codes to create/remove directories as needed.\n>\n> 2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\n> returning the last successfully resolved symref. This is necessary so\n> read_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\n> du-jour), and we know on which branch to create the first commit.\n>\n> Make this information flow explicit by adding a failure_errno to the signature\n> of read_raw_ref. All errnos from the files backend are still propagated\n> unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\n> relevant.\n\nI like the general direction to move away from errno, which may make\nsense only in the context of files backend (e.g. ENOENT would be\nleft in errno, only because the original \"loose ref\" implementation\nused one file per ref, when a ref we wanted to see did not exist)\nand having other backends use the same errno would not make much\nsense, as it is not the goal to make other backends like reftable\nemulate files backend.\n\nI wonder if in the ideal world, we'd rather want to define our own\nenum, not , that is specific to failure modes of ref API\nfunctions and signal failures by returning these values (and the\nenum includes 0 as a value to signal success, all other errors are\nnegative values).\n\nWhat I am really getting at is if we need an extra \"failure\"\nout-parameter-pointer in the internal API. I do not mind if it\nhelps the transition to the ideal world, but I offhand do not see if\nwe need result and failure as separate variables.\n\n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> refs.c | 7 +++++--\n> refs/debug.c | 4 ++--\n> refs/files-backend.c | 24 ++++++++++++------------\n> refs/packed-backend.c | 8 ++++----\n> refs/refs-internal.h | 16 +++++++++-------\n> 5 files changed, 32 insertions(+), 27 deletions(-)\n>\n> diff --git a/refs.c b/refs.c\n> index 261fd82beb98..43e2ad6b612a 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1675,13 +1675,16 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n> \t\t const char *refname, struct object_id *oid,\n> \t\t struct strbuf *referent, unsigned int *type)\n> {\n> +\tint result, failure;\n> \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n> \t\t\t\t\t type);\n> \t}\n> \n> -\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> -\t\t\t\t\t type);\n> +\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> +\t\t\t\t\t type, &failure);\n> +\terrno = failure;\n> +\treturn result;\n> }\n> \n> /* This function needs to return a meaningful errno on failure */\n> diff --git a/refs/debug.c b/refs/debug.c\n> index 922e64fa6ad9..887dbb14be6e 100644\n> --- a/refs/debug.c\n> +++ b/refs/debug.c\n> @@ -238,14 +238,14 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n> \n> static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> \t\t\t struct object_id *oid, struct strbuf *referent,\n> -\t\t\t unsigned int *type)\n> +\t\t\t unsigned int *type, int *failure_errno)\n> {\n> \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n> \tint res = 0;\n> \n> \toidcpy(oid, &null_oid);\n> \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n> -\t\t\t\t\t type);\n> +\t\t\t\t\t type, failure_errno);\n> \n> \tif (res == 0) {\n> \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index c9511da1d387..efe493ca1425 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n> \treturn refs->loose;\n> }\n> \n> -static int files_read_raw_ref(struct ref_store *ref_store,\n> -\t\t\t const char *refname, struct object_id *oid,\n> -\t\t\t struct strbuf *referent, unsigned int *type)\n> +static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> +\t\t\t struct object_id *oid, struct strbuf *referent,\n> +\t\t\t unsigned int *type, int *failure_errno)\n> {\n> \tstruct files_ref_store *refs =\n> \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n> @@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n> \tstruct stat st;\n> \tint fd;\n> \tint ret = -1;\n> -\tint save_errno;\n> \tint remaining_retries = 3;\n> \n> \t*type = 0;\n> @@ -459,10 +458,10 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n> \tret = parse_loose_ref_contents(buf, oid, referent, type);\n> \n> out:\n> -\tsave_errno = errno;\n> +\tif (failure_errno)\n> +\t\t*failure_errno = errno;\n> \tstrbuf_release(&sb_path);\n> \tstrbuf_release(&sb_contents);\n> -\terrno = save_errno;\n> \treturn ret;\n> }\n> \n> @@ -541,6 +540,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n> \tstruct strbuf ref_file = STRBUF_INIT;\n> \tint attempts_remaining = 3;\n> \tint ret = TRANSACTION_GENERIC_ERROR;\n> +\tint failure_errno = 0;\n> \n> \tassert(err);\n> \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n> @@ -629,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n> \t * fear that its value will change.\n> \t */\n> \n> -\tif (files_read_raw_ref(&refs->base, refname,\n> -\t\t\t &lock->old_oid, referent, type)) {\n> -\t\tif (errno == ENOENT) {\n> +\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n> +\t\t\t type, &failure_errno)) {\n> +\t\tif (failure_errno == ENOENT) {\n> \t\t\tif (mustexist) {\n> \t\t\t\t/* Garden variety missing reference. */\n> \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n> @@ -655,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n> \t\t\t\t * reference named \"refs/foo/bar/baz\".\n> \t\t\t\t */\n> \t\t\t}\n> -\t\t} else if (errno == EISDIR) {\n> +\t\t} else if (failure_errno == EISDIR) {\n> \t\t\t/*\n> \t\t\t * There is a directory in the way. It might have\n> \t\t\t * contained references that have been deleted. If\n> @@ -693,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,\n> \t\t\t\t\tgoto error_return;\n> \t\t\t\t}\n> \t\t\t}\n> -\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n> +\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n> \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n> \t\t\t\t \"reference broken\", refname);\n> \t\t\tgoto error_return;\n> \t\t} else {\n> \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n> -\t\t\t\t refname, strerror(errno));\n> +\t\t\t\t refname, strerror(failure_errno));\n> \t\t\tgoto error_return;\n> \t\t}\n> \n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index dfecdbc1db60..a457f18e93c8 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n> \treturn refs->snapshot;\n> }\n> \n> -static int packed_read_raw_ref(struct ref_store *ref_store,\n> -\t\t\t const char *refname, struct object_id *oid,\n> -\t\t\t struct strbuf *referent, unsigned int *type)\n> +static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> +\t\t\t struct object_id *oid, struct strbuf *referent,\n> +\t\t\t unsigned int *type, int *failure_errno)\n> {\n> \tstruct packed_ref_store *refs =\n> \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n> @@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n> \n> \tif (!rec) {\n> \t\t/* refname is not a packed reference. */\n> -\t\terrno = ENOENT;\n> +\t\t*failure_errno = ENOENT;\n> \t\treturn -1;\n> \t}\n> \n> diff --git a/refs/refs-internal.h b/refs/refs-internal.h\n> index 29728a339fed..ac8a14086724 100644\n> --- a/refs/refs-internal.h\n> +++ b/refs/refs-internal.h\n> @@ -617,10 +617,12 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n> * properly-formatted or even safe reference name. NEITHER INPUT NOR\n> * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n> *\n> - * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n> - * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n> - * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n> - * reading the ref, set errno appropriately and return -1.\n> + * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT\n> + * and return -1. If the ref exists but is neither a symbolic ref nor an object\n> + * ID, it is broken; set REF_ISBROKEN in type, and return -1. For the files\n> + * backend, EISDIR and ENOTDIR may be set if the ref name is a directory. If\n> + * there is another error reading the ref, set failure_errno appropriately and\n> + * return -1.\n> *\n> * Backend-specific flags might be set in type as well, regardless of\n> * outcome.\n> @@ -634,9 +636,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n> * - in all other cases, referent will be untouched, and therefore\n> * refname will still be valid and unchanged.\n> */\n> -typedef int read_raw_ref_fn(struct ref_store *ref_store,\n> -\t\t\t const char *refname, struct object_id *oid,\n> -\t\t\t struct strbuf *referent, unsigned int *type);\n> +typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n> +\t\t\t struct object_id *oid, struct strbuf *referent,\n> +\t\t\t unsigned int *type, int *failure_errno);\n> \n> struct ref_storage_be {\n> \tstruct ref_storage_be *next;\n"},{"id":"423322","messageId":"xmqqeeescntb.fsf@gitster.g","threadId":"55589","inReplyTo":"xmqq35v8e982.fsf@gitster.g","subject":"Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-04-30T06:02:24Z","receivedAt":"2021-04-30T06:02:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> I like the general direction to move away from errno, which may make\n> sense only in the context of files backend (e.g. ENOENT would be\n> left in errno, only because the original \"loose ref\" implementation\n> used one file per ref, when a ref we wanted to see did not exist)\n> and having other backends use the same errno would not make much\n> sense, as it is not the goal to make other backends like reftable\n> emulate files backend.\n>\n> I wonder if in the ideal world, we'd rather want to define our own\n> enum, not , that is specific to failure modes of ref API\n> functions and signal failures by returning these values (and the\n> enum includes 0 as a value to signal success, all other errors are\n> negative values).\n>\n> What I am really getting at is if we need an extra \"failure\"\n> out-parameter-pointer in the internal API. I do not mind if it\n> helps the transition to the ideal world, but I offhand do not see if\n> we need result and failure as separate variables.\n\nIn any case, the change in the function signature helped me catch\nthat this series invalidates what has been queued on hn/reftable\nwithout updating that topic to align with the new way to handle the\nerrno (it would have become a silent semantic failure if we instead\nencoded the \"failure\" in the return value).\n\nThe following is what I am tentatively using for tonight's\nintegration when merging this and hn/reftable together to the 'seen'\nbranch.\n\nThanks.\n\n\n refs/reftable-backend.c | 14 ++++++++------\n 1 file changed, 8 insertions(+), 6 deletions(-)\n\ndiff --git c/refs/reftable-backend.c w/refs/reftable-backend.c\nindex 1aff21adb4..4385d2d6f5 100644\n--- c/refs/reftable-backend.c\n+++ w/refs/reftable-backend.c\n@@ -52,7 +52,8 @@ static struct reftable_stack *stack_for(struct git_reftable_ref_store *store,\n static int git_reftable_read_raw_ref(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n \t\t\t\t struct strbuf *referent,\n-\t\t\t\t unsigned int *type);\n+\t\t\t\t unsigned int *type,\n+\t\t\t\t int *failure_errno);\n \n static void clear_reftable_log_record(struct reftable_log_record *log)\n {\n@@ -376,7 +377,8 @@ static int fixup_symrefs(struct ref_store *ref_store,\n \t\t\t\t\t\t&old_oid, &referent,\n \t\t\t\t\t\t/* mutate input, like\n \t\t\t\t\t\t files-backend.c */\n-\t\t\t\t\t\t&update->type);\n+\t\t\t\t\t\t&update->type,\n+\t\t\t\t\t\t&errno);\n \t\tif (err < 0 && errno == ENOENT &&\n \t\t is_null_oid(&update->old_oid)) {\n \t\t\terr = 0;\n@@ -1538,7 +1540,7 @@ static int reftable_error_to_errno(int err)\n static int git_reftable_read_raw_ref(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n \t\t\t\t struct strbuf *referent,\n-\t\t\t\t unsigned int *type)\n+\t\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct git_reftable_ref_store *refs =\n \t\t(struct git_reftable_ref_store *)ref_store;\n@@ -1560,12 +1562,12 @@ static int git_reftable_read_raw_ref(struct ref_store *ref_store,\n \n \terr = reftable_stack_read_ref(stack, refname, &ref);\n \tif (err > 0) {\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\terr = -1;\n \t\tgoto done;\n \t}\n \tif (err < 0) {\n-\t\terrno = reftable_error_to_errno(err);\n+\t\t*failure_errno = reftable_error_to_errno(err);\n \t\terr = -1;\n \t\tgoto done;\n \t}\n@@ -1578,7 +1580,7 @@ static int git_reftable_read_raw_ref(struct ref_store *ref_store,\n \t\toidread(oid, reftable_ref_record_val1(&ref));\n \t} else {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\terr = -1;\n \t}\n done:\n"},{"id":"424956","messageId":"CAFQ2z_MCySh2p+fL0uL8tAQhkpbnkGF6e=NOYicpAxN582aqag@mail.gmail.com","threadId":"55589","inReplyTo":"xmqqk0okebtb.fsf@gitster.g","subject":"Re: [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-05-19T12:25:37Z","receivedAt":"2021-05-19T12:25:51Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Fri, Apr 30, 2021 at 4:38 AM Junio C Hamano wrote:\n>\n> \"Han-Wen Nienhuys via GitGitGadget\" writes:\n>\n> > From: Han-Wen Nienhuys \n> >\n> > A grep for EINVAL */*c reveals that no code inspects EINVAL after reading\n> > references.\n>\n> We often use a pattern (which is common) like this:\n>\n> if (some_func_in_ref_API(...) < 0) {\n> if (errno == ENOENT || errno == EISDIR)\n> ... it is OK for the file to be missing ...\n> else\n> ... error ...\n> }\n>\n> If a piece of code currently sets EINVAL to errno manually when\n> signalling a failure by returning a negative value to communicate\n> with such a caller, we wouldn't see EINVAL mentioned, so such a grep\n> alone would not help us guarantee the correctness of an update to\n> stop assignment of EINVAL at all. The callers must be vetted more\n> carefully than \"we are happy that nobody explicitly mentions EINVAL\".\n\nSure. I looked at the callers, and documented further what I looked\nfor. But how far should one go? It's a global variable, so\ntransitively, almost all of the code could be observing the EINVAL\nvalue under very specific circumstances. But it would also be a\nterrible, fragile coding style and use of undocumented behavior.\n\n> > The files ref backend does use EINVAL so parse_loose_ref_contents() can\n> > communicate to lock_raw_ref() about garbage following the hex SHA1, or a short\n> > read in files_read_raw_ref(), but the files backend does not call into\n> > refs_read_raw_ref(), so its EINVAL sideband error is unused.\n>\n> This paragraph is confusing. It says EINVAL is used to signal\n> lock_raw_ref(), and it says EINVAL is not used by the same files\n> backend. Which is correct? If one part of the backend uses it, and\n> other parts don't, wouldn't the backend as a whole still use it?\n\nI tried to clarify the message. files-backend.c makes assumptions\nabout the errno return for files_read_raw_ref, but it's not making\nassumptions about the abstracted API in refs.h\n\n> That is because there is a codeflow like this:\n>\n> if (files_read_raw_ref(...)) {\n> if (errno == ENOENT) {\n> ... do various things ...\n> } else if (errno == EISDIR) {\n> ... do different and various things ...\n> } else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n> ... deal with broken ref ...\n> }\n> ...\n> }\n\nas mentioned above, this isn't calling refs_read_raw_ref, so it's not\naffected by this patch.\n\n> > + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n> > + * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n> > + * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n> > + * reading the ref, set errno appropriately and return -1.\n>\n> So, this is not sufficient to let caller correctly and safely handle\n> errors. \"set REF_ISBROKEN in type, set errno to something other\n> than ENOENT or EISDIR, and then return -1\" is necessary, I would\n> think.\n\nI tweaked the comment. Note that the function has only a handful of\ncallers (and only one caller where this behavior is relevant), and\nit's changed in a follow-on patch in this series. Is it worth the\neffort to wordsmith this further?\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"424957","messageId":"CAFQ2z_MVEFpq5qJx_BwbborFmUeaGhRN4N4cN-yiS1xToEw1LA@mail.gmail.com","threadId":"55589","inReplyTo":"xmqqbl9weabw.fsf@gitster.g","subject":"Re: [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-05-19T12:29:21Z","receivedAt":"2021-05-19T12:29:36Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Fri, Apr 30, 2021 at 5:10 AM Junio C Hamano wrote:\n>\n> I 100% agree with you that errno is cumbersome to use and carries\n> far less information than we want (we do not learn what operation\n> failed on what path) over a long distance. It only is useful when\n> the callchain still knows what path was operated on.\n>\n> But...\n>\n> \"Han-Wen Nienhuys via GitGitGadget\" writes:\n>\n> > For the copy/rename support, calls to lock_ref_oid_basic() in this file are\n> > followed by:\n> >\n> > * lock_ref_oid_basic (copy/rename rollback error path)\n> >\n> > * write_ref_to_lockfile (both in the rollback path and the success path of\n> > copy/rename)\n> >\n> > These calls do not inspect the incoming errno. As they perform I/O, they can\n> > clobber errno. For this reason, callers cannot reliably observe the errno that\n> > lock_ref_oid_basic() generated, so it is unsound for programmatic use.\n>\n> In the latter part of the above, \"callers\" refers to the callers of\n> \"the copy/rename support\" (aka files_copy_or_rename_ref())?\n>\n> Then I am not sure why \"callers cannot reliably observe the errno\n> that lock_ref_oid_basic() generated\" is a problem. They will see\n> the errno from the last system call that failed, if they care. So\n> their performing I/O is perfectly acceptable, too.\n>\n> Hence, I am not sure what change the above justifies, if any.\n>\n> If we can show that no caller of files_copy_or_rename_ref() uses\n> errno, it is a clear indication that lock_ref_oid_basic() is saving\n> and restoring errno for no good reason. I think that is what was\n> done for the other two callers below.\n>\n> So I traced what happens after the copy-rename thing gets called.\n>\n> refs_rename_ref(), rename_ref(), refs_copy_existing_ref() and\n> copy_existing_ref() (all in refs.c) should be the only callers of\n> the function. All callers in builtin/branch.c and builtin/remote.c\n> of these functions (by the way, refs_X() variants do not seem to be\n> called from anywhere---are they over-engineered?) just die() when\n> they signal a failure by returning non-zero, so I think it is safe\n> and much easier to understand to justify this change like so:\n>\n> refs/files-backend.c::lock_ref_oid_basic() tries hard to signal\n> how it failed to its callers using errno. The three callers of\n> this file-scope static function are\n>\n> * files_copy_or_rename_ref()\n> * files_create_symref()\n> * files_reflog_expire()\n>\n> None of them looks at errno after seeing a negative return from\n> lock_ref_oid_basic() to make any decision, and no caller of\n> these three functions looks at errno after they signal a failure\n> by returning a negative value.\n\nI stole your message here; hope that's OK. My original message tries\nto convey that if you do\n\n/* should return errno */\nint a() { .. }\n\nint b() {\n result = a();\n maybe_do_IO();\n return result;\n}\n\nthen callers of b() can't reason about the errno result of a(),\nbecause they can't know if an error code was generated by\nmaybe_do_IO() or a(). This means that the errno result of a() is\nuseless. (This is assuming that b() doesn't inspect errno, which I\nfailed to mention.)\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"424958","messageId":"CAFQ2z_PcxOmtg2ZvvhS6TTv+UOw1H+pzAxSd_-JMGQ-GYgWHgQ@mail.gmail.com","threadId":"55589","inReplyTo":"xmqqeeescntb.fsf@gitster.g","subject":"Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-05-19T12:33:59Z","receivedAt":"2021-05-19T12:34:16Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Fri, Apr 30, 2021 at 8:02 AM Junio C Hamano wrote:\n> > I like the general direction to move away from errno, which may make\n> > sense only in the context of files backend (e.g. ENOENT would be\n> > left in errno, only because the original \"loose ref\" implementation\n> > used one file per ref, when a ref we wanted to see did not exist)\n> > and having other backends use the same errno would not make much\n> > sense, as it is not the goal to make other backends like reftable\n> > emulate files backend.\n> >\n> > I wonder if in the ideal world, we'd rather want to define our own\n> > enum, not , that is specific to failure modes of ref API\n> > functions and signal failures by returning these values (and the\n> > enum includes 0 as a value to signal success, all other errors are\n> > negative values).\n\nI think it would be healthy to have a set of canonical error codes in\ngeneral, but I don't know if this will help here. The files and\npacked backend want to communicate about very specific conditions\n(ENOENT, EISDIR, ENOTDIR), which seem too low-level to put in a\ngeneric error code.\n\n> In any case, the change in the function signature helped me catch\n> that this series invalidates what has been queued on hn/reftable\n> without updating that topic to align with the new way to handle the\n> errno (it would have become a silent semantic failure if we instead\n> encoded the \"failure\" in the return value).\n>\n> The following is what I am tentatively using for tonight's\n> integration when merging this and hn/reftable together to the 'seen'\n> branch.\n\nThat sounds right. How should I post the next update to hn/reftable ?\n(based on which branch?)\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"426281","messageId":"20210603021356.342665-1-jonathantanmy@google.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/8] refs: cleanup errno sideband ref related functions","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-06-03T02:13:56Z","receivedAt":"2021-06-03T02:14:18Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> v3\n> \n> * remove errno as an implicit communication mechanism from refs support\n> completely.\n\nLooking at all the patches, it seems that this patch set is about\nfunctions in Git code that set errno themselves to indicate the category\nof failure encountered, and instead of setting errno, we want them to\ntransmit that information through an out param instead. I notice that\nthe cover letter talks about \"cleanup\" and \"remove errno\", but that\ncould have been explained in greater detail, I think.\n\nAs for whether it is a good idea overall, it could be said that errno is\nidiomatic in C and writing \"if (myfunc()) { store_errno = errno; ...\" is\nnot much more difficult than \"int store_errno; if (myfunc(&store_errno))\n{ ...\". But presumably Han-Wen thinks the opposite, since he wrote this\npatch set, and I'll defer to his opinion on this since he's working on\nthe ref code and I'm not. Besides that, having the out param is more\nexplicit (which might be better) and permits chaining of such functions\n(e.g. \"if (myfunc(&store_errno) || myotherfunc(&store_errno))\"). I'll\nreview this patch set as if this is the approach we want to take.\n"},{"id":"426282","messageId":"20210603021927.343351-1-jonathantanmy@google.com","threadId":"55589","inReplyTo":"7e8181e77d409af7595e357ad233b7781e026b78.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-06-03T02:19:27Z","receivedAt":"2021-06-03T02:19:44Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> From: Han-Wen Nienhuys \n> \n> A grep for EINVAL */*c reveals that no code inspects EINVAL after reading\n> references.\n> \n> The files ref backend does use EINVAL so parse_loose_ref_contents() can\n> communicate to lock_raw_ref() about garbage following the hex SHA1, or a short\n> read in files_read_raw_ref(), but the files backend does not call into\n> refs_read_raw_ref(), so its EINVAL sideband error is unused.\n\nDoes this mean that there is some code that sets EINVAL, but no code\nuses it? If yes, that seems to mean that we can't remove EINVAL from the\ndocumentation, since some code still sets it.\n\n> As the errno sideband is unintuitive and error-prone, remove EINVAL\n> value, as a step towards getting rid of the errno sideband altogether.\n\nHow is removing one possible value a step towards getting rid of the\nerrno sideband? I would have thought that we would be working towards\ntransmitting all existing values to the new sideband (the out param).\nUnless we are planning to get rid of all values in the sideband - in\nwhich case, this should be described in the commit message.\n\n[snip]\n\n> ---\n> refs/refs-internal.h | 9 ++++-----\n> 1 file changed, 4 insertions(+), 5 deletions(-)\n> \n> diff --git a/refs/refs-internal.h b/refs/refs-internal.h\n> index 467f4b3c936d..29728a339fed 100644\n> --- a/refs/refs-internal.h\n> +++ b/refs/refs-internal.h\n> @@ -617,11 +617,10 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n> * properly-formatted or even safe reference name. NEITHER INPUT NOR\n> * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n> *\n> - * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n> - * and return -1. If the ref exists but is neither a symbolic ref nor\n> - * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n> - * EINVAL, and return -1. If there is another error reading the ref,\n> - * set errno appropriately and return -1.\n> + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n> + * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n> + * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n> + * reading the ref, set errno appropriately and return -1.\n\nThe rewrapping was unnecessary and makes it hard to see what changed.\n"},{"id":"426284","messageId":"20210603023304.344682-1-jonathantanmy@google.com","threadId":"55589","inReplyTo":"b2c72097e5e8985e7fdd8e3eee66cdf43d1b65c0.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-06-03T02:33:04Z","receivedAt":"2021-06-03T02:34:08Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> From: Han-Wen Nienhuys \n> \n> Errno is a global variable written by almost all system calls, and therefore it\n> is hard to reason about its state. It's also useless for user-visible errors, as\n> it leaves no place to report the offending file and/or syscall.\n\nI don't think this paragraph is useful.\n\n> For the copy/rename support, calls to lock_ref_oid_basic() in this file are\n> followed by:\n> \n> * lock_ref_oid_basic (copy/rename rollback error path)\n> \n> * write_ref_to_lockfile (both in the rollback path and the success path of\n> copy/rename)\n> \n> These calls do not inspect the incoming errno. As they perform I/O, they can\n> clobber errno. For this reason, callers cannot reliably observe the errno that\n> lock_ref_oid_basic() generated, so it is unsound for programmatic use.\n> \n> For files_create_symref() and files_reflog_expire(), grepping over callers\n> showed no callers inspecting errno.\n\nThis is probably written more clearly as follows:\n\n No call to the static function lock_ref_oid_basic() is immediately\n followed by an errno check, so stopping setting errno is safe. But as a\n sanity check, lock_ref_oid_basic() is used in these functions:\n - files_copy_or_rename_ref() - here, calls are followed by error() (which\n performs I/O) or write_ref_to_lockfile() (which calls parse_object() which\n may perform I/O)\n - files_create_symref() - here, calls are followed by error() or\n create_symref_locked() (which performs I/O and does not inspect\n errno)\n - files_reflog_expire() - here, calls are followed by error() or\n refs_reflog_exists() (which calls a function in a vtable that is not\n documented to use and/or preserve errno)\n\n So it is safe to stop setting errno in lock_ref_oid_basic().\n\nThe diff itself looks good.\n"},{"id":"426285","messageId":"20210603023709.345324-1-jonathantanmy@google.com","threadId":"55589","inReplyTo":"ebd7b8380bf7aed84d23973a4809d1441aa21692.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-06-03T02:37:09Z","receivedAt":"2021-06-03T02:38:22Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> From: Han-Wen Nienhuys \n> \n> read_raw_ref_fn needs to supply a credible errno for a number of cases. These\n> are primarily:\n> \n> 1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\n> resulting error codes to create/remove directories as needed.\n> \n> 2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\n> returning the last successfully resolved symref. This is necessary so\n> read_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\n> du-jour), and we know on which branch to create the first commit.\n> \n> Make this information flow explicit by adding a failure_errno to the signature\n> of read_raw_ref. All errnos from the files backend are still propagated\n> unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\n> relevant.\n\nLooks good.\n\n> @@ -1675,13 +1675,16 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n> \t\t const char *refname, struct object_id *oid,\n> \t\t struct strbuf *referent, unsigned int *type)\n> {\n> +\tint result, failure;\n> \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n> \t\t\t\t\t type);\n> \t}\n> \n> -\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> -\t\t\t\t\t type);\n> +\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> +\t\t\t\t\t type, &failure);\n> +\terrno = failure;\n> +\treturn result;\n> }\n\nShould we initialize \"failure\" to 0 here? As a reader, I would assume\nthat \"failure\" has the semantics of errno here, which upon success, may\nor may not be set.\n"},{"id":"426288","messageId":"20210603025120.348393-1-jonathantanmy@google.com","threadId":"55589","inReplyTo":"dd3eceade4fcf09a679fdf20dddae247b4e0e879.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-06-03T02:51:20Z","receivedAt":"2021-06-03T02:51:54Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> From: Han-Wen Nienhuys \n> \n> This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\n> contract for the errno output explicit. The implementation still relies on\n> the global errno variable to ensure no side effects.\n\nLooking at the next few patches, I think the plan is:\n\n - Introduce a new function refs_resolve_ref_unsafe_with_errno() which\n returns the error information in an out param instead of errno. Right\n now, it wraps refs_resolve_ref_unsafe().\n - Migrate all callers that require the error information to\n refs_resolve_ref_unsafe_with_errno(), and leave all callers that do\n not require the error information alone (that is, using\n refs_resolve_ref_unsafe()).\n - To ensure that all callers using refs_resolve_ref_unsafe() really do\n not use errno, set it to 0. (This is allowed by the errno contract -\n successes can set errno to whatever they want.)\n\nIf this is the plan, it should be written in the commit message.\n\nAs it is, this patch handles the first and maybe part of the second\nstep, and patch 5 handles the rest of the second step? I think the\npatches should be more clearly divided.\n"},{"id":"426289","messageId":"20210603025340.349224-1-jonathantanmy@google.com","threadId":"55589","inReplyTo":"1bb350ea5d21eacf95426192c89173b502b7e06a.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe()","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-06-03T02:53:40Z","receivedAt":"2021-06-03T02:54:43Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> @@ -1685,10 +1685,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> }\n> \n> /* This function needs to return a meaningful errno on failure */\n> -const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n> -\t\t\t\t const char *refname,\n> -\t\t\t\t int resolve_flags,\n> -\t\t\t\t struct object_id *oid, int *flags)\n> +static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n> +\t\t\t\t\t\t const char *refname,\n> +\t\t\t\t\t\t int resolve_flags,\n> +\t\t\t\t\t\t struct object_id *oid,\n> +\t\t\t\t\t\t int *flags)\n\nSo a third function (refs_resolve_ref_unsafe_errno() - not to be\nconfused with refs_resolve_ref_unsafe_with_errno(), which has an extra\n\"with\")? Couldn't we just swap the other 2 functions directly instead of\ngoing through this intermediary?\n\n> +const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n> +\t\t\t\t int resolve_flags, struct object_id *oid,\n> +\t\t\t\t int *flags)\n> +{\n> +\tconst char *result = refs_resolve_ref_unsafe_errno(\n> +\t\trefs, refname, resolve_flags, oid, flags);\n> +\terrno = 0;\n> +\treturn result;\n> +}\n\nThis is the errno = 0 part that I was talking about in my review of patch 4.\n"},{"id":"426290","messageId":"20210603025553.349758-1-jonathantanmy@google.com","threadId":"55589","inReplyTo":"95d64d73353d8689e3928b8c9444490d0cdebfc9.1619710329.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-06-03T02:55:53Z","receivedAt":"2021-06-03T02:56:13Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> From: Han-Wen Nienhuys \n> \n> The only caller of refs_resolve_ref_unsafe_with_errno() is in\n> refs/files-backend.c, and it only cares about EISDIR and ENOTDIR.\n> \n> Signed-off-by: Han-Wen Nienhuys \n> ---\n> refs.c | 3 ---\n> 1 file changed, 3 deletions(-)\n> \n> diff --git a/refs.c b/refs.c\n> index 6e746cb01f24..597e4d1f18f9 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1706,7 +1706,6 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n> \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n> \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n> \t\t !refname_is_safe(refname)) {\n> -\t\t\terrno = EINVAL;\n> \t\t\treturn NULL;\n> \t\t}\n> \n\nI don't think this is related to avoiding errno and conveying error\ninformation through an out param. But besides that, as it is, I'm not\nsure that this is correct. Even if EINVAL is not checked by the caller,\nsetting errno to EINVAL here means avoiding exposing a potential\nEISDIR/ENOTDIR that a preceding call set. Same comment for the other\ninstances.\n"},{"id":"426875","messageId":"CAFQ2z_OHWSJT6Dz8iMSBaGG_=8KvCKCeaj8Dds0jBBQD5KheJQ@mail.gmail.com","threadId":"55589","inReplyTo":"20210603021927.343351-1-jonathantanmy@google.com","subject":"Re: [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-06-09T11:28:31Z","receivedAt":"2021-06-09T11:29:01Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jun 3, 2021 at 4:19 AM Jonathan Tan wrote:\n>\n> > From: Han-Wen Nienhuys \n> >\n> > A grep for EINVAL */*c reveals that no code inspects EINVAL after reading\n> > references.\n> >\n> > The files ref backend does use EINVAL so parse_loose_ref_contents() can\n> > communicate to lock_raw_ref() about garbage following the hex SHA1, or a short\n> > read in files_read_raw_ref(), but the files backend does not call into\n> > refs_read_raw_ref(), so its EINVAL sideband error is unused.\n>\n> Does this mean that there is some code that sets EINVAL, but no code\n> uses it? If yes, that seems to mean that we can't remove EINVAL from the\n> documentation, since some code still sets it.\n\nIt means that an alternate ref backend doesn't have to return EINVAL\nto work properly.\n\nadded to commit message.\n\n> > As the errno sideband is unintuitive and error-prone, remove EINVAL\n> > value, as a step towards getting rid of the errno sideband altogether.\n>\n> How is removing one possible value a step towards getting rid of the\n> errno sideband? I would have thought that we would be working towards\n> transmitting all existing values to the new sideband (the out param).\n> Unless we are planning to get rid of all values in the sideband - in\n> which case, this should be described in the commit message.\n\nIt means that we don't have to spend braincycles in further commits of\nthis series reasoning about EINVAL.\n\nThe existing functions potentially transmit all kinds of errors, eg.\nEIO if there was a problem with the media, or ENOTCONN if the git repo\nhappens to be on a FUSE filesystem that crashed. These are not\nrelevant to the ref backend, so we don't treat them specially either.\n\n> > - * set errno appropriately and return -1.\n> > + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n> > + * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n> > + * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n> > + * reading the ref, set errno appropriately and return -1.\n>\n> The rewrapping was unnecessary and makes it hard to see what changed.\n\nFixed. I suppose .clang-format settings should add some configuration\nabout the line size for wrapping comments.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"426876","messageId":"CAFQ2z_NQSbshLSstkBM9OZ7XWvydSQ=x0GCHj0a8mf1OC4t2jw@mail.gmail.com","threadId":"55589","inReplyTo":"20210603021356.342665-1-jonathantanmy@google.com","subject":"Re: [PATCH 0/8] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-06-09T11:29:32Z","receivedAt":"2021-06-09T11:29:48Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jun 3, 2021 at 4:14 AM Jonathan Tan wrote:\n>\n> > v3\n> >\n> > * remove errno as an implicit communication mechanism from refs support\n> > completely.\n>\n> Looking at all the patches, it seems that this patch set is about\n> functions in Git code that set errno themselves to indicate the category\n> of failure encountered, and instead of setting errno, we want them to\n> transmit that information through an out param instead. I notice that\n> the cover letter talks about \"cleanup\" and \"remove errno\", but that\n> could have been explained in greater detail, I think.\n\nWhat is the right place to document these considerations? The data in\nthe cover letter doesn't end up in the project history, so it seems\nlike the wrong place.\n\n> As for whether it is a good idea overall, it could be said that errno is\n> idiomatic in C and writing \"if (myfunc()) { store_errno = errno; ...\" is\n> not much more difficult than \"int store_errno; if (myfunc(&store_errno))\n> { ...\". But presumably Han-Wen thinks the opposite, since he wrote this\n> patch set, and I'll defer to his opinion on this since he's working on\n> the ref code and I'm not. Besides that, having the out param is more\n> explicit (which might be better) and permits chaining of such functions\n> (e.g. \"if (myfunc(&store_errno) || myotherfunc(&store_errno))\"). I'll\n> review this patch set as if this is the approach we want to take.\n\n\nit's certainly idiomatic and easy for writing. The problem is that it\nmakes reading code much more difficult: since all of the code has\naccess to errno, it's hard to tell which code depends on certain\nwrites to errno. One signal of this are the extensive discussions in\nthis series about how to argue (in the commit message) that a certain\ncode change is safe.\n\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"426972","messageId":"CAFQ2z_N5B4FC0z8_EHH9msBewQEZ1iyzgWbi7GJB0h9ztD-kQw@mail.gmail.com","threadId":"55589","inReplyTo":"20210603023304.344682-1-jonathantanmy@google.com","subject":"Re: [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-06-10T10:02:07Z","receivedAt":"2021-06-10T10:03:32Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jun 3, 2021 at 4:33 AM Jonathan Tan wrote:\n> > Errno is a global variable written by almost all system calls, and therefore it\n> > is hard to reason about its state. It's also useless for user-visible errors, as\n> > it leaves no place to report the offending file and/or syscall.\n>\n> I don't think this paragraph is useful.\n\nDropped.\n\n> This is probably written more clearly as follows:\n>\n> No call to the static function lock_ref_oid_basic() is immediately\n> followed by an errno check, so stopping setting errno is safe. But as a\n> sanity check, lock_ref_oid_basic() is used in these functions:\n> - files_copy_or_rename_ref() - here, calls are followed by error() (which\n> performs I/O) or write_ref_to_lockfile() (which calls parse_object() which\n> may perform I/O)\n> - files_create_symref() - here, calls are followed by error() or\n> create_symref_locked() (which performs I/O and does not inspect\n> errno)\n> - files_reflog_expire() - here, calls are followed by error() or\n> refs_reflog_exists() (which calls a function in a vtable that is not\n> documented to use and/or preserve errno)\n>\n> So it is safe to stop setting errno in lock_ref_oid_basic().\n\nI used some of the above text.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"426973","messageId":"CAFQ2z_N--3UeLFg7acPX6pmvvoRs0oKyBUE3zebb05G4m7_a5Q@mail.gmail.com","threadId":"55589","inReplyTo":"20210603023709.345324-1-jonathantanmy@google.com","subject":"Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-06-10T10:05:21Z","receivedAt":"2021-06-10T10:06:34Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jun 3, 2021 at 4:37 AM Jonathan Tan wrote:\n> Should we initialize \"failure\" to 0 here? As a reader, I would assume\n> that \"failure\" has the semantics of errno here, which upon success, may\n> or may not be set.\n\nDone.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"426981","messageId":"CAFQ2z_OH=HGThdHZV+=K3uctfj-DRZ0GeCrCb7thsF91igJR_g@mail.gmail.com","threadId":"55589","inReplyTo":"20210603025120.348393-1-jonathantanmy@google.com","subject":"Re: [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-06-10T11:27:07Z","receivedAt":"2021-06-10T11:27:36Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jun 3, 2021 at 4:51 AM Jonathan Tan wrote:\n> > This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\n> > contract for the errno output explicit. The implementation still relies on\n> > the global errno variable to ensure no side effects.\n>\n> Looking at the next few patches, I think the plan is:\n>\n> - Introduce a new function refs_resolve_ref_unsafe_with_errno() which\n> returns the error information in an out param instead of errno. Right\n> now, it wraps refs_resolve_ref_unsafe().\n> - Migrate all callers that require the error information to\n> refs_resolve_ref_unsafe_with_errno(), and leave all callers that do\n> not require the error information alone (that is, using\n> refs_resolve_ref_unsafe()).\n> - To ensure that all callers using refs_resolve_ref_unsafe() really do\n> not use errno, set it to 0. (This is allowed by the errno contract -\n> successes can set errno to whatever they want.)\n>\n> If this is the plan, it should be written in the commit message.\n\nI added more description to the commit message.\n\n> As it is, this patch handles the first and maybe part of the second\n> step, and patch 5 handles the rest of the second step? I think the\n> patches should be more clearly divided.\n\nDone.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"426982","messageId":"CAFQ2z_PL3Envot1oAg7-vwWyx6taJfD4W_1PEByfDFT8kXgOUA@mail.gmail.com","threadId":"55589","inReplyTo":"20210603025340.349224-1-jonathantanmy@google.com","subject":"Re: [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe()","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-06-10T11:45:58Z","receivedAt":"2021-06-10T11:46:12Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jun 3, 2021 at 4:53 AM Jonathan Tan wrote:\n>\n> > @@ -1685,10 +1685,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> > }\n> >\n> > /* This function needs to return a meaningful errno on failure */\n> > -const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n> > - const char *refname,\n> > - int resolve_flags,\n> > - struct object_id *oid, int *flags)\n> > +static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n> > + const char *refname,\n> > + int resolve_flags,\n> > + struct object_id *oid,\n> > + int *flags)\n>\n> So a third function (refs_resolve_ref_unsafe_errno() - not to be\n> confused with refs_resolve_ref_unsafe_with_errno(), which has an extra\n> \"with\")? Couldn't we just swap the other 2 functions directly instead of\n> going through this intermediary?\n\nI've clarified the name. I've done it this way, because it keeps the\ndiff small. Swapping the functions would require code changes that I\nthought would be more work to review.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"426983","messageId":"CAFQ2z_OfRGG5kF2OhYrm5OLMOOmz-dHd+-ocWwC_7Tt+yBYmkg@mail.gmail.com","threadId":"55589","inReplyTo":"20210603025553.349758-1-jonathantanmy@google.com","subject":"Re: [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-06-10T11:58:48Z","receivedAt":"2021-06-10T11:59:17Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jun 3, 2021 at 4:55 AM Jonathan Tan wrote:\n> I don't think this is related to avoiding errno and conveying error\n> information through an out param. But besides that, as it is, I'm not\n> sure that this is correct. Even if EINVAL is not checked by the caller,\n> setting errno to EINVAL here means avoiding exposing a potential\n> EISDIR/ENOTDIR that a preceding call set. Same comment for the other\n> instances.\n\nYou are right, but it's probably moot, because the follow-up commit\nstops using errno (making it impossible for EISDIR/ENOTDIR to\nhaphazardly appear), but I dropped this commit from the series.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"426987","messageId":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.git.git.1619710329.gitgitgadget@gmail.com","subject":"[PATCH v2 0/8] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:40Z","receivedAt":"2021-06-10T12:58:01Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"v4\n\n * commit msg tweaks in response to Jun.\n\nHan-Wen Nienhuys (8):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: make errno output explicit for refs_resolve_ref_unsafe\n refs: use refs_resolve_ref_unsafe_with_errno() where needed\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: clear errno return in refs_resolve_ref_unsafe()\n refs: explicitly propagate errno from refs_read_raw_ref\n\n refs.c | 51 +++++++++++++++++++++++--------------\n refs/debug.c | 4 +--\n refs/files-backend.c | 58 +++++++++++++++++++------------------------\n refs/packed-backend.c | 16 ++++++------\n refs/refs-internal.h | 31 +++++++++++++++--------\n 5 files changed, 90 insertions(+), 70 deletions(-)\n\n\nbase-commit: ebf3c04b262aa27fbb97f8a0156c2347fecafafb\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v2\nPull-Request: https://github.com/git/git/pull/1012\n\nRange-diff vs v1:\n\n 1: 7e8181e77d40 ! 1: f9b92e62b598 refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn\n @@ Metadata\n Author: Han-Wen Nienhuys \n \n ## Commit message ##\n - refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn\n + refs: remove EINVAL errno output from specification of read_raw_ref_fn\n \n - A grep for EINVAL */*c reveals that no code inspects EINVAL after reading\n - references.\n + This commit does not change code; it documents the fact that an alternate ref\n + backend does not need to return EINVAL from read_raw_ref_fn to function\n + properly.\n \n - The files ref backend does use EINVAL so parse_loose_ref_contents() can\n - communicate to lock_raw_ref() about garbage following the hex SHA1, or a short\n - read in files_read_raw_ref(), but the files backend does not call into\n - refs_read_raw_ref(), so its EINVAL sideband error is unused.\n + This is correct, because refs_read_raw_ref is only called from;\n +\n + * resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n +\n + * refs_verify_refname_available(), which does not inspect errno.\n +\n + * files-backend.c, where errno is overwritten on failure.\n +\n + * packed-backend.c (is_packed_transaction_needed), which calls it for the\n + packed ref backend, which never emits EINVAL.\n +\n + A grep for EINVAL */*c reveals that no code checks errno against EINVAL after\n + reading references. In addition, the refs.h file does not mention errno at all.\n +\n + A grep over resolve_ref_unsafe() turned up the following callers that inspect\n + errno:\n +\n + * sequencer.c::print_commit_summary, which uses it for die_errno\n +\n + * lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n +\n + The files ref backend does use EINVAL. The files backend does not call into\n + the generic API (refs_read_raw), but into the files-specific function\n + (files_read_raw_ref), which we are not changing in this commit.\n \n As the errno sideband is unintuitive and error-prone, remove EINVAL\n value, as a step towards getting rid of the errno sideband altogether.\n @@ Commit message\n Spotted by Ævar Arnfjörð Bjarmason .\n \n Signed-off-by: Han-Wen Nienhuys \n + Reviewed-by: Jonathan Tan \n \n ## refs/refs-internal.h ##\n @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,\n - * properly-formatted or even safe reference name. NEITHER INPUT NOR\n - * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n -- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n -- * and return -1. If the ref exists but is neither a symbolic ref nor\n + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n + * and return -1. If the ref exists but is neither a symbolic ref nor\n - * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n - * EINVAL, and return -1. If there is another error reading the ref,\n - * set errno appropriately and return -1.\n -+ * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n -+ * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n -+ * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n -+ * reading the ref, set errno appropriately and return -1.\n ++ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n ++ * (errno should not be ENOENT) If there is another error reading the\n ++ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n 2: b2c72097e5e8 ! 2: cbe09a48036c refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ Metadata\n ## Commit message ##\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n \n - Errno is a global variable written by almost all system calls, and therefore it\n - is hard to reason about its state. It's also useless for user-visible errors, as\n - it leaves no place to report the offending file and/or syscall.\n + refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\n + to its callers using errno.\n \n - For the copy/rename support, calls to lock_ref_oid_basic() in this file are\n - followed by:\n + It is safe to stop setting errno here, because the callers of this\n + file-scope static function are\n \n - * lock_ref_oid_basic (copy/rename rollback error path)\n + * files_copy_or_rename_ref()\n + * files_create_symref()\n + * files_reflog_expire()\n \n - * write_ref_to_lockfile (both in the rollback path and the success path of\n - copy/rename)\n + None of them looks at errno after seeing a negative return from\n + lock_ref_oid_basic() to make any decision, and no caller of these three\n + functions looks at errno after they signal a failure by returning a\n + negative value. In particular,\n \n - These calls do not inspect the incoming errno. As they perform I/O, they can\n - clobber errno. For this reason, callers cannot reliably observe the errno that\n - lock_ref_oid_basic() generated, so it is unsound for programmatic use.\n + * files_copy_or_rename_ref() - here, calls are followed by error()\n + (which performs I/O) or write_ref_to_lockfile() (which calls\n + parse_object() which may perform I/O)\n \n - For files_create_symref() and files_reflog_expire(), grepping over callers\n - showed no callers inspecting errno.\n + * files_create_symref() - here, calls are followed by error() or\n + create_symref_locked() (which performs I/O and does not inspect\n + errno)\n +\n + * files_reflog_expire() - here, calls are followed by error() or\n + refs_reflog_exists() (which calls a function in a vtable that is not\n + documented to use and/or preserve errno)\n \n Signed-off-by: Han-Wen Nienhuys \n + Reviewed-by: Jonathan Tan \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)\n 3: ebd7b8380bf7 ! 3: 3e2831e59c8e refs: make errno output explicit for read_raw_ref_fn\n @@ Commit message\n relevant.\n \n Signed-off-by: Han-Wen Nienhuys \n + Reviewed-by: Jonathan Tan \n \n ## refs.c ##\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n \n -\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n -\t\t\t\t\t type);\n ++\tfailure = 0;\n +\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n +\t\t\t\t\t type, &failure);\n +\terrno = failure;\n @@ refs/debug.c: debug_ref_iterator_begin(struct ref_store *ref_store, const char *\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n - \n - \toidcpy(oid, &null_oid);\n +@@ refs/debug.c: static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \toidcpy(oid, null_oid());\n + \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n -\t\t\t\t\t type);\n +\t\t\t\t\t type, failure_errno);\n @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n -- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return\n -- * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is\n -- * broken; set REF_ISBROKEN in type, and return -1. If there is another error\n -- * reading the ref, set errno appropriately and return -1.\n +- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n +- * and return -1. If the ref exists but is neither a symbolic ref nor\n +- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n +- * (errno should not be ENOENT) If there is another error reading the\n +- * ref, set errno appropriately and return -1.\n + * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT\n + * and return -1. If the ref exists but is neither a symbolic ref nor an object\n -+ * ID, it is broken; set REF_ISBROKEN in type, and return -1. For the files\n -+ * backend, EISDIR and ENOTDIR may be set if the ref name is a directory. If\n -+ * there is another error reading the ref, set failure_errno appropriately and\n -+ * return -1.\n ++ * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno\n ++ * should not be ENOENT). The files backend may return EISDIR (if the ref name\n ++ * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is\n ++ * another error reading the ref, set failure_errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n -: ------------ > 4: 11b2184044d7 refs: make errno output explicit for refs_resolve_ref_unsafe\n 4: dd3eceade4fc ! 5: 005ee8e6fb2a refs: make errno output explicit for refs_resolve_ref_unsafe\n @@ Metadata\n Author: Han-Wen Nienhuys \n \n ## Commit message ##\n - refs: make errno output explicit for refs_resolve_ref_unsafe\n + refs: use refs_resolve_ref_unsafe_with_errno() where needed\n \n - This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\n - contract for the errno output explicit. The implementation still relies on\n - the global errno variable to ensure no side effects.\n + lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\n + that needs error information to make logic decisions.\n \n Signed-off-by: Han-Wen Nienhuys \n -\n - ## refs.c ##\n -@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n - \treturn NULL;\n - }\n - \n -+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n -+\t\t\t\t\t const char *refname,\n -+\t\t\t\t\t int resolve_flags,\n -+\t\t\t\t\t struct object_id *oid,\n -+\t\t\t\t\t int *flags, int *failure_errno)\n -+{\n -+\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n -+\t\t\t\t\t\t resolve_flags, oid, flags);\n -+\t*failure_errno = errno;\n -+\treturn result;\n -+}\n -+\n - /* backend functions */\n - int refs_init_db(struct strbuf *err)\n - {\n -\n - ## refs.h ##\n -@@ refs.h: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n - \t\t\t\t int resolve_flags,\n - \t\t\t\t struct object_id *oid,\n - \t\t\t\t int *flags);\n -+\n - const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n - \t\t\t struct object_id *oid, int *flags);\n - \n + Reviewed-by: Jonathan Tan \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n -+\tint resolve_errno;\n ++\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \n \t\tgoto error_return;\n \t}\n -\n - ## refs/refs-internal.h ##\n -@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store,\n - \t\t const char *refname, struct object_id *oid,\n - \t\t struct strbuf *referent, unsigned int *type);\n - \n -+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n -+ * failure. */\n -+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n -+\t\t\t\t\t const char *refname,\n -+\t\t\t\t\t int resolve_flags,\n -+\t\t\t\t\t struct object_id *oid,\n -+\t\t\t\t\t int *flags, int *failure_errno);\n -+\n - /*\n - * Write an error to `err` and return a nonzero value iff the same\n - * refname appears multiple times in `refnames`. `refnames` must be\n 5: 039fc4be4b90 ! 6: 2b346caf1aed refs: add failure_errno to refs_read_raw_ref() signature\n @@ Commit message\n refs: add failure_errno to refs_read_raw_ref() signature\n \n This makes the errno output of refs_read_raw_ref explicit.\n - lock_raw_ref() now explicitly reads the errno output of refs_read_raw_ref.\n \n Signed-off-by: Han-Wen Nienhuys \n + Reviewed-by: Jonathan Tan \n \n ## refs.c ##\n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n +\t\t unsigned int *type, int *failure_errno)\n {\n -\tint result, failure;\n ++\tif (failure_errno)\n ++\t\t*failure_errno = 0;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n +-\tfailure = 0;\n -\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n -\t\t\t\t\t type, &failure);\n -\terrno = failure;\n 6: 1bb350ea5d21 ! 7: d86516219689 refs: clear errno return in refs_resolve_ref_unsafe()\n @@ Commit message\n This is done in a separate commit, to pinpoint the precise cause should there be\n regressions in error reporting.\n \n + This is implemented by renaming the existing logic to a static function\n + refs_resolve_unsafe_implicit_errno(), minimizing the code diff.\n +\n Signed-off-by: Han-Wen Nienhuys \n + Reviewed-by: Jonathan Tan \n \n ## refs.c ##\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n -\t\t\t\t const char *refname,\n -\t\t\t\t int resolve_flags,\n -\t\t\t\t struct object_id *oid, int *flags)\n -+static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n -+\t\t\t\t\t\t const char *refname,\n -+\t\t\t\t\t\t int resolve_flags,\n -+\t\t\t\t\t\t struct object_id *oid,\n -+\t\t\t\t\t\t int *flags)\n ++static const char *\n ++refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n ++\t\t\t\t const char *refname, int resolve_flags,\n ++\t\t\t\t struct object_id *oid, int *flags)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n +\t\t\t\t int resolve_flags, struct object_id *oid,\n +\t\t\t\t int *flags)\n +{\n -+\tconst char *result = refs_resolve_ref_unsafe_errno(\n ++\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n +\t\trefs, refname, resolve_flags, oid, flags);\n +\terrno = 0;\n +\treturn result;\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n {\n -\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n -\t\t\t\t\t\t resolve_flags, oid, flags);\n -+\tconst char *result = refs_resolve_ref_unsafe_errno(\n ++\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n +\t\trefs, refname, resolve_flags, oid, flags);\n \t*failure_errno = errno;\n \treturn result;\n 7: 95d64d73353d < -: ------------ refs: stop setting EINVAL and ELOOP in symref resolution\n 8: 9e161eeb5f6b ! 8: 2a9ebe43deac refs: explicitly propagate errno from refs_read_raw_ref\n @@ Commit message\n refs_read_raw_ref().\n \n Signed-off-by: Han-Wen Nienhuys \n + Reviewed-by: Jonathan Tan \n \n ## refs.c ##\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n }\n \n -/* This function needs to return a meaningful errno on failure */\n --static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n --\t\t\t\t\t\t const char *refname,\n --\t\t\t\t\t\t int resolve_flags,\n --\t\t\t\t\t\t struct object_id *oid,\n --\t\t\t\t\t\t int *flags)\n +-static const char *\n +-refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n +-\t\t\t\t const char *refname, int resolve_flags,\n +-\t\t\t\t struct object_id *oid, int *flags)\n +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n +\t\t\t\t\t const char *refname,\n +\t\t\t\t\t int resolve_flags,\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n -@@ refs.c: static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n +@@ refs.c: refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n \t\tflags = &unused_flags;\n \n \t*flags = 0;\n @@ refs.c: static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n -@@ refs.c: static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n + \t\t !refname_is_safe(refname)) {\n +-\t\t\terrno = EINVAL;\n ++\t\t\t*failure_errno = EINVAL;\n + \t\t\treturn NULL;\n + \t\t}\n + \n +@@ refs.c: refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n \t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n @@ refs.c: static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,\n \t\t\t/* In reading mode, refs must eventually resolve */\n \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n \t\t\t\treturn NULL;\n +@@ refs.c: refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n + \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n + \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n + \t\t\t !refname_is_safe(refname)) {\n +-\t\t\t\terrno = EINVAL;\n ++\t\t\t\t*failure_errno = EINVAL;\n + \t\t\t\treturn NULL;\n + \t\t\t}\n + \n +@@ refs.c: refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n + \t\t}\n + \t}\n + \n +-\terrno = ELOOP;\n ++\t*failure_errno = ELOOP;\n + \treturn NULL;\n + }\n + \n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n \t\t\t\t int resolve_flags, struct object_id *oid,\n \t\t\t\t int *flags)\n {\n --\tconst char *result = refs_resolve_ref_unsafe_errno(\n +-\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n -\t\trefs, refname, resolve_flags, oid, flags);\n -\terrno = 0;\n -\treturn result;\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *\n -\t\t\t\t\t struct object_id *oid,\n -\t\t\t\t\t int *flags, int *failure_errno)\n -{\n --\tconst char *result = refs_resolve_ref_unsafe_errno(\n +-\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n -\t\trefs, refname, resolve_flags, oid, flags);\n -\t*failure_errno = errno;\n -\treturn result;\n\n-- \ngitgitgadget\n"},{"id":"426988","messageId":"005ee8e6fb2a0f1a38801bed1dfe082999bfec14.1623329869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v2 5/8] refs: use refs_resolve_ref_unsafe_with_errno() where needed","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:45Z","receivedAt":"2021-06-10T12:58:05Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs/files-backend.c | 19 ++++++++++---------\n 1 file changed, 10 insertions(+), 9 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8f969c8f711f..5a430aabf623 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -924,6 +924,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -936,10 +937,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t refname, resolve_flags,\n-\t\t\t\t\t &lock->old_oid, type);\n-\tif (!resolved && errno == EISDIR) {\n+\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\t\tresolve_flags,\n+\t\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t\t&resolve_errno);\n+\tif (!resolved && resolve_errno == EISDIR) {\n \t\t/*\n \t\t * we are trying to lock foo but we used to\n \t\t * have foo/bar which now does not exist;\n@@ -959,12 +961,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tint last_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n-\t\t !refs_verify_refname_available(&refs->base, refname,\n-\t\t\t\t\t\t extras, skip, err))\n+\t\tif (resolve_errno != ENOTDIR ||\n+\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n+\t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n+\t\t\t\t refname, strerror(resolve_errno));\n \n \t\tgoto error_return;\n \t}\n-- \ngitgitgadget\n\n"},{"id":"426989","messageId":"11b2184044d766183edf5ee1888f38945e8a7fd1.1623329869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v2 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:44Z","receivedAt":"2021-06-10T12:58:17Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nIn a follow-on commits, we will\n\n* migrate callers that need this error information\n\n* clear errno in refs_resolve_ref_unsafe() to make sure these other callers\naren't using the error output accidentally.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs.c | 12 ++++++++++++\n refs/refs-internal.h | 8 ++++++++\n 2 files changed, 20 insertions(+)\n\ndiff --git a/refs.c b/refs.c\nindex bebe3f584da7..64e2d55adcfb 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1781,6 +1781,18 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n+{\n+\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n+\t\t\t\t\t\t resolve_flags, oid, flags);\n+\t*failure_errno = errno;\n+\treturn result;\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 904b2a9e51ae..eb97023658f8 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type);\n \n+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n+ * failure. */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n+\n /*\n * Write an error to `err` and return a nonzero value iff the same\n * refname appears multiple times in `refnames`. `refnames` must be\n-- \ngitgitgadget\n\n"},{"id":"426990","messageId":"2a9ebe43deac3da9baf241ff1e494f8210c27981.1623329869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:48Z","receivedAt":"2021-06-10T12:58:19Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe function refs_resolve_ref_unsafe_with_errno should produce an errno output.\nRather than taking the value from the errno (which might contain garbage\nbeforehand), explicitly propagate the failure_errno coming out of\nrefs_read_raw_ref().\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs.c | 38 ++++++++++++++------------------------\n 1 file changed, 14 insertions(+), 24 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 191cbf5a330f..92c4796078bb 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1686,11 +1686,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-static const char *\n-refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n-\t\t\t\t const char *refname, int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n@@ -1703,11 +1703,12 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n \t\tflags = &unused_flags;\n \n \t*flags = 0;\n+\t*failure_errno = 0;\n \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n+\t\t\t*failure_errno = EINVAL;\n \t\t\treturn NULL;\n \t\t}\n \n@@ -1730,6 +1731,8 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n \t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n+\t\t\t*failure_errno = read_failure;\n+\n \t\t\t/* In reading mode, refs must eventually resolve */\n \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n \t\t\t\treturn NULL;\n@@ -1767,7 +1770,7 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t\t !refname_is_safe(refname)) {\n-\t\t\t\terrno = EINVAL;\n+\t\t\t\t*failure_errno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \n@@ -1775,7 +1778,7 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n \t\t}\n \t}\n \n-\terrno = ELOOP;\n+\t*failure_errno = ELOOP;\n \treturn NULL;\n }\n \n@@ -1783,22 +1786,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n \t\t\t\t int resolve_flags, struct object_id *oid,\n \t\t\t\t int *flags)\n {\n-\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n-\t\trefs, refname, resolve_flags, oid, flags);\n-\terrno = 0;\n-\treturn result;\n-}\n-\n-const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n-\t\t\t\t\t const char *refname,\n-\t\t\t\t\t int resolve_flags,\n-\t\t\t\t\t struct object_id *oid,\n-\t\t\t\t\t int *flags, int *failure_errno)\n-{\n-\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n-\t\trefs, refname, resolve_flags, oid, flags);\n-\t*failure_errno = errno;\n-\treturn result;\n+\tint ignore;\n+\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &ignore);\n }\n \n /* backend functions */\n-- \ngitgitgadget\n"},{"id":"426992","messageId":"f9b92e62b5987e8ab51cac07ee1c44d3ec02f7a0.1623329869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v2 1/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:41Z","receivedAt":"2021-06-10T12:58:52Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 467f4b3c936d..f4445e329045 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \ngitgitgadget\n\n"},{"id":"426993","messageId":"cbe09a48036c0befafa0f26f72d188dc765f5b7b.1623329869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:42Z","receivedAt":"2021-06-10T12:58:54Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs/files-backend.c | 9 +--------\n 1 file changed, 1 insertion(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 677b7e4cdd2d..6aa0f5c41dd3 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t * to remain.\n \t\t */\n \t\tif (remove_empty_directories(&ref_file)) {\n-\t\t\tlast_errno = errno;\n \t\t\tif (!refs_verify_refname_available(\n \t\t\t\t\t &refs->base,\n \t\t\t\t\t refname, extras, skip, err))\n@@ -962,7 +959,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tlast_errno = errno;\n+\t\tint last_errno = errno;\n \t\tif (last_errno != ENOTDIR ||\n \t\t !refs_verify_refname_available(&refs->base, refname,\n \t\t\t\t\t\t extras, skip, err))\n@@ -981,20 +978,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n \t\t\t\t\t extras, skip, err)) {\n-\t\tlast_errno = ENOTDIR;\n \t\tgoto error_return;\n \t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n \n \tif (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {\n-\t\tlast_errno = errno;\n \t\tgoto error_return;\n \t}\n \tgoto out;\n@@ -1005,7 +999,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"426994","messageId":"3e2831e59c8e4cb8aef416a41b55083887b4559c.1623329869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:43Z","receivedAt":"2021-06-10T12:59:08Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs.c | 8 ++++++--\n refs/debug.c | 4 ++--\n refs/files-backend.c | 24 ++++++++++++------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 17 +++++++++--------\n 5 files changed, 33 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 8c9490235ea6..bebe3f584da7 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1675,13 +1675,17 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type)\n {\n+\tint result, failure;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\tfailure = 0;\n+\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, &failure);\n+\terrno = failure;\n+\treturn result;\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 7db4abccc341..f12413a9bc0f 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \toidcpy(oid, null_oid());\n \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 6aa0f5c41dd3..8f969c8f711f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,10 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\tif (failure_errno)\n+\t\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -541,6 +540,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno = 0;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -629,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -655,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -693,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex dfecdbc1db60..a457f18e93c8 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex f4445e329045..904b2a9e51ae 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,11 +617,12 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT\n+ * and return -1. If the ref exists but is neither a symbolic ref nor an object\n+ * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno\n+ * should not be ENOENT). The files backend may return EISDIR (if the ref name\n+ * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is\n+ * another error reading the ref, set failure_errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -635,9 +636,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \ngitgitgadget\n\n"},{"id":"426995","messageId":"2b346caf1aed265ca4787de4208375bd5cfb7bc7.1623329869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v2 6/8] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:46Z","receivedAt":"2021-06-10T12:59:19Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes the errno output of refs_read_raw_ref explicit.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs.c | 29 ++++++++++++++---------------\n refs/files-backend.c | 8 ++++----\n refs/packed-backend.c | 10 ++++++----\n refs/refs-internal.h | 6 +++---\n 4 files changed, 27 insertions(+), 26 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 64e2d55adcfb..ed2dde1c0c6d 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1671,21 +1671,19 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n-\tint result, failure;\n+\tif (failure_errno)\n+\t\t*failure_errno = 0;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\tfailure = 0;\n-\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &failure);\n-\terrno = failure;\n-\treturn result;\n+\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, failure_errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\n@@ -1726,9 +1724,10 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n+\t\tint read_failure = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n \t\t\t/* In reading mode, refs must eventually resolve */\n@@ -1740,9 +1739,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (read_failure != ENOENT && read_failure != EISDIR &&\n+\t\t\t read_failure != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -2254,7 +2252,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, NULL)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 5a430aabf623..01c9bd0dbf04 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -383,8 +383,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tif (lstat(path, &st) < 0) {\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -423,8 +423,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a457f18e93c8..03353ce48869 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -739,7 +739,8 @@ static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\t*failure_errno = ENOENT;\n+\t\tif (failure_errno)\n+\t\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \n@@ -1347,6 +1348,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1359,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure) ||\n+\t\t failure != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex eb97023658f8..c65d26580ce8 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n * failure. */\n-- \ngitgitgadget\n\n"},{"id":"426996","messageId":"d86516219689e9179f3ba18c2ad2391d8ca27076.1623329869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v2 7/8] refs: clear errno return in refs_resolve_ref_unsafe()","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-06-10T12:57:47Z","receivedAt":"2021-06-10T12:59:21Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis is done in a separate commit, to pinpoint the precise cause should there be\nregressions in error reporting.\n\nThis is implemented by renaming the existing logic to a static function\nrefs_resolve_unsafe_implicit_errno(), minimizing the code diff.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs.c | 22 ++++++++++++++++------\n 1 file changed, 16 insertions(+), 6 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex ed2dde1c0c6d..191cbf5a330f 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1687,10 +1687,10 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n }\n \n /* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+static const char *\n+refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n+\t\t\t\t const char *refname, int resolve_flags,\n+\t\t\t\t struct object_id *oid, int *flags)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n@@ -1779,14 +1779,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n+\t\trefs, refname, resolve_flags, oid, flags);\n+\terrno = 0;\n+\treturn result;\n+}\n+\n const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n \t\t\t\t\t const char *refname,\n \t\t\t\t\t int resolve_flags,\n \t\t\t\t\t struct object_id *oid,\n \t\t\t\t\t int *flags, int *failure_errno)\n {\n-\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n-\t\t\t\t\t\t resolve_flags, oid, flags);\n+\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n+\t\trefs, refname, resolve_flags, oid, flags);\n \t*failure_errno = errno;\n \treturn result;\n }\n-- \ngitgitgadget\n\n"},{"id":"427283","messageId":"CAFQ2z_NgAvvPTL2khMjVBSfUyBBA__pWS9NABHvGxTOyc0KnTQ@mail.gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 0/8] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-06-14T10:10:05Z","receivedAt":"2021-06-14T10:10:19Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jun 10, 2021 at 2:57 PM Han-Wen Nienhuys via GitGitGadget\n wrote:\n> v4\n>\n> * commit msg tweaks in response to Jun.\n\nI forgot to update the cover letter, but this includes the updates in\nresponse to review by Jonathan Tan.\n\nI think this topic could graduate?\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"428887","messageId":"87k0mae0ga.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"cbe09a48036c0befafa0f26f72d188dc765f5b7b.1623329869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-01T11:13:50Z","receivedAt":"2021-07-01T11:30:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\n> to its callers using errno.\n>\n> It is safe to stop setting errno here, because the callers of this\n> file-scope static function are\n>\n> * files_copy_or_rename_ref()\n> * files_create_symref()\n> * files_reflog_expire()\n>\n> None of them looks at errno after seeing a negative return from\n> lock_ref_oid_basic() to make any decision, and no caller of these three\n> functions looks at errno after they signal a failure by returning a\n> negative value. In particular,\n>\n> * files_copy_or_rename_ref() - here, calls are followed by error()\n> (which performs I/O) or write_ref_to_lockfile() (which calls\n> parse_object() which may perform I/O)\n>\n> * files_create_symref() - here, calls are followed by error() or\n> create_symref_locked() (which performs I/O and does not inspect\n> errno)\n>\n> * files_reflog_expire() - here, calls are followed by error() or\n> refs_reflog_exists() (which calls a function in a vtable that is not\n> documented to use and/or preserve errno)\n>\n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> ---\n> refs/files-backend.c | 9 +--------\n> 1 file changed, 1 insertion(+), 8 deletions(-)\n\nThis all looks good and well justified after the last commit (where we\neven mentioned refs_verify_refname_available() explicitly), but...\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 677b7e4cdd2d..6aa0f5c41dd3 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)\n> \n> /*\n> * Locks a ref returning the lock on success and NULL on failure.\n> - * On failure errno is set to something meaningful.\n> */\n> static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \t\t\t\t\t const char *refname,\n> @@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> {\n> \tstruct strbuf ref_file = STRBUF_INIT;\n> \tstruct ref_lock *lock;\n> -\tint last_errno = 0;\n> \tint mustexist = (old_oid && !is_null_oid(old_oid));\n> \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n> \tint resolved;\n> @@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \t\t * to remain.\n> \t\t */\n> \t\tif (remove_empty_directories(&ref_file)) {\n> -\t\t\tlast_errno = errno;\n> \t\t\tif (!refs_verify_refname_available(\n> \t\t\t\t\t &refs->base,\n> \t\t\t\t\t refname, extras, skip, err))\n> @@ -962,7 +959,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \t\t\t\t\t\t &lock->old_oid, type);\n> \t}\n> \tif (!resolved) {\n> -\t\tlast_errno = errno;\n> +\t\tint last_errno = errno;\n> \t\tif (last_errno != ENOTDIR ||\n> \t\t !refs_verify_refname_available(&refs->base, refname,\n> \t\t\t\t\t\t extras, skip, err))\n\n...this particular change gives me some pause, because all the rest is\nabout squirreling away our own errno for our own caller (which it turns\nout, we didn't need).\n\nBut in this case we're only guarding against\nrefs_verify_refname_available() possibly clobbering the errno we just\ngot on !resolved in refs_verify_refname_available().\n\nSo instead I'd expect either this on top:\n\t\n\tdiff --git a/refs/files-backend.c b/refs/files-backend.c\n\tindex 6aa0f5c41dd..28aa4932529 100644\n\t--- a/refs/files-backend.c\n\t+++ b/refs/files-backend.c\n\t@@ -959,12 +959,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n\t \t\t\t\t\t\t &lock->old_oid, type);\n\t \t}\n\t \tif (!resolved) {\n\t-\t\tint last_errno = errno;\n\t-\t\tif (last_errno != ENOTDIR ||\n\t+\t\tif (errno != ENOTDIR ||\n\t \t\t !refs_verify_refname_available(&refs->base, refname,\n\t \t\t\t\t\t\t extras, skip, err))\n\t \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n\t-\t\t\t\t refname, strerror(last_errno));\n\t+\t\t\t\t refname, strerror(errno));\n\t \n\t \t\tgoto error_return;\n\t \t}\n\nOr, if we are actually worried about the errno being reset as we report\nit:\n\t\n\tdiff --git a/refs/files-backend.c b/refs/files-backend.c\n\tindex 6aa0f5c41dd..8ee6af61f1a 100644\n\t--- a/refs/files-backend.c\n\t+++ b/refs/files-backend.c\n\t@@ -960,11 +960,20 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n\t \t}\n\t \tif (!resolved) {\n\t \t\tint last_errno = errno;\n\t+\t\terrno = 0;\n\t \t\tif (last_errno != ENOTDIR ||\n\t \t\t !refs_verify_refname_available(&refs->base, refname,\n\t-\t\t\t\t\t\t extras, skip, err))\n\t-\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n\t-\t\t\t\t refname, strerror(last_errno));\n\t+\t\t\t\t\t\t extras, skip, err)) {\n\t+\t\t\tif (errno)\n\t+\t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': '%s', \"\n\t+\t\t\t\t\t \"also got '%s when reporting the error!\",\n\t+\t\t\t\t\t refname, strerror(last_errno),\n\t+\t\t\t\t\t strerror(errno));\n\t+\t\t\telse\n\t+\t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n\t+\t\t\t\t\t refname, strerror(errno));\n\t+\n\t+\t\t}\n\t \n\t \t\tgoto error_return;\n\t \t}\n\nI think in the end it doesn't matter much, we hit our primary error, so\nwe're only potentially losing another error on our way out the door.\n\nIt's more about clarity, the \"last_errno\" pattern signals \"I'm about to\ncall something that'll reset the errno I care about\", but it's not clear\nif that's actually the case here, or if this is just leftover\nboilerplate.\n\nIn any case running the tests with my second hunk with just a:\n\n\tif (errno)\n\t\tBUG(\"lost it?\");\n\telse\n\t\t...\n\nPasses all our tests. I don't think it should be the scope of this\nseries to give this code 100% test coverage, but (looking ahead) there's\nno mention of /test/ anywhere in the commit messages/comments.\n\nI think even if we keep your \"last_errno\" as-is here it would be useful\nto at least say something like:\n\n\t/*\n\t * Just paranoia, we probably won't lose errno in\n\t * refs_verify_refname_available().\n\t */\n\tint last_errno = errno;\n\t...\n"},{"id":"428888","messageId":"87h7hedz9z.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"3e2831e59c8e4cb8aef416a41b55083887b4559c.1623329869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-01T11:34:15Z","receivedAt":"2021-07-01T11:55:25Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> read_raw_ref_fn needs to supply a credible errno for a number of cases. These\n> are primarily:\n>\n> 1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\n> resulting error codes to create/remove directories as needed.\n>\n> 2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\n> returning the last successfully resolved symref. This is necessary so\n> read_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\n> du-jour), and we know on which branch to create the first commit.\n>\n> Make this information flow explicit by adding a failure_errno to the signature\n> of read_raw_ref. All errnos from the files backend are still propagated\n> unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\n> relevant.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> ---\n> refs.c | 8 ++++++--\n> refs/debug.c | 4 ++--\n> refs/files-backend.c | 24 ++++++++++++------------\n> refs/packed-backend.c | 8 ++++----\n> refs/refs-internal.h | 17 +++++++++--------\n> 5 files changed, 33 insertions(+), 28 deletions(-)\n>\n> diff --git a/refs.c b/refs.c\n> index 8c9490235ea6..bebe3f584da7 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1675,13 +1675,17 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n> \t\t const char *refname, struct object_id *oid,\n> \t\t struct strbuf *referent, unsigned int *type)\n> {\n> +\tint result, failure;\n\nStyle nit: Different variables should be on different lines, except\nwhere they're the same, so \"int i, j\", not \"int ret, i\". Perhaps \"ret\"\nand \"errno\" are similar enough, but I'd split them, just my 0.02.\n\nAlso nit: s/failure/failure_errno/, just like the function signature.\n\n> \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n> \t\t\t\t\t type);\n> \t}\n> \n> -\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> -\t\t\t\t\t type);\n> +\tfailure = 0;\n> +\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> +\t\t\t\t\t type, &failure);\n> +\terrno = failure;\n> +\treturn result;\n> }\n\nIs there some subtlety here where this isn't equivalent to the more\nsimple/straightforward:\n\n\tdiff --git a/refs.c b/refs.c\n\tindex bebe3f584da..49ab7555de9 100644\n\t--- a/refs.c\n\t+++ b/refs.c\n\t@@ -1675,17 +1675,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n\t \t\t const char *refname, struct object_id *oid,\n\t \t\t struct strbuf *referent, unsigned int *type)\n\t {\n\t-\tint result, failure;\n\t \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n\t \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n\t \t\t\t\t\t type);\n\t \t}\n\t \n\t-\tfailure = 0;\n\t-\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n\t-\t\t\t\t\t type, &failure);\n\t-\terrno = failure;\n\t-\treturn result;\n\t+\terrno = 0;\n\t+\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n\t+\t\t\t\t\t type, &errno);\n\t }\n\t \n\t /* This function needs to return a meaningful errno on failure */\n\nIn that case adding the \"failure_errno\" to the signature won't be needed\neither, but I'm assuming we're heading towards some reftable end-state\nwhere that makes more sense.\n\nI can only imagine a case where we think files_read_raw_ref() would\nencounter a new errno after it assigned to *failure_errno, which is just\na couple of strbuf_release() calls.\n\nif that is a case we're worried about then like in my comment on 2/8\nshouldn't we be explicitly checking for such a lost/different errno?\nI.e. something like (should probably be less fatal than BUG(...):\n\t\t\n\tdiff --git a/refs.c b/refs.c\n\tindex bebe3f584da..9584ddae392 100644\n\t--- a/refs.c\n\t+++ b/refs.c\n\t@@ -1675,16 +1675,22 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n\t \t\t const char *refname, struct object_id *oid,\n\t \t\t struct strbuf *referent, unsigned int *type)\n\t {\n\t-\tint result, failure;\n\t+\tint result, failure_errno = 0;\n\t+\n\t \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n\t \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n\t \t\t\t\t\t type);\n\t \t}\n\t \n\t-\tfailure = 0;\n\t+\terrno = 0;\n\t \tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n\t-\t\t\t\t\t type, &failure);\n\t-\terrno = failure;\n\t+\t\t\t\t\t type, &failure_errno);\n\t+\tif (errno)\n\t+\t\tBUG(\"Got another errno from read_raw_ref()?: %s, failure errno: %s\",\n\t+\t\t strerror(errno),\n\t+\t\t strerror(failure_errno));\n\t+\telse\n\t+\t\terrno = failure_errno;\n\t \treturn result;\n\t }\n\t \n\tdiff --git a/refs/files-backend.c b/refs/files-backend.c\n\tindex 8f969c8f711..57cfdf738da 100644\n\t--- a/refs/files-backend.c\n\t+++ b/refs/files-backend.c\n\t@@ -458,8 +458,10 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n\t \tret = parse_loose_ref_contents(buf, oid, referent, type);\n\t \n\t out:\n\t-\tif (failure_errno)\n\t+\tif (failure_errno) {\n\t \t\t*failure_errno = errno;\n\t+\t\terrno = 0;\n\t+\t}\n\t \tstrbuf_release(&sb_path);\n\t \tstrbuf_release(&sb_contents);\n\t \treturn ret;\n\tdiff --git a/refs/packed-backend.c b/refs/packed-backend.c\n\tindex a457f18e93c..4bcb4777f0f 100644\n\t--- a/refs/packed-backend.c\n\t+++ b/refs/packed-backend.c\n\t@@ -740,6 +740,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n\t \tif (!rec) {\n\t \t\t/* refname is not a packed reference. */\n\t \t\t*failure_errno = ENOENT;\n\t+\t\terrno = 0;\n\t \t\treturn -1;\n\t \t}\n\nWhich also passes all our tests, and the packed_read_raw_ref() suggests\nhow you may have come up with the current pattern.\n> [...]\n> - * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n> - * and return -1. If the ref exists but is neither a symbolic ref nor\n> - * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n> - * (errno should not be ENOENT) If there is another error reading the\n> - * ref, set errno appropriately and return -1.\n> + * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT\n> + * and return -1. If the ref exists but is neither a symbolic ref nor an object\n> + * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno\n> + * should not be ENOENT). The files backend may return EISDIR (if the ref name\n> + * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is\n> + * another error reading the ref, set failure_errno appropriately and return -1.\n> *\n\nThis documentation is a bit confusing in light of the above comments. I\nassume that \"set failure_errno appropriately\" here means that it will do\nit, but it's really worded like it's suggesting that you should do it,\nand does the \"another error\" suggest that a caller may need to deal with\nan arbitrary errno, not just the ENOENT|EISDIR|ENOTDIR?\n"},{"id":"428889","messageId":"87eecidz74.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"11b2184044d766183edf5ee1888f38945e8a7fd1.1623329869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-01T11:56:19Z","receivedAt":"2021-07-01T11:57:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\n> contract for the errno output explicit. The implementation still relies on\n> the global errno variable to ensure no side effects of this refactoring.\n>\n> In a follow-on commits, we will\n>\n> * migrate callers that need this error information\n>\n> * clear errno in refs_resolve_ref_unsafe() to make sure these other callers\n> aren't using the error output accidentally.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> ---\n> refs.c | 12 ++++++++++++\n> refs/refs-internal.h | 8 ++++++++\n> 2 files changed, 20 insertions(+)\n>\n> diff --git a/refs.c b/refs.c\n> index bebe3f584da7..64e2d55adcfb 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1781,6 +1781,18 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n> \treturn NULL;\n> }\n> \n> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n> +\t\t\t\t\t const char *refname,\n> +\t\t\t\t\t int resolve_flags,\n> +\t\t\t\t\t struct object_id *oid,\n> +\t\t\t\t\t int *flags, int *failure_errno)\n> +{\n> +\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n> +\t\t\t\t\t\t resolve_flags, oid, flags);\n> +\t*failure_errno = errno;\n> +\treturn result;\n> +}\n> +\n> /* backend functions */\n> int refs_init_db(struct strbuf *err)\n> {\n> diff --git a/refs/refs-internal.h b/refs/refs-internal.h\n> index 904b2a9e51ae..eb97023658f8 100644\n> --- a/refs/refs-internal.h\n> +++ b/refs/refs-internal.h\n> @@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n> \t\t const char *refname, struct object_id *oid,\n> \t\t struct strbuf *referent, unsigned int *type);\n> \n> +/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n> + * failure. */\n> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n> +\t\t\t\t\t const char *refname,\n> +\t\t\t\t\t int resolve_flags,\n> +\t\t\t\t\t struct object_id *oid,\n> +\t\t\t\t\t int *flags, int *failure_errno);\n> +\n> /*\n> * Write an error to `err` and return a nonzero value iff the same\n> * refname appears multiple times in `refnames`. `refnames` must be\n\nOkey, this might be eligable for comments similar to what I've had so\nfar, but no callers yet. Let's read on to those commits, right now I'm\nthinking \"this looks small enough that it should be squashed into that\nfuture commit\"....\n"},{"id":"428890","messageId":"87bl7mdysf.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"005ee8e6fb2a0f1a38801bed1dfe082999bfec14.1623329869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 5/8] refs: use refs_resolve_ref_unsafe_with_errno() where needed","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-01T11:58:35Z","receivedAt":"2021-07-01T12:05:57Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\n> that needs error information to make logic decisions.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> ---\n> refs/files-backend.c | 19 ++++++++++---------\n> 1 file changed, 10 insertions(+), 9 deletions(-)\n>\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 8f969c8f711f..5a430aabf623 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -924,6 +924,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \tint mustexist = (old_oid && !is_null_oid(old_oid));\n> \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n> \tint resolved;\n> +\tint resolve_errno = 0;\n> \n> \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n> \tassert(err);\n> @@ -936,10 +937,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n> \n> \tfiles_ref_path(refs, &ref_file, refname);\n> -\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n> -\t\t\t\t\t refname, resolve_flags,\n> -\t\t\t\t\t &lock->old_oid, type);\n> -\tif (!resolved && errno == EISDIR) {\n> +\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n> +\t\t\t\t\t\t\tresolve_flags,\n> +\t\t\t\t\t\t\t&lock->old_oid, type,\n> +\t\t\t\t\t\t\t&resolve_errno);\n> +\tif (!resolved && resolve_errno == EISDIR) {\n> \t\t/*\n> \t\t * we are trying to lock foo but we used to\n> \t\t * have foo/bar which now does not exist;\n> @@ -959,12 +961,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \t\t\t\t\t\t &lock->old_oid, type);\n> \t}\n> \tif (!resolved) {\n> -\t\tint last_errno = errno;\n> -\t\tif (last_errno != ENOTDIR ||\n> -\t\t !refs_verify_refname_available(&refs->base, refname,\n> -\t\t\t\t\t\t extras, skip, err))\n> +\t\tif (resolve_errno != ENOTDIR ||\n> +\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n> +\t\t\t\t\t\t skip, err))\n> \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n> -\t\t\t\t refname, strerror(last_errno));\n> +\t\t\t\t refname, strerror(resolve_errno));\n> \n> \t\tgoto error_return;\n> \t}\n\nSo, having read 4/8 and this I wonder why\nrefs_resolve_ref_unsafe_with_errno() is needed at all. It's just a\nwrapper that sets errno to a variable you give it, but we could just\ndocument that tha caller should check errno.\n\nSo far I haven't seen anything that suggests the below diff-on-top\nwouldn't be OK (and all tests pass with it). It steps on the toes of\nsome of my earlier suggestions, but I'm doing these one-at-a-time.\n\nIn any case the comment I adjusted seems like something you should\nadjust to. It looks like a TODO for having the sort of function you've\njust implemented in refs_resolve_ref_unsafe_with_errno().\n\n\tdiff --git a/refs.c b/refs.c\n\tindex 64e2d55adcf..a07d852fcdc 100644\n\t--- a/refs.c\n\t+++ b/refs.c\n\t@@ -1688,7 +1688,10 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n\t \treturn result;\n\t }\n\t \n\t-/* This function needs to return a meaningful errno on failure */\n\t+/*\n\t+ * This function clears errno at the beginning. If it fails the errno\n\t+ * will be meaningful.\n\t+ */\n\t const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n\t \t\t\t\t const char *refname,\n\t \t\t\t\t int resolve_flags,\n\t@@ -1698,6 +1701,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n\t \tstruct object_id unused_oid;\n\t \tint unused_flags;\n\t \tint symref_count;\n\t+\terrno = 0;\n\t \n\t \tif (!oid)\n\t \t\toid = &unused_oid;\n\t@@ -1781,18 +1785,6 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n\t \treturn NULL;\n\t }\n\t \n\t-const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n\t-\t\t\t\t\t const char *refname,\n\t-\t\t\t\t\t int resolve_flags,\n\t-\t\t\t\t\t struct object_id *oid,\n\t-\t\t\t\t\t int *flags, int *failure_errno)\n\t-{\n\t-\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n\t-\t\t\t\t\t\t resolve_flags, oid, flags);\n\t-\t*failure_errno = errno;\n\t-\treturn result;\n\t-}\n\t-\n\t /* backend functions */\n\t int refs_init_db(struct strbuf *err)\n\t {\n\tdiff --git a/refs/files-backend.c b/refs/files-backend.c\n\tindex 5a430aabf62..5a400e55cbf 100644\n\t--- a/refs/files-backend.c\n\t+++ b/refs/files-backend.c\n\t@@ -924,7 +924,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n\t \tint mustexist = (old_oid && !is_null_oid(old_oid));\n\t \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n\t \tint resolved;\n\t-\tint resolve_errno = 0;\n\t \n\t \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n\t \tassert(err);\n\t@@ -937,11 +936,10 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n\t \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n\t \n\t \tfiles_ref_path(refs, &ref_file, refname);\n\t-\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n\t-\t\t\t\t\t\t\tresolve_flags,\n\t-\t\t\t\t\t\t\t&lock->old_oid, type,\n\t-\t\t\t\t\t\t\t&resolve_errno);\n\t-\tif (!resolved && resolve_errno == EISDIR) {\n\t+\tresolved = !!refs_resolve_ref_unsafe(&refs->base, refname,\n\t+\t\t\t\t\t resolve_flags,\n\t+\t\t\t\t\t &lock->old_oid, type);\n\t+\tif (!resolved && errno == EISDIR) {\n\t \t\t/*\n\t \t\t * we are trying to lock foo but we used to\n\t \t\t * have foo/bar which now does not exist;\n\t@@ -961,11 +959,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n\t \t\t\t\t\t\t &lock->old_oid, type);\n\t \t}\n\t \tif (!resolved) {\n\t-\t\tif (resolve_errno != ENOTDIR ||\n\t+\t\tif (errno != ENOTDIR ||\n\t \t\t !refs_verify_refname_available(&refs->base, refname, extras,\n\t \t\t\t\t\t\t skip, err))\n\t \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n\t-\t\t\t\t refname, strerror(resolve_errno));\n\t+\t\t\t\t refname, strerror(errno));\n\t \n\t \t\tgoto error_return;\n\t \t}\n\t\n"},{"id":"428891","messageId":"878s2qdy8y.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"2b346caf1aed265ca4787de4208375bd5cfb7bc7.1623329869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 6/8] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-01T12:06:37Z","receivedAt":"2021-07-01T12:17:42Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> This makes the errno output of refs_read_raw_ref explicit.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> ---\n> refs.c | 29 ++++++++++++++---------------\n> refs/files-backend.c | 8 ++++----\n> refs/packed-backend.c | 10 ++++++----\n> refs/refs-internal.h | 6 +++---\n> 4 files changed, 27 insertions(+), 26 deletions(-)\n>\n> diff --git a/refs.c b/refs.c\n> index 64e2d55adcfb..ed2dde1c0c6d 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1671,21 +1671,19 @@ static int refs_read_special_head(struct ref_store *ref_store,\n> \treturn result;\n> }\n> \n> -int refs_read_raw_ref(struct ref_store *ref_store,\n> -\t\t const char *refname, struct object_id *oid,\n> -\t\t struct strbuf *referent, unsigned int *type)\n> +int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> +\t\t struct object_id *oid, struct strbuf *referent,\n> +\t\t unsigned int *type, int *failure_errno)\n> {\n> -\tint result, failure;\n> +\tif (failure_errno)\n> +\t\t*failure_errno = 0;\n> \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n> \t\t\t\t\t type);\n> \t}\n> \n> -\tfailure = 0;\n> -\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> -\t\t\t\t\t type, &failure);\n> -\terrno = failure;\n> -\treturn result;\n> +\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> +\t\t\t\t\t type, failure_errno);\n> }\n\nAh, so here we drop the whole intermediate step of having this function\nnot take a failure_errno itself. I think this would be better squashed\ninto that earlier change.\n\n> /* This function needs to return a meaningful errno on failure */\n> @@ -1726,9 +1724,10 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n> \n> \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n> \t\tunsigned int read_flags = 0;\n> +\t\tint read_failure = 0;\n\nLet's call it failure_errno consistently if we end up keeping it.\n\n> -\t\tif (refs_read_raw_ref(refs, refname,\n> -\t\t\t\t oid, &sb_refname, &read_flags)) {\n> +\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n> +\t\t\t\t &read_flags, &read_failure)) {\n> \t\t\t*flags |= read_flags;\n> \n> \t\t\t/* In reading mode, refs must eventually resolve */\n> @@ -1740,9 +1739,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n> \t\t\t * may show errors besides ENOENT if there are\n> \t\t\t * similarly-named refs.\n> \t\t\t */\n> -\t\t\tif (errno != ENOENT &&\n> -\t\t\t errno != EISDIR &&\n> -\t\t\t errno != ENOTDIR)\n> +\t\t\tif (read_failure != ENOENT && read_failure != EISDIR &&\n> +\t\t\t read_failure != ENOTDIR)\n> \t\t\t\treturn NULL;\n\nBut ditto my previous comments, this seems like a whole dance to avoid\nreading errno directly in cases where doing so is actually OK. I.e. the\n\"last_errno\" pattern is for things that encounter an errno, do some\nother stuff (such as a printf) where they might get /another/ errno (or\nreset it), but in this case we can just document \"these will set errno\non failure\".\n\n> \t\t\toidclr(oid);\n> @@ -2254,7 +2252,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n> \t\tif (skip && string_list_has_string(skip, dirname.buf))\n> \t\t\tcontinue;\n> \n> -\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n> +\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n> +\t\t\t\t &type, NULL)) {\n> \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n> \t\t\t\t dirname.buf, refname);\n> \t\t\tgoto cleanup;\n\n\nAnd if we do care about errno at all, why would we not add it with a\nstrerror() here instead of explicitly ignoring it?\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 5a430aabf623..01c9bd0dbf04 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -383,8 +383,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> \tif (lstat(path, &st) < 0) {\n> \t\tif (errno != ENOENT)\n> \t\t\tgoto out;\n> -\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n> -\t\t\t\t oid, referent, type)) {\n> +\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n> +\t\t\t\t referent, type, NULL)) {\n> \t\t\terrno = ENOENT;\n> \t\t\tgoto out;\n> \t\t}\n\nIn this case that seems to make sense, since we substitute our own\nerrno. Maybe I haven't read this all carefully enough, but why are we\nnot caring about the errno refs_read_raw_ref() might return here? Is it\nbecause we might take the refs_read_special_head() codepath?\n\nIn any case, I for one would appreciate a comment/commit message note\nabout why we ignore the errno these \"I'll give you an errno on failure\"\nfunctions return in some cases, but not others.\n\nI think it's probably best to split those into another commit, i.e. do\nthe \"we just ferry errno around differently now (if we'll do that at\nall)\" as one step, and \"here we might get errno, but we like our own\nbetter\" as another.\n\n> @@ -423,8 +423,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> \t\t * ref is supposed to be, there could still be a\n> \t\t * packed ref:\n> \t\t */\n> -\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n> -\t\t\t\t oid, referent, type)) {\n> +\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n> +\t\t\t\t referent, type, NULL)) {\n> \t\t\terrno = EISDIR;\n> \t\t\tgoto out;\n\nDitto.\n\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index a457f18e93c8..03353ce48869 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -739,7 +739,8 @@ static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> \n> \tif (!rec) {\n> \t\t/* refname is not a packed reference. */\n> -\t\t*failure_errno = ENOENT;\n> +\t\tif (failure_errno)\n> +\t\t\t*failure_errno = ENOENT;\n> \t\treturn -1;\n> \t}\n\nFWIW I'd think it's better in terms of code readability to not do this,\nand make callers who don't care about our errno explicitly provide a\nthrowaway variable to show that they're not caring, but I don't have the\nfull picture yet. I.e. make them do something like:\n\n\tint ret;\n\tint got_errno;\n\tret = func(..., &got_errno);\n\tif (!ret) {\n\t\tif (!got_errno)\n\t\t\tBUG(\"error but no errno?\");\n\t\t/* We don't care, use our own */\n\t\tgot_errno = ENOSOMETHING\n\t\t...\n"},{"id":"428892","messageId":"875yxudxzf.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"d86516219689e9179f3ba18c2ad2391d8ca27076.1623329869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 7/8] refs: clear errno return in refs_resolve_ref_unsafe()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-01T12:19:18Z","receivedAt":"2021-07-01T12:23:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> This is done in a separate commit, to pinpoint the precise cause should there be\n> regressions in error reporting.\n>\n> This is implemented by renaming the existing logic to a static function\n> refs_resolve_unsafe_implicit_errno(), minimizing the code diff.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> ---\n> refs.c | 22 ++++++++++++++++------\n> 1 file changed, 16 insertions(+), 6 deletions(-)\n>\n> diff --git a/refs.c b/refs.c\n> index ed2dde1c0c6d..191cbf5a330f 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1687,10 +1687,10 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> }\n> \n> /* This function needs to return a meaningful errno on failure */\n> -const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n> -\t\t\t\t const char *refname,\n> -\t\t\t\t int resolve_flags,\n> -\t\t\t\t struct object_id *oid, int *flags)\n> +static const char *\n\nThe \"let's make this static\" seems like an unrelated change we should\nsquash into the earlier \"refs: use refs_resolve_ref_unsafe_with_errno()\nwhere needed\", we stopped using it in the files backend then. No?\n\n> +refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n> +\t\t\t\t const char *refname, int resolve_flags,\n> +\t\t\t\t struct object_id *oid, int *flags)\n> {\n> \tstatic struct strbuf sb_refname = STRBUF_INIT;\n> \tstruct object_id unused_oid;\n> @@ -1779,14 +1779,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n> \treturn NULL;\n> }\n> \n> +const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n> +\t\t\t\t int resolve_flags, struct object_id *oid,\n> +\t\t\t\t int *flags)\n> +{\n> +\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n> +\t\trefs, refname, resolve_flags, oid, flags);\n> +\terrno = 0;\n> +\treturn result;\n> +}\n> +\n> const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n> \t\t\t\t\t const char *refname,\n> \t\t\t\t\t int resolve_flags,\n> \t\t\t\t\t struct object_id *oid,\n> \t\t\t\t\t int *flags, int *failure_errno)\n> {\n> -\tconst char *result = refs_resolve_ref_unsafe(refs, refname,\n> -\t\t\t\t\t\t resolve_flags, oid, flags);\n> +\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n> +\t\trefs, refname, resolve_flags, oid, flags);\n> \t*failure_errno = errno;\n> \treturn result;\n> }\n\nPer my earlier comments this whole thing again seems a bit backwards. We\nexplicitly clear errno instead of telling the caller to care.\n\nHas this whole thing perhaps had the unstated aim that you were trying\nto distinguish the now-remaining refs_resolve_ref_unsafe() callers from\nthe \"I care about errno\" by explicitly clearing out errno for them, thus\nensuring that they need to use the wrapper function with \"errno\" in the\nname to get the errno they'd otherwise get?\n"},{"id":"428894","messageId":"8735sydvzs.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"2a9ebe43deac3da9baf241ff1e494f8210c27981.1623329869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-01T12:26:11Z","receivedAt":"2021-07-01T13:06:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> The function refs_resolve_ref_unsafe_with_errno should produce an errno output.\n> Rather than taking the value from the errno (which might contain garbage\n> beforehand), explicitly propagate the failure_errno coming out of\n> refs_read_raw_ref().\n>\n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> ---\n> refs.c | 38 ++++++++++++++------------------------\n> 1 file changed, 14 insertions(+), 24 deletions(-)\n>\n> diff --git a/refs.c b/refs.c\n> index 191cbf5a330f..92c4796078bb 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1686,11 +1686,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> \t\t\t\t\t type, failure_errno);\n> }\n> \n> -/* This function needs to return a meaningful errno on failure */\n> -static const char *\n> -refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n> -\t\t\t\t const char *refname, int resolve_flags,\n> -\t\t\t\t struct object_id *oid, int *flags)\n> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n> +\t\t\t\t\t const char *refname,\n> +\t\t\t\t\t int resolve_flags,\n> +\t\t\t\t\t struct object_id *oid,\n> +\t\t\t\t\t int *flags, int *failure_errno)\n> {\n\nAt this point I'm lost. We just introduced this function two commits\nago, and now it's gone again. I guess the \"we should update the comment\"\nfrom an earlier reply of mine is addressed in some way here...\n\n> \tstatic struct strbuf sb_refname = STRBUF_INIT;\n> \tstruct object_id unused_oid;\n> @@ -1703,11 +1703,12 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n\nAs an aside I really wish our hunk title detection would spot the rename\n& use the new name...\n\n> \t\tflags = &unused_flags;\n> \n> \t*flags = 0;\n> +\t*failure_errno = 0;\n> \n> \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n> \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n> \t\t !refname_is_safe(refname)) {\n> -\t\t\terrno = EINVAL;\n> +\t\t\t*failure_errno = EINVAL;\n> \t\t\treturn NULL;\n> \t\t}\n> \n> @@ -1730,6 +1731,8 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n> \t\t\t\t &read_flags, &read_failure)) {\n> \t\t\t*flags |= read_flags;\n> \n> +\t\t\t*failure_errno = read_failure;\n> +\n> \t\t\t/* In reading mode, refs must eventually resolve */\n> \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n> \t\t\t\treturn NULL;\n> @@ -1767,7 +1770,7 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n> \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n> \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n> \t\t\t !refname_is_safe(refname)) {\n> -\t\t\t\terrno = EINVAL;\n> +\t\t\t\t*failure_errno = EINVAL;\n> \t\t\t\treturn NULL;\n> \t\t\t}\n> \n> @@ -1775,7 +1778,7 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,\n> \t\t}\n> \t}\n> \n> -\terrno = ELOOP;\n> +\t*failure_errno = ELOOP;\n> \treturn NULL;\n> }\n> \n> @@ -1783,22 +1786,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n> \t\t\t\t int resolve_flags, struct object_id *oid,\n> \t\t\t\t int *flags)\n> {\n> -\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n> -\t\trefs, refname, resolve_flags, oid, flags);\n> -\terrno = 0;\n> -\treturn result;\n> -}\n> -\n> -const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n> -\t\t\t\t\t const char *refname,\n> -\t\t\t\t\t int resolve_flags,\n> -\t\t\t\t\t struct object_id *oid,\n> -\t\t\t\t\t int *flags, int *failure_errno)\n> -{\n> -\tconst char *result = refs_resolve_ref_unsafe_implicit_errno(\n> -\t\trefs, refname, resolve_flags, oid, flags);\n> -\t*failure_errno = errno;\n> -\treturn result;\n> +\tint ignore;\n> +\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n> +\t\t\t\t\t\t oid, flags, &ignore);\n> }\n\nOkey, so here we have something like the \"caller should explicitly\nignore it\" I suggested in <878s2qdy8y.fsf@evledraar.gmail.com>. We no\nlonger set \"errno\" at all for the benefit of the callers of that\nfunction, but instead explicitly throw it away, our\nrefs_resolve_ref_unsafe_with_errno() no longer sets errno.\n\nBut this end-state seems to have resulted in introducing new bugs. A\nbunch of functions are thin wrappers for\nrefs_resolve_ref_unsafe(). Previously they could inspect errno on a -1\nreturn, now they can't.\n\nI didn't look at them all, but just the second one I looked at,\nrefs_read_ref_full() has a verify_lock() caller which calls it, and that\nfunction then expects a meaningful errno that it in turn ferries up with\n\"can't verify ref\". It in turn is called by lock_ref_oid_basic(), it\nferries up that strbuf with the error, which'll now have a meaningless\nstrerror(0) in it.\n\nSo it seems to me that the refactoring being done here is halfway done,\nand results in a buggy end-state. The below diff is my attempt to fix\nthat up for just one caller, things thath call resolve_gitlink_ref()\n(it's literally the third caller out of N, so I'm not picking and\nchoosing).\n\nIn that case I believe that we don't actually have a regression because\nof your changes, but I think the changes below are the logical end-state\nof what you're starting here. I.e. we have a low-level function that may\nencounter an errno, and then callers removed from that via:\n\n index_path() -> resolve_gitlink_ref() -> refs_resolve_ref_unsafe()\n\nWhere the common case for index_path() is to report the error we got,\nbut it doesn't call die_errno() or error_errno() now, but really should,\nper the below diff (which you can consider to have by SOB if you want to\npick it up).\n\nThat passes all tests, except one where we actually fail because of what\nI'd argue is a better error message. i.e this in t3700-add.sh:\n\t\n\t+ diff -u expect actual\n\t--- expect 2021-07-01 13:01:23.267001881 +0000\n\t+++ actual 2021-07-01 13:01:23.267001881 +0000\n\t@@ -1,2 +1,2 @@\n\t-error: 'empty/' does not have a commit checked out\n\t+error: 'empty/' does not have a commit checked out: No such file or directory\n\t fatal: adding files failed\n\nIn one sense I don't think you need to do this excercise of going all\nthe way to the top of callers like index_path() that don't get an errno\nnow, but in another sense I think you really do.\n\nBy explicitly clearing \"errno\" as your end-state does and declaring that\ncallers of refs_resolve_ref_unsafe() must be as happy to ignore \"errno\"\nas your own callers are, you're potentially introducing subtle bugs\nunless we go all the way to the top of the callstack and check if\nanything above us cares about errno, which we'll need due to do due to\nits global nature.\n\nI think I'd really only be confident that we've got them all if your\nchance was amended to something where we add such \"int *maybe_errno\"\nparameters to all upstream callers that may relay the <0 return value we\ngot.\n\ndiff --git a/builtin/update-index.c b/builtin/update-index.c\nindex f1f16f2de52..54fcbc5be10 100644\n--- a/builtin/update-index.c\n+++ b/builtin/update-index.c\n@@ -272,6 +272,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len\n {\n \tint option;\n \tstruct cache_entry *ce;\n+\tint ignore_errno;\n \n \t/* Was the old index entry already up-to-date? */\n \tif (old && !ce_stage(old) && !ce_match_stat(old, st, 0))\n@@ -285,7 +286,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len\n \tce->ce_mode = ce_mode_from_stat(old, st->st_mode);\n \n \tif (index_path(&the_index, &ce->oid, path, st,\n-\t\t info_only ? 0 : HASH_WRITE_OBJECT)) {\n+\t\t info_only ? 0 : HASH_WRITE_OBJECT, &ignore_errno)) {\n \t\tdiscard_cache_entry(ce);\n \t\treturn -1;\n \t}\n@@ -325,14 +326,15 @@ static int process_directory(const char *path, int len, struct stat *st)\n {\n \tstruct object_id oid;\n \tint pos = cache_name_pos(path, len);\n+\tint their_errno = 0;\n \n \t/* Exact match: file or existing gitlink */\n \tif (pos >= 0) {\n \t\tconst struct cache_entry *ce = active_cache[pos];\n \t\tif (S_ISGITLINK(ce->ce_mode)) {\n-\n \t\t\t/* Do nothing to the index if there is no HEAD! */\n-\t\t\tif (resolve_gitlink_ref(path, \"HEAD\", &oid) < 0)\n+\t\t\tint ignore_errno;\n+\t\t\tif (resolve_gitlink_ref(path, \"HEAD\", &oid, &ignore_errno) < 0)\n \t\t\t\treturn 0;\n \n \t\t\treturn add_one_path(ce, path, len, st);\n@@ -358,11 +360,16 @@ static int process_directory(const char *path, int len, struct stat *st)\n \t}\n \n \t/* No match - should we add it as a gitlink? */\n-\tif (!resolve_gitlink_ref(path, \"HEAD\", &oid))\n+\tif (resolve_gitlink_ref(path, \"HEAD\", &oid, &their_errno) >= 0)\n \t\treturn add_one_path(NULL, path, len, st);\n \n \t/* Error out. */\n-\treturn error(\"%s: is a directory - add files inside instead\", path);\n+\tif (their_errno) {\n+\t\terrno = their_errno;\n+\t\treturn error_errno(\"%s: is a directory - add files inside instead\", path);\n+\t} else {\n+\t\treturn error(\"%s: is a directory - add files inside instead\", path);\n+\t}\n }\n \n static int process_path(const char *path, struct stat *st, int stat_errno)\ndiff --git a/cache.h b/cache.h\nindex ba04ff8bd36..62e5ac335b8 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -879,7 +879,9 @@ int ie_modified(struct index_state *, const struct cache_entry *, struct stat *,\n #define HASH_FORMAT_CHECK 2\n #define HASH_RENORMALIZE 4\n int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n-int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);\n+int index_path(struct index_state *istate, struct object_id *oid,\n+\t const char *path, struct stat *st, unsigned flags,\n+\t int *maybe_errno);\n \n /*\n * Record to sd the data from st that we use to check whether a file\ndiff --git a/combine-diff.c b/combine-diff.c\nindex 7d925ce9ce4..7ac9bc6e943 100644\n--- a/combine-diff.c\n+++ b/combine-diff.c\n@@ -1059,7 +1059,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,\n \t\t\telem->mode = canon_mode(st.st_mode);\n \t\t} else if (S_ISDIR(st.st_mode)) {\n \t\t\tstruct object_id oid;\n-\t\t\tif (resolve_gitlink_ref(elem->path, \"HEAD\", &oid) < 0)\n+\t\t\tint ignore_errno;\n+\t\t\tif (resolve_gitlink_ref(elem->path, \"HEAD\", &oid, &ignore_errno) < 0)\n \t\t\t\tresult = grab_blob(opt->repo, &elem->oid,\n \t\t\t\t\t\t elem->mode, &result_size,\n \t\t\t\t\t\t NULL, NULL);\ndiff --git a/diff-lib.c b/diff-lib.c\nindex c2ac9250fe9..4ae27851cb9 100644\n--- a/diff-lib.c\n+++ b/diff-lib.c\n@@ -40,6 +40,7 @@ static int check_removed(const struct index_state *istate, const struct cache_en\n \t\treturn 1;\n \tif (S_ISDIR(st->st_mode)) {\n \t\tstruct object_id sub;\n+\t\tint ignore_errno;\n \n \t\t/*\n \t\t * If ce is already a gitlink, we can have a plain\n@@ -53,7 +54,7 @@ static int check_removed(const struct index_state *istate, const struct cache_en\n \t\t * a directory --- the blob was removed!\n \t\t */\n \t\tif (!S_ISGITLINK(ce->ce_mode) &&\n-\t\t resolve_gitlink_ref(ce->name, \"HEAD\", &sub))\n+\t\t resolve_gitlink_ref(ce->name, \"HEAD\", &sub, &ignore_errno))\n \t\t\treturn 1;\n \t}\n \treturn 0;\ndiff --git a/diff.c b/diff.c\nindex 52c791574b7..972ad0e57de 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4426,6 +4426,7 @@ static void diff_fill_oid_info(struct diff_filespec *one, struct index_state *is\n {\n \tif (DIFF_FILE_VALID(one)) {\n \t\tif (!one->oid_valid) {\n+\t\t\tint maybe_errno = 0;\n \t\t\tstruct stat st;\n \t\t\tif (one->is_stdin) {\n \t\t\t\toidclr(&one->oid);\n@@ -4433,8 +4434,14 @@ static void diff_fill_oid_info(struct diff_filespec *one, struct index_state *is\n \t\t\t}\n \t\t\tif (lstat(one->path, &st) < 0)\n \t\t\t\tdie_errno(\"stat '%s'\", one->path);\n-\t\t\tif (index_path(istate, &one->oid, one->path, &st, 0))\n-\t\t\t\tdie(\"cannot hash %s\", one->path);\n+\t\t\tif (index_path(istate, &one->oid, one->path, &st, 0, &maybe_errno)) {\n+\t\t\t\tif (maybe_errno) {\n+\t\t\t\t\terrno = maybe_errno;\n+ \t\t\t\t\tdie_errno(\"cannot hash %s\", one->path);\n+\t\t\t\t} else {\n+ \t\t\t\t\tdie(\"cannot hash %s\", one->path);\n+\t\t\t\t}\n+\t\t\t}\n \t\t}\n \t}\n \telse\ndiff --git a/dir.c b/dir.c\nindex ebe5ec046e0..95eb4ada718 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2967,9 +2967,10 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)\n \tint only_empty = (flag & REMOVE_DIR_EMPTY_ONLY);\n \tint keep_toplevel = (flag & REMOVE_DIR_KEEP_TOPLEVEL);\n \tstruct object_id submodule_head;\n+\tint ignore_errno;\n \n \tif ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&\n-\t !resolve_gitlink_ref(path->buf, \"HEAD\", &submodule_head)) {\n+\t resolve_gitlink_ref(path->buf, \"HEAD\", &submodule_head, &ignore_errno) < 0) {\n \t\t/* Do not descend and nuke a nested git work tree. */\n \t\tif (kept_up)\n \t\t\t*kept_up = 1;\ndiff --git a/notes-merge.c b/notes-merge.c\nindex 46c1f7c7f11..f7e0091bd72 100644\n--- a/notes-merge.c\n+++ b/notes-merge.c\n@@ -698,6 +698,7 @@ int notes_merge_commit(struct notes_merge_options *o,\n \twhile ((e = readdir_skip_dot_and_dotdot(dir)) != NULL) {\n \t\tstruct stat st;\n \t\tstruct object_id obj_oid, blob_oid;\n+\t\tint maybe_errno = 0;\n \n \t\tif (get_oid_hex(e->d_name, &obj_oid)) {\n \t\t\tif (o->verbosity >= 3)\n@@ -710,8 +711,14 @@ int notes_merge_commit(struct notes_merge_options *o,\n \t\t/* write file as blob, and add to partial_tree */\n \t\tif (stat(path.buf, &st))\n \t\t\tdie_errno(\"Failed to stat '%s'\", path.buf);\n-\t\tif (index_path(o->repo->index, &blob_oid, path.buf, &st, HASH_WRITE_OBJECT))\n-\t\t\tdie(\"Failed to write blob object from '%s'\", path.buf);\n+\t\tif (index_path(o->repo->index, &blob_oid, path.buf, &st, HASH_WRITE_OBJECT, &maybe_errno)) {\n+\t\t\tif (maybe_errno) {\n+\t\t\t\terrno = maybe_errno;\n+\t\t\t\tdie_errno(\"Failed to write blob object from '%s'\", path.buf);\n+\t\t\t} else {\n+\t\t\t\tdie(\"Failed to write blob object from '%s'\", path.buf);\n+\t\t\t}\n+\t\t}\n \t\tif (add_note(partial_tree, &obj_oid, &blob_oid, NULL))\n \t\t\tdie(\"Failed to add resolved note '%s' to notes tree\",\n \t\t\t path.buf);\ndiff --git a/object-file.c b/object-file.c\nindex f233b440b22..0030854dcdf 100644\n--- a/object-file.c\n+++ b/object-file.c\n@@ -2260,7 +2260,8 @@ int index_fd(struct index_state *istate, struct object_id *oid,\n }\n \n int index_path(struct index_state *istate, struct object_id *oid,\n-\t const char *path, struct stat *st, unsigned flags)\n+\t const char *path, struct stat *st, unsigned flags,\n+\t int *maybe_errno)\n {\n \tint fd;\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -2286,7 +2287,7 @@ int index_path(struct index_state *istate, struct object_id *oid,\n \t\tstrbuf_release(&sb);\n \t\tbreak;\n \tcase S_IFDIR:\n-\t\treturn resolve_gitlink_ref(path, \"HEAD\", oid);\n+\t\treturn resolve_gitlink_ref(path, \"HEAD\", oid, maybe_errno);\n \tdefault:\n \t\treturn error(_(\"%s: unsupported file type\"), path);\n \t}\ndiff --git a/read-cache.c b/read-cache.c\nindex 77961a38854..a40a326587f 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -256,6 +256,7 @@ static int ce_compare_link(const struct cache_entry *ce, size_t expected_size)\n static int ce_compare_gitlink(const struct cache_entry *ce)\n {\n \tstruct object_id oid;\n+\tint ignore_errno;\n \n \t/*\n \t * We don't actually require that the .git directory\n@@ -265,7 +266,7 @@ static int ce_compare_gitlink(const struct cache_entry *ce)\n \t *\n \t * If so, we consider it always to match.\n \t */\n-\tif (resolve_gitlink_ref(ce->name, \"HEAD\", &oid) < 0)\n+\tif (resolve_gitlink_ref(ce->name, \"HEAD\", &oid, &ignore_errno) < 0)\n \t\treturn 0;\n \treturn !oideq(&oid, &ce->oid);\n }\n@@ -748,8 +749,13 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,\n \n \tnamelen = strlen(path);\n \tif (S_ISDIR(st_mode)) {\n-\t\tif (resolve_gitlink_ref(path, \"HEAD\", &oid) < 0)\n-\t\t\treturn error(_(\"'%s' does not have a commit checked out\"), path);\n+\t\tint their_errno = 0;\n+\t\tif (resolve_gitlink_ref(path, \"HEAD\", &oid, &their_errno) < 0) {\n+\t\t\tif (their_errno)\n+\t\t\t\treturn error_errno(_(\"'%s' does not have a commit checked out\"), path);\n+\t\t\telse\n+\t\t\t\treturn error(_(\"'%s' does not have a commit checked out\"), path);\n+\t\t}\n \t\twhile (namelen && path[namelen-1] == '/')\n \t\t\tnamelen--;\n \t}\n@@ -799,9 +805,16 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,\n \t\t}\n \t}\n \tif (!intent_only) {\n-\t\tif (index_path(istate, &ce->oid, path, st, hash_flags)) {\n+\t\tint maybe_errno = 0;\n+\t\tif (index_path(istate, &ce->oid, path, st, hash_flags,\n+\t\t\t &maybe_errno)) {\n \t\t\tdiscard_cache_entry(ce);\n-\t\t\treturn error(_(\"unable to index file '%s'\"), path);\n+\t\t\tif (maybe_errno) {\n+\t\t\t\terrno = maybe_errno;\n+\t\t\t\treturn error_errno(_(\"unable to index file '%s'\"), path);\n+\t\t\t} else {\n+\t\t\t\treturn error(_(\"unable to index file '%s'\"), path);\n+\t\t\t}\n \t\t}\n \t} else\n \t\tset_object_name_for_intent_to_add_entry(ce);\ndiff --git a/refs.c b/refs.c\nindex 92c4796078b..082b6b3d133 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1807,7 +1807,7 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n }\n \n int resolve_gitlink_ref(const char *submodule, const char *refname,\n-\t\t\tstruct object_id *oid)\n+\t\t\tstruct object_id *oid, int *maybe_errno)\n {\n \tstruct ref_store *refs;\n \tint flags;\n@@ -1817,7 +1817,8 @@ int resolve_gitlink_ref(const char *submodule, const char *refname,\n \tif (!refs)\n \t\treturn -1;\n \n-\tif (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||\n+\tif (!refs_resolve_ref_unsafe_with_errno(refs, refname, 0, oid, &flags,\n+\t\t maybe_errno) ||\n \t is_null_oid(oid))\n \t\treturn -1;\n \treturn 0;\ndiff --git a/refs.h b/refs.h\nindex 48970dfc7e0..b0505b35698 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -133,10 +133,16 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled);\n * Resolve refname in the nested \"gitlink\" repository in the specified\n * submodule (which must be non-NULL). If the resolution is\n * successful, return 0 and set oid to the name of the object;\n- * otherwise, return a non-zero value.\n+ * otherwise, return negative value.\n+ *\n+ * If we encounter an error we may have an errno to report from\n+ * refs_resolve_ref_unsafe_with_errno(), or we might have hit another\n+ * error. Set your `maybe_errno` to 0 beforehand, and check it after\n+ * an error where we return a negative value to see if you should\n+ * report the errno as well.\n */\n int resolve_gitlink_ref(const char *submodule, const char *refname,\n-\t\t\tstruct object_id *oid);\n+\t\t\tstruct object_id *oid, int *maybe_errno);\n \n /*\n * Return true iff abbrev_name is a possible abbreviation for\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 01c9bd0dbf0..9ed58c40de6 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -870,6 +870,7 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,\n \t\t\t &lock->old_oid, NULL)) {\n \t\tif (old_oid) {\n \t\t\tint save_errno = errno;\n+\t\t\tBUG(\"got here\");\n \t\t\tstrbuf_addf(err, \"can't verify ref '%s'\", lock->ref_name);\n \t\t\terrno = save_errno;\n \t\t\treturn -1;\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex f88a69f8e71..dd90787c56c 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1986,7 +1986,9 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \n \tif (S_ISGITLINK(ce->ce_mode)) {\n \t\tstruct object_id oid;\n-\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", &oid);\n+\t\tint ignore_errno;\n+\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", &oid,\n+\t\t\t\t\t\t &ignore_errno);\n \t\t/*\n \t\t * If we are not going to update the submodule, then\n \t\t * we don't care.\n"},{"id":"429242","messageId":"CAFQ2z_NpyJQLuM70MhJ8K1h2v3QXFuAZRjN=SvSsjnukNRJ8pw@mail.gmail.com","threadId":"55589","inReplyTo":"87k0mae0ga.fsf@evledraar.gmail.com","subject":"Re: [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-05T14:16:41Z","receivedAt":"2021-07-05T14:16:55Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jul 1, 2021 at 1:30 PM Ævar Arnfjörð Bjarmason wrote:\n> > }\n> > if (!resolved) {\n> > - last_errno = errno;\n> > + int last_errno = errno;\n> > if (last_errno != ENOTDIR ||\n> > !refs_verify_refname_available(&refs->base, refname,\n> > extras, skip, err))\n>\n> ...this particular change gives me some pause, because all the rest is\n> about squirreling away our own errno for our own caller (which it turns\n> out, we didn't need).\n>\n> But in this case we're only guarding against\n> refs_verify_refname_available() possibly clobbering the errno we just\n> got on !resolved in refs_verify_refname_available().\n\nThis comes from 5b2d8d6f2184381b76c13504a2f5ec8a62cd584e\n\n .. invoke verify_refname_available() to try\n to generate a more helpful error message.\n\n That function might not detect an error. For example, some\n non-reference file might be blocking the deletion of an\n otherwise-empty directory tree, or there might be a race with another\n process that just deleted the offending reference. In such cases,\n generate the strerror-based error message like before.\n\nI think we want to keep this behavior, hence I think this should be kept as is.\n\nI'll add a comment.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429243","messageId":"CAFQ2z_M5bbDgdmVmn-esxg5kAoT9tGS4azvvB5LGMb0UZg9QwA@mail.gmail.com","threadId":"55589","inReplyTo":"87h7hedz9z.fsf@evledraar.gmail.com","subject":"Re: [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-05T14:34:36Z","receivedAt":"2021-07-05T14:34:54Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jul 1, 2021 at 1:55 PM Ævar Arnfjörð Bjarmason wrote:\n> > diff --git a/refs.c b/refs.c\n> > index 8c9490235ea6..bebe3f584da7 100644\n> > --- a/refs.c\n> > +++ b/refs.c\n> > @@ -1675,13 +1675,17 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n> > const char *refname, struct object_id *oid,\n> > struct strbuf *referent, unsigned int *type)\n> > {\n> > + int result, failure;\n>\n> Style nit: Different variables should be on different lines, except\n> where they're the same, so \"int i, j\", not \"int ret, i\". Perhaps \"ret\"\n> and \"errno\" are similar enough, but I'd split them, just my 0.02.\n>\n> Also nit: s/failure/failure_errno/, just like the function signature.\n\ndone.\n\n> > if (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> > return refs_read_special_head(ref_store, refname, oid, referent,\n> > type);\n> > }\n> >\n> > - return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> > - type);\n> > + failure = 0;\n> > + result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> > + type, &failure);\n> > + errno = failure;\n> > + return result;\n> > }\n>\n> Is there some subtlety here where this isn't equivalent to the more\n> simple/straightforward:\n>\n> diff --git a/refs.c b/refs.c\n> index bebe3f584da..49ab7555de9 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1675,17 +1675,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n> const char *refname, struct object_id *oid,\n> struct strbuf *referent, unsigned int *type)\n> {\n> - int result, failure;\n> if (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> return refs_read_special_head(ref_store, refname, oid, referent,\n> type);\n> }\n>\n> - failure = 0;\n> - result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> - type, &failure);\n> - errno = failure;\n> - return result;\n> + errno = 0;\n> + return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> + type, &errno);\n\nThere is a big difference: in this case, in read_raw_ref, the\nfailure_errno pointer is aliased to errno. Which means that things\nlike\n\n *failure_errno = 0\n some_syscall();\n\nmight mysteriously clobber *failure_errno again, which is the kind of\neffect-at-a-distance we're trying to get rid of.\n\n> I can only imagine a case where we think files_read_raw_ref() would\n> encounter a new errno after it assigned to *failure_errno, which is just\n> a couple of strbuf_release() calls.\n>\n> if that is a case we're worried about then like in my comment on 2/8\n> shouldn't we be explicitly checking for such a lost/different errno?\n> I.e. something like (should probably be less fatal than BUG(...):\n>\n> diff --git a/refs.c b/refs.c\n> index bebe3f584da..9584ddae392 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1675,16 +1675,22 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n> const char *refname, struct object_id *oid,\n> struct strbuf *referent, unsigned int *type)\n> {\n> - int result, failure;\n> + int result, failure_errno = 0;\n> +\n> if (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> return refs_read_special_head(ref_store, refname, oid, referent,\n> type);\n> }\n>\n> - failure = 0;\n> + errno = 0;\n> result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> - type, &failure);\n> - errno = failure;\n> + type, &failure_errno);\n> + if (errno)\n> + BUG(\"Got another errno from read_raw_ref()?: %s, failure errno: %s\",\n> + strerror(errno),\n> + strerror(failure_errno));\n\nThe whole idea is to make sure that important (ie. errno values that\ninfluence downstream logic) are separate from unimportant (ie. used\nfor error messages). By adding a BUG() here, you're making a drastic\ndownstream logic change depending on errno, so your example code is\ngoing precisely in the opposite direction of where I want to go, and\nforces implementers of read_raw_ref to walk on eggshells again wrt.\nerrno handling to avoid triggering error messages.\n\n> > + * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT\n> > + * and return -1. If the ref exists but is neither a symbolic ref nor an object\n> > + * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno\n> > + * should not be ENOENT). The files backend may return EISDIR (if the ref name\n> > + * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is\n> > + * another error reading the ref, set failure_errno appropriately and return -1.\n> > *\n>\n> This documentation is a bit confusing in light of the above comments. I\n> assume that \"set failure_errno appropriately\" here means that it will do\n> it, but it's really worded like it's suggesting that you should do it,\n> and does the \"another error\" suggest that a caller may need to deal with\n> an arbitrary errno, not just the ENOENT|EISDIR|ENOTDIR?\n\nTried to clarify this.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429248","messageId":"CAFQ2z_OeO8ALHPv6JzL2UDWqEb6Um8e8nPYMWBH8mmaMvibMhQ@mail.gmail.com","threadId":"55589","inReplyTo":"8735sydvzs.fsf@evledraar.gmail.com","subject":"Re: [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-05T16:09:47Z","receivedAt":"2021-07-05T16:10:00Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Jul 1, 2021 at 3:06 PM Ævar Arnfjörð Bjarmason wrote:\n> > - const char *result = refs_resolve_ref_unsafe_implicit_errno(\n> > - refs, refname, resolve_flags, oid, flags);\n> > - *failure_errno = errno;\n> > - return result;\n> > + int ignore;\n> > + return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n> > + oid, flags, &ignore);\n> > }\n>\n> Okey, so here we have something like the \"caller should explicitly\n> ignore it\" I suggested in <878s2qdy8y.fsf@evledraar.gmail.com>. We no\n> longer set \"errno\" at all for the benefit of the callers of that\n> function, but instead explicitly throw it away, our\n> refs_resolve_ref_unsafe_with_errno() no longer sets errno.\n>\n> But this end-state seems to have resulted in introducing new bugs. A\n> bunch of functions are thin wrappers for\n> refs_resolve_ref_unsafe(). Previously they could inspect errno on a -1\n> return, now they can't.\n>\n> I didn't look at them all, but just the second one I looked at,\n> refs_read_ref_full() has a verify_lock() caller which calls it, and that\n\nIt looks like you were lucky. I looked at the output of\n\negrep -A7 -B7 --color -nH -e\n'(refs_resolve_refdup|resolve_refdup|refs_read_ref_full|read_ref_full|read_ref|refs_ref_exists|ref_exists|resolve_gitlink_ref|index_path)\\('\n*.[ch] */*.[ch]\n\nand the case you pointed out is the only one which inspects errno\nafter calling resolve_ref_unsafe (transitively through the grepped\nfunctions.)\n\n> function then expects a meaningful errno that it in turn ferries up with\n> \"can't verify ref\". It in turn is called by lock_ref_oid_basic(), it\n> ferries up that strbuf with the error, which'll now have a meaningless\n> strerror(0) in it.\n>\n> So it seems to me that the refactoring being done here is halfway done,\n> [..]\n> of what you're starting here. I.e. we have a low-level function that may\n> encounter an errno, and then callers removed from that via:\n>\n> index_path() -> resolve_gitlink_ref() -> refs_resolve_ref_unsafe()\n>..\n> Where the common case for index_path() is to report the error we got,\n> but it doesn't call die_errno() or error_errno() now, but really should,\n\n\"but really should\": Why?\n\nWhy should all functions that resolve a ref print out system level\nerrors? For example, that would entail large changes to sequencer.c\nwhich does \"error\" == \"does not exist\".\n\nAlso, the errors you print out will be fundamentally limited. An error\nin resolving HEAD could be caused by a problem in any of the following\nfiles\n\n .git/HEAD\n .git/packed-refs\n .git/refs/\n .git/refs/heads/\n .git/refs/heads/main\n\nSince we have no way of reporting the filename involved, what is a\nuser going to do with the error message (\"file doesn't exist\")? I\nthink refs_verify_refname_available() provides a more realistic\napproach: create custom error messages for situations that a user is\nmore likely to encounter.\n\nAlso, for something like the reftable backend, there are error classes\nthat have no direct errno counterpart. For example, if reftable finds\ncorrupt zlib data, should it invent a bogus errno code so it can work\nwith the errno reporting system?\n\nI'll take from this review that I should elide the part where errno is\ncleared, and leave it to someone else to figure out a more holistic\nstrategy to error reporting.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429253","messageId":"877di47e2j.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"CAFQ2z_OeO8ALHPv6JzL2UDWqEb6Um8e8nPYMWBH8mmaMvibMhQ@mail.gmail.com","subject":"Re: [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-05T19:08:50Z","receivedAt":"2021-07-05T19:31:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 05 2021, Han-Wen Nienhuys wrote:\n\n> On Thu, Jul 1, 2021 at 3:06 PM Ævar Arnfjörð Bjarmason wrote:\n>> > - const char *result = refs_resolve_ref_unsafe_implicit_errno(\n>> > - refs, refname, resolve_flags, oid, flags);\n>> > - *failure_errno = errno;\n>> > - return result;\n>> > + int ignore;\n>> > + return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n>> > + oid, flags, &ignore);\n>> > }\n>>\n>> Okey, so here we have something like the \"caller should explicitly\n>> ignore it\" I suggested in <878s2qdy8y.fsf@evledraar.gmail.com>. We no\n>> longer set \"errno\" at all for the benefit of the callers of that\n>> function, but instead explicitly throw it away, our\n>> refs_resolve_ref_unsafe_with_errno() no longer sets errno.\n>>\n>> But this end-state seems to have resulted in introducing new bugs. A\n>> bunch of functions are thin wrappers for\n>> refs_resolve_ref_unsafe(). Previously they could inspect errno on a -1\n>> return, now they can't.\n>>\n>> I didn't look at them all, but just the second one I looked at,\n>> refs_read_ref_full() has a verify_lock() caller which calls it, and that\n>\n> It looks like you were lucky. I looked at the output of\n\nMaybe so...\n\n> egrep -A7 -B7 --color -nH -e\n> '(refs_resolve_refdup|resolve_refdup|refs_read_ref_full|read_ref_full|read_ref|refs_ref_exists|ref_exists|resolve_gitlink_ref|index_path)\\('\n> *.[ch] */*.[ch]\n\nFWIW this gives better results:\n\n git grep -E -W '\\b(refs_resolve_refdup|resolve_refdup|refs_read_ref_full|read_ref_full|read_ref|refs_ref_exists|ref_exists|resolve_gitlink_ref|index_path)\\('\n\n> and the case you pointed out is the only one which inspects errno\n> after calling resolve_ref_unsafe (transitively through the grepped\n> functions.)\n\nIsn't the `errno == EISDIR` in files_copy_or_rename_ref() another one?\nIt's just after calling refs_read_ref_full().\n\nYes, but aside from that it does look like I got lucky. I assumed this\nissue was more widespread. Phew!\n\n>> function then expects a meaningful errno that it in turn ferries up with\n>> \"can't verify ref\". It in turn is called by lock_ref_oid_basic(), it\n>> ferries up that strbuf with the error, which'll now have a meaningless\n>> strerror(0) in it.\n>>\n>> So it seems to me that the refactoring being done here is halfway done,\n>> [..]\n>> of what you're starting here. I.e. we have a low-level function that may\n>> encounter an errno, and then callers removed from that via:\n>>\n>> index_path() -> resolve_gitlink_ref() -> refs_resolve_ref_unsafe()\n>>..\n>> Where the common case for index_path() is to report the error we got,\n>> but it doesn't call die_errno() or error_errno() now, but really should,\n>\n> \"but really should\": Why?\n>\n> Why should all functions that resolve a ref print out system level\n> errors? For example, that would entail large changes to sequencer.c\n> which does \"error\" == \"does not exist\".\n>\n> Also, the errors you print out will be fundamentally limited. An error\n> in resolving HEAD could be caused by a problem in any of the following\n> files\n>\n> .git/HEAD\n> .git/packed-refs\n> .git/refs/\n> .git/refs/heads/\n> .git/refs/heads/main\n>\n> Since we have no way of reporting the filename involved, what is a\n> user going to do with the error message (\"file doesn't exist\")? I\n> think refs_verify_refname_available() provides a more realistic\n> approach: create custom error messages for situations that a user is\n> more likely to encounter.\n\nLet me rephrase: \"we really shouldn't\", as in I agree that this API\nsucks and that refs_verify_refname_available() is a much better\napproach.\n\nBut as long as errno was the abstraction we were passing around it\nseemed like e.g. callers like process_directory() should be getting the\nerrno and using the *_errno() error functions, but of course it would be\nmuch better if they used a better error return flow.\n\nBut having looked at the grep above with fresh eyes I think it's fine to\nleave it at the refs_resolve_ref_unsafe_with_errno() boundary. The error\nmessages are clear enough, and we're trying to phase out errno anyway.\n\nI was more paranoid about things that would actually do equality checks\nagainst the errno, as it appearsfiles_copy_or_rename_ref() is still\ndoing.\n\n> Also, for something like the reftable backend, there are error classes\n> that have no direct errno counterpart. For example, if reftable finds\n> corrupt zlib data, should it invent a bogus errno code so it can work\n> with the errno reporting system?\n\nNo, that would be crazy. We should be moving away from errno\nentirely. My concern is only that we might be introducing subtle bugs\nfurther up the callstack now that we clear \"errno\" ...\n\n> I'll take from this review that I should elide the part where errno is\n> cleared, and leave it to someone else to figure out a more holistic\n> strategy to error reporting.\n\n...or if we don't clear errno introducing those sorts of subtle bugs\nwhen we use reftable instead of the files backend. I.e. no, we really\nshould be clearing errno, if not in this series then in some other\npre-reftable series.\n\nTo not do so would be kicking this particular can down the road, and\nleaving those bugs for reftable users to find. Which given that I've\nfound a few cases with no test coverage...\n"},{"id":"429255","messageId":"CAFQ2z_MuZJ7FmaZ8pjb51dhSHuHYipesHn8hum3Af49qwiZL6Q@mail.gmail.com","threadId":"55589","inReplyTo":"877di47e2j.fsf@evledraar.gmail.com","subject":"Re: [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-05T19:39:08Z","receivedAt":"2021-07-05T19:39:22Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Mon, Jul 5, 2021 at 9:31 PM Ævar Arnfjörð Bjarmason wrote:\n> > and the case you pointed out is the only one which inspects errno\n> > after calling resolve_ref_unsafe (transitively through the grepped\n> > functions.)\n>\n> Isn't the `errno == EISDIR` in files_copy_or_rename_ref() another one?\n> It's just after calling refs_read_ref_full().\n\nno. It's handling the EISDIR for refs_delete_ref.\n\n> > I'll take from this review that I should elide the part where errno is\n> > cleared, and leave it to someone else to figure out a more holistic\n> > strategy to error reporting.\n>\n> ...or if we don't clear errno introducing those sorts of subtle bugs\n> when we use reftable instead of the files backend. I.e. no, we really\n> should be clearing errno, if not in this series then in some other\n> pre-reftable series.\n>\n> To not do so would be kicking this particular can down the road, and\n> leaving those bugs for reftable users to find. Which given that I've\n> found a few cases with no test coverage...\n\ntoo late :-)\n\nI've already kicked the can down the road by not clearing errno in the\ncurrent version of the series.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429260","messageId":"pull.1012.v3.git.git.1625518566.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v2.git.git.1623329868.gitgitgadget@gmail.com","subject":"[PATCH v3 0/5] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-05T20:56:00Z","receivedAt":"2021-07-05T20:56:14Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"v5\n\n * address Ævar's comment; punt on clearing errno.\n\nHan-Wen Nienhuys (5):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: make errno output explicit for refs_resolve_ref_unsafe\n\n refs.c | 52 ++++++++++++++++++++++++++-----------\n refs/debug.c | 4 +--\n refs/files-backend.c | 60 ++++++++++++++++++++-----------------------\n refs/packed-backend.c | 15 ++++++-----\n refs/refs-internal.h | 34 ++++++++++++++++--------\n 5 files changed, 98 insertions(+), 67 deletions(-)\n\n\nbase-commit: 670b81a890388c60b7032a4f5b879f2ece8c4558\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v3\nPull-Request: https://github.com/git/git/pull/1012\n\nRange-diff vs v2:\n\n 1: f9b92e62b59 = 1: e2a0e5387ab refs: remove EINVAL errno output from specification of read_raw_ref_fn\n 2: cbe09a48036 ! 2: c594c9c5c67 refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ Commit message\n \n Signed-off-by: Han-Wen Nienhuys \n Reviewed-by: Jonathan Tan \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n -\t\tlast_errno = errno;\n +\t\tint last_errno = errno;\n \t\tif (last_errno != ENOTDIR ||\n - \t\t !refs_verify_refname_available(&refs->base, refname,\n - \t\t\t\t\t\t extras, skip, err))\n +-\t\t !refs_verify_refname_available(&refs->base, refname,\n +-\t\t\t\t\t\t extras, skip, err))\n ++\t\t /* in case of D/F conflict, try to generate a better error\n ++\t\t * message. If that fails, fall back to strerror(ENOTDIR).\n ++\t\t */\n ++\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n ++\t\t\t\t\t\t skip, err))\n + \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n + \t\t\t\t refname, strerror(last_errno));\n + \n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n 3: 3e2831e59c8 ! 3: b017caf54ba refs: make errno output explicit for read_raw_ref_fn\n @@ Metadata\n ## Commit message ##\n refs: make errno output explicit for read_raw_ref_fn\n \n + This makes it explicit how alternative ref backends should report errors in\n + read_raw_ref_fn.\n +\n read_raw_ref_fn needs to supply a credible errno for a number of cases. These\n are primarily:\n \n @@ Commit message\n \n Signed-off-by: Han-Wen Nienhuys \n Reviewed-by: Jonathan Tan \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type)\n {\n -+\tint result, failure;\n ++\tint result;\n ++\tint failure_errno;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n \n -\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n -\t\t\t\t\t type);\n -+\tfailure = 0;\n ++\tfailure_errno = 0;\n +\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n -+\t\t\t\t\t type, &failure);\n -+\terrno = failure;\n ++\t\t\t\t\t type, &failure_errno);\n ++\tif (failure_errno)\n ++\t\terrno = failure_errno;\n +\treturn result;\n }\n \n @@ refs/files-backend.c: stat_ref:\n \n out:\n -\tsave_errno = errno;\n -+\tif (failure_errno)\n -+\t\t*failure_errno = errno;\n ++\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n -\terrno = save_errno;\n @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,\n - * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n - * (errno should not be ENOENT) If there is another error reading the\n - * ref, set errno appropriately and return -1.\n -+ * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT\n -+ * and return -1. If the ref exists but is neither a symbolic ref nor an object\n -+ * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno\n -+ * should not be ENOENT). The files backend may return EISDIR (if the ref name\n -+ * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is\n -+ * another error reading the ref, set failure_errno appropriately and return -1.\n ++ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n ++ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n ++ * type, and return -1 (failure_errno should not be ENOENT)\n ++ *\n ++ * failure_errno provides errno codes that are interpreted beyond error\n ++ * reporting. The following error codes have special meaning:\n ++ * * ENOENT: the ref doesn't exist\n ++ * * EISDIR: ref name is a directory\n ++ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n 4: 11b2184044d < -: ----------- refs: make errno output explicit for refs_resolve_ref_unsafe\n 6: 2b346caf1ae ! 4: 4aaa9d3bd6f refs: add failure_errno to refs_read_raw_ref() signature\n @@ Metadata\n ## Commit message ##\n refs: add failure_errno to refs_read_raw_ref() signature\n \n - This makes the errno output of refs_read_raw_ref explicit.\n + This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n \n Signed-off-by: Han-Wen Nienhuys \n Reviewed-by: Jonathan Tan \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: done:\n @@ refs.c: done:\n +\t\t struct object_id *oid, struct strbuf *referent,\n +\t\t unsigned int *type, int *failure_errno)\n {\n --\tint result, failure;\n +-\tint result;\n +-\tint failure_errno;\n ++\tint ignore;\n +\tif (failure_errno)\n +\t\t*failure_errno = 0;\n ++\telse\n ++\t\tfailure_errno = &ignore;\n ++\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n --\tfailure = 0;\n +-\tfailure_errno = 0;\n -\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n --\t\t\t\t\t type, &failure);\n --\terrno = failure;\n +-\t\t\t\t\t type, &failure_errno);\n +-\tif (failure_errno)\n +-\t\terrno = failure_errno;\n -\treturn result;\n +\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n +\t\t\t\t\t type, failure_errno);\n }\n \n - /* This function needs to return a meaningful errno on failure */\n +-/* This function needs to return a meaningful errno on failure */\n +-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n +-\t\t\t\t const char *refname,\n +-\t\t\t\t int resolve_flags,\n +-\t\t\t\t struct object_id *oid, int *flags)\n ++const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n ++\t\t\t\t\t const char *refname,\n ++\t\t\t\t\t int resolve_flags,\n ++\t\t\t\t\t struct object_id *oid,\n ++\t\t\t\t\t int *flags, int *failure_errno)\n + {\n + \tstatic struct strbuf sb_refname = STRBUF_INIT;\n + \tstruct object_id unused_oid;\n + \tint unused_flags;\n ++\tint unused_errno;\n + \tint symref_count;\n + \n + \tif (!oid)\n +@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n + \t\tflags = &unused_flags;\n + \n + \t*flags = 0;\n ++\tif (!failure_errno)\n ++\t\tfailure_errno = &unused_errno;\n ++\t*failure_errno = 0;\n + \n + \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n + \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n +\t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n ++\t\t\t*failure_errno = read_failure;\n ++\n \t\t\t/* In reading mode, refs must eventually resolve */\n + \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n + \t\t\t\treturn NULL;\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n +@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n + \treturn NULL;\n + }\n + \n ++const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n ++\t\t\t\t int resolve_flags, struct object_id *oid,\n ++\t\t\t\t int *flags)\n ++{\n ++\tint ignore = 0;\n ++\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n ++\t\t\t\t\t\t oid, flags, &ignore);\n ++}\n ++\n + /* backend functions */\n + int refs_init_db(struct strbuf *err)\n + {\n @@ refs.c: int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n @@ refs/files-backend.c: stat_ref:\n \t\t}\n \n ## refs/packed-backend.c ##\n -@@ refs/packed-backend.c: static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n - \n - \tif (!rec) {\n - \t\t/* refname is not a packed reference. */\n --\t\t*failure_errno = ENOENT;\n -+\t\tif (failure_errno)\n -+\t\t\t*failure_errno = ENOENT;\n - \t\treturn -1;\n - \t}\n - \n @@ refs/packed-backend.c: int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n -+\t\tint failure;\n ++\t\tint failure_errno = 0;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n @@ refs/packed-backend.c: int is_packed_transaction_needed(struct ref_store *ref_st\n -\t\t\t\t &oid, &referent, &type) ||\n -\t\t errno != ENOENT) {\n +\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n -+\t\t\t\t &referent, &type, &failure) ||\n -+\t\t failure != ENOENT) {\n ++\t\t\t\t &referent, &type, &failure_errno) ||\n ++\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\n @@ refs/refs-internal.h: struct ref_update {\n +\t\t struct object_id *oid, struct strbuf *referent,\n +\t\t unsigned int *type, int *failure_errno);\n \n - /* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n - * failure. */\n + /*\n + * Write an error to `err` and return a nonzero value iff the same\n 5: 005ee8e6fb2 ! 5: ed5347d7bb6 refs: use refs_resolve_ref_unsafe_with_errno() where needed\n @@ Metadata\n Author: Han-Wen Nienhuys \n \n ## Commit message ##\n - refs: use refs_resolve_ref_unsafe_with_errno() where needed\n + refs: make errno output explicit for refs_resolve_ref_unsafe\n +\n + This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\n + contract for the errno output explicit. The implementation still relies on\n + the global errno variable to ensure no side effects of this refactoring.\n \n lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\n - that needs error information to make logic decisions.\n + that needs error information to make logic decisions, so update that caller\n \n Signed-off-by: Han-Wen Nienhuys \n Reviewed-by: Jonathan Tan \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \tif (!resolved) {\n -\t\tint last_errno = errno;\n -\t\tif (last_errno != ENOTDIR ||\n --\t\t !refs_verify_refname_available(&refs->base, refname,\n --\t\t\t\t\t\t extras, skip, err))\n +\t\tif (resolve_errno != ENOTDIR ||\n -+\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n -+\t\t\t\t\t\t skip, err))\n + \t\t /* in case of D/F conflict, try to generate a better error\n + \t\t * message. If that fails, fall back to strerror(ENOTDIR).\n + \t\t */\n + \t\t !refs_verify_refname_available(&refs->base, refname, extras,\n + \t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n -\t\t\t\t refname, strerror(last_errno));\n +\t\t\t\t refname, strerror(resolve_errno));\n \n \t\tgoto error_return;\n \t}\n +\n + ## refs/refs-internal.h ##\n +@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \t\t struct object_id *oid, struct strbuf *referent,\n + \t\t unsigned int *type, int *failure_errno);\n + \n ++/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n ++ * failure. */\n ++const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n ++\t\t\t\t\t const char *refname,\n ++\t\t\t\t\t int resolve_flags,\n ++\t\t\t\t\t struct object_id *oid,\n ++\t\t\t\t\t int *flags, int *failure_errno);\n ++\n + /*\n + * Write an error to `err` and return a nonzero value iff the same\n + * refname appears multiple times in `refnames`. `refnames` must be\n 7: d8651621968 < -: ----------- refs: clear errno return in refs_resolve_ref_unsafe()\n 8: 2a9ebe43dea < -: ----------- refs: explicitly propagate errno from refs_read_raw_ref\n\n-- \ngitgitgadget\n"},{"id":"429261","messageId":"e2a0e5387ab4d9bbe54d5d5303171a1a0a4fc8be.1625518566.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v3.git.git.1625518566.gitgitgadget@gmail.com","subject":"[PATCH v3 1/5] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-05T20:56:01Z","receivedAt":"2021-07-05T20:56:17Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 467f4b3c936..f4445e32904 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \ngitgitgadget\n\n"},{"id":"429262","messageId":"c594c9c5c676e9e4ae52adff8d2a0e985a2c481d.1625518566.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v3.git.git.1625518566.gitgitgadget@gmail.com","subject":"[PATCH v3 2/5] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-05T20:56:02Z","receivedAt":"2021-07-05T20:56:22Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 16 ++++++----------\n 1 file changed, 6 insertions(+), 10 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 677b7e4cdd2..83ddfb3b627 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t * to remain.\n \t\t */\n \t\tif (remove_empty_directories(&ref_file)) {\n-\t\t\tlast_errno = errno;\n \t\t\tif (!refs_verify_refname_available(\n \t\t\t\t\t &refs->base,\n \t\t\t\t\t refname, extras, skip, err))\n@@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tlast_errno = errno;\n+\t\tint last_errno = errno;\n \t\tif (last_errno != ENOTDIR ||\n-\t\t !refs_verify_refname_available(&refs->base, refname,\n-\t\t\t\t\t\t extras, skip, err))\n+\t\t /* in case of D/F conflict, try to generate a better error\n+\t\t * message. If that fails, fall back to strerror(ENOTDIR).\n+\t\t */\n+\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n+\t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n \t\t\t\t refname, strerror(last_errno));\n \n@@ -981,20 +981,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n \t\t\t\t\t extras, skip, err)) {\n-\t\tlast_errno = ENOTDIR;\n \t\tgoto error_return;\n \t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n \n \tif (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {\n-\t\tlast_errno = errno;\n \t\tgoto error_return;\n \t}\n \tgoto out;\n@@ -1005,7 +1002,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"429263","messageId":"b017caf54bae36b956f5a95d11d8a09d0fc59f96.1625518566.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v3.git.git.1625518566.gitgitgadget@gmail.com","subject":"[PATCH v3 3/5] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-05T20:56:03Z","receivedAt":"2021-07-05T20:56:24Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes it explicit how alternative ref backends should report errors in\nread_raw_ref_fn.\n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 10 ++++++++--\n refs/debug.c | 4 ++--\n refs/files-backend.c | 23 +++++++++++------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 20 ++++++++++++--------\n 5 files changed, 37 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 8c9490235ea..5e5e3af8da0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1675,13 +1675,19 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type)\n {\n+\tint result;\n+\tint failure_errno;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\tfailure_errno = 0;\n+\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, &failure_errno);\n+\tif (failure_errno)\n+\t\terrno = failure_errno;\n+\treturn result;\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 7db4abccc34..f12413a9bc0 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \toidcpy(oid, null_oid());\n \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 83ddfb3b627..f98b1be9386 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,9 @@ stat_ref:\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno = 0;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -629,9 +628,9 @@ retry:\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -655,7 +654,7 @@ retry:\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -693,13 +692,13 @@ retry:\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex dfecdbc1db6..a457f18e93c 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex f4445e32904..79dfb3af484 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n+ * type, and return -1 (failure_errno should not be ENOENT)\n+ *\n+ * failure_errno provides errno codes that are interpreted beyond error\n+ * reporting. The following error codes have special meaning:\n+ * * ENOENT: the ref doesn't exist\n+ * * EISDIR: ref name is a directory\n+ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \ngitgitgadget\n\n"},{"id":"429264","messageId":"4aaa9d3bd6f01b83932b25a9ba18ec9c1106405a.1625518566.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v3.git.git.1625518566.gitgitgadget@gmail.com","subject":"[PATCH v3 4/5] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-05T20:56:04Z","receivedAt":"2021-07-05T20:56:30Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 60 +++++++++++++++++++++++++++----------------\n refs/files-backend.c | 8 +++---\n refs/packed-backend.c | 7 ++---\n refs/refs-internal.h | 6 ++---\n 4 files changed, 49 insertions(+), 32 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 5e5e3af8da0..25d80e544d0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1671,34 +1671,35 @@ done:\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n-\tint result;\n-\tint failure_errno;\n+\tint ignore;\n+\tif (failure_errno)\n+\t\t*failure_errno = 0;\n+\telse\n+\t\tfailure_errno = &ignore;\n+\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\tfailure_errno = 0;\n-\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &failure_errno);\n-\tif (failure_errno)\n-\t\terrno = failure_errno;\n-\treturn result;\n+\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n+\tint unused_errno;\n \tint symref_count;\n \n \tif (!oid)\n@@ -1707,6 +1708,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\tflags = &unused_flags;\n \n \t*flags = 0;\n+\tif (!failure_errno)\n+\t\tfailure_errno = &unused_errno;\n+\t*failure_errno = 0;\n \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n@@ -1728,11 +1732,14 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n+\t\tint read_failure = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n+\t\t\t*failure_errno = read_failure;\n+\n \t\t\t/* In reading mode, refs must eventually resolve */\n \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n \t\t\t\treturn NULL;\n@@ -1742,9 +1749,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (read_failure != ENOENT && read_failure != EISDIR &&\n+\t\t\t read_failure != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -1783,6 +1789,15 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tint ignore = 0;\n+\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &ignore);\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n@@ -2244,7 +2259,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, NULL)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f98b1be9386..f38c9703504 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -383,8 +383,8 @@ stat_ref:\n \tif (lstat(path, &st) < 0) {\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -423,8 +423,8 @@ stat_ref:\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a457f18e93c..530dba29788 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure_errno = 0;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure_errno) ||\n+\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 79dfb3af484..54f57c6a2df 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /*\n * Write an error to `err` and return a nonzero value iff the same\n-- \ngitgitgadget\n\n"},{"id":"429265","messageId":"ed5347d7bb61257a164750835b754babb876e36a.1625518566.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v3.git.git.1625518566.gitgitgadget@gmail.com","subject":"[PATCH v3 5/5] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-05T20:56:05Z","receivedAt":"2021-07-05T20:56:31Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions, so update that caller\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Jonathan Tan \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 15 ++++++++-------\n refs/refs-internal.h | 8 ++++++++\n 2 files changed, 16 insertions(+), 7 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f38c9703504..8506c8b3bde 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -923,6 +923,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -935,10 +936,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t refname, resolve_flags,\n-\t\t\t\t\t &lock->old_oid, type);\n-\tif (!resolved && errno == EISDIR) {\n+\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\t\tresolve_flags,\n+\t\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t\t&resolve_errno);\n+\tif (!resolved && resolve_errno == EISDIR) {\n \t\t/*\n \t\t * we are trying to lock foo but we used to\n \t\t * have foo/bar which now does not exist;\n@@ -958,15 +960,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tint last_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n+\t\tif (resolve_errno != ENOTDIR ||\n \t\t /* in case of D/F conflict, try to generate a better error\n \t\t * message. If that fails, fall back to strerror(ENOTDIR).\n \t\t */\n \t\t !refs_verify_refname_available(&refs->base, refname, extras,\n \t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n+\t\t\t\t refname, strerror(resolve_errno));\n \n \t\tgoto error_return;\n \t}\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 54f57c6a2df..c52a64b081b 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n+ * failure. */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n+\n /*\n * Write an error to `err` and return a nonzero value iff the same\n * refname appears multiple times in `refnames`. `refnames` must be\n-- \ngitgitgadget\n"},{"id":"429273","messageId":"87v95o5ku8.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"pull.1012.v3.git.git.1625518566.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/5] refs: cleanup errno sideband ref related functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-06T00:38:49Z","receivedAt":"2021-07-06T00:48:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 05 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> v5\n\nv3?\n\n> * address Ævar's comment; punt on clearing errno.\n> [...]\n> \n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> + Reviewed-by: Ævar Arnfjörð Bjarmason \n\nFWIW per Documentation/SubmittingPatches:\n \n . `Reviewed-by:`, unlike the other tags, can only be offered by the\n reviewer and means that she is completely satisfied that the patch\n is ready for application. It is usually offered only after a\n detailed review.\n\nIt's not that I'm hard to please, but I can honestly say that I don't\nquite understand some parts of what you're gonig for, so that trailer is\nprobably premature :)\n\n> ## refs/files-backend.c ##\n> @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)\n> @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n> -\t\tlast_errno = errno;\n> +\t\tint last_errno = errno;\n> \t\tif (last_errno != ENOTDIR ||\n> - \t\t !refs_verify_refname_available(&refs->base, refname,\n> - \t\t\t\t\t\t extras, skip, err))\n> +-\t\t !refs_verify_refname_available(&refs->base, refname,\n> +-\t\t\t\t\t\t extras, skip, err))\n> ++\t\t /* in case of D/F conflict, try to generate a better error\n> ++\t\t * message. If that fails, fall back to strerror(ENOTDIR).\n> ++\t\t */\n> ++\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n> ++\t\t\t\t\t\t skip, err))\n> + \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n> + \t\t\t\t refname, strerror(last_errno));\n> + \n> @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \tif (is_null_oid(&lock->old_oid) &&\n> \t refs_verify_refname_available(refs->packed_ref_store, refname,\n> 3: 3e2831e59c8 ! 3: b017caf54ba refs: make errno output explicit for read_raw_ref_fn\n> @@ Metadata\n> ## Commit message ##\n> refs: make errno output explicit for read_raw_ref_fn\n> \n> + This makes it explicit how alternative ref backends should report errors in\n> + read_raw_ref_fn.\n> +\n> read_raw_ref_fn needs to supply a credible errno for a number of cases. These\n> are primarily:\n> \n> @@ Commit message\n> \n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-by: Jonathan Tan \n> + Reviewed-by: Ævar Arnfjörð Bjarmason \n> \n> ## refs.c ##\n> @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n> \t\t const char *refname, struct object_id *oid,\n> \t\t struct strbuf *referent, unsigned int *type)\n> {\n> -+\tint result, failure;\n> ++\tint result;\n> ++\tint failure_errno;\n> \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n> \t\t\t\t\t type);\n> @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n> \n> -\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> -\t\t\t\t\t type);\n> -+\tfailure = 0;\n> ++\tfailure_errno = 0;\n> +\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> -+\t\t\t\t\t type, &failure);\n> -+\terrno = failure;\n> ++\t\t\t\t\t type, &failure_errno);\n> ++\tif (failure_errno)\n> ++\t\terrno = failure_errno;\n> +\treturn result;\n> }\n\nTo rephrase my comment on the v2 to hopefully better get at the\npoint/question I had.\n\nIt wasn't that I don't get why you wouldn't save/restore errno in\ngeneral.\n\nIt's that the pattern of doing so seems backwards to me. I.e. surely the\ngoal here should be to one function at a time, and from the bottom-up,\nfigure out where we rely on \"errno\" and convert that to a\n\"failure_errno\".\n\nInstead not even files_read_raw_ref() resets \"errno = 0\" at the end, so\nthe errno /there/ can propagate upwards, and in this v3 we're not clearing it at all.\n\nI'm all for clearing it as mentioned in another mail, surely that should\nbe the point of this whole thing, i.e. to refactor this part of the API\nso that we're assured that nothing upstream of us relies on errno & prep\nthings for reftable (which won't set it at all, except to fake it).\n\nHaving dug a bit further, it seems what you're doing, whether it's\nintentional or not, is relying on the parse_loose_ref_contents() setting\nEINVAL, but you clobber your *failure_errno with it whether it returned\n-1 or not.\n\nSeemingly to make that logic work in files_read_raw_ref() you, after\ngetting an errno, assigned it \"errno\", instead of to a\nsaved_errno\". Thus requiring an inverse of the pattern where you need a\n\"saved_errno\" dance to save away an errno you explicitly don't care\nabout (because you didn't save the one you wanted earlier).\n\nI think something like the following diff on top, this whole dance was\nonly needed because you didn't pass the failure_errno down to\nrefs_read_special_head() and parse_loose_ref_contents().\n\ndiff --git a/refs.c b/refs.c\nindex 25d80e544d0..eca7310e7a4 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n \n static int refs_read_special_head(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t\t int *failure_errno)\n {\n \tstruct strbuf full_path = STRBUF_INIT;\n \tstruct strbuf content = STRBUF_INIT;\n@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n \t\tgoto done;\n \n-\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n+\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n+\t\t\t\t\t failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\n@@ -1683,7 +1685,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8506c8b3bde..823325cc97f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -381,11 +381,12 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\tgoto out;\n \n \tif (lstat(path, &st) < 0) {\n-\t\tif (errno != ENOENT)\n+\t\t*failure_errno = errno;\n+\t\tif (*failure_errno != ENOENT)\n \t\t\tgoto out;\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, NULL)) {\n-\t\t\terrno = ENOENT;\n+\t\t\t*failure_errno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -396,7 +397,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tif (S_ISLNK(st.st_mode)) {\n \t\tstrbuf_reset(&sb_contents);\n \t\tif (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {\n-\t\t\tif (errno == ENOENT || errno == EINVAL)\n+\t\t\t*failure_errno = errno;\n+\t\t\tif (*failure_errno == ENOENT || *failure_errno == EINVAL)\n \t\t\t\t/* inconsistent with lstat; retry */\n \t\t\t\tgoto stat_ref;\n \t\t\telse\n@@ -425,7 +427,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t */\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, NULL)) {\n-\t\t\terrno = EISDIR;\n+\t\t\t*failure_errno = errno = EISDIR;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -438,7 +440,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t */\n \tfd = open(path, O_RDONLY);\n \tif (fd < 0) {\n-\t\tif (errno == ENOENT && !S_ISLNK(st.st_mode))\n+\t\tif (*failure_errno == ENOENT && !S_ISLNK(st.st_mode))\n \t\t\t/* inconsistent with lstat; retry */\n \t\t\tgoto stat_ref;\n \t\telse\n@@ -446,26 +448,25 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t}\n \tstrbuf_reset(&sb_contents);\n \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n-\t\tint save_errno = errno;\n \t\tclose(fd);\n-\t\terrno = save_errno;\n \t\tgoto out;\n \t}\n \tclose(fd);\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n-\tret = parse_loose_ref_contents(buf, oid, referent, type);\n+\tret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);\n \n out:\n-\t*failure_errno = errno;\n+\terrno = 0; /* saved in *failure_errno */\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n \treturn ret;\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -484,7 +485,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n \tif (parse_oid_hex(buf, oid, &p) ||\n \t (*p != '\\0' && !isspace(*p))) {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n \treturn 0;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex c52a64b081b..530999f50df 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -700,7 +700,8 @@ struct ref_store {\n * Parse contents of a loose ref file.\n */\n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\n"},{"id":"429280","messageId":"CAFQ2z_N_J499X_wd4RGR7jobs3O3NvQuobJ9UcQijkNYN712dw@mail.gmail.com","threadId":"55589","inReplyTo":"87v95o5ku8.fsf@evledraar.gmail.com","subject":"Re: [PATCH v3 0/5] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-06T09:53:08Z","receivedAt":"2021-07-06T09:53:24Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Tue, Jul 6, 2021 at 2:48 AM Ævar Arnfjörð Bjarmason wrote:\n>\n>\n> On Mon, Jul 05 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n>\n> > v5\n>\n> v3?\n\nI erroneously closed out the github PR I used for the first 2 versions\nof this change series. (https://github.com/git/git/pull/1011)\n\n> > * address Ævar's comment; punt on clearing errno.\n> > [...]\n> >\n> > Signed-off-by: Han-Wen Nienhuys \n> > Reviewed-by: Jonathan Tan \n> > + Reviewed-by: Ævar Arnfjörð Bjarmason \n>\n> FWIW per Documentation/SubmittingPatches:\n>\n> . `Reviewed-by:`, unlike the other tags, can only be offered by the\n> reviewer and means that she is completely satisfied that the patch\n> is ready for application. It is usually offered only after a\n> detailed review.\n>\n> It's not that I'm hard to please, but I can honestly say that I don't\n> quite understand some parts of what you're gonig for, so that trailer is\n> probably premature :)\n\noh, ok. Removed them.\n\nDoes that mean you'd have to repost my patchseries just to add the\nreviewed-by header?\n\n> To rephrase my comment on the v2 to hopefully better get at the\n> point/question I had.\n>\n> It wasn't that I don't get why you wouldn't save/restore errno in\n> general.\n>\n> It's that the pattern of doing so seems backwards to me. I.e. surely the\n> goal here should be to one function at a time, and from the bottom-up,\n> figure out where we rely on \"errno\" and convert that to a\n> \"failure_errno\".\n>\n> Instead not even files_read_raw_ref() resets \"errno = 0\" at the end, so\n> the errno /there/ can propagate upwards, and in this v3 we're not clearing it at all.\n\nI would really want errno to be an explicit output for more functions\nin refs.h, but I don't have the time to see that through, and it's a\ndistraction from the reftable work, so I have changed the objective of\nthe patch series.\n\nI'm limiting myself to explicitly propagating errno values that are\nused beyond error reporting. The EINVAL value for\nparse_loose_ref_contents is a borderline case: it has an extra \"if ()\"\nbody in lock_raw_ref, but it's for an error message.\n\nThis means that we should avoid clearing errno, because we'd otherwise\nhave to track down all the places where it's being put into\nstrerror(). The one case you found earlier is still there, and by not\nclearing errno, I can keep this series smaller by not also having to\nrework verify_lock/read_refs_full.\n\nIn general, it's frustrating to observe that the files backend is a\ncomplex beast that nobody understands, but in order to replace it with\nsomething more principled, I have to spend lots of time cleaning it\nup.\n\n> Having dug a bit further, it seems what you're doing, whether it's\n> intentional or not, is relying on the parse_loose_ref_contents() setting\n> EINVAL, but you clobber your *failure_errno with it whether it returned\n> -1 or not.\n\nI've added a failure_errno argument parse_loose_ref_contents now.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429288","messageId":"87sg0r5x0m.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"CAFQ2z_N_J499X_wd4RGR7jobs3O3NvQuobJ9UcQijkNYN712dw@mail.gmail.com","subject":"Re: [PATCH v3 0/5] refs: cleanup errno sideband ref related functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-06T14:27:08Z","receivedAt":"2021-07-06T14:44:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jul 06 2021, Han-Wen Nienhuys wrote:\n\n> On Tue, Jul 6, 2021 at 2:48 AM Ævar Arnfjörð Bjarmason wrote:\n>>\n>>\n>> On Mon, Jul 05 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n>>\n>> > v5\n>>\n>> v3?\n>\n> I erroneously closed out the github PR I used for the first 2 versions\n> of this change series. (https://github.com/git/git/pull/1011)\n>\n>> > * address Ævar's comment; punt on clearing errno.\n>> > [...]\n>> >\n>> > Signed-off-by: Han-Wen Nienhuys \n>> > Reviewed-by: Jonathan Tan \n>> > + Reviewed-by: Ævar Arnfjörð Bjarmason \n>>\n>> FWIW per Documentation/SubmittingPatches:\n>>\n>> . `Reviewed-by:`, unlike the other tags, can only be offered by the\n>> reviewer and means that she is completely satisfied that the patch\n>> is ready for application. It is usually offered only after a\n>> detailed review.\n>>\n>> It's not that I'm hard to please, but I can honestly say that I don't\n>> quite understand some parts of what you're gonig for, so that trailer is\n>> probably premature :)\n>\n> oh, ok. Removed them.\n>\n> Does that mean you'd have to repost my patchseries just to add the\n> reviewed-by header?\n\nIt's fine to keep it. I just noted this (with a \":)\") since I\nReviewed-By is understood by Junio (I believe) to mean something closer\nto \"this person has fully understood this code & endorses it\". So far I\nwas only confident to say I was on right-hand-side of that \"&\",\ngive-or-take suggested changes :)\n\nWith the below suggestions for a v4 you can keep my Reviewed-By if you'd\nlike.\n\n>> To rephrase my comment on the v2 to hopefully better get at the\n>> point/question I had.\n>>\n>> It wasn't that I don't get why you wouldn't save/restore errno in\n>> general.\n>>\n>> It's that the pattern of doing so seems backwards to me. I.e. surely the\n>> goal here should be to one function at a time, and from the bottom-up,\n>> figure out where we rely on \"errno\" and convert that to a\n>> \"failure_errno\".\n>>\n>> Instead not even files_read_raw_ref() resets \"errno = 0\" at the end, so\n>> the errno /there/ can propagate upwards, and in this v3 we're not clearing it at all.\n>\n> I would really want errno to be an explicit output for more functions\n> in refs.h, but I don't have the time to see that through, and it's a\n> distraction from the reftable work, so I have changed the objective of\n> the patch series.\n>\n> I'm limiting myself to explicitly propagating errno values that are\n> used beyond error reporting. The EINVAL value for\n> parse_loose_ref_contents is a borderline case: it has an extra \"if ()\"\n> body in lock_raw_ref, but it's for an error message.\n>\n> This means that we should avoid clearing errno, because we'd otherwise\n> have to track down all the places where it's being put into\n> strerror(). The one case you found earlier is still there, and by not\n> clearing errno, I can keep this series smaller by not also having to\n> rework verify_lock/read_refs_full.\n\nI really think your v2 (or v4?, whatever) that set \"errno = 0\" plus\neyeballing that grep command as we've done is justification enough to\nkeep the \"errno = 0\".\n\nI just had the comment to the effect of \"uh, oh, maybe we've missed more\nof these\" on v2, but I think since then with your grep we can be fairly\nsure we haven't.\n\nI.e. the codepaths involved return non-zero anyway, so any breakage\nwould be limited to ignoring that and directly checking errno, or using\nerrno to format error messages.\n\nIf there is something like that still it'll be fairly far up the stack,\nI think it's unplausible that there's anything left.\n\nWe can do it in a follow-up series if you'd like, or not do it before\nreftable. I just think it's cleaner to be assured that we're past this\nparticular hump, and are guaranteed not to gain future callers that rely\non the errno in some subtle way (without the explicit pass-in variable,\nthat is).\n\n> In general, it's frustrating to observe that the files backend is a\n> complex beast that nobody understands, but in order to replace it with\n> something more principled, I have to spend lots of time cleaning it\n> up.\n\nYeah, it sucks. Or more specifically, that our \"abstract\" API isn't very\n\"abstract\", but basically a 1=1 mapping to the files backend.\n\n>> Having dug a bit further, it seems what you're doing, whether it's\n>> intentional or not, is relying on the parse_loose_ref_contents() setting\n>> EINVAL, but you clobber your *failure_errno with it whether it returned\n>> -1 or not.\n>\n> I've added a failure_errno argument parse_loose_ref_contents now.\n\nGreat, it's not clear if you picked up the full diff-on-top but I think\nconsistently using *failure_errno in files_read_raw_ref() instead of\n\"errno\" and the \"*failure_errno = errno\" at the end also makes the code\nmuch more readable (and allows for the removal of the \"saved_errno\" we\ndon't want).\n\nBy using \"errno\" itself for the body of the function it needs to be\nreally carefully read to assure oneself that one of the functions it\ncalls doesn't make a syscall, and even if we're assured of that now if\none of them has a new syscall added in the future it might be clobbered\nat a distance.\n"},{"id":"429310","messageId":"CAFQ2z_PXjiUNxgC5W=+BB4yRanHtNO+AtPq4u0k6S+t4zcj5FQ@mail.gmail.com","threadId":"55589","inReplyTo":"87sg0r5x0m.fsf@evledraar.gmail.com","subject":"Re: [PATCH v3 0/5] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-06T18:36:24Z","receivedAt":"2021-07-06T18:36:42Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Tue, Jul 6, 2021 at 4:37 PM Ævar Arnfjörð Bjarmason wrote:\n\n> > I've added a failure_errno argument parse_loose_ref_contents now.\n>\n> Great, it's not clear if you picked up the full diff-on-top but I think\n> consistently using *failure_errno in files_read_raw_ref() instead of\n> \"errno\" and the \"*failure_errno = errno\" at the end also makes the code\n> much more readable (and allows for the removal of the \"saved_errno\" we\n> don't want).\n\nIt's hard to get right: there are many system calls, and in most cases\nEISDIR/ENOTDIR have to be propagated to the caller. I tried doing\nthis, and promptly got a failing test related to dir/file conflicts.\nBy doing\n\n *failure_errno = errno\n\nnear the exit of files_read_raw_ref(), we can be sure there are no\nchanges in behavior. If anyone wants to understand the files backend\nin more depth, they're welcome to disentangle this in a follow-up\npatch.\n\n> By using \"errno\" itself for the body of the function it needs to be\n> really carefully read to assure oneself that one of the functions it\n> calls doesn't make a syscall, and even if we're assured of that now if\n> one of them has a new syscall added in the future it might be clobbered\n> at a distance.\n\nYes, anyone adding a syscall to files_read_raw_ref has to be very careful.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429314","messageId":"pull.1012.v4.git.git.1625597757.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v3.git.git.1625518566.gitgitgadget@gmail.com","subject":"[PATCH v4 0/6] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-06T18:55:51Z","receivedAt":"2021-07-06T18:56:02Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"v5\n\n * address Ævar's comment; punt on clearing errno.\n\nHan-Wen Nienhuys (6):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: explicitly return failure_errno from parse_loose_ref_contents\n refs: make errno output explicit for refs_resolve_ref_unsafe\n\n refs.c | 60 ++++++++++++++++++++++++-----------\n refs/debug.c | 4 +--\n refs/files-backend.c | 73 ++++++++++++++++++++++---------------------\n refs/packed-backend.c | 15 ++++-----\n refs/refs-internal.h | 40 ++++++++++++++++--------\n 5 files changed, 116 insertions(+), 76 deletions(-)\n\n\nbase-commit: 670b81a890388c60b7032a4f5b879f2ece8c4558\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v4\nPull-Request: https://github.com/git/git/pull/1012\n\nRange-diff vs v3:\n\n 1: e2a0e5387ab ! 1: d6a41c3c0cb refs: remove EINVAL errno output from specification of read_raw_ref_fn\n @@ Commit message\n Spotted by Ævar Arnfjörð Bjarmason .\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Jonathan Tan \n \n ## refs/refs-internal.h ##\n @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,\n 2: c594c9c5c67 ! 2: ff7ea6efcba refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ Commit message\n documented to use and/or preserve errno)\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Jonathan Tan \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Reviewed-By: Ævar Arnfjörð Bjarmason \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)\n 3: b017caf54ba ! 3: ff5696b0875 refs: make errno output explicit for read_raw_ref_fn\n @@ Commit message\n relevant.\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Jonathan Tan \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Reviewed-By: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n 4: 4aaa9d3bd6f ! 4: 9c933706cb0 refs: add failure_errno to refs_read_raw_ref() signature\n @@ Commit message\n This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Jonathan Tan \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Reviewed-By: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: done:\n -: ----------- > 5: ab147afb38d refs: explicitly return failure_errno from parse_loose_ref_contents\n 5: ed5347d7bb6 ! 6: 0526a41b4b3 refs: make errno output explicit for refs_resolve_ref_unsafe\n @@ Commit message\n that needs error information to make logic decisions, so update that caller\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Jonathan Tan \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Reviewed-By: Ævar Arnfjörð Bjarmason \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n\n-- \ngitgitgadget\n"},{"id":"429315","messageId":"d6a41c3c0cb8bcd3d65f1273431f266541a96754.1625597757.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v4.git.git.1625597757.gitgitgadget@gmail.com","subject":"[PATCH v4 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-06T18:55:52Z","receivedAt":"2021-07-06T18:56:03Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 467f4b3c936..f4445e32904 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \ngitgitgadget\n\n"},{"id":"429316","messageId":"ff7ea6efcba07848077555a54cabb9eb67cb1d2f.1625597757.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v4.git.git.1625597757.gitgitgadget@gmail.com","subject":"[PATCH v4 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-06T18:55:53Z","receivedAt":"2021-07-06T18:56:04Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-By: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 16 ++++++----------\n 1 file changed, 6 insertions(+), 10 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 677b7e4cdd2..83ddfb3b627 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t * to remain.\n \t\t */\n \t\tif (remove_empty_directories(&ref_file)) {\n-\t\t\tlast_errno = errno;\n \t\t\tif (!refs_verify_refname_available(\n \t\t\t\t\t &refs->base,\n \t\t\t\t\t refname, extras, skip, err))\n@@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tlast_errno = errno;\n+\t\tint last_errno = errno;\n \t\tif (last_errno != ENOTDIR ||\n-\t\t !refs_verify_refname_available(&refs->base, refname,\n-\t\t\t\t\t\t extras, skip, err))\n+\t\t /* in case of D/F conflict, try to generate a better error\n+\t\t * message. If that fails, fall back to strerror(ENOTDIR).\n+\t\t */\n+\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n+\t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n \t\t\t\t refname, strerror(last_errno));\n \n@@ -981,20 +981,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n \t\t\t\t\t extras, skip, err)) {\n-\t\tlast_errno = ENOTDIR;\n \t\tgoto error_return;\n \t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n \n \tif (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {\n-\t\tlast_errno = errno;\n \t\tgoto error_return;\n \t}\n \tgoto out;\n@@ -1005,7 +1002,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"429317","messageId":"ab147afb38d6e74e0e2fc1fbb83346e6ee32cdd5.1625597757.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v4.git.git.1625597757.gitgitgadget@gmail.com","subject":"[PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-06T18:55:56Z","receivedAt":"2021-07-06T18:56:05Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe EINVAL error from parse_loose_ref_contents is used in files-backend\nto create a custom error message.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-By: Ævar Arnfjörð Bjarmason \n---\n refs.c | 8 +++++---\n refs/files-backend.c | 13 +++++++++----\n refs/refs-internal.h | 6 ++++--\n 3 files changed, 18 insertions(+), 9 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 25d80e544d0..eca7310e7a4 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n \n static int refs_read_special_head(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t\t int *failure_errno)\n {\n \tstruct strbuf full_path = STRBUF_INIT;\n \tstruct strbuf content = STRBUF_INIT;\n@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n \t\tgoto done;\n \n-\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n+\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n+\t\t\t\t\t failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\n@@ -1683,7 +1685,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f38c9703504..b43ec4c66cb 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -455,9 +455,13 @@ stat_ref:\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n-\tret = parse_loose_ref_contents(buf, oid, referent, type);\n-\n+\tret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);\n+\terrno = *failure_errno;\n out:\n+\t/* Collect all types of failures from errno. Many system calls in this\n+\t * function can fail with ENOTDIR/EISDIR, and we want to collect all of\n+\t * them.\n+\t */\n \t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n@@ -465,7 +469,8 @@ out:\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -484,7 +489,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n \tif (parse_oid_hex(buf, oid, &p) ||\n \t (*p != '\\0' && !isspace(*p))) {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n \treturn 0;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 54f57c6a2df..bf581e70cf6 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -689,10 +689,12 @@ struct ref_store {\n };\n \n /*\n- * Parse contents of a loose ref file.\n+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for\n+ * invalid contents.\n */\n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\n-- \ngitgitgadget\n\n"},{"id":"429318","messageId":"ff5696b087571a28192c1d76a5e57451177e4174.1625597757.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v4.git.git.1625597757.gitgitgadget@gmail.com","subject":"[PATCH v4 3/6] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-06T18:55:54Z","receivedAt":"2021-07-06T18:56:06Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes it explicit how alternative ref backends should report errors in\nread_raw_ref_fn.\n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-By: Ævar Arnfjörð Bjarmason \n---\n refs.c | 10 ++++++++--\n refs/debug.c | 4 ++--\n refs/files-backend.c | 23 +++++++++++------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 20 ++++++++++++--------\n 5 files changed, 37 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 8c9490235ea..5e5e3af8da0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1675,13 +1675,19 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type)\n {\n+\tint result;\n+\tint failure_errno;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\tfailure_errno = 0;\n+\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, &failure_errno);\n+\tif (failure_errno)\n+\t\terrno = failure_errno;\n+\treturn result;\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 7db4abccc34..f12413a9bc0 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \toidcpy(oid, null_oid());\n \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 83ddfb3b627..f98b1be9386 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,9 @@ stat_ref:\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno = 0;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -629,9 +628,9 @@ retry:\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -655,7 +654,7 @@ retry:\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -693,13 +692,13 @@ retry:\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex dfecdbc1db6..a457f18e93c 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex f4445e32904..79dfb3af484 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n+ * type, and return -1 (failure_errno should not be ENOENT)\n+ *\n+ * failure_errno provides errno codes that are interpreted beyond error\n+ * reporting. The following error codes have special meaning:\n+ * * ENOENT: the ref doesn't exist\n+ * * EISDIR: ref name is a directory\n+ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \ngitgitgadget\n\n"},{"id":"429319","messageId":"9c933706cb09265b62213b29f4a632c8a4445de1.1625597757.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v4.git.git.1625597757.gitgitgadget@gmail.com","subject":"[PATCH v4 4/6] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-06T18:55:55Z","receivedAt":"2021-07-06T18:56:06Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-By: Ævar Arnfjörð Bjarmason \n---\n refs.c | 60 +++++++++++++++++++++++++++----------------\n refs/files-backend.c | 8 +++---\n refs/packed-backend.c | 7 ++---\n refs/refs-internal.h | 6 ++---\n 4 files changed, 49 insertions(+), 32 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 5e5e3af8da0..25d80e544d0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1671,34 +1671,35 @@ done:\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n-\tint result;\n-\tint failure_errno;\n+\tint ignore;\n+\tif (failure_errno)\n+\t\t*failure_errno = 0;\n+\telse\n+\t\tfailure_errno = &ignore;\n+\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\tfailure_errno = 0;\n-\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &failure_errno);\n-\tif (failure_errno)\n-\t\terrno = failure_errno;\n-\treturn result;\n+\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n+\tint unused_errno;\n \tint symref_count;\n \n \tif (!oid)\n@@ -1707,6 +1708,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\tflags = &unused_flags;\n \n \t*flags = 0;\n+\tif (!failure_errno)\n+\t\tfailure_errno = &unused_errno;\n+\t*failure_errno = 0;\n \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n@@ -1728,11 +1732,14 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n+\t\tint read_failure = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n+\t\t\t*failure_errno = read_failure;\n+\n \t\t\t/* In reading mode, refs must eventually resolve */\n \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n \t\t\t\treturn NULL;\n@@ -1742,9 +1749,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (read_failure != ENOENT && read_failure != EISDIR &&\n+\t\t\t read_failure != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -1783,6 +1789,15 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tint ignore = 0;\n+\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &ignore);\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n@@ -2244,7 +2259,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, NULL)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f98b1be9386..f38c9703504 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -383,8 +383,8 @@ stat_ref:\n \tif (lstat(path, &st) < 0) {\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -423,8 +423,8 @@ stat_ref:\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a457f18e93c..530dba29788 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure_errno = 0;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure_errno) ||\n+\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 79dfb3af484..54f57c6a2df 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /*\n * Write an error to `err` and return a nonzero value iff the same\n-- \ngitgitgadget\n\n"},{"id":"429320","messageId":"0526a41b4b30cee67c6fea818202a106dbd66cb2.1625597757.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v4.git.git.1625597757.gitgitgadget@gmail.com","subject":"[PATCH v4 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-06T18:55:57Z","receivedAt":"2021-07-06T18:56:07Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions, so update that caller\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-By: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 15 ++++++++-------\n refs/refs-internal.h | 8 ++++++++\n 2 files changed, 16 insertions(+), 7 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b43ec4c66cb..d86893c6e0a 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -928,6 +928,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -940,10 +941,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t refname, resolve_flags,\n-\t\t\t\t\t &lock->old_oid, type);\n-\tif (!resolved && errno == EISDIR) {\n+\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\t\tresolve_flags,\n+\t\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t\t&resolve_errno);\n+\tif (!resolved && resolve_errno == EISDIR) {\n \t\t/*\n \t\t * we are trying to lock foo but we used to\n \t\t * have foo/bar which now does not exist;\n@@ -963,15 +965,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tint last_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n+\t\tif (resolve_errno != ENOTDIR ||\n \t\t /* in case of D/F conflict, try to generate a better error\n \t\t * message. If that fails, fall back to strerror(ENOTDIR).\n \t\t */\n \t\t !refs_verify_refname_available(&refs->base, refname, extras,\n \t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n+\t\t\t\t refname, strerror(resolve_errno));\n \n \t\tgoto error_return;\n \t}\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex bf581e70cf6..df01d5dc8df 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n+ * failure. */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n+\n /*\n * Write an error to `err` and return a nonzero value iff the same\n * refname appears multiple times in `refnames`. `refnames` must be\n-- \ngitgitgadget\n"},{"id":"429325","messageId":"xmqqk0m3jl7b.fsf@gitster.g","threadId":"55589","inReplyTo":"4aaa9d3bd6f01b83932b25a9ba18ec9c1106405a.1625518566.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 4/5] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-07-06T19:28:40Z","receivedAt":"2021-07-06T19:28:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> -\tint result;\n> -\tint failure_errno;\n> +\tint ignore;\n> +\tif (failure_errno)\n> +\t\t*failure_errno = 0;\n> +\telse\n> +\t\tfailure_errno = &ignore;\n\nHmph, I would have expected that a piece of code that gives fallback\nlocation for ignored return parameter to initialize the location the\nsame way, i.e.\n\n\tint ignore\n\n\tif (!failure_errno)\n\t\tfailure_errno = &ignore\n\t*failure_errno = 0;\n\nThat way, the code that follows that takes failure_errno does not\nhave to care if that return parameter location was supplied by the\ncaller or given by the fallback logic.\n\n> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n> +\t\t\t\t\t const char *refname,\n> +\t\t\t\t\t int resolve_flags,\n> +\t\t\t\t\t struct object_id *oid,\n> +\t\t\t\t\t int *flags, int *failure_errno)\n> {\n> \tstatic struct strbuf sb_refname = STRBUF_INIT;\n> \tstruct object_id unused_oid;\n> \tint unused_flags;\n> +\tint unused_errno;\n> \tint symref_count;\n> \n> \tif (!oid)\n> @@ -1707,6 +1708,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n> \t\tflags = &unused_flags;\n> \n> \t*flags = 0;\n> +\tif (!failure_errno)\n> +\t\tfailure_errno = &unused_errno;\n> +\t*failure_errno = 0;\n\nAnd you do use that pattern correctly here. We probably would want\nto be consistent.\n"},{"id":"429327","messageId":"xmqqfswrjksb.fsf@gitster.g","threadId":"55589","inReplyTo":"ab147afb38d6e74e0e2fc1fbb83346e6ee32cdd5.1625597757.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-07-06T19:37:40Z","receivedAt":"2021-07-06T19:37:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> From: Han-Wen Nienhuys \n>\n> The EINVAL error from parse_loose_ref_contents is used in files-backend\n> to create a custom error message.\n>\n> Signed-off-by: Han-Wen Nienhuys \n> Reviewed-By: Ævar Arnfjörð Bjarmason \n\nI think a -By in the trailer is spelled with lowercase, i.e. \"Reviewed-by\".\n\n> ---\n> refs.c | 8 +++++---\n> refs/files-backend.c | 13 +++++++++----\n> refs/refs-internal.h | 6 ++++--\n> 3 files changed, 18 insertions(+), 9 deletions(-)\n>\n> diff --git a/refs.c b/refs.c\n> index 25d80e544d0..eca7310e7a4 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n> \n> static int refs_read_special_head(struct ref_store *ref_store,\n> \t\t\t\t const char *refname, struct object_id *oid,\n> -\t\t\t\t struct strbuf *referent, unsigned int *type)\n> +\t\t\t\t struct strbuf *referent, unsigned int *type,\n> +\t\t\t\t int *failure_errno)\n> {\n> \tstruct strbuf full_path = STRBUF_INIT;\n> \tstruct strbuf content = STRBUF_INIT;\n> @@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n> \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n> \t\tgoto done;\n> \n> -\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n> +\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n> +\t\t\t\t\t failure_errno);\n> \n> done:\n> \tstrbuf_release(&full_path);\n> @@ -1683,7 +1685,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> \n> \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n> -\t\t\t\t\t type);\n> +\t\t\t\t\t type, failure_errno);\n> \t}\n> \n> \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index f38c9703504..b43ec4c66cb 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -455,9 +455,13 @@ stat_ref:\n> \tstrbuf_rtrim(&sb_contents);\n> \tbuf = sb_contents.buf;\n> \n> -\tret = parse_loose_ref_contents(buf, oid, referent, type);\n> -\n> +\tret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);\n> +\terrno = *failure_errno;\n\nAll of the above makes sense.\n\n> out:\n> +\t/* Collect all types of failures from errno. Many system calls in this\n> +\t * function can fail with ENOTDIR/EISDIR, and we want to collect all of\n> +\t * them.\n> +\t */\n\nStyle.\n\n> \t*failure_errno = errno;\n> \tstrbuf_release(&sb_path);\n> \tstrbuf_release(&sb_contents);\n> @@ -465,7 +469,8 @@ out:\n> }\n> \n> int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n> -\t\t\t struct strbuf *referent, unsigned int *type)\n> +\t\t\t struct strbuf *referent, unsigned int *type,\n> +\t\t\t int *failure_errno)\n> {\n> \tconst char *p;\n> \tif (skip_prefix(buf, \"ref:\", &buf)) {\n> @@ -484,7 +489,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n> \tif (parse_oid_hex(buf, oid, &p) ||\n> \t (*p != '\\0' && !isspace(*p))) {\n> \t\t*type |= REF_ISBROKEN;\n> -\t\terrno = EINVAL;\n> +\t\t*failure_errno = EINVAL;\n> \t\treturn -1;\n> \t}\n\nOK.\n\n> \treturn 0;\n> diff --git a/refs/refs-internal.h b/refs/refs-internal.h\n> index 54f57c6a2df..bf581e70cf6 100644\n> --- a/refs/refs-internal.h\n> +++ b/refs/refs-internal.h\n> @@ -689,10 +689,12 @@ struct ref_store {\n> };\n> \n> /*\n> - * Parse contents of a loose ref file.\n> + * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for\n> + * invalid contents.\n> */\n\nOK.\n\n> int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n> -\t\t\t struct strbuf *referent, unsigned int *type);\n> +\t\t\t struct strbuf *referent, unsigned int *type,\n> +\t\t\t int *failure_errno);\n> \n> /*\n> * Fill in the generic part of refs and add it to our collection of\n"},{"id":"429328","messageId":"xmqqbl7fjkq2.fsf@gitster.g","threadId":"55589","inReplyTo":"9c933706cb09265b62213b29f4a632c8a4445de1.1625597757.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 4/6] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-07-06T19:39:01Z","receivedAt":"2021-07-06T19:39:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> -int refs_read_raw_ref(struct ref_store *ref_store,\n> -\t\t const char *refname, struct object_id *oid,\n> -\t\t struct strbuf *referent, unsigned int *type)\n> +int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> +\t\t struct object_id *oid, struct strbuf *referent,\n> +\t\t unsigned int *type, int *failure_errno)\n> {\n> -\tint result;\n> -\tint failure_errno;\n> +\tint ignore;\n> +\tif (failure_errno)\n> +\t\t*failure_errno = 0;\n> +\telse\n> +\t\tfailure_errno = &ignore;\n> +\n\nSame comment as v3(v5). Squashable fix-up follows.\n\ndiff --git a/refs.c b/refs.c\nindex eca7310e7a..aaea9b34c6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1677,11 +1677,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno)\n {\n-\tint ignore;\n-\tif (failure_errno)\n-\t\t*failure_errno = 0;\n-\telse\n-\t\tfailure_errno = &ignore;\n+\tint unused_errno;\n+\n+\tif (!failure_errno)\n+\t\tfailure_errno = &unused_errno;\n+\t*failure_errno = 0;\n \n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n"},{"id":"429405","messageId":"CAFQ2z_N+uMCG9ewv-P0GgZQ90m5i6Hk_HYEsRb_WuBvPMZKFFA@mail.gmail.com","threadId":"55589","inReplyTo":"xmqqfswrjksb.fsf@gitster.g","subject":"Re: [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-07T08:20:00Z","receivedAt":"2021-07-07T08:20:15Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Tue, Jul 6, 2021 at 9:37 PM Junio C Hamano wrote:\n>\n> \"Han-Wen Nienhuys via GitGitGadget\" writes:\n>\n> > From: Han-Wen Nienhuys \n> >\n> > The EINVAL error from parse_loose_ref_contents is used in files-backend\n> > to create a custom error message.\n> >\n> > Signed-off-by: Han-Wen Nienhuys \n> > Reviewed-By: Ævar Arnfjörð Bjarmason \n>\n> I think a -By in the trailer is spelled with lowercase, i.e. \"Reviewed-by\".\n\n.. All done.\n\nI also addressed your other fixup.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429435","messageId":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v4.git.git.1625597757.gitgitgadget@gmail.com","subject":"[PATCH v5 0/6] refs: cleanup errno sideband ref related functions","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-07T19:07:43Z","receivedAt":"2021-07-07T19:07:54Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"v5\n\n * address Ævar's comment; punt on clearing errno.\n\nHan-Wen Nienhuys (6):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: explicitly return failure_errno from parse_loose_ref_contents\n refs: make errno output explicit for refs_resolve_ref_unsafe\n\n refs.c | 58 ++++++++++++++++++++++-----------\n refs/debug.c | 4 +--\n refs/files-backend.c | 74 ++++++++++++++++++++++---------------------\n refs/packed-backend.c | 15 +++++----\n refs/refs-internal.h | 40 +++++++++++++++--------\n 5 files changed, 115 insertions(+), 76 deletions(-)\n\n\nbase-commit: 670b81a890388c60b7032a4f5b879f2ece8c4558\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v5\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v5\nPull-Request: https://github.com/git/git/pull/1012\n\nRange-diff vs v4:\n\n 1: d6a41c3c0cb = 1: d6a41c3c0cb refs: remove EINVAL errno output from specification of read_raw_ref_fn\n 2: ff7ea6efcba ! 2: 95025080c16 refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ Commit message\n documented to use and/or preserve errno)\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-By: Ævar Arnfjörð Bjarmason \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)\n 3: ff5696b0875 ! 3: 7feedb97201 refs: make errno output explicit for read_raw_ref_fn\n @@ Commit message\n relevant.\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-By: Ævar Arnfjörð Bjarmason \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n 4: 9c933706cb0 ! 4: ef91f5cee13 refs: add failure_errno to refs_read_raw_ref() signature\n @@ Commit message\n This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-By: Ævar Arnfjörð Bjarmason \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: done:\n @@ refs.c: done:\n -int refs_read_raw_ref(struct ref_store *ref_store,\n -\t\t const char *refname, struct object_id *oid,\n -\t\t struct strbuf *referent, unsigned int *type)\n +-{\n +-\tint result;\n +-\tint failure_errno;\n +int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n +\t\t struct object_id *oid, struct strbuf *referent,\n +\t\t unsigned int *type, int *failure_errno)\n - {\n --\tint result;\n --\tint failure_errno;\n -+\tint ignore;\n -+\tif (failure_errno)\n -+\t\t*failure_errno = 0;\n -+\telse\n -+\t\tfailure_errno = &ignore;\n -+\n ++{\n ++\tint unused_errno;\n ++\tif (!failure_errno)\n ++\t\tfailure_errno = &unused_errno;\n ++\t*failure_errno = 0;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n 5: ab147afb38d ! 5: 6918c214d1b refs: explicitly return failure_errno from parse_loose_ref_contents\n @@ Commit message\n to create a custom error message.\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-By: Ævar Arnfjörð Bjarmason \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: int for_each_fullref_in_prefixes(const char *namespace,\n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n done:\n \tstrbuf_release(&full_path);\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n - \n + \t*failure_errno = 0;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n -\t\t\t\t\t type);\n @@ refs/files-backend.c: stat_ref:\n +\tret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);\n +\terrno = *failure_errno;\n out:\n -+\t/* Collect all types of failures from errno. Many system calls in this\n -+\t * function can fail with ENOTDIR/EISDIR, and we want to collect all of\n -+\t * them.\n ++\t/*\n ++\t * Many system calls in this function can fail with ENOTDIR/EISDIR, and\n ++\t * we want to collect all of them, so simply copy the error out from\n ++\t * errno.\n +\t */\n \t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n 6: 0526a41b4b3 ! 6: 85a14bde904 refs: make errno output explicit for refs_resolve_ref_unsafe\n @@ Commit message\n that needs error information to make logic decisions, so update that caller\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-By: Ævar Arnfjörð Bjarmason \n + Reviewed-by: Ævar Arnfjörð Bjarmason \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n\n-- \ngitgitgadget\n"},{"id":"429436","messageId":"d6a41c3c0cb8bcd3d65f1273431f266541a96754.1625684869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","subject":"[PATCH v5 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-07T19:07:44Z","receivedAt":"2021-07-07T19:07:57Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 467f4b3c936..f4445e32904 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \ngitgitgadget\n\n"},{"id":"429437","messageId":"95025080c16f535599826ed4f013845d712b0e8d.1625684869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","subject":"[PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-07T19:07:45Z","receivedAt":"2021-07-07T19:07:58Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 16 ++++++----------\n 1 file changed, 6 insertions(+), 10 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 677b7e4cdd2..83ddfb3b627 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t * to remain.\n \t\t */\n \t\tif (remove_empty_directories(&ref_file)) {\n-\t\t\tlast_errno = errno;\n \t\t\tif (!refs_verify_refname_available(\n \t\t\t\t\t &refs->base,\n \t\t\t\t\t refname, extras, skip, err))\n@@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tlast_errno = errno;\n+\t\tint last_errno = errno;\n \t\tif (last_errno != ENOTDIR ||\n-\t\t !refs_verify_refname_available(&refs->base, refname,\n-\t\t\t\t\t\t extras, skip, err))\n+\t\t /* in case of D/F conflict, try to generate a better error\n+\t\t * message. If that fails, fall back to strerror(ENOTDIR).\n+\t\t */\n+\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n+\t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n \t\t\t\t refname, strerror(last_errno));\n \n@@ -981,20 +981,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n \t\t\t\t\t extras, skip, err)) {\n-\t\tlast_errno = ENOTDIR;\n \t\tgoto error_return;\n \t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n \n \tif (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {\n-\t\tlast_errno = errno;\n \t\tgoto error_return;\n \t}\n \tgoto out;\n@@ -1005,7 +1002,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"429438","messageId":"7feedb9720127605b5e84aa1143c6ddc736597f3.1625684869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","subject":"[PATCH v5 3/6] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-07T19:07:46Z","receivedAt":"2021-07-07T19:07:59Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes it explicit how alternative ref backends should report errors in\nread_raw_ref_fn.\n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 10 ++++++++--\n refs/debug.c | 4 ++--\n refs/files-backend.c | 23 +++++++++++------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 20 ++++++++++++--------\n 5 files changed, 37 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 8c9490235ea..5e5e3af8da0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1675,13 +1675,19 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type)\n {\n+\tint result;\n+\tint failure_errno;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\tfailure_errno = 0;\n+\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, &failure_errno);\n+\tif (failure_errno)\n+\t\terrno = failure_errno;\n+\treturn result;\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 7db4abccc34..f12413a9bc0 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \toidcpy(oid, null_oid());\n \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 83ddfb3b627..f98b1be9386 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,9 @@ stat_ref:\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno = 0;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -629,9 +628,9 @@ retry:\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -655,7 +654,7 @@ retry:\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -693,13 +692,13 @@ retry:\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex dfecdbc1db6..a457f18e93c 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex f4445e32904..79dfb3af484 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n+ * type, and return -1 (failure_errno should not be ENOENT)\n+ *\n+ * failure_errno provides errno codes that are interpreted beyond error\n+ * reporting. The following error codes have special meaning:\n+ * * ENOENT: the ref doesn't exist\n+ * * EISDIR: ref name is a directory\n+ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \ngitgitgadget\n\n"},{"id":"429439","messageId":"ef91f5cee13b1153f597f31451460bba543a63c3.1625684869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","subject":"[PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-07T19:07:47Z","receivedAt":"2021-07-07T19:08:00Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 60 ++++++++++++++++++++++++++-----------------\n refs/files-backend.c | 8 +++---\n refs/packed-backend.c | 7 ++---\n refs/refs-internal.h | 6 ++---\n 4 files changed, 48 insertions(+), 33 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 5e5e3af8da0..de3826f60c9 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1671,34 +1671,33 @@ done:\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n-{\n-\tint result;\n-\tint failure_errno;\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n+{\n+\tint unused_errno;\n+\tif (!failure_errno)\n+\t\tfailure_errno = &unused_errno;\n+\t*failure_errno = 0;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\tfailure_errno = 0;\n-\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &failure_errno);\n-\tif (failure_errno)\n-\t\terrno = failure_errno;\n-\treturn result;\n+\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n+\tint unused_errno;\n \tint symref_count;\n \n \tif (!oid)\n@@ -1707,6 +1706,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\tflags = &unused_flags;\n \n \t*flags = 0;\n+\tif (!failure_errno)\n+\t\tfailure_errno = &unused_errno;\n+\t*failure_errno = 0;\n \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n@@ -1728,11 +1730,14 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n+\t\tint read_failure = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, &read_failure)) {\n \t\t\t*flags |= read_flags;\n \n+\t\t\t*failure_errno = read_failure;\n+\n \t\t\t/* In reading mode, refs must eventually resolve */\n \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n \t\t\t\treturn NULL;\n@@ -1742,9 +1747,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (read_failure != ENOENT && read_failure != EISDIR &&\n+\t\t\t read_failure != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -1783,6 +1787,15 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tint ignore = 0;\n+\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &ignore);\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n@@ -2244,7 +2257,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, NULL)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f98b1be9386..f38c9703504 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -383,8 +383,8 @@ stat_ref:\n \tif (lstat(path, &st) < 0) {\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -423,8 +423,8 @@ stat_ref:\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, NULL)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a457f18e93c..530dba29788 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure_errno = 0;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure_errno) ||\n+\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 79dfb3af484..54f57c6a2df 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /*\n * Write an error to `err` and return a nonzero value iff the same\n-- \ngitgitgadget\n\n"},{"id":"429440","messageId":"6918c214d1b41b16c47625f0935389440253ec4c.1625684869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","subject":"[PATCH v5 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-07T19:07:48Z","receivedAt":"2021-07-07T19:08:02Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe EINVAL error from parse_loose_ref_contents is used in files-backend\nto create a custom error message.\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 8 +++++---\n refs/files-backend.c | 14 ++++++++++----\n refs/refs-internal.h | 6 ++++--\n 3 files changed, 19 insertions(+), 9 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex de3826f60c9..96df78a9509 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n \n static int refs_read_special_head(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t\t int *failure_errno)\n {\n \tstruct strbuf full_path = STRBUF_INIT;\n \tstruct strbuf content = STRBUF_INIT;\n@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n \t\tgoto done;\n \n-\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n+\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n+\t\t\t\t\t failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\n@@ -1681,7 +1683,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t*failure_errno = 0;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f38c9703504..7aafdf2ce3d 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -455,9 +455,14 @@ stat_ref:\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n-\tret = parse_loose_ref_contents(buf, oid, referent, type);\n-\n+\tret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);\n+\terrno = *failure_errno;\n out:\n+\t/*\n+\t * Many system calls in this function can fail with ENOTDIR/EISDIR, and\n+\t * we want to collect all of them, so simply copy the error out from\n+\t * errno.\n+\t */\n \t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n@@ -465,7 +470,8 @@ out:\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -484,7 +490,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n \tif (parse_oid_hex(buf, oid, &p) ||\n \t (*p != '\\0' && !isspace(*p))) {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n \treturn 0;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 54f57c6a2df..bf581e70cf6 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -689,10 +689,12 @@ struct ref_store {\n };\n \n /*\n- * Parse contents of a loose ref file.\n+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for\n+ * invalid contents.\n */\n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\n-- \ngitgitgadget\n\n"},{"id":"429441","messageId":"85a14bde90476bae765fe83edef414337fba377a.1625684869.git.gitgitgadget@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","subject":"[PATCH v5 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Han-Wen Nienhuys via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-07-07T19:07:49Z","receivedAt":"2021-07-07T19:08:06Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions, so update that caller\n\nSigned-off-by: Han-Wen Nienhuys \nReviewed-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 15 ++++++++-------\n refs/refs-internal.h | 8 ++++++++\n 2 files changed, 16 insertions(+), 7 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 7aafdf2ce3d..d6a7a0ee919 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -929,6 +929,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -941,10 +942,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t refname, resolve_flags,\n-\t\t\t\t\t &lock->old_oid, type);\n-\tif (!resolved && errno == EISDIR) {\n+\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\t\tresolve_flags,\n+\t\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t\t&resolve_errno);\n+\tif (!resolved && resolve_errno == EISDIR) {\n \t\t/*\n \t\t * we are trying to lock foo but we used to\n \t\t * have foo/bar which now does not exist;\n@@ -964,15 +966,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved) {\n-\t\tint last_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n+\t\tif (resolve_errno != ENOTDIR ||\n \t\t /* in case of D/F conflict, try to generate a better error\n \t\t * message. If that fails, fall back to strerror(ENOTDIR).\n \t\t */\n \t\t !refs_verify_refname_available(&refs->base, refname, extras,\n \t\t\t\t\t\t skip, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n+\t\t\t\t refname, strerror(resolve_errno));\n \n \t\tgoto error_return;\n \t}\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex bf581e70cf6..df01d5dc8df 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t struct object_id *oid, struct strbuf *referent,\n \t\t unsigned int *type, int *failure_errno);\n \n+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n+ * failure. */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n+\n /*\n * Write an error to `err` and return a nonzero value iff the same\n * refname appears multiple times in `refnames`. `refnames` must be\n-- \ngitgitgadget\n"},{"id":"429445","messageId":"xmqqa6mxetwh.fsf@gitster.g","threadId":"55589","inReplyTo":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 0/6] refs: cleanup errno sideband ref related functions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-07-07T20:44:14Z","receivedAt":"2021-07-07T20:44:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Han-Wen Nienhuys via GitGitGadget\" writes:\n\n> Han-Wen Nienhuys (6):\n> refs: remove EINVAL errno output from specification of read_raw_ref_fn\n> refs/files-backend: stop setting errno from lock_ref_oid_basic\n> refs: make errno output explicit for read_raw_ref_fn\n> refs: add failure_errno to refs_read_raw_ref() signature\n> refs: explicitly return failure_errno from parse_loose_ref_contents\n> refs: make errno output explicit for refs_resolve_ref_unsafe\n\nThanks, incremental changes look sensible.\nWill requeue.\n"},{"id":"429680","messageId":"87lf6d3wbz.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"95025080c16f535599826ed4f013845d712b0e8d.1625684869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T11:38:48Z","receivedAt":"2021-07-11T11:48:37Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> /*\n> * Locks a ref returning the lock on success and NULL on failure.\n> - * On failure errno is set to something meaningful.\n> */\n> static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \t\t\t\t\t const char *refname,\n> @@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> {\n> \tstruct strbuf ref_file = STRBUF_INIT;\n> \tstruct ref_lock *lock;\n> -\tint last_errno = 0;\n> \tint mustexist = (old_oid && !is_null_oid(old_oid));\n> \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n> \tint resolved;\n> @@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \t\t * to remain.\n> \t\t */\n> \t\tif (remove_empty_directories(&ref_file)) {\n> -\t\t\tlast_errno = errno;\n> \t\t\tif (!refs_verify_refname_available(\n> \t\t\t\t\t &refs->base,\n> \t\t\t\t\t refname, extras, skip, err))\n> @@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> \t\t\t\t\t\t &lock->old_oid, type);\n> \t}\n> \tif (!resolved) {\n> -\t\tlast_errno = errno;\n> +\t\tint last_errno = errno;\n> \t\tif (last_errno != ENOTDIR ||\n> -\t\t !refs_verify_refname_available(&refs->base, refname,\n> -\t\t\t\t\t\t extras, skip, err))\n> +\t\t /* in case of D/F conflict, try to generate a better error\n> +\t\t * message. If that fails, fall back to strerror(ENOTDIR).\n> +\t\t */\n> +\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n> +\t\t\t\t\t\t skip, err))\n> \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n> \t\t\t\t refname, strerror(last_errno));\n\nI don't think it's some dealbreaker and we can move on, but just FWIW I\nthink what I mentioned ending in your\nhttps://lore.kernel.org/git/CAFQ2z_NpyJQLuM70MhJ8K1h2v3QXFuAZRjN=SvSsjnukNRJ8pw@mail.gmail.com/\nis still outstanding.\n\nI.e. you added the comment, which is just says what the error emitting\nlooks like, that's all well & good.\n\nBut what I was pointing out that it didn't make sense to do any\n\"last_errno\" here at all anymore. You pointed to 5b2d8d6f218\n(lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,\n2015-05-11), we started setting \"last_errno\" there, but that was *not*\nto avoid clobbering between the !resolved and the\nstrbuf_add(strerror(last_errno)) here, but rather to carry the\n\"last_errno\" forward to the end of this lock_ref_oid_basic(), because\nother things (after this hunk) might reset/clear errno.\n\nAnyway, as noted there it doesn't actually matter, just reviewing &\nlooking if there's any loose ends, and for future source spelunking for\nanyone reading this thread.\n\nI.e. something like what I mentioned in\nhttps://lore.kernel.org/git/87k0mae0ga.fsf@evledraar.gmail.com/ could be\nsquashed in, or better yet (probably) this:\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 83ddfb3b627..f0ce0aac857 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -958,17 +958,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t refname, resolve_flags,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n-\tif (!resolved) {\n-\t\tint last_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n-\t\t /* in case of D/F conflict, try to generate a better error\n-\t\t * message. If that fails, fall back to strerror(ENOTDIR).\n-\t\t */\n-\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n-\t\t\t\t\t\t skip, err))\n-\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n-\n+\tif (!resolved &&\n+\t (errno != ENOTDIR ||\n+\t /* in case of D/F conflict, try to generate a better error\n+\t * message. If that fails, fall back to strerror(ENOTDIR).\n+\t */\n+\t !refs_verify_refname_available(&refs->base, refname, extras,\n+\t\t\t\t\t skip, err))) {\n+\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n+\t\t\t refname, strerror(errno));\n \t\tgoto error_return;\n \t}\n \n"},{"id":"429681","messageId":"87im1h3w3w.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"7feedb9720127605b5e84aa1143c6ddc736597f3.1625684869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 3/6] refs: make errno output explicit for read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T11:52:00Z","receivedAt":"2021-07-11T11:53:29Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> diff --git a/refs.c b/refs.c\n> index 8c9490235ea..5e5e3af8da0 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -1675,13 +1675,19 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n> \t\t const char *refname, struct object_id *oid,\n> \t\t struct strbuf *referent, unsigned int *type)\n> {\n> +\tint result;\n> +\tint failure_errno;\n> \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n> \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n> \t\t\t\t\t type);\n> \t}\n> \n> -\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> -\t\t\t\t\t type);\n> +\tfailure_errno = 0;\n> +\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n> +\t\t\t\t\t type, &failure_errno);\n> +\tif (failure_errno)\n> +\t\terrno = failure_errno;\n> +\treturn result;\n> }\n\nJust reading along again in sequence, and having forgotten where (if\nanywhere) this was headed I still find this verbose pattern for /looking\nlike/ we'll be doing something that's not setting errno, but actually\njust setting errno. I.e. the logic is the same as the below squash, but\nlet's read on...\n\ndiff --git a/refs.c b/refs.c\nindex 5e5e3af8da0..7b3f05a66ff 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1675,19 +1675,13 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t\t const char *refname, struct object_id *oid,\n \t\t struct strbuf *referent, unsigned int *type)\n {\n-\tint result;\n-\tint failure_errno;\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n-\tfailure_errno = 0;\n-\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &failure_errno);\n-\tif (failure_errno)\n-\t\terrno = failure_errno;\n-\treturn result;\n+\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n+\t\t\t\t\t type, &errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\n"},{"id":"429682","messageId":"87fswl3vsn.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"ef91f5cee13b1153f597f31451460bba543a63c3.1625684869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T11:59:45Z","receivedAt":"2021-07-11T12:00:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> From: Han-Wen Nienhuys \n>\n> This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n> [...]\n> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n\nThis and the subsequent commit don't compile for me, because this lacks\na prototype that you finally add in 6/6.\n"},{"id":"429684","messageId":"87czrp3tm4.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"6918c214d1b41b16c47625f0935389440253ec4c.1625684869.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T12:41:36Z","receivedAt":"2021-07-11T12:47:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n\n> Subject: refs: explicitly return failure_errno from parse_loose_ref_contents\n> [...]\n> The EINVAL error from parse_loose_ref_contents is used in files-backend\n> to create a custom error message.\n\nExcept as my https://lore.kernel.org/git/87v95o5ku8.fsf@evledraar.gmail.com/ shows, and if you remove this:\n\n> +\tret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);\n> +\terrno = *failure_errno;\n\nWe aren't explicitly returning things yet, instead we set it for some,\nbut others still fail if we don't support that one caller relying on\nthis \"errno\" and not \"failure_errno\" through the backdoor, why not\nsimply convert that remaining caller?\n"},{"id":"429687","messageId":"patch-01.17-57517368c34-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 01/17] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:25Z","receivedAt":"2021-07-11T16:30:49Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 467f4b3c936..f4445e32904 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \n2.32.0-dev\n\n"},{"id":"429689","messageId":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com","subject":"[PATCH v6? 00/17] refs API: get rid of errno setting entirely","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:24Z","receivedAt":"2021-07-11T16:30:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"After starting a review of v5 of Han-Wen's where I was coming up with\nsquashes-on-top I continued with it and saw if I could get rid of\nerrno setting entirely in refs.\n\nThe end-state of that is that we do that:\n\t\n\t$ git -P grep -w errno refs.c\n\trefs.c: if (errno != EPIPE) {\n\trefs.c: /* Don't leak errno outside this API */\n\trefs.c: errno = 0;\n\nThe rest has various bugfixes etc. to Han-Wen's version.\n\nI'm not sure about:\n\n refs: make errno ignoring explicit in lock_ref_oid_basic()\n\nBut it's an edge case that's present in Han-Wen's, but just\nignored/skipped past. Now it's a BUG(), I haven't been able to hit it\nin testing, but maybe we do hit it somehow.\n\nMy amendmend of:\n\n refs: explicitly return failure_errno from parse_loose_ref_contents\n\nHas it calling BUG() to assert that whenever we return non-zero we set\nerrno as expected, there were a few cases where functions omitted one\nerrno case, but set another.\n\nArguably everything as of 08/17 should be its follow-up series. I've\nincluded it because I think looking at those patches assures the\nreader that we didn't miss any subtle errno edge cases.\n\n(I removed my Reviewed-by trailer because it seemed redundant to SOB,\nand if I have SOB already on something saying I reviewed it seemed\nodd...)\n\n1. https://lore.kernel.org/git/pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com/\n\nHan-Wen Nienhuys (6):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: explicitly return failure_errno from parse_loose_ref_contents\n refs: make errno output explicit for refs_resolve_ref_unsafe\n\nÆvar Arnfjörð Bjarmason (11):\n refs: make errno ignoring explicit in lock_ref_oid_basic()\n refs file-backend.c: stop setting \"EBUSY\" in verify_lock()\n refs file-backend.c: deal with errno directly in verify_lock()\n refs API: remove refs_read_ref_full() wrapper\n refs API: make resolve_gitlink_ref() not set errno\n refs API: make refs_resolve_ref_unsafe() static\n refs API: make refs_resolve_refdup() not set errno\n refs API: make refs_ref_exists() not set errno\n refs API: make resolve_ref_unsafe() not set errno\n refs API: make expand_ref() and repo_dwim_log() not set errno\n refs API: don't leak \"errno\" in run_transaction_hook()\n\n refs.c | 122 ++++++++++++++++----------\n refs.h | 17 ++--\n refs/debug.c | 4 +-\n refs/files-backend.c | 180 +++++++++++++++++++++-----------------\n refs/packed-backend.c | 15 ++--\n refs/refs-internal.h | 32 ++++---\n sequencer.c | 10 ++-\n t/helper/test-ref-store.c | 5 +-\n worktree.c | 27 +++---\n 9 files changed, 244 insertions(+), 168 deletions(-)\n\nRange-diff:\n 1: d6a41c3c0cb ! 1: 57517368c34 refs: remove EINVAL errno output from specification of read_raw_ref_fn\n @@ Commit message\n Spotted by Ævar Arnfjörð Bjarmason .\n \n Signed-off-by: Han-Wen Nienhuys \n + Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## refs/refs-internal.h ##\n @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,\n 2: 95025080c16 ! 2: 61cf7611473 refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ Commit message\n refs_reflog_exists() (which calls a function in a vtable that is not\n documented to use and/or preserve errno)\n \n + In the case of the \"errno != ENOTDIR\" case that originates in 5b2d8d6f218\n + (lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,\n + 2015-05-11), there the \"last_errno\" was saved away to return it from\n + lock_ref_oid_basic(), now that we're no longer doing that we can skip\n + that entirely and use \"errno\" directly. A follow-up change will\n + extract the specific errno we want earlier in this function.\n +\n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \t\t\t\t\t &refs->base,\n \t\t\t\t\t refname, extras, skip, err))\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n + \t\t\t\t\t\t refname, resolve_flags,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n - \tif (!resolved) {\n +-\tif (!resolved) {\n -\t\tlast_errno = errno;\n -+\t\tint last_errno = errno;\n - \t\tif (last_errno != ENOTDIR ||\n +-\t\tif (last_errno != ENOTDIR ||\n -\t\t !refs_verify_refname_available(&refs->base, refname,\n -\t\t\t\t\t\t extras, skip, err))\n -+\t\t /* in case of D/F conflict, try to generate a better error\n -+\t\t * message. If that fails, fall back to strerror(ENOTDIR).\n -+\t\t */\n -+\t\t !refs_verify_refname_available(&refs->base, refname, extras,\n -+\t\t\t\t\t\t skip, err))\n - \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n - \t\t\t\t refname, strerror(last_errno));\n +-\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n +-\t\t\t\t refname, strerror(last_errno));\n +-\n ++\tif (!resolved &&\n ++\t (errno != ENOTDIR ||\n ++\t /* in case of D/F conflict, try to generate a better error\n ++\t * message. If that fails, fall back to strerror(ENOTDIR).\n ++\t */\n ++\t !refs_verify_refname_available(&refs->base, refname, extras,\n ++\t\t\t\t\t skip, err))) {\n ++\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n ++\t\t\t refname, strerror(errno));\n + \t\tgoto error_return;\n + \t}\n \n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n 3: 7feedb97201 ! 3: a4e5f2d02fb refs: make errno output explicit for read_raw_ref_fn\n @@ Commit message\n relevant.\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,\n - \t\t const char *refname, struct object_id *oid,\n - \t\t struct strbuf *referent, unsigned int *type)\n - {\n -+\tint result;\n -+\tint failure_errno;\n - \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n - \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n - \t\t\t\t\t type);\n \t}\n \n --\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n + \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n -\t\t\t\t\t type);\n -+\tfailure_errno = 0;\n -+\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n -+\t\t\t\t\t type, &failure_errno);\n -+\tif (failure_errno)\n -+\t\terrno = failure_errno;\n -+\treturn result;\n ++\t\t\t\t\t type, &errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\n @@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n -+\tint failure_errno = 0;\n ++\tint failure_errno;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n +@@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,\n + \tif (hold_lock_file_for_update_timeout(\n + \t\t\t &lock->lk, ref_file.buf, LOCK_NO_DEREF,\n + \t\t\t get_files_ref_lock_timeout_ms()) < 0) {\n +-\t\tif (errno == ENOENT && --attempts_remaining > 0) {\n ++\t\tint myerr = errno;\n ++\t\terrno = 0;\n ++\t\tif (myerr == ENOENT && --attempts_remaining > 0) {\n + \t\t\t/*\n + \t\t\t * Maybe somebody just deleted one of the\n + \t\t\t * directories leading to ref_file. Try\n +@@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,\n + \t\t\t */\n + \t\t\tgoto retry;\n + \t\t} else {\n +-\t\t\tunable_to_lock_message(ref_file.buf, errno, err);\n ++\t\t\tunable_to_lock_message(ref_file.buf, myerr, err);\n + \t\t\tgoto error_return;\n + \t\t}\n + \t}\n @@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,\n \t * fear that its value will change.\n \t */\n 4: ef91f5cee13 ! 4: 270cda29c3a refs: add failure_errno to refs_read_raw_ref() signature\n @@ Commit message\n \n This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n \n + Some of our callers explicitly do not care about the errno, rather\n + than understanding NULL let's have them declare that they don't care\n + by passing in an \"ignore_errno\". There's only three of them, and using\n + that pattern will make it more obvious that they want to throw away\n + data, let's also add a comment to one of the callers about why we'd\n + like to ignore the errno.\n +\n + Let's not extend that to refs_resolve_ref_unsafe() itself for now, it\n + has a large set of legacy callers, so we're faking up the old \"errno\"\n + behavior for it. We can convert those callers to\n + refs_resolve_ref_unsafe_with_errno() later.\n +\n + We are leaving out out the refs_read_special_head() in\n + refs_read_raw_ref() for now, as noted in the next commit moving it to\n + \"failure_errno\" will require some special consideration.\n +\n + We're intentionally mis-indenting the argument list of the new\n + refs_resolve_ref_unsafe_with_errno(), it will be non-static in a\n + subsequent commit, doing it this way makes that diff smaller.\n +\n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n -int refs_read_raw_ref(struct ref_store *ref_store,\n -\t\t const char *refname, struct object_id *oid,\n -\t\t struct strbuf *referent, unsigned int *type)\n --{\n --\tint result;\n --\tint failure_errno;\n +int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n +\t\t struct object_id *oid, struct strbuf *referent,\n +\t\t unsigned int *type, int *failure_errno)\n -+{\n -+\tint unused_errno;\n -+\tif (!failure_errno)\n -+\t\tfailure_errno = &unused_errno;\n -+\t*failure_errno = 0;\n + {\n ++\tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n --\tfailure_errno = 0;\n --\tresult = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n --\t\t\t\t\t type, &failure_errno);\n --\tif (failure_errno)\n --\t\terrno = failure_errno;\n --\treturn result;\n -+\treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n + \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n +-\t\t\t\t\t type, &errno);\n +\t\t\t\t\t type, failure_errno);\n }\n \n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n -\t\t\t\t const char *refname,\n -\t\t\t\t int resolve_flags,\n -\t\t\t\t struct object_id *oid, int *flags)\n -+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n ++static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n +\t\t\t\t\t const char *refname,\n +\t\t\t\t\t int resolve_flags,\n +\t\t\t\t\t struct object_id *oid,\n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n -+\tint unused_errno;\n \tint symref_count;\n \n ++\tassert(failure_errno);\n ++\n \tif (!oid)\n + \t\toid = &unused_oid;\n + \tif (!flags)\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n - \t\tflags = &unused_flags;\n - \n - \t*flags = 0;\n -+\tif (!failure_errno)\n -+\t\tfailure_errno = &unused_errno;\n -+\t*failure_errno = 0;\n - \n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n -@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n + \t\t !refname_is_safe(refname)) {\n +-\t\t\terrno = EINVAL;\n ++\t\t\t*failure_errno = EINVAL;\n + \t\t\treturn NULL;\n + \t\t}\n \n +@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n -+\t\tint read_failure = 0;\n \n -\t\tif (refs_read_raw_ref(refs, refname,\n -\t\t\t\t oid, &sb_refname, &read_flags)) {\n +\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n -+\t\t\t\t &read_flags, &read_failure)) {\n ++\t\t\t\t &read_flags, failure_errno)) {\n \t\t\t*flags |= read_flags;\n \n -+\t\t\t*failure_errno = read_failure;\n -+\n \t\t\t/* In reading mode, refs must eventually resolve */\n - \t\t\tif (resolve_flags & RESOLVE_REF_READING)\n - \t\t\t\treturn NULL;\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n -\t\t\tif (errno != ENOENT &&\n -\t\t\t errno != EISDIR &&\n -\t\t\t errno != ENOTDIR)\n -+\t\t\tif (read_failure != ENOENT && read_failure != EISDIR &&\n -+\t\t\t read_failure != ENOTDIR)\n ++\t\t\tif (*failure_errno != ENOENT &&\n ++\t\t\t *failure_errno != EISDIR &&\n ++\t\t\t *failure_errno != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n + \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n + \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n + \t\t\t !refname_is_safe(refname)) {\n +-\t\t\t\terrno = EINVAL;\n ++\t\t\t\t*failure_errno = EINVAL;\n + \t\t\t\treturn NULL;\n + \t\t\t}\n + \n +@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n + \t\t}\n + \t}\n + \n +-\terrno = ELOOP;\n ++\t*failure_errno = ELOOP;\n \treturn NULL;\n }\n \n @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n +\t\t\t\t int resolve_flags, struct object_id *oid,\n +\t\t\t\t int *flags)\n +{\n -+\tint ignore = 0;\n -+\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n -+\t\t\t\t\t\t oid, flags, &ignore);\n ++\tint failure_errno = 0;\n ++\tconst char *refn;\n ++\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n ++\t\t\t\t\t\t oid, flags, &failure_errno);\n ++\tif (!refn)\n ++\t\t/* For unmigrated legacy callers */\n ++\t\terrno = failure_errno;\n ++\treturn refn;\n +}\n +\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n +@@ refs.c: int refs_verify_refname_available(struct ref_store *refs,\n + \n + \tstrbuf_grow(&dirname, strlen(refname) + 1);\n + \tfor (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {\n ++\t\t/*\n ++\t\t * Just saying \"Is a directory\" when we e.g. can't\n ++\t\t * lock some multi-level ref isn't very informative,\n ++\t\t * the user won't be told *what* is a directory, so\n ++\t\t * let's not use strerror() below.\n ++\t\t */\n ++\t\tint ignore_errno;\n + \t\t/* Expand dirname to the new prefix, not including the trailing slash: */\n + \t\tstrbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);\n + \n @@ refs.c: int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n -\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n +\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n -+\t\t\t\t &type, NULL)) {\n ++\t\t\t\t &type, &ignore_errno)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \t\tgoto out;\n + \n \tif (lstat(path, &st) < 0) {\n ++\t\tint ignore_errno;\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n -\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n -\t\t\t\t oid, referent, type)) {\n +\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n -+\t\t\t\t referent, type, NULL)) {\n ++\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n @@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \n + \t/* Is it a directory? */\n + \tif (S_ISDIR(st.st_mode)) {\n ++\t\tint ignore_errno;\n + \t\t/*\n + \t\t * Even though there is a directory where the loose\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n -\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n -\t\t\t\t oid, referent, type)) {\n +\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n -+\t\t\t\t referent, type, NULL)) {\n ++\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\n @@ refs/packed-backend.c: int is_packed_transaction_needed(struct ref_store *ref_st\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n -+\t\tint failure_errno = 0;\n ++\t\tint failure_errno;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n 5: 6918c214d1b ! 5: f5197cdc0b8 refs: explicitly return failure_errno from parse_loose_ref_contents\n @@ Commit message\n The EINVAL error from parse_loose_ref_contents is used in files-backend\n to create a custom error message.\n \n + In untangling this we discovered a tricky edge case. The\n + refs_read_special_head() function was relying on\n + parse_loose_ref_contents() setting EINVAL.\n +\n + By converting it to use \"saved_errno\" we can migrate away from \"errno\"\n + in this part of the code entirely, and do away with an existing\n + \"save_errno\" pattern, its only purpose was to not clobber the \"errno\"\n + we previously needed at the end of files_read_raw_ref().\n +\n + Let's assert that we can do that by not having files_read_raw_ref()\n + itself operate on *failure_errno in addition to passing it on. Instead\n + we'll assert that if we return non-zero we actually do set errno, thus\n + assuring ourselves and callers that they can trust the resulting\n + \"failure_errno\".\n +\n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## refs.c ##\n @@ refs.c: int for_each_fullref_in_prefixes(const char *namespace,\n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n done:\n \tstrbuf_release(&full_path);\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n - \t*failure_errno = 0;\n + \tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n -\t\t\t\t\t type);\n @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \tint fd;\n + \tint ret = -1;\n + \tint remaining_retries = 3;\n ++\tint myerr = 0;\n + \n + \t*type = 0;\n + \tstrbuf_reset(&sb_path);\n +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \n + \tif (lstat(path, &st) < 0) {\n + \t\tint ignore_errno;\n +-\t\tif (errno != ENOENT)\n ++\t\tmyerr = errno;\n ++\t\terrno = 0;\n ++\t\tif (myerr != ENOENT)\n + \t\t\tgoto out;\n + \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n + \t\t\t\t referent, type, &ignore_errno)) {\n +-\t\t\terrno = ENOENT;\n ++\t\t\tmyerr = ENOENT;\n + \t\t\tgoto out;\n + \t\t}\n + \t\tret = 0;\n +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \tif (S_ISLNK(st.st_mode)) {\n + \t\tstrbuf_reset(&sb_contents);\n + \t\tif (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {\n +-\t\t\tif (errno == ENOENT || errno == EINVAL)\n ++\t\t\tmyerr = errno;\n ++\t\t\terrno = 0;\n ++\t\t\tif (myerr == ENOENT || myerr == EINVAL)\n + \t\t\t\t/* inconsistent with lstat; retry */\n + \t\t\t\tgoto stat_ref;\n + \t\t\telse\n +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \t\t */\n + \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n + \t\t\t\t referent, type, &ignore_errno)) {\n +-\t\t\terrno = EISDIR;\n ++\t\t\tmyerr = EISDIR;\n + \t\t\tgoto out;\n + \t\t}\n + \t\tret = 0;\n +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \t */\n + \tfd = open(path, O_RDONLY);\n + \tif (fd < 0) {\n +-\t\tif (errno == ENOENT && !S_ISLNK(st.st_mode))\n ++\t\tmyerr = errno;\n ++\t\tif (myerr == ENOENT && !S_ISLNK(st.st_mode))\n + \t\t\t/* inconsistent with lstat; retry */\n + \t\t\tgoto stat_ref;\n + \t\telse\n +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \t}\n + \tstrbuf_reset(&sb_contents);\n + \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n +-\t\tint save_errno = errno;\n + \t\tclose(fd);\n +-\t\terrno = save_errno;\n + \t\tgoto out;\n + \t}\n + \tclose(fd);\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n -\tret = parse_loose_ref_contents(buf, oid, referent, type);\n --\n -+\tret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);\n -+\terrno = *failure_errno;\n ++\tret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);\n + \n out:\n -+\t/*\n -+\t * Many system calls in this function can fail with ENOTDIR/EISDIR, and\n -+\t * we want to collect all of them, so simply copy the error out from\n -+\t * errno.\n -+\t */\n - \t*failure_errno = errno;\n +-\t*failure_errno = errno;\n ++\tif (ret && !myerr)\n ++\t\tBUG(\"returning non-zero %d, should have set myerr!\", ret);\n ++\t*failure_errno = myerr;\n ++\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n -@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \treturn ret;\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n 6: 85a14bde904 ! 6: 96689e523f1 refs: make errno output explicit for refs_resolve_ref_unsafe\n @@ Commit message\n that needs error information to make logic decisions, so update that caller\n \n Signed-off-by: Han-Wen Nienhuys \n - Reviewed-by: Ævar Arnfjörð Bjarmason \n + Signed-off-by: Ævar Arnfjörð Bjarmason \n +\n + ## refs.c ##\n +@@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \t\t\t\t\t type, failure_errno);\n + }\n + \n +-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n ++const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n + \t\t\t\t\t const char *refname,\n + \t\t\t\t\t int resolve_flags,\n + \t\t\t\t\t struct object_id *oid,\n +\n + ## refs.h ##\n +@@ refs.h: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n + \t\t\t\t int resolve_flags,\n + \t\t\t\t struct object_id *oid,\n + \t\t\t\t int *flags);\n ++/**\n ++ * refs_resolve_ref_unsafe_with_errno() is like\n ++ * refs_resolve_ref_unsafe(), but provide access to errno code that\n ++ * lead to a failure. We guarantee that errno is set to a meaningful\n ++ * value on non-zero return.\n ++ */\n ++const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n ++\t\t\t\t\t const char *refname,\n ++\t\t\t\t\t int resolve_flags,\n ++\t\t\t\t\t struct object_id *oid,\n ++\t\t\t\t\t int *flags, int *failure_errno);\n + const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n + \t\t\t struct object_id *oid, int *flags);\n + \n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n - \tif (!resolved) {\n --\t\tint last_errno = errno;\n --\t\tif (last_errno != ENOTDIR ||\n -+\t\tif (resolve_errno != ENOTDIR ||\n - \t\t /* in case of D/F conflict, try to generate a better error\n - \t\t * message. If that fails, fall back to strerror(ENOTDIR).\n - \t\t */\n - \t\t !refs_verify_refname_available(&refs->base, refname, extras,\n - \t\t\t\t\t\t skip, err))\n - \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n --\t\t\t\t refname, strerror(last_errno));\n -+\t\t\t\t refname, strerror(resolve_errno));\n - \n + \tif (!resolved &&\n +-\t (errno != ENOTDIR ||\n ++\t (resolve_errno != ENOTDIR ||\n + \t /* in case of D/F conflict, try to generate a better error\n + \t * message. If that fails, fall back to strerror(ENOTDIR).\n + \t */\n + \t !refs_verify_refname_available(&refs->base, refname, extras,\n + \t\t\t\t\t skip, err))) {\n + \t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n +-\t\t\t refname, strerror(errno));\n ++\t\t\t refname, strerror(resolve_errno));\n \t\tgoto error_return;\n \t}\n -\n - ## refs/refs-internal.h ##\n -@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n - \t\t struct object_id *oid, struct strbuf *referent,\n - \t\t unsigned int *type, int *failure_errno);\n \n -+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a\n -+ * failure. */\n -+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n -+\t\t\t\t\t const char *refname,\n -+\t\t\t\t\t int resolve_flags,\n -+\t\t\t\t\t struct object_id *oid,\n -+\t\t\t\t\t int *flags, int *failure_errno);\n -+\n - /*\n - * Write an error to `err` and return a nonzero value iff the same\n - * refname appears multiple times in `refnames`. `refnames` must be\n -: ----------- > 7: 10a40c9244e refs: make errno ignoring explicit in lock_ref_oid_basic()\n -: ----------- > 8: ff38a3f1936 refs file-backend.c: stop setting \"EBUSY\" in verify_lock()\n -: ----------- > 9: cdec272f06f refs file-backend.c: deal with errno directly in verify_lock()\n -: ----------- > 10: bd0639945a4 refs API: remove refs_read_ref_full() wrapper\n -: ----------- > 11: a359d1533bd refs API: make resolve_gitlink_ref() not set errno\n -: ----------- > 12: 07d550015ac refs API: make refs_resolve_ref_unsafe() static\n -: ----------- > 13: 73b70491515 refs API: make refs_resolve_refdup() not set errno\n -: ----------- > 14: 1e9de48d850 refs API: make refs_ref_exists() not set errno\n -: ----------- > 15: 446b50280be refs API: make resolve_ref_unsafe() not set errno\n -: ----------- > 16: 37c5b88d7d8 refs API: make expand_ref() and repo_dwim_log() not set errno\n -: ----------- > 17: f914df0bb25 refs API: don't leak \"errno\" in run_transaction_hook()\n-- \n2.32.0-dev\n\n"},{"id":"429688","messageId":"patch-03.17-a4e5f2d02fb-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 03/17] refs: make errno output explicit for read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:27Z","receivedAt":"2021-07-11T16:30:51Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes it explicit how alternative ref backends should report errors in\nread_raw_ref_fn.\n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs/debug.c | 4 ++--\n refs/files-backend.c | 29 +++++++++++++++--------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 20 ++++++++++++--------\n 5 files changed, 34 insertions(+), 29 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 8c9490235ea..7b3f05a66ff 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1681,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, &errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 7db4abccc34..f12413a9bc0 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \toidcpy(oid, null_oid());\n \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f0ce0aac857..b9df7862e57 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -611,7 +610,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tif (hold_lock_file_for_update_timeout(\n \t\t\t &lock->lk, ref_file.buf, LOCK_NO_DEREF,\n \t\t\t get_files_ref_lock_timeout_ms()) < 0) {\n-\t\tif (errno == ENOENT && --attempts_remaining > 0) {\n+\t\tint myerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr == ENOENT && --attempts_remaining > 0) {\n \t\t\t/*\n \t\t\t * Maybe somebody just deleted one of the\n \t\t\t * directories leading to ref_file. Try\n@@ -619,7 +620,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t */\n \t\t\tgoto retry;\n \t\t} else {\n-\t\t\tunable_to_lock_message(ref_file.buf, errno, err);\n+\t\t\tunable_to_lock_message(ref_file.buf, myerr, err);\n \t\t\tgoto error_return;\n \t\t}\n \t}\n@@ -629,9 +630,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -655,7 +656,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -693,13 +694,13 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex dfecdbc1db6..a457f18e93c 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex f4445e32904..79dfb3af484 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n+ * type, and return -1 (failure_errno should not be ENOENT)\n+ *\n+ * failure_errno provides errno codes that are interpreted beyond error\n+ * reporting. The following error codes have special meaning:\n+ * * ENOENT: the ref doesn't exist\n+ * * EISDIR: ref name is a directory\n+ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \n2.32.0-dev\n\n"},{"id":"429691","messageId":"patch-02.17-61cf7611473-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 02/17] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:26Z","receivedAt":"2021-07-11T16:30:51Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nIn the case of the \"errno != ENOTDIR\" case that originates in 5b2d8d6f218\n(lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,\n2015-05-11), there the \"last_errno\" was saved away to return it from\nlock_ref_oid_basic(), now that we're no longer doing that we can skip\nthat entirely and use \"errno\" directly. A follow-up change will\nextract the specific errno we want earlier in this function.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 24 +++++++++---------------\n 1 file changed, 9 insertions(+), 15 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 677b7e4cdd2..f0ce0aac857 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t * to remain.\n \t\t */\n \t\tif (remove_empty_directories(&ref_file)) {\n-\t\t\tlast_errno = errno;\n \t\t\tif (!refs_verify_refname_available(\n \t\t\t\t\t &refs->base,\n \t\t\t\t\t refname, extras, skip, err))\n@@ -961,14 +958,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t refname, resolve_flags,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n-\tif (!resolved) {\n-\t\tlast_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n-\t\t !refs_verify_refname_available(&refs->base, refname,\n-\t\t\t\t\t\t extras, skip, err))\n-\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n-\n+\tif (!resolved &&\n+\t (errno != ENOTDIR ||\n+\t /* in case of D/F conflict, try to generate a better error\n+\t * message. If that fails, fall back to strerror(ENOTDIR).\n+\t */\n+\t !refs_verify_refname_available(&refs->base, refname, extras,\n+\t\t\t\t\t skip, err))) {\n+\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n+\t\t\t refname, strerror(errno));\n \t\tgoto error_return;\n \t}\n \n@@ -981,20 +979,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n \t\t\t\t\t extras, skip, err)) {\n-\t\tlast_errno = ENOTDIR;\n \t\tgoto error_return;\n \t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n \n \tif (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {\n-\t\tlast_errno = errno;\n \t\tgoto error_return;\n \t}\n \tgoto out;\n@@ -1005,7 +1000,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \n2.32.0-dev\n\n"},{"id":"429690","messageId":"patch-04.17-270cda29c3a-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 04/17] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:28Z","receivedAt":"2021-07-11T16:30:52Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n\nSome of our callers explicitly do not care about the errno, rather\nthan understanding NULL let's have them declare that they don't care\nby passing in an \"ignore_errno\". There's only three of them, and using\nthat pattern will make it more obvious that they want to throw away\ndata, let's also add a comment to one of the callers about why we'd\nlike to ignore the errno.\n\nLet's not extend that to refs_resolve_ref_unsafe() itself for now, it\nhas a large set of legacy callers, so we're faking up the old \"errno\"\nbehavior for it. We can convert those callers to\nrefs_resolve_ref_unsafe_with_errno() later.\n\nWe are leaving out out the refs_read_special_head() in\nrefs_read_raw_ref() for now, as noted in the next commit moving it to\n\"failure_errno\" will require some special consideration.\n\nWe're intentionally mis-indenting the argument list of the new\nrefs_resolve_ref_unsafe_with_errno(), it will be non-static in a\nsubsequent commit, doing it this way makes that diff smaller.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 61 ++++++++++++++++++++++++++++++-------------\n refs/files-backend.c | 10 ++++---\n refs/packed-backend.c | 7 ++---\n refs/refs-internal.h | 6 ++---\n 4 files changed, 56 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 7b3f05a66ff..406e27213b8 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1671,30 +1671,33 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n+\tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &errno);\n+\t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n \tint symref_count;\n \n+\tassert(failure_errno);\n+\n \tif (!oid)\n \t\toid = &unused_oid;\n \tif (!flags)\n@@ -1705,7 +1708,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n+\t\t\t*failure_errno = EINVAL;\n \t\t\treturn NULL;\n \t\t}\n \n@@ -1723,8 +1726,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, failure_errno)) {\n \t\t\t*flags |= read_flags;\n \n \t\t\t/* In reading mode, refs must eventually resolve */\n@@ -1736,9 +1739,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (*failure_errno != ENOENT &&\n+\t\t\t *failure_errno != EISDIR &&\n+\t\t\t *failure_errno != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -1765,7 +1768,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t\t !refname_is_safe(refname)) {\n-\t\t\t\terrno = EINVAL;\n+\t\t\t\t*failure_errno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \n@@ -1773,10 +1776,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t}\n \t}\n \n-\terrno = ELOOP;\n+\t*failure_errno = ELOOP;\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tint failure_errno = 0;\n+\tconst char *refn;\n+\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &failure_errno);\n+\tif (!refn)\n+\t\t/* For unmigrated legacy callers */\n+\t\terrno = failure_errno;\n+\treturn refn;\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n@@ -2227,6 +2244,13 @@ int refs_verify_refname_available(struct ref_store *refs,\n \n \tstrbuf_grow(&dirname, strlen(refname) + 1);\n \tfor (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {\n+\t\t/*\n+\t\t * Just saying \"Is a directory\" when we e.g. can't\n+\t\t * lock some multi-level ref isn't very informative,\n+\t\t * the user won't be told *what* is a directory, so\n+\t\t * let's not use strerror() below.\n+\t\t */\n+\t\tint ignore_errno;\n \t\t/* Expand dirname to the new prefix, not including the trailing slash: */\n \t\tstrbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);\n \n@@ -2238,7 +2262,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, &ignore_errno)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b9df7862e57..79f96ba1058 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\tgoto out;\n \n \tif (lstat(path, &st) < 0) {\n+\t\tint ignore_errno;\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \t/* Is it a directory? */\n \tif (S_ISDIR(st.st_mode)) {\n+\t\tint ignore_errno;\n \t\t/*\n \t\t * Even though there is a directory where the loose\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a457f18e93c..928e5d2a994 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure_errno;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure_errno) ||\n+\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 79dfb3af484..54f57c6a2df 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /*\n * Write an error to `err` and return a nonzero value iff the same\n-- \n2.32.0-dev\n\n"},{"id":"429692","messageId":"patch-05.17-f5197cdc0b8-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 05/17] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:29Z","receivedAt":"2021-07-11T16:30:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe EINVAL error from parse_loose_ref_contents is used in files-backend\nto create a custom error message.\n\nIn untangling this we discovered a tricky edge case. The\nrefs_read_special_head() function was relying on\nparse_loose_ref_contents() setting EINVAL.\n\nBy converting it to use \"saved_errno\" we can migrate away from \"errno\"\nin this part of the code entirely, and do away with an existing\n\"save_errno\" pattern, its only purpose was to not clobber the \"errno\"\nwe previously needed at the end of files_read_raw_ref().\n\nLet's assert that we can do that by not having files_read_raw_ref()\nitself operate on *failure_errno in addition to passing it on. Instead\nwe'll assert that if we return non-zero we actually do set errno, thus\nassuring ourselves and callers that they can trust the resulting\n\"failure_errno\".\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 8 +++++---\n refs/files-backend.c | 30 +++++++++++++++++++-----------\n refs/refs-internal.h | 6 ++++--\n 3 files changed, 28 insertions(+), 16 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 406e27213b8..af01a692cb9 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n \n static int refs_read_special_head(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t\t int *failure_errno)\n {\n \tstruct strbuf full_path = STRBUF_INIT;\n \tstruct strbuf content = STRBUF_INIT;\n@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n \t\tgoto done;\n \n-\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n+\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n+\t\t\t\t\t failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\n@@ -1678,7 +1680,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 79f96ba1058..702e38a2b06 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tint fd;\n \tint ret = -1;\n \tint remaining_retries = 3;\n+\tint myerr = 0;\n \n \t*type = 0;\n \tstrbuf_reset(&sb_path);\n@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (lstat(path, &st) < 0) {\n \t\tint ignore_errno;\n-\t\tif (errno != ENOENT)\n+\t\tmyerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr != ENOENT)\n \t\t\tgoto out;\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = ENOENT;\n+\t\t\tmyerr = ENOENT;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tif (S_ISLNK(st.st_mode)) {\n \t\tstrbuf_reset(&sb_contents);\n \t\tif (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {\n-\t\t\tif (errno == ENOENT || errno == EINVAL)\n+\t\t\tmyerr = errno;\n+\t\t\terrno = 0;\n+\t\t\tif (myerr == ENOENT || myerr == EINVAL)\n \t\t\t\t/* inconsistent with lstat; retry */\n \t\t\t\tgoto stat_ref;\n \t\t\telse\n@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t */\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = EISDIR;\n+\t\t\tmyerr = EISDIR;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t */\n \tfd = open(path, O_RDONLY);\n \tif (fd < 0) {\n-\t\tif (errno == ENOENT && !S_ISLNK(st.st_mode))\n+\t\tmyerr = errno;\n+\t\tif (myerr == ENOENT && !S_ISLNK(st.st_mode))\n \t\t\t/* inconsistent with lstat; retry */\n \t\t\tgoto stat_ref;\n \t\telse\n@@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t}\n \tstrbuf_reset(&sb_contents);\n \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n-\t\tint save_errno = errno;\n \t\tclose(fd);\n-\t\terrno = save_errno;\n \t\tgoto out;\n \t}\n \tclose(fd);\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n-\tret = parse_loose_ref_contents(buf, oid, referent, type);\n+\tret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);\n \n out:\n-\t*failure_errno = errno;\n+\tif (ret && !myerr)\n+\t\tBUG(\"returning non-zero %d, should have set myerr!\", ret);\n+\t*failure_errno = myerr;\n+\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n \treturn ret;\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -486,7 +494,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n \tif (parse_oid_hex(buf, oid, &p) ||\n \t (*p != '\\0' && !isspace(*p))) {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n \treturn 0;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 54f57c6a2df..bf581e70cf6 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -689,10 +689,12 @@ struct ref_store {\n };\n \n /*\n- * Parse contents of a loose ref file.\n+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for\n+ * invalid contents.\n */\n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\n-- \n2.32.0-dev\n\n"},{"id":"429693","messageId":"patch-06.17-96689e523f1-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 06/17] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:30Z","receivedAt":"2021-07-11T16:30:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions, so update that caller\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs.h | 11 +++++++++++\n refs/files-backend.c | 14 ++++++++------\n 3 files changed, 20 insertions(+), 7 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex af01a692cb9..1dacb5fe27b 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1687,7 +1687,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t\t\t type, failure_errno);\n }\n \n-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n \t\t\t\t\t const char *refname,\n \t\t\t\t\t int resolve_flags,\n \t\t\t\t\t struct object_id *oid,\ndiff --git a/refs.h b/refs.h\nindex 48970dfc7e0..344a3c51a8c 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t\t int resolve_flags,\n \t\t\t\t struct object_id *oid,\n \t\t\t\t int *flags);\n+/**\n+ * refs_resolve_ref_unsafe_with_errno() is like\n+ * refs_resolve_ref_unsafe(), but provide access to errno code that\n+ * lead to a failure. We guarantee that errno is set to a meaningful\n+ * value on non-zero return.\n+ */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags);\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 702e38a2b06..d6ee7c1172f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -935,6 +935,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -947,10 +948,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t refname, resolve_flags,\n-\t\t\t\t\t &lock->old_oid, type);\n-\tif (!resolved && errno == EISDIR) {\n+\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\t\tresolve_flags,\n+\t\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t\t&resolve_errno);\n+\tif (!resolved && resolve_errno == EISDIR) {\n \t\t/*\n \t\t * we are trying to lock foo but we used to\n \t\t * have foo/bar which now does not exist;\n@@ -970,14 +972,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t &lock->old_oid, type);\n \t}\n \tif (!resolved &&\n-\t (errno != ENOTDIR ||\n+\t (resolve_errno != ENOTDIR ||\n \t /* in case of D/F conflict, try to generate a better error\n \t * message. If that fails, fall back to strerror(ENOTDIR).\n \t */\n \t !refs_verify_refname_available(&refs->base, refname, extras,\n \t\t\t\t\t skip, err))) {\n \t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t refname, strerror(errno));\n+\t\t\t refname, strerror(resolve_errno));\n \t\tgoto error_return;\n \t}\n \n-- \n2.32.0-dev\n\n"},{"id":"429694","messageId":"patch-07.17-10a40c9244e-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 07/17] refs: make errno ignoring explicit in lock_ref_oid_basic()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:31Z","receivedAt":"2021-07-11T16:30:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In the preceding commit we moved away from refs_resolve_ref_unsafe()\nin this function and started using a \"resolve_errno\" variable, which\nmeans that if this refs_resolve_ref_unsafe() invocation sets \"errno\"\nwe'd end up ignoring it for our earlier \"resolve_errno\" (if any).\n\nI'm not sure if this is what we should be doing, but let's add a BUG()\nhere for now to indicate that we hit a case we'd previously \"handle\",\nbut which we now ignore.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 13 ++++++++++---\n 1 file changed, 10 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex d6ee7c1172f..a4e9344ac8b 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -953,6 +953,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t\t\t&lock->old_oid, type,\n \t\t\t\t\t\t\t&resolve_errno);\n \tif (!resolved && resolve_errno == EISDIR) {\n+\t\tint ignore_errno = 0;\n \t\t/*\n \t\t * we are trying to lock foo but we used to\n \t\t * have foo/bar which now does not exist;\n@@ -967,9 +968,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t refname);\n \t\t\tgoto error_return;\n \t\t}\n-\t\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t\t refname, resolve_flags,\n-\t\t\t\t\t\t &lock->old_oid, type);\n+\t\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base,\n+\t\t\t\t\t\t\t\trefname, resolve_flags,\n+\t\t\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t\t\t&ignore_errno);\n+\t\tif (ignore_errno)\n+\t\t\tBUG(\"hit errno %d (%s) that will be ignored, \"\n+\t\t\t \"resolve errno is %d (%s)\",\n+\t\t\t ignore_errno, strerror(ignore_errno),\n+\t\t\t resolve_errno, strerror(resolve_errno));\n \t}\n \tif (!resolved &&\n \t (resolve_errno != ENOTDIR ||\n-- \n2.32.0-dev\n\n"},{"id":"429695","messageId":"patch-10.17-bd0639945a4-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 10/17] refs API: remove refs_read_ref_full() wrapper","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:34Z","receivedAt":"2021-07-11T16:30:59Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove the refs_read_ref_full() wrapper in favor of migrating various\nrefs.c API users to the underlying\nrefs_resolve_ref_unsafe_with_errno() function. A careful reading of\nthese callers shows that the callers of this function did not care\nabout \"errno\", by moving away from the refs_resolve_ref_unsafe()\nwrapper we can be sure that nothing relies on it anymore.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 22 ++++++++++------------\n refs.h | 2 --\n refs/files-backend.c | 30 +++++++++++++++++++-----------\n worktree.c | 9 +++++----\n 4 files changed, 34 insertions(+), 29 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 1dacb5fe27b..91320f092fb 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -294,20 +294,17 @@ struct ref_filter {\n \tvoid *cb_data;\n };\n \n-int refs_read_ref_full(struct ref_store *refs, const char *refname,\n-\t\t int resolve_flags, struct object_id *oid, int *flags)\n+int read_ref_full(const char *refname, int resolve_flags, struct object_id *oid, int *flags)\n {\n-\tif (refs_resolve_ref_unsafe(refs, refname, resolve_flags, oid, flags))\n+\tint ignore_errno;\n+\tstruct ref_store *refs = get_main_ref_store(the_repository);\n+\n+\tif (refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t oid, flags, &ignore_errno))\n \t\treturn 0;\n \treturn -1;\n }\n \n-int read_ref_full(const char *refname, int resolve_flags, struct object_id *oid, int *flags)\n-{\n-\treturn refs_read_ref_full(get_main_ref_store(the_repository), refname,\n-\t\t\t\t resolve_flags, oid, flags);\n-}\n-\n int read_ref(const char *refname, struct object_id *oid)\n {\n \treturn read_ref_full(refname, RESOLVE_REF_READING, oid, NULL);\n@@ -1398,9 +1395,10 @@ int refs_head_ref(struct ref_store *refs, each_ref_fn fn, void *cb_data)\n {\n \tstruct object_id oid;\n \tint flag;\n-\n-\tif (!refs_read_ref_full(refs, \"HEAD\", RESOLVE_REF_READING,\n-\t\t\t\t&oid, &flag))\n+\tint ignore_errno;\n+\t\n+\tif (refs_resolve_ref_unsafe_with_errno(refs, \"HEAD\", RESOLVE_REF_READING,\n+\t\t\t\t\t &oid, &flag, &ignore_errno))\n \t\treturn fn(\"HEAD\", &oid, flag, cb_data);\n \n \treturn 0;\ndiff --git a/refs.h b/refs.h\nindex 344a3c51a8c..d80184dd8e0 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -88,8 +88,6 @@ char *refs_resolve_refdup(struct ref_store *refs,\n char *resolve_refdup(const char *refname, int resolve_flags,\n \t\t struct object_id *oid, int *flags);\n \n-int refs_read_ref_full(struct ref_store *refs, const char *refname,\n-\t\t int resolve_flags, struct object_id *oid, int *flags);\n int read_ref_full(const char *refname, int resolve_flags,\n \t\t struct object_id *oid, int *flags);\n int read_ref(const char *refname, struct object_id *oid);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex e3d27001b86..b3bc2f57387 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1342,6 +1342,7 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store,\n \tstruct strbuf tmp_renamed_log = STRBUF_INIT;\n \tint log, ret;\n \tstruct strbuf err = STRBUF_INIT;\n+\tint ignore_errno;\n \n \tfiles_reflog_path(refs, &sb_oldref, oldrefname);\n \tfiles_reflog_path(refs, &sb_newref, newrefname);\n@@ -1399,9 +1400,9 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store,\n \t * the safety anyway; we want to delete the reference whatever\n \t * its current value.\n \t */\n-\tif (!copy && !refs_read_ref_full(&refs->base, newrefname,\n-\t\t\t\tRESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,\n-\t\t\t\tNULL, NULL) &&\n+\tif (!copy && refs_resolve_ref_unsafe_with_errno(&refs->base, newrefname,\n+\t\t\t\t\t\t\tRESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,\n+\t\t\t\t\t\t\tNULL, NULL, &ignore_errno) &&\n \t refs_delete_ref(&refs->base, NULL, newrefname,\n \t\t\t NULL, REF_NO_DEREF)) {\n \t\tif (errno == EISDIR) {\n@@ -1815,9 +1816,12 @@ static void update_symref_reflog(struct files_ref_store *refs,\n {\n \tstruct strbuf err = STRBUF_INIT;\n \tstruct object_id new_oid;\n+\tint ignore_errno;\n+\n \tif (logmsg &&\n-\t !refs_read_ref_full(&refs->base, target,\n-\t\t\t\tRESOLVE_REF_READING, &new_oid, NULL) &&\n+\t refs_resolve_ref_unsafe_with_errno(&refs->base, target,\n+\t\t\t\t\t RESOLVE_REF_READING, &new_oid, NULL,\n+\t\t\t\t\t &ignore_errno) &&\n \t files_log_ref_write(refs, refname, &lock->old_oid,\n \t\t\t\t&new_oid, logmsg, 0, &err)) {\n \t\terror(\"%s\", err.buf);\n@@ -2093,6 +2097,7 @@ static int files_reflog_iterator_advance(struct ref_iterator *ref_iterator)\n \t\t(struct files_reflog_iterator *)ref_iterator;\n \tstruct dir_iterator *diter = iter->dir_iterator;\n \tint ok;\n+\tint ignore_errno;\n \n \twhile ((ok = dir_iterator_advance(diter)) == ITER_OK) {\n \t\tint flags;\n@@ -2104,9 +2109,10 @@ static int files_reflog_iterator_advance(struct ref_iterator *ref_iterator)\n \t\tif (ends_with(diter->basename, \".lock\"))\n \t\t\tcontinue;\n \n-\t\tif (refs_read_ref_full(iter->ref_store,\n-\t\t\t\t diter->relative_path, 0,\n-\t\t\t\t &iter->oid, &flags)) {\n+\t\tif (!refs_resolve_ref_unsafe_with_errno(iter->ref_store,\n+\t\t\t\t\t\t\tditer->relative_path, 0,\n+\t\t\t\t\t\t\t&iter->oid, &flags,\n+\t\t\t\t\t\t\t&ignore_errno)) {\n \t\t\terror(\"bad ref for %s\", diter->path.buf);\n \t\t\tcontinue;\n \t\t}\n@@ -2450,9 +2456,11 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t\t\t * the transaction, so we have to read it here\n \t\t\t * to record and possibly check old_oid:\n \t\t\t */\n-\t\t\tif (refs_read_ref_full(&refs->base,\n-\t\t\t\t\t referent.buf, 0,\n-\t\t\t\t\t &lock->old_oid, NULL)) {\n+\t\t\tint ignore_errno;\n+\t\t\tif (!refs_resolve_ref_unsafe_with_errno(&refs->base,\n+\t\t\t\t\t\t\t\treferent.buf, 0,\n+\t\t\t\t\t\t\t\t&lock->old_oid, NULL,\n+\t\t\t\t\t\t\t\t&ignore_errno)) {\n \t\t\t\tif (update->flags & REF_HAVE_OLD) {\n \t\t\t\t\tstrbuf_addf(err, \"cannot lock ref '%s': \"\n \t\t\t\t\t\t \"error reading reference\",\ndiff --git a/worktree.c b/worktree.c\nindex 237517baee6..1dcdb481d61 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -562,16 +562,17 @@ int other_head_refs(each_ref_fn fn, void *cb_data)\n \t\tstruct worktree *wt = *p;\n \t\tstruct object_id oid;\n \t\tint flag;\n+\t\tint ignore_errno;\n \n \t\tif (wt->is_current)\n \t\t\tcontinue;\n \n \t\tstrbuf_reset(&refname);\n \t\tstrbuf_worktree_ref(wt, &refname, \"HEAD\");\n-\t\tif (!refs_read_ref_full(get_main_ref_store(the_repository),\n-\t\t\t\t\trefname.buf,\n-\t\t\t\t\tRESOLVE_REF_READING,\n-\t\t\t\t\t&oid, &flag))\n+\t\tif (refs_resolve_ref_unsafe_with_errno(get_main_ref_store(the_repository),\n+\t\t\t\t\t\t refname.buf,\n+\t\t\t\t\t\t RESOLVE_REF_READING,\n+\t\t\t\t\t\t &oid, &flag, &ignore_errno))\n \t\t\tret = fn(refname.buf, &oid, flag, cb_data);\n \t\tif (ret)\n \t\t\tbreak;\n-- \n2.32.0-dev\n\n"},{"id":"429696","messageId":"patch-11.17-a359d1533bd-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 11/17] refs API: make resolve_gitlink_ref() not set errno","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:35Z","receivedAt":"2021-07-11T16:31:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I have carefully read the upstream callers of resolve_gitlink_ref()\nand determined that they don't care about errno. So let's move away\nfrom the errno-setting refs_resolve_ref_unsafe() wrapper to\nrefs_resolve_ref_unsafe_with_errno(), and explicitly ignore the errno\nit sets for us.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 91320f092fb..7a9cdb1d720 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1814,14 +1814,15 @@ int resolve_gitlink_ref(const char *submodule, const char *refname,\n {\n \tstruct ref_store *refs;\n \tint flags;\n+\tint ignore_errno;\n \n \trefs = get_submodule_ref_store(submodule);\n \n \tif (!refs)\n \t\treturn -1;\n \n-\tif (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||\n-\t is_null_oid(oid))\n+\tif (!refs_resolve_ref_unsafe_with_errno(refs, refname, 0, oid, &flags,\n+\t\t &ignore_errno) || is_null_oid(oid))\n \t\treturn -1;\n \treturn 0;\n }\n-- \n2.32.0-dev\n\n"},{"id":"429697","messageId":"patch-08.17-ff38a3f1936-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 08/17] refs file-backend.c: stop setting \"EBUSY\" in verify_lock()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:32Z","receivedAt":"2021-07-11T16:31:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"This EBUSY was set in 835e3c992fd (refs.c: verify_lock should set\nerrno to something meaningful, 2014-06-20) to fix a bug in code that\nwas later refactored in cd94f765720 (fetch.c: change s_update_ref to\nuse a ref transaction, 2014-04-28) to not use errno at all, the two\nwere a part of the same series.\n\nSo this was only ever needed for an intra-series bugfix, and we kept\nit around for no reason. Removing it makes subsequent commits where we\nrefactor code surrounding verify_lock() (which may use errno) easier\nto reason about.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 3 ---\n 1 file changed, 3 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex a4e9344ac8b..abed0572fea 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -880,9 +880,7 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,\n \t\t\t mustexist ? RESOLVE_REF_READING : 0,\n \t\t\t &lock->old_oid, NULL)) {\n \t\tif (old_oid) {\n-\t\t\tint save_errno = errno;\n \t\t\tstrbuf_addf(err, \"can't verify ref '%s'\", lock->ref_name);\n-\t\t\terrno = save_errno;\n \t\t\treturn -1;\n \t\t} else {\n \t\t\toidclr(&lock->old_oid);\n@@ -894,7 +892,6 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,\n \t\t\t lock->ref_name,\n \t\t\t oid_to_hex(&lock->old_oid),\n \t\t\t oid_to_hex(old_oid));\n-\t\terrno = EBUSY;\n \t\treturn -1;\n \t}\n \treturn 0;\n-- \n2.32.0-dev\n\n"},{"id":"429698","messageId":"patch-09.17-cdec272f06f-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 09/17] refs file-backend.c: deal with errno directly in verify_lock()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:33Z","receivedAt":"2021-07-11T16:31:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Instead of using the refs_read_ref_full() wrapper function let's use\nthe refs_resolve_ref_unsafe_with_errno() function it resolves to (note\nthe inverted return values).\n\nPer a preceding change to remove an \"errno\" assignment from\nverify_lock() we can explicitly ignore errno here, we use the strbuf\n\"err\" API instead.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 7 ++++---\n 1 file changed, 4 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex abed0572fea..e3d27001b86 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -874,11 +874,12 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,\n \t\t const struct object_id *old_oid, int mustexist,\n \t\t struct strbuf *err)\n {\n+\tint ignore_errno;\n \tassert(err);\n \n-\tif (refs_read_ref_full(ref_store, lock->ref_name,\n-\t\t\t mustexist ? RESOLVE_REF_READING : 0,\n-\t\t\t &lock->old_oid, NULL)) {\n+\tif (!refs_resolve_ref_unsafe_with_errno(ref_store, lock->ref_name,\n+\t\t\t\t\t\tmustexist ? RESOLVE_REF_READING : 0,\n+\t\t\t\t\t\t&lock->old_oid, NULL, &ignore_errno)) {\n \t\tif (old_oid) {\n \t\t\tstrbuf_addf(err, \"can't verify ref '%s'\", lock->ref_name);\n \t\t\treturn -1;\n-- \n2.32.0-dev\n\n"},{"id":"429699","messageId":"patch-12.17-07d550015ac-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 12/17] refs API: make refs_resolve_ref_unsafe() static","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:36Z","receivedAt":"2021-07-11T16:31:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove public users of refs_resolve_ref_unsafe() over to\nrefs_resolve_ref_unsafe_with_errno(), and explicitly ignore the\nprovided errno in those callers. At that point the only remaining\nusers are the legacy functions in refs.c itself, so we can make the\nwrapper static.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 29 +++++++++++++++--------------\n refs.h | 8 +-------\n refs/files-backend.c | 24 ++++++++++++++----------\n t/helper/test-ref-store.c | 5 +++--\n worktree.c | 18 ++++++++++++------\n 5 files changed, 45 insertions(+), 39 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 7a9cdb1d720..214f3f4f79f 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -267,6 +267,21 @@ int ref_resolves_to_object(const char *refname,\n \treturn 1;\n }\n \n+static const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid, int *flags)\n+{\n+\tint failure_errno = 0;\n+\tconst char *refn;\n+\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &failure_errno);\n+\tif (!refn)\n+\t\t/* For unmigrated legacy callers */\n+\t\terrno = failure_errno;\n+\treturn refn;\n+}\n+\n char *refs_resolve_refdup(struct ref_store *refs,\n \t\t\t const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags)\n@@ -1780,20 +1795,6 @@ const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n \treturn NULL;\n }\n \n-const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n-\t\t\t\t int resolve_flags, struct object_id *oid,\n-\t\t\t\t int *flags)\n-{\n-\tint failure_errno = 0;\n-\tconst char *refn;\n-\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n-\t\t\t\t\t\t oid, flags, &failure_errno);\n-\tif (!refn)\n-\t\t/* For unmigrated legacy callers */\n-\t\terrno = failure_errno;\n-\treturn refn;\n-}\n-\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\ndiff --git a/refs.h b/refs.h\nindex d80184dd8e0..a3284adb8f8 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -63,14 +63,8 @@ struct worktree;\n #define RESOLVE_REF_NO_RECURSE 0x02\n #define RESOLVE_REF_ALLOW_BAD_NAME 0x04\n \n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid,\n-\t\t\t\t int *flags);\n /**\n- * refs_resolve_ref_unsafe_with_errno() is like\n- * refs_resolve_ref_unsafe(), but provide access to errno code that\n+ * refs_resolve_ref_unsafe_with_errno() provides access to errno code that\n * lead to a failure. We guarantee that errno is set to a meaningful\n * value on non-zero return.\n */\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b3bc2f57387..158c0146484 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -280,10 +280,11 @@ static void loose_fill_ref_dir(struct ref_store *ref_store,\n \t\t\t\t\t create_dir_entry(dir->cache, refname.buf,\n \t\t\t\t\t\t\t refname.len, 1));\n \t\t} else {\n-\t\t\tif (!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t\t refname.buf,\n-\t\t\t\t\t\t RESOLVE_REF_READING,\n-\t\t\t\t\t\t &oid, &flag)) {\n+\t\t\tint ignore_errno;\n+\t\t\tif (!refs_resolve_ref_unsafe_with_errno(&refs->base,\n+\t\t\t\t\t\t\t\trefname.buf,\n+\t\t\t\t\t\t\t\tRESOLVE_REF_READING,\n+\t\t\t\t\t\t\t\t&oid, &flag, &ignore_errno)) {\n \t\t\t\toidclr(&oid);\n \t\t\t\tflag |= REF_ISBROKEN;\n \t\t\t} else if (is_null_oid(&oid)) {\n@@ -1354,9 +1355,9 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store,\n \t\tgoto out;\n \t}\n \n-\tif (!refs_resolve_ref_unsafe(&refs->base, oldrefname,\n-\t\t\t\t RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,\n-\t\t\t\t&orig_oid, &flag)) {\n+\tif (!refs_resolve_ref_unsafe_with_errno(&refs->base, oldrefname,\n+\t\t\t\t\t\tRESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,\n+\t\t\t\t\t\t&orig_oid, &flag, &ignore_errno)) {\n \t\tret = error(\"refname %s not found\", oldrefname);\n \t\tgoto out;\n \t}\n@@ -1769,10 +1770,13 @@ static int commit_ref_update(struct files_ref_store *refs,\n \t\t */\n \t\tint head_flag;\n \t\tconst char *head_ref;\n+\t\tint ignore_errno;\n \n-\t\thead_ref = refs_resolve_ref_unsafe(&refs->base, \"HEAD\",\n-\t\t\t\t\t\t RESOLVE_REF_READING,\n-\t\t\t\t\t\t NULL, &head_flag);\n+\t\thead_ref = refs_resolve_ref_unsafe_with_errno(&refs->base,\n+\t\t\t\t\t\t\t \"HEAD\",\n+\t\t\t\t\t\t\t RESOLVE_REF_READING,\n+\t\t\t\t\t\t\t NULL, &head_flag,\n+\t\t\t\t\t\t\t &ignore_errno);\n \t\tif (head_ref && (head_flag & REF_ISSYMREF) &&\n \t\t !strcmp(head_ref, lock->ref_name)) {\n \t\t\tstruct strbuf log_err = STRBUF_INIT;\ndiff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c\nindex bba5f841c6a..ca69a793171 100644\n--- a/t/helper/test-ref-store.c\n+++ b/t/helper/test-ref-store.c\n@@ -123,9 +123,10 @@ static int cmd_resolve_ref(struct ref_store *refs, const char **argv)\n \tint resolve_flags = arg_flags(*argv++, \"resolve-flags\");\n \tint flags;\n \tconst char *ref;\n+\tint ignore_errno;\n \n-\tref = refs_resolve_ref_unsafe(refs, refname, resolve_flags,\n-\t\t\t\t &oid, &flags);\n+\tref = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t &oid, &flags, &ignore_errno);\n \tprintf(\"%s %s 0x%x\\n\", oid_to_hex(&oid), ref ? ref : \"(null)\", flags);\n \treturn ref ? 0 : 1;\n }\ndiff --git a/worktree.c b/worktree.c\nindex 1dcdb481d61..1dbdec82461 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -28,11 +28,13 @@ static void add_head_info(struct worktree *wt)\n {\n \tint flags;\n \tconst char *target;\n+\tint ignore_errno;\n \n-\ttarget = refs_resolve_ref_unsafe(get_worktree_ref_store(wt),\n-\t\t\t\t\t \"HEAD\",\n-\t\t\t\t\t 0,\n-\t\t\t\t\t &wt->head_oid, &flags);\n+\ttarget = refs_resolve_ref_unsafe_with_errno(get_worktree_ref_store(wt),\n+\t\t\t\t\t\t \"HEAD\",\n+\t\t\t\t\t\t 0,\n+\t\t\t\t\t\t &wt->head_oid, &flags,\n+\t\t\t\t\t\t &ignore_errno);\n \tif (!target)\n \t\treturn;\n \n@@ -417,6 +419,7 @@ const struct worktree *find_shared_symref(const char *symref,\n \t\tconst char *symref_target;\n \t\tstruct ref_store *refs;\n \t\tint flags;\n+\t\tint ignore_errno;\n \n \t\tif (wt->is_bare)\n \t\t\tcontinue;\n@@ -433,8 +436,11 @@ const struct worktree *find_shared_symref(const char *symref,\n \t\t}\n \n \t\trefs = get_worktree_ref_store(wt);\n-\t\tsymref_target = refs_resolve_ref_unsafe(refs, symref, 0,\n-\t\t\t\t\t\t\tNULL, &flags);\n+\t\tsymref_target = refs_resolve_ref_unsafe_with_errno(refs,\n+\t\t\t\t\t\t\t\t symref,\n+\t\t\t\t\t\t\t\t 0, NULL,\n+\t\t\t\t\t\t\t\t &flags,\n+\t\t\t\t\t\t\t\t &ignore_errno);\n \t\tif ((flags & REF_ISSYMREF) &&\n \t\t symref_target && !strcmp(symref_target, target)) {\n \t\t\texisting = wt;\n-- \n2.32.0-dev\n\n"},{"id":"429700","messageId":"patch-13.17-73b70491515-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 13/17] refs API: make refs_resolve_refdup() not set errno","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:37Z","receivedAt":"2021-07-11T16:31:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move refs_resolve_refdup() from the legacy refs_resolve_ref_unsafe()\nto the new refs_resolve_ref_unsafe_with_errno(). I have read its\ncallers and determined that they don't care about errno being set.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 214f3f4f79f..45b51c0a532 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -287,9 +287,11 @@ char *refs_resolve_refdup(struct ref_store *refs,\n \t\t\t struct object_id *oid, int *flags)\n {\n \tconst char *result;\n+\tint ignore_errno;\n \n-\tresult = refs_resolve_ref_unsafe(refs, refname, resolve_flags,\n-\t\t\t\t\t oid, flags);\n+\tresult = refs_resolve_ref_unsafe_with_errno(refs, refname,\n+\t\t\t\t\t\t resolve_flags, oid, flags,\n+\t\t\t\t\t\t &ignore_errno);\n \treturn xstrdup_or_null(result);\n }\n \n-- \n2.32.0-dev\n\n"},{"id":"429701","messageId":"patch-14.17-1e9de48d850-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 14/17] refs API: make refs_ref_exists() not set errno","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:38Z","receivedAt":"2021-07-11T16:31:05Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move refs_ref_exists from the legacy refs_resolve_ref_unsafe() to the\nnew refs_resolve_ref_unsafe_with_errno(). I have read its callers and\ndetermined that they don't care about errno being set, in particular:\n\n git grep -W -w -e refs_ref_exists -e ref_exists\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 5 ++++-\n 1 file changed, 4 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 45b51c0a532..0364b68f8fa 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -329,7 +329,10 @@ int read_ref(const char *refname, struct object_id *oid)\n \n int refs_ref_exists(struct ref_store *refs, const char *refname)\n {\n-\treturn !!refs_resolve_ref_unsafe(refs, refname, RESOLVE_REF_READING, NULL, NULL);\n+\tint ignore_errno;\n+\treturn !!refs_resolve_ref_unsafe_with_errno(refs, refname,\n+\t\t\t\t\t\t RESOLVE_REF_READING, NULL,\n+\t\t\t\t\t\t NULL, &ignore_errno);\n }\n \n int ref_exists(const char *refname)\n-- \n2.32.0-dev\n\n"},{"id":"429702","messageId":"patch-15.17-446b50280be-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 15/17] refs API: make resolve_ref_unsafe() not set errno","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:39Z","receivedAt":"2021-07-11T16:31:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the resolve_ref_unsafe() wrapper function to use\nrefs_resolve_ref_unsafe_with_errno() directly.\n\nFrom a reading of the callers I determined that the only one who cared\nabout errno was a sequencer.c caller added in e47c6cafcb5 (commit:\nmove print_commit_summary() to libgit, 2017-11-24), I'm migrating it\nto using refs_resolve_ref_unsafe_with_errno() directly.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 7 +++++--\n sequencer.c | 10 ++++++++--\n 2 files changed, 13 insertions(+), 4 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 0364b68f8fa..1d44027497e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1811,8 +1811,11 @@ int refs_init_db(struct strbuf *err)\n const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags)\n {\n-\treturn refs_resolve_ref_unsafe(get_main_ref_store(the_repository), refname,\n-\t\t\t\t resolve_flags, oid, flags);\n+\tstruct ref_store *refs = get_main_ref_store(the_repository);\n+\tint ignore_errno;\n+\n+\treturn refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &ignore_errno);\n }\n \n int resolve_gitlink_ref(const char *submodule, const char *refname,\ndiff --git a/sequencer.c b/sequencer.c\nindex 0bec01cf38e..c4772413b86 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -1267,6 +1267,8 @@ void print_commit_summary(struct repository *r,\n \tstruct pretty_print_context pctx = {0};\n \tstruct strbuf author_ident = STRBUF_INIT;\n \tstruct strbuf committer_ident = STRBUF_INIT;\n+\tstruct ref_store *refs;\n+\tint resolve_errno;\n \n \tcommit = lookup_commit(r, oid);\n \tif (!commit)\n@@ -1316,9 +1318,13 @@ void print_commit_summary(struct repository *r,\n \trev.diffopt.break_opt = 0;\n \tdiff_setup_done(&rev.diffopt);\n \n-\thead = resolve_ref_unsafe(\"HEAD\", 0, NULL, NULL);\n-\tif (!head)\n+\trefs = get_main_ref_store(the_repository);\n+\thead = refs_resolve_ref_unsafe_with_errno(refs, \"HEAD\", 0, NULL, NULL,\n+\t\t\t\t\t\t &resolve_errno);\n+\tif (!head) {\n+\t\terrno = resolve_errno;\n \t\tdie_errno(_(\"unable to resolve HEAD after creating commit\"));\n+\t}\n \tif (!strcmp(head, \"HEAD\"))\n \t\thead = _(\"detached HEAD\");\n \telse\n-- \n2.32.0-dev\n\n"},{"id":"429703","messageId":"patch-17.17-f914df0bb25-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 17/17] refs API: don't leak \"errno\" in run_transaction_hook()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:41Z","receivedAt":"2021-07-11T16:31:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In run_transaction_hook() we've checked errno since 67541597670 (refs:\nimplement reference transaction hook, 2020-06-19), let's reset errno\nafterwards to make sure nobody using refs.c directly or indirectly\nrelies on it.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 5 ++++-\n 1 file changed, 4 insertions(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 3357ba14e1d..f1477eed27a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -2108,8 +2108,11 @@ static int run_transaction_hook(struct ref_transaction *transaction,\n \t\t\t update->refname);\n \n \t\tif (write_in_full(proc.in, buf.buf, buf.len) < 0) {\n-\t\t\tif (errno != EPIPE)\n+\t\t\tif (errno != EPIPE) {\n+\t\t\t\t/* Don't leak errno outside this API */\n+\t\t\t\terrno = 0;\n \t\t\t\tret = -1;\n+\t\t\t}\n \t\t\tbreak;\n \t\t}\n \t}\n-- \n2.32.0-dev\n\n"},{"id":"429704","messageId":"patch-16.17-37c5b88d7d8-20210711T162803Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v6? 16/17] refs API: make expand_ref() and repo_dwim_log() not set errno","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-11T16:30:40Z","receivedAt":"2021-07-11T16:31:08Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The use of these two is rather trivial, and it's easy to see none of\ntheir callers care about errno. So let's move them from\nrefs_resolve_ref_unsafe() to refs_resolve_ref_unsafe_with_errno(),\nthese were the last two callers, so we can get rid of that wrapper\nfunction.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 32 +++++++++++---------------------\n 1 file changed, 11 insertions(+), 21 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 1d44027497e..3357ba14e1d 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -267,21 +267,6 @@ int ref_resolves_to_object(const char *refname,\n \treturn 1;\n }\n \n-static const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t\t const char *refname,\n-\t\t\t\t\t int resolve_flags,\n-\t\t\t\t\t struct object_id *oid, int *flags)\n-{\n-\tint failure_errno = 0;\n-\tconst char *refn;\n-\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n-\t\t\t\t\t\t oid, flags, &failure_errno);\n-\tif (!refn)\n-\t\t/* For unmigrated legacy callers */\n-\t\terrno = failure_errno;\n-\treturn refn;\n-}\n-\n char *refs_resolve_refdup(struct ref_store *refs,\n \t\t\t const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags)\n@@ -675,13 +660,16 @@ int expand_ref(struct repository *repo, const char *str, int len,\n \t\tstruct object_id oid_from_ref;\n \t\tstruct object_id *this_result;\n \t\tint flag;\n+\t\tstruct ref_store *refs = get_main_ref_store(repo);\n+\t\tint ignore_errno;\n \n \t\tthis_result = refs_found ? &oid_from_ref : oid;\n \t\tstrbuf_reset(&fullref);\n \t\tstrbuf_addf(&fullref, *p, len, str);\n-\t\tr = refs_resolve_ref_unsafe(get_main_ref_store(repo),\n-\t\t\t\t\t fullref.buf, RESOLVE_REF_READING,\n-\t\t\t\t\t this_result, &flag);\n+\t\tr = refs_resolve_ref_unsafe_with_errno(refs, fullref.buf,\n+\t\t\t\t\t\t RESOLVE_REF_READING,\n+\t\t\t\t\t\t this_result, &flag,\n+\t\t\t\t\t\t &ignore_errno);\n \t\tif (r) {\n \t\t\tif (!refs_found++)\n \t\t\t\t*ref = xstrdup(r);\n@@ -710,12 +698,14 @@ int repo_dwim_log(struct repository *r, const char *str, int len,\n \tfor (p = ref_rev_parse_rules; *p; p++) {\n \t\tstruct object_id hash;\n \t\tconst char *ref, *it;\n+\t\tint ignore_errno;\n \n \t\tstrbuf_reset(&path);\n \t\tstrbuf_addf(&path, *p, len, str);\n-\t\tref = refs_resolve_ref_unsafe(refs, path.buf,\n-\t\t\t\t\t RESOLVE_REF_READING,\n-\t\t\t\t\t &hash, NULL);\n+\t\tref = refs_resolve_ref_unsafe_with_errno(refs, path.buf,\n+\t\t\t\t\t\t\t RESOLVE_REF_READING,\n+\t\t\t\t\t\t\t &hash, NULL,\n+\t\t\t\t\t\t\t &ignore_errno);\n \t\tif (!ref)\n \t\t\tcontinue;\n \t\tif (refs_reflog_exists(refs, path.buf))\n-- \n2.32.0-dev\n\n"},{"id":"429869","messageId":"CAFQ2z_OftKcmR3ZgogAjiSGEBNmB7Akb01HLy9Abc0xSR5oSHQ@mail.gmail.com","threadId":"55589","inReplyTo":"87lf6d3wbz.fsf@evledraar.gmail.com","subject":"Re: [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-13T08:00:19Z","receivedAt":"2021-07-13T08:00:36Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Sun, Jul 11, 2021 at 1:48 PM Ævar Arnfjörð Bjarmason\n wrote:\n>\n>\n> On Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n>\n> > /*\n> > * Locks a ref returning the lock on success and NULL on failure.\n> > - * On failure errno is set to something meaningful.\n> > */\n> > static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> > const char *refname,\n> > @@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> > {\n> > struct strbuf ref_file = STRBUF_INIT;\n> > struct ref_lock *lock;\n> > - int last_errno = 0;\n> > int mustexist = (old_oid && !is_null_oid(old_oid));\n> > int resolve_flags = RESOLVE_REF_NO_RECURSE;\n> > int resolved;\n> > @@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> > * to remain.\n> > */\n> > if (remove_empty_directories(&ref_file)) {\n> > - last_errno = errno;\n> > if (!refs_verify_refname_available(\n> > &refs->base,\n> > refname, extras, skip, err))\n> > @@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n> > &lock->old_oid, type);\n> > }\n> > if (!resolved) {\n> > - last_errno = errno;\n> > + int last_errno = errno;\n> > if (last_errno != ENOTDIR ||\n> > - !refs_verify_refname_available(&refs->base, refname,\n> > - extras, skip, err))\n> > + /* in case of D/F conflict, try to generate a better error\n> > + * message. If that fails, fall back to strerror(ENOTDIR).\n> > + */\n> > + !refs_verify_refname_available(&refs->base, refname, extras,\n> > + skip, err))\n> > strbuf_addf(err, \"unable to resolve reference '%s': %s\",\n> > refname, strerror(last_errno));\n>\n> I don't think it's some dealbreaker and we can move on, but just FWIW I\n> think what I mentioned ending in your\n> https://lore.kernel.org/git/CAFQ2z_NpyJQLuM70MhJ8K1h2v3QXFuAZRjN=SvSsjnukNRJ8pw@mail.gmail.com/\n> is still outstanding.\n>\n> I.e. you added the comment, which is just says what the error emitting\n> looks like, that's all well & good.\n>\n> But what I was pointing out that it didn't make sense to do any\n> \"last_errno\" here at all anymore. You pointed to 5b2d8d6f218\n> (lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,\n> 2015-05-11), we started setting \"last_errno\" there, but that was *not*\n> to avoid clobbering between the !resolved and the\n> strbuf_add(strerror(last_errno)) here, but rather to carry the\n> \"last_errno\" forward to the end of this lock_ref_oid_basic(), because\n> other things (after this hunk) might reset/clear errno.\n\nI disagree. In your suggested change\n\n> + if (!resolved &&\n> + (errno != ENOTDIR ||\n> + /* in case of D/F conflict, try to generate a better error\n> + * message. If that fails, fall back to strerror(ENOTDIR).\n> + */\n> + !refs_verify_refname_available(&refs->base, refname, extras,\n> + skip, err))) {\n> + strbuf_addf(err, \"unable to resolve reference '%s': %s\",\n> + refname, strerror(errno));\n> goto error_return;\n\nthe refs_verify_refname_available() call happens only if\nerrno==ENOTDIR. The call might clobber the ENOTDIR errno and then\nfail. Then we'd be printing the last errno that\nrefs_verify_refname_available() saw, which may be different from\nENOTDIR.\n\nOther than that, for clarity's sake, it's always better to avoid the\nuse of global errno if we can.\n\n--\nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429870","messageId":"CAFQ2z_NzykfcBSvtHQ10RZWxLP48+qArZk9HbDJQB4tNX-GtYA@mail.gmail.com","threadId":"55589","inReplyTo":"87fswl3vsn.fsf@evledraar.gmail.com","subject":"Re: [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-13T08:02:29Z","receivedAt":"2021-07-13T08:02:42Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Sun, Jul 11, 2021 at 2:00 PM Ævar Arnfjörð Bjarmason\n wrote:\n>\n>\n> On Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:\n>\n> > From: Han-Wen Nienhuys \n> >\n> > This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n> > [...]\n> > +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n>\n> This and the subsequent commit don't compile for me, because this lacks\n> a prototype that you finally add in 6/6.\n\nWhoops! thanks for correcting.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429878","messageId":"CAFQ2z_PNLBCwZhfgk-9GBsBqoM_LSdeYR0ALSJhgF-h1uokWQw@mail.gmail.com","threadId":"55589","inReplyTo":"patch-08.17-ff38a3f1936-20210711T162803Z-avarab@gmail.com","subject":"Re: [PATCH v6? 08/17] refs file-backend.c: stop setting \"EBUSY\" in verify_lock()","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-13T08:08:45Z","receivedAt":"2021-07-13T08:08:59Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> This EBUSY was set in 835e3c992fd (refs.c: verify_lock should set\n> errno to something meaningful, 2014-06-20) to fix a bug in code that\n> was later refactored in cd94f765720 (fetch.c: change s_update_ref to\n> use a ref transaction, 2014-04-28) to not use errno at all, the two\n> were a part of the same series.\n>\n> So this was only ever needed for an intra-series bugfix, and we kept\n> it around for no reason. Removing it makes subsequent commits where we\n> refactor code surrounding verify_lock() (which may use errno) easier\n> to reason about.\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index a4e9344ac8b..abed0572fea 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -880,9 +880,7 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,\n> mustexist ? RESOLVE_REF_READING : 0,\n> &lock->old_oid, NULL)) {\n> if (old_oid) {\n> - int save_errno = errno;\n> strbuf_addf(err, \"can't verify ref '%s'\", lock->ref_name);\n> - errno = save_errno;\n\nyour message talks about EBUSY, and that change looks good, but how\ndoes it relate to this change?\n\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429879","messageId":"CAFQ2z_MNEb=tb+iVjE3n33Lq8tfKNKnaMOvxm6z0cwXChbuSQA@mail.gmail.com","threadId":"55589","inReplyTo":"patch-15.17-446b50280be-20210711T162803Z-avarab@gmail.com","subject":"Re: [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() not set errno","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-13T08:13:49Z","receivedAt":"2021-07-13T08:14:04Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason\n wrote:\n> + refs = get_main_ref_store(the_repository);\n> + head = refs_resolve_ref_unsafe_with_errno(refs, \"HEAD\", 0, NULL, NULL,\n> + &resolve_errno);\n> + if (!head) {\n> + errno = resolve_errno;\n\nThis adds another place where we'd use global errno to communicate.\nIsn't there a die() function that takes an errno value explicitly?\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429881","messageId":"CAFQ2z_M7FzR6HEea2Xj-j=LiTsjQvpGJc+h+D+GgU=ZEkWm50A@mail.gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"Re: [PATCH v6? 00/17] refs API: get rid of errno setting entirely","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-13T08:28:38Z","receivedAt":"2021-07-13T08:28:53Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> After starting a review of v5 of Han-Wen's where I was coming up with\n> squashes-on-top I continued with it and saw if I could get rid of\n> errno setting entirely in refs.\n\nthanks for taking this further! Is there a place where I can pull this\ncode (applying patches is bothersome).\n\nIt adds resolve_refs_unsafe_with_errno() to the public API in refs.h,\nand I think it has an ugly signature, but I suppose it's better than\nhaving the output be implicit through the global errno variable.\n\nI read over your patches, and they seem OK to me, modulo small\ncomments I posted.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"429918","messageId":"87h7gy2hn1.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"CAFQ2z_M7FzR6HEea2Xj-j=LiTsjQvpGJc+h+D+GgU=ZEkWm50A@mail.gmail.com","subject":"Re: [PATCH v6? 00/17] refs API: get rid of errno setting entirely","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-13T18:26:06Z","receivedAt":"2021-07-13T18:28:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jul 13 2021, Han-Wen Nienhuys wrote:\n\n> On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason\n> wrote:\n>>\n>> After starting a review of v5 of Han-Wen's where I was coming up with\n>> squashes-on-top I continued with it and saw if I could get rid of\n>> errno setting entirely in refs.\n>\n> thanks for taking this further! Is there a place where I can pull this\n> code (applying patches is bothersome).\n\nI've got it pushed to avar-review/pr-git-1012/hanwen/einval-sideband-v5\nat https://github.com/avar/git.git :\nhttps://github.com/avar/git/tree/avar-review/pr-git-1012%2Fhanwen%2Feinval-sideband-v5\n\n> It adds resolve_refs_unsafe_with_errno() to the public API in refs.h,\n> and I think it has an ugly signature, but I suppose it's better than\n> having the output be implicit through the global errno variable.\n\nYeah it is a bit ugly, perhaps we should just end up using the old name\nonce there's no callers of the old one?\n\n> I read over your patches, and they seem OK to me, modulo small\n> comments I posted.\n\nThanks, don't have time to look it over in detail now, will reply later.\n"},{"id":"430000","messageId":"87o8b51eea.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"CAFQ2z_MNEb=tb+iVjE3n33Lq8tfKNKnaMOvxm6z0cwXChbuSQA@mail.gmail.com","subject":"Re: [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() not set errno","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T08:32:38Z","receivedAt":"2021-07-14T08:35:40Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jul 13 2021, Han-Wen Nienhuys wrote:\n\n> On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason\n> wrote:\n>> + refs = get_main_ref_store(the_repository);\n>> + head = refs_resolve_ref_unsafe_with_errno(refs, \"HEAD\", 0, NULL, NULL,\n>> + &resolve_errno);\n>> + if (!head) {\n>> + errno = resolve_errno;\n>\n> This adds another place where we'd use global errno to communicate.\n> Isn't there a die() function that takes an errno value explicitly?\n\nNo, not that I know of. I suppose we could refactor fmt_with_err() in\nusage.c to somehow be exposed and add e.g. a:\n\n die_saved_errno(&saved_errno, \"msg %s\", ....);\n\nBut I really don't see the point.\n\nThe trouble with using errno is generally introducing undesired action\nat a distance, here we're setting it immediately before calling\ndie_errno(), so there's not much action or distance we should/could be\nworried about.\n\nE.g. xmkstemp() in wrapper.c uses this pattern, i.e.:\n\n errno = saved_errno;\n die_errno(...)\n"},{"id":"430002","messageId":"87lf691e6t.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"87h7gy2hn1.fsf@evledraar.gmail.com","subject":"Re: [PATCH v6? 00/17] refs API: get rid of errno setting entirely","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T08:38:06Z","receivedAt":"2021-07-14T08:40:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jul 13 2021, Ævar Arnfjörð Bjarmason wrote:\n\n> On Tue, Jul 13 2021, Han-Wen Nienhuys wrote:\n>\n>> On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason\n>> wrote:\n>>>\n>>> After starting a review of v5 of Han-Wen's where I was coming up with\n>>> squashes-on-top I continued with it and saw if I could get rid of\n>>> errno setting entirely in refs.\n>>\n>> thanks for taking this further! Is there a place where I can pull this\n>> code (applying patches is bothersome).\n>\n> I've got it pushed to avar-review/pr-git-1012/hanwen/einval-sideband-v5\n> at https://github.com/avar/git.git :\n> https://github.com/avar/git/tree/avar-review/pr-git-1012%2Fhanwen%2Feinval-sideband-v5\n>\n>> It adds resolve_refs_unsafe_with_errno() to the public API in refs.h,\n>> and I think it has an ugly signature, but I suppose it's better than\n>> having the output be implicit through the global errno variable.\n>\n> Yeah it is a bit ugly, perhaps we should just end up using the old name\n> once there's no callers of the old one?\n\nI looked at doing this, I don't think it's worth it.\n\nFor my additional patches on top we need to somehow migrate all callers\nincrementally, so we could\ns/refs_resolve_ref_unsafe_with_errno/refs_resolve_ref_unsafe/g at the\nend, but it just seems like a lot more churn, and for the patches to be\nself-contained I'd need to re-indent all the argument lists again.\n\nIt's a relatively obscure & scary API. I think it's OK that it's got a\nbit of a longer/more verbose name as a side-effect of the migration.\n\nAnd in any case with reftable I suspect that we'll end up refactoring\nfurther down the line sooner than later, i.e. to pass a \"struct strbuf\n*err\" or something, so getting it 100% right here seems premature.\n"},{"id":"430004","messageId":"CAFQ2z_NcBZS7w2YQ0CCT-0ffHdjk229OcLc5Zg4v7Tej62CGiA@mail.gmail.com","threadId":"55589","inReplyTo":"87lf691e6t.fsf@evledraar.gmail.com","subject":"Re: [PATCH v6? 00/17] refs API: get rid of errno setting entirely","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-07-14T08:43:44Z","receivedAt":"2021-07-14T08:43:58Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Wed, Jul 14, 2021 at 10:40 AM Ævar Arnfjörð Bjarmason\n wrote:\n> >> It adds resolve_refs_unsafe_with_errno() to the public API in refs.h,\n> >> and I think it has an ugly signature, but I suppose it's better than\n> >> having the output be implicit through the global errno variable.\n> >\n> > Yeah it is a bit ugly, perhaps we should just end up using the old name\n> > once there's no callers of the old one?\n>\n> I looked at doing this, I don't think it's worth it.\n..\n> And in any case with reftable I suspect that we'll end up refactoring\n> further down the line sooner than later, i.e. to pass a \"struct strbuf\n> *err\" or something, so getting it 100% right here seems premature.\n\nSGTM.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"430015","messageId":"cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com","subject":"[PATCH v7 0/6] refs: cleanup errno sideband ref related functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T11:43:28Z","receivedAt":"2021-07-14T11:43:41Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Per the discussion & feedback on the \"v6?\" at [1] this is a version of\nHan-Wen's patches which:\n\n A. Goes on top of my just-submitted one-patch\n https://lore.kernel.org/git/patch-1.1-de0838fe99-20210714T111351Z-avarab@gmail.com\n being able to remove that code in lock_ref_oid_basic() makes\n things simpler, the BUG() I added to that branch in my \"v6?\" is\n gone.\n\n That change simplifies this series quite a bit, more than one\n patch in this one dealt with that now-removed codepath.\n\n B. Rebases this on top of that\n\n C. I ejected my long series of RFC-esque patches at the end.\n\n Per [2] it looks like Han-Wen is happy to have some version of my\n re-roll picked up, but I hardly think it's fair that I inflate\n other people's serieses with a bunch of fixes on top, I cause\n myself enough trouble with that as it is :)\n\n The value of those patches of mine on top is to assert that the\n changes in this series are correct.\n\n But I think to just get this in it's enough that that work exists\n and that we did the full errno conversion and could demonstrate\n that there weren't bugs as a result of it.\n\n For now in 04/06 here we leave the \"For unmigrated legacy callers\"\n assignment of \"errno\" in refs_resolve_ref_unsafe().\n\n I'll just submit that cleanup series on top of this once this\n lands.\n\n1. http://lore.kernel.org/git/cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com\n2. http://lore.kernel.org/git/CAFQ2z_P=9Suh0kO6E44hUOyNFOAKcmcUg_x6AQFP9jhHrBA6RQ@mail.gmail.com\n\nHan-Wen Nienhuys (6):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: explicitly return failure_errno from parse_loose_ref_contents\n refs: make errno output explicit for refs_resolve_ref_unsafe\n\n refs.c | 69 ++++++++++++++++++++----------\n refs.h | 11 +++++\n refs/debug.c | 4 +-\n refs/files-backend.c | 98 +++++++++++++++++++++++--------------------\n refs/packed-backend.c | 15 +++----\n refs/refs-internal.h | 32 ++++++++------\n 6 files changed, 141 insertions(+), 88 deletions(-)\n\nRange-diff against v6:\n 1: 57517368c3 = 1: 4beba4443a refs: remove EINVAL errno output from specification of read_raw_ref_fn\n 2: 61cf761147 ! 2: fd8e356185 refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n - \t\t * to remain.\n - \t\t */\n - \t\tif (remove_empty_directories(&ref_file)) {\n --\t\t\tlast_errno = errno;\n - \t\t\tif (!refs_verify_refname_available(\n - \t\t\t\t\t &refs->base,\n - \t\t\t\t\t refname, extras, skip, err))\n -@@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n - \t\t\t\t\t\t refname, resolve_flags,\n - \t\t\t\t\t\t &lock->old_oid, type);\n - \t}\n + \tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n + \t\t\t\t\t refname, resolve_flags,\n + \t\t\t\t\t &lock->old_oid, type);\n -\tif (!resolved) {\n -\t\tlast_errno = errno;\n -\t\tif (last_errno != ENOTDIR ||\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n -\t\t\t\t\t\t extras, skip, err))\n -\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n -\t\t\t\t refname, strerror(last_errno));\n --\n +\tif (!resolved &&\n +\t (errno != ENOTDIR ||\n +\t /* in case of D/F conflict, try to generate a better error\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n +\t\t\t\t\t skip, err))) {\n +\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n +\t\t\t refname, strerror(errno));\n + \n \t\tgoto error_return;\n \t}\n - \n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n 3: a4e5f2d02f = 3: 913e72e9ff refs: make errno output explicit for read_raw_ref_fn\n 4: 270cda29c3 = 4: dd191768f6 refs: add failure_errno to refs_read_raw_ref() signature\n 5: f5197cdc0b = 5: 77c53dc0d9 refs: explicitly return failure_errno from parse_loose_ref_contents\n 6: 96689e523f ! 6: 5d4b1b4ddf refs: make errno output explicit for refs_resolve_ref_unsafe\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n -\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n -\t\t\t\t\t refname, resolve_flags,\n -\t\t\t\t\t &lock->old_oid, type);\n --\tif (!resolved && errno == EISDIR) {\n +\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n +\t\t\t\t\t\t\tresolve_flags,\n +\t\t\t\t\t\t\t&lock->old_oid, type,\n +\t\t\t\t\t\t\t&resolve_errno);\n -+\tif (!resolved && resolve_errno == EISDIR) {\n - \t\t/*\n - \t\t * we are trying to lock foo but we used to\n - \t\t * have foo/bar which now does not exist;\n -@@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n - \t\t\t\t\t\t &lock->old_oid, type);\n - \t}\n \tif (!resolved &&\n -\t (errno != ENOTDIR ||\n +\t (resolve_errno != ENOTDIR ||\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \t\t\t\t\t skip, err))) {\n \t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n -\t\t\t refname, strerror(errno));\n +-\n +\t\t\t refname, strerror(resolve_errno));\n \t\tgoto error_return;\n \t}\n 7: 10a40c9244 < -: ---------- refs: make errno ignoring explicit in lock_ref_oid_basic()\n 8: ff38a3f193 < -: ---------- refs file-backend.c: stop setting \"EBUSY\" in verify_lock()\n 9: cdec272f06 < -: ---------- refs file-backend.c: deal with errno directly in verify_lock()\n10: bd0639945a < -: ---------- refs API: remove refs_read_ref_full() wrapper\n11: a359d1533b < -: ---------- refs API: make resolve_gitlink_ref() not set errno\n12: 07d550015a < -: ---------- refs API: make refs_resolve_ref_unsafe() static\n13: 73b7049151 < -: ---------- refs API: make refs_resolve_refdup() not set errno\n14: 1e9de48d85 < -: ---------- refs API: make refs_ref_exists() not set errno\n15: 446b50280b < -: ---------- refs API: make resolve_ref_unsafe() not set errno\n16: 37c5b88d7d < -: ---------- refs API: make expand_ref() and repo_dwim_log() not set errno\n17: f914df0bb2 < -: ---------- refs API: don't leak \"errno\" in run_transaction_hook()\n-- \n2.32.0.851.g0fc62a9785\n\n"},{"id":"430016","messageId":"patch-1.6-4beba4443a-20210714T114301Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com","subject":"[PATCH v7 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T11:43:29Z","receivedAt":"2021-07-14T11:43:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 467f4b3c93..f4445e3290 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \n2.32.0.851.g0fc62a9785\n\n"},{"id":"430017","messageId":"patch-2.6-fd8e356185-20210714T114301Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com","subject":"[PATCH v7 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T11:43:30Z","receivedAt":"2021-07-14T11:43:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nIn the case of the \"errno != ENOTDIR\" case that originates in 5b2d8d6f218\n(lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,\n2015-05-11), there the \"last_errno\" was saved away to return it from\nlock_ref_oid_basic(), now that we're no longer doing that we can skip\nthat entirely and use \"errno\" directly. A follow-up change will\nextract the specific errno we want earlier in this function.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 22 +++++++++-------------\n 1 file changed, 9 insertions(+), 13 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 7e4963fd07..45f198c3fc 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n@@ -941,13 +939,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n \t\t\t\t\t refname, resolve_flags,\n \t\t\t\t\t &lock->old_oid, type);\n-\tif (!resolved) {\n-\t\tlast_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n-\t\t !refs_verify_refname_available(&refs->base, refname,\n-\t\t\t\t\t\t extras, skip, err))\n-\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n+\tif (!resolved &&\n+\t (errno != ENOTDIR ||\n+\t /* in case of D/F conflict, try to generate a better error\n+\t * message. If that fails, fall back to strerror(ENOTDIR).\n+\t */\n+\t !refs_verify_refname_available(&refs->base, refname, extras,\n+\t\t\t\t\t skip, err))) {\n+\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n+\t\t\t refname, strerror(errno));\n \n \t\tgoto error_return;\n \t}\n@@ -961,20 +961,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n \t\t\t\t\t extras, skip, err)) {\n-\t\tlast_errno = ENOTDIR;\n \t\tgoto error_return;\n \t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n \n \tif (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {\n-\t\tlast_errno = errno;\n \t\tgoto error_return;\n \t}\n \tgoto out;\n@@ -985,7 +982,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \n2.32.0.851.g0fc62a9785\n\n"},{"id":"430018","messageId":"patch-3.6-913e72e9ff-20210714T114301Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com","subject":"[PATCH v7 3/6] refs: make errno output explicit for read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T11:43:31Z","receivedAt":"2021-07-14T11:43:44Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes it explicit how alternative ref backends should report errors in\nread_raw_ref_fn.\n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs/debug.c | 4 ++--\n refs/files-backend.c | 29 +++++++++++++++--------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 20 ++++++++++++--------\n 5 files changed, 34 insertions(+), 29 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 8c9490235e..7b3f05a66f 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1681,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, &errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 7db4abccc3..f12413a9bc 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \toidcpy(oid, null_oid());\n \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 45f198c3fc..c4b5b70c53 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -611,7 +610,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tif (hold_lock_file_for_update_timeout(\n \t\t\t &lock->lk, ref_file.buf, LOCK_NO_DEREF,\n \t\t\t get_files_ref_lock_timeout_ms()) < 0) {\n-\t\tif (errno == ENOENT && --attempts_remaining > 0) {\n+\t\tint myerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr == ENOENT && --attempts_remaining > 0) {\n \t\t\t/*\n \t\t\t * Maybe somebody just deleted one of the\n \t\t\t * directories leading to ref_file. Try\n@@ -619,7 +620,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t */\n \t\t\tgoto retry;\n \t\t} else {\n-\t\t\tunable_to_lock_message(ref_file.buf, errno, err);\n+\t\t\tunable_to_lock_message(ref_file.buf, myerr, err);\n \t\t\tgoto error_return;\n \t\t}\n \t}\n@@ -629,9 +630,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -655,7 +656,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -693,13 +694,13 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex dfecdbc1db..a457f18e93 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex f4445e3290..79dfb3af48 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n+ * type, and return -1 (failure_errno should not be ENOENT)\n+ *\n+ * failure_errno provides errno codes that are interpreted beyond error\n+ * reporting. The following error codes have special meaning:\n+ * * ENOENT: the ref doesn't exist\n+ * * EISDIR: ref name is a directory\n+ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \n2.32.0.851.g0fc62a9785\n\n"},{"id":"430019","messageId":"patch-4.6-dd191768f6-20210714T114301Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com","subject":"[PATCH v7 4/6] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T11:43:32Z","receivedAt":"2021-07-14T11:43:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n\nSome of our callers explicitly do not care about the errno, rather\nthan understanding NULL let's have them declare that they don't care\nby passing in an \"ignore_errno\". There's only three of them, and using\nthat pattern will make it more obvious that they want to throw away\ndata, let's also add a comment to one of the callers about why we'd\nlike to ignore the errno.\n\nLet's not extend that to refs_resolve_ref_unsafe() itself for now, it\nhas a large set of legacy callers, so we're faking up the old \"errno\"\nbehavior for it. We can convert those callers to\nrefs_resolve_ref_unsafe_with_errno() later.\n\nWe are leaving out out the refs_read_special_head() in\nrefs_read_raw_ref() for now, as noted in the next commit moving it to\n\"failure_errno\" will require some special consideration.\n\nWe're intentionally mis-indenting the argument list of the new\nrefs_resolve_ref_unsafe_with_errno(), it will be non-static in a\nsubsequent commit, doing it this way makes that diff smaller.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 61 ++++++++++++++++++++++++++++++-------------\n refs/files-backend.c | 10 ++++---\n refs/packed-backend.c | 7 ++---\n refs/refs-internal.h | 6 ++---\n 4 files changed, 56 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 7b3f05a66f..406e27213b 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1671,30 +1671,33 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n+\tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &errno);\n+\t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n \tint symref_count;\n \n+\tassert(failure_errno);\n+\n \tif (!oid)\n \t\toid = &unused_oid;\n \tif (!flags)\n@@ -1705,7 +1708,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n+\t\t\t*failure_errno = EINVAL;\n \t\t\treturn NULL;\n \t\t}\n \n@@ -1723,8 +1726,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, failure_errno)) {\n \t\t\t*flags |= read_flags;\n \n \t\t\t/* In reading mode, refs must eventually resolve */\n@@ -1736,9 +1739,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (*failure_errno != ENOENT &&\n+\t\t\t *failure_errno != EISDIR &&\n+\t\t\t *failure_errno != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -1765,7 +1768,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t\t !refname_is_safe(refname)) {\n-\t\t\t\terrno = EINVAL;\n+\t\t\t\t*failure_errno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \n@@ -1773,10 +1776,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t}\n \t}\n \n-\terrno = ELOOP;\n+\t*failure_errno = ELOOP;\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tint failure_errno = 0;\n+\tconst char *refn;\n+\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &failure_errno);\n+\tif (!refn)\n+\t\t/* For unmigrated legacy callers */\n+\t\terrno = failure_errno;\n+\treturn refn;\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n@@ -2227,6 +2244,13 @@ int refs_verify_refname_available(struct ref_store *refs,\n \n \tstrbuf_grow(&dirname, strlen(refname) + 1);\n \tfor (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {\n+\t\t/*\n+\t\t * Just saying \"Is a directory\" when we e.g. can't\n+\t\t * lock some multi-level ref isn't very informative,\n+\t\t * the user won't be told *what* is a directory, so\n+\t\t * let's not use strerror() below.\n+\t\t */\n+\t\tint ignore_errno;\n \t\t/* Expand dirname to the new prefix, not including the trailing slash: */\n \t\tstrbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);\n \n@@ -2238,7 +2262,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, &ignore_errno)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c4b5b70c53..3eef442c24 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\tgoto out;\n \n \tif (lstat(path, &st) < 0) {\n+\t\tint ignore_errno;\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \t/* Is it a directory? */\n \tif (S_ISDIR(st.st_mode)) {\n+\t\tint ignore_errno;\n \t\t/*\n \t\t * Even though there is a directory where the loose\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a457f18e93..928e5d2a99 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure_errno;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure_errno) ||\n+\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 79dfb3af48..54f57c6a2d 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /*\n * Write an error to `err` and return a nonzero value iff the same\n-- \n2.32.0.851.g0fc62a9785\n\n"},{"id":"430020","messageId":"patch-5.6-77c53dc0d9-20210714T114301Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com","subject":"[PATCH v7 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T11:43:33Z","receivedAt":"2021-07-14T11:43:49Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe EINVAL error from parse_loose_ref_contents is used in files-backend\nto create a custom error message.\n\nIn untangling this we discovered a tricky edge case. The\nrefs_read_special_head() function was relying on\nparse_loose_ref_contents() setting EINVAL.\n\nBy converting it to use \"saved_errno\" we can migrate away from \"errno\"\nin this part of the code entirely, and do away with an existing\n\"save_errno\" pattern, its only purpose was to not clobber the \"errno\"\nwe previously needed at the end of files_read_raw_ref().\n\nLet's assert that we can do that by not having files_read_raw_ref()\nitself operate on *failure_errno in addition to passing it on. Instead\nwe'll assert that if we return non-zero we actually do set errno, thus\nassuring ourselves and callers that they can trust the resulting\n\"failure_errno\".\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 8 +++++---\n refs/files-backend.c | 30 +++++++++++++++++++-----------\n refs/refs-internal.h | 6 ++++--\n 3 files changed, 28 insertions(+), 16 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 406e27213b..af01a692cb 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n \n static int refs_read_special_head(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t\t int *failure_errno)\n {\n \tstruct strbuf full_path = STRBUF_INIT;\n \tstruct strbuf content = STRBUF_INIT;\n@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n \t\tgoto done;\n \n-\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n+\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n+\t\t\t\t\t failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\n@@ -1678,7 +1680,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 3eef442c24..e6b1cd9ff7 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tint fd;\n \tint ret = -1;\n \tint remaining_retries = 3;\n+\tint myerr = 0;\n \n \t*type = 0;\n \tstrbuf_reset(&sb_path);\n@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (lstat(path, &st) < 0) {\n \t\tint ignore_errno;\n-\t\tif (errno != ENOENT)\n+\t\tmyerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr != ENOENT)\n \t\t\tgoto out;\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = ENOENT;\n+\t\t\tmyerr = ENOENT;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tif (S_ISLNK(st.st_mode)) {\n \t\tstrbuf_reset(&sb_contents);\n \t\tif (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {\n-\t\t\tif (errno == ENOENT || errno == EINVAL)\n+\t\t\tmyerr = errno;\n+\t\t\terrno = 0;\n+\t\t\tif (myerr == ENOENT || myerr == EINVAL)\n \t\t\t\t/* inconsistent with lstat; retry */\n \t\t\t\tgoto stat_ref;\n \t\t\telse\n@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t */\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = EISDIR;\n+\t\t\tmyerr = EISDIR;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t */\n \tfd = open(path, O_RDONLY);\n \tif (fd < 0) {\n-\t\tif (errno == ENOENT && !S_ISLNK(st.st_mode))\n+\t\tmyerr = errno;\n+\t\tif (myerr == ENOENT && !S_ISLNK(st.st_mode))\n \t\t\t/* inconsistent with lstat; retry */\n \t\t\tgoto stat_ref;\n \t\telse\n@@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t}\n \tstrbuf_reset(&sb_contents);\n \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n-\t\tint save_errno = errno;\n \t\tclose(fd);\n-\t\terrno = save_errno;\n \t\tgoto out;\n \t}\n \tclose(fd);\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n-\tret = parse_loose_ref_contents(buf, oid, referent, type);\n+\tret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);\n \n out:\n-\t*failure_errno = errno;\n+\tif (ret && !myerr)\n+\t\tBUG(\"returning non-zero %d, should have set myerr!\", ret);\n+\t*failure_errno = myerr;\n+\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n \treturn ret;\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -486,7 +494,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n \tif (parse_oid_hex(buf, oid, &p) ||\n \t (*p != '\\0' && !isspace(*p))) {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n \treturn 0;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 54f57c6a2d..bf581e70cf 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -689,10 +689,12 @@ struct ref_store {\n };\n \n /*\n- * Parse contents of a loose ref file.\n+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for\n+ * invalid contents.\n */\n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\n-- \n2.32.0.851.g0fc62a9785\n\n"},{"id":"430021","messageId":"patch-6.6-5d4b1b4ddf-20210714T114301Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com","subject":"[PATCH v7 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-14T11:43:34Z","receivedAt":"2021-07-14T11:43:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions, so update that caller\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs.h | 11 +++++++++++\n refs/files-backend.c | 13 +++++++------\n 3 files changed, 19 insertions(+), 7 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex af01a692cb..1dacb5fe27 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1687,7 +1687,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t\t\t type, failure_errno);\n }\n \n-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n \t\t\t\t\t const char *refname,\n \t\t\t\t\t int resolve_flags,\n \t\t\t\t\t struct object_id *oid,\ndiff --git a/refs.h b/refs.h\nindex 48970dfc7e..344a3c51a8 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t\t int resolve_flags,\n \t\t\t\t struct object_id *oid,\n \t\t\t\t int *flags);\n+/**\n+ * refs_resolve_ref_unsafe_with_errno() is like\n+ * refs_resolve_ref_unsafe(), but provide access to errno code that\n+ * lead to a failure. We guarantee that errno is set to a meaningful\n+ * value on non-zero return.\n+ */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags);\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex e6b1cd9ff7..5acd636ba9 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -935,6 +935,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tint mustexist = (old_oid && !is_null_oid(old_oid));\n \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n \tint resolved;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -947,19 +948,19 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n-\t\t\t\t\t refname, resolve_flags,\n-\t\t\t\t\t &lock->old_oid, type);\n+\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\t\tresolve_flags,\n+\t\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t\t&resolve_errno);\n \tif (!resolved &&\n-\t (errno != ENOTDIR ||\n+\t (resolve_errno != ENOTDIR ||\n \t /* in case of D/F conflict, try to generate a better error\n \t * message. If that fails, fall back to strerror(ENOTDIR).\n \t */\n \t !refs_verify_refname_available(&refs->base, refname, extras,\n \t\t\t\t\t skip, err))) {\n \t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t refname, strerror(errno));\n-\n+\t\t\t refname, strerror(resolve_errno));\n \t\tgoto error_return;\n \t}\n \n-- \n2.32.0.851.g0fc62a9785\n\n"},{"id":"430331","messageId":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com","subject":"[PATCH v8 0/7] refs: cleanup errno sideband ref related functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-16T14:22:25Z","receivedAt":"2021-07-16T14:22:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"A v8 re-roll of v7 of this, covered in detail at:\nhttps://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/\n\nThis topic relies on the now-re-rolled and bigger dependency topic to\ncleanup the use of the reflog API, which is 1/4 callers (and the\ncomplex one) that need an API being simplified here, see\nhttps://lore.kernel.org/git/cover-00.11-00000000000-20210716T140631Z-avarab@gmail.com\n\nHan-Wen Nienhuys (6):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: explicitly return failure_errno from parse_loose_ref_contents\n refs: make errno output explicit for refs_resolve_ref_unsafe\n\nÆvar Arnfjörð Bjarmason (1):\n refs file backend: move raceproof_create_file() here\n\n cache.h | 43 ---------\n object-file.c | 68 --------------\n refs.c | 69 +++++++++-----\n refs.h | 11 +++\n refs/debug.c | 4 +-\n refs/files-backend.c | 207 +++++++++++++++++++++++++++++++++---------\n refs/packed-backend.c | 15 +--\n refs/refs-internal.h | 32 ++++---\n 8 files changed, 250 insertions(+), 199 deletions(-)\n\nRange-diff against v7:\n-: ----------- > 1: ce1ca2cf30f refs file backend: move raceproof_create_file() here\n1: 4beba4443ab = 2: 2a69bbea821 refs: remove EINVAL errno output from specification of read_raw_ref_fn\n2: fd8e3561851 ! 3: a3f80c6d2f7 refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n -\tint last_errno = 0;\n - \tint mustexist = (old_oid && !is_null_oid(old_oid));\n - \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n - \tint resolved;\n + \n + \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n + \tassert(err);\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n - \tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n - \t\t\t\t\t refname, resolve_flags,\n - \t\t\t\t\t &lock->old_oid, type);\n --\tif (!resolved) {\n + \tfiles_ref_path(refs, &ref_file, refname);\n + \tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n + \t\t\t\t RESOLVE_REF_NO_RECURSE,\n +-\t\t\t\t &lock->old_oid, type)) {\n -\t\tlast_errno = errno;\n -\t\tif (last_errno != ENOTDIR ||\n -\t\t !refs_verify_refname_available(&refs->base, refname,\n --\t\t\t\t\t\t extras, skip, err))\n +-\t\t\t\t\t\t NULL, NULL, err))\n -\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n -\t\t\t\t refname, strerror(last_errno));\n -+\tif (!resolved &&\n +-\n ++\t\t\t\t &lock->old_oid, type) &&\n +\t (errno != ENOTDIR ||\n +\t /* in case of D/F conflict, try to generate a better error\n +\t * message. If that fails, fall back to strerror(ENOTDIR).\n +\t */\n -+\t !refs_verify_refname_available(&refs->base, refname, extras,\n -+\t\t\t\t\t skip, err))) {\n ++\t !refs_verify_refname_available(&refs->base, refname, NULL,\n ++\t\t\t\t\t NULL, err))) {\n +\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n +\t\t\t refname, strerror(errno));\n - \n \t\tgoto error_return;\n \t}\n + \n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n + \t */\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n - \t\t\t\t\t extras, skip, err)) {\n +-\t\t\t\t\t NULL, NULL, err)) {\n -\t\tlast_errno = ENOTDIR;\n ++\t\t\t\t\t NULL, NULL, err))\n \t\tgoto error_return;\n - \t}\n +-\t}\n \n \tlock->ref_name = xstrdup(refname);\n \n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n - \n - \tif (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {\n --\t\tlast_errno = errno;\n - \t\tgoto error_return;\n - \t}\n - \tgoto out;\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n3: 913e72e9ffd = 4: 147058c8c3d refs: make errno output explicit for read_raw_ref_fn\n4: dd191768f66 = 5: b42a7474f18 refs: add failure_errno to refs_read_raw_ref() signature\n5: 77c53dc0d93 = 6: 93b770c8bea refs: explicitly return failure_errno from parse_loose_ref_contents\n6: 5d4b1b4ddff ! 7: cb32b5c0526 refs: make errno output explicit for refs_resolve_ref_unsafe\n @@ refs.h: const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n - \tint mustexist = (old_oid && !is_null_oid(old_oid));\n - \tint resolve_flags = RESOLVE_REF_NO_RECURSE;\n - \tint resolved;\n + {\n + \tstruct strbuf ref_file = STRBUF_INIT;\n + \tstruct ref_lock *lock;\n +\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n - \t\tresolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;\n + \tCALLOC_ARRAY(lock, 1);\n \n \tfiles_ref_path(refs, &ref_file, refname);\n --\tresolved = !!refs_resolve_ref_unsafe(&refs->base,\n --\t\t\t\t\t refname, resolve_flags,\n --\t\t\t\t\t &lock->old_oid, type);\n -+\tresolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n -+\t\t\t\t\t\t\tresolve_flags,\n -+\t\t\t\t\t\t\t&lock->old_oid, type,\n -+\t\t\t\t\t\t\t&resolve_errno);\n - \tif (!resolved &&\n +-\tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n +-\t\t\t\t RESOLVE_REF_NO_RECURSE,\n +-\t\t\t\t &lock->old_oid, type) &&\n -\t (errno != ENOTDIR ||\n ++\tif (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n ++\t\t\t\t\t\tRESOLVE_REF_NO_RECURSE,\n ++\t\t\t\t\t\t&lock->old_oid, type,\n ++\t\t\t\t\t\t&resolve_errno) &&\n +\t (resolve_errno != ENOTDIR ||\n \t /* in case of D/F conflict, try to generate a better error\n \t * message. If that fails, fall back to strerror(ENOTDIR).\n \t */\n - \t !refs_verify_refname_available(&refs->base, refname, extras,\n - \t\t\t\t\t skip, err))) {\n + \t !refs_verify_refname_available(&refs->base, refname, NULL,\n + \t\t\t\t\t NULL, err))) {\n \t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n -\t\t\t refname, strerror(errno));\n --\n +\t\t\t refname, strerror(resolve_errno));\n \t\tgoto error_return;\n \t}\n-- \n2.32.0.874.gfa1990a4f10\n\n"},{"id":"430332","messageId":"patch-1.7-ce1ca2cf30f-20210716T142032Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","subject":"[PATCH v8 1/7] refs file backend: move raceproof_create_file() here","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-16T14:22:26Z","receivedAt":"2021-07-16T14:22:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the raceproof_create_file() API added to cache.h and\nobject-file.c in 177978f56ad (raceproof_create_file(): new function,\n2017-01-06) to its only user, refs/files-backend.c.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n cache.h | 43 -----------------\n object-file.c | 68 ---------------------------\n refs/files-backend.c | 109 +++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 109 insertions(+), 111 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex ba04ff8bd36..eb4dfe6381f 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1202,49 +1202,6 @@ enum scld_error safe_create_leading_directories(char *path);\n enum scld_error safe_create_leading_directories_const(const char *path);\n enum scld_error safe_create_leading_directories_no_share(char *path);\n \n-/*\n- * Callback function for raceproof_create_file(). This function is\n- * expected to do something that makes dirname(path) permanent despite\n- * the fact that other processes might be cleaning up empty\n- * directories at the same time. Usually it will create a file named\n- * path, but alternatively it could create another file in that\n- * directory, or even chdir() into that directory. The function should\n- * return 0 if the action was completed successfully. On error, it\n- * should return a nonzero result and set errno.\n- * raceproof_create_file() treats two errno values specially:\n- *\n- * - ENOENT -- dirname(path) does not exist. In this case,\n- * raceproof_create_file() tries creating dirname(path)\n- * (and any parent directories, if necessary) and calls\n- * the function again.\n- *\n- * - EISDIR -- the file already exists and is a directory. In this\n- * case, raceproof_create_file() removes the directory if\n- * it is empty (and recursively any empty directories that\n- * it contains) and calls the function again.\n- *\n- * Any other errno causes raceproof_create_file() to fail with the\n- * callback's return value and errno.\n- *\n- * Obviously, this function should be OK with being called again if it\n- * fails with ENOENT or EISDIR. In other scenarios it will not be\n- * called again.\n- */\n-typedef int create_file_fn(const char *path, void *cb);\n-\n-/*\n- * Create a file in dirname(path) by calling fn, creating leading\n- * directories if necessary. Retry a few times in case we are racing\n- * with another process that is trying to clean up the directory that\n- * contains path. See the documentation for create_file_fn for more\n- * details.\n- *\n- * Return the value and set the errno that resulted from the most\n- * recent call of fn. fn is always called at least once, and will be\n- * called more than once if it returns ENOENT or EISDIR.\n- */\n-int raceproof_create_file(const char *path, create_file_fn fn, void *cb);\n-\n int mkdir_in_gitdir(const char *path);\n char *expand_user_path(const char *path, int real_home);\n const char *enter_repo(const char *path, int strict);\ndiff --git a/object-file.c b/object-file.c\nindex f233b440b22..ae766e18b4e 100644\n--- a/object-file.c\n+++ b/object-file.c\n@@ -414,74 +414,6 @@ enum scld_error safe_create_leading_directories_const(const char *path)\n \treturn result;\n }\n \n-int raceproof_create_file(const char *path, create_file_fn fn, void *cb)\n-{\n-\t/*\n-\t * The number of times we will try to remove empty directories\n-\t * in the way of path. This is only 1 because if another\n-\t * process is racily creating directories that conflict with\n-\t * us, we don't want to fight against them.\n-\t */\n-\tint remove_directories_remaining = 1;\n-\n-\t/*\n-\t * The number of times that we will try to create the\n-\t * directories containing path. We are willing to attempt this\n-\t * more than once, because another process could be trying to\n-\t * clean up empty directories at the same time as we are\n-\t * trying to create them.\n-\t */\n-\tint create_directories_remaining = 3;\n-\n-\t/* A scratch copy of path, filled lazily if we need it: */\n-\tstruct strbuf path_copy = STRBUF_INIT;\n-\n-\tint ret, save_errno;\n-\n-\t/* Sanity check: */\n-\tassert(*path);\n-\n-retry_fn:\n-\tret = fn(path, cb);\n-\tsave_errno = errno;\n-\tif (!ret)\n-\t\tgoto out;\n-\n-\tif (errno == EISDIR && remove_directories_remaining-- > 0) {\n-\t\t/*\n-\t\t * A directory is in the way. Maybe it is empty; try\n-\t\t * to remove it:\n-\t\t */\n-\t\tif (!path_copy.len)\n-\t\t\tstrbuf_addstr(&path_copy, path);\n-\n-\t\tif (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))\n-\t\t\tgoto retry_fn;\n-\t} else if (errno == ENOENT && create_directories_remaining-- > 0) {\n-\t\t/*\n-\t\t * Maybe the containing directory didn't exist, or\n-\t\t * maybe it was just deleted by a process that is\n-\t\t * racing with us to clean up empty directories. Try\n-\t\t * to create it:\n-\t\t */\n-\t\tenum scld_error scld_result;\n-\n-\t\tif (!path_copy.len)\n-\t\t\tstrbuf_addstr(&path_copy, path);\n-\n-\t\tdo {\n-\t\t\tscld_result = safe_create_leading_directories(path_copy.buf);\n-\t\t\tif (scld_result == SCLD_OK)\n-\t\t\t\tgoto retry_fn;\n-\t\t} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);\n-\t}\n-\n-out:\n-\tstrbuf_release(&path_copy);\n-\terrno = save_errno;\n-\treturn ret;\n-}\n-\n static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)\n {\n \tint i;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex bd99bc570de..36af99af844 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -852,6 +852,115 @@ static struct ref_iterator *files_ref_iterator_begin(\n \treturn ref_iterator;\n }\n \n+/*\n+ * Callback function for raceproof_create_file(). This function is\n+ * expected to do something that makes dirname(path) permanent despite\n+ * the fact that other processes might be cleaning up empty\n+ * directories at the same time. Usually it will create a file named\n+ * path, but alternatively it could create another file in that\n+ * directory, or even chdir() into that directory. The function should\n+ * return 0 if the action was completed successfully. On error, it\n+ * should return a nonzero result and set errno.\n+ * raceproof_create_file() treats two errno values specially:\n+ *\n+ * - ENOENT -- dirname(path) does not exist. In this case,\n+ * raceproof_create_file() tries creating dirname(path)\n+ * (and any parent directories, if necessary) and calls\n+ * the function again.\n+ *\n+ * - EISDIR -- the file already exists and is a directory. In this\n+ * case, raceproof_create_file() removes the directory if\n+ * it is empty (and recursively any empty directories that\n+ * it contains) and calls the function again.\n+ *\n+ * Any other errno causes raceproof_create_file() to fail with the\n+ * callback's return value and errno.\n+ *\n+ * Obviously, this function should be OK with being called again if it\n+ * fails with ENOENT or EISDIR. In other scenarios it will not be\n+ * called again.\n+ */\n+typedef int create_file_fn(const char *path, void *cb);\n+\n+/*\n+ * Create a file in dirname(path) by calling fn, creating leading\n+ * directories if necessary. Retry a few times in case we are racing\n+ * with another process that is trying to clean up the directory that\n+ * contains path. See the documentation for create_file_fn for more\n+ * details.\n+ *\n+ * Return the value and set the errno that resulted from the most\n+ * recent call of fn. fn is always called at least once, and will be\n+ * called more than once if it returns ENOENT or EISDIR.\n+ */\n+static int raceproof_create_file(const char *path, create_file_fn fn, void *cb)\n+{\n+\t/*\n+\t * The number of times we will try to remove empty directories\n+\t * in the way of path. This is only 1 because if another\n+\t * process is racily creating directories that conflict with\n+\t * us, we don't want to fight against them.\n+\t */\n+\tint remove_directories_remaining = 1;\n+\n+\t/*\n+\t * The number of times that we will try to create the\n+\t * directories containing path. We are willing to attempt this\n+\t * more than once, because another process could be trying to\n+\t * clean up empty directories at the same time as we are\n+\t * trying to create them.\n+\t */\n+\tint create_directories_remaining = 3;\n+\n+\t/* A scratch copy of path, filled lazily if we need it: */\n+\tstruct strbuf path_copy = STRBUF_INIT;\n+\n+\tint ret, save_errno;\n+\n+\t/* Sanity check: */\n+\tassert(*path);\n+\n+retry_fn:\n+\tret = fn(path, cb);\n+\tsave_errno = errno;\n+\tif (!ret)\n+\t\tgoto out;\n+\n+\tif (errno == EISDIR && remove_directories_remaining-- > 0) {\n+\t\t/*\n+\t\t * A directory is in the way. Maybe it is empty; try\n+\t\t * to remove it:\n+\t\t */\n+\t\tif (!path_copy.len)\n+\t\t\tstrbuf_addstr(&path_copy, path);\n+\n+\t\tif (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))\n+\t\t\tgoto retry_fn;\n+\t} else if (errno == ENOENT && create_directories_remaining-- > 0) {\n+\t\t/*\n+\t\t * Maybe the containing directory didn't exist, or\n+\t\t * maybe it was just deleted by a process that is\n+\t\t * racing with us to clean up empty directories. Try\n+\t\t * to create it:\n+\t\t */\n+\t\tenum scld_error scld_result;\n+\n+\t\tif (!path_copy.len)\n+\t\t\tstrbuf_addstr(&path_copy, path);\n+\n+\t\tdo {\n+\t\t\tscld_result = safe_create_leading_directories(path_copy.buf);\n+\t\t\tif (scld_result == SCLD_OK)\n+\t\t\t\tgoto retry_fn;\n+\t\t} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);\n+\t}\n+\n+out:\n+\tstrbuf_release(&path_copy);\n+\terrno = save_errno;\n+\treturn ret;\n+}\n+\n static int remove_empty_directories(struct strbuf *path)\n {\n \t/*\n-- \n2.32.0.874.gfa1990a4f10\n\n"},{"id":"430333","messageId":"patch-3.7-a3f80c6d2f7-20210716T142032Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","subject":"[PATCH v8 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-16T14:22:28Z","receivedAt":"2021-07-16T14:22:52Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nIn the case of the \"errno != ENOTDIR\" case that originates in 5b2d8d6f218\n(lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,\n2015-05-11), there the \"last_errno\" was saved away to return it from\nlock_ref_oid_basic(), now that we're no longer doing that we can skip\nthat entirely and use \"errno\" directly. A follow-up change will\nextract the specific errno we want earlier in this function.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 25 ++++++++++---------------\n 1 file changed, 10 insertions(+), 15 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 36af99af844..a6a9f2b99fa 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -982,7 +982,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -991,7 +990,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -1001,14 +999,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tfiles_ref_path(refs, &ref_file, refname);\n \tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n \t\t\t\t RESOLVE_REF_NO_RECURSE,\n-\t\t\t\t &lock->old_oid, type)) {\n-\t\tlast_errno = errno;\n-\t\tif (last_errno != ENOTDIR ||\n-\t\t !refs_verify_refname_available(&refs->base, refname,\n-\t\t\t\t\t\t NULL, NULL, err))\n-\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n-\n+\t\t\t\t &lock->old_oid, type) &&\n+\t (errno != ENOTDIR ||\n+\t /* in case of D/F conflict, try to generate a better error\n+\t * message. If that fails, fall back to strerror(ENOTDIR).\n+\t */\n+\t !refs_verify_refname_available(&refs->base, refname, NULL,\n+\t\t\t\t\t NULL, err))) {\n+\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n+\t\t\t refname, strerror(errno));\n \t\tgoto error_return;\n \t}\n \n@@ -1020,15 +1019,12 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t */\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n-\t\t\t\t\t NULL, NULL, err)) {\n-\t\tlast_errno = ENOTDIR;\n+\t\t\t\t\t NULL, NULL, err))\n \t\tgoto error_return;\n-\t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n@@ -1045,7 +1041,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \n2.32.0.874.gfa1990a4f10\n\n"},{"id":"430334","messageId":"patch-2.7-2a69bbea821-20210716T142032Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","subject":"[PATCH v8 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-16T14:22:27Z","receivedAt":"2021-07-16T14:22:54Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 467f4b3c936..f4445e32904 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \n2.32.0.874.gfa1990a4f10\n\n"},{"id":"430338","messageId":"patch-4.7-147058c8c3d-20210716T142032Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","subject":"[PATCH v8 4/7] refs: make errno output explicit for read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-16T14:22:29Z","receivedAt":"2021-07-16T14:22:57Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes it explicit how alternative ref backends should report errors in\nread_raw_ref_fn.\n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs/debug.c | 4 ++--\n refs/files-backend.c | 29 +++++++++++++++--------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 20 ++++++++++++--------\n 5 files changed, 34 insertions(+), 29 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 5e2330bf78e..136e2e4c78a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1682,7 +1682,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, &errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 18fd9bca595..d0aaea6d918 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \toidcpy(oid, null_oid());\n \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex a6a9f2b99fa..70970f6f770 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -540,6 +538,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -610,7 +609,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tif (hold_lock_file_for_update_timeout(\n \t\t\t &lock->lk, ref_file.buf, LOCK_NO_DEREF,\n \t\t\t get_files_ref_lock_timeout_ms()) < 0) {\n-\t\tif (errno == ENOENT && --attempts_remaining > 0) {\n+\t\tint myerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr == ENOENT && --attempts_remaining > 0) {\n \t\t\t/*\n \t\t\t * Maybe somebody just deleted one of the\n \t\t\t * directories leading to ref_file. Try\n@@ -618,7 +619,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t */\n \t\t\tgoto retry;\n \t\t} else {\n-\t\t\tunable_to_lock_message(ref_file.buf, errno, err);\n+\t\t\tunable_to_lock_message(ref_file.buf, myerr, err);\n \t\t\tgoto error_return;\n \t\t}\n \t}\n@@ -628,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -654,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -692,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 53d1ec27b60..1cb7f8e8f70 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex f4445e32904..79dfb3af484 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n+ * type, and return -1 (failure_errno should not be ENOENT)\n+ *\n+ * failure_errno provides errno codes that are interpreted beyond error\n+ * reporting. The following error codes have special meaning:\n+ * * ENOENT: the ref doesn't exist\n+ * * EISDIR: ref name is a directory\n+ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \n2.32.0.874.gfa1990a4f10\n\n"},{"id":"430335","messageId":"patch-5.7-b42a7474f18-20210716T142032Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","subject":"[PATCH v8 5/7] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-16T14:22:30Z","receivedAt":"2021-07-16T14:22:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n\nSome of our callers explicitly do not care about the errno, rather\nthan understanding NULL let's have them declare that they don't care\nby passing in an \"ignore_errno\". There's only three of them, and using\nthat pattern will make it more obvious that they want to throw away\ndata, let's also add a comment to one of the callers about why we'd\nlike to ignore the errno.\n\nLet's not extend that to refs_resolve_ref_unsafe() itself for now, it\nhas a large set of legacy callers, so we're faking up the old \"errno\"\nbehavior for it. We can convert those callers to\nrefs_resolve_ref_unsafe_with_errno() later.\n\nWe are leaving out out the refs_read_special_head() in\nrefs_read_raw_ref() for now, as noted in the next commit moving it to\n\"failure_errno\" will require some special consideration.\n\nWe're intentionally mis-indenting the argument list of the new\nrefs_resolve_ref_unsafe_with_errno(), it will be non-static in a\nsubsequent commit, doing it this way makes that diff smaller.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 61 ++++++++++++++++++++++++++++++-------------\n refs/files-backend.c | 10 ++++---\n refs/packed-backend.c | 7 ++---\n refs/refs-internal.h | 6 ++---\n 4 files changed, 56 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 136e2e4c78a..b451f917d39 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1672,30 +1672,33 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n+\tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &errno);\n+\t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n \tint symref_count;\n \n+\tassert(failure_errno);\n+\n \tif (!oid)\n \t\toid = &unused_oid;\n \tif (!flags)\n@@ -1706,7 +1709,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n+\t\t\t*failure_errno = EINVAL;\n \t\t\treturn NULL;\n \t\t}\n \n@@ -1724,8 +1727,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, failure_errno)) {\n \t\t\t*flags |= read_flags;\n \n \t\t\t/* In reading mode, refs must eventually resolve */\n@@ -1737,9 +1740,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (*failure_errno != ENOENT &&\n+\t\t\t *failure_errno != EISDIR &&\n+\t\t\t *failure_errno != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -1766,7 +1769,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t\t !refname_is_safe(refname)) {\n-\t\t\t\terrno = EINVAL;\n+\t\t\t\t*failure_errno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \n@@ -1774,10 +1777,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t}\n \t}\n \n-\terrno = ELOOP;\n+\t*failure_errno = ELOOP;\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tint failure_errno = 0;\n+\tconst char *refn;\n+\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &failure_errno);\n+\tif (!refn)\n+\t\t/* For unmigrated legacy callers */\n+\t\terrno = failure_errno;\n+\treturn refn;\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n@@ -2228,6 +2245,13 @@ int refs_verify_refname_available(struct ref_store *refs,\n \n \tstrbuf_grow(&dirname, strlen(refname) + 1);\n \tfor (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {\n+\t\t/*\n+\t\t * Just saying \"Is a directory\" when we e.g. can't\n+\t\t * lock some multi-level ref isn't very informative,\n+\t\t * the user won't be told *what* is a directory, so\n+\t\t * let's not use strerror() below.\n+\t\t */\n+\t\tint ignore_errno;\n \t\t/* Expand dirname to the new prefix, not including the trailing slash: */\n \t\tstrbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);\n \n@@ -2239,7 +2263,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, &ignore_errno)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 70970f6f770..25bb225d92a 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\tgoto out;\n \n \tif (lstat(path, &st) < 0) {\n+\t\tint ignore_errno;\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \t/* Is it a directory? */\n \tif (S_ISDIR(st.st_mode)) {\n+\t\tint ignore_errno;\n \t\t/*\n \t\t * Even though there is a directory where the loose\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 1cb7f8e8f70..a4bf3d22d77 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure_errno;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure_errno) ||\n+\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 79dfb3af484..54f57c6a2df 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /*\n * Write an error to `err` and return a nonzero value iff the same\n-- \n2.32.0.874.gfa1990a4f10\n\n"},{"id":"430337","messageId":"patch-6.7-93b770c8bea-20210716T142032Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","subject":"[PATCH v8 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-16T14:22:31Z","receivedAt":"2021-07-16T14:23:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe EINVAL error from parse_loose_ref_contents is used in files-backend\nto create a custom error message.\n\nIn untangling this we discovered a tricky edge case. The\nrefs_read_special_head() function was relying on\nparse_loose_ref_contents() setting EINVAL.\n\nBy converting it to use \"saved_errno\" we can migrate away from \"errno\"\nin this part of the code entirely, and do away with an existing\n\"save_errno\" pattern, its only purpose was to not clobber the \"errno\"\nwe previously needed at the end of files_read_raw_ref().\n\nLet's assert that we can do that by not having files_read_raw_ref()\nitself operate on *failure_errno in addition to passing it on. Instead\nwe'll assert that if we return non-zero we actually do set errno, thus\nassuring ourselves and callers that they can trust the resulting\n\"failure_errno\".\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 8 +++++---\n refs/files-backend.c | 30 +++++++++++++++++++-----------\n refs/refs-internal.h | 6 ++++--\n 3 files changed, 28 insertions(+), 16 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex b451f917d39..ceaaccc1680 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1654,7 +1654,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n \n static int refs_read_special_head(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t\t int *failure_errno)\n {\n \tstruct strbuf full_path = STRBUF_INIT;\n \tstruct strbuf content = STRBUF_INIT;\n@@ -1664,7 +1665,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n \t\tgoto done;\n \n-\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n+\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n+\t\t\t\t\t failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\n@@ -1679,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 25bb225d92a..45d7c346dea 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tint fd;\n \tint ret = -1;\n \tint remaining_retries = 3;\n+\tint myerr = 0;\n \n \t*type = 0;\n \tstrbuf_reset(&sb_path);\n@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (lstat(path, &st) < 0) {\n \t\tint ignore_errno;\n-\t\tif (errno != ENOENT)\n+\t\tmyerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr != ENOENT)\n \t\t\tgoto out;\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = ENOENT;\n+\t\t\tmyerr = ENOENT;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tif (S_ISLNK(st.st_mode)) {\n \t\tstrbuf_reset(&sb_contents);\n \t\tif (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {\n-\t\t\tif (errno == ENOENT || errno == EINVAL)\n+\t\t\tmyerr = errno;\n+\t\t\terrno = 0;\n+\t\t\tif (myerr == ENOENT || myerr == EINVAL)\n \t\t\t\t/* inconsistent with lstat; retry */\n \t\t\t\tgoto stat_ref;\n \t\t\telse\n@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t */\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = EISDIR;\n+\t\t\tmyerr = EISDIR;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t */\n \tfd = open(path, O_RDONLY);\n \tif (fd < 0) {\n-\t\tif (errno == ENOENT && !S_ISLNK(st.st_mode))\n+\t\tmyerr = errno;\n+\t\tif (myerr == ENOENT && !S_ISLNK(st.st_mode))\n \t\t\t/* inconsistent with lstat; retry */\n \t\t\tgoto stat_ref;\n \t\telse\n@@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t}\n \tstrbuf_reset(&sb_contents);\n \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n-\t\tint save_errno = errno;\n \t\tclose(fd);\n-\t\terrno = save_errno;\n \t\tgoto out;\n \t}\n \tclose(fd);\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n-\tret = parse_loose_ref_contents(buf, oid, referent, type);\n+\tret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);\n \n out:\n-\t*failure_errno = errno;\n+\tif (ret && !myerr)\n+\t\tBUG(\"returning non-zero %d, should have set myerr!\", ret);\n+\t*failure_errno = myerr;\n+\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n \treturn ret;\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -486,7 +494,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n \tif (parse_oid_hex(buf, oid, &p) ||\n \t (*p != '\\0' && !isspace(*p))) {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n \treturn 0;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 54f57c6a2df..bf581e70cf6 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -689,10 +689,12 @@ struct ref_store {\n };\n \n /*\n- * Parse contents of a loose ref file.\n+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for\n+ * invalid contents.\n */\n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\n-- \n2.32.0.874.gfa1990a4f10\n\n"},{"id":"430336","messageId":"patch-7.7-cb32b5c0526-20210716T142032Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","subject":"[PATCH v8 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-16T14:22:32Z","receivedAt":"2021-07-16T14:23:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions, so update that caller\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs.h | 11 +++++++++++\n refs/files-backend.c | 12 +++++++-----\n 3 files changed, 19 insertions(+), 6 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex ceaaccc1680..9e2592ee28a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1688,7 +1688,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t\t\t type, failure_errno);\n }\n \n-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n \t\t\t\t\t const char *refname,\n \t\t\t\t\t int resolve_flags,\n \t\t\t\t\t struct object_id *oid,\ndiff --git a/refs.h b/refs.h\nindex c009707438d..ba09ba0687b 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t\t int resolve_flags,\n \t\t\t\t struct object_id *oid,\n \t\t\t\t int *flags);\n+/**\n+ * refs_resolve_ref_unsafe_with_errno() is like\n+ * refs_resolve_ref_unsafe(), but provide access to errno code that\n+ * lead to a failure. We guarantee that errno is set to a meaningful\n+ * value on non-zero return.\n+ */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags);\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 45d7c346dea..be5f386f64d 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1001,6 +1001,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -1008,17 +1009,18 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tCALLOC_ARRAY(lock, 1);\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n-\t\t\t\t RESOLVE_REF_NO_RECURSE,\n-\t\t\t\t &lock->old_oid, type) &&\n-\t (errno != ENOTDIR ||\n+\tif (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\tRESOLVE_REF_NO_RECURSE,\n+\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t&resolve_errno) &&\n+\t (resolve_errno != ENOTDIR ||\n \t /* in case of D/F conflict, try to generate a better error\n \t * message. If that fails, fall back to strerror(ENOTDIR).\n \t */\n \t !refs_verify_refname_available(&refs->base, refname, NULL,\n \t\t\t\t\t NULL, err))) {\n \t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t refname, strerror(errno));\n+\t\t\t refname, strerror(resolve_errno));\n \t\tgoto error_return;\n \t}\n \n-- \n2.32.0.874.gfa1990a4f10\n\n"},{"id":"430588","messageId":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com","subject":"[PATCH v9 0/7] refs: cleanup errno sideband ref related functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-20T10:33:23Z","receivedAt":"2021-07-20T10:34:11Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"A v9 re-roll of the v8, see\nhttps://lore.kernel.org/git/cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com/\nand more imporantly the real summary of what this is in v7 at\nhttps://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/\n\nAs noted there this topic builds on my just-re-rolled v3 removing of\ndead code + fixing a gc race in the refs API, at:\nhttps://lore.kernel.org/git/cover-00.12-00000000000-20210720T102051Z-avarab@gmail.com/\n\nThe changes in this v8 are merely to re-roll on top of that. There was\na stray comment that didn't apply anymore with its new 12th patch. I\nalso removed the \"in case of D/F conflict...\" comment the 3rd patch\npreviously added, with the changes in the base topic I think this\ncodepath has become easily understood enough to not need an new\ncomment anymore (the initial version was more complex, see\nhttps://lore.kernel.org/git/95025080c16f535599826ed4f013845d712b0e8d.1625684869.git.gitgitgadget@gmail.com/).\n\nHan-Wen Nienhuys (6):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: add failure_errno to refs_read_raw_ref() signature\n refs: explicitly return failure_errno from parse_loose_ref_contents\n refs: make errno output explicit for refs_resolve_ref_unsafe\n\nÆvar Arnfjörð Bjarmason (1):\n refs file backend: move raceproof_create_file() here\n\n cache.h | 43 ----------\n object-file.c | 68 ---------------\n refs.c | 69 ++++++++++-----\n refs.h | 11 +++\n refs/debug.c | 4 +-\n refs/files-backend.c | 195 +++++++++++++++++++++++++++++++++---------\n refs/packed-backend.c | 15 ++--\n refs/refs-internal.h | 32 ++++---\n 8 files changed, 243 insertions(+), 194 deletions(-)\n\nRange-diff against v8:\n1: ce1ca2cf30f = 1: b7063c5af89 refs file backend: move raceproof_create_file() here\n2: 2a69bbea821 = 2: 5a63b64f53f refs: remove EINVAL errno output from specification of read_raw_ref_fn\n3: a3f80c6d2f7 ! 3: 0dd8a4c1209 refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ Commit message\n refs_reflog_exists() (which calls a function in a vtable that is not\n documented to use and/or preserve errno)\n \n - In the case of the \"errno != ENOTDIR\" case that originates in 5b2d8d6f218\n - (lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,\n - 2015-05-11), there the \"last_errno\" was saved away to return it from\n - lock_ref_oid_basic(), now that we're no longer doing that we can skip\n - that entirely and use \"errno\" directly. A follow-up change will\n - extract the specific errno we want earlier in this function.\n -\n Signed-off-by: Han-Wen Nienhuys \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n - \tfiles_ref_path(refs, &ref_file, refname);\n \tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n \t\t\t\t RESOLVE_REF_NO_RECURSE,\n --\t\t\t\t &lock->old_oid, type)) {\n + \t\t\t\t &lock->old_oid, type)) {\n -\t\tlast_errno = errno;\n --\t\tif (last_errno != ENOTDIR ||\n --\t\t !refs_verify_refname_available(&refs->base, refname,\n --\t\t\t\t\t\t NULL, NULL, err))\n --\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n + \t\tif (!refs_verify_refname_available(&refs->base, refname,\n + \t\t\t\t\t\t NULL, NULL, err))\n + \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n -\t\t\t\t refname, strerror(last_errno));\n --\n -+\t\t\t\t &lock->old_oid, type) &&\n -+\t (errno != ENOTDIR ||\n -+\t /* in case of D/F conflict, try to generate a better error\n -+\t * message. If that fails, fall back to strerror(ENOTDIR).\n -+\t */\n -+\t !refs_verify_refname_available(&refs->base, refname, NULL,\n -+\t\t\t\t\t NULL, err))) {\n -+\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n -+\t\t\t refname, strerror(errno));\n ++\t\t\t\t refname, strerror(errno));\n + \n \t\tgoto error_return;\n \t}\n - \n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t */\n \tif (is_null_oid(&lock->old_oid) &&\n4: 147058c8c3d = 4: c54fb99714a refs: make errno output explicit for read_raw_ref_fn\n5: b42a7474f18 = 5: 18825efce7d refs: add failure_errno to refs_read_raw_ref() signature\n6: 93b770c8bea = 6: 57e3f246f4f refs: explicitly return failure_errno from parse_loose_ref_contents\n7: cb32b5c0526 ! 7: 4b5e168b978 refs: make errno output explicit for refs_resolve_ref_unsafe\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re\n \tfiles_ref_path(refs, &ref_file, refname);\n -\tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n -\t\t\t\t RESOLVE_REF_NO_RECURSE,\n --\t\t\t\t &lock->old_oid, type) &&\n --\t (errno != ENOTDIR ||\n +-\t\t\t\t &lock->old_oid, type)) {\n +\tif (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n +\t\t\t\t\t\tRESOLVE_REF_NO_RECURSE,\n +\t\t\t\t\t\t&lock->old_oid, type,\n -+\t\t\t\t\t\t&resolve_errno) &&\n -+\t (resolve_errno != ENOTDIR ||\n - \t /* in case of D/F conflict, try to generate a better error\n - \t * message. If that fails, fall back to strerror(ENOTDIR).\n - \t */\n - \t !refs_verify_refname_available(&refs->base, refname, NULL,\n - \t\t\t\t\t NULL, err))) {\n - \t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n --\t\t\t refname, strerror(errno));\n -+\t\t\t refname, strerror(resolve_errno));\n ++\t\t\t\t\t\t&resolve_errno)) {\n + \t\tif (!refs_verify_refname_available(&refs->base, refname,\n + \t\t\t\t\t\t NULL, NULL, err))\n + \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n +-\t\t\t\t refname, strerror(errno));\n ++\t\t\t\t refname, strerror(resolve_errno));\n + \n \t\tgoto error_return;\n \t}\n - \n-- \n2.32.0.874.ge7a9d58bfcf\n\n"},{"id":"430589","messageId":"patch-1.7-b7063c5af89-20210720T102644Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"[PATCH v9 1/7] refs file backend: move raceproof_create_file() here","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-20T10:33:24Z","receivedAt":"2021-07-20T10:34:14Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the raceproof_create_file() API added to cache.h and\nobject-file.c in 177978f56ad (raceproof_create_file(): new function,\n2017-01-06) to its only user, refs/files-backend.c.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n cache.h | 43 -----------------\n object-file.c | 68 ---------------------------\n refs/files-backend.c | 109 +++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 109 insertions(+), 111 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex ba04ff8bd36..eb4dfe6381f 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1202,49 +1202,6 @@ enum scld_error safe_create_leading_directories(char *path);\n enum scld_error safe_create_leading_directories_const(const char *path);\n enum scld_error safe_create_leading_directories_no_share(char *path);\n \n-/*\n- * Callback function for raceproof_create_file(). This function is\n- * expected to do something that makes dirname(path) permanent despite\n- * the fact that other processes might be cleaning up empty\n- * directories at the same time. Usually it will create a file named\n- * path, but alternatively it could create another file in that\n- * directory, or even chdir() into that directory. The function should\n- * return 0 if the action was completed successfully. On error, it\n- * should return a nonzero result and set errno.\n- * raceproof_create_file() treats two errno values specially:\n- *\n- * - ENOENT -- dirname(path) does not exist. In this case,\n- * raceproof_create_file() tries creating dirname(path)\n- * (and any parent directories, if necessary) and calls\n- * the function again.\n- *\n- * - EISDIR -- the file already exists and is a directory. In this\n- * case, raceproof_create_file() removes the directory if\n- * it is empty (and recursively any empty directories that\n- * it contains) and calls the function again.\n- *\n- * Any other errno causes raceproof_create_file() to fail with the\n- * callback's return value and errno.\n- *\n- * Obviously, this function should be OK with being called again if it\n- * fails with ENOENT or EISDIR. In other scenarios it will not be\n- * called again.\n- */\n-typedef int create_file_fn(const char *path, void *cb);\n-\n-/*\n- * Create a file in dirname(path) by calling fn, creating leading\n- * directories if necessary. Retry a few times in case we are racing\n- * with another process that is trying to clean up the directory that\n- * contains path. See the documentation for create_file_fn for more\n- * details.\n- *\n- * Return the value and set the errno that resulted from the most\n- * recent call of fn. fn is always called at least once, and will be\n- * called more than once if it returns ENOENT or EISDIR.\n- */\n-int raceproof_create_file(const char *path, create_file_fn fn, void *cb);\n-\n int mkdir_in_gitdir(const char *path);\n char *expand_user_path(const char *path, int real_home);\n const char *enter_repo(const char *path, int strict);\ndiff --git a/object-file.c b/object-file.c\nindex ecca5a8da00..231a02997ba 100644\n--- a/object-file.c\n+++ b/object-file.c\n@@ -414,74 +414,6 @@ enum scld_error safe_create_leading_directories_const(const char *path)\n \treturn result;\n }\n \n-int raceproof_create_file(const char *path, create_file_fn fn, void *cb)\n-{\n-\t/*\n-\t * The number of times we will try to remove empty directories\n-\t * in the way of path. This is only 1 because if another\n-\t * process is racily creating directories that conflict with\n-\t * us, we don't want to fight against them.\n-\t */\n-\tint remove_directories_remaining = 1;\n-\n-\t/*\n-\t * The number of times that we will try to create the\n-\t * directories containing path. We are willing to attempt this\n-\t * more than once, because another process could be trying to\n-\t * clean up empty directories at the same time as we are\n-\t * trying to create them.\n-\t */\n-\tint create_directories_remaining = 3;\n-\n-\t/* A scratch copy of path, filled lazily if we need it: */\n-\tstruct strbuf path_copy = STRBUF_INIT;\n-\n-\tint ret, save_errno;\n-\n-\t/* Sanity check: */\n-\tassert(*path);\n-\n-retry_fn:\n-\tret = fn(path, cb);\n-\tsave_errno = errno;\n-\tif (!ret)\n-\t\tgoto out;\n-\n-\tif (errno == EISDIR && remove_directories_remaining-- > 0) {\n-\t\t/*\n-\t\t * A directory is in the way. Maybe it is empty; try\n-\t\t * to remove it:\n-\t\t */\n-\t\tif (!path_copy.len)\n-\t\t\tstrbuf_addstr(&path_copy, path);\n-\n-\t\tif (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))\n-\t\t\tgoto retry_fn;\n-\t} else if (errno == ENOENT && create_directories_remaining-- > 0) {\n-\t\t/*\n-\t\t * Maybe the containing directory didn't exist, or\n-\t\t * maybe it was just deleted by a process that is\n-\t\t * racing with us to clean up empty directories. Try\n-\t\t * to create it:\n-\t\t */\n-\t\tenum scld_error scld_result;\n-\n-\t\tif (!path_copy.len)\n-\t\t\tstrbuf_addstr(&path_copy, path);\n-\n-\t\tdo {\n-\t\t\tscld_result = safe_create_leading_directories(path_copy.buf);\n-\t\t\tif (scld_result == SCLD_OK)\n-\t\t\t\tgoto retry_fn;\n-\t\t} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);\n-\t}\n-\n-out:\n-\tstrbuf_release(&path_copy);\n-\terrno = save_errno;\n-\treturn ret;\n-}\n-\n static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)\n {\n \tint i;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 443182da102..9fc596fc75a 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -852,6 +852,115 @@ static struct ref_iterator *files_ref_iterator_begin(\n \treturn ref_iterator;\n }\n \n+/*\n+ * Callback function for raceproof_create_file(). This function is\n+ * expected to do something that makes dirname(path) permanent despite\n+ * the fact that other processes might be cleaning up empty\n+ * directories at the same time. Usually it will create a file named\n+ * path, but alternatively it could create another file in that\n+ * directory, or even chdir() into that directory. The function should\n+ * return 0 if the action was completed successfully. On error, it\n+ * should return a nonzero result and set errno.\n+ * raceproof_create_file() treats two errno values specially:\n+ *\n+ * - ENOENT -- dirname(path) does not exist. In this case,\n+ * raceproof_create_file() tries creating dirname(path)\n+ * (and any parent directories, if necessary) and calls\n+ * the function again.\n+ *\n+ * - EISDIR -- the file already exists and is a directory. In this\n+ * case, raceproof_create_file() removes the directory if\n+ * it is empty (and recursively any empty directories that\n+ * it contains) and calls the function again.\n+ *\n+ * Any other errno causes raceproof_create_file() to fail with the\n+ * callback's return value and errno.\n+ *\n+ * Obviously, this function should be OK with being called again if it\n+ * fails with ENOENT or EISDIR. In other scenarios it will not be\n+ * called again.\n+ */\n+typedef int create_file_fn(const char *path, void *cb);\n+\n+/*\n+ * Create a file in dirname(path) by calling fn, creating leading\n+ * directories if necessary. Retry a few times in case we are racing\n+ * with another process that is trying to clean up the directory that\n+ * contains path. See the documentation for create_file_fn for more\n+ * details.\n+ *\n+ * Return the value and set the errno that resulted from the most\n+ * recent call of fn. fn is always called at least once, and will be\n+ * called more than once if it returns ENOENT or EISDIR.\n+ */\n+static int raceproof_create_file(const char *path, create_file_fn fn, void *cb)\n+{\n+\t/*\n+\t * The number of times we will try to remove empty directories\n+\t * in the way of path. This is only 1 because if another\n+\t * process is racily creating directories that conflict with\n+\t * us, we don't want to fight against them.\n+\t */\n+\tint remove_directories_remaining = 1;\n+\n+\t/*\n+\t * The number of times that we will try to create the\n+\t * directories containing path. We are willing to attempt this\n+\t * more than once, because another process could be trying to\n+\t * clean up empty directories at the same time as we are\n+\t * trying to create them.\n+\t */\n+\tint create_directories_remaining = 3;\n+\n+\t/* A scratch copy of path, filled lazily if we need it: */\n+\tstruct strbuf path_copy = STRBUF_INIT;\n+\n+\tint ret, save_errno;\n+\n+\t/* Sanity check: */\n+\tassert(*path);\n+\n+retry_fn:\n+\tret = fn(path, cb);\n+\tsave_errno = errno;\n+\tif (!ret)\n+\t\tgoto out;\n+\n+\tif (errno == EISDIR && remove_directories_remaining-- > 0) {\n+\t\t/*\n+\t\t * A directory is in the way. Maybe it is empty; try\n+\t\t * to remove it:\n+\t\t */\n+\t\tif (!path_copy.len)\n+\t\t\tstrbuf_addstr(&path_copy, path);\n+\n+\t\tif (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))\n+\t\t\tgoto retry_fn;\n+\t} else if (errno == ENOENT && create_directories_remaining-- > 0) {\n+\t\t/*\n+\t\t * Maybe the containing directory didn't exist, or\n+\t\t * maybe it was just deleted by a process that is\n+\t\t * racing with us to clean up empty directories. Try\n+\t\t * to create it:\n+\t\t */\n+\t\tenum scld_error scld_result;\n+\n+\t\tif (!path_copy.len)\n+\t\t\tstrbuf_addstr(&path_copy, path);\n+\n+\t\tdo {\n+\t\t\tscld_result = safe_create_leading_directories(path_copy.buf);\n+\t\t\tif (scld_result == SCLD_OK)\n+\t\t\t\tgoto retry_fn;\n+\t\t} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);\n+\t}\n+\n+out:\n+\tstrbuf_release(&path_copy);\n+\terrno = save_errno;\n+\treturn ret;\n+}\n+\n static int remove_empty_directories(struct strbuf *path)\n {\n \t/*\n-- \n2.32.0.874.ge7a9d58bfcf\n\n"},{"id":"430590","messageId":"patch-2.7-5a63b64f53f-20210720T102644Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"[PATCH v9 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-20T10:33:25Z","receivedAt":"2021-07-20T10:34:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 3155708345f..dc0d826c3ac 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -622,9 +622,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \n2.32.0.874.ge7a9d58bfcf\n\n"},{"id":"430591","messageId":"patch-3.7-0dd8a4c1209-20210720T102644Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"[PATCH v9 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-20T10:33:26Z","receivedAt":"2021-07-20T10:35:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 11 ++---------\n 1 file changed, 2 insertions(+), 9 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 9fc596fc75a..9e00da76121 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -982,7 +982,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname,\n@@ -991,7 +990,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -1002,11 +1000,10 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n \t\t\t\t RESOLVE_REF_NO_RECURSE,\n \t\t\t\t &lock->old_oid, type)) {\n-\t\tlast_errno = errno;\n \t\tif (!refs_verify_refname_available(&refs->base, refname,\n \t\t\t\t\t\t NULL, NULL, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n+\t\t\t\t refname, strerror(errno));\n \n \t\tgoto error_return;\n \t}\n@@ -1019,15 +1016,12 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t */\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n-\t\t\t\t\t NULL, NULL, err)) {\n-\t\tlast_errno = ENOTDIR;\n+\t\t\t\t\t NULL, NULL, err))\n \t\tgoto error_return;\n-\t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n@@ -1044,7 +1038,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \n2.32.0.874.ge7a9d58bfcf\n\n"},{"id":"430593","messageId":"patch-4.7-c54fb99714a-20210720T102644Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"[PATCH v9 4/7] refs: make errno output explicit for read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-20T10:33:27Z","receivedAt":"2021-07-20T10:35:42Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes it explicit how alternative ref backends should report errors in\nread_raw_ref_fn.\n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs/debug.c | 4 ++--\n refs/files-backend.c | 29 +++++++++++++++--------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 20 ++++++++++++--------\n 5 files changed, 34 insertions(+), 29 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex d9635436759..fc6c0ddffae 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1682,7 +1682,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, &errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 18fd9bca595..d0aaea6d918 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \toidcpy(oid, null_oid());\n \terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 9e00da76121..85b4d791444 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -540,6 +538,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -610,7 +609,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tif (hold_lock_file_for_update_timeout(\n \t\t\t &lock->lk, ref_file.buf, LOCK_NO_DEREF,\n \t\t\t get_files_ref_lock_timeout_ms()) < 0) {\n-\t\tif (errno == ENOENT && --attempts_remaining > 0) {\n+\t\tint myerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr == ENOENT && --attempts_remaining > 0) {\n \t\t\t/*\n \t\t\t * Maybe somebody just deleted one of the\n \t\t\t * directories leading to ref_file. Try\n@@ -618,7 +619,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t */\n \t\t\tgoto retry;\n \t\t} else {\n-\t\t\tunable_to_lock_message(ref_file.buf, errno, err);\n+\t\t\tunable_to_lock_message(ref_file.buf, myerr, err);\n \t\t\tgoto error_return;\n \t\t}\n \t}\n@@ -628,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -654,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -692,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 24a360b719f..159ac776240 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex dc0d826c3ac..33a31d5c236 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -620,11 +620,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n+ * type, and return -1 (failure_errno should not be ENOENT)\n+ *\n+ * failure_errno provides errno codes that are interpreted beyond error\n+ * reporting. The following error codes have special meaning:\n+ * * ENOENT: the ref doesn't exist\n+ * * EISDIR: ref name is a directory\n+ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -638,9 +642,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \n2.32.0.874.ge7a9d58bfcf\n\n"},{"id":"430592","messageId":"patch-5.7-18825efce7d-20210720T102644Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"[PATCH v9 5/7] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-20T10:33:28Z","receivedAt":"2021-07-20T10:35:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n\nSome of our callers explicitly do not care about the errno, rather\nthan understanding NULL let's have them declare that they don't care\nby passing in an \"ignore_errno\". There's only three of them, and using\nthat pattern will make it more obvious that they want to throw away\ndata, let's also add a comment to one of the callers about why we'd\nlike to ignore the errno.\n\nLet's not extend that to refs_resolve_ref_unsafe() itself for now, it\nhas a large set of legacy callers, so we're faking up the old \"errno\"\nbehavior for it. We can convert those callers to\nrefs_resolve_ref_unsafe_with_errno() later.\n\nWe are leaving out out the refs_read_special_head() in\nrefs_read_raw_ref() for now, as noted in the next commit moving it to\n\"failure_errno\" will require some special consideration.\n\nWe're intentionally mis-indenting the argument list of the new\nrefs_resolve_ref_unsafe_with_errno(), it will be non-static in a\nsubsequent commit, doing it this way makes that diff smaller.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 61 ++++++++++++++++++++++++++++++-------------\n refs/files-backend.c | 10 ++++---\n refs/packed-backend.c | 7 ++---\n refs/refs-internal.h | 6 ++---\n 4 files changed, 56 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex fc6c0ddffae..728285c9220 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1672,30 +1672,33 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n+\tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &errno);\n+\t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n \tint symref_count;\n \n+\tassert(failure_errno);\n+\n \tif (!oid)\n \t\toid = &unused_oid;\n \tif (!flags)\n@@ -1706,7 +1709,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n+\t\t\t*failure_errno = EINVAL;\n \t\t\treturn NULL;\n \t\t}\n \n@@ -1724,8 +1727,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, failure_errno)) {\n \t\t\t*flags |= read_flags;\n \n \t\t\t/* In reading mode, refs must eventually resolve */\n@@ -1737,9 +1740,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (*failure_errno != ENOENT &&\n+\t\t\t *failure_errno != EISDIR &&\n+\t\t\t *failure_errno != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -1766,7 +1769,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t\t !refname_is_safe(refname)) {\n-\t\t\t\terrno = EINVAL;\n+\t\t\t\t*failure_errno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \n@@ -1774,10 +1777,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t}\n \t}\n \n-\terrno = ELOOP;\n+\t*failure_errno = ELOOP;\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tint failure_errno = 0;\n+\tconst char *refn;\n+\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &failure_errno);\n+\tif (!refn)\n+\t\t/* For unmigrated legacy callers */\n+\t\terrno = failure_errno;\n+\treturn refn;\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n@@ -2228,6 +2245,13 @@ int refs_verify_refname_available(struct ref_store *refs,\n \n \tstrbuf_grow(&dirname, strlen(refname) + 1);\n \tfor (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {\n+\t\t/*\n+\t\t * Just saying \"Is a directory\" when we e.g. can't\n+\t\t * lock some multi-level ref isn't very informative,\n+\t\t * the user won't be told *what* is a directory, so\n+\t\t * let's not use strerror() below.\n+\t\t */\n+\t\tint ignore_errno;\n \t\t/* Expand dirname to the new prefix, not including the trailing slash: */\n \t\tstrbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);\n \n@@ -2239,7 +2263,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, &ignore_errno)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 85b4d791444..bbd8caec624 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\tgoto out;\n \n \tif (lstat(path, &st) < 0) {\n+\t\tint ignore_errno;\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \t/* Is it a directory? */\n \tif (S_ISDIR(st.st_mode)) {\n+\t\tint ignore_errno;\n \t\t/*\n \t\t * Even though there is a directory where the loose\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 159ac776240..923b9ad89de 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure_errno;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure_errno) ||\n+\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 33a31d5c236..7beb38f79cc 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /*\n * Write an error to `err` and return a nonzero value iff the same\n-- \n2.32.0.874.ge7a9d58bfcf\n\n"},{"id":"430594","messageId":"patch-6.7-57e3f246f4f-20210720T102644Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"[PATCH v9 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-20T10:33:29Z","receivedAt":"2021-07-20T10:35:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe EINVAL error from parse_loose_ref_contents is used in files-backend\nto create a custom error message.\n\nIn untangling this we discovered a tricky edge case. The\nrefs_read_special_head() function was relying on\nparse_loose_ref_contents() setting EINVAL.\n\nBy converting it to use \"saved_errno\" we can migrate away from \"errno\"\nin this part of the code entirely, and do away with an existing\n\"save_errno\" pattern, its only purpose was to not clobber the \"errno\"\nwe previously needed at the end of files_read_raw_ref().\n\nLet's assert that we can do that by not having files_read_raw_ref()\nitself operate on *failure_errno in addition to passing it on. Instead\nwe'll assert that if we return non-zero we actually do set errno, thus\nassuring ourselves and callers that they can trust the resulting\n\"failure_errno\".\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 8 +++++---\n refs/files-backend.c | 30 +++++++++++++++++++-----------\n refs/refs-internal.h | 6 ++++--\n 3 files changed, 28 insertions(+), 16 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 728285c9220..b31dbdd0fa2 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1654,7 +1654,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n \n static int refs_read_special_head(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t\t int *failure_errno)\n {\n \tstruct strbuf full_path = STRBUF_INIT;\n \tstruct strbuf content = STRBUF_INIT;\n@@ -1664,7 +1665,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n \t\tgoto done;\n \n-\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n+\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n+\t\t\t\t\t failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\n@@ -1679,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex bbd8caec624..746831c86cb 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tint fd;\n \tint ret = -1;\n \tint remaining_retries = 3;\n+\tint myerr = 0;\n \n \t*type = 0;\n \tstrbuf_reset(&sb_path);\n@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (lstat(path, &st) < 0) {\n \t\tint ignore_errno;\n-\t\tif (errno != ENOENT)\n+\t\tmyerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr != ENOENT)\n \t\t\tgoto out;\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = ENOENT;\n+\t\t\tmyerr = ENOENT;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tif (S_ISLNK(st.st_mode)) {\n \t\tstrbuf_reset(&sb_contents);\n \t\tif (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {\n-\t\t\tif (errno == ENOENT || errno == EINVAL)\n+\t\t\tmyerr = errno;\n+\t\t\terrno = 0;\n+\t\t\tif (myerr == ENOENT || myerr == EINVAL)\n \t\t\t\t/* inconsistent with lstat; retry */\n \t\t\t\tgoto stat_ref;\n \t\t\telse\n@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t */\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = EISDIR;\n+\t\t\tmyerr = EISDIR;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t */\n \tfd = open(path, O_RDONLY);\n \tif (fd < 0) {\n-\t\tif (errno == ENOENT && !S_ISLNK(st.st_mode))\n+\t\tmyerr = errno;\n+\t\tif (myerr == ENOENT && !S_ISLNK(st.st_mode))\n \t\t\t/* inconsistent with lstat; retry */\n \t\t\tgoto stat_ref;\n \t\telse\n@@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t}\n \tstrbuf_reset(&sb_contents);\n \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n-\t\tint save_errno = errno;\n \t\tclose(fd);\n-\t\terrno = save_errno;\n \t\tgoto out;\n \t}\n \tclose(fd);\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n-\tret = parse_loose_ref_contents(buf, oid, referent, type);\n+\tret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);\n \n out:\n-\t*failure_errno = errno;\n+\tif (ret && !myerr)\n+\t\tBUG(\"returning non-zero %d, should have set myerr!\", ret);\n+\t*failure_errno = myerr;\n+\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n \treturn ret;\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -486,7 +494,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n \tif (parse_oid_hex(buf, oid, &p) ||\n \t (*p != '\\0' && !isspace(*p))) {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n \treturn 0;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 7beb38f79cc..9aa4af81836 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -692,10 +692,12 @@ struct ref_store {\n };\n \n /*\n- * Parse contents of a loose ref file.\n+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for\n+ * invalid contents.\n */\n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\n-- \n2.32.0.874.ge7a9d58bfcf\n\n"},{"id":"430595","messageId":"patch-7.7-4b5e168b978-20210720T102644Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"[PATCH v9 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-20T10:33:30Z","receivedAt":"2021-07-20T10:36:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions, so update that caller\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs.h | 11 +++++++++++\n refs/files-backend.c | 10 ++++++----\n 3 files changed, 18 insertions(+), 5 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex b31dbdd0fa2..9497b51e0a1 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1688,7 +1688,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t\t\t type, failure_errno);\n }\n \n-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n \t\t\t\t\t const char *refname,\n \t\t\t\t\t int resolve_flags,\n \t\t\t\t\t struct object_id *oid,\ndiff --git a/refs.h b/refs.h\nindex c009707438d..ba09ba0687b 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t\t int resolve_flags,\n \t\t\t\t struct object_id *oid,\n \t\t\t\t int *flags);\n+/**\n+ * refs_resolve_ref_unsafe_with_errno() is like\n+ * refs_resolve_ref_unsafe(), but provide access to errno code that\n+ * lead to a failure. We guarantee that errno is set to a meaningful\n+ * value on non-zero return.\n+ */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags);\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 746831c86cb..68182296c1b 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1001,6 +1001,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -1008,13 +1009,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tCALLOC_ARRAY(lock, 1);\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n-\t\t\t\t RESOLVE_REF_NO_RECURSE,\n-\t\t\t\t &lock->old_oid, type)) {\n+\tif (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\tRESOLVE_REF_NO_RECURSE,\n+\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t&resolve_errno)) {\n \t\tif (!refs_verify_refname_available(&refs->base, refname,\n \t\t\t\t\t\t NULL, NULL, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(resolve_errno));\n \n \t\tgoto error_return;\n \t}\n-- \n2.32.0.874.ge7a9d58bfcf\n\n"},{"id":"431261","messageId":"877dhcskh4.fsf@evledraar.gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"Re: [PATCH v9 0/7] refs: cleanup errno sideband ref related functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-07-26T23:49:40Z","receivedAt":"2021-07-26T23:50:24Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jul 20 2021, Ævar Arnfjörð Bjarmason wrote:\n\n> A v9 re-roll of the v8, see\n> https://lore.kernel.org/git/cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com/\n> and more imporantly the real summary of what this is in v7 at\n> https://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/\n\nFYI I did not re-roll a v10 of this on top of the just re-rolled base\ntopic:\nhttps://lore.kernel.org/git/cover-00.11-0000000000-20210726T234237Z-avarab@gmail.com/#t\n\nThe range-diff I have locally between the two (this version, and my\nrebased \"should I need a re-roll\" version) shows no changes, so this\nstill cleanly applies on top.\n"},{"id":"431265","messageId":"xmqqmtq8fw3d.fsf@gitster.g","threadId":"55589","inReplyTo":"877dhcskh4.fsf@evledraar.gmail.com","subject":"Re: [PATCH v9 0/7] refs: cleanup errno sideband ref related functions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-07-27T00:18:14Z","receivedAt":"2021-07-27T00:18:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Tue, Jul 20 2021, Ævar Arnfjörð Bjarmason wrote:\n>\n>> A v9 re-roll of the v8, see\n>> https://lore.kernel.org/git/cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com/\n>> and more imporantly the real summary of what this is in v7 at\n>> https://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/\n>\n> FYI I did not re-roll a v10 of this on top of the just re-rolled base\n> topic:\n> https://lore.kernel.org/git/cover-00.11-0000000000-20210726T234237Z-avarab@gmail.com/#t\n>\n> The range-diff I have locally between the two (this version, and my\n> rebased \"should I need a re-roll\" version) shows no changes, so this\n> still cleanly applies on top.\n\nThanks. I just rebased it on top of the base topic locally. No\nneed to resend.\n\n"},{"id":"432707","messageId":"20210813205449.1198555-1-jonathantanmy@google.com","threadId":"55589","inReplyTo":"patch-6.7-57e3f246f4f-20210720T102644Z-avarab@gmail.com","subject":"Re: [PATCH v9 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-08-13T20:54:49Z","receivedAt":"2021-08-13T20:54:55Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> @@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> \t}\n> \tstrbuf_reset(&sb_contents);\n> \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n> -\t\tint save_errno = errno;\n> \t\tclose(fd);\n> -\t\terrno = save_errno;\n> \t\tgoto out;\n> \t}\n\n[snip unrelated code]\n\n> out:\n> -\t*failure_errno = errno;\n> +\tif (ret && !myerr)\n> +\t\tBUG(\"returning non-zero %d, should have set myerr!\", ret);\n\nAt $DAYJOB, some people have observed this BUG triggering. Right now I\ndon't have a consistent reproduction recipe, but we noticed that the\nblock starting with \"if (strbuf_read\" quoted above does not set myerr\nupon an error, and instead immediately jumps to \"out\" with ret being -1.\n"},{"id":"432798","messageId":"CAFQ2z_M0LNmZn2xW_GWdwZOCi20xc9t3EnMzMzHP8ZcmWrW9EA@mail.gmail.com","threadId":"55589","inReplyTo":"patch-4.7-c54fb99714a-20210720T102644Z-avarab@gmail.com","subject":"Re: [PATCH v9 4/7] refs: make errno output explicit for read_raw_ref_fn","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-08-16T13:00:09Z","receivedAt":"2021-08-16T13:00:24Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Tue, Jul 20, 2021 at 12:33 PM Ævar Arnfjörð Bjarmason\n wrote:\n> --- a/refs/debug.c\n> +++ b/refs/debug.c\n> @@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n>\n> static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> struct object_id *oid, struct strbuf *referent,\n> - unsigned int *type)\n> + unsigned int *type, int *failure_errno)\n\nthe code in debug.c needs some tweaking:\n\n> {\n> struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n> int res = 0;\n> @@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n> oidcpy(oid, null_oid());\n> errno = 0;\n\nthis line can go\n\n> res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n> - type);\n> + type, failure_errno);\n>\n> if (res == 0) {\n> trace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\n\nA bit below, failure_errno (rather than errno) should be printed.\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\nRegistergericht und -nummer: Hamburg, HRB 86891\nSitz der Gesellschaft: Hamburg\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"433388","messageId":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com","subject":"[PATCH v10 0/8] refs: cleanup errno sideband ref related functions","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:36Z","receivedAt":"2021-08-23T11:52:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"A v10 re-roll of the v9, see\nhttps://lore.kernel.org/git/cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com/\nand more imporantly the real summary of what this is in v7 at\nhttps://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/\n\nThis topic builds on my just-re-rolled v5 removing of dead code +\nfixing a gc race in the refs API, at:\nhttps://lore.kernel.org/git/cover-v5-00.13-00000000000-20210823T113115Z-avarab@gmail.com/\n\nThe changes since v9 are:\n\n * Rebased on v5 of its \"base\" topic, see:\n https://lore.kernel.org/git/cover-v5-00.13-00000000000-20210823T113115Z-avarab@gmail.com/\n\n * Incorporate the test + fix Han-Wen submitted as\n https://lore.kernel.org/git/pull.1068.git.git.1629203489546.gitgitgadget@gmail.com/;\n that's here as 6/8.\n\n I fixed up the test code a bit, added a SYMLINKS prerequisite as\n suggested by Junio, used a \"test_when_finished\" instead of a \"rm\n -rf\" at the start, used test_cmp over \"test\", and other minor style\n changes. The actual BUG() fix is then squashed into 7/8.\n\n * Fixed issues in refs/debug.c, noted in\n https://lore.kernel.org/git/CAFQ2z_M0LNmZn2xW_GWdwZOCi20xc9t3EnMzMzHP8ZcmWrW9EA@mail.gmail.com/\n\nHan-Wen Nienhuys (7):\n refs: remove EINVAL errno output from specification of read_raw_ref_fn\n refs/files-backend: stop setting errno from lock_ref_oid_basic\n refs: make errno output explicit for read_raw_ref_fn\n refs: add failure_errno to refs_read_raw_ref() signature\n branch tests: test for errno propagating on failing read\n refs: explicitly return failure_errno from parse_loose_ref_contents\n refs: make errno output explicit for refs_resolve_ref_unsafe\n\nÆvar Arnfjörð Bjarmason (1):\n refs file backend: move raceproof_create_file() here\n\n cache.h | 43 ---------\n object-file.c | 68 ---------------\n refs.c | 69 ++++++++++-----\n refs.h | 11 +++\n refs/debug.c | 7 +-\n refs/files-backend.c | 196 +++++++++++++++++++++++++++++++++---------\n refs/packed-backend.c | 15 ++--\n refs/refs-internal.h | 32 ++++---\n t/t3200-branch.sh | 21 +++++\n 9 files changed, 266 insertions(+), 196 deletions(-)\n\nRange-diff against v9:\n1: b7063c5af89 = 1: f06b054e861 refs file backend: move raceproof_create_file() here\n2: 5a63b64f53f = 2: ba0f5f5fb0a refs: remove EINVAL errno output from specification of read_raw_ref_fn\n3: 0dd8a4c1209 ! 3: 2c4c30e8e06 refs/files-backend: stop setting errno from lock_ref_oid_basic\n @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)\n - * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n - \t\t\t\t\t const char *refname,\n + \t\t\t\t\t const char *refname, int *type,\n @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n4: c54fb99714a ! 4: 05f3a346e3b refs: make errno output explicit for read_raw_ref_fn\n @@ refs/debug.c: debug_ref_iterator_begin(struct ref_store *ref_store, const char *\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n -@@ refs/debug.c: static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \n \toidcpy(oid, null_oid());\n - \terrno = 0;\n +-\terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n -\t\t\t\t\t type);\n +\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\n +@@ refs/debug.c: static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n + \t} else {\n + \t\ttrace_printf_key(&trace_refs,\n + \t\t\t\t \"read_raw_ref: %s: %d (errno %d)\\n\", refname,\n +-\t\t\t\t res, errno);\n ++\t\t\t\t res, *failure_errno);\n + \t}\n + \treturn res;\n + }\n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n5: 18825efce7d = 5: fa9260f25fa refs: add failure_errno to refs_read_raw_ref() signature\n-: ----------- > 6: 6dae8b643ad branch tests: test for errno propagating on failing read\n6: 57e3f246f4f ! 7: 18bf4a0e97c refs: explicitly return failure_errno from parse_loose_ref_contents\n @@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store,\n \tstrbuf_reset(&sb_contents);\n \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n -\t\tint save_errno = errno;\n ++\t\tmyerr = errno;\n \t\tclose(fd);\n -\t\terrno = save_errno;\n \t\tgoto out;\n @@ refs/refs-internal.h: struct ref_store {\n \n /*\n * Fill in the generic part of refs and add it to our collection of\n +\n + ## t/t3200-branch.sh ##\n +@@ t/t3200-branch.sh: test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '\n + \t) &&\n + \tgit --git-dir subdir/.git/ branch rename-src &&\n + \tgit rev-parse rename-src >expect &&\n ++\t# Tests a BUG() assertion in files_read_raw_ref()\n + \tgit --git-dir subdir/.git/ branch -m rename-src rename-dest &&\n + \tgit rev-parse rename-dest >actual &&\n + \ttest_cmp expect actual &&\n7: 4b5e168b978 = 8: 7d94a32af83 refs: make errno output explicit for refs_resolve_ref_unsafe\n-- \n2.33.0.662.g438caf9576d\n\n"},{"id":"433389","messageId":"patch-v10-1.8-f06b054e861-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","subject":"[PATCH v10 1/8] refs file backend: move raceproof_create_file() here","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:37Z","receivedAt":"2021-08-23T11:52:57Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the raceproof_create_file() API added to cache.h and\nobject-file.c in 177978f56ad (raceproof_create_file(): new function,\n2017-01-06) to its only user, refs/files-backend.c.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n cache.h | 43 -----------------\n object-file.c | 68 ---------------------------\n refs/files-backend.c | 109 +++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 109 insertions(+), 111 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex bd4869beee4..1e838303654 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1211,49 +1211,6 @@ enum scld_error safe_create_leading_directories(char *path);\n enum scld_error safe_create_leading_directories_const(const char *path);\n enum scld_error safe_create_leading_directories_no_share(char *path);\n \n-/*\n- * Callback function for raceproof_create_file(). This function is\n- * expected to do something that makes dirname(path) permanent despite\n- * the fact that other processes might be cleaning up empty\n- * directories at the same time. Usually it will create a file named\n- * path, but alternatively it could create another file in that\n- * directory, or even chdir() into that directory. The function should\n- * return 0 if the action was completed successfully. On error, it\n- * should return a nonzero result and set errno.\n- * raceproof_create_file() treats two errno values specially:\n- *\n- * - ENOENT -- dirname(path) does not exist. In this case,\n- * raceproof_create_file() tries creating dirname(path)\n- * (and any parent directories, if necessary) and calls\n- * the function again.\n- *\n- * - EISDIR -- the file already exists and is a directory. In this\n- * case, raceproof_create_file() removes the directory if\n- * it is empty (and recursively any empty directories that\n- * it contains) and calls the function again.\n- *\n- * Any other errno causes raceproof_create_file() to fail with the\n- * callback's return value and errno.\n- *\n- * Obviously, this function should be OK with being called again if it\n- * fails with ENOENT or EISDIR. In other scenarios it will not be\n- * called again.\n- */\n-typedef int create_file_fn(const char *path, void *cb);\n-\n-/*\n- * Create a file in dirname(path) by calling fn, creating leading\n- * directories if necessary. Retry a few times in case we are racing\n- * with another process that is trying to clean up the directory that\n- * contains path. See the documentation for create_file_fn for more\n- * details.\n- *\n- * Return the value and set the errno that resulted from the most\n- * recent call of fn. fn is always called at least once, and will be\n- * called more than once if it returns ENOENT or EISDIR.\n- */\n-int raceproof_create_file(const char *path, create_file_fn fn, void *cb);\n-\n int mkdir_in_gitdir(const char *path);\n char *expand_user_path(const char *path, int real_home);\n const char *enter_repo(const char *path, int strict);\ndiff --git a/object-file.c b/object-file.c\nindex a8be8994814..9b318eecb19 100644\n--- a/object-file.c\n+++ b/object-file.c\n@@ -414,74 +414,6 @@ enum scld_error safe_create_leading_directories_const(const char *path)\n \treturn result;\n }\n \n-int raceproof_create_file(const char *path, create_file_fn fn, void *cb)\n-{\n-\t/*\n-\t * The number of times we will try to remove empty directories\n-\t * in the way of path. This is only 1 because if another\n-\t * process is racily creating directories that conflict with\n-\t * us, we don't want to fight against them.\n-\t */\n-\tint remove_directories_remaining = 1;\n-\n-\t/*\n-\t * The number of times that we will try to create the\n-\t * directories containing path. We are willing to attempt this\n-\t * more than once, because another process could be trying to\n-\t * clean up empty directories at the same time as we are\n-\t * trying to create them.\n-\t */\n-\tint create_directories_remaining = 3;\n-\n-\t/* A scratch copy of path, filled lazily if we need it: */\n-\tstruct strbuf path_copy = STRBUF_INIT;\n-\n-\tint ret, save_errno;\n-\n-\t/* Sanity check: */\n-\tassert(*path);\n-\n-retry_fn:\n-\tret = fn(path, cb);\n-\tsave_errno = errno;\n-\tif (!ret)\n-\t\tgoto out;\n-\n-\tif (errno == EISDIR && remove_directories_remaining-- > 0) {\n-\t\t/*\n-\t\t * A directory is in the way. Maybe it is empty; try\n-\t\t * to remove it:\n-\t\t */\n-\t\tif (!path_copy.len)\n-\t\t\tstrbuf_addstr(&path_copy, path);\n-\n-\t\tif (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))\n-\t\t\tgoto retry_fn;\n-\t} else if (errno == ENOENT && create_directories_remaining-- > 0) {\n-\t\t/*\n-\t\t * Maybe the containing directory didn't exist, or\n-\t\t * maybe it was just deleted by a process that is\n-\t\t * racing with us to clean up empty directories. Try\n-\t\t * to create it:\n-\t\t */\n-\t\tenum scld_error scld_result;\n-\n-\t\tif (!path_copy.len)\n-\t\t\tstrbuf_addstr(&path_copy, path);\n-\n-\t\tdo {\n-\t\t\tscld_result = safe_create_leading_directories(path_copy.buf);\n-\t\t\tif (scld_result == SCLD_OK)\n-\t\t\t\tgoto retry_fn;\n-\t\t} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);\n-\t}\n-\n-out:\n-\tstrbuf_release(&path_copy);\n-\terrno = save_errno;\n-\treturn ret;\n-}\n-\n static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)\n {\n \tint i;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ab666af4b75..950f9738ae0 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -852,6 +852,115 @@ static struct ref_iterator *files_ref_iterator_begin(\n \treturn ref_iterator;\n }\n \n+/*\n+ * Callback function for raceproof_create_file(). This function is\n+ * expected to do something that makes dirname(path) permanent despite\n+ * the fact that other processes might be cleaning up empty\n+ * directories at the same time. Usually it will create a file named\n+ * path, but alternatively it could create another file in that\n+ * directory, or even chdir() into that directory. The function should\n+ * return 0 if the action was completed successfully. On error, it\n+ * should return a nonzero result and set errno.\n+ * raceproof_create_file() treats two errno values specially:\n+ *\n+ * - ENOENT -- dirname(path) does not exist. In this case,\n+ * raceproof_create_file() tries creating dirname(path)\n+ * (and any parent directories, if necessary) and calls\n+ * the function again.\n+ *\n+ * - EISDIR -- the file already exists and is a directory. In this\n+ * case, raceproof_create_file() removes the directory if\n+ * it is empty (and recursively any empty directories that\n+ * it contains) and calls the function again.\n+ *\n+ * Any other errno causes raceproof_create_file() to fail with the\n+ * callback's return value and errno.\n+ *\n+ * Obviously, this function should be OK with being called again if it\n+ * fails with ENOENT or EISDIR. In other scenarios it will not be\n+ * called again.\n+ */\n+typedef int create_file_fn(const char *path, void *cb);\n+\n+/*\n+ * Create a file in dirname(path) by calling fn, creating leading\n+ * directories if necessary. Retry a few times in case we are racing\n+ * with another process that is trying to clean up the directory that\n+ * contains path. See the documentation for create_file_fn for more\n+ * details.\n+ *\n+ * Return the value and set the errno that resulted from the most\n+ * recent call of fn. fn is always called at least once, and will be\n+ * called more than once if it returns ENOENT or EISDIR.\n+ */\n+static int raceproof_create_file(const char *path, create_file_fn fn, void *cb)\n+{\n+\t/*\n+\t * The number of times we will try to remove empty directories\n+\t * in the way of path. This is only 1 because if another\n+\t * process is racily creating directories that conflict with\n+\t * us, we don't want to fight against them.\n+\t */\n+\tint remove_directories_remaining = 1;\n+\n+\t/*\n+\t * The number of times that we will try to create the\n+\t * directories containing path. We are willing to attempt this\n+\t * more than once, because another process could be trying to\n+\t * clean up empty directories at the same time as we are\n+\t * trying to create them.\n+\t */\n+\tint create_directories_remaining = 3;\n+\n+\t/* A scratch copy of path, filled lazily if we need it: */\n+\tstruct strbuf path_copy = STRBUF_INIT;\n+\n+\tint ret, save_errno;\n+\n+\t/* Sanity check: */\n+\tassert(*path);\n+\n+retry_fn:\n+\tret = fn(path, cb);\n+\tsave_errno = errno;\n+\tif (!ret)\n+\t\tgoto out;\n+\n+\tif (errno == EISDIR && remove_directories_remaining-- > 0) {\n+\t\t/*\n+\t\t * A directory is in the way. Maybe it is empty; try\n+\t\t * to remove it:\n+\t\t */\n+\t\tif (!path_copy.len)\n+\t\t\tstrbuf_addstr(&path_copy, path);\n+\n+\t\tif (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))\n+\t\t\tgoto retry_fn;\n+\t} else if (errno == ENOENT && create_directories_remaining-- > 0) {\n+\t\t/*\n+\t\t * Maybe the containing directory didn't exist, or\n+\t\t * maybe it was just deleted by a process that is\n+\t\t * racing with us to clean up empty directories. Try\n+\t\t * to create it:\n+\t\t */\n+\t\tenum scld_error scld_result;\n+\n+\t\tif (!path_copy.len)\n+\t\t\tstrbuf_addstr(&path_copy, path);\n+\n+\t\tdo {\n+\t\t\tscld_result = safe_create_leading_directories(path_copy.buf);\n+\t\t\tif (scld_result == SCLD_OK)\n+\t\t\t\tgoto retry_fn;\n+\t\t} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);\n+\t}\n+\n+out:\n+\tstrbuf_release(&path_copy);\n+\terrno = save_errno;\n+\treturn ret;\n+}\n+\n static int remove_empty_directories(struct strbuf *path)\n {\n \t/*\n-- \n2.33.0.662.g438caf9576d\n\n"},{"id":"433390","messageId":"patch-v10-2.8-ba0f5f5fb0a-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","subject":"[PATCH v10 2/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:38Z","receivedAt":"2021-08-23T11:52:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis commit does not change code; it documents the fact that an alternate ref\nbackend does not need to return EINVAL from read_raw_ref_fn to function\nproperly.\n\nThis is correct, because refs_read_raw_ref is only called from;\n\n* resolve_ref_unsafe(), which does not care for the EINVAL errno result.\n\n* refs_verify_refname_available(), which does not inspect errno.\n\n* files-backend.c, where errno is overwritten on failure.\n\n* packed-backend.c (is_packed_transaction_needed), which calls it for the\n packed ref backend, which never emits EINVAL.\n\nA grep for EINVAL */*c reveals that no code checks errno against EINVAL after\nreading references. In addition, the refs.h file does not mention errno at all.\n\nA grep over resolve_ref_unsafe() turned up the following callers that inspect\nerrno:\n\n* sequencer.c::print_commit_summary, which uses it for die_errno\n\n* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.\n\nThe files ref backend does use EINVAL. The files backend does not call into\nthe generic API (refs_read_raw), but into the files-specific function\n(files_read_raw_ref), which we are not changing in this commit.\n\nAs the errno sideband is unintuitive and error-prone, remove EINVAL\nvalue, as a step towards getting rid of the errno sideband altogether.\n\nSpotted by Ævar Arnfjörð Bjarmason .\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/refs-internal.h | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex d496c5ed52d..1c6e5ab51d7 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -622,9 +622,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n *\n * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to\n- * EINVAL, and return -1. If there is another error reading the ref,\n- * set errno appropriately and return -1.\n+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n+ * (errno should not be ENOENT) If there is another error reading the\n+ * ref, set errno appropriately and return -1.\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n-- \n2.33.0.662.g438caf9576d\n\n"},{"id":"433391","messageId":"patch-v10-3.8-2c4c30e8e06-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","subject":"[PATCH v10 3/8] refs/files-backend: stop setting errno from lock_ref_oid_basic","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:39Z","receivedAt":"2021-08-23T11:53:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nrefs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed\nto its callers using errno.\n\nIt is safe to stop setting errno here, because the callers of this\nfile-scope static function are\n\n* files_copy_or_rename_ref()\n* files_create_symref()\n* files_reflog_expire()\n\nNone of them looks at errno after seeing a negative return from\nlock_ref_oid_basic() to make any decision, and no caller of these three\nfunctions looks at errno after they signal a failure by returning a\nnegative value. In particular,\n\n* files_copy_or_rename_ref() - here, calls are followed by error()\n(which performs I/O) or write_ref_to_lockfile() (which calls\nparse_object() which may perform I/O)\n\n* files_create_symref() - here, calls are followed by error() or\ncreate_symref_locked() (which performs I/O and does not inspect\nerrno)\n\n* files_reflog_expire() - here, calls are followed by error() or\nrefs_reflog_exists() (which calls a function in a vtable that is not\ndocumented to use and/or preserve errno)\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs/files-backend.c | 11 ++---------\n 1 file changed, 2 insertions(+), 9 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 950f9738ae0..c1794dcd295 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -982,7 +982,6 @@ static int create_reflock(const char *path, void *cb)\n \n /*\n * Locks a ref returning the lock on success and NULL on failure.\n- * On failure errno is set to something meaningful.\n */\n static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t\t\t\t\t const char *refname, int *type,\n@@ -990,7 +989,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n-\tint last_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -1001,11 +999,10 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n \t\t\t\t RESOLVE_REF_NO_RECURSE,\n \t\t\t\t &lock->old_oid, type)) {\n-\t\tlast_errno = errno;\n \t\tif (!refs_verify_refname_available(&refs->base, refname,\n \t\t\t\t\t\t NULL, NULL, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(last_errno));\n+\t\t\t\t refname, strerror(errno));\n \n \t\tgoto error_return;\n \t}\n@@ -1018,15 +1015,12 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \t */\n \tif (is_null_oid(&lock->old_oid) &&\n \t refs_verify_refname_available(refs->packed_ref_store, refname,\n-\t\t\t\t\t NULL, NULL, err)) {\n-\t\tlast_errno = ENOTDIR;\n+\t\t\t\t\t NULL, NULL, err))\n \t\tgoto error_return;\n-\t}\n \n \tlock->ref_name = xstrdup(refname);\n \n \tif (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {\n-\t\tlast_errno = errno;\n \t\tunable_to_lock_message(ref_file.buf, errno, err);\n \t\tgoto error_return;\n \t}\n@@ -1043,7 +1037,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \n out:\n \tstrbuf_release(&ref_file);\n-\terrno = last_errno;\n \treturn lock;\n }\n \n-- \n2.33.0.662.g438caf9576d\n\n"},{"id":"433392","messageId":"patch-v10-4.8-05f3a346e3b-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","subject":"[PATCH v10 4/8] refs: make errno output explicit for read_raw_ref_fn","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:40Z","receivedAt":"2021-08-23T11:53:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis makes it explicit how alternative ref backends should report errors in\nread_raw_ref_fn.\n\nread_raw_ref_fn needs to supply a credible errno for a number of cases. These\nare primarily:\n\n1) The files backend calls read_raw_ref from lock_raw_ref, and uses the\nresulting error codes to create/remove directories as needed.\n\n2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,\nreturning the last successfully resolved symref. This is necessary so\nread_raw_ref(\"HEAD\") on an empty repo returns refs/heads/main (or the default branch\ndu-jour), and we know on which branch to create the first commit.\n\nMake this information flow explicit by adding a failure_errno to the signature\nof read_raw_ref. All errnos from the files backend are still propagated\nunchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are\nrelevant.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs/debug.c | 7 +++----\n refs/files-backend.c | 29 +++++++++++++++--------------\n refs/packed-backend.c | 8 ++++----\n refs/refs-internal.h | 20 ++++++++++++--------\n 5 files changed, 35 insertions(+), 31 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 05944d8e725..da0cc82ca66 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1682,7 +1682,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, &errno);\n }\n \n /* This function needs to return a meaningful errno on failure */\ndiff --git a/refs/debug.c b/refs/debug.c\nindex bf4a82bccb6..8667c640237 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -239,15 +239,14 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,\n \n static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t struct object_id *oid, struct strbuf *referent,\n-\t\t\t unsigned int *type)\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n \tint res = 0;\n \n \toidcpy(oid, null_oid());\n-\terrno = 0;\n \tres = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \n \tif (res == 0) {\n \t\ttrace_printf_key(&trace_refs, \"read_raw_ref: %s: %s (=> %s) type %x: %d\\n\",\n@@ -255,7 +254,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t} else {\n \t\ttrace_printf_key(&trace_refs,\n \t\t\t\t \"read_raw_ref: %s: %d (errno %d)\\n\", refname,\n-\t\t\t\t res, errno);\n+\t\t\t\t res, *failure_errno);\n \t}\n \treturn res;\n }\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c1794dcd295..b3de3633969 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)\n \treturn refs->loose;\n }\n \n-static int files_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tstruct stat st;\n \tint fd;\n \tint ret = -1;\n-\tint save_errno;\n \tint remaining_retries = 3;\n \n \t*type = 0;\n@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,\n \tret = parse_loose_ref_contents(buf, oid, referent, type);\n \n out:\n-\tsave_errno = errno;\n+\t*failure_errno = errno;\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n-\terrno = save_errno;\n \treturn ret;\n }\n \n@@ -540,6 +538,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tint attempts_remaining = 3;\n \tint ret = TRANSACTION_GENERIC_ERROR;\n+\tint failure_errno;\n \n \tassert(err);\n \tfiles_assert_main_repository(refs, \"lock_raw_ref\");\n@@ -610,7 +609,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \tif (hold_lock_file_for_update_timeout(\n \t\t\t &lock->lk, ref_file.buf, LOCK_NO_DEREF,\n \t\t\t get_files_ref_lock_timeout_ms()) < 0) {\n-\t\tif (errno == ENOENT && --attempts_remaining > 0) {\n+\t\tint myerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr == ENOENT && --attempts_remaining > 0) {\n \t\t\t/*\n \t\t\t * Maybe somebody just deleted one of the\n \t\t\t * directories leading to ref_file. Try\n@@ -618,7 +619,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t */\n \t\t\tgoto retry;\n \t\t} else {\n-\t\t\tunable_to_lock_message(ref_file.buf, errno, err);\n+\t\t\tunable_to_lock_message(ref_file.buf, myerr, err);\n \t\t\tgoto error_return;\n \t\t}\n \t}\n@@ -628,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t * fear that its value will change.\n \t */\n \n-\tif (files_read_raw_ref(&refs->base, refname,\n-\t\t\t &lock->old_oid, referent, type)) {\n-\t\tif (errno == ENOENT) {\n+\tif (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,\n+\t\t\t type, &failure_errno)) {\n+\t\tif (failure_errno == ENOENT) {\n \t\t\tif (mustexist) {\n \t\t\t\t/* Garden variety missing reference. */\n \t\t\t\tstrbuf_addf(err, \"unable to resolve reference '%s'\",\n@@ -654,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t * reference named \"refs/foo/bar/baz\".\n \t\t\t\t */\n \t\t\t}\n-\t\t} else if (errno == EISDIR) {\n+\t\t} else if (failure_errno == EISDIR) {\n \t\t\t/*\n \t\t\t * There is a directory in the way. It might have\n \t\t\t * contained references that have been deleted. If\n@@ -692,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,\n \t\t\t\t\tgoto error_return;\n \t\t\t\t}\n \t\t\t}\n-\t\t} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {\n+\t\t} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': \"\n \t\t\t\t \"reference broken\", refname);\n \t\t\tgoto error_return;\n \t\t} else {\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(failure_errno));\n \t\t\tgoto error_return;\n \t\t}\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 65ba4214b8d..47247a14917 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)\n \treturn refs->snapshot;\n }\n \n-static int packed_read_raw_ref(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno)\n {\n \tstruct packed_ref_store *refs =\n \t\tpacked_downcast(ref_store, REF_STORE_READ, \"read_raw_ref\");\n@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,\n \n \tif (!rec) {\n \t\t/* refname is not a packed reference. */\n-\t\terrno = ENOENT;\n+\t\t*failure_errno = ENOENT;\n \t\treturn -1;\n \t}\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 1c6e5ab51d7..ddd6d7f8ebd 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -620,11 +620,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * properly-formatted or even safe reference name. NEITHER INPUT NOR\n * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.\n *\n- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT\n- * and return -1. If the ref exists but is neither a symbolic ref nor\n- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1\n- * (errno should not be ENOENT) If there is another error reading the\n- * ref, set errno appropriately and return -1.\n+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a\n+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in\n+ * type, and return -1 (failure_errno should not be ENOENT)\n+ *\n+ * failure_errno provides errno codes that are interpreted beyond error\n+ * reporting. The following error codes have special meaning:\n+ * * ENOENT: the ref doesn't exist\n+ * * EISDIR: ref name is a directory\n+ * * ENOTDIR: ref prefix is not a directory\n *\n * Backend-specific flags might be set in type as well, regardless of\n * outcome.\n@@ -638,9 +642,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,\n * - in all other cases, referent will be untouched, and therefore\n * refname will still be valid and unchanged.\n */\n-typedef int read_raw_ref_fn(struct ref_store *ref_store,\n-\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,\n+\t\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t\t unsigned int *type, int *failure_errno);\n \n struct ref_storage_be {\n \tstruct ref_storage_be *next;\n-- \n2.33.0.662.g438caf9576d\n\n"},{"id":"433393","messageId":"patch-v10-5.8-fa9260f25fa-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","subject":"[PATCH v10 5/8] refs: add failure_errno to refs_read_raw_ref() signature","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:41Z","receivedAt":"2021-08-23T11:53:08Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.\n\nSome of our callers explicitly do not care about the errno, rather\nthan understanding NULL let's have them declare that they don't care\nby passing in an \"ignore_errno\". There's only three of them, and using\nthat pattern will make it more obvious that they want to throw away\ndata, let's also add a comment to one of the callers about why we'd\nlike to ignore the errno.\n\nLet's not extend that to refs_resolve_ref_unsafe() itself for now, it\nhas a large set of legacy callers, so we're faking up the old \"errno\"\nbehavior for it. We can convert those callers to\nrefs_resolve_ref_unsafe_with_errno() later.\n\nWe are leaving out out the refs_read_special_head() in\nrefs_read_raw_ref() for now, as noted in the next commit moving it to\n\"failure_errno\" will require some special consideration.\n\nWe're intentionally mis-indenting the argument list of the new\nrefs_resolve_ref_unsafe_with_errno(), it will be non-static in a\nsubsequent commit, doing it this way makes that diff smaller.\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 61 ++++++++++++++++++++++++++++++-------------\n refs/files-backend.c | 10 ++++---\n refs/packed-backend.c | 7 ++---\n refs/refs-internal.h | 6 ++---\n 4 files changed, 56 insertions(+), 28 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex da0cc82ca66..4c782fa2978 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1672,30 +1672,33 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \treturn result;\n }\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type)\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno)\n {\n+\tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n \t\t\t\t\t type);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\n-\t\t\t\t\t type, &errno);\n+\t\t\t\t\t type, failure_errno);\n }\n \n-/* This function needs to return a meaningful errno on failure */\n-const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n-\t\t\t\t const char *refname,\n-\t\t\t\t int resolve_flags,\n-\t\t\t\t struct object_id *oid, int *flags)\n+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno)\n {\n \tstatic struct strbuf sb_refname = STRBUF_INIT;\n \tstruct object_id unused_oid;\n \tint unused_flags;\n \tint symref_count;\n \n+\tassert(failure_errno);\n+\n \tif (!oid)\n \t\toid = &unused_oid;\n \tif (!flags)\n@@ -1706,7 +1709,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t !refname_is_safe(refname)) {\n-\t\t\terrno = EINVAL;\n+\t\t\t*failure_errno = EINVAL;\n \t\t\treturn NULL;\n \t\t}\n \n@@ -1724,8 +1727,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \tfor (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {\n \t\tunsigned int read_flags = 0;\n \n-\t\tif (refs_read_raw_ref(refs, refname,\n-\t\t\t\t oid, &sb_refname, &read_flags)) {\n+\t\tif (refs_read_raw_ref(refs, refname, oid, &sb_refname,\n+\t\t\t\t &read_flags, failure_errno)) {\n \t\t\t*flags |= read_flags;\n \n \t\t\t/* In reading mode, refs must eventually resolve */\n@@ -1737,9 +1740,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t * may show errors besides ENOENT if there are\n \t\t\t * similarly-named refs.\n \t\t\t */\n-\t\t\tif (errno != ENOENT &&\n-\t\t\t errno != EISDIR &&\n-\t\t\t errno != ENOTDIR)\n+\t\t\tif (*failure_errno != ENOENT &&\n+\t\t\t *failure_errno != EISDIR &&\n+\t\t\t *failure_errno != ENOTDIR)\n \t\t\t\treturn NULL;\n \n \t\t\toidclr(oid);\n@@ -1766,7 +1769,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\tif (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\t\tif (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||\n \t\t\t !refname_is_safe(refname)) {\n-\t\t\t\terrno = EINVAL;\n+\t\t\t\t*failure_errno = EINVAL;\n \t\t\t\treturn NULL;\n \t\t\t}\n \n@@ -1774,10 +1777,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t}\n \t}\n \n-\terrno = ELOOP;\n+\t*failure_errno = ELOOP;\n \treturn NULL;\n }\n \n+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,\n+\t\t\t\t int resolve_flags, struct object_id *oid,\n+\t\t\t\t int *flags)\n+{\n+\tint failure_errno = 0;\n+\tconst char *refn;\n+\trefn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,\n+\t\t\t\t\t\t oid, flags, &failure_errno);\n+\tif (!refn)\n+\t\t/* For unmigrated legacy callers */\n+\t\terrno = failure_errno;\n+\treturn refn;\n+}\n+\n /* backend functions */\n int refs_init_db(struct strbuf *err)\n {\n@@ -2228,6 +2245,13 @@ int refs_verify_refname_available(struct ref_store *refs,\n \n \tstrbuf_grow(&dirname, strlen(refname) + 1);\n \tfor (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {\n+\t\t/*\n+\t\t * Just saying \"Is a directory\" when we e.g. can't\n+\t\t * lock some multi-level ref isn't very informative,\n+\t\t * the user won't be told *what* is a directory, so\n+\t\t * let's not use strerror() below.\n+\t\t */\n+\t\tint ignore_errno;\n \t\t/* Expand dirname to the new prefix, not including the trailing slash: */\n \t\tstrbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);\n \n@@ -2239,7 +2263,8 @@ int refs_verify_refname_available(struct ref_store *refs,\n \t\tif (skip && string_list_has_string(skip, dirname.buf))\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {\n+\t\tif (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,\n+\t\t\t\t &type, &ignore_errno)) {\n \t\t\tstrbuf_addf(err, _(\"'%s' exists; cannot create '%s'\"),\n \t\t\t\t dirname.buf, refname);\n \t\t\tgoto cleanup;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b3de3633969..f962495e456 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\tgoto out;\n \n \tif (lstat(path, &st) < 0) {\n+\t\tint ignore_errno;\n \t\tif (errno != ENOENT)\n \t\t\tgoto out;\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = ENOENT;\n \t\t\tgoto out;\n \t\t}\n@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \t/* Is it a directory? */\n \tif (S_ISDIR(st.st_mode)) {\n+\t\tint ignore_errno;\n \t\t/*\n \t\t * Even though there is a directory where the loose\n \t\t * ref is supposed to be, there could still be a\n \t\t * packed ref:\n \t\t */\n-\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname,\n-\t\t\t\t oid, referent, type)) {\n+\t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n+\t\t\t\t referent, type, &ignore_errno)) {\n \t\t\terrno = EISDIR;\n \t\t\tgoto out;\n \t\t}\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 47247a14917..52cdc94a26e 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \tret = 0;\n \tfor (i = 0; i < transaction->nr; i++) {\n \t\tstruct ref_update *update = transaction->updates[i];\n+\t\tint failure_errno;\n \t\tunsigned int type;\n \t\tstruct object_id oid;\n \n@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,\n \t\t\t */\n \t\t\tcontinue;\n \n-\t\tif (!refs_read_raw_ref(ref_store, update->refname,\n-\t\t\t\t &oid, &referent, &type) ||\n-\t\t errno != ENOENT) {\n+\t\tif (!refs_read_raw_ref(ref_store, update->refname, &oid,\n+\t\t\t\t &referent, &type, &failure_errno) ||\n+\t\t failure_errno != ENOENT) {\n \t\t\t/*\n \t\t\t * We have to actually delete that reference\n \t\t\t * -> this transaction is needed.\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex ddd6d7f8ebd..7ade286260c 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -149,9 +149,9 @@ struct ref_update {\n \tconst char refname[FLEX_ARRAY];\n };\n \n-int refs_read_raw_ref(struct ref_store *ref_store,\n-\t\t const char *refname, struct object_id *oid,\n-\t\t struct strbuf *referent, unsigned int *type);\n+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n+\t\t struct object_id *oid, struct strbuf *referent,\n+\t\t unsigned int *type, int *failure_errno);\n \n /*\n * Write an error to `err` and return a nonzero value iff the same\n-- \n2.33.0.662.g438caf9576d\n\n"},{"id":"433394","messageId":"patch-v10-6.8-6dae8b643ad-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","subject":"[PATCH v10 6/8] branch tests: test for errno propagating on failing read","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:42Z","receivedAt":"2021-08-23T11:53:10Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nAdd a test for \"git branch\" to cover the case where .git/refs is\nsymlinked. To check availability, refs_verify_refname_available() will\nrun refs_read_raw_ref() on each prefix, leading to a read() from\n.git/refs (which is a directory).\n\nIt would probably be more robust to re-issue the lstat() as a normal\nstat(), in which case, we would fall back to the directory case, but\nfor now let's just test for the existing behavior as-is. This test\ncovers a regression in a commit that only ever made it to \"next\", see\n[1].\n\n1. http://lore.kernel.org/git/pull.1068.git.git.1629203489546.gitgitgadget@gmail.com\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/t3200-branch.sh | 20 ++++++++++++++++++++\n 1 file changed, 20 insertions(+)\n\ndiff --git a/t/t3200-branch.sh b/t/t3200-branch.sh\nindex cc4b10236e2..9fae13c2dea 100755\n--- a/t/t3200-branch.sh\n+++ b/t/t3200-branch.sh\n@@ -731,6 +731,26 @@ test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for\n \ttest_must_fail git branch -m u v\n '\n \n+test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '\n+\ttest_when_finished \"rm -rf subdir\" &&\n+\tgit init subdir &&\n+\n+\t(\n+\t\tcd subdir &&\n+\t\tfor d in refs objects packed-refs\n+\t\tdo\n+\t\t\trm -rf .git/$d &&\n+\t\t\tln -s ../../.git/$d .git/$d\n+\t\tdone\n+\t) &&\n+\tgit --git-dir subdir/.git/ branch rename-src &&\n+\tgit rev-parse rename-src >expect &&\n+\tgit --git-dir subdir/.git/ branch -m rename-src rename-dest &&\n+\tgit rev-parse rename-dest >actual &&\n+\ttest_cmp expect actual &&\n+\tgit branch -D rename-dest\n+'\n+\n test_expect_success 'test tracking setup via --track' '\n \tgit config remote.local.url . &&\n \tgit config remote.local.fetch refs/heads/*:refs/remotes/local/* &&\n-- \n2.33.0.662.g438caf9576d\n\n"},{"id":"433395","messageId":"patch-v10-7.8-18bf4a0e97c-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","subject":"[PATCH v10 7/8] refs: explicitly return failure_errno from parse_loose_ref_contents","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:43Z","receivedAt":"2021-08-23T11:53:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThe EINVAL error from parse_loose_ref_contents is used in files-backend\nto create a custom error message.\n\nIn untangling this we discovered a tricky edge case. The\nrefs_read_special_head() function was relying on\nparse_loose_ref_contents() setting EINVAL.\n\nBy converting it to use \"saved_errno\" we can migrate away from \"errno\"\nin this part of the code entirely, and do away with an existing\n\"save_errno\" pattern, its only purpose was to not clobber the \"errno\"\nwe previously needed at the end of files_read_raw_ref().\n\nLet's assert that we can do that by not having files_read_raw_ref()\nitself operate on *failure_errno in addition to passing it on. Instead\nwe'll assert that if we return non-zero we actually do set errno, thus\nassuring ourselves and callers that they can trust the resulting\n\"failure_errno\".\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 8 +++++---\n refs/files-backend.c | 31 ++++++++++++++++++++-----------\n refs/refs-internal.h | 6 ++++--\n t/t3200-branch.sh | 1 +\n 4 files changed, 30 insertions(+), 16 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 4c782fa2978..b83fd8c36b3 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1654,7 +1654,8 @@ int for_each_fullref_in_prefixes(const char *namespace,\n \n static int refs_read_special_head(struct ref_store *ref_store,\n \t\t\t\t const char *refname, struct object_id *oid,\n-\t\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t\t int *failure_errno)\n {\n \tstruct strbuf full_path = STRBUF_INIT;\n \tstruct strbuf content = STRBUF_INIT;\n@@ -1664,7 +1665,8 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \tif (strbuf_read_file(&content, full_path.buf, 0) < 0)\n \t\tgoto done;\n \n-\tresult = parse_loose_ref_contents(content.buf, oid, referent, type);\n+\tresult = parse_loose_ref_contents(content.buf, oid, referent, type,\n+\t\t\t\t\t failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\n@@ -1679,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tassert(failure_errno);\n \tif (!strcmp(refname, \"FETCH_HEAD\") || !strcmp(refname, \"MERGE_HEAD\")) {\n \t\treturn refs_read_special_head(ref_store, refname, oid, referent,\n-\t\t\t\t\t type);\n+\t\t\t\t\t type, failure_errno);\n \t}\n \n \treturn ref_store->be->read_raw_ref(ref_store, refname, oid, referent,\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f962495e456..41efe5352b5 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tint fd;\n \tint ret = -1;\n \tint remaining_retries = 3;\n+\tint myerr = 0;\n \n \t*type = 0;\n \tstrbuf_reset(&sb_path);\n@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \n \tif (lstat(path, &st) < 0) {\n \t\tint ignore_errno;\n-\t\tif (errno != ENOENT)\n+\t\tmyerr = errno;\n+\t\terrno = 0;\n+\t\tif (myerr != ENOENT)\n \t\t\tgoto out;\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = ENOENT;\n+\t\t\tmyerr = ENOENT;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \tif (S_ISLNK(st.st_mode)) {\n \t\tstrbuf_reset(&sb_contents);\n \t\tif (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {\n-\t\t\tif (errno == ENOENT || errno == EINVAL)\n+\t\t\tmyerr = errno;\n+\t\t\terrno = 0;\n+\t\t\tif (myerr == ENOENT || myerr == EINVAL)\n \t\t\t\t/* inconsistent with lstat; retry */\n \t\t\t\tgoto stat_ref;\n \t\t\telse\n@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t */\n \t\tif (refs_read_raw_ref(refs->packed_ref_store, refname, oid,\n \t\t\t\t referent, type, &ignore_errno)) {\n-\t\t\terrno = EISDIR;\n+\t\t\tmyerr = EISDIR;\n \t\t\tgoto out;\n \t\t}\n \t\tret = 0;\n@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t */\n \tfd = open(path, O_RDONLY);\n \tif (fd < 0) {\n-\t\tif (errno == ENOENT && !S_ISLNK(st.st_mode))\n+\t\tmyerr = errno;\n+\t\tif (myerr == ENOENT && !S_ISLNK(st.st_mode))\n \t\t\t/* inconsistent with lstat; retry */\n \t\t\tgoto stat_ref;\n \t\telse\n@@ -448,26 +454,29 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t}\n \tstrbuf_reset(&sb_contents);\n \tif (strbuf_read(&sb_contents, fd, 256) < 0) {\n-\t\tint save_errno = errno;\n+\t\tmyerr = errno;\n \t\tclose(fd);\n-\t\terrno = save_errno;\n \t\tgoto out;\n \t}\n \tclose(fd);\n \tstrbuf_rtrim(&sb_contents);\n \tbuf = sb_contents.buf;\n \n-\tret = parse_loose_ref_contents(buf, oid, referent, type);\n+\tret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);\n \n out:\n-\t*failure_errno = errno;\n+\tif (ret && !myerr)\n+\t\tBUG(\"returning non-zero %d, should have set myerr!\", ret);\n+\t*failure_errno = myerr;\n+\n \tstrbuf_release(&sb_path);\n \tstrbuf_release(&sb_contents);\n \treturn ret;\n }\n \n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type)\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -486,7 +495,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n \tif (parse_oid_hex(buf, oid, &p) ||\n \t (*p != '\\0' && !isspace(*p))) {\n \t\t*type |= REF_ISBROKEN;\n-\t\terrno = EINVAL;\n+\t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n \treturn 0;\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 7ade286260c..7a3a61ac22f 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -692,10 +692,12 @@ struct ref_store {\n };\n \n /*\n- * Parse contents of a loose ref file.\n+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for\n+ * invalid contents.\n */\n int parse_loose_ref_contents(const char *buf, struct object_id *oid,\n-\t\t\t struct strbuf *referent, unsigned int *type);\n+\t\t\t struct strbuf *referent, unsigned int *type,\n+\t\t\t int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t3200-branch.sh b/t/t3200-branch.sh\nindex 9fae13c2dea..63fbd71dc5c 100755\n--- a/t/t3200-branch.sh\n+++ b/t/t3200-branch.sh\n@@ -745,6 +745,7 @@ test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '\n \t) &&\n \tgit --git-dir subdir/.git/ branch rename-src &&\n \tgit rev-parse rename-src >expect &&\n+\t# Tests a BUG() assertion in files_read_raw_ref()\n \tgit --git-dir subdir/.git/ branch -m rename-src rename-dest &&\n \tgit rev-parse rename-dest >actual &&\n \ttest_cmp expect actual &&\n-- \n2.33.0.662.g438caf9576d\n\n"},{"id":"433396","messageId":"patch-v10-8.8-7d94a32af83-20210823T114712Z-avarab@gmail.com","threadId":"55589","inReplyTo":"cover-v10-0.8-00000000000-20210823T114712Z-avarab@gmail.com","subject":"[PATCH v10 8/8] refs: make errno output explicit for refs_resolve_ref_unsafe","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-23T11:52:44Z","receivedAt":"2021-08-23T11:53:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"From: Han-Wen Nienhuys \n\nThis introduces refs_resolve_ref_unsafe_with_errno(), which makes the API\ncontract for the errno output explicit. The implementation still relies on\nthe global errno variable to ensure no side effects of this refactoring.\n\nlock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()\nthat needs error information to make logic decisions, so update that caller\n\nSigned-off-by: Han-Wen Nienhuys \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n refs.c | 2 +-\n refs.h | 11 +++++++++++\n refs/files-backend.c | 10 ++++++----\n 3 files changed, 18 insertions(+), 5 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex b83fd8c36b3..e3b6d8f8dc0 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1688,7 +1688,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,\n \t\t\t\t\t type, failure_errno);\n }\n \n-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n \t\t\t\t\t const char *refname,\n \t\t\t\t\t int resolve_flags,\n \t\t\t\t\t struct object_id *oid,\ndiff --git a/refs.h b/refs.h\nindex fda8aef1547..a5685c891a9 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,\n \t\t\t\t int resolve_flags,\n \t\t\t\t struct object_id *oid,\n \t\t\t\t int *flags);\n+/**\n+ * refs_resolve_ref_unsafe_with_errno() is like\n+ * refs_resolve_ref_unsafe(), but provide access to errno code that\n+ * lead to a failure. We guarantee that errno is set to a meaningful\n+ * value on non-zero return.\n+ */\n+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,\n+\t\t\t\t\t const char *refname,\n+\t\t\t\t\t int resolve_flags,\n+\t\t\t\t\t struct object_id *oid,\n+\t\t\t\t\t int *flags, int *failure_errno);\n const char *resolve_ref_unsafe(const char *refname, int resolve_flags,\n \t\t\t struct object_id *oid, int *flags);\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 41efe5352b5..a14bb6eb96e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1001,6 +1001,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n {\n \tstruct strbuf ref_file = STRBUF_INIT;\n \tstruct ref_lock *lock;\n+\tint resolve_errno = 0;\n \n \tfiles_assert_main_repository(refs, \"lock_ref_oid_basic\");\n \tassert(err);\n@@ -1008,13 +1009,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,\n \tCALLOC_ARRAY(lock, 1);\n \n \tfiles_ref_path(refs, &ref_file, refname);\n-\tif (!refs_resolve_ref_unsafe(&refs->base, refname,\n-\t\t\t\t RESOLVE_REF_NO_RECURSE,\n-\t\t\t\t &lock->old_oid, type)) {\n+\tif (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,\n+\t\t\t\t\t\tRESOLVE_REF_NO_RECURSE,\n+\t\t\t\t\t\t&lock->old_oid, type,\n+\t\t\t\t\t\t&resolve_errno)) {\n \t\tif (!refs_verify_refname_available(&refs->base, refname,\n \t\t\t\t\t\t NULL, NULL, err))\n \t\t\tstrbuf_addf(err, \"unable to resolve reference '%s': %s\",\n-\t\t\t\t refname, strerror(errno));\n+\t\t\t\t refname, strerror(resolve_errno));\n \n \t\tgoto error_return;\n \t}\n-- \n2.33.0.662.g438caf9576d\n\n"}]}