git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] refs file backend: remove dead "errno == EISDIR" code

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Jul 14, 2021, 19:07 UTC
Message-ID
<871r801yp6.fsf@evledraar.gmail.com>
In-Reply-To
<YO8PBBJZ2Q+5ZqFs@coredump.intra.peff.net>
On Wed, Jul 14 2021, Jeff King wrote:
Show 46 quoted lines
> On Wed, Jul 14, 2021 at 01:17:14PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> Since a1c1d8170d (refs_resolve_ref_unsafe: handle d/f conflicts for
>> writes, 2017-10-06) we don't, because our our callstack will look
>> something like:
>> 
>>     files_copy_or_rename_ref() -> lock_ref_oid_basic() -> refs_resolve_ref_unsafe()
>> 
>> And then the refs_resolve_ref_unsafe() call here will in turn (in the
>> code added in a1c1d8170d) do the equivalent of this (via a call to
>> refs_read_raw_ref()):
>> 
>> 	/* Via refs_read_raw_ref() */
>> 	fd = open(path, O_RDONLY);
>> 	if (fd < 0)
>> 		/* get errno == EISDIR */
>> 	/* later, in refs_resolve_ref_unsafe() */
>> 	if ([...] && errno != EISDIR)
>> 		return NULL;
>> 	[...]
>> 	/* returns the refs/heads/foo to the caller, even though it's a directory */
>> 	return refname;
>
> Isn't that pseudo-code missing a conditional that's there in the real
> code? In refs_resolve_ref_unsafe(), I see:
>
>        if (refs_read_raw_ref(refs, refname,
>                              oid, &sb_refname, &read_flags)) {
>                *flags |= read_flags;
>
>                /* In reading mode, refs must eventually resolve */
>                if (resolve_flags & RESOLVE_REF_READING)
>                        return NULL;
>
>                /*
>                 * Otherwise a missing ref is OK. But the files backend
>                 * may show errors besides ENOENT if there are
>                 * similarly-named refs.
>                 */
>                if (errno != ENOENT &&
>                    errno != EISDIR &&
>                    errno != ENOTDIR)
>                        return NULL;
>
> So if RESOLVE_REF_READING is set, we can return NULL immediately, with
> errno set to EISDIR. Which contradicts this:

I opted (perhaps unwisely) to elide that since as you note above we don't take that path in relation to the removed code. I.e. I'm describing the relevant codepath we take nowadays given the code & its callers.

But will reword etc., thanks.
Show 16 quoted lines
>> I.e. even though we got an "errno == EISDIR" we won't take this
>> branch, since in cases of EISDIR "resolved" is always
>> non-NULL. I.e. we pretend at this point as though everything's OK and
>> there is no "foo" directory.
>
> So when is RESOLVE_REF_READING set? The resolve_flags parameter is
> passed in by the caller. In lock_ref_oid_basic(), it comes from this:
>
>     int mustexist = (old_oid && !is_null_oid(old_oid));
>     [...]
>     if (mustexist)
>             resolve_flags |= RESOLVE_REF_READING;
>
> So do any callers pass in old_oid? Surprisingly few. It used to be
> called from other locking functions, but these days it looks like it is
> only files_reflog_expire().
In general (and not being too familiar with this area) and per:
    7521cc4611 (refs.c: make delete_ref use a transaction, 2014-04-30)
    92b1551b1d (refs: resolve symbolic refs first, 2016-04-25)
    029cdb4ab2 (refs.c: make prune_ref use a transaction to delete the ref, 2014-04-30)
And:
    https://lore.kernel.org/git/20140902205841.GA18279@google.com/    

I wonder if these remaining cases can be migrated over to lock_raw_ref() or the transaction API, as many other similar callers have been already.

But that's a bigger change, I won't be doing that now, just wondering if these are some #leftoverbits or if there's a good reason they were left.

Show 7 quoted lines
> I'm not sure if this case is important or not. If we're expecting the
> ref to exist, then an in-the-way directory is going to mean failure
> either way. It could still exist within the packed-refs file, but then
> refs_read_raw_ref() would not return failure.
>
> So...I think it's fine? But the argument in your commit message seems to
> have missed this case entirely.

Perhaps more succinctly: If we have a directory in the way, it's going to be impossible for the "old_oid" condition to be satisfied in any case in the file backend.

Even if we still had a caller that did "care" about that what could they hope to get from an "old_oid=<some-OID>" for a lock on "foo/bar" where "foo" is an empty directory?

Except of course for the case where it's not a directory but packed, but as you noted that's handled in another case.

Perhaps it's informative that the below diff-on-top also passes all tests, i.e. that we have largely the same "refs_read_raw_ref(refs->packed_ref_store" copy/pasted in files_read_raw_ref() in two adjacent places, we're just changing what errno we pass upwards.

It thoroughly tramples on Han-Wen's series, and it's easier to deal with (if at all) once his lands, just thought it might be interesting:

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7e4963fd07..4a97cd48d9 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -356,6 +356,8 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	int ret = -1;
 	int save_errno;
 	int remaining_retries = 3;
+	int lstat_bad_or_not_file = 0;
+	int lstat_errno = 0;
 
 	*type = 0;
 	strbuf_reset(&sb_path);
@@ -382,11 +384,28 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 		goto out;
 
 	if (lstat(path, &st) < 0) {
-		if (errno != ENOENT)
+		lstat_bad_or_not_file = 1;
+		lstat_errno = errno;
+	} else if (S_ISDIR(st.st_mode)) {
+		/*
+		 * Maybe it's an empty directory, maybe it's not, in
+		 * either case this ref does not exist in the files
+		 * backend (but may be packet), later code will handle
+		 * the "create and maybe remove_empty_directories()"
+		 * case if needed, or die otherwise.
+		 */
+		lstat_bad_or_not_file = 1;
+	}
+
+	if (lstat_bad_or_not_file) {
+		if (lstat_errno && lstat_errno != ENOENT)
 			goto out;
 		if (refs_read_raw_ref(refs->packed_ref_store, refname,
 				      oid, referent, type)) {
-			errno = ENOENT;
+			if (lstat_errno)
+				errno = ENOENT;
+			else
+				errno = EISDIR;
 			goto out;
 		}
 		ret = 0;
@@ -417,22 +436,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 		 */
 	}
 
-	/* Is it a directory? */
-	if (S_ISDIR(st.st_mode)) {
-		/*
-		 * Even though there is a directory where the loose
-		 * ref is supposed to be, there could still be a
-		 * packed ref:
-		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
-			errno = EISDIR;
-			goto out;
-		}
-		ret = 0;
-		goto out;
-	}
-
 	/*
 	 * Anything else, just open it and try to use it as
 	 * a ref
Previous: Jeff KingNext: Jeff King
Message 3 of 88 in “refs file backend: remove dead "errno == EISDIR" code”
  1. refs file backend: remove dead "errno == EISDIR" codeÆvar Arnfjörð Bjarmason, Jul 14, 2021
  2. Jeff KingJul 14, 2021
  3. Ævar Arnfjörð BjarmasonJul 14, 2021
  4. Jeff KingJul 14, 2021
  5. Ævar Arnfjörð BjarmasonJul 15, 2021
  6. Jeff KingJul 15, 2021
  7. Junio C HamanoJul 17, 2021
  8. Jeff KingJul 17, 2021
  9. Junio C HamanoJul 19, 2021
  10. Junio C HamanoJul 19, 2021
  11. Ævar Arnfjörð BjarmasonJul 17, 2021
  12. 00/11 fix "git reflog expire" race & get rid of EISDIR in refs APIÆvar Arnfjörð Bjarmason, Jul 16, 2021
  13. 01/11 refs/packet: add missing BUG() invocations to reflog callbacksÆvar Arnfjörð Bjarmason, Jul 16, 2021
  14. 02/11 refs/files: remove unused REF_DELETING in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 16, 2021
  15. Jeff KingJul 17, 2021
  16. Junio C HamanoJul 19, 2021
  17. Jeff KingJul 20, 2021
  18. 03/11 refs/files: remove unused "extras/skip" in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 16, 2021
  19. 04/11 refs/files: remove unused "skip" in lock_raw_ref() tooÆvar Arnfjörð Bjarmason, Jul 16, 2021
  20. 05/11 refs/debug: re-indent argument list for "prepare"Ævar Arnfjörð Bjarmason, Jul 16, 2021
  21. 06/11 refs API: pass the "lock OID" to reflog "prepare"Ævar Arnfjörð Bjarmason, Jul 16, 2021
  22. Jeff KingJul 17, 2021
  23. Junio C HamanoJul 19, 2021
  24. Ævar Arnfjörð BjarmasonJul 19, 2021
  25. 07/11 refs: make repo_dwim_log() accept a NULL oidÆvar Arnfjörð Bjarmason, Jul 16, 2021
  26. 08/11 refs/files: add a comment about refs_reflog_exists() callÆvar Arnfjörð Bjarmason, Jul 16, 2021
  27. Jeff KingJul 17, 2021
  28. Junio C HamanoJul 19, 2021
  29. Jeff KingJul 20, 2021
  30. 09/11 reflog expire: don't lock reflogs using previously seen OIDÆvar Arnfjörð Bjarmason, Jul 16, 2021
  31. Jeff KingJul 17, 2021
  32. Han-Wen NienhuysAug 17, 2021
  33. Junio C HamanoAug 18, 2021
  34. Carlo Marcelo Arenas BelónAug 19, 2021
  35. Junio C HamanoAug 20, 2021
  36. 10/11 refs/files: remove unused "oid" in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 16, 2021
  37. Jeff KingJul 17, 2021
  38. 11/11 refs/files: remove unused "errno == EISDIR" codeÆvar Arnfjörð Bjarmason, Jul 16, 2021
  39. Jeff KingJul 17, 2021
  40. Jeff KingJul 17, 2021
  41. 00/12 fix "git reflog expire" race & get rid of EISDIR in refs APIÆvar Arnfjörð Bjarmason, Jul 20, 2021
  42. 10/12 refs/files: remove unused "oid" in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 20, 2021
  43. 05/12 refs/debug: re-indent argument list for "prepare"Ævar Arnfjörð Bjarmason, Jul 20, 2021
  44. 08/12 refs/files: add a comment about refs_reflog_exists() callÆvar Arnfjörð Bjarmason, Jul 20, 2021
  45. 07/12 refs: make repo_dwim_log() accept a NULL oidÆvar Arnfjörð Bjarmason, Jul 20, 2021
  46. 02/12 refs/files: remove unused REF_DELETING in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 20, 2021
  47. Junio C HamanoAug 2, 2021
  48. 01/12 refs/packet: add missing BUG() invocations to reflog callbacksÆvar Arnfjörð Bjarmason, Jul 20, 2021
  49. 11/12 refs/files: remove unused "errno == EISDIR" codeÆvar Arnfjörð Bjarmason, Jul 20, 2021
  50. 09/12 reflog expire: don't lock reflogs using previously seen OIDÆvar Arnfjörð Bjarmason, Jul 20, 2021
  51. 03/12 refs/files: remove unused "extras/skip" in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 20, 2021
  52. 06/12 refs API: pass the "lock OID" to reflog "prepare"Ævar Arnfjörð Bjarmason, Jul 20, 2021
  53. Junio C HamanoJul 21, 2021
  54. Junio C HamanoJul 21, 2021
  55. 12/12 refs/files: remove unused "errno != ENOTDIR" conditionÆvar Arnfjörð Bjarmason, Jul 20, 2021
  56. 04/12 refs/files: remove unused "skip" in lock_raw_ref() tooÆvar Arnfjörð Bjarmason, Jul 20, 2021
  57. 00/11 fix "git reflog expire" race & get rid of EISDIR in refs APIÆvar Arnfjörð Bjarmason, Jul 26, 2021
  58. 02/11 refs/files: remove unused REF_DELETING in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 26, 2021
  59. 01/11 refs/packet: add missing BUG() invocations to reflog callbacksÆvar Arnfjörð Bjarmason, Jul 26, 2021
  60. 03/11 refs/files: remove unused "extras/skip" in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 26, 2021
  61. 05/11 refs/debug: re-indent argument list for "prepare"Ævar Arnfjörð Bjarmason, Jul 26, 2021
  62. 04/11 refs/files: remove unused "skip" in lock_raw_ref() tooÆvar Arnfjörð Bjarmason, Jul 26, 2021
  63. 07/11 refs/files: add a comment about refs_reflog_exists() callÆvar Arnfjörð Bjarmason, Jul 26, 2021
  64. 09/11 refs/files: remove unused "oid" in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Jul 26, 2021
  65. 08/11 reflog expire: don't lock reflogs using previously seen OIDÆvar Arnfjörð Bjarmason, Jul 26, 2021
  66. Junio C HamanoAug 2, 2021
  67. Ævar Arnfjörð BjarmasonAug 4, 2021
  68. 10/11 refs/files: remove unused "errno == EISDIR" codeÆvar Arnfjörð Bjarmason, Jul 26, 2021
  69. 11/11 refs/files: remove unused "errno != ENOTDIR" conditionÆvar Arnfjörð Bjarmason, Jul 26, 2021
  70. 06/11 refs: make repo_dwim_log() accept a NULL oidÆvar Arnfjörð Bjarmason, Jul 26, 2021
  71. 00/13 fix "git reflog expire" race & get rid of EISDIR in refs APIÆvar Arnfjörð Bjarmason, Aug 23, 2021
  72. 01/13 refs/packet: add missing BUG() invocations to reflog callbacksÆvar Arnfjörð Bjarmason, Aug 23, 2021
  73. 02/13 refs/files: remove unused REF_DELETING in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Aug 23, 2021
  74. 03/13 refs: drop unused "flags" parameter to lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Aug 23, 2021
  75. 04/13 refs/files: remove unused "extras/skip" in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Aug 23, 2021
  76. 05/13 refs/files: remove unused "skip" in lock_raw_ref() tooÆvar Arnfjörð Bjarmason, Aug 23, 2021
  77. 06/13 refs/debug: re-indent argument list for "prepare"Ævar Arnfjörð Bjarmason, Aug 23, 2021
  78. 07/13 refs: make repo_dwim_log() accept a NULL oidÆvar Arnfjörð Bjarmason, Aug 23, 2021
  79. 08/13 refs/files: add a comment about refs_reflog_exists() callÆvar Arnfjörð Bjarmason, Aug 23, 2021
  80. 09/13 reflog expire: don't lock reflogs using previously seen OIDÆvar Arnfjörð Bjarmason, Aug 23, 2021
  81. 10/13 refs API: remove OID argument to reflog_expire()Ævar Arnfjörð Bjarmason, Aug 23, 2021
  82. 11/13 refs/files: remove unused "oid" in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason, Aug 23, 2021
  83. 12/13 refs/files: remove unused "errno == EISDIR" codeÆvar Arnfjörð Bjarmason, Aug 23, 2021
  84. 13/13 refs/files: remove unused "errno != ENOTDIR" conditionÆvar Arnfjörð Bjarmason, Aug 23, 2021
  85. Ævar Arnfjörð BjarmasonJul 23, 2021
  86. Junio C HamanoJul 23, 2021
  87. Ævar Arnfjörð BjarmasonJul 26, 2021
  88. Junio C HamanoJul 26, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.