git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 13/21] refs: resolve symbolic refs first

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
Feb 12, 2016, 14:09 UTC
Message-ID
<56BDE780.2030407@alum.mit.edu>
In-Reply-To
<1454701462-3817-14-git-send-email-dturner@twopensource.com>
On 02/05/2016 08:44 PM, David Turner wrote:
Show 9 quoted lines
> Before committing ref updates, split symbolic ref updates into two
> parts: an update to the underlying ref, and a log-only update to the
> symbolic ref.  This ensures that both references are locked correctly
> while their reflogs are updated.
> 
> It is still possible to confuse git by concurrent updates, since the
> splitting of symbolic refs does not happen under lock. So a symbolic ref
> could be replaced by a plain ref in the middle of this operation, which
> would lead to reflog discontinuities and missed old-ref checks.
This patch is doing too much at once for my little brain to follow.

My first hangup is the change to setting RESOLVE_REF_NO_RECURSE unconditionally in lock_ref_sha1_basic(). I count five callers of that function and see no justification for why the change is OK in the context of each caller. Here are some thoughts:

* The call from files_create_symref() sets REF_NODEREF, so it is
unaffected by this change.
* The call from files_transaction_commit() is preceded by a call to
dereference_symrefs(), which I assume effectively replaces the need for
RESOLVE_REF_NO_RECURSE.
* There are two calls from files_rename_ref(). Why is it OK to do
without RESOLVE_REF_NO_RECURSE there?
  * For the oldrefname call, I suppose the justification is the "(flag &
REF_ISSYMREF)" check earlier in the function. (But does this introduce a
significant TOCTOU race?)
  * For the newrefname call, I suppose it's because the code a little
higher up tries to delete any existing reference with that name. It
looks to me like the old code was slightly broken: if newrefname was an
unborn symbolic reference, then: read_ref_full() would fail;
delete_ref() would be skipped; lock_ref_sha1_basic() would lock the
*referred-to* reference; the referred-to reference would be overwritten
instead of newrefname. So it could be that here REF_NODEREF indirectly
fixes a bug?
* The last call, from files_reflog_expire(), is also questionable before
your patch. If refname is a symref, then the function is expiring the
reflog of the symref. But (before this patch) it locks not the symref
but its referent. This was discussed in some length before on the
mailing list [1], and the conclusion was that the current behavior is
wrong, but for backwards compatibility reasons it would be safest to
change it to locking *both* the symref and its referent.

If possible, it would be better to split this patch up into several: the first few would each add the REF_NODEREF flag at one callsite, with a careful justification of why that is OK. Once all the callsites (except the one in files_transaction_commit()) have been changed, then the last patch could add the dereference_symrefs() machinery and change the last callsite.

(I'm not certain that those steps are actually doable independently, given that REF_NODEREF has other effects besides setting RESOLVE_REF_NO_RECURSE.)

I'm not just being pedantic here. The patch as written is really too big to review effectively.

Michael

[1] http://thread.gmane.org/gmane.comp.version-control.git/263552/focus=263555

-- 
Michael Haggerty
mhagger@alum.mit.edu
Previous: David TurnerNext: David Turner
Message 25 of 53 in “refs backend”
  1. 00/20 refs backendDavid Turner, Feb 5, 2016
  2. 01/21 refs: add a backend method structure with transaction functionsDavid Turner, Feb 5, 2016
  3. 02/21 refs: add methods for misc ref operationsDavid Turner, Feb 5, 2016
  4. Michael HaggertyFeb 11, 2016
  5. David TurnerFeb 12, 2016
  6. 03/21 refs: add methods for the ref iteratorsDavid Turner, Feb 5, 2016
  7. Michael HaggertyFeb 11, 2016
  8. David TurnerFeb 12, 2016
  9. 04/21 refs: add do_for_each_per_worktree_refDavid Turner, Feb 5, 2016
  10. 05/21 refs: add methods for reflogDavid Turner, Feb 5, 2016
  11. 06/21 refs: add method for initial ref transaction commitDavid Turner, Feb 5, 2016
  12. 07/21 refs: add method for delete_refsDavid Turner, Feb 5, 2016
  13. 08/21 refs: add methods to init refs dbDavid Turner, Feb 5, 2016
  14. Michael HaggertyFeb 11, 2016
  15. David TurnerFeb 11, 2016
  16. 09/21 refs: add method to rename refsDavid Turner, Feb 5, 2016
  17. Michael HaggertyFeb 11, 2016
  18. David TurnerFeb 11, 2016
  19. 10/21 refs: make lock genericDavid Turner, Feb 5, 2016
  20. 11/21 refs: move duplicate check to common codeDavid Turner, Feb 5, 2016
  21. 12/21 refs: allow log-only updatesDavid Turner, Feb 5, 2016
  22. Michael HaggertyFeb 11, 2016
  23. David TurnerFeb 11, 2016
  24. 13/21 refs: resolve symbolic refs firstDavid Turner, Feb 5, 2016
  25. Michael HaggertyFeb 12, 2016
  26. David TurnerFeb 18, 2016
  27. Michael HaggertyFeb 18, 2016
  28. 14/21 refs: always handle non-normal refs in files backendDavid Turner, Feb 5, 2016
  29. Michael HaggertyFeb 12, 2016
  30. David TurnerFeb 18, 2016
  31. Michael HaggertyFeb 18, 2016
  32. David TurnerFeb 18, 2016
  33. 15/21 init: allow alternate ref strorage to be set for new reposDavid Turner, Feb 5, 2016
  34. Michael HaggertyFeb 12, 2016
  35. David TurnerFeb 17, 2016
  36. Michael HaggertyFeb 18, 2016
  37. 16/21 refs: check submodules ref storage configDavid Turner, Feb 5, 2016
  38. 17/21 clone: allow ref storage backend to be set for cloneDavid Turner, Feb 5, 2016
  39. 18/21 svn: learn ref-storage argumentDavid Turner, Feb 5, 2016
  40. 19/21 refs: add register_ref_storage_backends()David Turner, Feb 5, 2016
  41. Michael HaggertyFeb 12, 2016
  42. David TurnerFeb 17, 2016
  43. 20/21 refs: add LMDB refs storage backendDavid Turner, Feb 5, 2016
  44. Michael HaggertyFeb 11, 2016
  45. David TurnerFeb 11, 2016
  46. Michael HaggertyFeb 12, 2016
  47. David TurnerFeb 13, 2016
  48. Duy NguyenFeb 14, 2016
  49. Duy NguyenFeb 15, 2016
  50. David TurnerFeb 16, 2016
  51. David TurnerFeb 17, 2016
  52. 21/21 refs: tests for lmdb backendDavid Turner, Feb 5, 2016
  53. Junio C HamanoFeb 8, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.