git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 25/27] attr: store attribute stack in attr_check structure

From
BWBrandon Williams <bmwill@google.com>
Date
Jan 25, 2017, 21:54 UTC
Message-ID
<20170125215426.GB83343@google.com>
In-Reply-To
<xmqq1svqc2ww.fsf@gitster.mtv.corp.google.com>
On 01/25, Junio C Hamano wrote:
Show 59 quoted lines
> Brandon Williams <bmwill@google.com> writes:
> 
> >> In my mind, the value of having a constant check_attr is primarily
> >> that it gives us a stable pointer to serve as a hashmap key,
> >> i.e. the identifier for each call site, in a later iteration.
> >
> > We didn't really discuss this notion of having the pointer be a key into
> > a hashmap, what sort of information are you envisioning being stored in
> > this sort of hashmap?
> 
> The "entries relevant to this attr_check() call, that is specific to
> the <check_attr instance, the thread> tuple" (aka "what used to be
> called the global attr_stack") we discussed would be the primary
> example.  A thread is likely be looping in a caller that has many
> paths inside a directory, calling a function that has a call to
> attr_check() for each path.  Having something that can use to
> identify the check_attr instance in a stable way, even when the
> inner function is called and returns many times, would allow us to
> populate the "attr stack" just once for the thread when it enters a
> directory for the first time (remember, another thread may be
> executing the same codepath, checking for paths in a different
> directory) and keep using it.  There may be other mechanisms you can
> come up with, so I wouldn't say it is the only valid way, but it is
> a way.  That is why I said:
> 
> >> But all of the above comes from my intuition, and I'll very much
> >> welcome to be proven wrong with an alternative design, or better
> >> yet, a working code based on an alternative design ;-).
> 
> near the end of my message.
> 
> > One issue I can see with this is that the
> > functions which have a static attr_check struct would still not be thread
> > safe if the initialization of the structure isn't surrounded by a mutex
> > itself. ie
> 
> Yes, that goes without saying.  That is why I suggested Stefan to do
> not this:
> 
> > static struct attr_check *check;
> >
> > if (!check)
> >   init(check);
> >
> > would need to be:
> >
> > lock()
> > if (!check)
> >   init(check);
> > unlock();
> 
> but this:
> 
> 	static struct attr_check *check;
> 	init(&check);
> 
> and hide the lock/unlock gymnastics inside the API.  I thought that
> already was in what you inherited from him and started your work
> on top of?

I essentially built off of the series you had while using Stefan's patches as inspiration, but I don't believe the kind of mechanism you are describing existed in Stefan's series. His series had a single lock for the entire system, only allowing a single caller to be in it at any given time. This definitely isn't ideal, hence why I picked it up.

Implementation aside I want to try and nail down what the purpose of this refactor is. There are roughly two notions of being "thread-safe".

1. The first is that the subsystem itself is thread safe, that is
   multiple threads can be executing inside the subsystem without stepping
   on each others work.
2. The second is that the object itself is thread safe or that multiple
   threads can use the same object.

I thought that the main purpose of this was to achieve (1) since currently that is not the case.

-- 
Brandon Williams
Previous: Junio C HamanoNext: Brandon Williams
Message 75 of 111 in “Revamp the attribute system; another round”
  1. 00/27 Revamp the attribute system; another roundBrandon Williams, Jan 12, 2017
  2. 20/27 attr: change validity check for attribute names to use positive logicBrandon Williams, Jan 12, 2017
  3. 19/27 attr: pass struct attr_check to collect_some_attrsBrandon Williams, Jan 12, 2017
  4. 18/27 attr: retire git_check_attrs() APIBrandon Williams, Jan 12, 2017
  5. 17/27 attr: convert git_check_attrs() callers to use the new APIBrandon Williams, Jan 12, 2017
  6. 15/27 attr: (re)introduce git_check_attr() and struct attr_checkBrandon Williams, Jan 12, 2017
  7. 16/27 attr: convert git_all_attrs() to use "struct attr_check"Brandon Williams, Jan 12, 2017
  8. 23/27 attr: remove maybe-real, maybe-macro from git_attrBrandon Williams, Jan 12, 2017
  9. 24/27 attr: tighten const correctness with git_attr and match_attrBrandon Williams, Jan 12, 2017
  10. 21/27 attr: use hashmap for attribute dictionaryBrandon Williams, Jan 12, 2017
  11. Stefan BellerJan 18, 2017
  12. Brandon WilliamsJan 18, 2017
  13. 25/27 attr: store attribute stacks in hashmapBrandon Williams, Jan 12, 2017
  14. Junio C HamanoJan 13, 2017
  15. Brandon WilliamsJan 18, 2017
  16. Brandon WilliamsJan 23, 2017
  17. Stefan BellerJan 18, 2017
  18. Brandon WilliamsJan 18, 2017
  19. Stefan BellerJan 18, 2017
  20. 22/27 attr: eliminate global check_all_attr arrayBrandon Williams, Jan 12, 2017
  21. 27/27 attr: reformat git_attr_set_direction() functionBrandon Williams, Jan 12, 2017
  22. 26/27 attr: push the bare repo check into read_attr()Brandon Williams, Jan 12, 2017
  23. 13/27 attr.c: outline the future plans by heavily commentingBrandon Williams, Jan 12, 2017
  24. 12/27 Documentation: fix a typoBrandon Williams, Jan 12, 2017
  25. 01/27 commit.c: use strchrnul() to scan for one lineBrandon Williams, Jan 12, 2017
  26. 02/27 attr.c: use strchrnul() to scan for one lineBrandon Williams, Jan 12, 2017
  27. 11/27 attr.c: add push_stack() helperBrandon Williams, Jan 12, 2017
  28. 08/27 attr.c: tighten constness around "git_attr" structureBrandon Williams, Jan 12, 2017
  29. 14/27 attr: rename function and struct related to checking attributesBrandon Williams, Jan 12, 2017
  30. 06/27 attr.c: mark where #if DEBUG ends more clearlyBrandon Williams, Jan 12, 2017
  31. 07/27 attr.c: simplify macroexpand_one()Brandon Williams, Jan 12, 2017
  32. 10/27 attr: support quoting pathname patterns in C styleBrandon Williams, Jan 12, 2017
  33. 09/27 attr.c: plug small leak in parse_attr_line()Brandon Williams, Jan 12, 2017
  34. 04/27 attr.c: explain the lack of attr-name syntax check in parse_attr()Brandon Williams, Jan 12, 2017
  35. 05/27 attr.c: complete a sentence in a commentBrandon Williams, Jan 12, 2017
  36. 03/27 attr.c: update a stale comment on "struct match_attr"Brandon Williams, Jan 12, 2017
  37. Junio C HamanoJan 15, 2017
  38. Jeff KingJan 16, 2017
  39. 00/27 Revamp the attribute system; another roundBrandon Williams, Jan 23, 2017
  40. 01/27 commit.c: use strchrnul() to scan for one lineBrandon Williams, Jan 23, 2017
  41. 02/27 attr.c: use strchrnul() to scan for one lineBrandon Williams, Jan 23, 2017
  42. 03/27 attr.c: update a stale comment on "struct match_attr"Brandon Williams, Jan 23, 2017
  43. 04/27 attr.c: explain the lack of attr-name syntax check in parse_attr()Brandon Williams, Jan 23, 2017
  44. 05/27 attr.c: complete a sentence in a commentBrandon Williams, Jan 23, 2017
  45. 07/27 attr.c: simplify macroexpand_one()Brandon Williams, Jan 23, 2017
  46. 10/27 attr: support quoting pathname patterns in C styleBrandon Williams, Jan 23, 2017
  47. 12/27 Documentation: fix a typoBrandon Williams, Jan 23, 2017
  48. 13/27 attr.c: outline the future plans by heavily commentingBrandon Williams, Jan 23, 2017
  49. 09/27 attr.c: plug small leak in parse_attr_line()Brandon Williams, Jan 23, 2017
  50. 11/27 attr.c: add push_stack() helperBrandon Williams, Jan 23, 2017
  51. 14/27 attr: rename function and struct related to checking attributesBrandon Williams, Jan 23, 2017
  52. 20/27 attr: change validity check for attribute names to use positive logicBrandon Williams, Jan 23, 2017
  53. 21/27 attr: use hashmap for attribute dictionaryBrandon Williams, Jan 23, 2017
  54. 22/27 attr: eliminate global check_all_attr arrayBrandon Williams, Jan 23, 2017
  55. Junio C HamanoJan 23, 2017
  56. 16/27 attr: convert git_all_attrs() to use "struct attr_check"Brandon Williams, Jan 23, 2017
  57. 23/27 attr: remove maybe-real, maybe-macro from git_attrBrandon Williams, Jan 23, 2017
  58. 24/27 attr: tighten const correctness with git_attr and match_attrBrandon Williams, Jan 23, 2017
  59. 17/27 attr: convert git_check_attrs() callers to use the new APIBrandon Williams, Jan 23, 2017
  60. 18/27 attr: retire git_check_attrs() APIBrandon Williams, Jan 23, 2017
  61. 15/27 attr: (re)introduce git_check_attr() and struct attr_checkBrandon Williams, Jan 23, 2017
  62. 06/27 attr.c: mark where #if DEBUG ends more clearlyBrandon Williams, Jan 23, 2017
  63. 08/27 attr.c: tighten constness around "git_attr" structureBrandon Williams, Jan 23, 2017
  64. 19/27 attr: pass struct attr_check to collect_some_attrsBrandon Williams, Jan 23, 2017
  65. 26/27 attr: push the bare repo check into read_attr()Brandon Williams, Jan 23, 2017
  66. 27/27 attr: reformat git_attr_set_direction() functionBrandon Williams, Jan 23, 2017
  67. 25/27 attr: store attribute stack in attr_check structureBrandon Williams, Jan 23, 2017
  68. Junio C HamanoJan 23, 2017
  69. Brandon WilliamsJan 23, 2017
  70. Brandon WilliamsJan 24, 2017
  71. Junio C HamanoJan 24, 2017
  72. Brandon WilliamsJan 25, 2017
  73. Stefan BellerJan 25, 2017
  74. Junio C HamanoJan 25, 2017
  75. Brandon WilliamsJan 25, 2017
  76. Brandon WilliamsJan 25, 2017
  77. 00/27 Revamp the attribute system; another roundBrandon Williams, Jan 28, 2017
  78. 01/27 commit.c: use strchrnul() to scan for one lineBrandon Williams, Jan 28, 2017
  79. 02/27 attr.c: use strchrnul() to scan for one lineBrandon Williams, Jan 28, 2017
  80. 05/27 attr.c: complete a sentence in a commentBrandon Williams, Jan 28, 2017
  81. 03/27 attr.c: update a stale comment on "struct match_attr"Brandon Williams, Jan 28, 2017
  82. 07/27 attr.c: simplify macroexpand_one()Brandon Williams, Jan 28, 2017
  83. 06/27 attr.c: mark where #if DEBUG ends more clearlyBrandon Williams, Jan 28, 2017
  84. 08/27 attr.c: tighten constness around "git_attr" structureBrandon Williams, Jan 28, 2017
  85. 04/27 attr.c: explain the lack of attr-name syntax check in parse_attr()Brandon Williams, Jan 28, 2017
  86. 09/27 attr.c: plug small leak in parse_attr_line()Brandon Williams, Jan 28, 2017
  87. 10/27 attr: support quoting pathname patterns in C styleBrandon Williams, Jan 28, 2017
  88. 11/27 attr.c: add push_stack() helperBrandon Williams, Jan 28, 2017
  89. 12/27 Documentation: fix a typoBrandon Williams, Jan 28, 2017
  90. 15/27 attr: (re)introduce git_check_attr() and struct attr_checkBrandon Williams, Jan 28, 2017
  91. 15/27 attr: (re)introduce git_check_attr() and struct attr_checkBrandon Williams, Jan 30, 2017
  92. 13/27 attr.c: outline the future plans by heavily commentingBrandon Williams, Jan 28, 2017
  93. 16/27 attr: convert git_all_attrs() to use "struct attr_check"Brandon Williams, Jan 28, 2017
  94. Stefan BellerJan 28, 2017
  95. Brandon WilliamsJan 29, 2017
  96. 16/27 attr: convert git_all_attrs() to use "struct attr_check"Brandon Williams, Jan 30, 2017
  97. 14/27 attr: rename function and struct related to checking attributesBrandon Williams, Jan 28, 2017
  98. 18/27 attr: retire git_check_attrs() APIBrandon Williams, Jan 28, 2017
  99. 19/27 attr: pass struct attr_check to collect_some_attrsBrandon Williams, Jan 28, 2017
  100. 17/27 attr: convert git_check_attrs() callers to use the new APIBrandon Williams, Jan 28, 2017
  101. 22/27 attr: eliminate global check_all_attr arrayBrandon Williams, Jan 28, 2017
  102. 21/27 attr: use hashmap for attribute dictionaryBrandon Williams, Jan 28, 2017
  103. 26/27 attr: push the bare repo check into read_attr()Brandon Williams, Jan 28, 2017
  104. 25/27 attr: store attribute stack in attr_check structureBrandon Williams, Jan 28, 2017
  105. 20/27 attr: change validity check for attribute names to use positive logicBrandon Williams, Jan 28, 2017
  106. 24/27 attr: tighten const correctness with git_attr and match_attrBrandon Williams, Jan 28, 2017
  107. 27/27 attr: reformat git_attr_set_direction() functionBrandon Williams, Jan 28, 2017
  108. 23/27 attr: remove maybe-real, maybe-macro from git_attrBrandon Williams, Jan 28, 2017
  109. Junio C HamanoFeb 2, 2017
  110. Brandon WilliamsFeb 9, 2017
  111. Junio C HamanoFeb 9, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.