git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] update-server-info: avoid needless overwrites

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
May 14, 2019, 11:57 UTC
Message-ID
<87h89xw8nk.fsf@evledraar.gmail.com>
In-Reply-To
<20190514112451.GB26957@sigill.intra.peff.net>
On Tue, May 14 2019, Jeff King wrote:
Show 38 quoted lines
> On Tue, May 14, 2019 at 12:33:11PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> > I think it would work because any update-server-info, whether from A or
>> > B, will take into account the full current repo state (and we don't look
>> > at that state until we take the lock). So you might get an interleaved
>> > "A-push, B-push, B-maint, A-maint", but that's OK. A-maint will
>> > represent B's state when it runs.
>>
>> Maybe we're talking about different things. I mean the following
>> sequence:
>>
>>  1. Refs "X" and "Y" are at X=A Y=A
>>  2. Concurrent push #1 happens, updating X from A..F
>>  3. Concurrent push #2 happens, updating Y from A..F
>>  4. Concurrent push #1 succeeds
>>  5. Concurrent push #1 starts update-server-info. Reads X=F Y=A
>>  5. Concurrent push #2 succeeds
>>  6. Concurrent push #2 starts update-server-info. Reads X=F Y=F
>>  7. Concurrent push #2's update-server-info finishes, X=F Y=F written to "info"
>>  8. Concurrent push #1's update-server-info finishes, X=A Y=F written to "info"
>>
>> I.e. because we have per-ref locks and no lock at all on
>> update-server-info (but that would need to be a global ref lock, not
>> just on the "info" files) we can have a push that's already read "X"'s
>> value as "A" while updating "Y" win the race against an
>> update-server-info that updated "X"'s value to "F".
>>
>> It will get fixed on the next push (at least as far as "X"'s value
>> goes), but until that time dumb clients will falsely see that "X" hasn't
>> been updated.
>
> That's the same situation. But I thought we were talking about having an
> update-server-info lock. In which case the #2 update-server-info or the
> #1 update-server-info runs in its entirety, and cannot have their read
> and write steps interleaved (that's what I meant by "don't look at the
> state until we take the lock"). Then that gives us a strict ordering: we
> know that _some_ update-server-info (be it #1 or #2's) will run after
> any given update.

Yeah you're right. I *thought* in my last E-mail we were talking about the current state, but re-reading upthread I see that was a fail on my part.

An update-server-info lock would solve this indeed. We could still end up with a situation where whatever a naïve version of the lockfile API would fail for the "new" update since the old one was underway, so we'd need something similar to core.*Ref*Timeout, but if we ran into a *.lock or the timeout we could exit non-zero, as opposed to silently failing like it does now when it races.

Previous: Jeff KingNext: Eric Wong
Message 12 of 26 in “update-server-info: avoid needless overwrites”
  1. update-server-info: avoid needless overwritesEric Wong, May 11, 2019
  2. Eric SunshineMay 11, 2019
  3. update-server-info: avoid needless overwritesEric Wong, May 11, 2019
  4. Eric WongMay 11, 2019
  5. Ævar Arnfjörð BjarmasonMay 11, 2019
  6. Eric WongMay 12, 2019
  7. Jeff KingMay 12, 2019
  8. Ævar Arnfjörð BjarmasonMay 12, 2019
  9. Jeff KingMay 14, 2019
  10. Ævar Arnfjörð BjarmasonMay 14, 2019
  11. Jeff KingMay 14, 2019
  12. Ævar Arnfjörð BjarmasonMay 14, 2019
  13. Eric WongMay 14, 2019
  14. dumb HTTP things I want to doEric Wong, May 14, 2019
  15. Jeff KingMay 14, 2019
  16. Ævar Arnfjörð BjarmasonMay 14, 2019
  17. Jeff KingMay 14, 2019
  18. 2/1 server-info: conditionally update on fetchEric Wong, May 15, 2019
  19. [WIP] repack leaving stale entries in objects/info/packsEric Wong, May 15, 2019
  20. Jeff KingMay 15, 2019
  21. server-info: do not list unlinked packsEric Wong, May 23, 2019
  22. Jeff KingMay 23, 2019
  23. server-info: do not list unlinked packsEric Wong, May 23, 2019
  24. Jeff KingMay 24, 2019
  25. Ævar Arnfjörð BjarmasonMay 24, 2019
  26. update-server-info: avoid needless overwritesEric Wong, May 13, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.