git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [GSoC] How to protect cached_objects

From
Duy Nguyen <pclouds@gmail.com>
Date
May 24, 2019, 09:55 UTC
Message-ID
<CACsJy8DFw1Y_bhE=k2ZEMTk+vFvwwmx4GDnRXEQB9cp58M3vLg@mail.gmail.com>
In-Reply-To
<CAHd-oW40x3-4rye3QeVGix5pfTCoCHmiwg8ddCeq1E6YYongag@mail.gmail.com>

On Thu, May 23, 2019 at 11:51 PM Matheus Tavares Bernardino <matheus.bernardino@usp.br> wrote:

>
Show 25 quoted lines
> Hi, everyone
>
> As one of my first tasks in GSoC, I'm looking to protect the global
> states at sha1-file.c for future parallelizations. Currently, I'm
> analyzing how to deal with the cached_objects array, which is a small
> set of in-memory objects that read_object_file() is able to return
> although they don't really exist on disk. The only current user of
> this set is git-blame, which adds a fake commit containing
> non-committed changes.
>
> As it is now, if we start parallelizing blame, cached_objects won't be
> a problem since it is written to only once, at the beginning, and read
> from a couple times latter, with no possible race conditions.
>
> But should we make these operations thread safe for future uses that
> could involve potential parallel writes and reads too?
>
> If so, we have two options:
> - Make the array thread local, which would oblige us to replicate data, or
> - Protect it with locks, which could impact the sequential
> performance. We could have a macro here, to skip looking on
> single-threaded use cases. But we don't know, a priori, the number of
> threads that would want to use the pack access code.
>
> Any thought on this?

I would go with "that's the problem of the future me". I'll go with a simple global (I mean per-object store) mutex. After we have a complete picture how many locks we need, and can run some tests to see the amount of lock contention we have (or even cache missess if we have so many locks), then we can start thinking of an optimal strategy.

I mean, this is an implementation detail and can't affect object access API right? That gives us some breathing room to change stuff without preparing for something that we don't need right now (like multiple cached_objects writers)

> Thanks,
> Matheus Tavares
-- 
Duy
Previous: Matheus Tavares BernardinoNext: Matheus Tavares Bernardino
Message 4 of 6 in “[GSoC] How to protect cached_objects”
  1. Matheus Tavares BernardinoMay 23, 2019
  2. Jeff KingMay 24, 2019
  3. Matheus Tavares BernardinoMay 25, 2019
  4. Duy NguyenMay 24, 2019
  5. Matheus Tavares BernardinoMay 25, 2019
  6. Duy NguyenMay 26, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.