git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC/PATCH 8/8] read-cache: unlink old sharedindex files

From
Duy Nguyen <pclouds@gmail.com>
Date
Jul 13, 2016, 15:16 UTC
Message-ID
<CACsJy8DHd44MnFUkJwh9n0JK5J5P6kER0kgopX0UrS=ROo_URQ@mail.gmail.com>
In-Reply-To
<CAP8UFD2pes3MD9FNBtvLYS5785aJoGPz00Fj+fHjy03j0aKjFg@mail.gmail.com>

On Tue, Jul 12, 2016 at 9:45 PM, Christian Couder <christian.couder@gmail.com> wrote:

Show 14 quoted lines
> On Tue, Jul 12, 2016 at 5:12 PM, Duy Nguyen <pclouds@gmail.com> wrote:
>>
>> No. People could create an index file anywhere in theory. So you don't
>> know how many index files there are.
>
> Maybe when an index file is created, its path and its sharedindex file
> could be appended into a log file.
> We could check this log file to see if we can remove sharedindex files.
> We would need to remove the entries in this log file for the indexes
> that are no longer there.
>
> Or instead of one log file we could have a file for each index file in
> a special directory called for example "indexinfo".
> So we could just delete the file if its related index is no longer there.

New files will require locking so people don't append at the same time. And maybe another new host of problems. I think we can just go with the garbage collection way that we have done for unreachable objects.

Your indexinfo idea looks very close to multiple locking, an index would lock the shared index it's linked to, preventing it from being removed. For single locking, we can just create a file named $X.lock, but for multiple locks, maybe we can go with $X.lock-$index_trailing_sha1. Will it work? I don't know. Just thinking out loud.

Show 7 quoted lines
>> It really depends. If the shared part is too small for old indexes, we
>> might as well unsplit them. In practice though, the only long-term
>> index file is $GIT_DIR/index. If we don't delete old shared index
>> files too close to their creation time, temp index files will go away.
>
> We could treat $GIT_DIR/index specially so that if there are no temp
> index files, there should be nothing in "indexinfo".

No, temp index files are needed. And it will be hard to treat $GIT_DIR/index specially because updating it involves a temp index: you first prepare a new index in $GIT_DIR/index.lock. If everything goes well, you atomically rename it to $GIT_DIR/index. You may be able to treat $GIT_DIR/index.lock special too, but that's starting to get out of hand.

-- 
Duy
Previous: Christian CouderNext: Christian Couder
Message 13 of 19 in “Add configuration options for split-index”
  1. 0/8 Add configuration options for split-indexChristian Couder, Jul 11, 2016
  2. 2/8 split-index: add {add,remove}_split_index() functionsChristian Couder, Jul 11, 2016
  3. 3/8 read-cache: add and then use tweak_split_index()Christian Couder, Jul 11, 2016
  4. 5/8 Documentation/config: add information for core.splitIndexChristian Couder, Jul 11, 2016
  5. 6/8 Documentation/git-update-index: talk about core.splitIndex config varChristian Couder, Jul 11, 2016
  6. 7/8 t1700: add tests for core.splitIndexChristian Couder, Jul 11, 2016
  7. 4/8 update-index: warn in case of split-index incoherencyChristian Couder, Jul 11, 2016
  8. 8/8 read-cache: unlink old sharedindex filesChristian Couder, Jul 11, 2016
  9. Duy NguyenJul 11, 2016
  10. Christian CouderJul 12, 2016
  11. Duy NguyenJul 12, 2016
  12. Christian CouderJul 12, 2016
  13. Duy NguyenJul 13, 2016
  14. Christian CouderJul 13, 2016
  15. 1/8 config: add git_config_get_split_index()Christian Couder, Jul 11, 2016
  16. Duy NguyenJul 12, 2016
  17. Christian CouderJul 23, 2016
  18. Duy NguyenJul 25, 2016
  19. Christian CouderJul 25, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.