git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 23/23] untracked cache: guard and disable on system changes

From
Duy Nguyen <pclouds@gmail.com>
Date
Dec 15, 2014, 03:02 UTC
Message-ID
<CACsJy8DLus+M65FCw=rrBStLXnWOmCJRm2NidWth9Roj8zAodw@mail.gmail.com>
In-Reply-To
<548A0175.6000008@web.de>
On Fri, Dec 12, 2014 at 3:41 AM, Torsten Bögershausen <tboegi@web.de> wrote:
Show 20 quoted lines
> Even if I share the the concerns that the cache may work on one system,
> but not on the other, there should be better ways to protect from that.
>
> Using the uname does not really help, if you move one repo from NTFS to VFAT,
> we will not detect it (assuming we use Windows).
> (And how much do we need to support the move of a repo ?)
>
> There is a concern that this may not work, when different clients are accessing
> the repo, using the UNTR extension.
>
> Some kind of sanity check would be good to have, what can be done ?
> The most important things are the timestamps.
> I can think of 2 sanity checks:
> - If the modified time stamp of a directory is older then the create time of any file,
>   the UNTR cache can not be used.
> - If the timestamp of a file changes, but the sha1 sum is the same, what does this mean?
>   The file (or the whole repo) has been copied, or the time stamping does not work.
>
> A simple verification of the FS could be to stat() .git/, create a temp file, delete it and
> stat() again. If mtime does not change, the FS is unusable for UNTR.

This is a slow test. Some filesytem only supports second resolution timestamps. If you create and delete the file so fast, mtime may remain in the same second even if the fs is supported. We need to wait a second between those operations (this is why "update-index --untracked-cache" takes several seconds). So it cannot be done often (i.e. at startup of every command)

Show 11 quoted lines
> Then we could extend the uname idea:
> Create a string in UNTR which is a collection of lines like this:
>
> Working-For: Linux;/mnt/nfs/projects/project1
> Not-OK-For: WIndows:/a:/project1
> (Of course the strings can be made nicer, and '\n' is URL-encoded.)
>
> Each system that is not listed needs to probe the repo, add another line
> and re-write the index.
>
> We can even add a "best-for" line, and invalidate the UNTR every 12 hours or so.

It starts to look complicated. How about letting the user deal with it? UNTR will support storing multiple "location" lines. Whenever UNTR is needed at a new location, it's disabled. The user has to use 'update-index' to test the new location and add it to UNTR. The user can choose to keep current locations (network access), or replace them all with the new one (repo move), or just mark the new location unusable so the extension is kept for use in other places, but warning at this place is suppressed. THe "localtion" consists of worktree path, system name and host name.

> Should we think about having an ASCII area for additional information, which is part
> of the stone, but the content is flexible ?

These lines are already in free form. If the running system generates the same string as stored in UNTR, it's allowed to use the extension. We need code to understand this content, so flexibility must be within limit..

-- 
Duy
Previous: Torsten Bögershausen
Message 35 of 35 in “nd/untracked-cache updates”
  1. 00/23 nd/untracked-cache updatesNguyễn Thái Ngọc Duy, Dec 8, 2014
  2. checkout: add --ignore-other-wortreesNguyễn Thái Ngọc Duy, Dec 8, 2014
  3. Duy NguyenDec 8, 2014
  4. 01/23 dir.c: optionally compute sha-1 of a .gitignore fileNguyễn Thái Ngọc Duy, Dec 8, 2014
  5. 02/23 untracked cache: record .gitignore information and dir hierarchyNguyễn Thái Ngọc Duy, Dec 8, 2014
  6. 03/23 untracked cache: initial untracked cache validationNguyễn Thái Ngọc Duy, Dec 8, 2014
  7. 04/23 untracked cache: invalidate dirs recursively if .gitignore changesNguyễn Thái Ngọc Duy, Dec 8, 2014
  8. 05/23 untracked cache: make a wrapper around {open,read,close}dir()Nguyễn Thái Ngọc Duy, Dec 8, 2014
  9. 06/23 untracked cache: record/validate dir mtime and reuse cached outputNguyễn Thái Ngọc Duy, Dec 8, 2014
  10. 07/23 untracked cache: mark what dirs should be recursed/savedNguyễn Thái Ngọc Duy, Dec 8, 2014
  11. 08/23 untracked cache: don't open non-existent .gitignoreNguyễn Thái Ngọc Duy, Dec 8, 2014
  12. 09/23 ewah: add convenient wrapper ewah_serialize_strbuf()Nguyễn Thái Ngọc Duy, Dec 8, 2014
  13. 10/23 untracked cache: save to an index extensionNguyễn Thái Ngọc Duy, Dec 8, 2014
  14. Eric SunshineDec 9, 2014
  15. Duy NguyenDec 10, 2014
  16. 11/23 untracked cache: load from UNTR index extensionNguyễn Thái Ngọc Duy, Dec 8, 2014
  17. Junio C HamanoDec 15, 2014
  18. 12/23 untracked cache: invalidate at index addition or removalNguyễn Thái Ngọc Duy, Dec 8, 2014
  19. 13/23 read-cache.c: split racy stat test to a separate functionNguyễn Thái Ngọc Duy, Dec 8, 2014
  20. 14/23 untracked cache: avoid racy timestampsNguyễn Thái Ngọc Duy, Dec 8, 2014
  21. 15/23 untracked cache: print stats with $GIT_TRACE_UNTRACKED_STATSNguyễn Thái Ngọc Duy, Dec 8, 2014
  22. 16/23 untracked cache: mark index dirty if untracked cache is updatedNguyễn Thái Ngọc Duy, Dec 8, 2014
  23. 17/23 untracked-cache: temporarily disable with $GIT_DISABLE_UNTRACKED_CACHENguyễn Thái Ngọc Duy, Dec 8, 2014
  24. 18/23 status: enable untracked cacheNguyễn Thái Ngọc Duy, Dec 8, 2014
  25. 19/23 update-index: manually enable or disable untracked cacheNguyễn Thái Ngọc Duy, Dec 8, 2014
  26. 20/23 update-index: test the system before enabling untracked cacheNguyễn Thái Ngọc Duy, Dec 8, 2014
  27. 21/23 t7063: tests for untracked cacheNguyễn Thái Ngọc Duy, Dec 8, 2014
  28. 22/23 mingw32: add uname()Nguyễn Thái Ngọc Duy, Dec 8, 2014
  29. 23/23 untracked cache: guard and disable on system changesNguyễn Thái Ngọc Duy, Dec 8, 2014
  30. brian m. carlsonDec 9, 2014
  31. Duy NguyenDec 9, 2014
  32. Torsten BögershausenDec 10, 2014
  33. Duy NguyenDec 10, 2014
  34. Torsten BögershausenDec 11, 2014
  35. Duy NguyenDec 15, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.