git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 13/22] documentation: add documentation of the index-v5 file format

From
Duy Nguyen <pclouds@gmail.com>
Date
Jul 11, 2013, 10:39 UTC
Message-ID
<CACsJy8ALSBPq1+TP_YxJ=ecUwpKRY-i2O=+q8qMjtXbjShg3mA@mail.gmail.com>
In-Reply-To
<1373184720-29767-14-git-send-email-t.gummerer@gmail.com>
On Sun, Jul 7, 2013 at 3:11 PM, Thomas Gummerer <t.gummerer@gmail.com> wrote:
Show 57 quoted lines
> +== File entry (fileentries)
> +
> +  File entries are sorted in ascending order on the name field, after the
> +  respective offset given by the directory entries. All file names are
> +  prefix compressed, meaning the file name is relative to the directory.
> +
> +  filename (variable length, nul terminated). The exact encoding is
> +    undefined, but the filename cannot contain a NUL byte (iow, the same
> +    encoding as a UNIX pathname).
> +
> +  flags (16-bits): 'flags' field split into (high to low bits)
> +
> +    assumevalid (1-bit): assume-valid flag
> +
> +    intenttoadd (1-bit): intent-to-add flag, used by "git add -N".
> +      Extended flag in index v3.
> +
> +    stage (2-bit): stage of the file during merge
> +
> +    skipworktree (1-bit): skip-worktree flag, used by sparse checkout.
> +      Extended flag in index v3.
> +
> +    smudged (1-bit): indicates if the file is racily smudged.
> +
> +    10-bit unused, must be zero [6]
> +
> +  mode (16-bits): file mode, split into (high to low bits)
> +
> +    objtype (4-bits): object type
> +      valid values in binary are 1000 (regular file), 1010 (symbolic
> +      link) and 1110 (gitlink)
> +
> +    3-bit unused
> +
> +    permission (9-bits): unix permission. Only 0755 and 0644 are valid
> +      for regular files. Symbolic links and gitlinks have value 0 in
> +      this field.
> +
> +  mtimes (32-bits): mtime seconds, the last time a file's data changed
> +    this is stat(2) data
> +
> +  mtimens (32-bits): mtime nanosecond fractions
> +    this is stat(2) data
> +
> +  file size (32-bits): The on-disk size, trucated to 32-bit.
> +    this is stat(2) data
> +
> +  statcrc (32-bits): crc32 checksum over ctime seconds, ctime
> +    nanoseconds, ino, dev, uid, gid (All stat(2) data
> +    except mtime and file size). If the statcrc is 0 it will
> +    be ignored. [7]
> +
> +  objhash (160-bits): SHA-1 for the represented object
> +
> +  entrycrc (32-bits): crc32 checksum for the file entry. The crc code
> +    includes the offset to the offset to the file, relative to the
> +    beginning of the file.

Question about the possibility of updating index file directly. If git updates a few fields of an entry (but not entrycrc yet) and crashes, the entry would become corrupt because its entrycrc does not match the content. What do we do? Do we need to save a copy of the entry somewhere in the index file (maybe in the conflict data section), so that the reader can recover the index? Losing the index because of bugs is big deal in my opinion. pre-v5 never faces this because we keep the original copy til the end.

Maybe entrycrc should not cover stat fields and statcrc. It would make refreshing safer. If the above happens during refresh, only statcrc is corrupt and we can just refresh the entry. entrycrc still says the other fields are good (and they are). -- Duy

Previous: Thomas GummererNext: Thomas Gummerer
Message 34 of 51 in “Index v5”
  1. 00/22 Index v5Thomas Gummerer, Jul 7, 2013
  2. 01/22 t2104: Don't fail for index versions other than [23]Thomas Gummerer, Jul 7, 2013
  3. 02/22 read-cache: split index file version specific functionalityThomas Gummerer, Jul 7, 2013
  4. 03/22 read-cache: move index v2 specific functions to their own fileThomas Gummerer, Jul 7, 2013
  5. 04/22 read-cache: Re-read index if index file changedThomas Gummerer, Jul 7, 2013
  6. 05/22 read-cache: add index reading apiThomas Gummerer, Jul 7, 2013
  7. Duy NguyenJul 8, 2013
  8. Thomas GummererJul 8, 2013
  9. Duy NguyenJul 8, 2013
  10. Thomas GummererJul 8, 2013
  11. Duy NguyenJul 8, 2013
  12. Thomas GummererJul 8, 2013
  13. 5/22 Add documentation for the index apiThomas Gummerer, Jul 8, 2013
  14. Duy NguyenJul 9, 2013
  15. Thomas GummererJul 9, 2013
  16. Duy NguyenJul 10, 2013
  17. Thomas GummererJul 11, 2013
  18. Duy NguyenJul 11, 2013
  19. Duy NguyenJul 11, 2013
  20. Junio C HamanoJul 8, 2013
  21. Thomas GummererJul 8, 2013
  22. Junio C HamanoJul 8, 2013
  23. Thomas GummererJul 9, 2013
  24. 06/22 make sure partially read index is not changedThomas Gummerer, Jul 7, 2013
  25. Junio C HamanoJul 8, 2013
  26. Thomas GummererJul 8, 2013
  27. 07/22 dir.c: use index apiThomas Gummerer, Jul 7, 2013
  28. 08/22 tree.c: use index apiThomas Gummerer, Jul 7, 2013
  29. 09/22 name-hash.c: use index apiThomas Gummerer, Jul 7, 2013
  30. 10/22 grep.c: Use index apiThomas Gummerer, Jul 7, 2013
  31. 11/22 ls-files.c: use the index apiThomas Gummerer, Jul 7, 2013
  32. 12/22 read-cache: make read_blob_data_from_index use index apiThomas Gummerer, Jul 7, 2013
  33. 13/22 documentation: add documentation of the index-v5 file formatThomas Gummerer, Jul 7, 2013
  34. Duy NguyenJul 11, 2013
  35. Thomas GummererJul 11, 2013
  36. Duy NguyenJul 11, 2013
  37. Thomas GummererJul 11, 2013
  38. Duy NguyenJul 11, 2013
  39. 14/22 read-cache: make in-memory format aware of stat_crcThomas Gummerer, Jul 7, 2013
  40. 15/22 read-cache: read index-v5Thomas Gummerer, Jul 7, 2013
  41. Eric SunshineJul 7, 2013
  42. Thomas GummererJul 8, 2013
  43. 16/22 read-cache: read resolve-undo dataThomas Gummerer, Jul 7, 2013
  44. 17/22 read-cache: read cache-tree in index-v5Thomas Gummerer, Jul 7, 2013
  45. Eric SunshineJul 7, 2013
  46. 18/22 read-cache: write index-v5Thomas Gummerer, Jul 7, 2013
  47. Eric SunshineJul 7, 2013
  48. 19/22 read-cache: write index-v5 cache-tree dataThomas Gummerer, Jul 7, 2013
  49. 20/22 read-cache: write resolve-undo data for index-v5Thomas Gummerer, Jul 7, 2013
  50. 21/22 update-index.c: rewrite index when index-version is givenThomas Gummerer, Jul 7, 2013
  51. 22/22 p0003-index.sh: add perf test for the index formatsThomas Gummerer, Jul 7, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.