git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 04/16] update-index: generalize 'read_index_info'

From
Junio C Hamano <gitster@pobox.com>
Date
Jun 11, 2024, 22:45 UTC
Message-ID
<xmqqa5jrt7x4.fsf@gitster.g>
In-Reply-To
<9d0689e9c285b375b0067760929011038c085d65.1718130288.git.gitgitgadget@gmail.com>
"Victoria Dye via GitGitGadget" <gitgitgadget@gmail.com> writes:
Show 13 quoted lines
> From: Victoria Dye <vdye@github.com>
>
> Move 'read_index_info()' into a new header 'index-info.h' and generalize the
> function to call a provided callback for each parsed line. Update
> 'update-index.c' to use this generalized 'read_index_info()', adding the
> callback 'apply_index_info()' to verify the parsed line and update the index
> according to its contents.
>
> The input parsing done by 'read_index_info()' is similar to, but more
> flexible than, the parsing done in 'mktree' by 'mktree_line()' (handling not
> only 'git ls-tree' output but also the outputs of 'git apply --index-info'
> and 'git ls-files --stage' outputs). To make 'mktree' more flexible, a later
> patch will replace mktree's custom parsing with 'read_index_info()'.
"git apply --index-info"?  

That is a blast from the past. It no longer exists since 7a988699 (apply: get rid of --index-info in favor of --build-fake-ancestor, 2007-09-17).

As to the scriptability, supporting "ls-files -s" and "ls-tree -r" output as our input do help, but the third one is not natively emitted and it is very unlikely that there are third-party tools that give output in that format. After all these years, I suspect that it is sufficient to say

    "update-index --index-info" and "mktree" both read information
    necessary to eventually build trees, but having two separate
    parsers is a maintenance burden, so we are massaging the code
    from the former to be reusable.
without mentioning where the old third format comes from.
Show 28 quoted lines
> diff --git a/builtin/update-index.c b/builtin/update-index.c
> index d343416ae26..77df380cb54 100644
> --- a/builtin/update-index.c
> +++ b/builtin/update-index.c
> @@ -11,6 +11,7 @@
>  #include "gettext.h"
>  #include "hash.h"
>  #include "hex.h"
> +#include "index-info.h"
>  #include "lockfile.h"
>  #include "quote.h"
>  #include "cache-tree.h"
> @@ -509,100 +510,29 @@ static void update_one(const char *path)
>  	report("add '%s'", path);
>  }
>  
> +static int apply_index_info(unsigned int mode, struct object_id *oid, int stage,
> +			    const char *path_name, void *cbdata UNUSED)
>  {
> +	if (!verify_path(path_name, mode)) {
> +		fprintf(stderr, "Ignoring path %s\n", path_name);
> +		return 0;
> +	}
>  
> +	if (!mode) {
> +		/* mode == 0 means there is no such path -- remove */
> +		if (remove_file_from_index(the_repository->index, path_name))
> +			die("git update-index: unable to remove %s", path_name);

This changes the error message. We used to feed "ptr" (no longer visible to this function, as the caller unquotes before calling us) that pointed at the original the user gave to the program; now we report the path_name which is the result of the unquoting.

Show 8 quoted lines
> +	}
> +	else {
> +		/* mode ' ' sha1 '\t' name
> +		 * ptr[-1] points at tab,
> +		 * ptr[-41] is at the beginning of sha1
>  		 */
> +		if (add_cacheinfo(mode, oid, path_name, stage))
> +			die("git update-index: unable to update %s", path_name);

But this side used to report the path_name as the result of unquoting in the original. So the above change would probably be OK in the name of consistency?

973d6a20 (update-index --index-info: adjust for funny-path quoting., 2005-10-16) was the origin of the unquoting, and looking at that commit, I have a feeling that the "ptr" thing above (i.e., the one I pointed out as changing the behaviour) was simply forgotten (as opposed to deliberately made to report the original) while updating the code to deal with quoted original into unquoted paths.

So I think the change is more than OK. It is a very welcome (belated) bugfix for 973d6a20 ;-).

>  	}
> +
> +	return 0;
>  }

It looks a bit disappointing that we die in the callback like above, when the main parser loop that moved to the other file to be more reusable is now capable of returning to the caller with an error, but at this step, it is a good place to stop. A refactor that does not change the behaviour.

Nicely done.
Show 13 quoted lines
> diff --git a/t/t2107-update-index-basic.sh b/t/t2107-update-index-basic.sh
> index cc72ead79f3..29696ade0d0 100755
> --- a/t/t2107-update-index-basic.sh
> +++ b/t/t2107-update-index-basic.sh
> @@ -142,4 +142,31 @@ test_expect_success '--index-version' '
>  	test_must_be_empty actual
>  '
>  
> +test_expect_success '--index-info fails on malformed input' '
> +	# empty line
> +	echo "" |
> +	test_must_fail git update-index --index-info 2>err &&
> +	grep "malformed input line" err &&

Using "test_grep" would make it easier to diagnose when test breaks. A failing "grep" will be silent. A failing "test_grep" will tell us "I was told to find THIS, but didn't find any in THAT".

Show 22 quoted lines
> +	# bad whitespace
> +	printf "100644 $EMPTY_BLOB A" |
> +	test_must_fail git update-index --index-info 2>err &&
> +	grep "malformed input line" err &&
> +
> +	# invalid stage value
> +	printf "100644 $EMPTY_BLOB 5\tA" |
> +	test_must_fail git update-index --index-info 2>err &&
> +	grep "malformed input line" err &&
> +
> +	# invalid OID length
> +	printf "100755 abc123\tA" |
> +	test_must_fail git update-index --index-info 2>err &&
> +	grep "malformed input line" err &&
> +
> +	# bad quoting
> +	printf "100644 $EMPTY_BLOB\t\"A" |
> +	test_must_fail git update-index --index-info 2>err &&
> +	grep "bad quoting of path name" err
> +'
> +
>  test_done
Previous: Victoria Dye via GitGitGadgetNext: Victoria Dye via GitGitGadget
Message 9 of 65 in “mktree: support more flexible usage”
  1. 00/16 mktree: support more flexible usageVictoria Dye via GitGitGadget, Jun 11, 2024
  2. 01/16 mktree: use OPT_BOOLVictoria Dye via GitGitGadget, Jun 11, 2024
  3. 02/16 mktree: rename treeent to tree_entryVictoria Dye via GitGitGadget, Jun 11, 2024
  4. Patrick SteinhardtJun 12, 2024
  5. 03/16 mktree: use non-static tree_entry arrayVictoria Dye via GitGitGadget, Jun 11, 2024
  6. Eric SunshineJun 11, 2024
  7. Patrick SteinhardtJun 12, 2024
  8. 04/16 update-index: generalize 'read_index_info'Victoria Dye via GitGitGadget, Jun 11, 2024
  9. Junio C HamanoJun 11, 2024
  10. 06/16 index-info.c: parse object type in provided in read_index_infoVictoria Dye via GitGitGadget, Jun 11, 2024
  11. Junio C HamanoJun 12, 2024
  12. 05/16 index-info.c: identify empty input lines in read_index_infoVictoria Dye via GitGitGadget, Jun 11, 2024
  13. Junio C HamanoJun 11, 2024
  14. Victoria DyeJun 18, 2024
  15. 07/16 mktree: use read_index_info to read stdin linesVictoria Dye via GitGitGadget, Jun 11, 2024
  16. Junio C HamanoJun 12, 2024
  17. Patrick SteinhardtJun 12, 2024
  18. Junio C HamanoJun 12, 2024
  19. 08/16 mktree: add a --literally optionVictoria Dye via GitGitGadget, Jun 11, 2024
  20. Junio C HamanoJun 12, 2024
  21. 09/16 mktree: validate paths more carefullyVictoria Dye via GitGitGadget, Jun 11, 2024
  22. Junio C HamanoJun 12, 2024
  23. Victoria DyeJun 12, 2024
  24. Junio C HamanoJun 12, 2024
  25. 10/16 mktree: overwrite duplicate entriesVictoria Dye via GitGitGadget, Jun 11, 2024
  26. Patrick SteinhardtJun 12, 2024
  27. Victoria DyeJun 12, 2024
  28. 11/16 mktree: create tree using an in-core indexVictoria Dye via GitGitGadget, Jun 11, 2024
  29. Patrick SteinhardtJun 12, 2024
  30. 12/16 mktree: use iterator struct to add tree entries to indexVictoria Dye via GitGitGadget, Jun 11, 2024
  31. Patrick SteinhardtJun 12, 2024
  32. Victoria DyeJun 13, 2024
  33. 13/16 mktree: add directory-file conflict hashmapVictoria Dye via GitGitGadget, Jun 11, 2024
  34. 14/16 mktree: optionally add to an existing treeVictoria Dye via GitGitGadget, Jun 11, 2024
  35. Patrick SteinhardtJun 12, 2024
  36. Junio C HamanoJun 12, 2024
  37. Victoria DyeJun 17, 2024
  38. 15/16 mktree: allow deeper paths in inputVictoria Dye via GitGitGadget, Jun 11, 2024
  39. 16/16 mktree: remove entries when mode is 0Victoria Dye via GitGitGadget, Jun 11, 2024
  40. 00/17 mktree: support more flexible usageVictoria Dye via GitGitGadget, Jun 19, 2024
  41. 01/17 mktree: use OPT_BOOLVictoria Dye via GitGitGadget, Jun 19, 2024
  42. 02/17 mktree: rename treeent to tree_entryVictoria Dye via GitGitGadget, Jun 19, 2024
  43. 03/17 mktree: use non-static tree_entry arrayVictoria Dye via GitGitGadget, Jun 19, 2024
  44. 04/17 update-index: generalize 'read_index_info'Victoria Dye via GitGitGadget, Jun 19, 2024
  45. 05/17 index-info.c: return unrecognized lines to callerVictoria Dye via GitGitGadget, Jun 19, 2024
  46. 06/17 index-info.c: parse object type in provided in read_index_infoVictoria Dye via GitGitGadget, Jun 19, 2024
  47. 08/17 mktree.c: do not fail on mismatched submodule typeVictoria Dye via GitGitGadget, Jun 19, 2024
  48. 07/17 mktree: use read_index_info to read stdin linesVictoria Dye via GitGitGadget, Jun 19, 2024
  49. Junio C HamanoJun 20, 2024
  50. 09/17 mktree: add a --literally optionVictoria Dye via GitGitGadget, Jun 19, 2024
  51. 10/17 mktree: validate paths more carefullyVictoria Dye via GitGitGadget, Jun 19, 2024
  52. 11/17 mktree: overwrite duplicate entriesVictoria Dye via GitGitGadget, Jun 19, 2024
  53. Junio C HamanoJun 20, 2024
  54. 12/17 mktree: create tree using an in-core indexVictoria Dye via GitGitGadget, Jun 19, 2024
  55. Junio C HamanoJun 20, 2024
  56. 13/17 mktree: use iterator struct to add tree entries to indexVictoria Dye via GitGitGadget, Jun 19, 2024
  57. Junio C HamanoJun 26, 2024
  58. 14/17 mktree: add directory-file conflict hashmapVictoria Dye via GitGitGadget, Jun 19, 2024
  59. 15/17 mktree: optionally add to an existing treeVictoria Dye via GitGitGadget, Jun 19, 2024
  60. Junio C HamanoJun 26, 2024
  61. 16/17 mktree: allow deeper paths in inputVictoria Dye via GitGitGadget, Jun 19, 2024
  62. Junio C HamanoJun 27, 2024
  63. 17/17 mktree: remove entries when mode is 0Victoria Dye via GitGitGadget, Jun 19, 2024
  64. Junio C HamanoJun 25, 2024
  65. Junio C HamanoJul 10, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.