git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v6 1/7] archive: optionally add "virtual" files

From
Junio C Hamano <gitster@pobox.com>
Date
May 25, 2022, 21:11 UTC
Message-ID
<xmqqfskx5ndd.fsf@gitster.g>
In-Reply-To
<0005cfae31d52a157d4df5ba3db9f9f5b2167ddc.1653145696.git.gitgitgadget@gmail.com>

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com> writes:

Show 9 quoted lines
> @@ -61,6 +61,17 @@ OPTIONS
>  	by concatenating the value for `--prefix` (if any) and the
>  	basename of <file>.
>  
> +--add-virtual-file=<path>:<content>::
> +	Add the specified contents to the archive.  Can be repeated to add
> +	multiple files.  The path of the file in the archive is built
> +	by concatenating the value for `--prefix` (if any) and the
> +	basename of <file>.

This sentence was copy-pasted from --add-file without adjusting. There is no <file>; this new feature gives <path>.

Also, I suspect that the feature is losing end-user supplied information without a good reason. --add-file=<file> may have prepared an input in a randomly named temporary directory and it would make quite a lot of sense to strip the leading directory components from <file> and use only the basename part. But the <path> given to "--add-virtual-file" does not refer to anything on the filesystem. Its ONLY use is to be used as the path in the archive to store the content. There is no justification why we would discard the leading path components from it. I am not decided, but I am inclined to say that we should not honor "--prefix".

   $ git archive --prefix=2.36.0 v2.36.0

would be a way to create a single directory and put everything in the tree-ish in there, but there probably are cases where the user of an "extra file" feature wants to add untracked cruft _in_ that directory, and there are other cases where an extra file wants to go to the top-level next to the 2.36.0 directory. A user can use the same string as --prefix=<base> in front of <path> if the extra file should go next to the top-level of the tree-ish, or without such prefixing to place the extra file at the top-level.

Hence
	Add the specified contents to the archive.  Can be repeated
	to add multiple files.  `<path>` is used as the path of the
	file in the archive.
	
would be what I would expect in a version of this feature that is
reasonably designed.
Show 5 quoted lines
> ++
> +The `<path>` cannot contain any colon, the file mode is limited to
> +a regular file, and the option may be subject to platform-dependent
> +command-line limits. For non-trivial cases, write an untracked file
> +and use `--add-file` instead.
OK.
Show 18 quoted lines
> diff --git a/archive.c b/archive.c
> index a3bbb091256..d20e16fa819 100644
> --- a/archive.c
> +++ b/archive.c
> @@ -263,6 +263,7 @@ static int queue_or_write_archive_entry(const struct object_id *oid,
>  struct extra_file_info {
>  	char *base;
>  	struct stat stat;
> +	void *content;
>  };
>  
>  int write_archive_entries(struct archiver_args *args,
> @@ -337,7 +338,13 @@ int write_archive_entries(struct archiver_args *args,
>  		strbuf_addstr(&path_in_archive, basename(path));
>  
>  		strbuf_reset(&content);
> -		if (strbuf_read_file(&content, path, info->stat.st_size) < 0)
> +		if (info->content)

We ended up with the problematic "leading <path> components are discarded" design only because the implementation reuses the logic path_in_archive computation (the last line is seen in precontext), which is a bit unfortunate. I think we could rewrite the inside of that "for each extra file" loop like so, instead:

	for (i = 0; i < args->extra_files.nr; i++) {
		struct string_list_item *item = args->extra_files.items + i;
		char *path = item->string;
		struct extra_file_info *info = item->util;
		put_be64(fake_oid.hash, i + 1);
		if (!info->content) {
			strbuf_reset(&path_in_archive);
			if (info->base)
				strbuf_addstr(&path_in_archive, info->base);
			strbuf_addstr(&path_in_archive, basename(path));
			strbuf_reset(&content);
			if (strbuf_read_file(&content, path, info->stat.st_size) < 0)
				err = error_errno(_("could not read '%s'"), path);
			else
				err = write_entry(args, &fake_oid, path_in_archive.buf,
						  path_in_archive.len,
						  info->stat.st_mode,
						  content.buf, content.len);
		} else {
			err = write_entry(args, &fake_oid,
					  path, strlen(path),
					  info->stat.st_mode,
					  info->content, info->stat.st_size);
		}
		if (err)
			break;
	}

The first half is the original code for "--add-file", which clears info->content to NULL. We mangle the filename to come up with the name in the archive (i.e. take basename and prefix with info->base).

The "else" side is the new code. "--add-virtual-file" has the "<path>" thing in item->string, and info has the contents, so we just write it out.

Previous: Johannes Schindelin via GitGitGadgetNext: René Scharfe
Message 105 of 140 in “scalar: implement the subcommand "diagnose"”
  1. 0/5 scalar: implement the subcommand "diagnose"Johannes Schindelin via GitGitGadget, Jan 26, 2022
  2. 1/5 Implement `scalar diagnose`Johannes Schindelin via GitGitGadget, Jan 26, 2022
  3. René ScharfeJan 26, 2022
  4. Taylor BlauJan 26, 2022
  5. Johannes SchindelinFeb 6, 2022
  6. Elijah NewrenJan 27, 2022
  7. 2/5 scalar diagnose: include disk space informationJohannes Schindelin via GitGitGadget, Jan 26, 2022
  8. 3/5 scalar: teach `diagnose` to gather packfile infoMatthew John Cheetham via GitGitGadget, Jan 26, 2022
  9. Taylor BlauJan 26, 2022
  10. Derrick StoleeJan 27, 2022
  11. Johannes SchindelinFeb 6, 2022
  12. 4/5 scalar: teach `diagnose` to gather loose objects informationMatthew John Cheetham via GitGitGadget, Jan 26, 2022
  13. Taylor BlauJan 26, 2022
  14. Derrick StoleeJan 27, 2022
  15. Elijah NewrenJan 27, 2022
  16. Johannes SchindelinFeb 6, 2022
  17. 5/5 scalar diagnose: show a spinner while staging contentJohannes Schindelin via GitGitGadget, Jan 26, 2022
  18. Derrick StoleeJan 27, 2022
  19. Johannes SchindelinFeb 6, 2022
  20. 0/6 scalar: implement the subcommand "diagnose"Johannes Schindelin via GitGitGadget, Feb 6, 2022
  21. 2/6 scalar: validate the optional enlistment argumentJohannes Schindelin via GitGitGadget, Feb 6, 2022
  22. 1/6 archive: optionally add "virtual" filesJohannes Schindelin via GitGitGadget, Feb 6, 2022
  23. René ScharfeFeb 7, 2022
  24. Junio C HamanoFeb 7, 2022
  25. Johannes SchindelinFeb 8, 2022
  26. Junio C HamanoFeb 8, 2022
  27. René ScharfeFeb 8, 2022
  28. Junio C HamanoFeb 9, 2022
  29. René ScharfeFeb 10, 2022
  30. Junio C HamanoFeb 10, 2022
  31. René ScharfeFeb 11, 2022
  32. Junio C HamanoFeb 11, 2022
  33. René ScharfeFeb 12, 2022
  34. Junio C HamanoFeb 13, 2022
  35. René ScharfeFeb 13, 2022
  36. Junio C HamanoFeb 14, 2022
  37. Johannes SchindelinFeb 8, 2022
  38. 3/6 Implement `scalar diagnose`Johannes Schindelin via GitGitGadget, Feb 6, 2022
  39. René ScharfeFeb 7, 2022
  40. Johannes SchindelinFeb 8, 2022
  41. 4/6 scalar diagnose: include disk space informationJohannes Schindelin via GitGitGadget, Feb 6, 2022
  42. 5/6 scalar: teach `diagnose` to gather packfile infoMatthew John Cheetham via GitGitGadget, Feb 6, 2022
  43. 6/6 scalar: teach `diagnose` to gather loose objects informationMatthew John Cheetham via GitGitGadget, Feb 6, 2022
  44. 0/7 scalar: implement the subcommand "diagnose"Johannes Schindelin via GitGitGadget, May 4, 2022
  45. 2/7 archive --add-file-with-contents: allow paths containing colonsJohannes Schindelin via GitGitGadget, May 4, 2022
  46. Elijah NewrenMay 7, 2022
  47. Johannes SchindelinMay 9, 2022
  48. 1/7 archive: optionally add "virtual" filesJohannes Schindelin via GitGitGadget, May 4, 2022
  49. 3/7 scalar: validate the optional enlistment argumentJohannes Schindelin via GitGitGadget, May 4, 2022
  50. 5/7 scalar diagnose: include disk space informationJohannes Schindelin via GitGitGadget, May 4, 2022
  51. 7/7 scalar: teach `diagnose` to gather loose objects informationMatthew John Cheetham via GitGitGadget, May 4, 2022
  52. 6/7 scalar: teach `diagnose` to gather packfile infoMatthew John Cheetham via GitGitGadget, May 4, 2022
  53. 4/7 Implement `scalar diagnose`Johannes Schindelin via GitGitGadget, May 4, 2022
  54. Elijah NewrenMay 7, 2022
  55. 0/7 scalar: implement the subcommand "diagnose"Johannes Schindelin via GitGitGadget, May 10, 2022
  56. 2/7 archive --add-file-with-contents: allow paths containing colonsJohannes Schindelin via GitGitGadget, May 10, 2022
  57. Junio C HamanoMay 10, 2022
  58. rsbecker@nexbridge.comMay 10, 2022
  59. Johannes SchindelinMay 19, 2022
  60. Johannes SchindelinMay 19, 2022
  61. Junio C HamanoMay 19, 2022
  62. 3/7 scalar: validate the optional enlistment argumentJohannes Schindelin via GitGitGadget, May 10, 2022
  63. Ævar Arnfjörð BjarmasonMay 17, 2022
  64. Junio C HamanoMay 18, 2022
  65. Ævar Arnfjörð BjarmasonMay 20, 2022
  66. Johannes SchindelinMay 20, 2022
  67. Ævar Arnfjörð BjarmasonMay 21, 2022
  68. Junio C HamanoMay 22, 2022
  69. Johannes SchindelinMay 24, 2022
  70. Ævar Arnfjörð BjarmasonMay 24, 2022
  71. Junio C HamanoMay 24, 2022
  72. Johannes SchindelinMay 25, 2022
  73. 4/7 Implement `scalar diagnose`Johannes Schindelin via GitGitGadget, May 10, 2022
  74. Ævar Arnfjörð BjarmasonMay 17, 2022
  75. 6/7 scalar: teach `diagnose` to gather packfile infoMatthew John Cheetham via GitGitGadget, May 10, 2022
  76. 5/7 scalar diagnose: include disk space informationJohannes Schindelin via GitGitGadget, May 10, 2022
  77. 1/7 archive: optionally add "virtual" filesJohannes Schindelin via GitGitGadget, May 10, 2022
  78. Junio C HamanoMay 10, 2022
  79. rsbecker@nexbridge.comMay 10, 2022
  80. Junio C HamanoMay 10, 2022
  81. René ScharfeMay 11, 2022
  82. Junio C HamanoMay 11, 2022
  83. René ScharfeMay 12, 2022
  84. Junio C HamanoMay 12, 2022
  85. Junio C HamanoMay 12, 2022
  86. René ScharfeMay 14, 2022
  87. fixup! archive: optionally add "virtual" filesJunio C Hamano, May 12, 2022
  88. 7/7 scalar: teach `diagnose` to gather loose objects informationMatthew John Cheetham via GitGitGadget, May 10, 2022
  89. Ævar Arnfjörð BjarmasonMay 17, 2022
  90. rsbecker@nexbridge.comMay 17, 2022
  91. Johannes SchindelinMay 19, 2022
  92. 0/7 scalar: implement the subcommand "diagnose"Johannes Schindelin via GitGitGadget, May 19, 2022
  93. 1/7 archive: optionally add "virtual" filesJohannes Schindelin via GitGitGadget, May 19, 2022
  94. René ScharfeMay 20, 2022
  95. Junio C HamanoMay 20, 2022
  96. 2/7 archive --add-file-with-contents: allow paths containing colonsJohannes Schindelin via GitGitGadget, May 19, 2022
  97. 5/7 scalar diagnose: include disk space informationJohannes Schindelin via GitGitGadget, May 19, 2022
  98. 4/7 Implement `scalar diagnose`Johannes Schindelin via GitGitGadget, May 19, 2022
  99. 3/7 scalar: validate the optional enlistment argumentJohannes Schindelin via GitGitGadget, May 19, 2022
  100. 6/7 scalar: teach `diagnose` to gather packfile infoMatthew John Cheetham via GitGitGadget, May 19, 2022
  101. 7/7 scalar: teach `diagnose` to gather loose objects informationMatthew John Cheetham via GitGitGadget, May 19, 2022
  102. Junio C HamanoMay 19, 2022
  103. 0/7 scalar: implement the subcommand "diagnose"Johannes Schindelin via GitGitGadget, May 21, 2022
  104. 1/7 archive: optionally add "virtual" filesJohannes Schindelin via GitGitGadget, May 21, 2022
  105. Junio C HamanoMay 25, 2022
  106. René ScharfeMay 26, 2022
  107. Junio C HamanoMay 26, 2022
  108. René ScharfeMay 26, 2022
  109. Junio C HamanoMay 26, 2022
  110. René ScharfeMay 27, 2022
  111. Junio C HamanoMay 27, 2022
  112. René ScharfeMay 28, 2022
  113. 3/7 scalar: validate the optional enlistment argumentJohannes Schindelin via GitGitGadget, May 21, 2022
  114. 2/7 archive --add-virtual-file: allow paths containing colonsJohannes Schindelin via GitGitGadget, May 21, 2022
  115. Junio C HamanoMay 25, 2022
  116. Junio C HamanoMay 25, 2022
  117. Junio C HamanoMay 25, 2022
  118. 6/7 scalar: teach `diagnose` to gather packfile infoMatthew John Cheetham via GitGitGadget, May 21, 2022
  119. 4/7 Implement `scalar diagnose`Johannes Schindelin via GitGitGadget, May 21, 2022
  120. 5/7 scalar diagnose: include disk space informationJohannes Schindelin via GitGitGadget, May 21, 2022
  121. 7/7 scalar: teach `diagnose` to gather loose objects informationMatthew John Cheetham via GitGitGadget, May 21, 2022
  122. 0/7 js/scalar-diagnose rebasedJunio C Hamano, May 28, 2022
  123. 1/7 archive: optionally add "virtual" filesJunio C Hamano, May 28, 2022
  124. 3/7 scalar: validate the optional enlistment argumentJunio C Hamano, May 28, 2022
  125. 2/7 archive --add-virtual-file: allow paths containing colonsJunio C Hamano, May 28, 2022
  126. Adam DinwoodieJun 15, 2022
  127. Junio C HamanoJun 15, 2022
  128. Adam DinwoodieJun 15, 2022
  129. Johannes SchindelinJun 18, 2022
  130. Junio C HamanoJun 18, 2022
  131. Adam DinwoodieJun 20, 2022
  132. 4/7 scalar: implement `scalar diagnose`Junio C Hamano, May 28, 2022
  133. Ævar Arnfjörð BjarmasonJun 10, 2022
  134. Junio C HamanoJun 10, 2022
  135. Ævar Arnfjörð BjarmasonJun 10, 2022
  136. 6/7 scalar: teach `diagnose` to gather packfile infoJunio C Hamano, May 28, 2022
  137. 7/7 scalar: teach `diagnose` to gather loose objects informationJunio C Hamano, May 28, 2022
  138. 5/7 scalar diagnose: include disk space informationJunio C Hamano, May 28, 2022
  139. Johannes SchindelinMay 30, 2022
  140. Junio C HamanoMay 30, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.