git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/9] pack-write: refactor renaming in finish_tmp_packfile()

From
Junio C Hamano <gitster@pobox.com>
Date
Sep 9, 2021, 19:29 UTC
Message-ID
<xmqqk0jpzgxu.fsf@gitster.g>
In-Reply-To
<35052ef494dbc55119614f3e22742d8d814b21b1.1631157880.git.me@ttaylorr.com>
Taylor Blau <me@ttaylorr.com> writes:
Show 19 quoted lines
> @@ -1250,8 +1251,9 @@ static void write_pack_file(void)
>  					    &pack_idx_opts, hash);
>  
>  			if (write_bitmap_index) {
> -				strbuf_addf(&tmpname, "%s.bitmap", hash_to_hex(hash));
> +				size_t tmpname_len = tmpname.len;
>  
> +				strbuf_addstr(&tmpname, "bitmap");
>  				stop_progress(&progress_state);
>  
>  				bitmap_writer_show_progress(progress);
> @@ -1260,6 +1262,7 @@ static void write_pack_file(void)
>  				bitmap_writer_finish(written_list, nr_written,
>  						     tmpname.buf, write_bitmap_options);
>  				write_bitmap_index = 0;
> +				strbuf_setlen(&tmpname, tmpname_len);
>  			}
>  
>  			strbuf_release(&tmpname);

This runs setlen on tmpname strbuf and immediately releases (the close brace before release closes the "if (write_bitmap_index)", not any loop. If we plan to insert more code to use tmpname where the blank line we see above is in the future, it may make sense, but even in such a case, adding setlen() only when it starts to matter may make it easier to follow.

In any case, the above correctly adjusts tmpname to have a <hash> plus '.' at the end of tmpname to call finish_tmp_packfile().

Show 14 quoted lines
> diff --git a/bulk-checkin.c b/bulk-checkin.c
> index 6283bc8bd9..c19d471f0b 100644
> --- a/bulk-checkin.c
> +++ b/bulk-checkin.c
> @@ -46,7 +46,8 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state)
>  		close(fd);
>  	}
>  
> -	strbuf_addf(&packname, "%s/pack/pack-", get_object_directory());
> +	strbuf_addf(&packname, "%s/pack/pack-%s.", get_object_directory(),
> +		    hash_to_hex(hash));
>  	finish_tmp_packfile(&packname, state->pack_tmp_name,
>  			    state->written, state->nr_written,
>  			    &state->pack_idx_opts, hash);
OK.
Show 17 quoted lines
> diff --git a/pack-write.c b/pack-write.c
> index 2767b78619..95b063be94 100644
> --- a/pack-write.c
> +++ b/pack-write.c
> @@ -458,6 +458,18 @@ struct hashfile *create_tmp_packfile(char **pack_tmp_name)
>  	return hashfd(fd, *pack_tmp_name);
>  }
>  
> +static void rename_tmp_packfile(struct strbuf *nb, const char *source,
> +				const char *ext)
> +{
> +	size_t nb_len = nb->len;
> +
> +	strbuf_addstr(nb, ext);
> +	if (rename(source, nb->buf))
> +		die_errno("unable to rename temporary '*.%s' file to '%s'",
> +			  ext, nb->buf);

I do not like '*.%s' here. Without '*' it is clear enough, and because nb->buf has already the same ext information,

	unable to rename temporary file to '%s'.

would be even simpler without losing any information than this rewrite does.

The original explicitly used the more human-facing terms like "pack file", so we are losing information by this refactoring, but the loss is not too bad. In one case, namely ".idx" files, it is even better, as the original says "temporary index file" to refer to the new .idx file, which makes it ambiguous with _the_ index file.

> +	strbuf_setlen(nb, nb_len);
> +}

In the longer term if we were to add more auxiliary files next to each of the .pack file, it makes perfect sense that the common prefix is fed to rename_tmp_packfile() and the function reverts contents of the prefix buffer back when it is done with it. But it would be more friendly to those adding more calls to this function in the future to document the convention in a comment before the function.

Also, the name "nb" would need rethinking. As far as the callers are concerned, that is not a full name, but they are supplying the common prefix to the function. Perhaps "struct strbuf *name_prefix" or soemthing might be better? I dunno.

Show 14 quoted lines
>  void finish_tmp_packfile(struct strbuf *name_buffer,
>  			 const char *pack_tmp_name,
>  			 struct pack_idx_entry **written_list,
> @@ -466,7 +478,6 @@ void finish_tmp_packfile(struct strbuf *name_buffer,
>  			 unsigned char hash[])
>  {
>  	const char *idx_tmp_name, *rev_tmp_name = NULL;
> -	int basename_len = name_buffer->len;
>  
>  	if (adjust_shared_perm(pack_tmp_name))
>  		die_errno("unable to make temporary pack file readable");
> @@ -479,26 +490,10 @@ void finish_tmp_packfile(struct strbuf *name_buffer,
>  	rev_tmp_name = write_rev_file(NULL, written_list, nr_written, hash,
>  				      pack_idx_opts->flags);

It is unfortunate that write_idx_file() and write_rev_file() take hash and come up with the temporary filename on their own (which means their resulting filenames may not share the same prefix as .pack and .idx files), but it is just the naming of temporaries and inconsistencies among them is, eh, temporary, so it is OK.

Show 7 quoted lines
> +	rename_tmp_packfile(name_buffer, pack_tmp_name, "pack");
> +	rename_tmp_packfile(name_buffer, idx_tmp_name, "idx");
> +	if (rev_tmp_name)
> +		rename_tmp_packfile(name_buffer, rev_tmp_name, "rev");
>  
>  	free((void *)idx_tmp_name);
>  }
Previous: Taylor BlauNext: Taylor Blau
Message 45 of 81 in “pack-write,repack: prevent opening packs too early”
  1. 0/2 pack-write,repack: prevent opening packs too earlyTaylor Blau, Sep 1, 2021
  2. 1/2 pack-write.c: rename `.idx` file into place lastTaylor Blau, Sep 1, 2021
  3. 2/2 builtin/repack.c: move `.idx` files into place lastTaylor Blau, Sep 1, 2021
  4. Jeff KingSep 1, 2021
  5. Taylor BlauSep 1, 2021
  6. Jeff KingSep 1, 2021
  7. Taylor BlauSep 1, 2021
  8. Jeff KingSep 1, 2021
  9. Taylor BlauSep 1, 2021
  10. Jeff KingSep 7, 2021
  11. 0/3 rename *.idx file into place last (also after *.bitmap)Ævar Arnfjörð Bjarmason, Sep 7, 2021
  12. 1/3 pack-write: use more idiomatic strbuf usage for packname constructionÆvar Arnfjörð Bjarmason, Sep 7, 2021
  13. Taylor BlauSep 7, 2021
  14. Ævar Arnfjörð BjarmasonSep 7, 2021
  15. 2/3 pack-write: split up finish_tmp_packfile() functionÆvar Arnfjörð Bjarmason, Sep 7, 2021
  16. Taylor BlauSep 7, 2021
  17. 3/3 pack-write: rename *.idx file into place last (really!)Ævar Arnfjörð Bjarmason, Sep 7, 2021
  18. Taylor BlauSep 7, 2021
  19. Taylor BlauSep 7, 2021
  20. Ævar Arnfjörð BjarmasonSep 7, 2021
  21. 0/4 rename *.idx file into place last (also after *.bitmap)Ævar Arnfjörð Bjarmason, Sep 8, 2021
  22. 1/4 pack.h: line-wrap the definition of finish_tmp_packfile()Ævar Arnfjörð Bjarmason, Sep 8, 2021
  23. 2/4 pack-write: refactor renaming in finish_tmp_packfile()Ævar Arnfjörð Bjarmason, Sep 8, 2021
  24. Taylor BlauSep 8, 2021
  25. 3/4 pack-write: split up finish_tmp_packfile() functionÆvar Arnfjörð Bjarmason, Sep 8, 2021
  26. 4/4 pack-write: rename *.idx file into place last (really!)Ævar Arnfjörð Bjarmason, Sep 8, 2021
  27. Ævar Arnfjörð BjarmasonSep 8, 2021
  28. Ævar Arnfjörð BjarmasonSep 8, 2021
  29. Taylor BlauSep 8, 2021
  30. 0/3 prevent opening packs too earlyTaylor Blau, Sep 8, 2021
  31. 1/3 pack-write.c: rename `.idx` files into place lastTaylor Blau, Sep 8, 2021
  32. 2/3 builtin/repack.c: move `.idx` files into place lastTaylor Blau, Sep 8, 2021
  33. 3/3 builtin/index-pack.c: move `.idx` files into place lastTaylor Blau, Sep 8, 2021
  34. Ævar Arnfjörð BjarmasonSep 8, 2021
  35. Ævar Arnfjörð BjarmasonSep 9, 2021
  36. Taylor BlauSep 9, 2021
  37. Ævar Arnfjörð BjarmasonSep 9, 2021
  38. Ævar Arnfjörð BjarmasonSep 9, 2021
  39. Taylor BlauSep 9, 2021
  40. 0/9 packfile: avoid .idx rename racesTaylor Blau, Sep 9, 2021
  41. 1/9 pack.h: line-wrap the definition of finish_tmp_packfile()Taylor Blau, Sep 9, 2021
  42. 2/9 bulk-checkin.c: store checksum directlyTaylor Blau, Sep 9, 2021
  43. Ævar Arnfjörð BjarmasonSep 9, 2021
  44. 3/9 pack-write: refactor renaming in finish_tmp_packfile()Taylor Blau, Sep 9, 2021
  45. Junio C HamanoSep 9, 2021
  46. Taylor BlauSep 9, 2021
  47. Junio C HamanoSep 9, 2021
  48. Taylor BlauSep 9, 2021
  49. Junio C HamanoSep 10, 2021
  50. 4/9 pack-write.c: rename `.idx` files after `*.rev`Taylor Blau, Sep 9, 2021
  51. Ævar Arnfjörð BjarmasonSep 9, 2021
  52. Taylor BlauSep 9, 2021
  53. Junio C HamanoSep 9, 2021
  54. 5/9 builtin/repack.c: move `.idx` files into place lastTaylor Blau, Sep 9, 2021
  55. Junio C HamanoSep 9, 2021
  56. Taylor BlauSep 9, 2021
  57. 6/9 index-pack: refactor renaming in final()Taylor Blau, Sep 9, 2021
  58. Junio C HamanoSep 9, 2021
  59. Taylor BlauSep 9, 2021
  60. 8/9 pack-write: split up finish_tmp_packfile() functionTaylor Blau, Sep 9, 2021
  61. 7/9 builtin/index-pack.c: move `.idx` files into place lastTaylor Blau, Sep 9, 2021
  62. Ævar Arnfjörð BjarmasonSep 9, 2021
  63. Junio C HamanoSep 9, 2021
  64. 9/9 pack-objects: rename .idx files into place after .bitmap filesTaylor Blau, Sep 9, 2021
  65. Ævar Arnfjörð BjarmasonSep 9, 2021
  66. Junio C HamanoSep 9, 2021
  67. Taylor BlauSep 9, 2021
  68. Ævar Arnfjörð BjarmasonSep 9, 2021
  69. Taylor BlauSep 9, 2021
  70. Junio C HamanoSep 9, 2021
  71. 0/9 packfile: avoid .idx rename racesTaylor Blau, Sep 9, 2021
  72. 1/9 pack.h: line-wrap the definition of finish_tmp_packfile()Taylor Blau, Sep 9, 2021
  73. 2/9 bulk-checkin.c: store checksum directlyTaylor Blau, Sep 9, 2021
  74. 3/9 pack-write: refactor renaming in finish_tmp_packfile()Taylor Blau, Sep 9, 2021
  75. 4/9 pack-write.c: rename `.idx` files after `*.rev`Taylor Blau, Sep 9, 2021
  76. 5/9 builtin/repack.c: move `.idx` files into place lastTaylor Blau, Sep 9, 2021
  77. 6/9 index-pack: refactor renaming in final()Taylor Blau, Sep 9, 2021
  78. 8/9 pack-write: split up finish_tmp_packfile() functionTaylor Blau, Sep 9, 2021
  79. 7/9 builtin/index-pack.c: move `.idx` files into place lastTaylor Blau, Sep 9, 2021
  80. 9/9 pack-objects: rename .idx files into place after .bitmap filesTaylor Blau, Sep 9, 2021
  81. Junio C HamanoSep 10, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.