git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 3/4] bulk-checkin: require transaction for index_blob_bulk_checkin()

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 22, 2025, 16:49 UTC
Message-ID
<xmqqqzx3xpwc.fsf@gitster.g>
In-Reply-To
<20250821232249.319427-4-jltobler@gmail.com>
Justin Tobler <jltobler@gmail.com> writes:
Show 45 quoted lines
>  /*
> - * This creates one packfile per large blob unless bulk-checkin
> - * machinery is "plugged".
> + * This writes the specified object to a packfile. Objects written here
> + * during the same transaction are written to the same packfile. The
> + * packfile is not flushed until the transaction is flushed. The caller
> + * is expected to ensure a valid transaction is setup for objects to be
> + * recorded to.
>   *
>   * This also bypasses the usual "convert-to-git" dance, and that is on
>   * purpose. We could write a streaming version of the converting
> diff --git a/object-file.c b/object-file.c
> index 1740aa2b2e3..bc15af42450 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -1253,19 +1253,26 @@ int index_fd(struct index_state *istate, struct object_id *oid,
>  	 * Call xsize_t() only when needed to avoid potentially unnecessary
>  	 * die() for large files.
>  	 */
> -	if (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path))
> +	if (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path)) {
>  		ret = index_stream_convert_blob(istate, oid, fd, path, flags);
> -	else if (!S_ISREG(st->st_mode))
> +	} else if (!S_ISREG(st->st_mode)) {
>  		ret = index_pipe(istate, oid, fd, type, path, flags);
> -	else if ((st->st_size >= 0 && (size_t) st->st_size <= repo_settings_get_big_file_threshold(istate->repo)) ||
> -		 type != OBJ_BLOB ||
> -		 (path && would_convert_to_git(istate, path)))
> +	} else if ((st->st_size >= 0 &&
> +		    (size_t)st->st_size <= repo_settings_get_big_file_threshold(istate->repo)) ||
> +		   type != OBJ_BLOB ||
> +		   (path && would_convert_to_git(istate, path))) {
>  		ret = index_core(istate, oid, fd, xsize_t(st->st_size),
>  				 type, path, flags);
> -	else
> -		ret = index_blob_bulk_checkin(the_repository->objects->transaction,
> +	} else {
> +		struct odb_transaction *transaction;
> +
> +		transaction = begin_odb_transaction(the_repository->objects);
> +		ret = index_blob_bulk_checkin(transaction,
>  					      oid, fd, xsize_t(st->st_size),
>  					      path, flags);
> +		end_odb_transaction(transaction);
> +	}

Interesting. If the caller does the odb transaction management itself by calling begin/end before calling this function and fed two or more large objects, the original code did the right thing with .nesting set to 1. In the new code, it still does the right thing even during the call to index_blob_bulk_checkin we raise .nesting by one, because the all non-zero .nesting values mean the same thing to the machinery, thanks to lazy initialization of the .objdir.

So, isn't the comment above the function now less accurate than before? The caller of this function does not have to do anything and we do not expect the caller to "ensure a valid transaction" at all, no?

Thanks.
Previous: Justin ToblerNext: Justin Tobler
Message 12 of 27 in “bulk-checkin: remove global transaction state”
  1. 0/3 bulk-checkin: remove global transaction stateJustin Tobler, Aug 20, 2025
  2. 1/3 bulk-checkin: introduce object database transaction structureJustin Tobler, Aug 20, 2025
  3. 2/3 bulk-checkin: remove global transaction stateJustin Tobler, Aug 20, 2025
  4. 3/3 bulk-checkin: wire repository variableJustin Tobler, Aug 20, 2025
  5. Junio C HamanoAug 21, 2025
  6. Justin ToblerAug 21, 2025
  7. Junio C HamanoAug 21, 2025
  8. Junio C HamanoAug 21, 2025
  9. 0/4 bulk-checkin: remove global transaction stateJustin Tobler, Aug 21, 2025
  10. 1/4 bulk-checkin: introduce object database transaction structureJustin Tobler, Aug 21, 2025
  11. 3/4 bulk-checkin: require transaction for index_blob_bulk_checkin()Justin Tobler, Aug 21, 2025
  12. Junio C HamanoAug 22, 2025
  13. Justin ToblerAug 22, 2025
  14. Junio C HamanoAug 22, 2025
  15. 2/4 bulk-checkin: remove global transaction stateJustin Tobler, Aug 21, 2025
  16. Junio C HamanoAug 22, 2025
  17. Justin ToblerAug 22, 2025
  18. Junio C HamanoAug 22, 2025
  19. 4/4 bulk-checkin: use repository variable from transactionJustin Tobler, Aug 21, 2025
  20. Junio C HamanoAug 22, 2025
  21. Justin ToblerAug 22, 2025
  22. 0/4 bulk-checkin: remove global transaction stateJustin Tobler, Aug 22, 2025
  23. 1/4 bulk-checkin: introduce object database transaction structureJustin Tobler, Aug 22, 2025
  24. 2/4 bulk-checkin: remove global transaction stateJustin Tobler, Aug 22, 2025
  25. 3/4 bulk-checkin: require transaction for index_blob_bulk_checkin()Justin Tobler, Aug 22, 2025
  26. 4/4 bulk-checkin: use repository variable from transactionJustin Tobler, Aug 22, 2025
  27. Junio C HamanoAug 25, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.