git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/8] reftable/stack: simplify tracking of table locks

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 2, 2024, 23:00 UTC
Message-ID
<xmqqy15e8r8b.fsf@gitster.g>
In-Reply-To
<40d9f75cf20d4b76adb1683709e054e264d4e06f.1722435214.git.ps@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 23 quoted lines
> -	size_t i;
> +	size_t i, nlocks = 0;
>  
>  	if (first > last || (!expiry && first == last)) {
>  		err = 0;
> @@ -1051,7 +1051,7 @@ static int stack_compact_range(struct reftable_stack *st,
>  	for (i = first; i <= last; i++) {
>  		stack_filename(&table_name, st, reader_name(st->readers[i]));
>  
> -		err = hold_lock_file_for_update(&table_locks[i - first],
> +		err = hold_lock_file_for_update(&table_locks[nlocks],
>  						table_name.buf, LOCK_NO_DEREF);
>  		if (err < 0) {
>  			if (errno == EEXIST)
> @@ -1066,7 +1066,7 @@ static int stack_compact_range(struct reftable_stack *st,
>  		 * run into file descriptor exhaustion when we compress a lot
>  		 * of tables.
>  		 */
> -		err = close_lock_file_gently(&table_locks[i - first]);
> +		err = close_lock_file_gently(&table_locks[nlocks++]);
>  		if (err < 0) {
>  			err = REFTABLE_IO_ERROR;
>  			goto done;

The only unusual control flow in this loop that runs i from first to last is to leave it upon an error, so "i - first" and "nlocks" is always the same, at this step in the patch series.

Show 11 quoted lines
> @@ -1183,8 +1183,8 @@ static int stack_compact_range(struct reftable_stack *st,
>  	 * Delete the old tables. They may still be in use by concurrent
>  	 * readers, so it is expected that unlinking tables may fail.
>  	 */
> -	for (i = first; i <= last; i++) {
> -		struct lock_file *table_lock = &table_locks[i - first];
> +	for (i = 0; i < nlocks; i++) {
> +		struct lock_file *table_lock = &table_locks[i];
>  		char *table_path = get_locked_file_path(table_lock);
>  		unlink(table_path);
>  		free(table_path);

And this one at this step in the patch series is skipped if the earlier loop saw even a single error, so again, this is a benign noop change.

Show 8 quoted lines
> @@ -1192,8 +1192,8 @@ static int stack_compact_range(struct reftable_stack *st,
>  
>  done:
>  	rollback_lock_file(&tables_list_lock);
> -	for (i = first; table_locks && i <= last; i++)
> -		rollback_lock_file(&table_locks[i - first]);
> +	for (i = 0; table_locks && i < nlocks; i++)
> +		rollback_lock_file(&table_locks[i]);

This is a true bugfix, isn't it? If we failed to create lock file somewhere in the middle, we used to still go ahead and attempted rollback_lock_file() on all of them. Now we rollback only what we successfully called hold_lock_file_for_update().

I wonder why nobody segfaulted where after a failed lock. The answer probably is that lk->tempfile that is NULL will safely bypass most of the things because is_tempfile_active() would say "false" on such a lockfile. But still it probably still were wrong to call rollback_lock_file() on a "struct lockfile" full of NUL-bytes, and it is good that we no longer do that.

>  	reftable_free(table_locks);
>  
>  	delete_tempfile(&new_table);
Previous: Justin ToblerNext: Patrick Steinhardt
Message 10 of 50 in “reftable: improvements and fixes for compaction”
  1. 0/8 reftable: improvements and fixes for compactionPatrick Steinhardt, Jul 31, 2024
  2. 1/8 reftable/stack: refactor function to gather table sizesPatrick Steinhardt, Jul 31, 2024
  3. Junio C HamanoAug 2, 2024
  4. 2/8 reftable/stack: test compaction with already-locked tablesPatrick Steinhardt, Jul 31, 2024
  5. Junio C HamanoAug 2, 2024
  6. Patrick SteinhardtAug 5, 2024
  7. 3/8 reftable/stack: update stats on failed full compactionPatrick Steinhardt, Jul 31, 2024
  8. 4/8 reftable/stack: simplify tracking of table locksPatrick Steinhardt, Jul 31, 2024
  9. Justin ToblerJul 31, 2024
  10. Junio C HamanoAug 2, 2024
  11. Patrick SteinhardtAug 5, 2024
  12. 5/8 reftable/stack: do not die when fsyncing lock file filesPatrick Steinhardt, Jul 31, 2024
  13. 6/8 reftable/stack: use lock_file when adding table to "tables.list"Patrick Steinhardt, Jul 31, 2024
  14. Justin ToblerJul 31, 2024
  15. Patrick SteinhardtAug 1, 2024
  16. 7/8 reftable/stack: fix corruption on concurrent compactionPatrick Steinhardt, Jul 31, 2024
  17. Justin ToblerAug 1, 2024
  18. Patrick SteinhardtAug 1, 2024
  19. 8/8 reftable/stack: handle locked tables during auto-compactionPatrick Steinhardt, Jul 31, 2024
  20. Junio C HamanoAug 2, 2024
  21. Patrick SteinhardtAug 5, 2024
  22. 0/9 reftable: improvements and fixes for compactionPatrick Steinhardt, Aug 5, 2024
  23. 1/9 reftable/stack: refactor function to gather table sizesPatrick Steinhardt, Aug 5, 2024
  24. 2/9 reftable/stack: extract function to setup stack with N tablesPatrick Steinhardt, Aug 5, 2024
  25. 3/9 reftable/stack: test compaction with already-locked tablesPatrick Steinhardt, Aug 5, 2024
  26. Karthik NayakAug 8, 2024
  27. 4/9 reftable/stack: update stats on failed full compactionPatrick Steinhardt, Aug 5, 2024
  28. 5/9 reftable/stack: simplify tracking of table locksPatrick Steinhardt, Aug 5, 2024
  29. 6/9 reftable/stack: do not die when fsyncing lock file filesPatrick Steinhardt, Aug 5, 2024
  30. 7/9 reftable/stack: use lock_file when adding table to "tables.list"Patrick Steinhardt, Aug 5, 2024
  31. 8/9 reftable/stack: fix corruption on concurrent compactionPatrick Steinhardt, Aug 5, 2024
  32. Karthik NayakAug 8, 2024
  33. Patrick SteinhardtAug 8, 2024
  34. 9/9 reftable/stack: handle locked tables during auto-compactionPatrick Steinhardt, Aug 5, 2024
  35. Justin ToblerAug 6, 2024
  36. Patrick SteinhardtAug 7, 2024
  37. Justin ToblerAug 7, 2024
  38. Karthik NayakAug 8, 2024
  39. Karthik NayakAug 8, 2024
  40. 0/9 reftable: improvements and fixes for compactionPatrick Steinhardt, Aug 8, 2024
  41. 1/9 reftable/stack: refactor function to gather table sizesPatrick Steinhardt, Aug 8, 2024
  42. 2/9 reftable/stack: extract function to setup stack with N tablesPatrick Steinhardt, Aug 8, 2024
  43. 3/9 reftable/stack: test compaction with already-locked tablesPatrick Steinhardt, Aug 8, 2024
  44. 4/9 reftable/stack: update stats on failed full compactionPatrick Steinhardt, Aug 8, 2024
  45. 5/9 reftable/stack: simplify tracking of table locksPatrick Steinhardt, Aug 8, 2024
  46. 6/9 reftable/stack: do not die when fsyncing lock file filesPatrick Steinhardt, Aug 8, 2024
  47. 7/9 reftable/stack: use lock_file when adding table to "tables.list"Patrick Steinhardt, Aug 8, 2024
  48. 8/9 reftable/stack: fix corruption on concurrent compactionPatrick Steinhardt, Aug 8, 2024
  49. 9/9 reftable/stack: handle locked tables during auto-compactionPatrick Steinhardt, Aug 8, 2024
  50. Karthik NayakAug 8, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.