Re: [PATCH v4 4/4] object-file: write ".keep" lockfiles during transaction commit
- From
Karthik Nayak <karthik.188@gmail.com>
- Date
- Oct 26, 2026, 21:49 UTC
- Message-ID
- <CAOLa=ZTDm25XDambq4ohmeURaBuE29EwS6ORhHc6PDVaKyGWRA@mail.gmail.com>
- In-Reply-To
- <20261021182528.3055494-5-jltobler@gmail.com>
Justin Tobler <jltobler@gmail.com> writes:
Show 25 quoted lines
> The "files" ODB backend writes a ".keep" lockfile into the transaction > quarantine directory via `odb_transaction_files_write_pack()`. When the > ODB transaction is committed, the ".keep" lockfile is migrated to the > primary ODB alongside its packfile. Writing the temporary lockfile in > this manner has a couple of downsides: > > - Migrating quarantined objects via `tmp_objdir_migrate()` doesn't > immediately stop if a ".keep" lockfile already exists for the same > packfile in the primary ODB and instead allows all the remaining > objects to be migrated. So even though `odb_transaction_commit()` > ultimately returns an error here, the objects are needlessly written > to the primary ODB anyways. > > - Even more concerning, the ".keep" tempfile atexit(3) handler tries > to remove any lockfile of the same name regardless of whether it was > created by the same process. If the ".keep" lockfile of the same > name for a concurrent process gets deleted prematurely, its packfile > becomes at risk of being deleted by concurrent housekeeping. > > Stop creating/registering the temporary lockfiles when writing the > packfile to the transaction and instead do so when committing the > transaction. This ensures the lockfile is properly acquired prior to > migrating objects and that only the owning process can delete its own > lockfiles. >
So to reiterate, the old method was to ask index-pack to also write the '.keep' file in the quarantine directory. Post that, all the files in the directory were moved to the canonical location.
Now, we handle '.keep' file creation directly. Nit: There is a logical jump from the last-1 sentence to the last sentence. How does moving the lockfile creation to the commit phase solve these issues?
Show 74 quoted lines
> Reported-by: Qin ShiCheng <qeesung@live.com>
> Signed-off-by: Justin Tobler <jltobler@gmail.com>
> ---
> object-file.c | 76 +++++++++++++++++++++-----------------
> t/t5547-push-quarantine.sh | 30 +++++++++++++--
> 2 files changed, 70 insertions(+), 36 deletions(-)
>
> diff --git a/object-file.c b/object-file.c
> index 74a31760bc..e67c0865a2 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -492,6 +492,7 @@ struct odb_transaction_files {
> struct transaction_packfile packfile;
> const char *prefix;
>
> + struct strvec pack_names;
> struct tempfile **pack_lockfiles;
> size_t pack_lockfiles_nr;
> size_t pack_lockfiles_alloc;
> @@ -1257,7 +1258,7 @@ int read_loose_object(struct repository *repo,
> }
>
> static int write_pack_lockfile(struct odb_source *source, const char *pack_name,
> - const char *keep_msg)
> + const char *keep_msg, struct tempfile **out)
> {
> struct strbuf lock_path = STRBUF_INIT;
> struct strbuf contents = STRBUF_INIT;
> @@ -1288,6 +1289,7 @@ static int write_pack_lockfile(struct odb_source *source, const char *pack_name,
> goto out;
> }
>
> + *out = register_tempfile(lock_path.buf);
> ret = 0;
>
> out:
> @@ -1297,6 +1299,40 @@ static int write_pack_lockfile(struct odb_source *source, const char *pack_name,
> return ret;
> }
>
> +static int write_pack_lockfiles(struct odb_transaction_files *transaction)
> +{
> + struct strbuf keep_msg = STRBUF_INIT;
> + char hostname[HOST_NAME_MAX + 1];
> + int ret = 0;
> +
> + if (!transaction->pack_names.nr)
> + return 0;
> +
> + if (xgethostname(hostname, sizeof(hostname)))
> + xsnprintf(hostname, sizeof(hostname), "localhost");
> + strbuf_addf(&keep_msg, "receive-pack %"PRIuMAX" on %s",
> + (uintmax_t)getpid(), hostname);
> +
> + for (size_t i = 0; i < transaction->pack_names.nr; i++) {
> + struct tempfile *lock;
> +
> + ret = write_pack_lockfile(transaction->base.source,
> + transaction->pack_names.v[i],
> + keep_msg.buf, &lock);
> + if (ret < 0)
> + break;
> +
> + ALLOC_GROW(transaction->pack_lockfiles,
> + transaction->pack_lockfiles_nr + 1,
> + transaction->pack_lockfiles_alloc);
> + transaction->pack_lockfiles[transaction->pack_lockfiles_nr++] = lock;
> + }
> +
> + strbuf_release(&keep_msg);
> +
> + return ret;
> +}
>Okay so this function handles the lockfile creation for each of the packs. Looks good.
[snip]
Show 57 quoted lines
> @@ -1434,14 +1471,9 @@ static int odb_transaction_files_write_pack(struct odb_transaction *base,
> return -1;
> }
> } else {
> - char hostname[HOST_NAME_MAX + 1];
> -
> strvec_pushl(&child.args, "index-pack", "--stdin", NULL);
> push_header_arg(&child.args, &hdr);
>
> - if (xgethostname(hostname, sizeof(hostname)))
> - xsnprintf(hostname, sizeof(hostname), "localhost");
> -
> if (!opts->quiet && err_fd)
> strvec_push(&child.args, "--show-resolving-progress");
> if (err_fd)
> @@ -1479,29 +1511,7 @@ static int odb_transaction_files_write_pack(struct odb_transaction *base,
> goto out;
> }
>
> - strbuf_addf(&keep_msg, "receive-pack %"PRIuMAX" on %s",
> - (uintmax_t)getpid(), hostname);
> -
> - if (write_pack_lockfile(transaction->quarantine, pack_name.buf,
> - keep_msg.buf)) {
> - strbuf_addstr(err_msg, "lockfile write failed");
> - goto out;
> - }
> -
> - ALLOC_GROW(transaction->pack_lockfiles,
> - transaction->pack_lockfiles_nr + 1,
> - transaction->pack_lockfiles_alloc);
> -
> - /*
> - * The lockfile filepath is expected to be the final location of
> - * the ".keep" file after being migrated to the main ODB source.
> - * This ensures the lockfile can be found and removed later
> - * after the ODB transaction has been committed.
> - */
> - strbuf_addf(&final_path, "%s/pack/pack-%s.keep",
> - base->source->path, pack_name.buf);
> - transaction->pack_lockfiles[transaction->pack_lockfiles_nr++] =
> - register_tempfile(final_path.buf);
> + strvec_push(&transaction->pack_names, pack_name.buf);
>
> odb_source_prepare(transaction->quarantine,
> ODB_PREPARE_FLUSH_CACHES);
> @@ -1510,9 +1520,7 @@ static int odb_transaction_files_write_pack(struct odb_transaction *base,
> ret = 0;
>
> out:
> - strbuf_release(&final_path);
> strbuf_release(&pack_name);
> - strbuf_release(&keep_msg);
>
> return ret;
> }
>So writing the individual pack no longer is concerned about also writing the keep file. Makes sense.
[snip]