threads / patch / 52284

patchenable a timeout for hold_lock_file_for_update

Subject: [PATCH] enable a timeout for hold_lock_file_for_update

## tl;dr

4 messages between Nov 18, 2019 and Nov 20, 2019. Diffs are folded; open one to read it.

replies: 3people: 3as markdown or json

Martin Nicolay· Nov 18, 2019, 13:42 UTC · lore

The new funktion get_files_lock_timeout_ms reads the config core.filesLockTimeout analog get_files_ref_lock_timeout_ms.

This value is used in hold_lock_file_for_update instead of the fixed value 0. --- While working with complex scripts invoking git multiple times my editor (emacs with standard version control) detects the changes and apparently calls "git status". This leads to abort in "git stash". With this patch and an appropriate value core.fileslocktimeout this problem goes away.

While it may be possible to patch the elisp scripts of emacs (and all other similar callers) to call "git status" with --no-optional-locks it seems to me a better approarch to solve this problem at its root: calling hold_lock_file_for_update_timeout with a timeout of 0 ms.

The implementation with the function get_files_lock_timeout_ms is adopted from a similar usage of get_files_ref_lock_timeout_ms.

BTW: is there a way to link this version of the patch to the previous
version to reduce the work for reviewers?
 Documentation/config/core.txt |  6 ++++++
 lockfile.c                    | 16 ++++++++++++++++
 lockfile.h                    |  4 +++-
 3 files changed, 25 insertions(+), 1 deletion(-)
Show changes to 3 files +25 −1

Documentation/config/core.txt, lockfile.c, lockfile.h

diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt
index 852d2ba37a..230ea02560 100644
--- a/Documentation/config/core.txt
+++ b/Documentation/config/core.txt
@@ -482,6 +482,12 @@ core.packedRefsTimeout::
 	all; -1 means to try indefinitely. Default is 1000 (i.e.,
 	retry for 1 second).
 
+core.filesLockTimeout::
+	The length of time, in milliseconds, to retry when trying to
+	lock an individual file. Value 0 means not to retry at
+	all; -1 means to try indefinitely. Default is 0 (i.e., don't
+	retry at all).
+
 core.pager::
 	Text viewer for use by Git commands (e.g., 'less').  The value
 	is meant to be interpreted by the shell.  The order of preference
diff --git a/lockfile.c b/lockfile.c
index 8e8ab4f29f..7301f393d6 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -145,6 +145,22 @@ static int lock_file_timeout(struct lock_file *lk, const char *path,
 	}
 }
 
+/*
+ * Get timeout for hold_lock_file_for_update.
+ */
+long get_files_lock_timeout_ms(void)
+{
+	static int configured = 0;
+
+	static int timeout_ms = 0; /* default */
+	if (!configured) {
+		git_config_get_int("core.fileslocktimeout", &timeout_ms);
+		configured = 1;
+	}
+
+	return timeout_ms;
+}
+
 void unable_to_lock_message(const char *path, int err, struct strbuf *buf)
 {
 	if (err == EEXIST) {
diff --git a/lockfile.h b/lockfile.h
index 9843053ce8..a0520e6a7b 100644
--- a/lockfile.h
+++ b/lockfile.h
@@ -163,6 +163,8 @@ int hold_lock_file_for_update_timeout(
 		struct lock_file *lk, const char *path,
 		int flags, long timeout_ms);
 
+long get_files_lock_timeout_ms(void);
+
 /*
  * Attempt to create a lockfile for the file at `path` and return a
  * file descriptor for writing to it, or -1 on error. The flags
@@ -172,7 +174,7 @@ static inline int hold_lock_file_for_update(
 		struct lock_file *lk, const char *path,
 		int flags)
 {
-	return hold_lock_file_for_update_timeout(lk, path, flags, 0);
+	return hold_lock_file_for_update_timeout(lk, path, flags, get_files_lock_timeout_ms() );
 }
 
 /*

-- 
2.13.7
Denton Liu· Nov 18, 2019, 17:57 UTC · re: Martin Nicolay · lore

Re: [PATCH] enable a timeout for hold_lock_file_for_update

Hi Martin,

Thanks for the patch. I'm not too familiar with the lockfile codebase of git but here are some general comments:

> Subject: [PATCH] enable a timeout for hold_lock_file_for_update

We prefer subjects in the format of "<area>: <brief subject>" so perhaps, we could rewrite this to:

	lockfile: learn core.filesLockTimeout configuration
On Mon, Nov 18, 2019 at 02:42:17PM +0100, Martin Nicolay wrote:
> The new funktion get_files_lock_timeout_ms reads the config
s/funktion/function/
> core.filesLockTimeout analog get_files_ref_lock_timeout_ms.
Perhaps s/analog/similar to/ ?
Show 18 quoted lines
> 
> This value is used in hold_lock_file_for_update instead of the
> fixed value 0.
> ---
> While working with complex scripts invoking git multiple times
> my editor (emacs with standard version control) detects the
> changes and apparently calls "git status". This leads to abort
> in "git stash". With this patch and an appropriate value
> core.fileslocktimeout this problem goes away.
> 
> While it may be possible to patch the elisp scripts of emacs (and
> all other similar callers) to call "git status" with
> --no-optional-locks it seems to me a better approarch to solve this
> problem at its root: calling hold_lock_file_for_update_timeout with
> a timeout of 0 ms.
> 
> The implementation with the function get_files_lock_timeout_ms is 
> adopted from a similar usage of get_files_ref_lock_timeout_ms.

It might be good to include the above three paragraphs in your commit message. Not only do they describe the change but, more importantly, they describe _why_ the change is being made.

> 
> BTW: is there a way to link this version of the patch to the previous
> version to reduce the work for reviewers?
When you generate your patches, run
	git format-patch --in-reply-to=<r> -v<n>

where <r> is the Message-ID of your last patch and where <n> is the version of the patch (in this case, it should've been 2 since you sent out one before).

Show 69 quoted lines
> 
>  Documentation/config/core.txt |  6 ++++++
>  lockfile.c                    | 16 ++++++++++++++++
>  lockfile.h                    |  4 +++-
>  3 files changed, 25 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt
> index 852d2ba37a..230ea02560 100644
> --- a/Documentation/config/core.txt
> +++ b/Documentation/config/core.txt
> @@ -482,6 +482,12 @@ core.packedRefsTimeout::
>  	all; -1 means to try indefinitely. Default is 1000 (i.e.,
>  	retry for 1 second).
>  
> +core.filesLockTimeout::
> +	The length of time, in milliseconds, to retry when trying to
> +	lock an individual file. Value 0 means not to retry at
> +	all; -1 means to try indefinitely. Default is 0 (i.e., don't
> +	retry at all).
> +
>  core.pager::
>  	Text viewer for use by Git commands (e.g., 'less').  The value
>  	is meant to be interpreted by the shell.  The order of preference
> diff --git a/lockfile.c b/lockfile.c
> index 8e8ab4f29f..7301f393d6 100644
> --- a/lockfile.c
> +++ b/lockfile.c
> @@ -145,6 +145,22 @@ static int lock_file_timeout(struct lock_file *lk, const char *path,
>  	}
>  }
>  
> +/*
> + * Get timeout for hold_lock_file_for_update.
> + */
> +long get_files_lock_timeout_ms(void)
> +{
> +	static int configured = 0;
> +
> +	static int timeout_ms = 0; /* default */
> +	if (!configured) {
> +		git_config_get_int("core.fileslocktimeout", &timeout_ms);
> +		configured = 1;
> +	}
> +
> +	return timeout_ms;
> +}
> +
>  void unable_to_lock_message(const char *path, int err, struct strbuf *buf)
>  {
>  	if (err == EEXIST) {
> diff --git a/lockfile.h b/lockfile.h
> index 9843053ce8..a0520e6a7b 100644
> --- a/lockfile.h
> +++ b/lockfile.h
> @@ -163,6 +163,8 @@ int hold_lock_file_for_update_timeout(
>  		struct lock_file *lk, const char *path,
>  		int flags, long timeout_ms);
>  
> +long get_files_lock_timeout_ms(void);
> +
>  /*
>   * Attempt to create a lockfile for the file at `path` and return a
>   * file descriptor for writing to it, or -1 on error. The flags
> @@ -172,7 +174,7 @@ static inline int hold_lock_file_for_update(
>  		struct lock_file *lk, const char *path,
>  		int flags)
>  {
> -	return hold_lock_file_for_update_timeout(lk, path, flags, 0);
> +	return hold_lock_file_for_update_timeout(lk, path, flags, get_files_lock_timeout_ms() );
Style nit: remove the space after the function call.
Thanks,
Denton
Show 6 quoted lines
>  }
>  
>  /*
> -- 
> 2.13.7
>
Martin Nicolay· Nov 19, 2019, 14:56 UTC · re: Martin Nicolay · lore

[PATCH v3] lockfile: learn core.filesLockTimeout configuration

The new function get_files_lock_timeout_ms reads the config core.filesLockTimeout similar to get_files_ref_lock_timeout_ms. This value is used in hold_lock_file_for_update instead of the fixed value 0.

While working with complex scripts invoking git multiple times my editor (emacs with standard version control) detects the changes and apparently calls "git status". This leads to abort in "git stash". With this patch and an appropriate value core.filesLockTimeout this problem goes away.

While it may be possible to patch the elisp scripts of emacs (and all other similar callers) to call "git status" with --no-optional-locks it seems to me a better approarch to solve this problem at its root: calling hold_lock_file_for_update_timeout with a timeout of 0 ms.

The implementation with the function get_files_lock_timeout_ms is
adopted from a similar usage of get_files_ref_lock_timeout_ms.
---
 Documentation/config/core.txt |  6 ++++++
 lockfile.c                    | 16 ++++++++++++++++
 lockfile.h                    |  4 +++-
 3 files changed, 25 insertions(+), 1 deletion(-)
Show changes to 3 files +25 −1

Documentation/config/core.txt, lockfile.c, lockfile.h

diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt
index 852d2ba37a..230ea02560 100644
--- a/Documentation/config/core.txt
+++ b/Documentation/config/core.txt
@@ -482,6 +482,12 @@ core.packedRefsTimeout::
 	all; -1 means to try indefinitely. Default is 1000 (i.e.,
 	retry for 1 second).
 
+core.filesLockTimeout::
+	The length of time, in milliseconds, to retry when trying to
+	lock an individual file. Value 0 means not to retry at
+	all; -1 means to try indefinitely. Default is 0 (i.e., don't
+	retry at all).
+
 core.pager::
 	Text viewer for use by Git commands (e.g., 'less').  The value
 	is meant to be interpreted by the shell.  The order of preference
diff --git a/lockfile.c b/lockfile.c
index 8e8ab4f29f..7301f393d6 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -145,6 +145,22 @@ static int lock_file_timeout(struct lock_file *lk, const char *path,
 	}
 }
 
+/*
+ * Get timeout for hold_lock_file_for_update.
+ */
+long get_files_lock_timeout_ms(void)
+{
+	static int configured = 0;
+
+	static int timeout_ms = 0; /* default */
+	if (!configured) {
+		git_config_get_int("core.fileslocktimeout", &timeout_ms);
+		configured = 1;
+	}
+
+	return timeout_ms;
+}
+
 void unable_to_lock_message(const char *path, int err, struct strbuf *buf)
 {
 	if (err == EEXIST) {
diff --git a/lockfile.h b/lockfile.h
index 9843053ce8..c7e7db0d5e 100644
--- a/lockfile.h
+++ b/lockfile.h
@@ -163,6 +163,8 @@ int hold_lock_file_for_update_timeout(
 		struct lock_file *lk, const char *path,
 		int flags, long timeout_ms);
 
+long get_files_lock_timeout_ms(void);
+
 /*
  * Attempt to create a lockfile for the file at `path` and return a
  * file descriptor for writing to it, or -1 on error. The flags
@@ -172,7 +174,7 @@ static inline int hold_lock_file_for_update(
 		struct lock_file *lk, const char *path,
 		int flags)
 {
-	return hold_lock_file_for_update_timeout(lk, path, flags, 0);
+	return hold_lock_file_for_update_timeout(lk, path, flags, get_files_lock_timeout_ms());
 }
 
 /*

-- 
2.13.7
Junio C Hamano· Nov 20, 2019, 02:38 UTC · re: Martin Nicolay · lore

Re: [PATCH v3] lockfile: learn core.filesLockTimeout configuration

Martin Nicolay <m.nicolay@osm-ag.de> writes:
Show 20 quoted lines
> The new function get_files_lock_timeout_ms reads the config
> core.filesLockTimeout similar to get_files_ref_lock_timeout_ms.
> This value is used in hold_lock_file_for_update instead of the
> fixed value 0.
>
> While working with complex scripts invoking git multiple times
> my editor (emacs with standard version control) detects the
> changes and apparently calls "git status". This leads to abort
> in "git stash". With this patch and an appropriate value
> core.filesLockTimeout this problem goes away.
>
> While it may be possible to patch the elisp scripts of emacs (and
> all other similar callers) to call "git status" with
> --no-optional-locks it seems to me a better approarch to solve this
> problem at its root: calling hold_lock_file_for_update_timeout with
> a timeout of 0 ms.
>
> The implementation with the function get_files_lock_timeout_ms is
> adopted from a similar usage of get_files_ref_lock_timeout_ms.
> ---
Missing sign-off before the three-dash line.

I think the last paragraph can be left without. It is not like there are many other sensible ways to get a configured value without making repeated calls to git_config_get_int().

Show 5 quoted lines
> +core.filesLockTimeout::
> +	The length of time, in milliseconds, to retry when trying to
> +	lock an individual file. Value 0 means not to retry at
> +	all; -1 means to try indefinitely. Default is 0 (i.e., don't
> +	retry at all).

Will there be *NO* callers of the lockfile API functions that do not honor the value taken from this configuration variable after this patch is applied?

Otherwise, users who set this configuration variable and hit a codepath that locks files without asking get_files_lock_timeout_ms() how it should retry would find the above description inaccurate, wouldn't they?

There is another question---is it safe to make all attempts to create a lockfile retry, possibly forever? I do not offhand think of any concrete example, but I would not be surprised if there is a codepath that would never want to retry but want to fail upon the first failure (i.e. wants to always use value 0 without allowing the users to configure).

So, unless the answers to the above two questions are "with this patch, all attempts to lock will honor this variable" and "yes it is safe because ...", some tweak of the description may be necessary to hint the readers that not all the locks will retry by honoring this variable.

Show 12 quoted lines
> diff --git a/lockfile.c b/lockfile.c
> index 8e8ab4f29f..7301f393d6 100644
> --- a/lockfile.c
> +++ b/lockfile.c
> @@ -145,6 +145,22 @@ static int lock_file_timeout(struct lock_file *lk, const char *path,
>  	}
>  }
>  
> +/*
> + * Get timeout for hold_lock_file_for_update.
> + */
> +long get_files_lock_timeout_ms(void)

Shouldn't this return "int", which is the type you get from the underlying configuration API?

I also wondered if this has to be extern at all; the reason why this patch makes it extern is purely because hold_lock_file_for_update() is defined as a static inline in lockfile.h to expand to another function, so any file that includes lockfile.h and calls that static inline must be able to see this name.

Because all of the lockfile public API functions are about accessing filesystem entities, I am not sure if making this many thin wrappers static inlines to potentially save one extra intermediate call is worth it (there are 10 of them). For now, I think the organization this patch leaves is OK, but we may later want to examine these static inline wrappers and consider turning them into a regular extern functions.

I do not think other static inline wrappers in the lockfile.h is hurting right now, but with this change, hold_lock_file_for_update() certainly is. If it becomes just a usual extern function, we do not have to expose the get-files-lock-timeout-ms helper at all (and worry about its name, as globally visible names needs extra care to help developers). In any case, that is outside the scope of this patch, but a potential follow-on work after this patch stabilizes.

Show 5 quoted lines
> +{
> +	static int configured = 0;
> +
> +	static int timeout_ms = 0; /* default */
> +	if (!configured) {
 - Do not explicitly initialize statics to zero (instead, let BSS take
   care of it).
 - Lose the blank line between the declarations.
 - Have a blank line after the last decl and the first statement.
Show 6 quoted lines
> +		git_config_get_int("core.fileslocktimeout", &timeout_ms);
> +		configured = 1;
> +	}
> +
> +	return timeout_ms;
> +}

By the way, why are these called file*S*locktimeout (both the end-user facing configuration variable and the function name)?

Also, "lock timeout" is a misleading name for both the configuration variable and the function. It sounds as if after that many milliseconds, the system will automatically break your lock if you do not perform the action under the lock quickly enough, but that is a wrong message to send to the end users.

The timeout is about retrying to acquire the lock, so the name most likely needs to have words "lock", "retry", and "timeout" somewhere in it.

Perhaps core.lockRetryTimeout or something? I dunno. You may want to wait before others offer a better name before rerolling, as I am not very good at naming things.

Thanks.

← back to recent threads