# Do porcelain command require lock management? 4 messages from 2020-10-24 to 2020-10-26. Participants: Konstantin Ryabitsev, brian m. carlson, Junio C Hamano. Thread: https://gitlist.dev/t/54496 ## Konstantin Ryabitsev, 2020-10-24 14:46 Subject: Do porcelain command require lock management? Message-ID: <20201024144637.cvwa22f2y4tvfn4z@chatter.i7.local> URL: https://gitlist.dev/e/20201024144637.cvwa22f2y4tvfn4z%40chatter.i7.local ``` Hello: A script I'm writing performs a succession of porcelain commands to create a commit in a bare git repository: git hash-object git mktree git commit-tree git update-ref Do I need to manage external locking around these commands to avoid any concurrency problems, or will git take care of that? -K ``` ## brian m. carlson, 2020-10-25 23:02 Subject: Re: Do porcelain command require lock management? Message-ID: <20201025230203.GH860779@camp.crustytoothpaste.net> URL: https://gitlist.dev/e/20201025230203.GH860779%40camp.crustytoothpaste.net In-Reply-To: <20201024144637.cvwa22f2y4tvfn4z@chatter.i7.local> ``` On 2020-10-24 at 14:46:37, Konstantin Ryabitsev wrote: > Hello: > > A script I'm writing performs a succession of porcelain commands to > create a commit in a bare git repository: > > git hash-object > git mktree > git commit-tree > git update-ref > > Do I need to manage external locking around these commands to avoid any > concurrency problems, or will git take care of that? I'm almost certain that Git will do the same locking and object creation semantics that it does in porcelain commands as in the plumbing commands you're using. For example, I happen to know that all loose object creation goes through one function, which should gracefully handle concurrent accesses. Git is in general safe against concurrent accesses and is designed not to lose or corrupt data in this case. However, I will point out that ref updates may conflict and if so, Git will fail instead of waiting. So while your repository will remain consistent and won't experience corruption, that doesn't mean that all operations will complete successfully. Some sort of retry mechanism or other error handling will probably be warranted. -- brian m. carlson (he/him or they/them) Houston, Texas, US ``` ## Konstantin Ryabitsev, 2020-10-26 16:36 Subject: Re: Do porcelain command require lock management? Message-ID: <20201026163643.6cp4jwj5lqkg6vns@chatter.i7.local> URL: https://gitlist.dev/e/20201026163643.6cp4jwj5lqkg6vns%40chatter.i7.local In-Reply-To: <20201025230203.GH860779@camp.crustytoothpaste.net> ``` On Sun, Oct 25, 2020 at 11:02:03PM +0000, brian m. carlson wrote: > > A script I'm writing performs a succession of porcelain commands to > > create a commit in a bare git repository: > > > > git hash-object > > git mktree > > git commit-tree > > git update-ref > > > > Do I need to manage external locking around these commands to avoid any > > concurrency problems, or will git take care of that? > > I'm almost certain that Git will do the same locking and object creation > semantics that it does in porcelain commands as in the plumbing commands > you're using. For example, I happen to know that all loose object > creation goes through one function, which should gracefully handle > concurrent accesses. Git is in general safe against concurrent accesses > and is designed not to lose or corrupt data in this case. > > However, I will point out that ref updates may conflict and if so, Git > will fail instead of waiting. So while your repository will remain > consistent and won't experience corruption, that doesn't mean that all > operations will complete successfully. Some sort of retry mechanism or > other error handling will probably be warranted. Thanks for your advice. I think in this case it's easier to just use simple locking to make sure that concurrent processes don't step on each-other's toes. Best regards, -K ``` ## Junio C Hamano, 2020-10-26 17:56 Subject: Re: Do porcelain command require lock management? Message-ID: URL: https://gitlist.dev/e/xmqqy2jseuqn.fsf%40gitster.c.googlers.com In-Reply-To: <20201024144637.cvwa22f2y4tvfn4z@chatter.i7.local> ``` Konstantin Ryabitsev writes: > A script I'm writing performs a succession of porcelain commands to > create a commit in a bare git repository: Meaning 'plumbing' commands? > > git hash-object > git mktree These take input and create a new object or if the data fed to them were somehow identical to an existing object, becomes a no-op. We take the data, hash to compute its name while writing it out as a loose object to a temporary file. After finishing this compute/write we can check if we already have the object and if so we can just clean up and leave. There is no race in this case. If there is no existing object, we rename the temporary file to the filename derived from the object name. If somebody else were creating exactly the same object at the same time, the renaming to the final name will fail for all but one of them, but in the end everybody gets the object they wanted to see in the repository unless you found a hash collision. These other "failing" folks would probably need to "redo" the operation, but when they do so, their compute/writes would safely become no-op. > git commit-tree This may write more than one objects; compute/write for each object competes with compute/write by other processes independently, but the overall story is the same. > git update-ref You can use the "old value MUST BE this" variant to avoid a race with another process that tries to update the same ref to a different value, but if the other process updates the same ref to a different value _after_ you are done, their update will overwrite what you did, so they also need to be using the same "old value MUST BE this" variant. ```