{"thread":{"id":"54496","subject":"Do porcelain command require lock management?","startedAt":"2020-10-24T14:46:42Z","lastAt":"2020-10-26T17:56:55Z","messageCount":4,"participants":["Konstantin Ryabitsev","brian m. carlson","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"408292","messageId":"20201024144637.cvwa22f2y4tvfn4z@chatter.i7.local","threadId":"54496","inReplyTo":null,"subject":"Do porcelain command require lock management?","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-10-24T14:46:37Z","receivedAt":"2020-10-24T14:46:42Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"Hello:\n\nA script I'm writing performs a succession of porcelain commands to \ncreate a commit in a bare git repository:\n\ngit hash-object\ngit mktree\ngit commit-tree\ngit update-ref\n\nDo I need to manage external locking around these commands to avoid any \nconcurrency problems, or will git take care of that?\n\n-K\n"},{"id":"408380","messageId":"20201025230203.GH860779@camp.crustytoothpaste.net","threadId":"54496","inReplyTo":"20201024144637.cvwa22f2y4tvfn4z@chatter.i7.local","subject":"Re: Do porcelain command require lock management?","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-10-25T23:02:03Z","receivedAt":"2020-10-25T23:02:44Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-10-24 at 14:46:37, Konstantin Ryabitsev wrote:\n> Hello:\n> \n> A script I'm writing performs a succession of porcelain commands to\n> create a commit in a bare git repository:\n> \n> git hash-object\n> git mktree\n> git commit-tree\n> git update-ref\n> \n> Do I need to manage external locking around these commands to avoid any\n> concurrency problems, or will git take care of that?\n\nI'm almost certain that Git will do the same locking and object creation\nsemantics that it does in porcelain commands as in the plumbing commands\nyou're using.  For example, I happen to know that all loose object\ncreation goes through one function, which should gracefully handle\nconcurrent accesses.  Git is in general safe against concurrent accesses\nand is designed not to lose or corrupt data in this case.\n\nHowever, I will point out that ref updates may conflict and if so, Git\nwill fail instead of waiting.  So while your repository will remain\nconsistent and won't experience corruption, that doesn't mean that all\noperations will complete successfully.  Some sort of retry mechanism or\nother error handling will probably be warranted.\n-- \nbrian m. carlson (he/him or they/them)\nHouston, Texas, US\n"},{"id":"408405","messageId":"20201026163643.6cp4jwj5lqkg6vns@chatter.i7.local","threadId":"54496","inReplyTo":"20201025230203.GH860779@camp.crustytoothpaste.net","subject":"Re: Do porcelain command require lock management?","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-10-26T16:36:43Z","receivedAt":"2020-10-26T16:36:48Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Sun, Oct 25, 2020 at 11:02:03PM +0000, brian m. carlson wrote:\n> > A script I'm writing performs a succession of porcelain commands to\n> > create a commit in a bare git repository:\n> > \n> > git hash-object\n> > git mktree\n> > git commit-tree\n> > git update-ref\n> > \n> > Do I need to manage external locking around these commands to avoid any\n> > concurrency problems, or will git take care of that?\n> \n> I'm almost certain that Git will do the same locking and object creation\n> semantics that it does in porcelain commands as in the plumbing commands\n> you're using.  For example, I happen to know that all loose object\n> creation goes through one function, which should gracefully handle\n> concurrent accesses.  Git is in general safe against concurrent accesses\n> and is designed not to lose or corrupt data in this case.\n> \n> However, I will point out that ref updates may conflict and if so, Git\n> will fail instead of waiting.  So while your repository will remain\n> consistent and won't experience corruption, that doesn't mean that all\n> operations will complete successfully.  Some sort of retry mechanism or\n> other error handling will probably be warranted.\n\nThanks for your advice. I think in this case it's easier to just use \nsimple locking to make sure that concurrent processes don't step on \neach-other's toes.\n\nBest regards,\n-K\n"},{"id":"408428","messageId":"xmqqy2jseuqn.fsf@gitster.c.googlers.com","threadId":"54496","inReplyTo":"20201024144637.cvwa22f2y4tvfn4z@chatter.i7.local","subject":"Re: Do porcelain command require lock management?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-10-26T17:56:48Z","receivedAt":"2020-10-26T17:56:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:\n\n> A script I'm writing performs a succession of porcelain commands to \n> create a commit in a bare git repository:\n\nMeaning 'plumbing' commands?\n\n>\n> git hash-object\n> git mktree\n\nThese take input and create a new object or if the data fed to them\nwere somehow identical to an existing object, becomes a no-op.\n\nWe take the data, hash to compute its name while writing it out as a\nloose object to a temporary file.  After finishing this\ncompute/write we can check if we already have the object and if so\nwe can just clean up and leave.  There is no race in this case.  If\nthere is no existing object, we rename the temporary file to the\nfilename derived from the object name.  If somebody else were\ncreating exactly the same object at the same time, the renaming to\nthe final name will fail for all but one of them, but in the end\neverybody gets the object they wanted to see in the repository\nunless you found a hash collision.  These other \"failing\" folks\nwould probably need to \"redo\" the operation, but when they do so,\ntheir compute/writes would safely become no-op.\n\n> git commit-tree\n\nThis may write more than one objects; compute/write for each object\ncompetes with compute/write by other processes independently, but\nthe overall story is the same.\n\n> git update-ref\n\nYou can use the \"old value MUST BE this\" variant to avoid a race\nwith another process that tries to update the same ref to a\ndifferent value, but if the other process updates the same ref to a\ndifferent value _after_ you are done, their update will overwrite\nwhat you did, so they also need to be using the same \"old value MUST\nBE this\" variant.\n"}]}