{"thread":{"id":"15630","subject":"Re: Locking binary files","startedAt":"2008-09-23T06:39:41Z","lastAt":"2008-09-24T15:00:56Z","messageCount":19,"participants":["Mario Pareja","Andreas Ericsson","Boaz Harrosh","Dmitry Potapov","Alex Riesen","Daniel Barkalow","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"91356","messageId":"94c1db200809222339t7d65081eq7471fef86fb5ec73@mail.gmail.com","threadId":"15630","inReplyTo":"94c1db200809222333q4953a6b9g8ce0c1cd4b8f5eb4@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Mario Pareja","fromEmail":"mpareja.dev@gmail.com","sentAt":"2008-09-23T06:39:41Z","receivedAt":"2008-09-23T06:39:41Z","isPatch":false,"sender":{"key":"mpareja.dev@gmail.com","avatar":null},"body":"Hi,\n\nFor one and a half years, I have been keeping my eyes on the git\ncommunity in hopes of making the switch away from SVN.  One particular\nissue holding me back is the inability to lock binary files.\nThroughout the past year, I have yet to see developments on this\nissue.  I understand that locking files goes against the fundamental\nprinciples of distributed source control, but I think we need to come\nup with some workarounds.  For Linux kernel development this is may\nnot be an issue; however, for application development this is a major\nissue. How else can one developer be sure that time spent editing a\nbinary file will not be wasted because another developer submitted a\nchange?\n\nTo achieve the effects of locking, a \"central\" repository must be\nidentified.  Regardless of the distributed nature of git, most\n_companies_ will have a \"central\" repository for a software project.\nWe should be able to mark a file as requiring a lock from the\ngoverning git repository at a specified address.  Is this made\ndifficult because git tracks file contents not files?\n\nIn any case, I think this is a crucial issue that needs to be\naddressed if git is going to be adopted by companies with binary file\nconflict potential. I don't see how a web development company can take\nadvantage of git to track source code and image file changes.  Any\nadvice would be great!\n\nRegards,\n\nMario\n"},{"id":"91359","messageId":"48D8983C.7070506@op5.se","threadId":"15630","inReplyTo":"94c1db200809222339t7d65081eq7471fef86fb5ec73@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-09-23T07:18:20Z","receivedAt":"2008-09-23T07:18:20Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Mario Pareja wrote:\n> Hi,\n> \n> For one and a half years, I have been keeping my eyes on the git\n> community in hopes of making the switch away from SVN.  One particular\n> issue holding me back is the inability to lock binary files.\n> Throughout the past year, I have yet to see developments on this\n> issue.  I understand that locking files goes against the fundamental\n> principles of distributed source control, but I think we need to come\n> up with some workarounds.  For Linux kernel development this is may\n> not be an issue; however, for application development this is a major\n> issue. How else can one developer be sure that time spent editing a\n> binary file will not be wasted because another developer submitted a\n> change?\n> \n\nBecause they will cause merge conflicts when you try to bring the\nhistories together. Some binary formats can be edited by multiple\nusers at the same time, while others can't, so git will try to merge\nthose binary files for you. For images, that almost certainly won't\ngo so well so it will result in a conflict.\n\n> To achieve the effects of locking, a \"central\" repository must be\n> identified.\n\nTo achieve distributedness no central repository must exist. Locking\ncan be done by some other means.\n\n>  Regardless of the distributed nature of git, most\n> _companies_ will have a \"central\" repository for a software project.\n\nActually, all projects with some sort of userbase will probably have\nsome official \"here's the published code suitable for production use\"\nrepository. To say that it's the \"central\" one is a bit off though.\nIt's merely a public place that can be referred to for convenience.\n\n> We should be able to mark a file as requiring a lock from the\n> governing git repository at a specified address.  Is this made\n> difficult because git tracks file contents not files?\n> \n> In any case, I think this is a crucial issue that needs to be\n> addressed if git is going to be adopted by companies with binary file\n> conflict potential. I don't see how a web development company can take\n> advantage of git to track source code and image file changes.  Any\n> advice would be great!\n> \n\nTry and find out.\n\nmkdir foo && cd foo && git init\ncp /random/binary/file.png image.png\ngit add image.png && git commit -m\"first commit\"\ngit checkout -b A\ncp /other/random/binary/file.png image.png\ngit add image.png && git commit -m\"conflicting commit\"\ngit checkout -b B master\ncp /third/random/binary/file.png image.png\ngit add image.png && git commit -m\"non-conflicting commit\"\ngit checkout master\ncp /third/random/binary/file.png image.png\ngit add image.png && git commit -m\"master says 'so be it'\"\n\ngit merge B; # works, since the binary files are the same\ngit merge A; # produces a conflict message\n\n\nIn which way is that not exactly the right behaviour?\nHow would locking have helped?\n\nIf your colleagues are replacing files you committed so\nthat your code suddenly fails, you have a communication\n(and QA) issue at work. Adding locking to git is not the\nsolution to that problem. Introducing a sort of builtin\nnotion of a central repository is, frankly, disgusting.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n"},{"id":"91368","messageId":"48D8A97E.8070003@op5.se","threadId":"15630","inReplyTo":"94c1db200809230054t20e7e61dh5022966d4112eee6@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-09-23T08:31:58Z","receivedAt":"2008-09-23T08:31:58Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Mario, please don't reply in private. That way your mails won't\nget indexed and you don't have a chance to get help from others\non the mailing list.\n\nWhile we're at it; don't top-post. Most people who frequent email\nlists with moderate to high traffic read hundreds of emails every\nday, so a quick reminder of what the discussion was about is useful\nwhen getting a reply. That reminder gets a lot trickier to get to\nif you first have to scroll down and then back up. Besides that,\nit feels totally backwards.\n\nMario Pareja wrote:\n> Andreas,\n> \n> Thanks for the quick reply.  You asked how I thought locking could\n> have helped. I think locking helps notify a developer that a file is\n> being modified _before_ the developer begins his/her own\n> modifications. If I followed your example correctly, the conflict is\n> identified after the work has been done - this is too late if you ask\n> me.\n> \n\nSo it's a communication issue then. The way I understand locks in svn\nand cvs is that they also only bother you when you want to check in the\nfile you've just recently modified, or if multiple people want to lock\nthe same file at the same time.\n\nIf that's the case, I see no problem what so ever with teaching specific\ngit commands to interact with a locking server. git lock (and git unlock)\nwould have to be coupled with a git-lock-daemon with wich everyone\ncommunicates. It should probably have the ability to run a hook or\nsomething (centrally) when a lock is obtained and released, so as to be\nable to notify others that a lock is held.\n\nI might write this for fun some day, but it's really not my itch to\nscratch, and it would be a terrible mistake to add something like a\ncentral repository to take care of it when a single rather stupid\ndaemon and an equally stupid program could do the same work but much\nmore efficiently.\n\nNote that locking would be completely advisory though, and nothing\nwould prevent people from committing changes to a locked file. Then\nagain, insofar as I understand SVN/CVS locking, that's how those\nwork too, except that an SVN \"checkin\" would be the equivalent of\n\"git commit && git push\" (the push part of the git sequence won't\nwork).\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n"},{"id":"91377","messageId":"48D8CFF1.8030403@panasas.com","threadId":"15630","inReplyTo":"94c1db200809222339t7d65081eq7471fef86fb5ec73@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Boaz Harrosh","fromEmail":"bharrosh@panasas.com","sentAt":"2008-09-23T11:16:01Z","receivedAt":"2008-09-23T11:16:01Z","isPatch":false,"sender":{"key":"bharrosh@panasas.com","avatar":"https://gravatar.com/avatar/347e426f8ca0409f6891ceeefd4c3c2d8769608382323057efeaa362b4c3dbf3?d=mp&s=160"},"body":"Mario Pareja wrote:\n> Hi,\n> \n> For one and a half years, I have been keeping my eyes on the git\n> community in hopes of making the switch away from SVN.  One particular\n> issue holding me back is the inability to lock binary files.\n> Throughout the past year, I have yet to see developments on this\n> issue.  I understand that locking files goes against the fundamental\n> principles of distributed source control, but I think we need to come\n> up with some workarounds.  For Linux kernel development this is may\n> not be an issue; however, for application development this is a major\n> issue. How else can one developer be sure that time spent editing a\n> binary file will not be wasted because another developer submitted a\n> change?\n> \n> To achieve the effects of locking, a \"central\" repository must be\n> identified.  Regardless of the distributed nature of git, most\n> _companies_ will have a \"central\" repository for a software project.\n> We should be able to mark a file as requiring a lock from the\n> governing git repository at a specified address.  Is this made\n> difficult because git tracks file contents not files?\n> \n> In any case, I think this is a crucial issue that needs to be\n> addressed if git is going to be adopted by companies with binary file\n> conflict potential. I don't see how a web development company can take\n> advantage of git to track source code and image file changes.  Any\n> advice would be great!\n> \n> Regards,\n> \n> Mario\n> --\n\nIt should be easy for a company to set a policy where a couple of scripts\nmust be run for particular type of files. Given that, the implementation\nof such scripts is easy:\n\nFor every foo.bin there is possibly a foo.bin.lock file.\n\nLock-script look for absence of the lock-file at upstream then git-add\nthe file (With some info that tells users things like who has the file).\nIf git-push fails, since I'm adding a file and someone already added\nit while I was pushing, then the lock is not granted.\n\nUnlock-script will git-rm the lock-file and push.\n\nIn both scripts mod-bits of original file can be toggled for\nread-only/write signaling to the user. (At upstream the file is always\nread-only)\n\nThis can also work in a distributed system with more then one tier of\nservers. (Locks pushed to the most upstream server)\n\nCombine that with git's mail notifications for commits and you have a\nsystem far more robust then svn will ever want to be\n\nMy $0.017\nBoaz\n"},{"id":"91379","messageId":"48D8D0F0.6050909@panasas.com","threadId":"15630","inReplyTo":"48D8CFF1.8030403@panasas.com","subject":"Re: Locking binary files","fromName":"Boaz Harrosh","fromEmail":"bharrosh@panasas.com","sentAt":"2008-09-23T11:20:16Z","receivedAt":"2008-09-23T11:20:16Z","isPatch":false,"sender":{"key":"bharrosh@panasas.com","avatar":"https://gravatar.com/avatar/347e426f8ca0409f6891ceeefd4c3c2d8769608382323057efeaa362b4c3dbf3?d=mp&s=160"},"body":"Boaz Harrosh wrote:\n> Mario Pareja wrote:\n>> Hi,\n>>\n>> For one and a half years, I have been keeping my eyes on the git\n>> community in hopes of making the switch away from SVN.  One particular\n>> issue holding me back is the inability to lock binary files.\n>> Throughout the past year, I have yet to see developments on this\n>> issue.  I understand that locking files goes against the fundamental\n>> principles of distributed source control, but I think we need to come\n>> up with some workarounds.  For Linux kernel development this is may\n>> not be an issue; however, for application development this is a major\n>> issue. How else can one developer be sure that time spent editing a\n>> binary file will not be wasted because another developer submitted a\n>> change?\n>>\n>> To achieve the effects of locking, a \"central\" repository must be\n>> identified.  Regardless of the distributed nature of git, most\n>> _companies_ will have a \"central\" repository for a software project.\n>> We should be able to mark a file as requiring a lock from the\n>> governing git repository at a specified address.  Is this made\n>> difficult because git tracks file contents not files?\n>>\n>> In any case, I think this is a crucial issue that needs to be\n>> addressed if git is going to be adopted by companies with binary file\n>> conflict potential. I don't see how a web development company can take\n>> advantage of git to track source code and image file changes.  Any\n>> advice would be great!\n>>\n>> Regards,\n>>\n>> Mario\n>> --\n> \n> It should be easy for a company to set a policy where a couple of scripts\n> must be run for particular type of files. Given that, the implementation\n> of such scripts is easy:\n> \n> For every foo.bin there is possibly a foo.bin.lock file.\n> \n> Lock-script look for absence of the lock-file at upstream then git-add\n> the file (With some info that tells users things like who has the file).\n> If git-push fails, since I'm adding a file and someone already added\n> it while I was pushing, then the lock is not granted.\n> \n> Unlock-script will git-rm the lock-file and push.\n> \n> In both scripts mod-bits of original file can be toggled for\n> read-only/write signaling to the user. (At upstream the file is always\n> read-only)\n> \n> This can also work in a distributed system with more then one tier of\n> servers. (Locks pushed to the most upstream server)\n> \n> Combine that with git's mail notifications for commits and you have a\n> system far more robust then svn will ever want to be\n> \n> My $0.017\n> Boaz\n> \n\nOK combine that with a technic presented in this ml thread:\n  \"Management of opendocument (openoffice.org) files in git\" \nAnd make all this automatic for particular type of files\n\nBoaz\n"},{"id":"91391","messageId":"20080923134446.GM21650@dpotapov.dyndns.org","threadId":"15630","inReplyTo":"94c1db200809222339t7d65081eq7471fef86fb5ec73@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-09-23T13:44:46Z","receivedAt":"2008-09-23T13:44:46Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Tue, Sep 23, 2008 at 02:39:41AM -0400, Mario Pareja wrote:\n> \n> How else can one developer be sure that time spent editing a\n> binary file will not be wasted because another developer submitted a\n> change?\n\nThat sounds to me more like a communication problem than anything\nrelated to Git itself.\n\n> \n> To achieve the effects of locking, a \"central\" repository must be\n> identified.  Regardless of the distributed nature of git, most\n> _companies_ will have a \"central\" repository for a software project.\n> We should be able to mark a file as requiring a lock from the\n> governing git repository at a specified address.  Is this made\n> difficult because git tracks file contents not files?\n\nThe problem exists regardless the distributed nature of git. Let's\nconsider a single repository with only two branches: A and B. Now, one\ndeveloper has decided to edit some binary file called pretty.img on A.\nShould this file be locked only on the branch A or on both branches? The\nanswer is if A is going to merge to B then this file on B too and remain\nlocking till A is merged to B. In fact, it may be *absolutely* pointless\nto lock the file on the developer's topic branch, because another\ndeveloper can edit it on another topic branch without noticing that this\nlock exists at all. So, it may be enough to lock it only B enough, but\nthis is impossible to Git to know, because Git does not understand\n_your_ particular workflow, and without any locking scheme is rather\nmeaningless.\n\nPerhaps, a more general solution can be based exactly on the content,\nnot on the name, i.e. in some share directory on the server I create\na file with name based on SHA-1 of the binary file where I put comment\nexplaining why I locked it. Obviously, this lock is purely advisory,\nbut it is good, in some situation you really may want to edit two\nfiles with the same SHA-1 on different branches that never get merge.\nMoreover, this lock is never deleted. So, it could make sense instead\nof having a separate file per lock to organize it in some more compact\nstorage, which may look like history of editing binary files... But it\nis just an idea how I would do that.\n\nDmitry\n"},{"id":"91395","messageId":"94c1db200809230656q4a9a765dw2354c0058b1d940c@mail.gmail.com","threadId":"15630","inReplyTo":"48D8A97E.8070003@op5.se","subject":"Re: Locking binary files","fromName":"Mario Pareja","fromEmail":"mpareja.dev@gmail.com","sentAt":"2008-09-23T13:56:57Z","receivedAt":"2008-09-23T13:56:57Z","isPatch":false,"sender":{"key":"mpareja.dev@gmail.com","avatar":null},"body":"> So it's a communication issue then.\n\nYes, but I think the communication of this information needs to happen\nas part of a developers normal work-flow rather than requiring them to\nremember to check an external system.\n\n> The way I understand locks in svn\n> and cvs is that they also only bother you when you want to check in the\n> file you've just recently modified, or if multiple people want to lock\n> the same file at the same time.\n\nThe SVN client will make locked files read-only until a lock is\nobtained for them.  This helps \"remind\" you that a lock should be\nobtained before editing such a file. Requiring the developer to obtain\na lock ensures that nobody else is editing the file and prevents\nwasted work.  Upon commit, the file is marked as unlocked and the\nlocal file is once again read-only.\n\n>\n> Note that locking would be completely advisory though, and nothing\n> would prevent people from committing changes to a locked file.\n\nIf git were to support locking then it could prevent people from\ncommitting without first locking.  Even if it is not supported\ndirectly by git - perhaps using a lock daemon - a wrapper would need\nto be written around git commit/push to prevent developers from\ncommitting/pushing changes that would cause binary merging conflicts.\n\n> Then\n> again, insofar as I understand SVN/CVS locking, that's how those\n> work too, except that an SVN \"checkin\" would be the equivalent of\n> \"git commit && git push\" (the push part of the git sequence won't\n> work).\n>\n\nGenerally in SVN you need to lock the file before being able to commit.\n\nReally, I am just curious about how others deal with this issue.  Do\nyou simply start editing binary files and hope nobody else edits the\nsame file?  Do you send out an email telling people you are working on\nsuch a file?\n\nMario\n"},{"id":"91398","messageId":"94c1db200809230714k6b007919yfd8ad1b86cbcd385@mail.gmail.com","threadId":"15630","inReplyTo":"48D8CFF1.8030403@panasas.com","subject":"Re: Locking binary files","fromName":"Mario Pareja","fromEmail":"mpareja.dev@gmail.com","sentAt":"2008-09-23T14:14:07Z","receivedAt":"2008-09-23T14:14:07Z","isPatch":false,"sender":{"key":"mpareja.dev@gmail.com","avatar":null},"body":"> It should be easy for a company to set a policy where a couple of scripts\n> must be run for particular type of files. Given that, the implementation\n> of such scripts is easy:\n>\n> For every foo.bin there is possibly a foo.bin.lock file.\n>\n> Lock-script look for absence of the lock-file at upstream then git-add\n> the file (With some info that tells users things like who has the file).\n> If git-push fails, since I'm adding a file and someone already added\n> it while I was pushing, then the lock is not granted.\n>\n> Unlock-script will git-rm the lock-file and push.\n>\n> In both scripts mod-bits of original file can be toggled for\n> read-only/write signaling to the user. (At upstream the file is always\n> read-only)\n>\n> This can also work in a distributed system with more then one tier of\n> servers. (Locks pushed to the most upstream server)\n>\n> Combine that with git's mail notifications for commits and you have a\n> system far more robust then svn will ever want to be\n>\n> My $0.017\n> Boaz\n>\n\nThis is a reasonable approach to obtaining the desired functionality.\nUnfortunately, I have not seen any third-party packages implementing\nsuch a feature.  It seems to me the problem is general enough to be\nsolved once rather than requiring organizations wishing to use git to\nimplement an in-house locking system. It simply creates more friction.\nPerhaps, when I have the time, I will come up with something others\ncan use.  For now, unfortunately, it seems I am out of luck?\n\nMario\n"},{"id":"91399","messageId":"81b0412b0809230728p4e81572bg67a3bec4c32c7dfb@mail.gmail.com","threadId":"15630","inReplyTo":"94c1db200809230656q4a9a765dw2354c0058b1d940c@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2008-09-23T14:28:00Z","receivedAt":"2008-09-23T14:28:00Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"2008/9/23 Mario Pareja <mpareja.dev@gmail.com>:\n>> So it's a communication issue then.\n>\n> Yes, but I think the communication of this information needs to happen\n> as part of a developers normal work-flow rather than requiring them to\n> remember to check an external system.\n\nLook at pre-receive and update hooks. They can deny a push operation and\nget enough information to notice a change to the path of your unlucky file.\n\nAnd yes, *you* have to do that yourself.\n"},{"id":"91400","messageId":"48D8FEC5.3060503@panasas.com","threadId":"15630","inReplyTo":"94c1db200809230714k6b007919yfd8ad1b86cbcd385@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Boaz Harrosh","fromEmail":"bharrosh@panasas.com","sentAt":"2008-09-23T14:35:49Z","receivedAt":"2008-09-23T14:35:49Z","isPatch":false,"sender":{"key":"bharrosh@panasas.com","avatar":"https://gravatar.com/avatar/347e426f8ca0409f6891ceeefd4c3c2d8769608382323057efeaa362b4c3dbf3?d=mp&s=160"},"body":"Mario Pareja wrote:\n>> It should be easy for a company to set a policy where a couple of scripts\n>> must be run for particular type of files. Given that, the implementation\n>> of such scripts is easy:\n>>\n>> For every foo.bin there is possibly a foo.bin.lock file.\n>>\n>> Lock-script look for absence of the lock-file at upstream then git-add\n>> the file (With some info that tells users things like who has the file).\n>> If git-push fails, since I'm adding a file and someone already added\n>> it while I was pushing, then the lock is not granted.\n>>\n>> Unlock-script will git-rm the lock-file and push.\n>>\n>> In both scripts mod-bits of original file can be toggled for\n>> read-only/write signaling to the user. (At upstream the file is always\n>> read-only)\n>>\n>> This can also work in a distributed system with more then one tier of\n>> servers. (Locks pushed to the most upstream server)\n>>\n>> Combine that with git's mail notifications for commits and you have a\n>> system far more robust then svn will ever want to be\n>>\n>> My $0.017\n>> Boaz\n>>\n> \n> This is a reasonable approach to obtaining the desired functionality.\n> Unfortunately, I have not seen any third-party packages implementing\n> such a feature.  It seems to me the problem is general enough to be\n> solved once rather than requiring organizations wishing to use git to\n> implement an in-house locking system. It simply creates more friction.\n> Perhaps, when I have the time, I will come up with something others\n> can use.  For now, unfortunately, it seems I am out of luck?\n> \n> Mario\n> --\n\nThe open-source my friend. First comes first implements. More and more\ndevelopment platforms use XML files in where they used a binary file\nformat before. Just for these cases. Git is mostly used with open-source\nand/or very new systems that don't have binary file formats. OK graphics is\nanother thing, I guess.\n\nSo you are welcome to it. \"git-lock\" is available\n\nBoaz\n"},{"id":"91409","messageId":"alpine.LNX.1.00.0809231216350.19665@iabervon.org","threadId":"15630","inReplyTo":"94c1db200809230656q4a9a765dw2354c0058b1d940c@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2008-09-23T17:32:15Z","receivedAt":"2008-09-23T17:32:15Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 23 Sep 2008, Mario Pareja wrote:\n\n> > So it's a communication issue then.\n> \n> Yes, but I think the communication of this information needs to happen\n> as part of a developers normal work-flow rather than requiring them to\n> remember to check an external system.\n> \n> > The way I understand locks in svn\n> > and cvs is that they also only bother you when you want to check in the\n> > file you've just recently modified, or if multiple people want to lock\n> > the same file at the same time.\n> \n> The SVN client will make locked files read-only until a lock is\n> obtained for them.  This helps \"remind\" you that a lock should be\n> obtained before editing such a file. Requiring the developer to obtain\n> a lock ensures that nobody else is editing the file and prevents\n> wasted work.  Upon commit, the file is marked as unlocked and the\n> local file is once again read-only.\n\nI think the right tool on the git side is actually a \"smudge/clean\" \nscript. When you check something out, git converts it from the \nrepository-stored form to a working tree form using a script (if there is \none configured); this could check whether you've got the appropriate lock, \nand make the file unwritable if you don't. Then you have a script that \ngets or releases a lock and sets any write bits on files already checked \nout appropriately. There could also be locking-server magic to detect that \nyou've pushed a change and release the lock, telling you so that it makes \nyour file unwritable, but that's optional.\n\n(Side note: consider version-specific logos; which lock you need depends \non which version you're working on, and you may want to pick up locks for \nmultiple versions and make changes to each logo, switching between the \nbranches, and make sure you can get all the locks before you start \nworking on any of the files, despite not having any individual file \nchecked out continuously in the process)\n\n> > Note that locking would be completely advisory though, and nothing\n> > would prevent people from committing changes to a locked file.\n> \n> If git were to support locking then it could prevent people from\n> committing without first locking.  Even if it is not supported\n> directly by git - perhaps using a lock daemon - a wrapper would need\n> to be written around git commit/push to prevent developers from\n> committing/pushing changes that would cause binary merging conflicts.\n\nIf you've gotten to the point of committing (let alone pushing), and you \nhaven't got exclusive access, git should certainly not prevent you; the \npoint of the locking is to prevent people from doing work that will be \nwasted, and the work is already done at this point. It's better then to \nactually try the binary merge, which comes down to apologizing profusely \nand then somebody openning the 3 versions (theirs, the other side's, and \nthe common ancestor) in their graphics program and modifying the other \nside's to include their change. It wouldn't help anything to prevent \npeople from being able to get all of these versions to each other, once \nthey're made. It's also helpful to have people commit what they did before \nredoing it, so that they can use it for reference in the process and won't \nlose it.\n\n(Actually, I bet it would be not-too-hard to set up gimp for three-way \nmerge of images; open the result file with \"theirs\" as the contents, and \nopen the common ancestor and \"yours\" as extra layers and set the ancestor \nto negative, and make the user clean up the mess)\n\nOn the other hand, the locking server should reject your push if somebody \nelse has got the lock, so that the person who editted the file without \nhaving the lock is the one stuck redoing things.\n\nIn any case, the fundamental idea is: (a) you want some server to favor \npeople who declare their intent to change something in advance, and give \nall the work of redoing stuff to people who didn't declare their intent in \nadvance; and (b) you want to prompt people to declare their intent in case \nthey forget.\n\n(a) is a pre-update hook that checks the diffstat against other people's \nlocks. (b) is a smudge script that makes files you're supposed to lock and \nhaven't a-w. Of course, git doesn't have the code for manipulating a \nper-user set of locks, but it shouldn't be too hard to find some project \nthat just does that.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"91417","messageId":"7v7i92tzgb.fsf@gitster.siamese.dyndns.org","threadId":"15630","inReplyTo":"alpine.LNX.1.00.0809231216350.19665@iabervon.org","subject":"Re: Locking binary files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-09-23T19:49:24Z","receivedAt":"2008-09-23T19:49:24Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Daniel Barkalow <barkalow@iabervon.org> writes:\n\n> I think the right tool on the git side is actually a \"smudge/clean\" \n> script. When you check something out, git converts it from the \n> repository-stored form to a working tree form using a script (if there is \n> one configured); this could check whether you've got the appropriate lock, \n> and make the file unwritable if you don't.\n\nAn obvious question is \"how would such a script check the lock when you\nare 30,000 ft above ground\"; in other words, this \"locking mechanism\"\ncontradicts the very nature of distributed development theme.  The best\nmechanism should always be on the human side.  An SCM auguments\ninter-developer communication, but it is not a _substitute_ for\ncommunication.\n\nBut if you limit the use case to an always tightly connected environment\n(aka \"not distributed at all\"), I agree the above would be a very\nreasonable approach.\n\nSuch a setup would need a separate locking infrastructure and an end user\ncommand that grabs the lock and when successful makes the file in the work\ntree read/write.  The user butchers the contents after taking the lock,\nsaves, and then when running \"git commit\", probably the post-commit hook\nwould release any relevant locks.\n\nAll these can be left outside the scope of git, as they can be hooked into\ngit with the existing infrastructure. Once a BCP materializes it could be\nadded to contrib/ just like the \"paranoid\" update hook.\n"},{"id":"91422","messageId":"20080923204616.GS21650@dpotapov.dyndns.org","threadId":"15630","inReplyTo":"94c1db200809230656q4a9a765dw2354c0058b1d940c@mail.gmail.com","subject":"Re: Locking binary files","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-09-23T20:46:16Z","receivedAt":"2008-09-23T20:46:16Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Tue, Sep 23, 2008 at 09:56:57AM -0400, Mario Pareja wrote:\n> \n> The SVN client will make locked files read-only until a lock is\n> obtained for them.  This helps \"remind\" you that a lock should be\n> obtained before editing such a file. Requiring the developer to obtain\n> a lock ensures that nobody else is editing the file and prevents\n> wasted work.  Upon commit, the file is marked as unlocked and the\n> local file is once again read-only.\n\nThe approach that SVN takes is not only impossible for distributed\nenvironment, it does not work even in a _single_ repository where you\nhave branching and merging. If you have a topic branch then your lock\nwill have a zero effect on other developers or lock of other developers\non you. Obviously, you are going to have the binary merge conflict at\nthe end. But it is even worse than that. Somebody locked a file on the\nmaster branch and you clone from it. Now, this somebody unlocked this\nfile, but this file on your branch remains locked but this person, and\nthis person may even not aware that about your branch. That is insane!\n\nDmitry\n"},{"id":"91428","messageId":"alpine.LNX.1.00.0809231551320.19665@iabervon.org","threadId":"15630","inReplyTo":"7v7i92tzgb.fsf@gitster.siamese.dyndns.org","subject":"Re: Locking binary files","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2008-09-23T21:13:29Z","receivedAt":"2008-09-23T21:13:29Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 23 Sep 2008, Junio C Hamano wrote:\n\n> Daniel Barkalow <barkalow@iabervon.org> writes:\n> \n> > I think the right tool on the git side is actually a \"smudge/clean\" \n> > script. When you check something out, git converts it from the \n> > repository-stored form to a working tree form using a script (if there is \n> > one configured); this could check whether you've got the appropriate lock, \n> > and make the file unwritable if you don't.\n> \n> An obvious question is \"how would such a script check the lock when you\n> are 30,000 ft above ground\"; in other words, this \"locking mechanism\"\n> contradicts the very nature of distributed development theme.  The best\n> mechanism should always be on the human side.  An SCM auguments\n> inter-developer communication, but it is not a _substitute_ for\n> communication.\n\nIf you're offline, you can't get new locks, nor release them. But it can \nmake reasonable decisions if it remembers what locks you got before.\n\nOn the other hand, you can just make the file writable yourself while \ndisconnected, and nothing bad happens to anybody else; if someone else \nlocks the file and starts working, they'll block your eventual push until \nthey push and you merge. And nothing too bad happens to you; you get stuck \nredoing the change later (as a merge), but (a) you would have had to do \nthe work then anyway; (b) you knew you weren't protecting yourself; and \n(c) at least you got to practice on the plane.\n\nThe point of the locking is just that, if you get the lock for a \nparticular file in a particular branch on a particular shared repository, \nyou can be sure you won't have to merge that file in order to push there, \nand you can get this worked out in advance of having the push ready. A \nsecondary concern is that you might want to stop yourself from working on \ncertain things without this kind of reservation, but that's a local \ndecision.\n\n> But if you limit the use case to an always tightly connected environment\n> (aka \"not distributed at all\"), I agree the above would be a very\n> reasonable approach.\n> \n> Such a setup would need a separate locking infrastructure and an end user\n> command that grabs the lock and when successful makes the file in the work\n> tree read/write.  The user butchers the contents after taking the lock,\n> saves, and then when running \"git commit\", probably the post-commit hook\n> would release any relevant locks.\n\nThe lock needs to last until you push to the repository the lock is for; \notherwise you have the exclusive ability to make changes, but someone who \ngrabs the lock right after you release it will still be working on the \nversion without your change, which is what the lock is supposed to \nprevent.\n\n> All these can be left outside the scope of git, as they can be hooked into\n> git with the existing infrastructure. Once a BCP materializes it could be\n> added to contrib/ just like the \"paranoid\" update hook.\n\nIt would be handy to link against some of git, since it will want to use \ngit config files and remotes and refspecs to figure out what lock to ask \nfor on the client side, and how to communicate with the target remote \nrepository, and the process of getting a lock requires checking that \nyou're up-to-date, and git's also got a bunch of useful code for atomic \nfile updates and repository-scoped filename management. But adding this \ndoesn't have to modify any existing behavior.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"91436","messageId":"20080923215422.GV21650@dpotapov.dyndns.org","threadId":"15630","inReplyTo":"alpine.LNX.1.00.0809231551320.19665@iabervon.org","subject":"Re: Locking binary files","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-09-23T21:54:22Z","receivedAt":"2008-09-23T21:54:22Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Tue, Sep 23, 2008 at 05:13:29PM -0400, Daniel Barkalow wrote:\n> \n> The lock needs to last until you push to the repository the lock is for; \n> otherwise you have the exclusive ability to make changes, but someone who \n> grabs the lock right after you release it will still be working on the \n> version without your change, which is what the lock is supposed to \n> prevent.\n\nIt still will happen if developers work on topic branches, and it is not\na rate situation with Git. Thus locking some particular path is stupid.\nWhat you may want instead is too mark SHA-1 of this file as being edited\nand later maybe as being replaced with another one. In this case, anyone\nwho has the access to the central information storage will get warning\nabout attempt to edit a file that is edited or already replaced with a\nnew version.\n\nDmitry\n"},{"id":"91438","messageId":"alpine.LNX.1.00.0809231811560.19665@iabervon.org","threadId":"15630","inReplyTo":"20080923215422.GV21650@dpotapov.dyndns.org","subject":"Re: Locking binary files","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2008-09-23T22:29:53Z","receivedAt":"2008-09-23T22:29:53Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Wed, 24 Sep 2008, Dmitry Potapov wrote:\n\n> On Tue, Sep 23, 2008 at 05:13:29PM -0400, Daniel Barkalow wrote:\n> > \n> > The lock needs to last until you push to the repository the lock is for; \n> > otherwise you have the exclusive ability to make changes, but someone who \n> > grabs the lock right after you release it will still be working on the \n> > version without your change, which is what the lock is supposed to \n> > prevent.\n> \n> It still will happen if developers work on topic branches, and it is not\n> a rate situation with Git. Thus locking some particular path is stupid.\n> What you may want instead is too mark SHA-1 of this file as being edited\n> and later maybe as being replaced with another one. In this case, anyone\n> who has the access to the central information storage will get warning\n> about attempt to edit a file that is edited or already replaced with a\n> new version.\n\nNo, your goal is to avoid having to do a merge in order to do a particular \npush. That push is the push to the shared location. It doesn't matter if \nyou use topic branches, because your eventual goal is still to push to the \nshared location (or, possibly, to have the project maintainer push to the \nshared location with some sort of interesting delegation), so you lock the \nshared location, not your topic branch.\n\nOn the other hand, it's easily possible that other people (or you) want to \nfork the image, such that only some locations (either different paths in \nthe project or the same path in different branches) get your change and \nother branches get different changes made at the same time. Of course, if \nyou want to change multiple things, you need to get multiple locks.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"91441","messageId":"20080923232154.GW21650@dpotapov.dyndns.org","threadId":"15630","inReplyTo":"alpine.LNX.1.00.0809231811560.19665@iabervon.org","subject":"Re: Locking binary files","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-09-23T23:21:54Z","receivedAt":"2008-09-23T23:21:54Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Tue, Sep 23, 2008 at 06:29:53PM -0400, Daniel Barkalow wrote:\n> On Wed, 24 Sep 2008, Dmitry Potapov wrote:\n> \n> > It still will happen if developers work on topic branches, and it is not\n> > a rate situation with Git. Thus locking some particular path is stupid.\n> > What you may want instead is too mark SHA-1 of this file as being edited\n> > and later maybe as being replaced with another one. In this case, anyone\n> > who has the access to the central information storage will get warning\n> > about attempt to edit a file that is edited or already replaced with a\n> > new version.\n> \n> No, your goal is to avoid having to do a merge in order to do a particular \n> push. That push is the push to the shared location. It doesn't matter if \n> you use topic branches, because your eventual goal is still to push to the \n> shared location (or, possibly, to have the project maintainer push to the \n> shared location with some sort of interesting delegation), so you lock the \n> shared location, not your topic branch.\n\nWhat are you saying is that when I am locking some file on the current\nbranch, Git (or whatever script that performs this locking) should figure\nout what is the original shared branch for it and lock the file there.\nWhen you have finished to edit and push changes then the lock should be\nremoved if changes are pushed to this shared branch, otherwise it should\nbe some token of delegation to the project maintainer who is going to\npush (or probably first merge, because other files may need that) to\nthis branch.\n\nMaybe, it can work, but it sounds too complex to me. I believe that my\nidea using SHA-1 is better. After all, what is file? It is its content.\nAt least, in Git, we always identify files by their content. Thus if you\nlock some file, you put a lock on certain SHA-1. Now, regardless of\nbranches and paths, this lock can work provided that you have access to\nsome shared location. Of course, this lock is purely advisory, but it is\ngood, because you may want to ignore it in some case. For instance, you\nwant to created a new branch based on the current shared location and\nhave no plan to ever merge it back. In this case, the lock on the shared\nbranch should not matter to you. This is true regardless how you\nimplement locking, and in your scheme it will another special case.\n\n\nDmitry\n"},{"id":"91446","messageId":"alpine.LNX.1.00.0809232330050.19665@iabervon.org","threadId":"15630","inReplyTo":"20080923232154.GW21650@dpotapov.dyndns.org","subject":"Re: Locking binary files","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2008-09-24T04:15:39Z","receivedAt":"2008-09-24T04:15:39Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Wed, 24 Sep 2008, Dmitry Potapov wrote:\n\n> On Tue, Sep 23, 2008 at 06:29:53PM -0400, Daniel Barkalow wrote:\n> > On Wed, 24 Sep 2008, Dmitry Potapov wrote:\n> > \n> > > It still will happen if developers work on topic branches, and it is not\n> > > a rate situation with Git. Thus locking some particular path is stupid.\n> > > What you may want instead is too mark SHA-1 of this file as being edited\n> > > and later maybe as being replaced with another one. In this case, anyone\n> > > who has the access to the central information storage will get warning\n> > > about attempt to edit a file that is edited or already replaced with a\n> > > new version.\n> > \n> > No, your goal is to avoid having to do a merge in order to do a particular \n> > push. That push is the push to the shared location. It doesn't matter if \n> > you use topic branches, because your eventual goal is still to push to the \n> > shared location (or, possibly, to have the project maintainer push to the \n> > shared location with some sort of interesting delegation), so you lock the \n> > shared location, not your topic branch.\n> \n> What are you saying is that when I am locking some file on the current\n> branch, Git (or whatever script that performs this locking) should figure\n> out what is the original shared branch for it and lock the file there.\n\nOr you should have to say. But \"git lock <filename>\" should probably \nput the lock on whatever branch \"git push\" would push to, and similarly \nfor the other argument combinations that \"git push\" permits.\n\n> When you have finished to edit and push changes then the lock should be\n> removed if changes are pushed to this shared branch, otherwise it should\n> be some token of delegation to the project maintainer who is going to\n> push (or probably first merge, because other files may need that) to\n> this branch.\n\nCorrect.\n\n> Maybe, it can work, but it sounds too complex to me. I believe that my\n> idea using SHA-1 is better. After all, what is file? It is its content.\n> At least, in Git, we always identify files by their content.\n\nNot at all; there are plenty of cases where what matters is the path, and \nsome things are relevant by virtue of the form of the filename which names \nthat content.\n\n> Thus if you lock some file, you put a lock on certain SHA-1. Now, \n> regardless of branches and paths, this lock can work provided that you \n> have access to some shared location. Of course, this lock is purely \n> advisory, but it is good, because you may want to ignore it in some \n> case.\n\nIn my design, the lock (on the shared repository) is not advisory; if \nsomeone else has it, you can't push if the new commit doesn't match the \nold commit for that path. (Of course, the system might let you break the \nother person's lock.) I don't think locks are particularly useful if you \ndon't get some particular guarantee out ofhaving them (in my case, that \nsomebody else will have to do any merge for the file if one is needed).\n\n> For instance, you want to created a new branch based on the \n> current shared location and have no plan to ever merge it back. In this \n> case, the lock on the shared branch should not matter to you. This is \n> true regardless how you implement locking, and in your scheme it will \n> another special case.\n\nIf you have no intention to merge a local branch back to the remote branch \nit is based on, then you won't have the remote configured for this. If the \nlocking and lock-checking code uses the push configuration to determine \nwhat locks make sense, it'll automatically be unrelated.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"91472","messageId":"20080924150056.GY21650@dpotapov.dyndns.org","threadId":"15630","inReplyTo":"alpine.LNX.1.00.0809232330050.19665@iabervon.org","subject":"Re: Locking binary files","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-09-24T15:00:56Z","receivedAt":"2008-09-24T15:00:56Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Sep 24, 2008 at 12:15:39AM -0400, Daniel Barkalow wrote:\n> On Wed, 24 Sep 2008, Dmitry Potapov wrote:\n> \n> > \n> > What are you saying is that when I am locking some file on the current\n> > branch, Git (or whatever script that performs this locking) should figure\n> > out what is the original shared branch for it and lock the file there.\n> \n> Or you should have to say. But \"git lock <filename>\" should probably \n> put the lock on whatever branch \"git push\" would push to, and similarly \n> for the other argument combinations that \"git push\" permits.\n\nIt seems to me very fragile to rely on the push configuration in deciding\nwhat can be locked and what cannot. Besides this configuration can change\nover time. So what is going to happen with locks then? Another problem:\nwhat if I don't push anyway but usually send pull-requests?\n\nThe fact is if you cannot get your locking working in _one_ repository\nthen any hope that it will work when you have more than one is nothing\nbut a pipe dream.\n\n> \n> > Maybe, it can work, but it sounds too complex to me. I believe that my\n> > idea using SHA-1 is better. After all, what is file? It is its content.\n> > At least, in Git, we always identify files by their content.\n> \n> Not at all; there are plenty of cases where what matters is the path, and \n> some things are relevant by virtue of the form of the filename which names \n> that content.\n\nWhether it matters or not depends on a particular workflow and what the\ndeveloper wants to achieve. Such decisions should be taken by human\nbeing, otherwise you are prone to do the wrong things too often.\n\n> \n> > Thus if you lock some file, you put a lock on certain SHA-1. Now, \n> > regardless of branches and paths, this lock can work provided that you \n> > have access to some shared location. Of course, this lock is purely \n> > advisory, but it is good, because you may want to ignore it in some \n> > case.\n> \n> In my design, the lock (on the shared repository) is not advisory; if \n> someone else has it, you can't push if the new commit doesn't match the \n> old commit for that path.\n\nHey, if someone wants to push this file, it means it is already late,\nbecause you _already_ have the situation where two people have edited\nexactly the same binary file. Isn't the situation that the lock was\nintended to prevent?\n\nSo, the goal should be to warn someone who is going to edit file locked\nby someone else. You cannot prevent him/her from doing so, only to warn\nabout that.\n\nAs to pushing, it can be different policies. IMHO, the update hook is\nthe best place to express what push you want to allow and what not, but\nsome workflow may not use push at all, yet ability to lock (perhaps,\n'synchronize' would be a better word here) may still be needed.\n\n\nDmitry\n"}]}