threads / discuss / 32653

Re: Question re. git remote repository

Subject: Re: Question re. git remote repository

## tl;dr

8 messages between Jan 16, 2013 and Jan 17, 2013.

replies: 7people: 2as markdown or json

Matt Seitz (matseitz)· Jan 16, 2013, 23:32 UTC · lore
"David Lang" <david@lang.hm> wrote in message news:<alpine.DEB.2.02.1301161459060.21503@nftneq.ynat.uz>...
> But if you try to have one filesystem, with multiple people running git on their 
> machines against that shared filesystem, I would expect you to have all sorts of 
> problems.
What leads you to think you will have problems?
Why would there be more of a problem on a network file system as opposed to local file system that can be accessed by multiple users?
Linus seemed to think it should work:
http://permalink.gmane.org/gmane.comp.version-control.git/122670
And "git init" specifically has a "shared" option:

--shared[=(false|true|umask|group|all|world|everybody|0xxx)] Specify that the git repository is to be shared amongst several users. This allows users belonging to the same group to push into that repository. When specified, the config variable "core.sharedRepository" is set so that files and directories under $GIT_DIR are created with the requested permissions. When not specified, git will use permissions reported by umask(2).

David Lang· Jan 17, 2013, 00:21 UTC · re: Matt Seitz (matseitz) · lore
On Wed, 16 Jan 2013, Matt Seitz (matseitz) wrote:
Show 9 quoted lines
> "David Lang" <david@lang.hm> wrote in message news:<alpine.DEB.2.02.1301161459060.21503@nftneq.ynat.uz>...
>> But if you try to have one filesystem, with multiple people running git on their
>> machines against that shared filesystem, I would expect you to have all sorts of
>> problems.
>
> What leads you to think you will have problems?
>
> Why would there be more of a problem on a network file system as opposed to 
> local file system that can be accessed by multiple users?

There are safety checks and synchronization primitives that work between mulitiple users on one machine (where you can see what other processes are running for example) that don't work with separate machines using a filesystem

> Linus seemed to think it should work:
>
> http://permalink.gmane.org/gmane.comp.version-control.git/122670

well, he knows git better than I do, but using git over NFS/CIFS is not the same as saying that you have multiple users on different systems making changes.

In the link you point at, he says that you can have problems with some types of actions. He points out things like git prune, but I would also say that there are probably race conditions if you have two git processes that try to change the HEAD to different things at the same time.

Show 10 quoted lines
> And "git init" specifically has a "shared" option:
>
> --shared[=(false|true|umask|group|all|world|everybody|0xxx)]
>
> Specify that the git repository is to be shared amongst several users. This 
> allows users belonging to the same group to push into that repository. When 
> specified, the config variable "core.sharedRepository" is set so that files 
> and directories under $GIT_DIR are created with the requested permissions. 
> When not specified, git will use permissions reported by umask(2).
>

I think this is dealing with multiple users _reading_ a repository, not making updates to it at the same time.

David Lang
Matt Seitz (matseitz)· Jan 17, 2013, 01:09 UTC · re: David Lang · lore

RE: Question re. git remote repository

Show 11 quoted lines
> From: David Lang [mailto:david@lang.hm]
> 
> On Wed, 16 Jan 2013, Matt Seitz (matseitz) wrote:
> 
> > Linus seemed to think it should work:
> >
> > http://permalink.gmane.org/gmane.comp.version-control.git/122670
> 
> In the link you point at, he says that you can have problems with some
> types of
> actions. He points out things like git prune, 
Linus wrote:

You do need to be a bit careful if you do maintenance operations concurrently (I would suggest avoiding doing concurrent "git gc --prune", for example), but any normal git workflow should be fine.

> but I would also say that there
> are probably race conditions if you have two git processes that try to
> change the HEAD to different things at the same time.
What makes you think there are race conditions?
Linus wrote:

And git doesn't have "proper locking", because it doesn't need it for database ops: git objects are stable. For refs, git should be using the proper NFS-safe "create and atomic rename" ops.

Show 7 quoted lines
> > And "git init" specifically has a "shared" option:
> >
> > --shared[=(false|true|umask|group|all|world|everybody|0xxx)]
> 
> I think this is dealing with multiple users _reading_ a repository, not
> making
> updates to it at the same time.
The description of "shared" says "This allows users belonging to the same group to push into that repository."  The "push" command is about making updates.
David Lang· Jan 17, 2013, 01:26 UTC · re: Matt Seitz (matseitz) · lore

RE: Question re. git remote repository

On Thu, 17 Jan 2013, Matt Seitz (matseitz) wrote:
Show 29 quoted lines
>> From: David Lang [mailto:david@lang.hm]
>>
>> On Wed, 16 Jan 2013, Matt Seitz (matseitz) wrote:
>>
>>> Linus seemed to think it should work:
>>>
>>> http://permalink.gmane.org/gmane.comp.version-control.git/122670
>>
>> In the link you point at, he says that you can have problems with some
>> types of
>> actions. He points out things like git prune,
>
> Linus wrote:
>
> You do need to be a bit careful if you do maintenance operations
> concurrently (I would suggest avoiding doing concurrent "git gc --prune",
> for example), but any normal git workflow should be fine.
>
>> but I would also say that there
>> are probably race conditions if you have two git processes that try to
>> change the HEAD to different things at the same time.
>
> What makes you think there are race conditions?
>
> Linus wrote:
>
> And git doesn't have "proper locking", because it doesn't need it for
> database ops: git objects are stable. For refs, git should be using the
> proper NFS-safe "create and atomic rename" ops.

As Linus points out, objects are stable, so when you create objects you don't have to worry about locking, if two things write an object at the same time, the same contents are being written so races don't matter.

However, if you have two people doing a commit or merge, you will get different results based on the order they are happening in. This seems to be exactly the type of thing that falls into the 'maintinance operations' category.

Linus says that git does not have "proper locking", so think about it, what do you think will happen if person A does git add a/b; git commit and person B does git add c/d; git commit?

Since the tree will look different depending on what order these four commands execute in, the resulting HEAD could have multiple different values depending on the order. The individual commits may even be different.

David Lang
Show 11 quoted lines
>>> And "git init" specifically has a "shared" option:
>>>
>>> --shared[=(false|true|umask|group|all|world|everybody|0xxx)]
>>
>> I think this is dealing with multiple users _reading_ a repository, not
>> making
>> updates to it at the same time.
>
> The description of "shared" says "This allows users belonging to the same 
> group to push into that repository."  The "push" command is about making 
> updates.
Matt Seitz (matseitz)· Jan 17, 2013, 01:46 UTC · re: David Lang · lore

RE: Question re. git remote repository

Show 7 quoted lines
> From: David Lang [mailto:david@lang.hm]
> 
> Linus says that git does not have "proper locking", so think about it,
> what do
> you think will happen if person A does git add a/b; git commit and person
> B does
> git add c/d; git commit?
Sorry, I wasn't clear. My assumption is that a shared repository on a network file system will either be: 
1. a bare repository that is normally accessed only by "git push" and "git pull" (or "git fetch"), the central repository model.
2. a repository where only one user does "git add" and "git commit", while other users will do "git pull", the peer-to-peer model (you pull changes from me, I pull changes from you).
David Lang· Jan 17, 2013, 01:59 UTC · re: Matt Seitz (matseitz) · lore

RE: Question re. git remote repository

On Thu, 17 Jan 2013, Matt Seitz (matseitz) wrote:
Show 11 quoted lines
>> From: David Lang [mailto:david@lang.hm]
>>
>> Linus says that git does not have "proper locking", so think about it,
>> what do
>> you think will happen if person A does git add a/b; git commit and person
>> B does
>> git add c/d; git commit?
>
> Sorry, I wasn't clear. My assumption is that a shared repository on a network file system will either be:
>
> 1. a bare repository that is normally accessed only by "git push" and "git pull" (or "git fetch"), the central repository model.

pulling from it would not be a problem, I could see issues with multiple pushes taking place (the underlying repository would not get corrupted, but you will very quickly hit conflicts where the push is not a fast forward and you need to merge, not just push)

> 2. a repository where only one user does "git add" and "git commit", while 
> other users will do "git pull", the peer-to-peer model (you pull changes from 
> me, I pull changes from you).

At this point only one system is writing to the repository and it doesn't matter that it's on network storage vs local storage.

pulling from a shared repository is probably safe, but I wouldn't bet against there being any conditions where a pull at the same time someone is doing an update being able to cause problems.

The normal thing is to do the pulls through git-daemon, and that does make sure that what you are pulling is consistant.

David Lang
Matt Seitz (matseitz)· Jan 17, 2013, 02:25 UTC · re: David Lang · lore

RE: Question re. git remote repository

Show 11 quoted lines
> From: David Lang [mailto:david@lang.hm]
> 
> On Thu, 17 Jan 2013, Matt Seitz (matseitz) wrote:
> 
> > 1. a bare repository that is normally accessed only by "git push" and
> > "git pull" (or "git fetch"), the central repository model.
> 
> pulling from it would not be a problem, I could see issues with multiple
> pushes taking place (the underlying repository would not get corrupted, but you
> will very quickly hit conflicts where the push is not a fast forward and you
> need to merge, not just push)
How is that different on a network file system, as opposed to using http, ssh, or git-daemon?  Don't you get a "not a fast-forward" error, regardless of the protocol?
Show 12 quoted lines
> > 2. a repository where only one user does "git add" and "git commit",
> while
> > other users will do "git pull", the peer-to-peer model (you pull changes
> from
> > me, I pull changes from you).
> 
> 
> pulling from a shared repository is probably safe, but I wouldn't bet
> against
> there being any conditions where a pull at the same time someone is doing
> an
> update being able to cause problems.
Why do you think there would be a problem?
> The normal thing is to do the pulls through git-daemon, and that does make
> sure
> that what you are pulling is consistant.
What does "git pull" via git-daemon do to ensure consistency that is different from "git pull" on a network file system?
David Lang· Jan 17, 2013, 02:49 UTC · re: Matt Seitz (matseitz) · lore

RE: Question re. git remote repository

On Thu, 17 Jan 2013, Matt Seitz (matseitz) wrote:
Show 13 quoted lines
>> From: David Lang [mailto:david@lang.hm]
>>
>> On Thu, 17 Jan 2013, Matt Seitz (matseitz) wrote:
>>
>>> 1. a bare repository that is normally accessed only by "git push" and
>>> "git pull" (or "git fetch"), the central repository model.
>>
>> pulling from it would not be a problem, I could see issues with multiple
>> pushes taking place (the underlying repository would not get corrupted, but you
>> will very quickly hit conflicts where the push is not a fast forward and you
>> need to merge, not just push)
>
> How is that different on a network file system, as opposed to using http, ssh, or git-daemon?  Don't you get a "not a fast-forward" error, regardless of the protocol?
true.
Show 20 quoted lines
>>> 2. a repository where only one user does "git add" and "git commit",
>> while
>>> other users will do "git pull", the peer-to-peer model (you pull changes
>> from
>>> me, I pull changes from you).
>>
>>
>> pulling from a shared repository is probably safe, but I wouldn't bet
>> against
>> there being any conditions where a pull at the same time someone is doing
>> an
>> update being able to cause problems.
>
> Why do you think there would be a problem?
>
>> The normal thing is to do the pulls through git-daemon, and that does make
>> sure
>> that what you are pulling is consistant.
>
> What does "git pull" via git-daemon do to ensure consistency that is different from "git pull" on a network file system?

git pull via the daemon looks at what tree items you have on each end, and then it sends you the items needed to make you match the server. If there are partial updates on the server (due to some update in process) the daemon should not see that, but if you are grabbing the files directly, I would be less confident that you are always safe.

you may _be_ safe, and if others who really know the internals speak up, take their word on it. But, absent assurances that we know that everything is done in the right order in the face of a networked filesystem (which may break visibility of changes due to caching), I would not trust such raw access for updates at all, and only somewhat trust it for read-only use.

David Lang

← back to recent threads