Re: [PATCH v3 3/4] gitfaq: add entry about syncing working trees

From: brian m. carlson <sandals@crustytoothpaste.net>
Date: Jul 4, 2024, 21:08 UTC
Message-ID: <ZocPYKyVzSDIekTK@tapette.crustytoothpaste.net>
In-Reply-To: <xmqqjzi191fw.fsf@gitster.g>

On 2024-07-04 at 05:21:55, Junio C Hamano wrote:

Show 11 quoted lines

> "brian m. carlson" <sandals@crustytoothpaste.net> writes:
> 
> > -Credentials
> > ------------
> > +Credentials and Transfers
> > +-------------------------
> 
> I can see (and appreciate) that you struggled to find a good section
> to piggyback on, instead of giving this topic its own section.  But
> do these two make a good mix?  They seem to be totally different
> topics.

I can try again.

Show 24 quoted lines

> > +It is important not to use a cloud syncing service to sync any portion of a Git
> > +repository, since this can cause corruption, such as missing objects, changed
> > +or added files, broken refs, and a wide variety of other corruption.  These
> > +services tend to sync file by file on a continuous basis and don't understand
> > +the structure of a Git repository.  This is especially bad if they sync the
> > +repository in the middle of it being updated, since that is very likely to
> > +cause incomplete or partial updates and therefore data loss.
> 
> A naïve reader may say "but isn't it the point of these cloud
> syncing service that they will eventually catch up???" and we may
> want to have a good story why it does not work.
> 
>     You create many objects in one repository in loose form, cloud
>     syncing service kicks in to transfer them to the second
>     repository, and then in the original repository an auto-gc kicks
>     in so some of the loose objects fail to propagate.  The packfile
>     that is the result of auto-gc will eventually propagate to the
>     second repository, but before it completes, the second
>     repository would be in an inconsistent state, and especially if
>     the ref updates are propagated before objects, then the second
>     repository will be in a corrupt state.  It would be a disaster
>     if another auto-gc kicked in there.
> 
> is one scenario I came up with.

The most common situation we see is that refs tend to be renamed to things like "refs/heads/main 2", which is obviously not a valid refname and doesn't work, or the ref gets rolled back to an older version. Working trees also get stuck into weird states where files keep coming back or getting deleted, or the index gets two differently named copies, neither of which is "index".

It is _less_ likely that objects are renamed, but it could be that the tool thinks they've been legitimately deleted if the loose objects get packed and then they do get deleted elsewhere without another source of those objects existing. I'm not sure how object loss happens in the real world with these services, but there have been users reporting it on StackOverflow, so I'm confident it does occur.

If we have users who ask about this, I'm happy to answer them on the list. I don't want to explain the various and sundry scenarios in the FAQ entry in order to keep it short, but I can find several examples of problems if need be.

-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA