Re: rsync busy non-bare git repo 'source' to quiet

From: Neal Kreitzinger <nkreitzinger@gmail.com>
Date: Mar 8, 2011, 22:20 UTC
Message-ID: <4D76ABB1.9080001@gmail.com>
In-Reply-To: <20110308213959.GB5786@sigill.intra.peff.net>

On 3/8/2011 3:39 PM, Jeff King wrote:

Show 42 quoted lines

> On Tue, Mar 08, 2011 at 03:25:00PM -0600, Neal Kreitzinger wrote:
>
>> Does anyone have an example of an rsync bash script that will make
>> a good copy of a non-bare git repo (including the working tree)
>> while the "source" git repo is busy and the "destination" git repo
>> is quiet?
>
> Don't copy the working tree. It's redundant with the repo data
> (assuming your working tree is clean), so you are probably faster to
> sync .git and then "reset --hard". And then you don't have to worry
> about fetching an inconsistent working tree state.
>
> For syncing the repo, I think you would need to do:
>
> 1. Copy the refs to a local temp space.
>
> 2. Copy the object db.
>
> 3. Install your temp refs into place.
>
> That way, for any updates in progress you will either not copy them
> (because the refs weren't in place in step 1, though you may have
> some of their objects), or if you do copy them, you are guaranteed to
> have all of the necessary objects (because git will not update the
> ref until all objects are in place).
>
> But I really have to wonder why you don't simply use git to do the
> fetch? It already does the right thing with respect to updates, and
> it will be way more efficient than rsync in the face of repacking (or
> if you really do want to use rsync, there is even an rsync transport
> for git already). It sounds like:
>
>> This would make it very easy for us to refresh our "beta" livebox
>> to emulate the current "gold" livebox using a single rsync instead
>> of a combination of rsync and git-clone/pull due to the pieces that
>> git does not replicate (ie, hooks) and the non-git components of
>> our git based change control menu system (which is written in bash
>> scripts on linux).
>
> You just don't want to do a pull in addition to an rsync. But I
> don't think this solution will be any less complex.
>

One reason is that we only have one development box. We would have to open up the canonical repo and development repos to git:// protocol access. If I use git I have to do the initial clones for creation and then the pulls for refreshes. rsync will do the creation and refreshes via the same script. If I use git I would have to rsync the hooks, config, and anything else git doesn't bring over. On the goldbox I have a bare repo that mirrors the canonical repo and then has additional branches and is in turn mirrored by another bare mirror which has all the additional branches. I would have to recreate the original remote branch setup and then still maintain the remotes to the goldbox. Then I wouldn't really have a simulation of the goldbox anymore because I have extra remotes (maybe that wouldn't really hurt anything). Rsync seems like a simpler solution and more accurate solution for creating a copy of an ecosystem of interrelated git repos colocated on the same box.

A previous post in the newsgroup states: "> If you want your rsync backup to be fine, you need to follow some

Show 5 quoted lines

> ordering.  You need to copy the refs first (.git/packed-refs and
> .git/refs/), then the loose objects (.git/objects/??/*), and then all
> the rest.  If files are copied in a different order while some write
> operations are performed on the source repository then you may end up
> with an incoherent repository."

Would that work?

v/r, neal