backup or mirror a repository

Hello list,

I have been poking around on IRC and gmane, but haven't yet seen a solution to my problem:

What I really want is something semantically identical to a recursive copy of my entire git repository on one machine sent to another machine. Right now I can't simply use "rsync -ar --delete ..." because the pack changes names, thus fooling rsync and sending a bunch of bits over the network.

I saw git-mirror on gmane and asked around in IRC, but the former seems to have died sometime last November and the latter told me I'd probably have to write some glue to process git show-ref and git ls-remote. This seems viable, but suboptimal. Another user chimed in that they also had the same use case as myself.

fdr

(Justifications: I am super paranoid, so I want to push my local branches to another disk to be safe, but then they proliferate over time since deleted local branches are not also deleted on the target repository. I also want to work on multiple machines seamlessly, and as such doing a bunch of "git checkout -r ..." when I need to change branches is also suboptimal. As I said, I can get what I want by simply using rsync, so this is entirely about efficiency by exploiting git's structure)

Hi,

On Thu, 27 Sep 2007, Dan Farina wrote:

I guess you're looking for "git remote add --mirror origin <url>". AFAIK this is not in any released version yet, though.

Ciao, Dscho

On Thu, 2007-09-27 at 22:56 +0100, Johannes Schindelin wrote:

I just tried a bleeding edge git with --mirror.

It's close, except (unless there is some other provision to take care of this) remote branch deletes. (Which, after more gmane trolling, seemed to be "git remote rm", which doesn't seem present just yet).

Of course, it is completely possible I'm using it incorrectly. In brief, I did "git init" "git remote add --mirror backup <url>" "git pull" (perhaps I should have used "git remote update"?) And the behavior of this is more or less what I wanted, as far as sending branches. Convenient removal of branches and propagating that removal still stumps me. Is there a preferred way to do this yet?

Thanks, fdr

Hi,

On Thu, 27 Sep 2007, Dan Farina wrote:

git remote rm is about removing the remote nick, not a remote branch.

"git pull" without further parameters accesses the remote "origin", not "backup", and it merges _one_ branch into the current branch.

What you should have done is "git fetch backup".

Or use "origin" instead of "backup" right from the start, and then use "git fetch".

But no pull.

You want to remove branches from the local repo which are no longer there on the remote side?

Then "git remote prune" is for you.  For details, please see the man page.

Ciao, Dscho

On Fri, 2007-09-28 at 02:56 +0100, Johannes Schindelin wrote:

Ah, I thought for some reason there'd be a "remove" (since the other operations have long names) and, much like "ls-remote" there'd be some "rm-remote." An insane presupposition of mine from having too little information.

I did use "pull backup" (my error for being sloppy and omitting it from my mail) and it does work provided I do the dirty thing of doing a subsequent reset --hard (I thought perhaps things were not yet fully-there). Fetch does it perfectly, as you say, thanks!

I did look at prune and update, but my problem is the opposite: I want something that will remove branches from the remote repo when they no longer exist locally. As-is over time I will proliferate little local branches unless I occasionally sit down and delete branches by operating directly on the bare backup repository. (and then use prune on the remote nodes)

Thanks, fdr

Dan Farina <drfarina@gmail.com> writes:

The "git remote add --mirror" setup is about setting up the local repository _AS_ the backup of the remote. In other words, the contents come from the remote by fetching from it and safely kept away from disaster on the local side. And for that, "remote prune" is a perfect thing to do.

I think what you are asking for is an opposite, a backup remote site you would push into. That is not what "remote add --mirror" is about.

You can almost do it with

	git push --all $remote

except there is no way to automagically remove the branch you removed from the local repository. For that, we would need a new --mirror option to "git-push".

I think it is trivial to do for native transports, as we first get the list of all refs from the remote side before starting the transfer. You need to change the last parameter called 'all' to remote.c::match_refs() into an enum ('push_all' being one of choices), introduce another enum 'push_mirror', and teach it to "match" the remote (i.e. dst) ref that does not have corresponding entry on our side (i.e. src) with an empty object name to mark it removed. Then the part marked as "Finally, tell the other end!" in send-pack.c::send_pack() will take care of the actual removal.

On Thu, 2007-09-27 at 23:27 -0700, Junio C Hamano wrote:

Yup! I actually was looking for such a thing in the git-push man page, but was unsuccessful. I was living with git-push --all for a little while before I thought I'd ask. Unfortunately, the preferred setup (having the backup machine actively perform fetch and prune) is not very nice for me due to firewalls.

I should have a look, but if someone else wants to work on this they shouldn't block on me: work already made me get hit by a bus on one project (luckily I wasn't important), and the storm is not over.

Thanks, fdr

On Thu, Sep 27, 2007 at 11:27:06PM -0700, Junio C Hamano wrote:

I have tried to do that but I am getting a warning:
$ git remote prune origin
Warning: unrecognized mapping in remotes.origin.fetch: +refs/*:refs/*
and no branch is removed.

I suspect that the change that introduced --mirror option for the 'add' command did not adjust the prune procedure to handle the new situation properly. Or is just me doing something wrong?

Anyway, because the official released git still does not have --mirror and it could be difficult to convince the admin to use the bleading edge version of Git for secure backup, I wrote a simple script instead, which provides the same functionality but using the released version of Git. Maybe, someone else finds it useful.

$ cat git-mirror #!/bin/sh

usage() {
	echo >&2 "Usage: $0 URL"
	exit 1
}

set -e

test $# -eq 1 || usage URL="$1"

# Initialize Git bare directory git --bare init

# Initialize the mirror git remote add origin "$URL" git fetch rmdir refs/heads ln -s remotes/origin refs/heads

# Adding creating a script that will be run by cron cat <<EOF > git-mirror-sync cd "$PWD" git fetch git remote prune origin EOF chmod +x git-mirror-sync

# Adding git-mirror-sync to cron echo "Please, add $PWD/git-mirror-sync to cron"

#########################################################

Dmitry

Hi,

On Thu, 18 Oct 2007, Dmitry Potapov wrote:

No, you're right. I did not anticipate git-remote to be written the way it is. After fiddling with it for several hours, I am giving up for now. IMHO the script is too married to the idea that the remote branches live in refs/remotes/<remote>/*. Probably it would be very easy by now to implement it as a builtin, using remote.[ch].

Ciao, Dscho