threads / discuss / 6742

Question on git fetch to bare repo

Subject: Question on git fetch to bare repo

## tl;dr

10 messages between Feb 8, 2007 and Feb 13, 2007.

replies: 9people: 5as markdown or json

Bill Lear· Feb 8, 2007, 22:28 UTC · lore

If I have not yet made myself unwelcome, I now have another problem using git 1.4.4.1.

I have a public bare repo I created yesterday:

% mkdir project && cd project % git --bare init-db --shared % git --bare fetch git://source/project [All seems well]

I have a private repo:

% mkdir project && cd project % git clone /repos/git/project [All is well]

A co-worker checks something in to our company repo, so I go to my public repo to fetch the changes:

% cd /repos/git/project
% git --bare fetch -v git://source/project
remote: Generating pack...
remote: Done counting 230 objects.
remote: Result has 152 objects.
remote: Deltifying 152 objects.
remote:  100% (152/152) done
Unpacking 152 objects
remote: Total 152, written 152 (delta 109), reused 90 (delta 51)
 100% (152/152) done
* fetched git://source/project
  commit: 5c2d43d
I then go to my private repo to pull from my public one:
% cd ~/project
% git branch
  topic
* master
% git pull
Already up-to-date.
% cat .git/remotes/origin
URL: /repos/git/project
Pull: refs/heads/master:refs/heads/origin
Pull: refs/heads/topic:refs/heads/topic
[All seems well with this repo ??]
I try to push from my private to my public:

% git push /repos/git/project Everything up-to-date

I go back to my public repo, and poking around, can't see what is wrong.
The commit that the fetch says it pulled is there, if I do:

% cd /repos/git/project % git --bare show -t 5c2d43d

the patch comes out exactly as it did in my email notification, so I know it's there.

I tried then to be more forceful, and did another fetch like this, which I sort of expected to fail:

% git --bare fetch -v git://source/project master:master
remote: Generating pack...
remote: Done counting 230 objects.
remote: Result has 152 objects.
remote: Deltifying 152 objects.
remote:  100% (152/152) done
Unpacking 152 objects
remote: Total 152, written 152 (delta 109), reused 90 (delta 51)
 100% (152/152) done
* refs/heads/master: fast forward to branch 'master' of git://source/project
  old..new: 37e2298..5c2d43d
Cannot fetch into the current branch.
Having enabled the logs, I can investigate a bit:

% cat logs/refs/heads/master 37e229835103a11365b1e081f9b9987a88437e62 5c2d43dc819fc1bc37ebae1696c3fbfd6a4401db Bill Lear <rael@zopyra.com> 1170973321 -0600 fetch git://source/project: fast-forward 5c2d43dc819fc1bc37ebae1696c3fbfd6a4401db 37e229835103a11365b1e081f9b9987a88437e62 Bill Lear <rael@zopyra.com> 1170973321 -0600 fetch git://source/project: Undoing incorrectly fetched HEAD.

And, my branches seem to be there, properly:
% git --bare branch
  topic
* master

So, I'm confused. Why does my fetch seem to fetch things the first time, yet I cannot pull these into my private repo? I could swear I just did this same sequence of operations on Monday and it worked.

Perhaps I just need a vacation ...
Bill
Jakub Narebski· Feb 8, 2007, 22:34 UTC · re: Bill Lear · lore

Re: Question on git fetch to bare repo

Bill Lear wrote:
Show 6 quoted lines
> I have a public bare repo I created yesterday:
> 
> % mkdir project && cd project
> % git --bare init-db --shared
> % git --bare fetch git://source/project
> [All seems well]
_Seems_ well
  % git clone --bare --shared git://source/project project
is a proper invocation.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
Johannes Schindelin· Feb 9, 2007, 00:22 UTC · re: Jakub Narebski · lore

Re: Question on git fetch to bare repo

Hi,
[Jakub, you broke the thread. I Cc' Bill, but I cannot join the thread]
On Thu, 8 Feb 2007, Jakub Narebski wrote:
Show 14 quoted lines
> Bill Lear wrote:
> 
> > I have a public bare repo I created yesterday:
> > 
> > % mkdir project && cd project
> > % git --bare init-db --shared
> > % git --bare fetch git://source/project
> > [All seems well]
> 
> _Seems_ well
> 
>   % git clone --bare --shared git://source/project project
> 
> is a proper invocation.
No.

init --shared means that the repository is set up such that different users can update branches.

clone --shared means that if the original repository is on the local machine (which it is not in your example), then that is used as an alternate, i.e. the objects are not copied at all, but reused from the other location.

I complained about that ambiguity, but I am probably the guilty person: AFAIR it was me who introduced "init --shared".

Ciao, Dscho

Johannes Schindelin· Feb 9, 2007, 00:24 UTC · re: Johannes Schindelin · lore

Re: Question on git fetch to bare repo

Hi,
On Fri, 9 Feb 2007, Johannes Schindelin wrote:
> [Jakub, you broke the thread. I Cc' Bill, but I cannot join the thread]

Sorry, PEBCAK. The thread is not broken, but you -- again -- did not reply to the original poster.

Ciao, Dscho

Jeff King· Feb 9, 2007, 02:39 UTC · re: Bill Lear · lore

Re: Question on git fetch to bare repo

On Thu, Feb 08, 2007 at 04:28:29PM -0600, Bill Lear wrote:
> % mkdir project && cd project
> % git --bare init-db --shared
> % git --bare fetch git://source/project
> [All seems well]

It's not. You have fetched the objects from git://source/project, and FETCH_HEAD (an ephemeral pointer to the things you just fetched) points to them. But look in project/refs/heads; you have no branches!

What you want instead is:
  git --bare fetch git://source/project master:master
which means
  "fetch project's master and store it as my master; if it's not a
  fast-forward, then complain loudly. Don't do any merges."
Show 5 quoted lines
> I have a private repo:
> 
> % mkdir project && cd project
> % git clone /repos/git/project
> [All is well]

Is it? You shouldn't have any branches in /repos/git/project at this point.

Show 15 quoted lines
> A co-worker checks something in to our company repo, so I go to my
> public repo to fetch the changes:
> 
> % cd /repos/git/project
> % git --bare fetch -v git://source/project
> remote: Generating pack...
> remote: Done counting 230 objects.
> remote: Result has 152 objects.
> remote: Deltifying 152 objects.
> remote:  100% (152/152) done
> Unpacking 152 objects
> remote: Total 152, written 152 (delta 109), reused 90 (delta 51)
>  100% (152/152) done
> * fetched git://source/project
>   commit: 5c2d43d

Again, you're not _storing_ those changes in a branch, you're just putting them in FETCH_HEAD.

Show 8 quoted lines
> I then go to my private repo to pull from my public one:
> 
> % cd ~/project
> % git branch
>   topic
> * master
> % git pull
> Already up-to-date.

Not surprising, since you didn't actually update any branches in the previous step.

Show 5 quoted lines
> % cat .git/remotes/origin
> URL: /repos/git/project
> Pull: refs/heads/master:refs/heads/origin
> Pull: refs/heads/topic:refs/heads/topic
> [All seems well with this repo ??]

I'm confused as to how there are branches in the 'public' repo at this step, since your initial fetch shouldn't have actually made any, especially not a 'topic' branch.

Show 5 quoted lines
> % cd /repos/git/project
> % git --bare show -t 5c2d43d
> 
> the patch comes out exactly as it did in my email notification, so I
> know it's there.
Sure, you have the object, but you don't have any _pointers_ to it.
Show 15 quoted lines
> I tried then to be more forceful, and did another fetch like this,
> which I sort of expected to fail:
> 
> % git --bare fetch -v git://source/project master:master
> remote: Generating pack...
> remote: Done counting 230 objects.
> remote: Result has 152 objects.
> remote: Deltifying 152 objects.
> remote:  100% (152/152) done
> Unpacking 152 objects
> remote: Total 152, written 152 (delta 109), reused 90 (delta 51)
>  100% (152/152) done
> * refs/heads/master: fast forward to branch 'master' of git://source/project
>   old..new: 37e2298..5c2d43d
> Cannot fetch into the current branch.

You're more on the right track here, but not quite. If you want to save things in branches, you either need to use a 'remotes' shorthand (which defines pull lines with refspecs) or you need to specify the refspec on the command line (like master:master). However, you _don't_ want to fetch directly into your master branch. fetch is for copying refs and objects, not for merging (and you've presumably made some commits on master that the upstream doesn't have).

It seems like you're trying to merge in your "public" repository, which is a mistake. I think you would be much better served to think of your public repository as a place where you publish changes (_only_ from your private repository), and do all of your external fetching and merging in your private repository. Everyone has a public repo, so you always "pull" from other people's public repos, and "push" into your public repo. Thus you don't need --shared at all.

IOW, do this:

# set up public repository; initially empty, but we will push something # useful into it soon. mkdir /git/repo/project && cd /git/repo/project git --bare init

# set up our private repository, which is a clone of the company repo cd $HOME git clone git://source/project

# at this point, our .git/remotes/origin file is set up to pull from
# the company repo. But we still want to publish our changes in our
# personal public repo. Let's set that up. We always want the public
# branches to match ours, even if we've reset or rebased, so we use '+'
# to always overwrite. This is safe, because data is never getting into
# the public repo in any way _except_ for us pushing it.
cat >.git/remotes/publish <<'EOF'
URL: /git/repo/project
Push: +master:master
Push: +topic:topic

# now we can make our first publication, which at this point is the same # as the source repo git push publish

# and now we make some changes hack hack hack git commit

# and we can publish more changes git push publish

# but now we want to grab some changes from Bob's public repo. Let's set
# up a new remote for him. Again, we use '+' to overwrite, since we are
# just tracking what Bob is doing. Note that we have Bob's branch now,
# but we don't _publish_ it, since it's not in our publish remote.
cat >.git/remotes/bob <<'EOF'
URL: /path/to/bobs/repo
Pull: +master:bob-master
EOF

# and now we can fetch/pull from bob git fetch bob # what has bob done that we haven't? gitk bob-master..master # ok, let's merge git pull . bob-master # or do the fetch/pull in one step git pull bob master

And Bob can of course do the same to us.

Does that make more sense? Or have I completely missed what you are trying to accomplish? :)

-Peff
Bill Lear· Feb 9, 2007, 04:19 UTC · re: Jeff King · lore

Re: Question on git fetch to bare repo

On Thursday, February 8, 2007 at 21:39:41 (-0500) Jeff King writes:
>...
>Does that make more sense? Or have I completely missed what you are
>trying to accomplish? :)

No, you haven't misunderstood. This gives me yet more options, so thank you very much.

Bill
Bill Lear· Feb 12, 2007, 20:47 UTC · re: Jeff King · lore

Re: Question on git fetch to bare repo

On Thursday, February 8, 2007 at 21:39:41 (-0500) Jeff King writes:
Show 16 quoted lines
>...
>You're more on the right track here, but not quite. If you want to save
>things in branches, you either need to use a 'remotes' shorthand (which
>defines pull lines with refspecs) or you need to specify the refspec on
>the command line (like master:master). However, you _don't_ want to
>fetch directly into your master branch. fetch is for copying refs and
>objects, not for merging (and you've presumably made some commits on
>master that the upstream doesn't have).
>
>It seems like you're trying to merge in your "public" repository, which
>is a mistake. I think you would be much better served to think of your
>public repository as a place where you publish changes (_only_ from your
>private repository), and do all of your external fetching and merging in
>your private repository. Everyone has a public repo, so you always
>"pull" from other people's public repos, and "push" into your public
>repo. Thus you don't need --shared at all.

Ok, I've gotten some time to read through this. Kinda sad that something published 4 days ago seems like the distant past ...

The problem I have with doing all of my fetching and merging in my private repo is this: I have an update hook in my public repo that I use to communicate my changes to my peers. The problem is when I pull from a peer's repo into my private repo, make some of my changes, and then publish (push) my changes to the public repo, HIS changes are pushed as well, and the update script naturally picks up on these and broadcasts them. My peer group ends up getting the same update message about his commits that they have already received. Multiply this among 6 peers and it becomes a real headache.

On the other hand, if I fetch his changes into my public (bare) repo first, and then pull from there into my private, make changes in my private and then commit and publish to my public, the update script will send out my changes --- and only MY changes --- to my peers.

So, what I have (just now) tried to do (using the latest 1.5 code), is clone my public repo to create my private repo:

% git clone /repos/git/project
and my .git/config file is now:
% cat .git/config
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = /repos/git/project
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master
and no .git/remotes/origin exists.
I notice that this did not "clone" all the branches in my public repo:
% git branch
* master
whereas in my public repo:
% cd /repos/git/project
% git --bare branch
  topic
* master

So, regardless if I can correct that (very confused as to why the clone did not grab the branch), it seems that I did not make my intent clear...

I'm still not sure if I'm obeying the "don't develop on anything on the RHS of ':' dictum".

I'm going to puzzle over the rest of your advice to see if I can grok it and fold it into the new (1.5) way of life.

Bill
Jeff King· Feb 13, 2007, 04:54 UTC · re: Bill Lear · lore

Re: Question on git fetch to bare repo

On Mon, Feb 12, 2007 at 02:47:02PM -0600, Bill Lear wrote:
Show 9 quoted lines
> The problem I have with doing all of my fetching and merging in my
> private repo is this: I have an update hook in my public repo that I
> use to communicate my changes to my peers.  The problem is when I pull
> from a peer's repo into my private repo, make some of my changes, and
> then publish (push) my changes to the public repo, HIS changes are
> pushed as well, and the update script naturally picks up on these and
> broadcasts them.  My peer group ends up getting the same update
> message about his commits that they have already received.  Multiply
> this among 6 peers and it becomes a real headache.

Interesting. Why not have your update hook know who you are, and send out changes only for commits that are either authored by you, or committed by you (depending on your workflow, these may have different results)?

Something like this (mostly untested!) on top of the stock update hook:
diff --git a/templates/hooks--update b/templates/hooks--update
index d4253cb..1598f1b 100644
--- a/templates/hooks--update
+++ b/templates/hooks--update
@@ -55,6 +55,7 @@ projectdesc=$(cat $GIT_DIR/description)
 recipients=$(git-repo-config hooks.mailinglist)
 announcerecipients=$(git-repo-config hooks.announcelist)
 allowunannotated=$(git-repo-config --bool hooks.allowunannotated)
+authorfilter=$(git-repo-config hooks.authorfilter)
 
 # --- Check types
 newrev_type=$(git-cat-file -t "$newrev")
@@ -148,7 +149,7 @@ case "$refname_type" in
 			# This shows all log entries that are not already covered by
 			# another ref - i.e. commits that are now accessible from this
 			# ref that were previously not accessible
-			git-rev-list --pretty $newref $(git-rev-parse --not --all)
+			git-rev-list ${authorfilter:+--author="$authorfilter"} --pretty $newref $(git-rev-parse --not --all)
 			echo $LOGEND
 		else
 			# oldrev is valid
@@ -165,7 +166,7 @@ case "$refname_type" in
 			baserev=$(git-merge-base $oldrev $newrev)
 
 			# Commit with a parent
-			for rev in $(git-rev-list $newrev ^$baserev)
+			for rev in $(git-rev-list ${authorfilter:+--author="$authorfilter"} $newrev ^$baserev)
 			do
 				revtype=$(git-cat-file -t "$rev")
 				echo "       via  $rev ($revtype)"
@@ -190,7 +191,7 @@ case "$refname_type" in
 			fi
 			echo ""
 			echo $LOGBEGIN
-			git-rev-list --pretty $newrev ^$baserev
+			git-rev-list ${authorfilter:+--author="$authorfilter"} --pretty $newrev ^$baserev
 			echo $LOGEND
 			echo ""
 			echo "Diffstat:"


Just set hooks.authorfilter in your config to 'Bill Lear'.  Note that
this still gives the _full_ diffstat between the two endpoints. I would
think you would really want to show a diffstat for each filtered commit
individually. And you're probably not using this hook currently, but I
hope it should be obvious how to modify whatever you are using.

Of course, what you're doing now isn't _wrong_, once we fix the
"committing on tracking branches" problem (which it looks like you are
addressing below), so don't let me drag you too far from your workflow.
:)

> So, what I have (just now) tried to do (using the latest 1.5 code),

Good, 1.5 is much more pleasant to work with. :)

> [remote "origin"]
>         url = /repos/git/project
>         fetch = +refs/heads/*:refs/remotes/origin/*

So this means everything in your public repo's refs/heads/* is mirrored
in your private refs/remotes/origin/*.

> [branch "master"]
>         remote = origin
>         merge = refs/heads/master

And the local branch master will merge from origin's refs/heads/master.

> and no .git/remotes/origin exists.

Right, the config above supersedes it.

> I notice that this did not "clone" all the branches in my public repo:
> 
> % git branch
> * master
> 
> whereas in my public repo:
> 
> % cd /repos/git/project
> % git --bare branch
>   topic
> * master

It did clone it; git-branch just doesn't print remote branches by
default. Try 'git branch -a'. You can create a new topic branch from
your origin topic branch like this:

  git checkout -b topic origin/topic

If you want it to pull automagically from the upstream topic (when you
do a git-pull without any arguments), then you need a config similar to
what it automatically set up for master:

  [branch "topic"]
    remote = origin
    merge = refs/heads/topic

> I'm still not sure if I'm obeying the "don't develop on anything
> on the RHS of ':' dictum".

You are, because 1.5 makes it much harder to do so. Doing a
'git-checkout topic' won't work until you create a local topic branch.
With the detached head work in 1.5, you _can_ do this:

  git checkout origin/topic

but it will issue a warning.

-Peff
Junio C Hamano· Feb 13, 2007, 05:28 UTC · re: Bill Lear · lore

Re: Question on git fetch to bare repo

Bill Lear <rael@zopyra.com> writes:
Show 9 quoted lines
> The problem I have with doing all of my fetching and merging in my
> private repo is this: I have an update hook in my public repo that I
> use to communicate my changes to my peers.  The problem is when I pull
> from a peer's repo into my private repo, make some of my changes, and
> then publish (push) my changes to the public repo, HIS changes are
> pushed as well, and the update script naturally picks up on these and
> broadcasts them.  My peer group ends up getting the same update
> message about his commits that they have already received.  Multiply
> this among 6 peers and it becomes a real headache.

I suspect that is because your "email when pushed" hook lists what commits are new on the branch. I do not use any "email when pushed" hook myself, so I do not know how yours is set up, but I suspect it is doing a moral equivalent of:

	#!/bin/sh
	name=<name of the branch the email will talk about>
	old=<commit before this push updates the branch tip>
        new=<commit this push is trying to update the branch tip with>
	git shortlog $old..$new |
        mailx -s "branch $name updated" recipients

That behaviour might be desirable when branches in the repositories are more or less independent, but in situations like yours where commit are cross pushed into each other's branches, you are not necessarily interested in the progress of each branch.

One solution would be to list only the new commits introduced by the push into the repository, regardless of the branch. You can replace the "git-shortlog $old..$new" with:

	git shortlog $new --not --all
to get that information, I think.
Jeff King· Feb 13, 2007, 05:31 UTC · re: Junio C Hamano · lore

Re: Question on git fetch to bare repo

On Mon, Feb 12, 2007 at 09:28:10PM -0800, Junio C Hamano wrote:
Show 7 quoted lines
> One solution would be to list only the new commits introduced by
> the push into the repository, regardless of the branch.  You can
> replace the "git-shortlog $old..$new" with:
> 
> 	git shortlog $new --not --all
> 
> to get that information, I think.

I don't think that addresses his concern. I had suggested to him that he do all pulling/merging in his private repo, then simply keep the public repo as a mirror. In that case, when pushing out to the public repo, _all_ commits will likely be new to the public repo, including those ones that he pulled from elsewhere.

-Peff

← back to recent threads