Testing for existence of a remote branch from a script

Hi,

I look after some scripts we use at $dayjob for pushing changes though our review system.

For some of our repositories we operate a triangular workflow where changes are fetched from one branch (e.g. 'foo') but are pushed to a different one ('foo_incoming'). Our CI system runs to test the changes and when they pass 'foo_incoming' is merged (fast-forward most of the time) into 'foo'.

The problem I have is not all our projects use this workflow so I've tried to automate the detection of this. My script does something like

  br=$(git rev-parse --symbolic-full-name
refs/remotes/origin/foo_incoming -- 2>/dev/null || echo
refs/remotes/origin/foo)

The '--' is necessary because if foo_incoming doesn't exist then there is extra output on stdout that puts off users. But when foo_incoming does exist then br gets set to `refs/remotes/origin/foo_incoming\n--`.

Is there a better way of checking for the existence of a remote branch?

Thanks, Chris

On Mon, Jan 06, 2025 at 05:40:36PM +1300, Chris Packham wrote:

I may have missed something, would that work: git fetch -p git branch -r

Torsten Bögershausen <tboegi@web.de> writes:

Or "git ls-remote origin refs/heads/foo-incoming" and see if it yields anything?

The "workflow" makes me wonder how it would be bootstrapped, though.

When you add a new branch, "bar", to be pre-reviewed before getting merged at the server side, you want people to push to "bar-incoming". Before the very initial update to "bar-incoming" that pushes into "bar-incoming" and creates it, there wouldn't be "bar-incoming" on the remote side, would there? So "see if foo-incoming exists and change the behaviour on the client end accordingly" may not be a strategy to pursue.

If we wanted to support the "workflow" natively, the way we would do so would be to introduce a protocol capability that allows the server side to advertise:

    If you want to update branch B, you are not allowed to do so
    directly.  Instead, you are expected to push your changes to
    update 'refs/for/B'

and then "git push" on the client end would notice the capability and turns "git push origin B" (or, more likely, the user is on local branch B that is to build on the remote branch B from 'origin', and "git push" with no arguments would do the right thing) into such an update.

I am not suggesting that we jump to the above immediately. But the reason why I am bringing it up is because The "how would I see if they have foo-incoming?" smells like seeking a way to implement such a custom capability advertisement outside Git.

A few random thoughts.

 - Would it be useful if we introduced the ability to advertise
   "custom capabilities" from the receiving end of the connection,
   that does not affect how the rest of Git behaves at all?  It
   would be sort of the reverse of --server-option, which is a
   mechanism to let the client to tell the other side out of band
   information that the rest of Git is oblivious.

   The other side of course needs a way to inspect what capabilities
   are advertised.  For "--server-option", I do not think our server
   end does anything special, but other implementations can act on
   them.  This new thing can start the same way.

 - If this is a poor-man's custom capability advertisement, perhaps
   the server end can create a ref "refs/capabilities/incoming"
   (whose value does not really matter) and your client side can see
   if there is such a ref with "ls-remote"?  That may be a more
   robust thing to do instead, perhaps, as you do not need to worry
   about "What about a new branch 'bar'?" bootstrapping issues.

On Mon, Jan 06, 2025 at 07:30:32AM -0800, Junio C Hamano wrote:

So if enhancing the git server's functionality (either via git ls-remotes or some other operation) is on the table, one of the things that I would really love is some way of asking the question is "git commit <SHA hash>" in the remote repository reachable via some branch or git tag?", and optionally, "which git branch/tag should be fetched if the testing infrastructure wants to be able to test that specific git commit ID?"

I could solve that problem by doing a "git fetch", but in my case, the specific repository in question might not even be on the local client where the user is executing "gce-xfstests -c ext4/4k -g auto --repo stable-rc.git --commit 47c2f92131c4". (This command launches a VM which actually builds the kernel, and I'd like to be able to give immediate feedback about whether the command will fail due to the user typo'ing the git commit id without either waiting for the Google Cloud VM to launch and then fail, or doing a "git fetch" and determining whether the git commit is there. I might be executing this command while on a very slow networking link, such as when I'm in an airplane or on a cruise ship, and so the cost of doing the "git fetch" might be prohibitive.)

Thanks,

						- Ted

"Theodore Ts'o" <tytso@mit.edu> writes:

Enhancements that do not require breaking backward compatibility is always on the table ;-).

Both sounds like a useful thing to do, but I wonder how generic these should be and at the same time how common a narrowed-down feature would suffice. If we try to make it generally very useful, at some point, we'd cross the line where we'd be better off doing "run ssh and execute these Git commands" over the wire X-<.

There are server-side-minded folks who are extending "cat-file --batch" to allow you to ask about objects you do not have but the other end has, if I am not mistaken, by the "remote-object-info" feature?

I wonder if these more advanced "info" about objects you mentioned fit into the picture well as part of it.

Thanks.

On Tue, Jan 7, 2025 at 4:30 AM Junio C Hamano <gitster@pobox.com> wrote:

git ls-remote --exit-code origin refs/heads/foo_incoming would work for me. And I think it would save me a fetch to get past one bootstrapping issue.

Currently we require the first person to know what they're doing and do the initial push as `HEAD:refs/heads/foo_incoming` the CI tooling we have set up knows to create `foo` if it doesn't already exist. That's probably not too bad because for an unborn branch you'd need to say `HEAD:refs/heads/foo` anyway before subsequent pushes would work without the refspec.

Our workflow does require some discipline as that first push should just be either an existing commit (when branching in an existing repo) or a largely empty "Initial commit" for a new repository. That all relies on a "gentlepersons's agreement" that the first push to one of these branches doesn't require review because there's nothing really to enforce that policy. Pushing to the non-incoming branch is restricted to the CI user via a hook on the server for plain git or with access permissions for gerrit.

I know git has some tooling for a triangular workflow (branch.<name>.pushRemote) I'm not sure if that can be setup to push to a differently named branch. Some kind of branch.<name>.pushRefSpec might help if it could support *:*_incoming.

The main thing I'm scripting on top of is Gerrit's magic refs/for/* so even if git supported our version of a triangular workflow workflow I'd still need something to decide when to use refs/for/foo or refs/for/foo_incoming when dealing with Gerrit.

For the most part we're happy with using a pre-receive hook on the server to block updates when required. We're able to put out an error message that tells the user to push to foo_incoming.

Having some kind of config that made `git push` do the right thing would be great as it would save the user some typing. I don't know how much advertising from the git server this would really require (we could easily update our reject hook to advertise some `git config` settings).

On Mon, Jan 06, 2025 at 05:40:36PM +1300, Chris Packham wrote:

In gce-xfstests[1] I do this via "git ls-remote":

validate_branch_name()
{
    if test -z "$GIT_REPO"
    then
        echo "GIT_REPO is neither found in the config file nor provided with --repo"
        exit 1
    fi
    if [[ "$GIT_REPO" != *".git" ]]; then
        GIT_REPO="$GIT_REPO.git"
    fi
    if ! git ls-remote "$GIT_REPO" > /dev/null; then
	echo -e "Repo not found: $GIT_REPO\n"
	exit 1
    elif ! git ls-remote --heads  --exit-code "$GIT_REPO" $1 > /dev/null; then
	echo -e "$1 is not a valid branch of $GIT_REPO"
	exit 1
    fi
}

See run-fstests/util/parse_cli[2] for this function, and a related function, validate_commit_name if you also want to accept git tags. (The validate_commit_name function isn't perfect, since I don't want to fetch the full git repo to validate a SHA hash specifier, but it's good enough to validate typo'ed tag or branch names.)

[1] https://thunk.org/gce-xfstests [2] https://github.com/tytso/xfstests-bld/blob/master/run-fstests/util/parse_cli

Cheers,

						- Ted