threads / rfc / 55219

[RFC] new subcommand: git sync

Subject: [RFC] new subcommand: git sync

## tl;dr

7 messages between Feb 26, 2021 and Mar 1, 2021.

replies: 6people: 4as markdown or json

Jean-Marie Lemetayer· Feb 26, 2021, 09:08 UTC · lore
Hi folks,

I created a new "git sync" sub-command a few months ago to deal with the pull request workflow.

Its goals are to:
 - keep all configured branches synchronized with the remotes (--set-upstream)
 - do not touch your wip feature branches (which has diverged from upstream)
 - prune the remotes

As I use it on a daily basis, to synchronize the remotes and then be able to quickly rebase my pull requests. I think it's worth sharing. What do you think?

For now it is a simple shell script available here: https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync

If you think it's a good idea, I'll propose a series of patches with the new sub-command, the manual page and the associated tests.

Best regards, Jean-Marie Lemetayer

Ævar Arnfjörð Bjarmason· Feb 26, 2021, 15:25 UTC · re: Jean-Marie Lemetayer · lore

Re: [RFC] new subcommand: git sync

On Fri, Feb 26 2021, Jean-Marie Lemetayer wrote:
Show 18 quoted lines
> Hi folks,
>
> I created a new "git sync" sub-command a few months ago to deal with the pull
> request workflow.
>
> Its goals are to:
>  - keep all configured branches synchronized with the remotes (--set-upstream)
>  - do not touch your wip feature branches (which has diverged from upstream)
>  - prune the remotes
>
> As I use it on a daily basis, to synchronize the remotes and then be able to
> quickly rebase my pull requests. I think it's worth sharing. What do you think?
>
> For now it is a simple shell script available here:
> https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync
>
> If you think it's a good idea, I'll propose a series of patches with the new
> sub-command, the manual page and the associated tests.

Have you seen 'git branch -v' and 'git branch -v --format=*'? There seems to be a high amount of overlap between this wrapper you've written and it.

I suspect most of what you have here could be turned into an %(if:*) directive where you emit the pull/push command as appropriate.

If you search the internet for "git-sync" there's dozens of such command (and I've personally observed at least two of them being written by co-workers in real time, not sure if either of those is in the Google results).

So I think there's probably a worthwhile problem to be solved here that could be turned into patches to git.git, something between "git [clone|push] --mirror" and "git branch -v".

I don't think there's any interest in getting new shellscript built-ins in the future. We've been actively migrating away from those.

But most of the logic in your script is just calling the ref-filter.c API behind the scenes.

B.t.w. you can probably speed up & simplify your script a lot by making use of IFS="" in the shell and not calling N for-each-ref commands when it seems to me that one invocation would do. Just dump the N fields you need split on some token, and split on that token in your loop.

Randall S. Becker· Feb 26, 2021, 16:42 UTC · re: Ævar Arnfjörð Bjarmason · lore

RE: [RFC] new subcommand: git sync

On February 26, 2021 10:25 AM, : Ævar Arnfjörð Bjarmason wrote:
Show 52 quoted lines
> To: Jean-Marie Lemetayer <jeanmarie.lemetayer@gmail.com>
> Cc: git@vger.kernel.org
> Subject: Re: [RFC] new subcommand: git sync
> 
> On Fri, Feb 26 2021, Jean-Marie Lemetayer wrote:
> 
> > Hi folks,
> >
> > I created a new "git sync" sub-command a few months ago to deal with
> > the pull request workflow.
> >
> > Its goals are to:
> >  - keep all configured branches synchronized with the remotes
> > (--set-upstream)
> >  - do not touch your wip feature branches (which has diverged from
> > upstream)
> >  - prune the remotes
> >
> > As I use it on a daily basis, to synchronize the remotes and then be
> > able to quickly rebase my pull requests. I think it's worth sharing. What do
> you think?
> >
> > For now it is a simple shell script available here:
> > https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync
> >
> > If you think it's a good idea, I'll propose a series of patches with
> > the new sub-command, the manual page and the associated tests.
> 
> Have you seen 'git branch -v' and 'git branch -v --format=*'? There seems to
> be a high amount of overlap between this wrapper you've written and it.
> 
> I suspect most of what you have here could be turned into an %(if:*)
> directive where you emit the pull/push command as appropriate.
> 
> If you search the internet for "git-sync" there's dozens of such command
> (and I've personally observed at least two of them being written by co-
> workers in real time, not sure if either of those is in the Google results).
> 
> So I think there's probably a worthwhile problem to be solved here that
> could be turned into patches to git.git, something between "git [clone|push]
> --mirror" and "git branch -v".
> 
> I don't think there's any interest in getting new shellscript built-ins in the
> future. We've been actively migrating away from those.
> 
> But most of the logic in your script is just calling the ref-filter.c API behind the
> scenes.
> 
> B.t.w. you can probably speed up & simplify your script a lot by making use of
> IFS="" in the shell and not calling N for-each-ref commands when it seems to
> me that one invocation would do. Just dump the N fields you need split on
> some token, and split on that token in your loop.
Yeah, I'm one of those that has made extensive use of the name "sync" when using git to synchronize between (POSIX) OSS/USS and (Non-POSIX) NonStop/MVS respectively. If you're going somewhere with it, could I suggest something like "reconcile" or of it is specific to the pull workflow, maybe "pull-sync"? I agree that scripts are not desirable long-term.

My $0.02. Randall

-- Brief whoami:
 NonStop developer since approximately 211288444200000000
 UNIX developer since approximately 421664400
 MVS developer a long time ago in a galaxy far far away
-- In my real life, I talk too much.
Jean-Marie Lemetayer· Mar 1, 2021, 09:48 UTC · re: Randall S. Becker · lore

Re: [RFC] new subcommand: git sync

On Fri, Feb 26, 2021 at 5:42 PM Randall S. Becker <rsbecker@nexbridge.com> wrote:

Show 56 quoted lines
>
> On February 26, 2021 10:25 AM, : Ævar Arnfjörð Bjarmason wrote:
> > To: Jean-Marie Lemetayer <jeanmarie.lemetayer@gmail.com>
> > Cc: git@vger.kernel.org
> > Subject: Re: [RFC] new subcommand: git sync
> >
> > On Fri, Feb 26 2021, Jean-Marie Lemetayer wrote:
> >
> > > Hi folks,
> > >
> > > I created a new "git sync" sub-command a few months ago to deal with
> > > the pull request workflow.
> > >
> > > Its goals are to:
> > >  - keep all configured branches synchronized with the remotes
> > > (--set-upstream)
> > >  - do not touch your wip feature branches (which has diverged from
> > > upstream)
> > >  - prune the remotes
> > >
> > > As I use it on a daily basis, to synchronize the remotes and then be
> > > able to quickly rebase my pull requests. I think it's worth sharing. What do
> > you think?
> > >
> > > For now it is a simple shell script available here:
> > > https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync
> > >
> > > If you think it's a good idea, I'll propose a series of patches with
> > > the new sub-command, the manual page and the associated tests.
> >
> > Have you seen 'git branch -v' and 'git branch -v --format=*'? There seems to
> > be a high amount of overlap between this wrapper you've written and it.
> >
> > I suspect most of what you have here could be turned into an %(if:*)
> > directive where you emit the pull/push command as appropriate.
> >
> > If you search the internet for "git-sync" there's dozens of such command
> > (and I've personally observed at least two of them being written by co-
> > workers in real time, not sure if either of those is in the Google results).
> >
> > So I think there's probably a worthwhile problem to be solved here that
> > could be turned into patches to git.git, something between "git [clone|push]
> > --mirror" and "git branch -v".
> >
> > I don't think there's any interest in getting new shellscript built-ins in the
> > future. We've been actively migrating away from those.
> >
> > But most of the logic in your script is just calling the ref-filter.c API behind the
> > scenes.
> >
> > B.t.w. you can probably speed up & simplify your script a lot by making use of
> > IFS="" in the shell and not calling N for-each-ref commands when it seems to
> > me that one invocation would do. Just dump the N fields you need split on
> > some token, and split on that token in your loop.
>
> Yeah, I'm one of those that has made extensive use of the name "sync" when using git to synchronize between (POSIX) OSS/USS and (Non-POSIX) NonStop/MVS respectively. If you're going somewhere with it, could I suggest something like "reconcile" or of it is specific to the pull workflow, maybe "pull-sync"? I agree that scripts are not desirable long-term.
Thanks for the feedback !

To sum up -- if I have the time to do it -- I must do it in native C and not use a generic name like "sync".

As it is a way to keep my branches up to date with the remotes, maybe create an option for the "git branch" submodule instead of a new subcommand? Maybe "git branch --synchronize" or something better ...

As using "Pull request" is not the only way to use this command and also because there is a big trolling issue with the termes"Pull request" and "Merge request" I don't think using "pull-sync" is a good idea either.

I know that the first golden rule of coding is "Naming", but damn, sometimes it's hard to find a good name.

Best regards, Jean-Marie

Randall S. Becker· Mar 1, 2021, 13:55 UTC · re: Jean-Marie Lemetayer · lore

RE: [RFC] new subcommand: git sync

On March 1, 2021 4:48 AM, Jean-Marie Lemetayer wrote:
Show 87 quoted lines
> To: Randall S. Becker <rsbecker@nexbridge.com>
> Cc: Ævar Arnfjörð Bjarmason <avarab@gmail.com>; git@vger.kernel.org
> Subject: Re: [RFC] new subcommand: git sync
> 
> On Fri, Feb 26, 2021 at 5:42 PM Randall S. Becker <rsbecker@nexbridge.com>
> wrote:
> >
> > On February 26, 2021 10:25 AM, : Ævar Arnfjörð Bjarmason wrote:
> > > To: Jean-Marie Lemetayer <jeanmarie.lemetayer@gmail.com>
> > > Cc: git@vger.kernel.org
> > > Subject: Re: [RFC] new subcommand: git sync
> > >
> > > On Fri, Feb 26 2021, Jean-Marie Lemetayer wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I created a new "git sync" sub-command a few months ago to deal
> > > > with the pull request workflow.
> > > >
> > > > Its goals are to:
> > > >  - keep all configured branches synchronized with the remotes
> > > > (--set-upstream)
> > > >  - do not touch your wip feature branches (which has diverged from
> > > > upstream)
> > > >  - prune the remotes
> > > >
> > > > As I use it on a daily basis, to synchronize the remotes and then
> > > > be able to quickly rebase my pull requests. I think it's worth
> > > > sharing. What do
> > > you think?
> > > >
> > > > For now it is a simple shell script available here:
> > > > https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync
> > > >
> > > > If you think it's a good idea, I'll propose a series of patches
> > > > with the new sub-command, the manual page and the associated tests.
> > >
> > > Have you seen 'git branch -v' and 'git branch -v --format=*'? There
> > > seems to be a high amount of overlap between this wrapper you've
> written and it.
> > >
> > > I suspect most of what you have here could be turned into an %(if:*)
> > > directive where you emit the pull/push command as appropriate.
> > >
> > > If you search the internet for "git-sync" there's dozens of such
> > > command (and I've personally observed at least two of them being
> > > written by co- workers in real time, not sure if either of those is in the
> Google results).
> > >
> > > So I think there's probably a worthwhile problem to be solved here
> > > that could be turned into patches to git.git, something between "git
> > > [clone|push] --mirror" and "git branch -v".
> > >
> > > I don't think there's any interest in getting new shellscript
> > > built-ins in the future. We've been actively migrating away from those.
> > >
> > > But most of the logic in your script is just calling the
> > > ref-filter.c API behind the scenes.
> > >
> > > B.t.w. you can probably speed up & simplify your script a lot by
> > > making use of IFS="" in the shell and not calling N for-each-ref
> > > commands when it seems to me that one invocation would do. Just
> dump
> > > the N fields you need split on some token, and split on that token in your
> loop.
> >
> > Yeah, I'm one of those that has made extensive use of the name "sync"
> when using git to synchronize between (POSIX) OSS/USS and (Non-POSIX)
> NonStop/MVS respectively. If you're going somewhere with it, could I
> suggest something like "reconcile" or of it is specific to the pull workflow,
> maybe "pull-sync"? I agree that scripts are not desirable long-term.
> 
> Thanks for the feedback !
> 
> To sum up -- if I have the time to do it -- I must do it in native C and not use a
> generic name like "sync".
> 
> As it is a way to keep my branches up to date with the remotes, maybe
> create an option for the "git branch" submodule instead of a new
> subcommand? Maybe "git branch --synchronize" or something better ...
> 
> As using "Pull request" is not the only way to use this command and also
> because there is a big trolling issue with the termes"Pull request" and
> "Merge request" I don't think using "pull-sync" is a good idea either.
> 
> I know that the first golden rule of coding is "Naming", but damn, sometimes
> it's hard to find a good name.
It's almost feels like you are thinking along the lines of a variant of a mirror rather than pulls and merges. I wonder whether that analogy would hold up? I would love to have a stable mechanism of mirroring repositories when doing development on 2-3 platforms at the same time. Yes, pull/merge can be a bit heavy handed in this situation. I have considered something like rsync to do the job (which it can), but within a git context it sounds more like a mirror - although it would be nice to mirror work in progress. So perhaps this should go through the stash mechanism.
Junio C Hamano· Feb 26, 2021, 22:31 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [RFC] new subcommand: git sync

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> B.t.w. you can probably speed up & simplify your script a lot by making
> use of IFS="" in the shell and not calling N for-each-ref commands when
> it seems to me that one invocation would do. Just dump the N fields you
> need split on some token, and split on that token in your loop.

A hidden gem in for-each-ref is its ability to quote the placeholder values in a language specific way, and that is to allow the --format to generate a script that can be eval'ed. E.g.

  $ git for-each-ref --shell \
    --format='doit %(authorname) %(subject)' refs/heads/ab/\* |
    head -n 3
  doit 'Ævar Arnfjörð Bjarmason' 'branch: show "HEAD detached" first ...'
  doit 'Ævar Arnfjörð Bjarmason' 'CoC: update to version 2.0 + local ...'
  doit 'Ævar Arnfjörð Bjarmason' 'config.mak.uname: remove unused NEE...'
so that you can prepare
	doit () {
		person=$1 subject=$2
		... do things on the branch data ...
	}

beforehand and then eval the output from the script you wrote with "git for-each-ref".

Junio C Hamano· Feb 26, 2021, 23:20 UTC · re: Junio C Hamano · lore

Re: [RFC] new subcommand: git sync

Junio C Hamano <gitster@pobox.com> writes:
> so that you can prepare
> ...
> beforehand and then eval the output from the script you wrote with
> "git for-each-ref".

Ehhh, not "eval the output from the script", but "eval the script". In essense, you use "for-each-ref --format" as a tool to write a script, and then execute it.

← back to recent threads