Recommended work flow with git to send in patches

Hi,

Compressing my "life long story" into a single question -- what's the recommended work flow to work with git and send in patches, when upstream might be slow in respond, and require squashing relevant patches into one?

You can use my following message as a start point, and please answer my last question, which I've been asking twice (in different ways) with no answer.

Please CC me when replying.

Thanks

---------- Forwarded message ----------
From: Tong Sun <suntong@cpan.org>
Date: Sun, Jun 6, 2010 at 8:56 PM
Subject: Working with git and sending in patches
To: grml-devel@ml.grml.org

Hi,

Just trying to put all jigsaw puzzle together here. Please correct me if I'm wrong.

First of all, philosophy for version control with git:

. While developing, small/independent commits are good thing, so that it's easy to decouple different changes.

. But when integrating something in a main branch, commits should contain all logical/related changes.

Steps (using grml-debootstrap as an example):

- do initial git pull into grml-debootstrap

  git pull git://git.grml.org/grml-debootstrap master

- Go into grml-debootstrap and start a new branch

  git checkout -b t/my-working-branch

- work on the code, commit, hack, commit, hack, commit -- commit often
& commit small

- when AOK and need to integrate patches into main branch, squash all
patches into one

  git rebase -i origin/master

- send in patches via email (to grml-devel@ml.grml.org)

  git format-patch origin
  git send-email --to grml-devel@ml.grml.org ...

Please correct me if anything above is wrong.

Now, question, having done above, if I start to work some logically unrelated patches, what steps should I take? (I don't want 'git rebase' to pick up patches that I've already sent in).

On Tue, Jul 27, 2010 at 15:31, Tong Sun <suntong@cpan.org> wrote:

So far so good.

That's just silly. Why is the policy to destroy commit information when submitting patches?

On Tue, Jul 27, 2010 at 11:35 AM, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

Sorry about the confusing. The original message was composed under the condition that all patches are relevant patches that work together to form a big logical changes. I was required to do the squashing.

thanks

Hi Tong,

Tong Sun writes:

Personally, I use `git symbolic-ref` to create a new branch without history, stage and create commits to send off to the list from there; it's also worth noting that I keep the branch to fixup my commits and re-roll after reviews.

-- Ram

On Tue, Jul 27, 2010 at 11:39 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

Thanks a lot for the comment.

Could you elaborate it a bit with actual commands, starting from 'git pull git://remote/project master' please?

I'm new to git and the above comment barely helps me to put all jigsaw puzzle together.

thanks

Hi Tong,

Tong Sun writes:

$ git symbolic-ref HEAD refs/heads/rollout $ rm .git/index # Git recreates it automatically anyway $ git add LICENSE $ git commit -m "Add LICENSE" # root commit $ # git add ... git commit cycle

-- Ram

Ramkumar Ramachandra <artagnon@gmail.com> writes:

Ramkumar, in git 1.7.2 you can simply use "git checkout --orphan newbranch".

Hi Jakub,

Jakub Narebski writes:

Thanks for pointing this out! I'm still using many plubming tools- I ought to find out their equivalents in the new porcelain.

-- Ram

Tong Sun <suntong@cpan.org> writes:

I think that in final results, i.e. in patches that you send, or commits that you send pull request for, you should have commits that do one thing, and do it completely and without breaking. Nevertheless having small commits that you publish / send to maintainer is a good thing; it is always easy to review a few small patches, than one mage-patch.

Why not git-clone (possibly shallow, if you are working on one-shot patch or patch series)?

If you plan to continue working on this repository, and it is not one-shot patch or patch series, it would be better (easier in the future) to use "git remote add".

You can always use 'git commit' + 'git commit --amend' if you want to fix previous commit, instead of creating new commit.

Reorder, edit, squash patches.

With larger patch series, it could possibly be of the form:

    mkdir patches/
    git format-patch -o patches/ --cover-letter
    [edit cover letter, filling in template]
    [edit patches if necessary, adding comments between "---" and diffstat]

You can also use some patch management interface, like StGit, Guilt or TopGit. I personally use StGit, so the above description is modified in that there is 'stg init', there is 'stg new some-patch' and possibly multiple 'stg refresh' when working on commit, there might be 'stg goto's and 'stg push'es and 'stg pop's to go back and forth between patches, and fix them.

It's a matter of taste whether to use some kind of patch management / / mail queue interface, or to use interactive rebase instead.

If you want to start to work on some logically unrelated patches, you should start a new branch for that.

Alternatively, when some or all patches are accepted upstream, you download new changes using 'git pull' or 'git fetch' or 'git remote update', and then rebase your branch on top of branch it is based on, or to be more exact on top of branch your patches are now in. 'git rebase' (or 'git pull --rebase' or even 'git pull', if configured appropriately) would automatically skip patches that got applied (sometimes you need to tell git to 'git rebase --skip' if it didn't detect this).

It would be 'stg rebase <base branch>' (usually <base branch> is 'origin'='origin/master' or 'master'), and 'stg clean -a' to remove empty applied patches.

P.S. Doesn't GRML have web page / wiki page for developers?
     http://grml.org/git/

On Tue, Jul 27, 2010 at 1:35 PM, Jakub Narebski <jnareb@gmail.com> wrote:

Yes, http://grml.org/git/, but it is kind of reference book style, doesn't cover much on the work flow, especially on the topic of no-writing-privilege user contributing back.

Ok, to explain it, I have to touch upon my "life long story" of using git. Long story short on this, the recommended work flow that I searched and found from the Inet was to do 'git clone' from web then 'git clone' a local working copy. Here is my trying log:

# Download the latest version of the repository without downloading all the history, using "shallow checkouts".

  git clone --depth 1 git://git.grml.org/grml-debootstrap.git

create working repo:

  $ git-clone --depth 1 grml-debootstrap grml-debootstrap.working
  Initialized empty Git repository in
/export/repositories/gitwork/grml/grml-debootstrap.working/.git/
  fatal: attempt to fetch/clone from a shallow repository
  ^^^^^^^^^^^^^^^

Seeing that fatal error, and not knowing where to get help from, I just gave up the 'git clone' approach. Please be specific (with git commands), how would I use 'git clone' for working on one-shot patch or patch series.

Got to run, will comment on the rest later.

thanks

On Tue, Jul 27, 2010, Tong Sun wrote:

You could point lack of information for "leaf" contributors to them...

O.K.
 

I don't understand this second step. Why do you want this second clone? It is totally unnecessary. The clone you make is working repository you can make your contributions in.

In short, the workflow should look like this (I use here grml as example, but in practice I used git repository as I know it is fairly small, so the numbers, SHA-1 identifiers and branches won't match)

  $ cd <somewhere>
  $ git clone --depth 1 git://git.grml.org/grml-debootstrap.git
  Initialized empty Git repository in <somwehere>/grml-debootstrap/.git/
  [...]
  $ cd grml-debootstrap
  $ git branch -a
  * master
    remotes/origin/HEAD -> origin/master
    remotes/origin/master
  [...]

Now you have two choices: you can work on 'master' branch (reasonable if you want to do only single thing), or you can create your own branch (one branch for one independent unrelated feature). Let's create new branch off 'origin' == 'origin/master' == 'remotes/origin/master'

  $ git checkout -b t/my-working-branch origin
  Branch t/my-working-branch set up to track remote branch master from origin.
  Switched to a new branch 't/my-working-branch'

We could first setup git in such way, that it sets up to track upstream via rebasing rather than via merge (using `branch.autosetuprebase` config variable), but let's not complicate matters.

Now you work on branch

  [edit, edit, edit]
  [commit or commit --amend]
  [repeat as necessary]
  
Now, after some time you feel that your changes are ready for sending.
First, download any new changes, if any.

  $ git fetch

Then use interactive rebase to both update your changes to apply to most recent upstream code, and also reorder and edit your patches (e.g. fix typo in commit message, move fix commit and squash it with fixed commit, etc.)

  $ git rebase --interactive origin
  [...]
  Successfully rebased and updated refs/heads/t/my-working-branch.

Then you need to generate patches. If its more of them, I found it good idea to save them in separate subdirectory, but it might be not necessary for you

  $ git format-patch --cover-letter origin..

There are some situations where you don't need cover letter, e.g. if you are sending single patch, but they are quite useful especially with longer series.

  [edit '0000-cover-letter.patch' at least]
  [add comments to individual patches, if necessary]

Then you can either use your email client, or just

  $ git send-email --dry-run *.patch

(Here you see why saving patch series to separate subdirectory by using '-o' option makes it easy to pick up relevant patches... ;-))

HTH

On Tue, Jul 27, 2010 at 1:35 PM, Jakub Narebski <jnareb@gmail.com> wrote:

Yeah, that's actually exactly what I believed before getting feedbacks from grml developers for squashed patches. It's an interesting topic to me, so let's dig deeper into it.

Say that I need to add a feature to a CLI program. I would instinctively divide it into 3 logical steps/patches, 1st to the user interface (the command line handling), 2nd to the implementation, and 3rd to the document.

Do you think 3 small patches is the way to go, or a single patch is, since all 3 are logically related?

Now back to our topic, thanks for your work flow explanation, I'll answer/ask in this single message.

That's what I searched and found from the Inet when I was looking for the recommended work flow, which was to do 'git clone' from web once then 'git clone' several local working copies to work on several independent unrelated features. Now I know creating my own local branches is the way to go.

Could you elaborate more on this with git commands please, so that I can have a full picture?

Thanks again for your clearly explanations, I think I don't any further questions for the moment.

cheers,

On Thu, Jul 29, 2010, Tong Sun wrote:

In this situation it is obvious to me that you should send single squashed patch, as otherwise commit does not contain all related and connected changes: command line handling without implementation doesn't make sense, and if you are implementing something, you should document it.

An example of change that might, or might be not split into two commits is simple bug fix. You can either first write failing test showing the breakage, and then write fix and change test expectation to pass, or you can write fix and test in one commit.

Here are a few generalized examples from git repository history where you might want to send a change as a series of patches rather than in one large single patch:

1. If you are improving documentation, you can split your patch into
   one adding missing documentation for some feature, and second adding
   examples of using said feature.

2. If you plan to make some part of code used more widely, you can in
   first commit make API public, in second add support for feature to
   wider codebase, and in third add tests for this support.

   Similar thing with having doing refactoring first, then using this
   refactoring to easy add new feature.

3. If you are fixing some compiler warnings, fixing each class/type
   of warnings could be made into separate commits.

4. You can have one commit adding feature, and second adding support
   for said feature (for parameters / subcommands that use said feature)
   to shell completion.

Etc.

I think "fork (clone) to branch" was from some very ancient git tutorials (git has in-place branching and branch switching from time immemorial), or from some (outdated?) Mercurial documentation.

That said shallow clone should be improved, so you can clone from shallow clone with the same depth or shallower. Current implementation is a bit lacking.

I don't think that would apply in your situation, but "git remote add" is used if you want to fetch changes from more than one upstream repository (or you want to configure repository to push into). This is an alternative to one-shot "git pull <URL> <branch>" which does not save _any_ information about upstream you fetched (pulled) from.

See git-remote manpage for details.

On Wed, Jul 28, 2010 at 7:20 PM, Jakub Narebski <jnareb@gmail.com> wrote:

Got it.

and http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html as well.

thanks again. no more questions for now.

cheers