{"thread":{"id":"29890","subject":"Removing useless merge commit with \"filter-branch\"","startedAt":"2012-03-08T23:21:04Z","lastAt":"2012-03-29T18:26:47Z","messageCount":4,"participants":["Anatol Pomozov","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"186509","messageId":"CAOMFOmWMsXgepY0-ZWFymd9uHSUmbOk66r75qa-Kv5TWx_U=EA@mail.gmail.com","threadId":"29890","inReplyTo":null,"subject":"Removing useless merge commit with \"filter-branch\"","fromName":"Anatol Pomozov","fromEmail":"anatol.pomozov@gmail.com","sentAt":"2012-03-08T23:21:04Z","receivedAt":"2012-03-08T23:21:04Z","isPatch":false,"sender":{"key":"anatol.pomozov@gmail.com","avatar":"https://gravatar.com/avatar/71fc20093402ce987148294ec0999d025209e52da6762789ff248cf5c317645f?d=mp&s=160"},"body":"Hi,\n\nI have a large project (~100K commits) and I need to split a part of\nit into separate project. What I usually do in this case is\n\ngit filter-branch --prune-empty --index-filter 'git rm -rfq --cached\n--ignore-unmatch UNNEEDED_DIRECTORIES' HEAD\n\nthat works more or less fine for me.\n\nThe original project has a lot of merge commits (don't ask me why).\nBasically every non-merge commit is merged back to master branch\ninstead of rebasing on top of the master. In the command above I use\n--prune-empty parameter that removes empty commits, but not their\nmerge points. This leaves a lot of \"useless commit points\" like this:\n\n|\no      - merge commit that previously merged feature X\n|\\\n| \\\n|  \\\no  |   - real commit\n|   |\n|  /\n|/\n|\n\n\nAs of me such merge left-overs are completely useless and I would like\nto remove them. Actually this task can be split into 2 steps:\n1) Remove useless parents. A useless part is the one that points to a\ncommit that is *already* reachable by some other parent. This step\nconverts useless merge points to regular empty commits.\n2) run filter branch with --prune-empty that removes such empty commits.\n\n\nSo my questions are:\n1) What is the best way to remove \"useless parents\" as in the algorithm above?\n2) Should such behavior (remove useless parent/merge commits) be\nenabled when flag --prune-empty is used?\n"},{"id":"186510","messageId":"7v62eebri3.fsf@alter.siamese.dyndns.org","threadId":"29890","inReplyTo":"CAOMFOmWMsXgepY0-ZWFymd9uHSUmbOk66r75qa-Kv5TWx_U=EA@mail.gmail.com","subject":"Re: Removing useless merge commit with \"filter-branch\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-03-08T23:30:12Z","receivedAt":"2012-03-08T23:30:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Anatol Pomozov writes:\n\n> |\n> o - merge commit that previously merged feature X\n> |\\\n> | \\\n> | \\\n> o | - real commit\n> | |\n> | /\n> |/\n> |\n\nIt is unclear how many commits are drawn in the above picture and\nwhat \"feature X\" is about in the above picture. Care to redraw the\ncommit DAG to explain what you are trying to do a bit better?\n\nThe way I read it is that you start from a history like this (note\nthat when we draw an ascii art history we often write it sideways,\ntime flows from left to right):\n\n ---A-----B-----M---\n \\ /\n C-------D\n\nwhere a side branch to implement \"feature X\" that has C and D forked\nat A, and it was merged at M after somebody else committed B on the\nmainline. When you filtered out some parts of the tree, it turns\nout that C and D are totally unintereseting because their changes\ntouch parts outside of your interest, i.e. the history is:\n\n ---A-----B-----M---\n \\ /\n o-------o\n\nwhere 'o' are now no-op.\n\nIs that what you are talking about?\n\nI think \"log --simplify-merges A..M -- path\" may already has logic\nthat deals with this, so it may help if you study what it does and\nhow it does what it does.\n"},{"id":"186911","messageId":"CAOMFOmUT+2q7jC9Z1=zFJdBi_KAg=A66yHJNF2LRvjfQcgbdFw@mail.gmail.com","threadId":"29890","inReplyTo":"7v62eebri3.fsf@alter.siamese.dyndns.org","subject":"Re: Removing useless merge commit with \"filter-branch\"","fromName":"Anatol Pomozov","fromEmail":"anatol.pomozov@gmail.com","sentAt":"2012-03-13T22:27:42Z","receivedAt":"2012-03-13T22:27:42Z","isPatch":false,"sender":{"key":"anatol.pomozov@gmail.com","avatar":"https://gravatar.com/avatar/71fc20093402ce987148294ec0999d025209e52da6762789ff248cf5c317645f?d=mp&s=160"},"body":"Hi\n\nOn Thu, Mar 8, 2012 at 3:30 PM, Junio C Hamano wrote:\n> Anatol Pomozov writes:\n>\n>> |\n>> o      - merge commit that previously merged feature X\n>> |\\\n>> | \\\n>> |  \\\n>> o  |   - real commit\n>> |   |\n>> |  /\n>> |/\n>> |\n>\n> It is unclear how many commits are drawn in the above picture and\n> what \"feature X\" is about in the above picture.  Care to redraw the\n> commit DAG to explain what you are trying to do a bit better?\n>\n> The way I read it is that you start from a history like this (note\n> that when we draw an ascii art history we often write it sideways,\n> time flows from left to right):\n>\n>    ---A-----B-----M---\n>        \\         /\n>         C-------D\n>\n> where a side branch to implement \"feature X\" that has C and D forked\n> at A, and it was merged at M after somebody else committed B on the\n> mainline.  When you filtered out some parts of the tree, it turns\n> out that C and D are totally unintereseting because their changes\n> touch parts outside of your interest, i.e. the history is:\n>\n>    ---A-----B-----M---\n>        \\         /\n>         o-------o\n>\n> where 'o' are now no-op.\n>\n> Is that what you are talking about?\n\nYes, in fact --prune-empty flag removes empty commits so the history looks like\n\n-----A-------B-------M--------\n \\ /\n --------------\n\n\nSo M is a merge that has 2 parents A and B. I would like to remove\nthis merge M and leave the history as\n\n-----A-----B-----\n\nas only these commits have changes in my library that I am trying to extract.\n\nI think some trickery with \"git filter-branch --parent-filter\" should help here.\n\nFirst one runs filter-branch with --parent-filter and removes useless\nparents from merges (in this example with will be parent A---M), this\nconverts such merges to regular empty commits\n\nthen run filter-branch one more time with --prune-empty - it removes\nempty commits.\n>\n> I think \"log --simplify-merges A..M -- path\" may already has logic\n> that deals with this, so it may help if you study what it does and\n> how it does what it does.\n"},{"id":"188088","messageId":"CAOMFOmWrG9WGZCF6j3P=Somo1DDxoB=ALCiYYbjO__8=uxYokg@mail.gmail.com","threadId":"29890","inReplyTo":"CAOMFOmUT+2q7jC9Z1=zFJdBi_KAg=A66yHJNF2LRvjfQcgbdFw@mail.gmail.com","subject":"Re: Removing useless merge commit with \"filter-branch\"","fromName":"Anatol Pomozov","fromEmail":"anatol.pomozov@gmail.com","sentAt":"2012-03-29T18:26:47Z","receivedAt":"2012-03-29T18:26:47Z","isPatch":false,"sender":{"key":"anatol.pomozov@gmail.com","avatar":"https://gravatar.com/avatar/71fc20093402ce987148294ec0999d025209e52da6762789ff248cf5c317645f?d=mp&s=160"},"body":"Hi,\n\nI solved my issue by using \"git filter-branch --parent-filter\". The\nidea is to visit all commits and remove all \"dependent\" parents. To\nfind independent parents I used \"git show-branch --independent\nPARENT_1..PARENT_N\" command. In my case (we have a lot of short-term\ndevelopment branches) this converted ~80% of all merges to \"empty\nnon-merge commits\" and later \"git filter-branch --prune-empty\" removed\nthem. This made history of my project much more simpler and linear.\n\nI think such functionality should be available in git and enabled when\n\"--prune-empty\" flag is used. So \"--prune-empty\" removes not only\nsimple commits but also empty useless merge commits. Or maybe add a\n--prune-empty-merges flag?\n\nAnyway here is the script that I use, future readers might find it useful:\n\n$ git filter-branch -f --prune-empty --parent-filter\nPATH_TO/rewrite_parent.rb master\n$ cat rewrite_parent.rb\n#!/usr/bin/ruby\nold_parents = gets.chomp.gsub('-p ', ' ')\n\nif old_parents.empty? then\n new_parents = []\nelse\n new_parents = `git show-branch --independent #{old_parents}`.split\nend\n\nputs new_parents.map{|p| '-p ' + p}.join(' ')\n\nMost likely the script can be rewritten as one-line shell script.\n\nOn Tue, Mar 13, 2012 at 3:27 PM, Anatol Pomozov\n wrote:\n> Hi\n>\n> On Thu, Mar 8, 2012 at 3:30 PM, Junio C Hamano wrote:\n>> Anatol Pomozov writes:\n>>\n>>> |\n>>> o      - merge commit that previously merged feature X\n>>> |\\\n>>> | \\\n>>> |  \\\n>>> o  |   - real commit\n>>> |   |\n>>> |  /\n>>> |/\n>>> |\n>>\n>> It is unclear how many commits are drawn in the above picture and\n>> what \"feature X\" is about in the above picture.  Care to redraw the\n>> commit DAG to explain what you are trying to do a bit better?\n>>\n>> The way I read it is that you start from a history like this (note\n>> that when we draw an ascii art history we often write it sideways,\n>> time flows from left to right):\n>>\n>>    ---A-----B-----M---\n>>        \\         /\n>>         C-------D\n>>\n>> where a side branch to implement \"feature X\" that has C and D forked\n>> at A, and it was merged at M after somebody else committed B on the\n>> mainline.  When you filtered out some parts of the tree, it turns\n>> out that C and D are totally unintereseting because their changes\n>> touch parts outside of your interest, i.e. the history is:\n>>\n>>    ---A-----B-----M---\n>>        \\         /\n>>         o-------o\n>>\n>> where 'o' are now no-op.\n>>\n>> Is that what you are talking about?\n>\n> Yes, in fact --prune-empty flag removes empty commits so the history looks like\n>\n> -----A-------B-------M--------\n>       \\               /\n>        --------------\n>\n>\n> So M is a merge that has 2 parents A and B. I would like to remove\n> this merge M and leave the history as\n>\n> -----A-----B-----\n>\n> as only these commits have changes in my library that I am trying to extract.\n>\n> I think some trickery with \"git filter-branch --parent-filter\" should help here.\n>\n> First one runs filter-branch with --parent-filter and removes useless\n> parents from merges (in this example with will be parent A---M), this\n> converts such merges to regular empty commits\n>\n> then run filter-branch one more time with --prune-empty - it removes\n> empty commits.\n>>\n>> I think \"log --simplify-merges A..M -- path\" may already has logic\n>> that deals with this, so it may help if you study what it does and\n>> how it does what it does.\n"}]}