threads / discuss / 37010
Subject: Passing rev-list options in git-filter-branch broken
Hey.
When trying to rewrite svn revisions in commit messages to corresponding git commit hashes, I came across the following problem (using git 1.9.1):
$ git filter-branch --msg-filter svnrev2git.py -- --date-order --all fatal: options not supported in --stdin mode Could not get the commits
This seems to have been caused by 3361a548db. After reverting this commit, using --date-order appears to work again.
Regards felix
Junio C HamanoRe: Passing rev-list options in git-filter-branch broken
Felix Eckhofer <felix@tribut.de> writes:
> When trying to rewrite svn revisions in commit messages to > corresponding git commit hashes, I came across the following problem > (using git 1.9.1): > > $ git filter-branch --msg-filter svnrev2git.py -- --date-order --all > fatal: options not supported in --stdin mode > Could not get the commits > > This seems to have been caused by 3361a548db. After reverting this > commit, using --date-order appears to work again.
Hmph, unfortunate.
3361a548 (Allow git-filter-branch to process large repositories with lots of branches., 2013-09-10) has this change:
-rev_args=$(git rev-parse --revs-only "$@")
+git rev-parse --revs-only "$@" >../parseand then later feeds ../parse from the standard input of rev-list.
The --revs-only option, because --date-order *is* a rev-list related argument, is emitted by the above rev-parse, along with the tip of refs (which come from --all). But --stdin mode of rev-list is meant to receive *only* revisions, not options. When it gets to the point to accept the list of tips to start traversing from, it is too late to give it an option.
Changing the above to something like:
git rev-parse --revs-only --no-flags "$@" >../parse
would be a better workaround that would not break repositories with large number of references, but it obviously will lose --date-order option (why would it be even necessary, though? I suspect that forcing the date-order will make the resulting pack a lot worse by robbing the data stream of locality).
Junio C HamanoRe: Passing rev-list options in git-filter-branch broken
Junio C Hamano <gitster@pobox.com> writes:
> Felix Eckhofer <felix@tribut.de> writes: > >> This seems to have been caused by 3361a548db. After reverting this >> commit, using --date-order appears to work again. > > Hmph, unfortunate. > > 3361a548 (Allow git-filter-branch to process large repositories with > lots of branches., 2013-09-10) has this change: > > > -rev_args=$(git rev-parse --revs-only "$@") > +git rev-parse --revs-only "$@" >../parse > > and then later feeds ../parse from the standard input of rev-list. > > The --revs-only option, because --date-order *is* a rev-list related > argument, is emitted by the above rev-parse, along with the tip of > refs (which come from --all). But --stdin mode of rev-list is meant > to receive *only* revisions, not options. When it gets to the point > to accept the list of tips to start traversing from, it is too late > to give it an option. > > Changing the above to something like: > > git rev-parse --revs-only --no-flags "$@" >../parse > > would be a better workaround that would not break repositories with > large number of references, but it obviously will lose --date-order > option (why would it be even necessary, though? I suspect that > forcing the date-order will make the resulting pack a lot worse by > robbing the data stream of locality).
As the original "../parse" file will have mixture of revs (i.e. the object names) and whatever rev-list options (e.g. "--date-order") you gave from the command line, here is a band-aid to sift them into two, to be applied on top of 3361a548 (Allow git-filter-branch to process large repositories with lots of branches., 2013-09-10).
I wish there was a way to tell rev-parse "give us only flags relevant to rev-list, don't include revs in the output", but I do not think there was ever any such option (we can get revs without any flags with --no-flags, though).
git-filter-branch.sh | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/git-filter-branch.sh b/git-filter-branch.sh index ca3d539..ed55c01 100755 --- a/git-filter-branch.sh +++ b/git-filter-branch.sh @@ -255,7 +255,9 @@ else remap_to_ancestor=t fi -git rev-parse --revs-only "$@" >../parse +git rev-parse --revs-only "$@" >../parse.revs.flags +sed -ne '/^[0-9a-f][0-9a-f][0-9a-f]*[0-9a-f]$/p' <../parse.revs.flags >../parse.revs +sed -e '/^[0-9a-f][0-9a-f][0-9a-f]*[0-9a-f]$/d' <../parse.revs.flags >../parse.flags case "$filter_subdir" in "") @@ -268,7 +270,8 @@ case "$filter_subdir" in esac git rev-list --reverse --topo-order --default HEAD \ - --parents --simplify-merges --stdin "$@" <../parse >../revs || + --parents --simplify-merges --stdin "$@" \ + $(cat ../parse.flags) <../parse.revs >../revs || die "Could not get the commits" commits=$(wc -l <../revs | tr -d " ")
Re: Passing rev-list options in git-filter-branch broken
Junio,
thanks for your reply and your patch.
Am 27.06.2014 20:31, schrieb Junio C Hamano:
> [...] > would be a better workaround that would not break repositories with > large number of references, but it obviously will lose --date-order > option (why would it be even necessary, though? I suspect that > forcing the date-order will make the resulting pack a lot worse by > robbing the data stream of locality).
We're migrating some svn repositories to git. The use case is to replace mentions of svn revisions in commit messages by the corresponding commit hash. Therefore, the commits must be ordered by svn revision number, which is not guaranteed for commits on different branches with --topo-order.
Regards felix