{"thread":{"id":"50368","subject":"New command/tool: git filter-repo","startedAt":"2019-01-31T08:57:44Z","lastAt":"2019-06-08T16:22:00Z","messageCount":13,"participants":["Elijah Newren","Junio C Hamano","Roberto Tyley","Johannes Schindelin","Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"368228","messageId":"CABPp-BFC--s+D0ijRkFCRxP5Lxfi+__YF4EdxkpO5z+GoNW7Gg@mail.gmail.com","threadId":"50368","inReplyTo":null,"subject":"New command/tool: git filter-repo","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-01-31T08:57:36Z","receivedAt":"2019-01-31T08:57:44Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi everyone,\n\ngit-filter-repo[1], a filter-branch-like tool for rewriting repository\nhistory, is ready for more widespread testing and feedback.  The rough\nedges I previously mentioned have been fixed, and it has several useful\nfeatures already, though more development work is ongoing (docs are a\nbit sparse right now, though -h provides some help).\n\nWhy filter-repo vs. filter-branch?\n\n  * filter-branch is extremely to unusably slow (multiple orders of\n    magnitude slower than it should be) for non-trivial repositories.\n\n  * filter-branch made a number of usability choices that are okay for\n    small repos, but these choices sometimes conflict as more options\n    are combined, and the overall usability often causes difficulties\n    for users trying to work with intermediate or larger repos.\n\n  * filter-branch is missing some basic features.\n\nThe first two are intrinsic to filter-branch's design at this point\nand cannot be backward-compatibly fixed.\n\nRequirements:\n  * Python 2 (for now?)\n  * A version of git with en/fast-export-import topic (in master of git.git)\n  * A version of git with the --combined-all-names option to diff-tree;\n    I have submitted[2] this patch, but it hasn't been picked up yet.\n\nWhat's the future?  (Core command of git.git?  place it in contrib?  keep it\nin a separate repo?)  I'm hoping to discuss that at the contributor summit\ntoday, but feedback on the list is also welcome.\n\n\nThanks,\nElijah\n\n[1] https://github.com/newren/git-filter-repo; it's a ~2800 line\n    single-file python script, depending only on the python standard\n    library (and execution of git commands), all of which is designed\n    to make build/installation trivial: you just need to add it to\n    your $PATH.\n[2] https://public-inbox.org/git/20190126221811.20241-1-newren@gmail.com/\n"},{"id":"368259","messageId":"xmqq36p83aq4.fsf@gitster-ct.c.googlers.com","threadId":"50368","inReplyTo":"CABPp-BFC--s+D0ijRkFCRxP5Lxfi+__YF4EdxkpO5z+GoNW7Gg@mail.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-01-31T19:09:39Z","receivedAt":"2019-01-31T19:09:44Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren <newren@gmail.com> writes:\n\n> git-filter-repo[1], a filter-branch-like tool for rewriting repository\n> history, is ready for more widespread testing and feedback.  The rough\n> edges I previously mentioned have been fixed, and it has several useful\n> features already, though more development work is ongoing (docs are a\n> bit sparse right now, though -h provides some help).\n>\n> Why filter-repo vs. filter-branch?\n\nHow does it compare with bfg-repo-cleaner?  Somehow I was led to\nbelieve that all serious users of filter-branch like functionality\nare using bfg-repo-cleaner instead.\n"},{"id":"368267","messageId":"CABPp-BH==w5APkz9cvUYq7m4qieJ3LWCsYySevgJuZ8bi2RzjQ@mail.gmail.com","threadId":"50368","inReplyTo":"xmqq36p83aq4.fsf@gitster-ct.c.googlers.com","subject":"Re: New command/tool: git filter-repo","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-01-31T20:43:42Z","receivedAt":"2019-01-31T20:44:03Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Jan 31, 2019 at 8:09 PM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Elijah Newren <newren@gmail.com> writes:\n>\n> > git-filter-repo[1], a filter-branch-like tool for rewriting repository\n> > history, is ready for more widespread testing and feedback.  The rough\n> > edges I previously mentioned have been fixed, and it has several useful\n> > features already, though more development work is ongoing (docs are a\n> > bit sparse right now, though -h provides some help).\n> >\n> > Why filter-repo vs. filter-branch?\n>\n> How does it compare with bfg-repo-cleaner?  Somehow I was led to\n> believe that all serious users of filter-branch like functionality\n> are using bfg-repo-cleaner instead.\n\nNo, bfg-repo-cleaner only covers an important subset of the usecases.\nbfg-repo-cleaner does a really good job if your goal is to remove a\nfew big files and/or to remove some sensitive text (matched via\nregexes) from all blobs.  It was designed for that specific role and\nhas more options in this area than filter-repo currently has.  But\neven within this design space it was optimized for, it is missing two\nthings that I really want:\n\n  * pruning of commits which become empty due to filtering\n  * providing a way for the user to know what needs to be cleaned up.\nIt has options like --strip-blobs-bigger-than <size> or\n--strip-biggest-blobs <NUM>, but no way for the user to figure out\nwhat <size> or <NUM> should be.  Also, since it just focuses on really\nbig blobs, it misses cases like someone checking in directories with a\nhuge number of small-to-moderately sized files (e.g. bower_components/\nor node_modules/, though these could also contain a few big blobs\ntoo), or someone checking in a lot of moderately sized files of a\nuniform extension (e.g. .webm, .tar.gz, .zip, .mp4, .avi).  I've seen\ncases in the wild where the correct cleaning of history was more about\nfiltering out directories or extensions than a couple big files.\nfilter-repo's --analyze option creates some reports that help with\nthis tremendously.  Also, the options to delete files by glob/basename\noverlook the fact that renames may have occurred.  Having a report\nthat mentions renames that have occurred in history (also part of\nfilter-repo's --analyze option) can be very helpful.\n\nOutside of this specific usecase, bfg-repo-cleaner is not very useful.\nIt simply lacks more general filtering capabilties:\n\n  * While bfg-repo-cleaner has facilities to remove certain paths, it\nhas none to say you only want to keep certain paths.  Unlike\nfilter-branch where you can use a pipeline to list all files, grep to\nremove the ones you want to keep from the list, then pipe the\nremainder of paths to xargs git rm, bfg-repo-cleaner doesn't have a\nfacility for shell commands.  Instead in bfg-repo-cleaner you would\nneed to emulate this by exhaustively listing directories and\npaths/globs of file basenames to delete, but that assumes the user\nknows all paths that have ever existed making this solution not only\nonerous but error prone.  More of the filterings I see these days are\nabout just keeping a directory (or perhaps a handful of them) rather\nthan just removing or cleaning a few files.  Also, this makes pruning\nof commits which become empty much more important, but as noted above,\nbfg-repo-cleaner lacks that ability.\n  * It has no facilities for renaming paths.  You'd have to use a\ndifferent tool to do that, but then why not use the other tool to do\nthe whole job?  Even if you do decide to use both tools, some\ncapabilities of one tool can be neutered by such an approach (e.g.\nbfg-repo-cleaner's carefully rewritten commit messages that tried to\nensure abbreviated commit shas referred to the new commit ids)\n  * It has no facilities for affecting other parts of history, such as\nchanging author/committer/tagger names or emails, changing commit\ntimestamp or timezone, reparenting commits, splicing repository\nhistories together, filtering files differently based on commit\ntimestamp, etc. -- all of which can be done with filter-repo (though\nsome of those things requires writing a small python script; see basic\nexamples in t/lib-usage/*)\n\nPersonally, I also find it kind of annoying that bfg-repo-cleaner\ndoesn't automatically repack and shrink the repo when it is done and\ninstead prints multiple commands the user can run to achieve that,\neven though it's the core use case for the tool.  Granted, they may\nhave had last-ditch recovery-of-the-original-repo in mind in case the\nuser ran in a repository they shouldn't have, but I much prefer to\nhave the tool just check if the repo looks like a fresh clone and bail\nif not, so that users have a far easier recovery mechanism -- just\nthrow away the clone you were filtering and re-clone.  Once you do\nthat, auto repacking and shrinking is pretty natural.  (And you can\nalways provide a --force option to allow filtering & rewriting in a\nrepo that isn't a fresh clone.)\n\n\nElijah\n"},{"id":"368269","messageId":"CABPp-BHOwfnpd=thMrGNQN0t+btN=jZhWHuhJ59HZ2oH5qDrBA@mail.gmail.com","threadId":"50368","inReplyTo":"CABPp-BFC--s+D0ijRkFCRxP5Lxfi+__YF4EdxkpO5z+GoNW7Gg@mail.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-01-31T20:47:59Z","receivedAt":"2019-01-31T20:48:13Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Jan 31, 2019 at 9:57 AM Elijah Newren <newren@gmail.com> wrote:\n>\n> Hi everyone,\n>\n> git-filter-repo[1], a filter-branch-like tool for rewriting repository\n> history, is ready for more widespread testing and feedback.\n...\n> What's the future?  (Core command of git.git?  place it in contrib?  keep it\n> in a separate repo?)  I'm hoping to discuss that at the contributor summit\n> today, but feedback on the list is also welcome.\n\nTurns out we didn't have enough time and didn't discuss it at the\ncontributor summit.  So, I'm even more interested in feedback from the\nmailing list.\n\nThanks,\nElijah\n"},{"id":"368283","messageId":"CAFY1edZ6hS5Dz9z5ZAhU59he9SjxetvfTN+ndzUZkjxhsuwEZA@mail.gmail.com","threadId":"50368","inReplyTo":"CABPp-BH==w5APkz9cvUYq7m4qieJ3LWCsYySevgJuZ8bi2RzjQ@mail.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Roberto Tyley","fromEmail":"roberto.tyley@gmail.com","sentAt":"2019-01-31T23:36:12Z","receivedAt":"2019-01-31T23:36:27Z","isPatch":false,"sender":{"key":"roberto.tyley@gmail.com","avatar":"https://avatars.githubusercontent.com/u/52038?v=4"},"body":"On Thu, 31 Jan 2019 at 22:37, Elijah Newren <newren@gmail.com> wrote:\n> On Thu, Jan 31, 2019 at 8:09 PM Junio C Hamano <gitster@pobox.com> wrote:\n> > Elijah Newren <newren@gmail.com> writes:\n> >\n> > > git-filter-repo[1], a filter-branch-like tool for rewriting repository\n> > > history, is ready for more widespread testing and feedback.  The rough\n> > > edges I previously mentioned have been fixed, and it has several useful\n> > > features already, though more development work is ongoing (docs are a\n> > > bit sparse right now, though -h provides some help).\n> > >\n> > > Why filter-repo vs. filter-branch?\n\nI like the name! I think a lot of users are interested in filtering\ntheir entire repo, rather than rewriting a single branch.\n\n> > How does it compare with bfg-repo-cleaner?  Somehow I was led to\n> > believe that all serious users of filter-branch like functionality\n> > are using bfg-repo-cleaner instead.\n>\n> No, bfg-repo-cleaner only covers an important subset of the usecases.\n\nThat's true - the focus with BFG Repo-Cleaner is on removing unwanted\ndata - completely eradicating it from a repo's history. There are some\nmistakes in history that repo owners just really *do not* want to\nshare (ie large files, private data/credentials), and they can be a\ncritical blocker to sharing or working with a Git repo. In terms of\nrewriting history, my internal criterion for what I features I really\nwant to be in the BFG is: is this unwanted data completely stopping\nmany users from sharing their code or doing their work?\n\nI understand that when it comes to rewriting history, there are loads\nof other operations that people sometimes want to perform, beyond\nremoving unwanted data - merging/splitting of history,\nanonymization/renaming of committers, etc. Some of those might be nice\nto add to the BFG - but as with many OSS-maintainers, I have limited\ntime, and a life to balance outside of software...!\n\n> bfg-repo-cleaner does a really good job if your goal is to remove a\n> few big files and/or to remove some sensitive text (matched via\n> regexes) from all blobs.  It was designed for that specific role and\n> has more options in this area than filter-repo currently has.  But\n> even within this design space it was optimized for, it is missing two\n> things that I really want:\n>\n>   * pruning of commits which become empty due to filtering\n\nThere certainly have been several users asking for this feature on the\nBFG, and even a kindly contributed PR for the functionality which I've\nyet to merge. As it doesn't actually stop users from doing work - so\nfar as I can see - it's something that I've done a poor job of\nfollowing up.\n\n>   * providing a way for the user to know what needs to be cleaned up.\n> It has options like --strip-blobs-bigger-than <size> or\n> --strip-biggest-blobs <NUM>, but no way for the user to figure out\n> what <size> or <NUM> should be.\n\nFor users of GitHub, It's normally 100MB with\n--strip-blobs-bigger-than <size> :-)\n\n> Also, since it just focuses on really\n> big blobs, it misses cases like someone checking in directories with a\n> huge number of small-to-moderately sized files (e.g. bower_components/\n> or node_modules/, though these could also contain a few big blobs\n\nFor those use-cases, it might be that BFG's --delete-folders flag is\nuseful, especially given the protected-head-commit feature of the BFG.\n\n\nIt's getting late for me, must be even later in Brussels - I wish I\ncould have made it there to join in! Merry Git Merge to you all, and\ngood luck to you Elijah with git-filter-repo.\n\nRoberto\n"},{"id":"368306","messageId":"CABPp-BFvKk9D9h-XMjAnOrziKDhfGDG-rX410kb-pt+jF5cinA@mail.gmail.com","threadId":"50368","inReplyTo":"CAFY1edZ6hS5Dz9z5ZAhU59he9SjxetvfTN+ndzUZkjxhsuwEZA@mail.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-02-01T07:38:40Z","receivedAt":"2019-02-01T07:38:57Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi Roberto,\n\nFirst of all, thanks for the feedback, and for the awesome work on BFG\nrepo filter!\n\nOn Fri, Feb 1, 2019 at 12:36 AM Roberto Tyley <roberto.tyley@gmail.com> wrote:\n>\n> On Thu, 31 Jan 2019 at 22:37, Elijah Newren <newren@gmail.com> wrote:\n> > On Thu, Jan 31, 2019 at 8:09 PM Junio C Hamano <gitster@pobox.com> wrote:\n> > > Elijah Newren <newren@gmail.com> writes:\n> > >\n> > > > git-filter-repo[1], a filter-branch-like tool for rewriting repository\n> > > > history, is ready for more widespread testing and feedback.  The rough\n> > > > edges I previously mentioned have been fixed, and it has several useful\n> > > > features already, though more development work is ongoing (docs are a\n> > > > bit sparse right now, though -h provides some help).\n> > > >\n> > > > Why filter-repo vs. filter-branch?\n>\n> I like the name! I think a lot of users are interested in filtering\n> their entire repo, rather than rewriting a single branch.\n>\n> > > How does it compare with bfg-repo-cleaner?  Somehow I was led to\n> > > believe that all serious users of filter-branch like functionality\n> > > are using bfg-repo-cleaner instead.\n> >\n> > No, bfg-repo-cleaner only covers an important subset of the usecases.\n>\n> That's true - the focus with BFG Repo-Cleaner is on removing unwanted\n> data - completely eradicating it from a repo's history. There are some\n> mistakes in history that repo owners just really *do not* want to\n> share (ie large files, private data/credentials), and they can be a\n> critical blocker to sharing or working with a Git repo. In terms of\n> rewriting history, my internal criterion for what I features I really\n> want to be in the BFG is: is this unwanted data completely stopping\n> many users from sharing their code or doing their work?\n>\n> I understand that when it comes to rewriting history, there are loads\n> of other operations that people sometimes want to perform, beyond\n> removing unwanted data - merging/splitting of history,\n> anonymization/renaming of committers, etc. Some of those might be nice\n> to add to the BFG - but as with many OSS-maintainers, I have limited\n> time, and a life to balance outside of software...!\n\nTotally understand; you picked a certain set of usecases and provided\na really good tool for those ones.  I focused more on repository\nmigration (not between different version control systems, but some\nother special flag-day event.  e.g. we have a whole bunch of plugins,\nseveral of them are unmaintained, they are in lots of different code\nhosting platforms; and someone decides we want to put all the widely\nused plugins into a big monorepo for various reasons.)  It's just that\nwhen doing some kind of big migration is a good time to do cleanup as\nwell, especially when typical actual plugin size history might be a\nfew hundred kilobytes, but for some reason many are dozens or hundreds\nof megabytes, so filter-repo also provides some facilities for\ncleaning unwanted stuff out as well.\n\n> > bfg-repo-cleaner does a really good job if your goal is to remove a\n> > few big files and/or to remove some sensitive text (matched via\n> > regexes) from all blobs.  It was designed for that specific role and\n> > has more options in this area than filter-repo currently has.  But\n> > even within this design space it was optimized for, it is missing two\n> > things that I really want:\n> >\n> >   * pruning of commits which become empty due to filtering\n>\n> There certainly have been several users asking for this feature on the\n> BFG, and even a kindly contributed PR for the functionality which I've\n> yet to merge. As it doesn't actually stop users from doing work - so\n> far as I can see - it's something that I've done a poor job of\n> following up.\n\nJust as a heads up: It may be a lot uglier than it at first looks;\nthere's all kinds of special cases and I'm not quite sure I've got it\nall right in filter-repo yet:\n\n* Some projects intentionally create empty commits for versioning or\npublishing or other reasons, and do not want these commits to be\nremoved.  Thus, it's important to remove commits that *become* empty\n(due to other filtering rules), not commits which started empty.\n* There's a special case for the above rule: if a user only wants the\nhistory of a certain directory, then any empty commits that pre-dated\nthat directory aren't wanted/needed.  The way I handle this is that if\na commit which had no changes relative to its parent became an orphan\n(its parent and all other ancestors were pruned due to becoming\nempty), then the fact that it was orphaned implies that it *became*\nempty and is thus prunable.\n* There are also topological changes possible: If a parent and all\nfurther ancestors on one side of history are pruned, a merge commit\nmay become a non-merge commit.  Often that will make the merge commit\nitself prunable, though there is the possibility that the merge had\nother changes tucked into it.\n* Merges also may become degenerate:\n  * Case 1: both sides of history may have commits pruned away (due to\nbecoming empty) all the way back to the merge base, meaning the merge\ncommit now has the merge base for both parents.  I think having\nredundant parents is senseless and the redundant ones should be\npruned.  In most cases, this should also mean the merge commit is no\nlonger empty as it'll likely have no changes relative to the merge\nbase.\n  * Case 2: one side of history may have commits pruned away back to\nthe merge base, meaning that the merge commit now merges one parent\nwith an ancestor of that parent.  If the merge commit has no changes\nrelative to the newer parent, then it could potentially be pruned\naway.  However: What if this merge commit *started* as a merge of some\ncommit with its own ancestor?  If the second parent is the newer\ncommit, this was likely a --no-ff merge and probably shouldn't be\npruned.  But, if a project has a strong policy of always doing --no-ff\nmerges to incorporate changes, then even if the merge commit didn't\nstart out looking like a --no-ff merge we should probably keep it even\nif it ends up looking like one.  (But, of course, in either case if\nthe first parent is the newer one, then it's not a --no-ff merge and\nunless the merge commit has extra file changes in it, we should be\nable to prune it.)\n* Also, if your scheme checks for changes against the first parent to\nsee what changes exist in a merge commit, if the first parent\nhistory is pruned away due to filtering choices (e.g. it didn't touch\nthe directory of interest), then your methodology may miss the fact\nthat this merge commit has an empty set of changes relative to its\nremaining parent and thus mistakenly retain some commits which became\nempty and were expected to be pruned.  I believe I saw filter-branch\nmessing this up, though I'd have to double check.\n* There might be other special cases I've overlooked, though I've\ntried to cover them all.\n\nI'm currently thinking that perhaps a flag or pair of flags might be\nuseful here (perhaps --empty-pruning={always,never,auto}\n--no-ff-pruning={always,never,auto}) with 'auto' perhaps being pruning\nof commits which didn't start out that way (empty or a no-ff merge)\nbut became so.  I'm still undecided on this...\n\n> >   * providing a way for the user to know what needs to be cleaned up.\n> > It has options like --strip-blobs-bigger-than <size> or\n> > --strip-biggest-blobs <NUM>, but no way for the user to figure out\n> > what <size> or <NUM> should be.\n>\n> For users of GitHub, It's normally 100MB with\n> --strip-blobs-bigger-than <size> :-)\n\nIf you're only interested in what GitHub won't allow for you to\ncontinue working with your repo, sure.  If you are migrating your repo\nfor some reason and want to take the chance to clean it up at the same\ntime so everyone doesn't have to pay heavy clone costs, there are many\nother interesting values.  :-)\n\n> > Also, since it just focuses on really\n> > big blobs, it misses cases like someone checking in directories with a\n> > huge number of small-to-moderately sized files (e.g. bower_components/\n> > or node_modules/, though these could also contain a few big blobs\n>\n> For those use-cases, it might be that BFG's --delete-folders flag is\n> useful, especially given the protected-head-commit feature of the BFG.\n\nAbsolutely.  However, this somewhat assumes the user doing the\nfiltering knows what parts of history are extraneous.  I've seen many\ncases where the original author(s) of some repo weren't that familiar\nwith version control and committed a whole bunch of things they\nshouldn't.  They then left the company or project and a few years\nlater, someone else deletes those files in a new commit, possibly even\nmentioning in the commit message that it was stuff that was never\nneeded.  Several more years go by, the second individual isn't with\nthe project any more either, and someone else comes along, possibly\nnot even familiar with the language or build tools the project uses\nbut was tasked with migrating the history anyway.  Having a tool which\ncan tell them why this small plugin happens to be much bigger than\nexpected with pointers to relevant bits (\"what's this\nbower_components/ directory that accounts for most the size and was\ndeleted four years ago?\") is very helpful.\n\nI've also seen this with big repos where there are lots of ugly\ntravesties in all kinds of different places, and no one still around\nknows about more than a couple of them.  The developers may be willing\nto go through a flag day to have the history rewritten to expunge the\nthings that never should have been committed to get clone times down,\nbut only removing a few big blobs or expecting there to be an\nindividual who knows which directories can be nuked isn't always the\nbest answer.  Given enough time, anyone can run enough git commands to\nkind of figure out what are the big things, but I wanted something\nthat made that job easier.\n\n\nGranted, 'git filter-repo --analyze' is a separate read-only step\n(other than the reports it writes), so even if people wanted to use\nBFG repo cleaner or other filtering tools they could use this aspect\nof filter-repo to guide decisions.\n\n> It's getting late for me, must be even later in Brussels - I wish I\n> could have made it there to join in! Merry Git Merge to you all, and\n> good luck to you Elijah with git-filter-repo.\n\nThanks again for all the feedback; maybe we'll get to meet at a future\nconference.  Best of luck to you as well.\n\nElijah\n"},{"id":"368766","messageId":"CABPp-BGOz8nks0+Tdw5GyGqxeYR-3FF6FT5JcgVqZDYVRQ6qog@mail.gmail.com","threadId":"50368","inReplyTo":"CABPp-BFC--s+D0ijRkFCRxP5Lxfi+__YF4EdxkpO5z+GoNW7Gg@mail.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-02-08T01:25:01Z","receivedAt":"2019-02-08T01:25:20Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nOn Thu, Jan 31, 2019 at 12:57 AM Elijah Newren <newren@gmail.com> wrote:\n> git-filter-repo[1], a filter-branch-like tool for rewriting repository\n> history, is ready for more widespread testing and feedback.  The rough\n\n\nSomeone at the Contributor Summit (Michael Haggerty perhaps?) asked me\nabout performance numbers on known repositories for filter-repo and\nhow it compared to other tools; I gave extremely rough estimates, but\nhere I belatedly provide some more detailed figures.  In each case, I\nreport both filtering time, and cleanup (gc or clone) time[0]:\n\n\nTestcase 1: Remove a single file (configure.ac) from each commit in git.git:\n\n  * filter-branch[1a]:  2413.978s + 34.812s\n  * BFG (8-core)[1b]:     38.743s + 30.333s\n  * BFG (40-core)[1b]:    24.680s + 35.165s\n  * filter-repo[1c]:      35.582s + 15.690s\n\n  Caveats: filter-repo failed and needed workarounds; see [1d]\n\nTestcase 2: Keep two directories (guides/ and tools/) from rails.git:\n\n  * filter-branch[2a]: 14586.655s + 22.726s\n  * BFG (8-core)[2b]:     27.675s + 15.786s\n  * BFG (40-core)[2b]:    24.883s + 20.463s\n  * filter-repo[2c]:      10.951s + 12.500s\n\n  Caveats: filter-branch failed at the end of this operation; see [2d].\n           AFAICT, BFG can't do this operation; used approximations instead[2e].\n\nTestcase 3: Replacing one string with another throughout all files in linux.git:\n\n  * filter-branch[3a]: Estimated at about 3.5 months (~8.9e6 seconds)\n  * BFG (8-core)[3b]:   2144.904s + 693.79s\n  * BFG (40-core)[3b]:  1178.577s + 636.887s\n  * filter-repo[3c]:    1203.147s + 159.620s\n\n  Caveats: filter-branch failed at ~12 hours; see [3d].\n\n\nOther details about measurements at [4].  Take-aways and biased\nopinions at [5].\n\n\nHope this was interesting,\nElijah\n\n\n\n*************** Footnotes (Minutiae for the curious) ***************\n\n[0] git-filter-branch's manpage suggests re-cloning to get rid of old objects,\n    BFG as its last step provides the user commands to execute in order to\n    clean out old objects, and filter-repo automatically runs such commands.\n    As such, time of post-run gc seems like a relevant thing to report.\n    Commands used and timed:\n\n  * filter-branch: time git clone file://$(pwd) ../nuke-me-clone\n  * BFG:           git reflog expire --expire=now --all && time git gc\n--prune=now\n  * filter-repo:   N/A (internally runs same commands as I manually ran for BFG)\n\n\n[1a] time git filter-branch --index-filter 'git rm --quiet --cached\n--ignore-unmatch configure.ac' --tag-name-filter cat --prune-empty --\n--all\n\n[1b] time java -jar ~/Downloads/bfg-1.13.0.jar --delete-files configure.ac\n\n[1c] git tag | grep v1.0rc | xargs git tag -d\n     git tag -d junio-gpg-pub\n     time git filter-repo --path configure.ac --invert-paths\n\n[1d] git fast-export when run with certain flags will abort in repos\n     with tags of blobs or tags of tags.  I had to first delete 7 tags\n     to get this testcase to run, as shown in the commands above in\n     [1c].  I'll probably patch fast-export to fix this.\n\n\n[2a] time git filter-branch --index-filter 'git ls-files -z | tr \"\\0\"\n\"\\n\" | grep -v -e ^guides/ -e ^tools/ | tr \"\\n\" \"\\0\" | xargs -0 git rm\n--quiet --cached --ignore-unmatch' --tag-name-filter cat --prune-empty\n-- --all\n\n[2b] git log --format=%n --name-only | sort | uniq | grep -v ^$ > all-files.txt\n     time java -jar ~/Downloads/bfg-1.13.0.jar --delete-folders\n\"{$(grep / all-files.txt | sed -e 's/\"//' -e s%/.*%% | uniq | grep -v\n-e guides -e tools | tr '\\n' ,)}\" --delete-files \"{$(comm -23 <(grep\n-v / all-files.txt) <(grep -e guides/ -e tools/ all-files.txt | sed -e\ns%.*/%% | sort) | tr '\\n' ,)}\"\n\n[2c] time git filter-repo --path guides --path tools\n\n[2d] filter-branch fails at the very end when noting which refs were\n     deleted/rewritten with:\n         error: cannot lock ref 'refs/tags/v0.10.0': is at\nb68b47672e613e94a7859c9549e9cd4b401f7b79 but expected\ne2724aa1856253f4fc48ddc251583042c5f06029\n         Could not delete refs/tags/v0.10.0\n     Turns out b68b47672e613e94a7859c9549e9cd4b401f7b79 is an\n     annotated tag in the original repo pointing to the commit\n     e2724aa1856253f4fc48ddc251583042c5f06029.  I do not know the\n     cause of this bug, but since it was almost at the very end, I\n     just reported the time used before it hit this error.\n\n[2e] Unless I am misunderstanding, BFG is not capable of this\n     filtering operation because it uses basenames for --delete-files\n     and --delete-folders, and some names appear in several\n     directories (e.g. .gitignore, Rakefile, tasks).  As such, with\n     the BFG you either have to delete files/directories that\n     shouldn't be, or leave files and folders around that you wanted\n     to have deleted.  The command in [2b] has some of both, but\n     should still give a good estimate of how long it would take BFG\n     to do this kind of operation if file and directory basenames in\n     the rails repository happened to be named uniquely.\n\n[3a] time git filter-branch -d /dev/shm/tmp --tree-filter 'git\nls-files | xargs sed -i s/secretly/covertly/' --tag-name-filter cat --\n--all\n\n[3b] time java -jar ~/Downloads/bfg-1.13.0.jar --replace-text <(echo\n'secretly==>covertly')\n\n[3c] time git filter-repo --replace-text <(echo 'secretly==>covertly')\n\n[3d] filter-branch failed after 45704 seconds, predicting another\n     8836429 seconds (~102 days) remaining at the time.  As commits\n     earlier in history tend to be smaller, filter-branch nearly\n     always underestimates the time required, sometimes considerably.\n     filter-branch failed on commit\n     af25e94d4dcfb9608846242fabdd4e6014e5c9f0 due to an empty ident.\n     I possibly could have worked around it with --env-filter, but\n     it's not like I'm going to wait for it to finish anyway.\n\n[4] Other notes about timings:\n  * All tests were run on an 8 cpu system, except for the \"BFG\n    40-core\" tests which were run on a 40 core system.  (filter-branch\n    and filter-repo are not multi-threaded and gain nothing from more\n    cores.)\n  * More precisely, I ran on AWS with an m4.2xlarge with two 50-GB GP2\n    volumes (150 Iops) for tests.  The 40-core system was an\n    m4.10xlarge.\n  * Before each command, to try to avoid warm disk caches helping or\n    hurting depending on the order I ran commands in, I first ran:\n    * rsync -az --delete ../$REPO-orig/ ./\n    * git status\n    * $TOOL -h\n  * Testing was imperfect; I just ran once and recorded the time.  It took\n    long enough to gather the data as it was.\n  * when additional commands were needed for the filtering\n    (e.g. getting the all-files.txt list to generate the BFG command,\n    or deleting tags that fast-export couldn't handle for\n    filter-repo), I did not include the times of those commands in the\n    overall execution time.  It would have added a few hundredths of a\n    second to filter-repo's git.git time, and about 5-6 seconds to BFG's\n    rails.git time.\n  * filter-repo self-reports time until filtering finishes and time\n    until entirely done.  I took difference between its self-report of\n    overall time and the \"time\" command's report of overall time (which\n    was typically order ~ 0.1s), and added that to filter-repo's\n    filtering time, assuming that most the discrepancy would be due to\n    python startup.\n\n[5] Performance is only one measurement.  Features, capabilities,\nusability, etc. matter too.  filter-branch is a general purpose\nfiltering tool, but in my opinion, not a good one -- and not just\nbecause of performance.  BFG Repo Cleaner is a good tool, but it is\nspecial purpose; it is designed for a few particular usecases\n(limiting the kinds of things I could try in my comparison above).  My\nhope is that filter-repo serves as a good general purpose filtering\ntool so that people can stop suffering from filter-branch.\n"},{"id":"368816","messageId":"nycvar.QRO.7.76.6.1902081119320.41@tvgsbejvaqbjf.bet","threadId":"50368","inReplyTo":"CABPp-BGOz8nks0+Tdw5GyGqxeYR-3FF6FT5JcgVqZDYVRQ6qog@mail.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-02-08T10:22:57Z","receivedAt":"2019-02-08T10:23:00Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Elijah,\n\nOn Thu, 7 Feb 2019, Elijah Newren wrote:\n\n> On Thu, Jan 31, 2019 at 12:57 AM Elijah Newren <newren@gmail.com> wrote:\n> > git-filter-repo[1], a filter-branch-like tool for rewriting repository\n> > history, is ready for more widespread testing and feedback.  The rough\n> \n> Someone at the Contributor Summit (Michael Haggerty perhaps?) asked me\n> about performance numbers on known repositories for filter-repo and\n> how it compared to other tools; I gave extremely rough estimates, but\n> here I belatedly provide some more detailed figures.  In each case, I\n> report both filtering time, and cleanup (gc or clone) time[0]:\n> \n> [...]\n\nThose are pretty good numbers right there.\n\n>  My hope is that filter-repo serves as a good general purpose filtering\n>  tool so that people can stop suffering from filter-branch.\n\nI agree. `git filter-branch` was simply pulled out of Cogito when that\nproject was declared discontinued, as the last nugget we could steal from\nthe carcass (we stole such a lot from Cogito that we're probably the\nreason it died).\n\nAnd when somebody started to rewrite it in C (calling it `git\nrewrite-commits` if memory serves), I asked to change it a little so it\ncould be used as a drop-in replacement for `filter-branch`, and\nunfortunately stopped that effort in its tracks.\n\nSo it is really good for me to see you picking up the initiative and make\nthat particular shell script obsolete.\n\nCiao,\nDscho\n\nP.S.: No, I am not willing to even attempt to run filter-branch on\nWindows.\n"},{"id":"368857","messageId":"875ztukt77.fsf@evledraar.gmail.com","threadId":"50368","inReplyTo":"CABPp-BFC--s+D0ijRkFCRxP5Lxfi+__YF4EdxkpO5z+GoNW7Gg@mail.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-02-08T18:53:32Z","receivedAt":"2019-02-08T18:53:36Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 31 2019, Elijah Newren wrote:\n\n> What's the future?  (Core command of git.git?  place it in contrib?  keep it\n> in a separate repo?)  I'm hoping to discuss that at the contributor summit\n> today, but feedback on the list is also welcome.\n\nSome of this I may have mentioned at the summit, but here for the list:\n\n* I think it should be a candidate for a core (not \"just contrib\")\n  git.git command, given that we have someone willing to maintain it &\n  deal with bugs etc. I'm not worried about that given the author.\n\n* It's unfortunate in terms of API we need to support going forward that\n  this obligates us to support a fairly intricate python API going\n  forward, so it's similar (but more detailed) to Git.pm (which I also\n  tried to get rid of as an external API a while ago).\n\n  However, as you correctly note that's the only way a command like this\n  can be really fast, we already have the \"no special API\" command with\n  git-filter-branch, and that's horribly slow.\n\n  But perhaps there's ways we can in advance deal with a potential\n  future breaking API change. E.g. some Pythonic way of versioning the\n  API, or just prominently documenting whatever (low?) stability\n  guarantees we're making.\n\n  I imagine if we need to make breaking changes in the future that'll\n  less big of a deal than in other cases, since we'd expect the API use\n  to be one-off migration scripts, although maybe it'll get used for\n  all-the-time exports (e.g. mirroring internal->external repos with\n  filtering).\n\n* The rest of our commands are hooked up to the i18n framework. I don't\n  think this should be a blocker, but it's worth thinking about what the\n  plan for this is.\n\n  Are we going to need the equivalent of Git::I18N for Python (which\n  presumably will be a run-time dependency on something needing the\n  Python API that links to gettext).\n\n  Or perhaps we could do the translated strings in C, by making the\n  program you're invoking be a C command, invoking the Python part as a\n  helper (which would need to re-invoke a helper if it prints its own\n  messages).\n\nThanks for working on this!\n"},{"id":"368875","messageId":"nycvar.QRO.7.76.6.1902082111320.41@tvgsbejvaqbjf.bet","threadId":"50368","inReplyTo":"875ztukt77.fsf@evledraar.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-02-08T20:13:26Z","receivedAt":"2019-02-08T20:13:28Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Fri, 8 Feb 2019, Ævar Arnfjörð Bjarmason wrote:\n\n> [...]\n>\n>   But perhaps there's ways we can in advance deal with a potential\n>   future breaking API change. E.g. some Pythonic way of versioning the\n>   API, or just prominently documenting whatever (low?) stability\n>   guarantees we're making.\n\nAnother thing to keep in mind: it being in Python prevents it from being\ndistributed with Git for Windows. The Git for Windows installer already\nweighs way more than it used to (it used to be under 30MB, now it is\n44MB), and I am simply not willing to increase the footprint dramatically\njust for one rarely used command.\n\nIf only it were written as a built-in...\n\nCiao,\nDscho"},{"id":"369013","messageId":"CABPp-BEPw0ZrK8o+T74uuA3q2hgpqiy+VDHo1x3zTtYzuH7fwA@mail.gmail.com","threadId":"50368","inReplyTo":"875ztukt77.fsf@evledraar.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-02-11T15:47:51Z","receivedAt":"2019-02-11T15:48:06Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi Ævar,\n\nOn Fri, Feb 8, 2019 at 10:53 AM Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n>\n> On Thu, Jan 31 2019, Elijah Newren wrote:\n>\n> > What's the future?  (Core command of git.git?  place it in contrib?  keep it\n> > in a separate repo?)  I'm hoping to discuss that at the contributor summit\n> > today, but feedback on the list is also welcome.\n>\n> Some of this I may have mentioned at the summit, but here for the list:\n>\n> * I think it should be a candidate for a core (not \"just contrib\")\n>   git.git command, given that we have someone willing to maintain it &\n>   deal with bugs etc. I'm not worried about that given the author.\n>\n> * It's unfortunate in terms of API we need to support going forward that\n>   this obligates us to support a fairly intricate python API going\n>   forward, so it's similar (but more detailed) to Git.pm (which I also\n>   tried to get rid of as an external API a while ago).\n>\n>   However, as you correctly note that's the only way a command like this\n>   can be really fast, we already have the \"no special API\" command with\n>   git-filter-branch, and that's horribly slow.\n>\n>   But perhaps there's ways we can in advance deal with a potential\n>   future breaking API change. E.g. some Pythonic way of versioning the\n>   API, or just prominently documenting whatever (low?) stability\n>   guarantees we're making.\n>\n>   I imagine if we need to make breaking changes in the future that'll\n>   less big of a deal than in other cases, since we'd expect the API use\n>   to be one-off migration scripts, although maybe it'll get used for\n>   all-the-time exports (e.g. mirroring internal->external repos with\n>   filtering).\n>\n> * The rest of our commands are hooked up to the i18n framework. I don't\n>   think this should be a blocker, but it's worth thinking about what the\n>   plan for this is.\n>\n>   Are we going to need the equivalent of Git::I18N for Python (which\n>   presumably will be a run-time dependency on something needing the\n>   Python API that links to gettext).\n>\n>   Or perhaps we could do the translated strings in C, by making the\n>   program you're invoking be a C command, invoking the Python part as a\n>   helper (which would need to re-invoke a helper if it prints its own\n>   messages).\n>\n> Thanks for working on this!\n\nGood points.  I'll dig in to the i18n story.  As you point out, the\nAPI stability may be tricky, but you may be right that we just need to\nprominently document whatever guarantee we want to make and that it's\ndesigned more for one-off migration scripts than continuing exports.\n"},{"id":"369014","messageId":"CABPp-BEhjkP4kUgT9unhbM5kzykEwcP1bnrhFKQ+N9Ci9ZY=eg@mail.gmail.com","threadId":"50368","inReplyTo":"nycvar.QRO.7.76.6.1902082111320.41@tvgsbejvaqbjf.bet","subject":"Re: New command/tool: git filter-repo","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-02-11T16:00:30Z","receivedAt":"2019-02-11T16:00:57Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi Dscho,\n\nOn Fri, Feb 8, 2019 at 12:13 PM Johannes Schindelin\n<Johannes.Schindelin@gmx.de> wrote:\n>\n> Hi Ævar,\n>\n> On Fri, 8 Feb 2019, Ævar Arnfjörð Bjarmason wrote:\n>\n> > [...]\n> >\n> >   But perhaps there's ways we can in advance deal with a potential\n> >   future breaking API change. E.g. some Pythonic way of versioning the\n> >   API, or just prominently documenting whatever (low?) stability\n> >   guarantees we're making.\n>\n> Another thing to keep in mind: it being in Python prevents it from being\n> distributed with Git for Windows. The Git for Windows installer already\n> weighs way more than it used to (it used to be under 30MB, now it is\n> 44MB), and I am simply not willing to increase the footprint dramatically\n> just for one rarely used command.\n\nThat would be unfortunate, though understandable.  I am curious,\nthough: do you include and does anyone use filter-branch on windows?\nYou mentioned elsewhere in this thread that you weren't even willing\nto attempt to run filter-branch there.  If people aren't using\nfilter-branch on windows, then there's nothing for me to save them\nfrom anyway.  If they are, I'm curious to hear more about the usecases\nand motivations, even if the cost of my tool is too high for you to\ninclude.\n\nAlso, since filter-branch and filter-repo are meant mostly as one-shot\nmigration tools, it is already not uncommon for people to do it on a\ndifferent machine (perhaps one with more RAM, or faster disks), and at\nmost one person on the team needs to run it (sometimes folks even look\nto an \"expert\" outside the team to run the migration for them).  Once\nmigrated, they push the results back and are done with the tool.\n\n> If only it were written as a built-in...\n\nA built-in would be great, IF it could provide all the same\ncapabilities and with at least the same speed.  However, making it a\nbuilt-in would fundamentally remove a significant chunk of its power\nand flexibility, which was part of the driving force for creating this\ntool.\n\nElijah\n"},{"id":"376862","messageId":"CABPp-BHqdKvsUDx40Tz-A3z32TN_KWcDENWb8zsaidxxf8SNCg@mail.gmail.com","threadId":"50368","inReplyTo":"875ztukt77.fsf@evledraar.gmail.com","subject":"Re: New command/tool: git filter-repo","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2019-06-08T16:20:42Z","receivedAt":"2019-06-08T16:22:00Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nNow that there's a released version of git that has all necessary\nflags and features[1] to run git filter-repo\n(https://github.com/newren/git-filter-repo), I thought I'd send an\nupdate...\n\nOn Fri, Feb 8, 2019 at 10:53 AM Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n> On Thu, Jan 31 2019, Elijah Newren wrote:\n>\n> > What's the future?  (Core command of git.git?  place it in contrib?  keep it\n> > in a separate repo?)  I'm hoping to discuss that at the contributor summit\n> > today, but feedback on the list is also welcome.\n>\n> Some of this I may have mentioned at the summit, but here for the list:\n>\n> * I think it should be a candidate for a core (not \"just contrib\")\n>   git.git command, given that we have someone willing to maintain it &\n>   deal with bugs etc. I'm not worried about that given the author.\n>\n> * It's unfortunate in terms of API we need to support going forward that\n>   this obligates us to support a fairly intricate python API going\n>   forward, so it's similar (but more detailed) to Git.pm (which I also\n>   tried to get rid of as an external API a while ago).\n>\n>   However, as you correctly note that's the only way a command like this\n>   can be really fast, we already have the \"no special API\" command with\n>   git-filter-branch, and that's horribly slow.\n>\n>   But perhaps there's ways we can in advance deal with a potential\n>   future breaking API change. E.g. some Pythonic way of versioning the\n>   API, or just prominently documenting whatever (low?) stability\n>   guarantees we're making.\n>\n>   I imagine if we need to make breaking changes in the future that'll\n>   less big of a deal than in other cases, since we'd expect the API use\n>   to be one-off migration scripts, although maybe it'll get used for\n>   all-the-time exports (e.g. mirroring internal->external repos with\n>   filtering).\n>\n> * The rest of our commands are hooked up to the i18n framework. I don't\n>   think this should be a blocker, but it's worth thinking about what the\n>   plan for this is.\n>\n>   Are we going to need the equivalent of Git::I18N for Python (which\n>   presumably will be a run-time dependency on something needing the\n>   Python API that links to gettext).\n>\n>   Or perhaps we could do the translated strings in C, by making the\n>   program you're invoking be a C command, invoking the Python part as a\n>   helper (which would need to re-invoke a helper if it prints its own\n>   messages).\n>\n> Thanks for working on this!\n\nI've implemented these, and several other things too.  Changes since last time:\n\n* Now i18n-ized\n* Several disclaimers about API backcompat (this is more of a one-shot\nconversion tool) [2]\n* Converted to Python3 (Python2 is EOL at EOY)\n* Pruning of become-empty and become-degenerate+empty commits has been\nfixed up (I mentioned this as a concern last time)\n* Testsuite has been fleshed out, including not only multiple small\nfixes to filter repo, but more fixes to git itself[3]\n* Usage, Examples, Internals, and Limitations documentation now exists\n(in README.md format and built-in -h help; no manpage yet)\n* Several new filters and abilities have been added\n\nNow that filter-repo is complete and more easily tested, what are\nfolks thoughts on incorporating it in git.git?  (And if we do go that\nroute, can we avoid losing its history so I can bisect issues if\nnecessary?)\n\nThanks,\nElijah\n\n\n[1] Well, except people will get an error if they use\n--preserve-commit-encoding saying they don't have a new enough git\nsince en/fast-export-encoding is still sitting in next and didn't make\nit in to git-2.22.  But I only know of one repo that even uses special\ncommit encodings, so I suspect few if any will even care about that\nflag.\n\n[2] The warnings appear in several places to try to make sure people\nnotice them; a not quite complete list:\nhttps://github.com/newren/git-filter-repo/blame/master/README.md#L620-L625\nhttps://github.com/newren/git-filter-repo/blame/master/README.md#L727-L730\nhttps://github.com/newren/git-filter-repo/blame/master/README.md#L989-L993\nhttps://github.com/newren/git-filter-repo/blob/master/git-filter-repo#L13-L30\nhttps://github.com/newren/git-filter-repo/blob/master/git-filter-repo#L1523,L1524\nhttps://github.com/newren/git-filter-repo/blob/master/t/t9391/commit_info.py#L4-L6\nhttps://github.com/newren/git-filter-repo/blob/master/t/t9391/create_fast_export_output.py#L4-L6\nhttps://github.com/newren/git-filter-repo/blob/master/t/t9391/file_filter.py#L4-L6\nhttps://github.com/newren/git-filter-repo/blob/master/t/t9391/splice_repos.py#L4-L6\nhttps://github.com/newren/git-filter-repo/blob/master/t/t9391/strip-cvs-keywords.py#L4-L6\n\n[3] https://github.com/newren/git-filter-repo/tree/develop#upstream-improvements\n"}]}