git/list[1] front-page[2] threads[3] people[4] search[5] about
 

History surgery with fast-import (Re: Summer of Code project ideas due this Friday)

From
Jonathan Nieder <jrnieder@gmail.com>
Date
Mar 12, 2011, 00:20 UTC
Message-ID
<20110312002017.GA16081@elie>
In-Reply-To
<m339mu7u6n.fsf@localhost.localdomain>
Jakub Narebski wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
Show 7 quoted lines
>> 4. filter-branch killer: using fast-import's new features to implement
>>    common filter-branch operations (--subdirectory-filter,
>>    --prune-empty, obliterating certain files) faster.
>
> How it would be different from existing reposurgeon tool by ESR
> (cross-VC thanks to using fast-import format), or git_fast_filter by
> Elijah Newren (more of a library than a ready tool)?
Good question.
. reposurgeon loads the whole history into a convenient format,
  manipulates it, then dumps it as a separate step.  So it is even more
  flexible than filter-branch, but with a large history and especially
  on a small machine it can be slower.
. git_fast_filter is what Thomas mentioned --- a filter to go between
  fast-export and fast-import.

Both involve unpacking all trees. Which is not a huge expense, mind you, but there is room to go faster.

What I was suggesting is instead a tool that relies on fast-import to do the heavy lifting. So, for example, when you ask to delete path/to/remove.txt from all commits, it would write:

 commit refs/heads/master
 mark :172
 author A U Thor <author@example.com> ...
 committer ...
 data ...
 from ...
 merge ...
 M 040000 [old tree name for that commit] ""
 D path/to/remove.txt
 commit refs/heads/mater
 mark :173
 ...

The idea is inspired by http://thread.gmane.org/gmane.comp.version-control.git/158375 I am interested in it because I imagine something like this could be useful for splitting out branches from a naive import of an svn repo (and similar surgeries).

Previous: Thomas RastNext: Ramkumar Ramachandra
Message 28 of 63 in “Google Summer of Code 2011”
  1. Shawn PearceMar 3, 2011
  2. Jeff KingMar 3, 2011
  3. Shawn PearceMar 3, 2011
  4. Jeff KingMar 3, 2011
  5. Jakub NarebskiMar 3, 2011
  6. Jeff KingMar 9, 2011
  7. Jeff KingMar 9, 2011
  8. Shawn PearceMar 9, 2011
  9. Jeff KingMar 9, 2011
  10. Shawn PearceMar 9, 2011
  11. Summer of Code project ideas due this FridayJeff King, Mar 9, 2011
  12. Jonathan NiederMar 10, 2011
  13. Jeff KingMar 10, 2011
  14. Shawn PearceMar 10, 2011
  15. Alexander MiselerMar 10, 2011
  16. Thomas RastMar 10, 2011
  17. Santi BéjarMar 10, 2011
  18. Jeff KingMar 10, 2011
  19. Junio C HamanoMar 10, 2011
  20. Jeff KingMar 10, 2011
  21. Junio C HamanoMar 10, 2011
  22. Jeff KingMar 10, 2011
  23. Junio C HamanoMar 10, 2011
  24. Jeff KingMar 10, 2011
  25. Thomas RastMar 11, 2011
  26. Jakub NarebskiMar 10, 2011
  27. Thomas RastMar 11, 2011
  28. History surgery with fast-import (Re: Summer of Code project ideas due this Friday)Jonathan Nieder, Mar 12, 2011
  29. Ramkumar RamachandraMar 13, 2011
  30. Nguyen Thai Ngoc DuyMar 10, 2011
  31. Jeff KingMar 10, 2011
  32. Alexander MiselerMar 10, 2011
  33. Jeff KingMar 10, 2011
  34. Alexander MiselerMar 11, 2011
  35. Alexander MiselerMar 12, 2011
  36. Alexander MiselerMar 11, 2011
  37. Ilari LiusvaaraMar 11, 2011
  38. Nguyen Thai Ngoc DuyMar 11, 2011
  39. Alexander MiselerMar 11, 2011
  40. Nguyen Thai Ngoc DuyMar 11, 2011
  41. Sam VilainMar 11, 2011
  42. Alexander MiselerMar 12, 2011
  43. Ævar Arnfjörð BjarmasonMar 11, 2011
  44. code.sculptor@gmail.comMar 11, 2011
  45. Jakub NarebskiMar 17, 2011
  46. Heiko VoigtMar 22, 2011
  47. J.H.Mar 22, 2011
  48. Pat ThoytsMar 25, 2011
  49. Jakub NarebskiMar 25, 2011
  50. Ramkumar RamachandraMar 3, 2011
  51. Jonathan NiederMar 3, 2011
  52. Sverre RabbelierMar 7, 2011
  53. Ramkumar RamachandraMar 8, 2011
  54. Sverre RabbelierMar 8, 2011
  55. Jens LehmannMar 3, 2011
  56. Christian CouderMar 5, 2011
  57. Sam VilainMar 6, 2011
  58. Heiko VoigtMar 7, 2011
  59. Fredrik GustafssonMar 7, 2011
  60. Heiko VoigtMar 9, 2011
  61. Fredrik GustafssonMar 9, 2011
  62. Heiko VoigtMar 10, 2011
  63. Thomas RastMar 9, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.