git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH/RFC 0/4] Add option to enable filters in git-svn

From
Matteo Bertini <naufraghi@develer.com>
Date
May 31, 2016, 15:07 UTC
Message-ID
<20160531150749.24840-1-naufraghi@develer.com>

=========== Description ===========

This is a RFC for a patch that allows to enable filters in the repositories created/managed by git-svn.

================== Example scenario ==================

The usage scenario I’m facing is the migration of a large SVN repository to a git repository, with big files being transparently handled through git-lfs

To recap, these migrations are usually handled through a transitioning repo created by git-svn, which is then pushed to a remote git server; this server acts as a read-only mirror for a while, until the main SVN server is shut down. For large repos, the process can take several weeks to adjust the migration scripts, branch mapping, authorship, etc., and it’s also common to update the mirror incrementally.

Without this patch ==================

Without this patch, the migration needs to be setup with a pipeline with:
* incremental git svn fetch,
* bfg-repo-cleaner with —convert-to-git-lfs option (which puts invalid
  .gitattributes files, see https://github.com/rtyley/bfg-repo-cleaner/issues/143)
* git-filter-branch (to put in place the correct .gitattributes files)

A minimal change in the pipeline results in unrelated branches after the rebases.

Each deviation from the svn sources must be committed back or saved as patch and reapplied after the rebase.

With this patch ===============

Everything is much simpler: filters can be triggered during the fetch, so that git-lfs already runs while the repository is fetched. At the end of the process, the local git repository is already in its final form, and no further git-filter-branch/rebases is required.

The transitioning git repo can diverge partially from the svn source thanks to git svn rebase.

Details =======

In the current implementation git-svn uses the hash-object command with the --no-filters option. There is no gain in removing that option because the git-svn uses temporary file names.

In this patch hash-object accepts a new format for the streaming input, files and paths, tab separated. git-svn can provide the real path to hash-object and thanks to that we can enable the filters.

Status ======

The patch is in RFC, no new test where written (yet), just runned for regressions (https://travis-ci.org/naufraghi/git/builds/134170350).

I'd like to gain some feedback before spending more time on the feature, mainly:

 * feature comments,
 * coding comments (neither Perl or C are my main languages),
 * user interface (option names, config placements) comments.

Best, Matteo

Matteo Bertini (4):
  hash-object.c: Allow distinct file/path in stdin mode too.
  Git.pm: Add $path and $enable_filters arguments to
    hash_and_insert_object.
  SVN/Fetcher.pm: Add svn-remote.<id>.enable-filters to enable the
    filters.
  git-svn.perl: Add git svn init --enable-filters option.
 builtin/hash-object.c   | 29 +++++++++++++++++++++++++++--
 git-svn.perl            |  4 ++++
 perl/Git.pm             | 19 +++++++++++++------
 perl/Git/SVN/Fetcher.pm | 16 ++++++++++++----
 4 files changed, 56 insertions(+), 12 deletions(-)
-- 
2.9.0.rc0.39.gb9f310b.dirty
Next: Matteo Bertini
Message 1 of 9 in “Add option to enable filters in git-svn”
  1. 0/4 Add option to enable filters in git-svnMatteo Bertini, May 31, 2016
  2. 1/4 hash-object.c: Allow distinct file/path in stdin mode too.Matteo Bertini, May 31, 2016
  3. 2/4 Git.pm: Add $path and $enable_filters arguments to hash_and_insert_object.Matteo Bertini, May 31, 2016
  4. 3/4 SVN/Fetcher.pm: Add svn-remote.<id>.enable-filters to enable the filters.Matteo Bertini, May 31, 2016
  5. 4/4 git-svn.perl: Add git svn init --enable-filters option.Matteo Bertini, May 31, 2016
  6. Matteo BertiniMay 31, 2016
  7. Eric WongMay 31, 2016
  8. Matteo BertiniMay 31, 2016
  9. Eric WongJun 2, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.