{"thread":{"id":"25039","subject":"Why does git-svn redownload revisions?","startedAt":"2010-09-08T16:13:33Z","lastAt":"2010-09-08T23:50:44Z","messageCount":4,"participants":["Daniel Trebbien","Brian Gernhardt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"150308","messageId":"loom.20100908T181056-819@post.gmane.org","threadId":"25039","inReplyTo":null,"subject":"Why does git-svn redownload revisions?","fromName":"Daniel Trebbien","fromEmail":"dtrebbien@gmail.com","sentAt":"2010-09-08T16:13:33Z","receivedAt":"2010-09-08T16:13:33Z","isPatch":false,"sender":{"key":"dtrebbien@gmail.com","avatar":null},"body":"For my first time using git-svn, I decided that I wanted to convert the GNU Nano\nSubversion repository to a git repo. I finally settled on the following series\nof commands after some initial trial-and-error (e.g. having the wrong\n`i18n.commitencoding` and not specifying a Subversion authors file):\ngit svn init -s svn://svn.sv.gnu.org/nano\ngit config svn.authorsfile ~/projects/nano/svn.authorsfile\ngit config i18n.commitencoding 'ISO-8859-1'\ngit svn fetch\n\nThe issue is that the fetch runs up to revision 4250, but then mysteriously\nbegins redownloading revisions 1 up to four thousand something in importing\nthe `nano_2_1_1` tag:\n...\nr4248 = 7c444dd667b9629ce92a53e8d35ad2d178e5735f (refs/remotes/trunk)\n\tM\tnano/ChangeLog\n\tM\tnano/configure.ac\n\tM\tnano/po/cs.po\n\tM\tnano/po/pt_BR.po\n\tM\tnano/po/es.po\n\tM\tnano/po/eu.po\n\tM\tnano/po/hu.po\n\tM\tnano/po/vi.po\n\tM\tnano/po/nano.pot\n\tM\tnano/po/ms.po\n\tM\tnano/po/uk.po\n\tM\tnano/po/ro.po\n\tM\tnano/po/ru.po\n\tM\tnano/po/rw.po\n\tM\tnano/po/id.po\n\tM\tnano/po/nb.po\n\tM\tnano/po/gl.po\n\tM\tnano/po/fr.po\n\tM\tnano/po/nl.po\n\tM\tnano/po/nn.po\n\tM\tnano/po/pl.po\n\tM\tnano/po/it.po\n\tM\tnano/po/ca.po\n\tM\tnano/po/da.po\n\tM\tnano/po/sr.po\n\tM\tnano/po/tr.po\n\tM\tnano/po/ga.po\n\tM\tnano/po/bg.po\n\tM\tnano/po/sv.po\n\tM\tnano/po/de.po\n\tM\tnano/po/zh_TW.po\n\tM\tnano/po/fi.po\n\tM\tnano/po/zh_CN.po\nr4249 = 6fbad8a00fa067d1e3de913f77db08c6117843c7 (refs/remotes/trunk)\n\tM\tnano/NEWS\nr4250 = 704700855e5112d75654e3e7461e896f49e10fd8 (refs/remotes/trunk)\nFound possible branch point: svn://svn.sv.gnu.org/nano/trunk/nano =>\nsvn://svn.sv.gnu.org/nano/tags/nano_2_1_1, 4248\nInitializing parent: refs/remotes/tags/nano_2_1_1@4248\n\tA\tmkinstalldirs\n\tA\tutils.c\n\tA\tnano.h\n\tA\tglobal.c\n\tA\tconfigure\n\tA\tMakefile.in\n\tA\tAUTHORS\n\tA\tconfigure.in\n\tA\tChangeLog\n\tA\tproto.h\n\tA\tnano.1\n\tA\tnano.1.html\n\tA\tREADME\n\tA\tacconfig.h\n\tA\tBUGS\n\tA\tconfig.h.in\n\tA\tABOUT-NLS\n\tA\tTODO\n\tA\tINSTALL\n\tA\tintl/po2tbl.sed.in\n\tA\tintl/loadinfo.h\n\tA\tintl/Makefile.in\n\tA\tintl/explodename.c\n\tA\tintl/VERSION\n\tA\tintl/xopen-msg.sed\n\tA\tintl/ChangeLog\n\tA\tintl/finddomain.c\n\tA\tintl/localealias.c\n\tA\tintl/gettextP.h\n\tA\tintl/textdomain.c\n\tA\tintl/linux-msg.sed\n\tA\tintl/l10nflist.c\n\tA\tintl/loadmsgcat.c\n\tA\tintl/libgettext.h\n\tA\tintl/bindtextdom.c\n\tA\tintl/gettext.c\n\tA\tintl/intl-compat.c\n\tA\tintl/dgettext.c\n\tA\tintl/cat-compat.c\n\tA\tintl/gettext.h\n\tA\tintl/dcgettext.c\n\tA\tintl/hash-string.h\n\tA\twinio.c\n\tA\tCOPYING\n\tA\tMakefile.am\n\tA\tmissing\n\tA\tNEWS\n\tA\tcut.c\n\tA\tnano.c\n\tA\taclocal.m4\n\tA\tinstall-sh\n\tA\tpo/cat-id-tbl.c\n\tA\tpo/stamp-cat-id\n\tA\tpo/es.po\n\tA\tpo/fr.po\n\tA\tpo/de.po\n\tA\tpo/ChangeLog\n\tA\tpo/Makefile.in.in\n\tA\tpo/es.gmo\n\tA\tpo/fr.gmo\n\tA\tpo/de.gmo\n\tA\tpo/it.po\n\tA\tpo/POTFILES.in\n\tA\tpo/nano.pot\n\tA\tpo/it.gmo\n\tA\tstamp-h.in\nr2 = 842a208235bd2a1181250766996f7797e74a8608\n(refs/remotes/tags/nano_2_1_1@4248)\n\tM\twinio.c\nr6 = 280cfdcb8533ffcb424b3a86b7392ccfbc054e60\n(refs/remotes/tags/nano_2_1_1@4248)\n\tM\tChangeLog\nr7 = 6951e65102dd1e005761dec6eb082f0a2d9327a2\n(refs/remotes/tags/nano_2_1_1@4248)\n\tM\tAUTHORS\n...\n\n\nWhy is git-svn redownloading revision history? Also, why hasn't git-svn\nredownloaded revisions to import other tags such as `nano_1_3_9`, `nano_2_0_0`,\nand `nano_0_9_14`? I have yet to complete the fetch, but git-svn finished\nredownloading revisions for `nano_2_1_1`, seemed to skip redownloading for\n`nano_2_1_2`, and is now redownloading for `nano_2_1_3`.\n"},{"id":"150310","messageId":"B1E94440-DDCA-4C10-A0EE-E08A66DF3D7E@silverinsanity.com","threadId":"25039","inReplyTo":"loom.20100908T181056-819@post.gmane.org","subject":"Re: Why does git-svn redownload revisions?","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2010-09-08T17:19:16Z","receivedAt":"2010-09-08T17:19:16Z","isPatch":false,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"This happens when you have a tag or branch that is a subdirectory of trunk.  It looks like tags/nano_2_1_1 is a branch off trunk/nano.  Git-svn doesn't recognize that trunk/nano@4248 is a subdirectory of trunk@4248 so it starts downloading the complete history of trunk/nano again.\n\nOn Sep 8, 2010, at 12:13 PM, Daniel Trebbien wrote:\n\n> The issue is that the fetch runs up to revision 4250, but then mysteriously\n> begins redownloading revisions 1 up to four thousand something in importing\n> the `nano_2_1_1` tag:\n> ...\n> r4248 = 7c444dd667b9629ce92a53e8d35ad2d178e5735f (refs/remotes/trunk)\n> \tM\tnano/ChangeLog\n> \tM\tnano/configure.ac\n> \tM\tnano/po/cs.po\n\n> \tM\tnano/po/fi.po\n> \tM\tnano/po/zh_CN.po\n> r4249 = 6fbad8a00fa067d1e3de913f77db08c6117843c7 (refs/remotes/trunk)\n> \tM\tnano/NEWS\n> r4250 = 704700855e5112d75654e3e7461e896f49e10fd8 (refs/remotes/trunk)\n> Found possible branch point: svn://svn.sv.gnu.org/nano/trunk/nano =>\n> svn://svn.sv.gnu.org/nano/tags/nano_2_1_1, 4248\n> Initializing parent: refs/remotes/tags/nano_2_1_1@4248\n> \tA\tmkinstalldirs\n> \tA\tutils.c\n> \tA\tnano.h\n> \tA\tglobal.c\n> \tA\tconfigure\n> \tA\tMakefile.in\n\nI've encountered the same problem when importing other SVN repos.  If nano is the only subdirectory in trunk, you can fix this by changing your .git/config like this:\n\n-\tfetch = trunk:refs/remotes/trunk\n+\tfetch = trunk/nano:refs/remotes/trunk\n\nThis will cause issues if nano is not the only directory in trunk (it won't fetch the others) or if you have branches or tags that branch off the full trunk (you get the same problem).\n\nOr you can downloading both trunk and nano as separate branches.  I think it fetches each revision twice, but it will prevent it from downloading the entire history for each subdirectory branch.\n\n \tfetch = trunk:refs/remotes/trunk\n+\tfetch = trunk/nano:refs/remotes/nano\n\nThis will make your history look a little odd since trunk will have every commit that nano does but be a completely separate branch.  (It also seems to confuse `git-svn find-rev`.)\n\nYou can also speed imports like this by creating a local mirror with svnsync and using `git svn clone --use-svnsync-props file:///local/mirror`.  This probably isn't a good long term solution, but can be very handy when you're experimenting with getting it right.  Once you have it set up as you like delete svn-remote.svn.useSvnsyncProps, change svn-remote.svn.url to the original URL, and delete the local mirror.\n\nI've been investigating altering the algorithm that finds branch points to understand branching off subdirectories, but haven't had the time.  It's good to know that the Minix 3 repo isn't the only one that does it.\n\n~~ Brian\n"},{"id":"150329","messageId":"AANLkTinOTgB7HkeVdB-+ttvVMT1HFg91nt6pdD6Y_uML@mail.gmail.com","threadId":"25039","inReplyTo":"B1E94440-DDCA-4C10-A0EE-E08A66DF3D7E@silverinsanity.com","subject":"Re: Why does git-svn redownload revisions?","fromName":"Daniel Trebbien","fromEmail":"dtrebbien@gmail.com","sentAt":"2010-09-08T22:36:56Z","receivedAt":"2010-09-08T22:36:56Z","isPatch":false,"sender":{"key":"dtrebbien@gmail.com","avatar":null},"body":"On Wed, Sep 8, 2010 at 10:19 AM, Brian Gernhardt\n<benji@silverinsanity.com> wrote:\n> This happens when you have a tag or branch that is a subdirectory of trunk.  It looks like tags/nano_2_1_1 is a branch off trunk/nano.  Git-svn doesn't recognize that trunk/nano@4248 is a subdirectory of trunk@4248 so it starts downloading the complete history of trunk/nano again.\n>\n> I've encountered the same problem when importing other SVN repos.  If nano is the only subdirectory in trunk, you can fix this by changing your .git/config like this:\n>\n> -       fetch = trunk:refs/remotes/trunk\n> +       fetch = trunk/nano:refs/remotes/trunk\n>\n> This will cause issues if nano is not the only directory in trunk (it won't fetch the others) or if you have branches or tags that branch off the full trunk (you get the same problem).\n>\n> Or you can downloading both trunk and nano as separate branches.  I think it fetches each revision twice, but it will prevent it from downloading the entire history for each subdirectory branch.\n>\n>        fetch = trunk:refs/remotes/trunk\n> +       fetch = trunk/nano:refs/remotes/nano\n>\n> This will make your history look a little odd since trunk will have every commit that nano does but be a completely separate branch.  (It also seems to confuse `git-svn find-rev`.)\n>\n> You can also speed imports like this by creating a local mirror with svnsync and using `git svn clone --use-svnsync-props file:///local/mirror`.  This probably isn't a good long term solution, but can be very handy when you're experimenting with getting it right.  Once you have it set up as you like delete svn-remote.svn.useSvnsyncProps, change svn-remote.svn.url to the original URL, and delete the local mirror.\n>\n> I've been investigating altering the algorithm that finds branch points to understand branching off subdirectories, but haven't had the time.  It's good to know that the Minix 3 repo isn't the only one that does it.\n>\n> ~~ Brian\n\nLooking through the Nano repository, it appears that early tags were\ncopies of the entire trunk, but later tags (`nano_2_0_8` and onward)\nwere copies of just `trunk/nano`.\n\nIs there a way that I can inform git-svn that tags `nano_2_0_8`,\n`nano_2_0_9`, ... should all be based off of `remotes/trunk/nano`?\n"},{"id":"150335","messageId":"1C149599-1AC6-464F-9FE0-298D224E9B49@silverinsanity.com","threadId":"25039","inReplyTo":"AANLkTinOTgB7HkeVdB-+ttvVMT1HFg91nt6pdD6Y_uML@mail.gmail.com","subject":"Re: Why does git-svn redownload revisions?","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2010-09-08T23:50:44Z","receivedAt":"2010-09-08T23:50:44Z","isPatch":false,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"\nOn Sep 8, 2010, at 6:36 PM, Daniel Trebbien wrote:\n\n>> Or you can downloading both trunk and nano as separate branches.  I think it fetches each revision twice, but it will prevent it from downloading the entire history for each subdirectory branch.\n>> \n>>        fetch = trunk:refs/remotes/trunk\n>> +       fetch = trunk/nano:refs/remotes/nano\n>> \n>> This will make your history look a little odd since trunk will have every commit that nano does but be a completely separate branch.  (It also seems to confuse `git-svn find-rev`.)\n\n> Looking through the Nano repository, it appears that early tags were\n> copies of the entire trunk, but later tags (`nano_2_0_8` and onward)\n> were copies of just `trunk/nano`.\n> \n> Is there a way that I can inform git-svn that tags `nano_2_0_8`,\n> `nano_2_0_9`, ... should all be based off of `remotes/trunk/nano`?\n\ngit-svn will automatically determine that it's branched off of trunk/nano, but will only save the commits for trunk/nano if you create a branch for it.  That's I think adding a fetch line for trunk/nano as the lesser of two evils.\n\n~~ Brian\n"}]}