{"thread":{"id":"3091","subject":"[QUESTION] about .git/info/grafts file","startedAt":"2006-01-17T17:32:01Z","lastAt":"2006-01-20T20:48:51Z","messageCount":21,"participants":["Franck","Junio C Hamano","Andreas Ericsson","Petr Baudis","Linus Torvalds","Ryan Anderson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"14784","messageId":"cda58cb80601170932o6f955469y@mail.gmail.com","threadId":"3091","inReplyTo":"cda58cb80601170928r252a6e34y@mail.gmail.com","subject":"[QUESTION] about .git/info/grafts file","fromName":"Franck","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-01-17T17:32:01Z","receivedAt":"2006-01-17T17:32:01Z","isPatch":false,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"Hi,\n\nI'm wondering why the \"grafts\" files is not involved during\npush/pull/clone operations ?\n\nAnother question regarding grafting use case. Let's say I have my\norigin branch looks like:\n\n origin ---0---1------300 000---300 001---300 002\n\nLet's say that the 300 000th commit is where I started my work by using:\n\n $ git-checkout -b master <300 000 shaid>\n\nI do some work on master branch and get the following\n\n a---b---c---d master\n /\n origin ---0---1---...---300,000---300,001---300,002\n\nNow, I would like to make my own public repository based on my work\nbut before pushing master branch in that repo I would like to get rid\nof all unused commits [0 299,999]. Indeed each of these commits do not\nhave useful history for my work. So I used grafts things to have:\n\n a---b---c---d master\n /\n origin 300,000---300,001---300,002\n\nBut now if I ask to git for:\n\n $ git-merge-base master origin\n # nothing\n\nSo git failed to found the common commit object which should be 300,000. Why ?\n\nIn other the hand, if I use grafting to get:\n\n a---b---c---d master\n /\n origin 2999,999---300,000---300,001---300,002\n\n $ git-merge-base master origin\n 2dcaaf2decd31ac9a21d616604c0a7c1fa65d5a4\n\nSo now git found the common commit. Can anybody explain me why ?\n\nDo you think it's a good usage of git ? Or should I do otherwise to\nsetup my public repository ?\n\nThanks\n--\n Franck\n"},{"id":"14832","messageId":"cda58cb80601180947j48a5396t@mail.gmail.com","threadId":"3091","inReplyTo":"cda58cb80601170932o6f955469y@mail.gmail.com","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Franck","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-01-18T17:47:54Z","receivedAt":"2006-01-18T17:47:54Z","isPatch":false,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"Hi,\n\nCould anybody shed some light there ? It would be very nice.\n\nThanks\n Franck\n\n2006/1/17, Franck :\n> Hi,\n>\n> I'm wondering why the \"grafts\" files is not involved during\n> push/pull/clone operations ?\n>\n> Another question regarding grafting use case. Let's say I have my\n> origin branch looks like:\n>\n> origin ---0---1------300 000---300 001---300 002\n>\n> Let's say that the 300 000th commit is where I started my work by using:\n>\n> $ git-checkout -b master <300 000 shaid>\n>\n> I do some work on master branch and get the following\n>\n> a---b---c---d master\n> /\n> origin ---0---1---...---300,000---300,001---300,002\n>\n> Now, I would like to make my own public repository based on my work\n> but before pushing master branch in that repo I would like to get rid\n> of all unused commits [0 299,999]. Indeed each of these commits do not\n> have useful history for my work. So I used grafts things to have:\n>\n> a---b---c---d master\n> /\n> origin 300,000---300,001---300,002\n>\n> But now if I ask to git for:\n>\n> $ git-merge-base master origin\n> # nothing\n>\n> So git failed to found the common commit object which should be 300,000. Why ?\n>\n> In other the hand, if I use grafting to get:\n>\n> a---b---c---d master\n> /\n> origin 2999,999---300,000---300,001---300,002\n>\n> $ git-merge-base master origin\n> 2dcaaf2decd31ac9a21d616604c0a7c1fa65d5a4\n>\n> So now git found the common commit. Can anybody explain me why ?\n>\n> Do you think it's a good usage of git ? Or should I do otherwise to\n> setup my public repository ?\n>\n> Thanks\n> --\n> Franck\n>\n"},{"id":"14860","messageId":"7v8xtdrqwg.fsf@assigned-by-dhcp.cox.net","threadId":"3091","inReplyTo":"cda58cb80601170932o6f955469y@mail.gmail.com","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-19T00:40:47Z","receivedAt":"2006-01-19T00:40:47Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Franck writes:\n\n> I'm wondering why the \"grafts\" files is not involved during\n> push/pull/clone operations ?\n\nCommit ancestry grafting is a local repository issue and even if\nyou manage to lie to your local git that 300,000th commit is the\nepoch, the commit object you send out to the downloader would\nrecord its true parent (or parents, if it is a merge), so the\ndownloader would want to go further back. And no, rewriting\nthat commit and feeding a parentless commit to the downloader is\nnot an option, because such a commit object would have different\nobject name and unpack-objects would be unhappy.\n\nIf you choose not to have full history in your public repository\nfor whatever reason (ISP server diskquota comes to mind) that is\nOK, but be honest about it to your downloaders. Tell them that\nyou do not have the full history, and they first need to clone\nfrom some other repository you started your development upon, in\norder to use what you added upon. \"This repository does not\nhave all the history -- please first clone from XX repository\n(you need at least xxx commit), and then do another 'git pull'\nfrom here\", or something like that.\n\nIt _might_ work if you tell your downloader to have a proper\ngraft file in his repository to cauterize the commit ancestry\nchain _before_ he pulls from you, though. I haven't tried it\n(and honestly I did not feel that is something important to\nsupport, so it might work by accident but that is not by\ndesign).\n\n> $ git-merge-base master origin\n> # nothing\n\nMaybe you did not use grafts properly to cauterize? I tried the\nfollowing and am getting expected results. I did not have\npatience to do 300,000, so I cut things at #4, though.\n\n-- 8< -- \n\n#!/bin/sh\n\nrm -fr .git\ngit init-db\necho 0 >path\ngit add path\n\nfor i in 1 2 3 4 5 6 7\ndo\n\techo $i >path\n\tgit commit -a -m \"Iteration #$i\"\n\tgit tag \"iter#$i\"\ndone\n\n\ngit checkout -b mine iter#4\n\nfor i in A B C D\ndo\n\techo $i >path\n\tgit commit -a -m \"Alternate #$i\"\n\tgit tag \"alt#$i\"\ndone\n\ngit log --pretty=oneline --topo-order\necho merge base is `git merge-base master mine` | git name-rev --stdin\n\ngit-rev-parse iter#4 >.git/info/grafts\necho \"Cauterize away history before #4\"\n\ngit log --pretty=oneline --topo-order\necho merge base is `git merge-base master mine` | git name-rev --stdin\n"},{"id":"14883","messageId":"cda58cb80601190251v5251c8bdh@mail.gmail.com","threadId":"3091","inReplyTo":"7v8xtdrqwg.fsf@assigned-by-dhcp.cox.net","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Franck","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-01-19T10:51:22Z","receivedAt":"2006-01-19T10:51:22Z","isPatch":false,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"Thanks Junio for answering\n\n2006/1/19, Junio C Hamano :\n> Franck writes:\n>\n> > I'm wondering why the \"grafts\" files is not involved during\n> > push/pull/clone operations ?\n>\n> Commit ancestry grafting is a local repository issue and even if\n> you manage to lie to your local git that 300,000th commit is the\n> epoch, the commit object you send out to the downloader would\n> record its true parent (or parents, if it is a merge), so the\n> downloader would want to go further back. And no, rewriting\n> that commit and feeding a parentless commit to the downloader is\n> not an option, because such a commit object would have different\n> object name and unpack-objects would be unhappy.\n>\n> If you choose not to have full history in your public repository\n> for whatever reason (ISP server diskquota comes to mind)\n\nwell, dealing with a repo that has more than 300,000 objects becomes a\nburden. A lots of git commands are slow, and cloning it take a while !\n\n> that is\n> OK, but be honest about it to your downloaders. Tell them that\n> you do not have the full history, and they first need to clone\n> from some other repository you started your development upon, in\n> order to use what you added upon. \"This repository does not\n> have all the history -- please first clone from XX repository\n> (you need at least xxx commit), and then do another 'git pull'\n> from here\", or something like that.\n>\n\nI don't try to hide or lie to my downloaders. I just want them to\navoid to deal with totaly pointless history. My work have been started\nrecently and is based on current XX repository. IMHO storing, dealing\nwith objects which are more than 10 years old is useless.\n\nI don't see why it is so bad to create a \"grafted\" repository ? I want\nit to be small but still want to merge by using git-resolve with XX\nrepository.\n\n>\n> > $ git-merge-base master origin\n> > # nothing\n>\n> Maybe you did not use grafts properly to cauterize?\n\nWell in my graft file I did:\n\n $ cat > .git/info/grafts\n \n\n $\n\nBy reading \"Documentation/repository-layout.txt\", I thought it would\nhave been the right thing to do. If I did the same like you did ie:\n\n $ cat > .git/info/grafts\n \n\n $\n\nIt works.\n\nThanks\n--\n Franck\n"},{"id":"14884","messageId":"43CF739F.2030204@op5.se","threadId":"3091","inReplyTo":"7v8xtdrqwg.fsf@assigned-by-dhcp.cox.net","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2006-01-19T11:10:23Z","receivedAt":"2006-01-19T11:10:23Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Junio C Hamano wrote:\n> Franck writes:\n> \n> \n>>I'm wondering why the \"grafts\" files is not involved during\n>>push/pull/clone operations ?\n> \n> \n> Commit ancestry grafting is a local repository issue and even if\n> you manage to lie to your local git that 300,000th commit is the\n> epoch, the commit object you send out to the downloader would\n> record its true parent (or parents, if it is a merge), so the\n> downloader would want to go further back. And no, rewriting\n> that commit and feeding a parentless commit to the downloader is\n> not an option, because such a commit object would have different\n> object name and unpack-objects would be unhappy.\n> \n\n\nI'm a bit curious about how this was done for the public kernel repo. \nI'd like to import glibc to git, but keeping history since 1972 seems a \nbloody waste, really.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"14890","messageId":"20060119130519.GB28365@pasky.or.cz","threadId":"3091","inReplyTo":"43CF739F.2030204@op5.se","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-19T13:05:19Z","receivedAt":"2006-01-19T13:05:19Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Thu, Jan 19, 2006 at 12:10:23PM CET, I got a letter\nwhere Andreas Ericsson said that...\n> Junio C Hamano wrote:\n> >Franck writes:\n> >\n> >\n> >>I'm wondering why the \"grafts\" files is not involved during\n> >>push/pull/clone operations ?\n> >\n> >\n> >Commit ancestry grafting is a local repository issue and even if\n> >you manage to lie to your local git that 300,000th commit is the\n> >epoch, the commit object you send out to the downloader would\n> >record its true parent (or parents, if it is a merge), so the\n> >downloader would want to go further back. And no, rewriting\n> >that commit and feeding a parentless commit to the downloader is\n> >not an option, because such a commit object would have different\n> >object name and unpack-objects would be unhappy.\n> \n> I'm a bit curious about how this was done for the public kernel repo. \n> I'd like to import glibc to git, but keeping history since 1972 seems a \n> bloody waste, really.\n\nFWIW, with the ELinks GIT repository we just started from scratch and\nthen converted the old CVS repository, and provided this script in\ncontrib/grafthistory.sh:\n\n\n#!/bin/sh\n#\n# Graft the ELinks development history to the current tree.\n#\n# Note that this will download about 80M.\n\nif [ -z \"`which wget 2>/dev/null`\" ]; then\n echo \"Error: You need to have wget installed so that I can fetch the history.\" >&2\n exit 1\nfi\n\n[ \"$GIT_DIR\" ] || GIT_DIR=.git\nif ! [ -d \"$GIT_DIR\" ]; then\n echo \"Error: You must run this from the project root (or set GIT_DIR to your .git directory).\" >&2\n exit 1\nfi\ncd \"$GIT_DIR\"\n\necho \"[grafthistory] Downloading the history\"\nmkdir -p objects/pack\ncd objects/pack\nwget -c http://elinks.cz/elinks-history.git/objects/pack/pack-0d6c5c67aab3b9d5d9b245da5929c15d79124a48.idx\nwget -c http://elinks.cz/elinks-history.git/objects/pack/pack-0d6c5c67aab3b9d5d9b245da5929c15d79124a48.pack\n\necho \"[grafthistory] Setting up the grafts\"\ncd ../..\nmkdir -p info\n# master\necho 0f6d4310ad37550be3323fab80456e4953698bf0 06135dc2b8bb7ed2e441305bdaa82048396de633 >>info/grafts\n# REL_0_10\necho 43a9a406737fd22a8558c47c74b4ad04d4c92a2b 730242dcf2cdeed13eae7e8b0c5f47bb03326792 >>info/grafts\n\necho \"[grafthistory] Refreshing the dumb server info wrt. new packs\"\ncd ..\ngit-update-server-info\n\n\nSo you checkout the ELinks repository and if you want the full history\nyou just run this script and it does everything for you.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe. -- Douglas Adams\n"},{"id":"14891","messageId":"20060119130940.GC28365@pasky.or.cz","threadId":"3091","inReplyTo":"cda58cb80601190251v5251c8bdh@mail.gmail.com","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-19T13:09:40Z","receivedAt":"2006-01-19T13:09:40Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Thu, Jan 19, 2006 at 11:51:22AM CET, I got a letter\nwhere Franck said that...\n> well, dealing with a repo that has more than 300,000 objects becomes a\n> burden. A lots of git commands are slow, and cloning it take a while !\n\nWere the objects packed? It would be interesting to have some data about\nhow GIT performs with that much objects...\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe. -- Douglas Adams\n"},{"id":"14892","messageId":"cda58cb80601190531q238a8f40r@mail.gmail.com","threadId":"3091","inReplyTo":"43CF739F.2030204@op5.se","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Franck","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-01-19T13:31:58Z","receivedAt":"2006-01-19T13:31:58Z","isPatch":false,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"2006/1/19, Andreas Ericsson :\n> Junio C Hamano wrote:\n> > Franck writes:\n> >\n> >\n> >>I'm wondering why the \"grafts\" files is not involved during\n> >>push/pull/clone operations ?\n> >\n> >\n> > Commit ancestry grafting is a local repository issue and even if\n> > you manage to lie to your local git that 300,000th commit is the\n> > epoch, the commit object you send out to the downloader would\n> > record its true parent (or parents, if it is a merge), so the\n> > downloader would want to go further back. And no, rewriting\n> > that commit and feeding a parentless commit to the downloader is\n> > not an option, because such a commit object would have different\n> > object name and unpack-objects would be unhappy.\n> >\n>\n>\n> I'm a bit curious about how this was done for the public kernel repo.\n> I'd like to import glibc to git, but keeping history since 1972 seems a\n> bloody waste, really.\n>\n\nThat's exactly my point. Futhermore make your downloaders import that\nuseless history spread this waste.\n\nI guess kernel repo will encounter this problem in short term. It's\nbeing bigger and bigger and developpers may be borred to deal with so\nmany useless objects. But I'm not saying that it's bad thing to keep\nthat history. It just would be nice to allow developpers that don't\ncare about old history to get rid of it.\n\nThanks\n--\n Franck\n"},{"id":"14893","messageId":"43CF97AF.9060300@op5.se","threadId":"3091","inReplyTo":"cda58cb80601190531q238a8f40r@mail.gmail.com","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2006-01-19T13:44:15Z","receivedAt":"2006-01-19T13:44:15Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Franck wrote:\n> 2006/1/19, Andreas Ericsson :\n>>\n>>I'm a bit curious about how this was done for the public kernel repo.\n>>I'd like to import glibc to git, but keeping history since 1972 seems a\n>>bloody waste, really.\n>>\n> \n> \n> That's exactly my point. Futhermore make your downloaders import that\n> useless history spread this waste.\n> \n> I guess kernel repo will encounter this problem in short term. It's\n> being bigger and bigger and developpers may be borred to deal with so\n> many useless objects.\n\n\nAch, no. The current kernel repo only has history since April 17 (around \n155 MB of objects, with less than optimal packing), when it started \nusing git for versioning. The kernel repo also sees a lot of very rapid \ndevelopment.\n\nThe full kernel tree, with history since 1991 or some such, is about 3.2 \nGB. It was for this reason that the early history was dropped. I don't \nthink another drop will be necessary any time soon, since incremental \nupdates are fairly cheap over git and git+ssh. Only gitk suffers, but \nthat's just for a short while.\n\n\n> But I'm not saying that it's bad thing to keep\n> that history. It just would be nice to allow developpers that don't\n> care about old history to get rid of it.\n> \n\nYou could ofcourse create a new repository with the files from the \nversion you want, but then you'd have a hard time merging the two repos \nif you ever want to import the old history.\n\nLinus; Is this what you did with the public kernel repo?\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"14901","messageId":"Pine.LNX.4.64.0601190842270.3240@g5.osdl.org","threadId":"3091","inReplyTo":"20060119130940.GC28365@pasky.or.cz","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-01-19T16:58:09Z","receivedAt":"2006-01-19T16:58:09Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 19 Jan 2006, Petr Baudis wrote:\n>\n> Dear diary, on Thu, Jan 19, 2006 at 11:51:22AM CET, I got a letter\n> where Franck said that...\n> > well, dealing with a repo that has more than 300,000 objects becomes a\n> > burden. A lots of git commands are slow, and cloning it take a while !\n> \n> Were the objects packed? It would be interesting to have some data about\n> how GIT performs with that much objects...\n\nThe historical linux archive has a lot more than 300,000 objects. In fact, \neven the _current_ kernel archive has almost 200,000 objects.\n\nMaybe somebody was thinking \"commits\", not \"objects\". Something with \n300,000 commits is indeed a pretty big project.\n\nAnyway, from a scalability standpoint, git should have no problem at all \nwith tons of objects, as long as you pack the old history. There are a few \nthings that get slower:\n\n - if you end up doing things that look at history, they are obviously at \n least linear is history size. Often there are other downsides too \n (using lots of memory).\n\n Example: try even just a simple \"gitk\" on the (regular, new) kernel \n archive, and it will take a while before the whole thing has been done. \n Of course, you'll see the top entries interactively, so mostly you \n won't care, but I routinely limit it some way just to make it not make \n the CPU fans come on. So I do something like\n\n\tgitk --since=1.week.ago\n\tgitk v2.6.15..\n\n instead of plain gitk, just because it makes operations cheaper.\n\n - a full clone takes a long time. Git _could_ fairly easily have an \n extension to add a date specifier to clone too:\n\n\tgit clone --since=1.month.ago \n\n and just leave any older stuff (you could always fetch it later), but \n we've just never done it. Maybe we should. It _should_ be pretty simple \n to do from a conceptual standpoint.\n\nbut \"everyday\" operations shouldn't slow down from having a long history. \nI can still apply 4-5 patches a second to the kernel archive, for example, \nas you can see from\n\n\tgit log --pretty=fuller | grep CommitDate | less -S\n\nand looking for one of the patch series I've applied from Andrew..\n\n\t\tLinus\n"},{"id":"14902","messageId":"20060119173006.GE28365@pasky.or.cz","threadId":"3091","inReplyTo":"43CF97AF.9060300@op5.se","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-19T17:30:07Z","receivedAt":"2006-01-19T17:30:07Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Thu, Jan 19, 2006 at 05:58:09PM CET, I got a letter\nwhere Linus Torvalds said that...\n> On Thu, 19 Jan 2006, Petr Baudis wrote:\n> >\n> > Dear diary, on Thu, Jan 19, 2006 at 11:51:22AM CET, I got a letter\n> > where Franck said that...\n> > > well, dealing with a repo that has more than 300,000 objects becomes a\n> > > burden. A lots of git commands are slow, and cloning it take a while !\n> > \n> > Were the objects packed? It would be interesting to have some data about\n> > how GIT performs with that much objects...\n> \n> The historical linux archive has a lot more than 300,000 objects. In fact, \n> even the _current_ kernel archive has almost 200,000 objects.\n\nEek. I was burnt by git-count-objects' misleading name. I guess\n\n\tgit-rev-list --objects --all | wc -l\n\nshould give accurate results - 145941 for kernel repository back from\nDecember. I will follow up later with a patch for git-count-objects.\n\n> - a full clone takes a long time. Git _could_ fairly easily have an \n> extension to add a date specifier to clone too:\n> \n> \tgit clone --since=1.month.ago \n> \n> and just leave any older stuff (you could always fetch it later), but \n> we've just never done it. Maybe we should. It _should_ be pretty simple \n> to do from a conceptual standpoint.\n\nYes. I receive wishes for this time by time and it is buried somewhere\ndeep in my TODO list. I'm not sure how happy the GIT tools will be about\ninvalid parent references.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe. -- Douglas Adams\n"},{"id":"14903","messageId":"cda58cb80601190933o4cedde92x@mail.gmail.com","threadId":"3091","inReplyTo":"Pine.LNX.4.64.0601190842270.3240@g5.osdl.org","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Franck","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-01-19T17:33:17Z","receivedAt":"2006-01-19T17:33:17Z","isPatch":false,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"2006/1/19, Linus Torvalds :\n> - a full clone takes a long time. Git _could_ fairly easily have an\n> extension to add a date specifier to clone too:\n>\n> git clone --since=1.month.ago \n>\n> and just leave any older stuff (you could always fetch it later), but\n> we've just never done it. Maybe we should. It _should_ be pretty simple\n> to do from a conceptual standpoint.\n>\n\nthat would be great ! something like:\n\n git clone --since=v2.6.15 \n\nwould be very useful for me. How would it work ? Does it automatically\nset up a graft file for me ?\n\n> but \"everyday\" operations shouldn't slow down from having a long history.\n\nbut it's really a pain to run for example git-repack or git-prune commands.\n\nThanks\n--\n Franck\n"},{"id":"14904","messageId":"20060119174536.GF28365@pasky.or.cz","threadId":"3091","inReplyTo":"43CF97AF.9060300@op5.se","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-19T17:45:36Z","receivedAt":"2006-01-19T17:45:36Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Thu, Jan 19, 2006 at 02:44:15PM CET, I got a letter\nwhere Andreas Ericsson said that...\n> Ach, no. The current kernel repo only has history since April 17 (around \n> 155 MB of objects, with less than optimal packing), when it started \n> using git for versioning. The kernel repo also sees a lot of very rapid \n> development.\n> \n> The full kernel tree, with history since 1991 or some such, is about 3.2 \n> GB.\n\nThere is some \"accurate\" history only from the moment the kernel got\ntracked in BK, and it is certainly far less.\n\nThe question is, what is the \"official\" kernel history repository?\nThere is at least\n\n\thttp://www.kernel.org/pub/scm/linux/kernel/git/tglx/history.git\n\nwith a 251M pack and\n\n\thttp://www.kernel.org/pub/scm/linux/kernel/git/torvalds/old-2.6-bkcvs.git\n\nwith a 165M pack - IIRC the latter is obsoleted by the former and\nperhaps should be blasted to prevent confusion?\n\nGetting a little offtopic here... Linus, would it be deemed useful to\nhave the script I've pasted in <20060119130519.GB28365@pasky.or.cz>\n(earlier in this thread) in the kernel's scripts/ directory, pointing at\nthe canonical history repository?\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe. -- Douglas Adams\n"},{"id":"14905","messageId":"Pine.LNX.4.64.0601190938110.3240@g5.osdl.org","threadId":"3091","inReplyTo":"cda58cb80601190933o4cedde92x@mail.gmail.com","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-01-19T17:49:35Z","receivedAt":"2006-01-19T17:49:35Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 19 Jan 2006, Franck wrote:\n> \n> that would be great ! something like:\n> \n> git clone --since=v2.6.15 \n> \n> would be very useful for me. How would it work ? Does it automatically\n> set up a graft file for me ?\n\nI think we'd have to set up the grafts file, yes. However, it's actually \nless of an advantage than you'd think: especially for long development \nhistories, the incremental packing is very very efficient. In contrast, if \nyou only get recent versions, there's nothing to be incremental against, \nso the size of the pack will not be that much smaller.\n\nSo getting just a tenth of the development history will _not_ cause the \npack to be just a tenth in size. It's probably closer to half the size of \nthe full history.\n\nAnyway, it's _conceptually_ something that git wouldn't have any problems \nwith, but that doesn't mean that it's totally trivial either. The easiest \nway to do it (by far) would be to expand the native git protocol with a \n\"get all objects of this one version\" or something like that, and then \nyou'd just do a \"pull and mark all unknown commits in the grafts file\".\n\nSo in effect, instead of getting the whole history pack, you'd get a pack \nthat contains _one_ version (no history at all), and then (if you want to) \nyou can get a pack that gets all stuff that isn't reachable from that one \n(ie \"newer\").\n\nThat would have the advantage that it's quite possible that many users \nmight want to do just\n\n\tgit clone --only=v2.6.15 \n\nwhich would do that \"one single version\" variant of the clone. Then, later \non, you could just do\n\n\tgit pull --graft-unknown \n\nto update the history.\n\nAnybody want to try that? It would be a new command to \"git-daemon\" \n(instead of \"git-upoload-pack\", you'd do a new \"git-upload-version\" \ncommand internally: it would look a lot like upload-pack, and use the same \nunpacking protocol).\n\n> but it's really a pain to run for example git-repack or git-prune commands.\n\nWell, you really don't need to do that very often.\n\n\t\tLinus\n"},{"id":"14911","messageId":"7v3bjkf54k.fsf@assigned-by-dhcp.cox.net","threadId":"3091","inReplyTo":"cda58cb80601190251v5251c8bdh@mail.gmail.com","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-19T18:24:11Z","receivedAt":"2006-01-19T18:24:11Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Franck writes:\n\n> I don't see why it is so bad to create a \"grafted\" repository ? I want\n> it to be small but still want to merge by using git-resolve with XX\n> repository.\n\nFranck, and people on the list,\n\nI have a bad habit of responding to a \"call for help\" request by\nstating how things are currently done and why, sometimes with an\noutline of how the limitation in the current way can be (or at\nleast I think it could be, without testing that solution myself)\nworked around, but without making it explicit if the limitation\nis something that should not be there or if it is something\nfundamental. This often makes it sound as if I am saying I\nthink the original request is unreasonable, and/or the current\nstate of affairs is perfect. This is one of such cases.\n\nI agree it would be nice to support \"strictly speaking, the\nrepository is incomplete but has everything necessary as long as\nyou operate near the tip of the development\" mode of operation.\n\nIt only has never been a high priority.\n\n> Well in my graft file I did:\n>\n> $ cat > .git/info/grafts\n> \n>\n> $\n\nThe trailing empty line at the end is discarded as a comment, I\nthink, so that should be fine. \"terminated by a newline\" in the\ndocumentation talks about each line being terminated by a LF,\nnot about terminating the file itself with an extra newline.\n\nI think you spotted a bug in a documentation and another in the\ncode. I presume these two are the same in what you did;\nyou are saying \"this commit has itself as its parent\", but that\ncan never be the case and the graft parser should reject such\nline and complain but I do not think the current code does so.\n\nThe documentation says \"a commit and its fake parents ...\nseparated by a space and terminated by a newline\". We should at\nleast say \"zero or more fake parents\", or make it ever clearer\nby giving a couple of examples.\n"},{"id":"14906","messageId":"7vwtgwdqk0.fsf@assigned-by-dhcp.cox.net","threadId":"3091","inReplyTo":"Pine.LNX.4.64.0601190842270.3240@g5.osdl.org","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-19T18:24:15Z","receivedAt":"2006-01-19T18:24:15Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> - a full clone takes a long time. Git _could_ fairly easily have an \n> extension to add a date specifier to clone too:\n>\n> \tgit clone --since=1.month.ago \n>\n> and just leave any older stuff (you could always fetch it later), but \n> we've just never done it. Maybe we should. It _should_ be pretty simple \n> to do from a conceptual standpoint.\n\nTrue, except some implementation details you forgot to mention\nin your other message that you talked about upload-version.\nBoth commit walkers and git native transfer fundamentally\noperate by trusting that our current refs are complete, which\nmakes \"could always fetch it later\" part a bit involved.\n\nIt fortunately would not be a rocket science. We would need to\nhave a mode \"do not trust our current refs are complete\" with an\nexplicit command line option, or automatically fall back to that\nmode when seeing the $GIT_DIR/info/grafts file has changed, and\nrevalidate the commit ancestry chain we have in a repository\ncloned that way.\n"},{"id":"14934","messageId":"7virsf7lb9.fsf@assigned-by-dhcp.cox.net","threadId":"3091","inReplyTo":"7v8xtdrqwg.fsf@assigned-by-dhcp.cox.net","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-20T01:14:02Z","receivedAt":"2006-01-20T01:14:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> It _might_ work if you tell your downloader to have a proper\n> graft file in his repository to cauterize the commit ancestry\n> chain _before_ he pulls from you, though. I haven't tried it\n> (and honestly I did not feel that is something important to\n> support, so it might work by accident but that is not by\n> design).\n\nI just tried it and it actually works.\n\n\t$ git clone git.git junk\n $ cd junk ;# I am not brave enough to risk the real thing ;-)\n\t$ git rev-parse master~4 >.git/refs/info/grafts\n $ cd ..\n $ mkdir cloned\n $ cd cloned\n $ git init-db\n $ cp ../junk/.git/info/grafts .git/info/\n\t$ git clone-pack ../baz\n\t$ git fsck-objects --full\n\t$ git log --pretty=short | cat\n\nThis \"only the tip of the git.git\" repository has about 450\nobjects in it, fully packed because of clone-pack, with one 680K\npackfile. I think the true full history of git.git/ packed into\none is aruond a 5MB packfile. I suspect a bigger repository\nwould not see that much size reduction, as Linus already\nexplained here.\n\nYou could emulate what I just did above to prepare the\nequivalent of \"baz\" above, and make it available over git://\nprotocol, say at git://franck.example.com/franck.git/.\n\nThen you tell your downloaders something like this:\n\n\tThis repository has been cauterized, and cannot be\n\tcloned in a usual manner, but once you make a clone\n\teverything including further incremental updates should\n\twork.\n\n To clone this repository:\n\n\t\t$ mkdir franckproject ;# make a new repository\n\t\t$ cd franckproject && git init-db\n\t\t$ echo 'XXxxxxXXxxx' >.git/info/grafts\n\t\t$ git clone-pack git://franck.example.com/franck.git/\n"},{"id":"14944","messageId":"cda58cb80601200207g7c3495ddx@mail.gmail.com","threadId":"3091","inReplyTo":"7virsf7lb9.fsf@assigned-by-dhcp.cox.net","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Franck","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-01-20T10:07:09Z","receivedAt":"2006-01-20T10:07:09Z","isPatch":false,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"2006/1/20, Junio C Hamano :\n> Junio C Hamano writes:\n>\n> > It _might_ work if you tell your downloader to have a proper\n> > graft file in his repository to cauterize the commit ancestry\n> > chain _before_ he pulls from you, though. I haven't tried it\n> > (and honestly I did not feel that is something important to\n> > support, so it might work by accident but that is not by\n> > design).\n>\n> I just tried it and it actually works.\n>\n> $ git clone git.git junk\n> $ cd junk ;# I am not brave enough to risk the real thing ;-)\n> $ git rev-parse master~4 >.git/refs/info/grafts\n> $ cd ..\n> $ mkdir cloned\n> $ cd cloned\n> $ git init-db\n> $ cp ../junk/.git/info/grafts .git/info/\n> $ git clone-pack ../baz\n> $ git fsck-objects --full\n> $ git log --pretty=short | cat\n>\n\nJust to be sure, what you call baz is actually junk ?\n\n> This \"only the tip of the git.git\" repository has about 450\n> objects in it, fully packed because of clone-pack, with one 680K\n> packfile.\n\nI tried that but I don't have same results. Did you delete all branchs\nexcept master before running clone-pack ? In my case I cloned the\nwhole thing. So junk and cloned repos are the same size\n\n> I think the true full history of git.git/ packed into\n> one is aruond a 5MB packfile. I suspect a bigger repository\n> would not see that much size reduction, as Linus already\n> explained here.\n\nsorry, but I didn't understand his explaination, surely because of my\nvery limited knowledge about git internals...\n\n>\n> You could emulate what I just did above to prepare the\n> equivalent of \"baz\" above, and make it available over git://\n> protocol, say at git://franck.example.com/franck.git/.\n>\n\ndoes the git protocol is really needed in your example ? or can rsync\nwork fine too since \"franck.git\" repo is cautorized so every objects\nof this repo shouldn't be part of old history, so they should be\nusefull, no ?\n\nThanks.\n--\n Franck\n"},{"id":"14946","messageId":"cda58cb80601200543s41b3f63ch@mail.gmail.com","threadId":"3091","inReplyTo":"7v3bjkf54k.fsf@assigned-by-dhcp.cox.net","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Franck","fromEmail":"vagabon.xyz@gmail.com","sentAt":"2006-01-20T13:43:10Z","receivedAt":"2006-01-20T13:43:10Z","isPatch":false,"sender":{"key":"vagabon.xyz@gmail.com","avatar":null},"body":"2006/1/19, Junio C Hamano :\n> Franck writes:\n>\n> > I don't see why it is so bad to create a \"grafted\" repository ? I want\n> > it to be small but still want to merge by using git-resolve with XX\n> > repository.\n>\n> Franck, and people on the list,\n>\n> I have a bad habit of responding to a \"call for help\" request by\n> stating how things are currently done and why, sometimes with an\n\nwhat ? Hey, I would say that you, Linus and other people on the list\nhave a GREAT habit to spend time to answer others how things work. And\nthere are usually accurate explanations, examples with a lot of\ndetails with them.\n\nThanks !\n--\n Franck\n"},{"id":"14958","messageId":"7v4q3ybx0m.fsf@assigned-by-dhcp.cox.net","threadId":"3091","inReplyTo":"cda58cb80601200207g7c3495ddx@mail.gmail.com","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-20T17:59:53Z","receivedAt":"2006-01-20T17:59:53Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Franck writes:\n\n> 2006/1/20, Junio C Hamano :\n>\n>> $ git clone git.git junk\n>> $ cd junk ;# I am not brave enough to risk the real thing ;-)\n>> $ git rev-parse master~4 >.git/refs/info/grafts\n\nTypo: \t's|.git/refs/info/grafts|.git/info/grafts|'\n\nBTW the above exact sequence will not work with my \"master\"\ntoday, since I merged up bunch of things last night. You have\nto cauterize all the paths that lead to earlier history. For\nexample, if I have this:\n\n ---o---o---x---o---o---o---o (master)\n \\ /\n o---o---o\n\ncauterizing at master~4 ('x') will still leak history via the\nside branch, if you follow the history from the tip and go\nbackwards. I have to also cauterize the merge commit after that\nto remove the side branch, or cauterize the leftmost branch\npoint and live with a bit deeper history. The choice depends on\nhow much real history I want to keep in the pruned history.\n\nFor example, to pretend the history was like this:\n\n ---o---o x---o---o---o---o (master)\n \\ \n o---o---o\n\n\t$ git rev-parse master~4 >.git/info/grafts ;# 'x'\n $ git rev-parse master~3 master~4 >.git/info/grafts\n\nThe second line says master~3 (the one that comes after 'x') has\nonly a single parent, which is master~4, in order to throw the\nside branch away [*1*].\n\nBack to the original example...\n\n>> $ cd ..\n>> $ mkdir cloned\n>> $ cd cloned\n>> $ git init-db\n>> $ cp ../junk/.git/info/grafts .git/info/\n>> $ git clone-pack ../baz\n\nThere are a couple of typos here and that was the reason your\nexperiment did not work. Sorry. The \"clone-pack\" should have\nbeen like this:\n\n $ git clone-pack ../junk master\n Packing 471 objects\n e7555785f4edcf4988c53305349e3f525216e2cb refs/heads/master\n\t$ git-rev-parse e7555785f >.git/refs/heads/master\n\nThis 'cloned' is the lightweight one.\n\n> does the git protocol is really needed in your example ? or can rsync\n> work fine too since \"franck.git\" repo is cautorized so every objects\n> of this repo shouldn't be part of old history, so they should be\n> usefull, no ?\n\nrsync may for the initial clone but its use afterwards is\nfrowned upon for other reasons these days.\n\n\n[Footnote]\n\n*1* There still is an anomaly if you look at \"git log\" after\npruning side branch this way; master~3 commit is still shown as\n\"merge\". I think you could call it a bug, but I am not sure it\nis worth fixing.\n"},{"id":"14963","messageId":"20060120204851.GB19212@mythryan2.michonline.com","threadId":"3091","inReplyTo":"43CF97AF.9060300@op5.se","subject":"Re: [QUESTION] about .git/info/grafts file","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2006-01-20T20:48:51Z","receivedAt":"2006-01-20T20:48:51Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"On Thu, Jan 19, 2006 at 02:44:15PM +0100, Andreas Ericsson wrote:\n> \n> The full kernel tree, with history since 1991 or some such, is about 3.2 \n> GB. It was for this reason that the early history was dropped. I don't \n> think another drop will be necessary any time soon, since incremental \n> updates are fairly cheap over git and git+ssh. Only gitk suffers, but \n> that's just for a short while.\n\nJust to make sure this is corrected, the 3.2GB was for a fully unpacked\ntree, which is still fairly bad in the current tree.\n\nThe historical tree, packed, runs about 266M in a single pack.\nAdmittedly, I still refuse to try to run gitk on it.\n\n> >But I'm not saying that it's bad thing to keep\n> >that history. It just would be nice to allow developpers that don't\n> >care about old history to get rid of it.\n> \n> You could ofcourse create a new repository with the files from the \n> version you want, but then you'd have a hard time merging the two repos \n> if you ever want to import the old history.\n\nIt's always possible to use a \"graft\" to tie the history together, and\nif you really need to merge changes across the boundary, my graft-ripple\n(in the archives) tool can make it happen, though it does some ... nasty\nthings to the history tree in the process. (It might be useful on a\nthrowaway tree to provide a way to merge, then, from which a set of\ndiffs could be taken and applied back on an un-messy tree.)\n\n-- \n\nRyan Anderson\n sometimes Pug Majere\n"}]}