{"thread":{"id":"18476","subject":"Summer of Code - Cached Packs/Object Lists","startedAt":"2009-03-23T01:53:49Z","lastAt":"2009-03-23T14:41:11Z","messageCount":5,"participants":["thisnukes4u@gmail.com","Shawn O. Pearce","Thomas Coppi","Nicolas Pitre"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"108981","messageId":"fsmi80et2dp57f5bdxUYAxe124vaj_firegpg@mail.gmail.com","threadId":"18476","inReplyTo":null,"subject":"Summer of Code - Cached Packs/Object Lists","fromName":"","fromEmail":"thisnukes4u@gmail.com","sentAt":"2009-03-23T01:53:49Z","receivedAt":"2009-03-23T01:53:49Z","isPatch":false,"sender":{"key":"thisnukes4u@gmail.com","avatar":null},"body":"Hey list,\n\n My name is Thomas Coppi, and I am a junior studying Computer Science at New Mexico Tech. I've been using git for a few years now for managing coursework and personal projects, but have never looked much into the git source code or internals, so I figure that Summer of Code is a good way to get started.\n\n I am particularly interested in the packfile caching project mentioned on the wiki, but I have a couple of questions:\n\n1. Would it be possible to implement both the packfile and object list caching mechanisms, or might would one interfere with the other in some way?\n2. With just a quick perusal of the daemon source, I noticed that it shells out to the upload-pack command. Where would it be appropriate to implement such a caching mechanism, in the daemon proper, the upload-pack code, or would both need to be updated?\n\nThanks in advance, and sorry if my questions seem a little bit newbish, as I mentioned this is my first time diving into git internals.\n-- \nThomas Coppi\n\n"},{"id":"108982","messageId":"20090323015936.GM23521@spearce.org","threadId":"18476","inReplyTo":"fsmi80et2dp57f5bdxUYAxe124vaj_firegpg@mail.gmail.com","subject":"Re: Summer of Code - Cached Packs/Object Lists","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-03-23T01:59:36Z","receivedAt":"2009-03-23T01:59:36Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Please line wrap your email at something useful to others when\nquoting, like 70-72 characters per line.\n\nthisnukes4u@gmail.com wrote:\n> I am particularly interested in the packfile caching project\n> mentioned on the wiki, but I have a couple of questions:\n>\n> 1. Would it be possible to implement both the packfile and\n> object list caching mechanisms, or might would one interfere with\n> the other in some way?\n\nYou could do both. But I think most people on the list will argue\nthat doing both is overkill and only one is necessary, and further,\nthat only the one that offers the \"biggest bank for the buck\"\nshould be implemented.\n\nWhole pack file caching has been discussed on list a few times as a\nnice feature to have, but it raises some issues of cache management,\nnot to mention the issue I posed about it being relatively useless\non frequently changing repositories.\n\n> 2. With just a quick perusal of the daemon source, I noticed\n> that it shells out to the upload-pack command. Where would it be\n> appropriate to implement such a caching mechanism, in the daemon\n> proper, the upload-pack code, or would both need to be updated?\n\nThe daemon doesn't get enough data from the client in order to\nperform any sort of caching.\n\nSo the caching has to happen in upload-pack, and/or pack-objects.\n(upload-pack forks out to pack-objects to create the pack file to\nsend to the client)\n\n-- \nShawn.\n"},{"id":"108990","messageId":"fsmk86y8vqj4f56b08UYAxe124vaj_firegpg@mail.gmail.com","threadId":"18476","inReplyTo":"20090323015936.GM23521@spearce.org","subject":"Re: Summer of Code - Cached Packs/Object Lists","fromName":"Thomas Coppi","fromEmail":"thisnukes4u@gmail.com","sentAt":"2009-03-23T02:49:57Z","receivedAt":"2009-03-23T02:49:57Z","isPatch":false,"sender":{"key":"thisnukes4u@gmail.com","avatar":null},"body":"On Sun, Mar 22, 2009 at 7:59 PM, Shawn O. Pearce wrote:\n> Please line wrap your email at something useful to others when\n> quoting, like 70-72 characters per line.\n\nSorry about that.\n\n> You could do both.  But I think most people on the list will argue\n> that doing both is overkill and only one is necessary, and further,\n> that only the one that offers the \"biggest bank for the buck\"\n> should be implemented.\n\nAlright, that seems reasonable. Given that I think I would lean\ntowards implementing an object list caching mechanism, since that seems\nto be more generally applicable. The logic for this would then need to\nbe in the rev-list code(as mentioned in the JGit discussion), correct?\n\nOh, and I forgot to mention in my previous email that I am also\navailable on freenode IRC under the nick tcoppi.\n\nThanks again,\n\n--\nThomas Coppi\n\n"},{"id":"108991","messageId":"20090323025244.GO23521@spearce.org","threadId":"18476","inReplyTo":"fsmk86y8vqj4f56b08UYAxe124vaj_firegpg@mail.gmail.com","subject":"Re: Summer of Code - Cached Packs/Object Lists","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-03-23T02:52:44Z","receivedAt":"2009-03-23T02:52:44Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Thomas Coppi wrote:\n> On Sun, Mar 22, 2009 at 7:59 PM, Shawn O. Pearce wrote:\n>> You could do both. ??But I think most people on the list will argue\n>> that doing both is overkill and only one is necessary, and further,\n>> that only the one that offers the \"biggest bank for the buck\"\n>> should be implemented.\n>\n> Alright, that seems reasonable. Given that I think I would lean\n> towards implementing an object list caching mechanism, since that seems\n> to be more generally applicable. The logic for this would then need to\n> be in the rev-list code(as mentioned in the JGit discussion), correct?\n\nProbably. IIRC upload-pack forks a rev-list to produce the\nobject list, and pipes that into the forked pack-objects' stdin.\nThus rev-list is probably what would need to know how to include\nthe cached list to its output.\n\n-- \nShawn.\n"},{"id":"109073","messageId":"alpine.LFD.2.00.0903231037100.30483@xanadu.home","threadId":"18476","inReplyTo":"20090323025244.GO23521@spearce.org","subject":"Re: Summer of Code - Cached Packs/Object Lists","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2009-03-23T14:41:11Z","receivedAt":"2009-03-23T14:41:11Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sun, 22 Mar 2009, Shawn O. Pearce wrote:\n\n> Thomas Coppi wrote:\n> > On Sun, Mar 22, 2009 at 7:59 PM, Shawn O. Pearce wrote:\n> >> You could do both. ??But I think most people on the list will argue\n> >> that doing both is overkill and only one is necessary, and further,\n> >> that only the one that offers the \"biggest bank for the buck\"\n> >> should be implemented.\n> >\n> > Alright, that seems reasonable. Given that I think I would lean\n> > towards implementing an object list caching mechanism, since that seems\n> > to be more generally applicable. The logic for this would then need to\n> > be in the rev-list code(as mentioned in the JGit discussion), correct?\n> \n> Probably. IIRC upload-pack forks a rev-list to produce the\n> object list, and pipes that into the forked pack-objects' stdin.\n> Thus rev-list is probably what would need to know how to include\n> the cached list to its output.\n\nRelated to this, the first optimization is probably to avoid the fork \naltogether. The pack-objects code knows how to list objects by itself \nalready, and that is used by git-repack. At the moment, packed tree \nobjects during a fetch are probably accessed one extra time needlessly.\n\n\nNicolas\n"}]}