{"thread":{"id":"1689","subject":"Re: please pull ppc64-2.6.git","startedAt":"2005-08-29T17:32:09Z","lastAt":"2005-09-02T00:20:02Z","messageCount":11,"participants":["Linus Torvalds","Russell King","Paul Mackerras","Christian Meder","Junio C Hamano","Sergey Vlasov"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"7903","messageId":"Pine.LNX.4.58.0508291006440.3243@g5.osdl.org","threadId":"1689","inReplyTo":"17170.25803.413408.44080@cargo.ozlabs.ibm.com","subject":"Re: please pull ppc64-2.6.git","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-08-29T17:32:09Z","receivedAt":"2005-08-29T17:32:09Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 29 Aug 2005, Paul Mackerras wrote:\n> \n> Please do a pull from:\n> \n> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/paulus/ppc64-2.6.git\n\nGaah.\n\nThis is not a valid git repository.\n\nGuys, if you do partially populated repositories, _please_ make sure that \nyou still make it a valid git repository. These days you can trivially do \nso by doing a\n\n\techo /pub/scm/linux/kernel/git/torvalds/linux-2.6/objects > objects/info/alternates\n\nor similar. That also makes gitweb able to show diffs etc, something it \ncan't do for a broken partial repository.\n\nNot pulled,\n\n\t\tLinus\n"},{"id":"7905","messageId":"20050829184510.A20605@flint.arm.linux.org.uk","threadId":"1689","inReplyTo":"Pine.LNX.4.58.0508291006440.3243@g5.osdl.org","subject":"Re: please pull ppc64-2.6.git","fromName":"Russell King","fromEmail":"rmk@arm.linux.org.uk","sentAt":"2005-08-29T17:45:10Z","receivedAt":"2005-08-29T17:45:10Z","isPatch":false,"sender":{"key":"rmk@arm.linux.org.uk","avatar":null},"body":"On Mon, Aug 29, 2005 at 10:32:09AM -0700, Linus Torvalds wrote:\n> On Mon, 29 Aug 2005, Paul Mackerras wrote:\n> > Please do a pull from:\n> > \n> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/paulus/ppc64-2.6.git\n> \n> Gaah.\n> \n> This is not a valid git repository.\n> \n> Guys, if you do partially populated repositories, _please_ make sure that \n> you still make it a valid git repository. These days you can trivially do \n> so by doing a\n> \n> \techo /pub/scm/linux/kernel/git/torvalds/linux-2.6/objects > objects/info/alternates\n> \n> or similar. That also makes gitweb able to show diffs etc, something it \n> can't do for a broken partial repository.\n\nIs the expected filesystem layout documented somewhere online (_external_\nto the source code) ?\n\nThe reason I stress external to the code is that some of us do not track\ngit developments. (Except via the ctrl-d method in our mail readers.)\n\nAlternatively, when changes occur to the repostory format, please can\nthey be marked with some obvious subject so that folk know when things\nare going to break?\n\n-- \nRussell King\n"},{"id":"7907","messageId":"Pine.LNX.4.58.0508291057380.3243@g5.osdl.org","threadId":"1689","inReplyTo":"20050829184510.A20605@flint.arm.linux.org.uk","subject":"Re: please pull ppc64-2.6.git","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-08-29T18:02:38Z","receivedAt":"2005-08-29T18:02:38Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 29 Aug 2005, Russell King wrote:\n> \n> Is the expected filesystem layout documented somewhere online (_external_\n> to the source code) ?\n\nNope, I don't think so. \n\n> Alternatively, when changes occur to the repostory format, please can\n> they be marked with some obvious subject so that folk know when things\n> are going to break?\n\nThe only actual filesystem _breakage_ has been the introduction of \npack-files (and the old, old _old_ thing where I changed the actual object \ncompression/hashing order).\n\nThe \"objects/info/alternates\" thing is an extension, which allows you to \nhave a partial object store, and point to the \"rest of it\", and still have \nall the tools understand it and be able to parse the totality of it. So it \ndoesn't break or change old formats, it only allows a new one.\n\n(Partial repos have always worked with the rsync protocol, and with the\nclient-side pulling. But that was more of an accident than anything else,\nand they fundamentally were broken for any real work - gitweb can't show\nanything really sane, server-side serving - whether anonymous or ssh -\ndoesn't work, etc etc).\n\n\t\tLinus\n"},{"id":"7909","messageId":"20050829192013.B20605@flint.arm.linux.org.uk","threadId":"1689","inReplyTo":"Pine.LNX.4.58.0508291057380.3243@g5.osdl.org","subject":"Re: please pull ppc64-2.6.git","fromName":"Russell King","fromEmail":"rmk@arm.linux.org.uk","sentAt":"2005-08-29T18:20:13Z","receivedAt":"2005-08-29T18:20:13Z","isPatch":false,"sender":{"key":"rmk@arm.linux.org.uk","avatar":null},"body":"On Mon, Aug 29, 2005 at 11:02:38AM -0700, Linus Torvalds wrote:\n> The \"objects/info/alternates\" thing is an extension, which allows you to \n> have a partial object store, and point to the \"rest of it\", and still have \n> all the tools understand it and be able to parse the totality of it. So it \n> doesn't break or change old formats, it only allows a new one.\n\nAh, ok. I thought it was a new requirement, and I had visions of\nsimilar complaints about my repositories. Thanks for explaining\nthe situation.\n\n-- \nRussell King\n"},{"id":"7922","messageId":"17171.39652.237263.484079@cargo.ozlabs.ibm.com","threadId":"1689","inReplyTo":"Pine.LNX.4.58.0508291006440.3243@g5.osdl.org","subject":"Re: please pull ppc64-2.6.git","fromName":"Paul Mackerras","fromEmail":"paulus@samba.org","sentAt":"2005-08-29T23:31:48Z","receivedAt":"2005-08-29T23:31:48Z","isPatch":false,"sender":{"key":"paulus@samba.org","avatar":"https://avatars.githubusercontent.com/u/1606439?v=4"},"body":"Linus Torvalds writes:\n\n> \techo /pub/scm/linux/kernel/git/torvalds/linux-2.6/objects > objects/info/alternates\n\nDid that (s/linux-2.6/linux-2.6.git/ actually...)\n\nWhat can one put in the alternates file? Just an absolute path, or\ndoes a relative path or a URL work too?\n\n> or similar. That also makes gitweb able to show diffs etc, something it \n> can't do for a broken partial repository.\n\nGitweb still doesn't work; I think someone needs to update the git on\n*.kernel.org.\n\nPaul.\n"},{"id":"7927","messageId":"Pine.LNX.4.58.0508291706230.3243@g5.osdl.org","threadId":"1689","inReplyTo":"17171.39652.237263.484079@cargo.ozlabs.ibm.com","subject":"Re: please pull ppc64-2.6.git","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-08-30T00:20:21Z","receivedAt":"2005-08-30T00:20:21Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 30 Aug 2005, Paul Mackerras wrote:\n> \n> What can one put in the alternates file? Just an absolute path, or\n> does a relative path or a URL work too?\n\nOnly an absolute path.\n\nURL's fundamentally do not work, and relative paths end up being parsed as\nrelative to where-ever the user happens to be (and some commands will \n\"chdir()\" into the .git directory, while others will not).\n\nFor a raw git directory like the ones on master.kernel.org, with a\nrelative pathname (relative to the main git directory itself) the\n_pulling_ should actually happen to work, since that will be happening in\nthe xxx.git directory using \"GIT_DIR=.\".\n\nBut anything who uses GIT_DIR= from another directory\nwouldn't be able to use it. That probably includes gitweb, btw.\n\nSo only do the absolute ones. Maybe we should define some well-specified \nmeaning for relative ones, but it definitely isn't there now.\n\n> > or similar. That also makes gitweb able to show diffs etc, something it \n> > can't do for a broken partial repository.\n> \n> Gitweb still doesn't work; I think someone needs to update the git on\n> *.kernel.org.\n\nHmm, yes. kernel.org is at 0.99.4, and the objects/info/alternates thing \nwas done after that.\n\nIt's in 0.99.5, though, so next time kernel.org updates its git version, \nit will automagically start working.\n\n(And I use my own git installation, so it works fine for me, and I just \npulled the thing without problems).\n\n\t\t\tLinus\n"},{"id":"7961","messageId":"1125438048.9705.43.camel@localhost","threadId":"1689","inReplyTo":"Pine.LNX.4.58.0508291706230.3243@g5.osdl.org","subject":"Re: please pull ppc64-2.6.git","fromName":"Christian Meder","fromEmail":"chris@absolutegiganten.org","sentAt":"2005-08-30T21:40:48Z","receivedAt":"2005-08-30T21:40:48Z","isPatch":false,"sender":{"key":"chris@absolutegiganten.org","avatar":null},"body":"On Mon, 2005-08-29 at 17:20 -0700, Linus Torvalds wrote:\n> \n> On Tue, 30 Aug 2005, Paul Mackerras wrote:\n> > \n> > What can one put in the alternates file? Just an absolute path, or\n> > does a relative path or a URL work too?\n> \n> Only an absolute path.\n> \n> URL's fundamentally do not work, and relative paths end up being parsed as\n> relative to where-ever the user happens to be (and some commands will \n> \"chdir()\" into the .git directory, while others will not).\n\nIs alternates unthinkable with URLs (e.g. remote alternates) i.e. broken\nby design or thinkable but not there yet ?\n\nGreetings,\n\n\n\t\t\t\tChristian\n\n-- \nChristian Meder, email: chris@absolutegiganten.org\n\nThe Way-Seeking Mind of a tenzo is actualized \nby rolling up your sleeves.\n\n (Eihei Dogen Zenji)\n"},{"id":"7962","messageId":"7v8xyjhxkd.fsf@assigned-by-dhcp.cox.net","threadId":"1689","inReplyTo":"1125438048.9705.43.camel@localhost","subject":"Re: please pull ppc64-2.6.git","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-08-30T22:25:22Z","receivedAt":"2005-08-30T22:25:22Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Meder writes:\n\n> Is alternates unthinkable with URLs (e.g. remote alternates).\n\nIn order to read an object data, the low level core GIT layer\ndoes open()/mmap() of a file on the locally mounted filesystem.\nIt also does opendir()/readdir()/closedir() to find what's\navailable.\n\nIf you can arrange to automount via luserfs (or whatever; we do\nnot care much about particular implementation) so that it can do\nthese filesystem operations, git will happily use it.\n\nEven in that case, the alternate you would tell git should look\nlike a local filesystem path --- your automounting from remote\nspecified with URL for such a path would happen outside GIT.\n"},{"id":"7971","messageId":"20050831170852.2f549318.vsu@altlinux.ru","threadId":"1689","inReplyTo":"7v8xyjhxkd.fsf@assigned-by-dhcp.cox.net","subject":"Re: please pull ppc64-2.6.git","fromName":"Sergey Vlasov","fromEmail":"vsu@altlinux.ru","sentAt":"2005-08-31T13:08:52Z","receivedAt":"2005-08-31T13:08:52Z","isPatch":false,"sender":{"key":"vsu@altlinux.ru","avatar":"https://avatars.githubusercontent.com/u/616082?v=4"},"body":"On Tue, 30 Aug 2005 15:25:22 -0700 Junio C Hamano wrote:\n\n> Christian Meder writes:\n> \n> > Is alternates unthinkable with URLs (e.g. remote alternates).\n> \n> In order to read an object data, the low level core GIT layer\n> does open()/mmap() of a file on the locally mounted filesystem.\n> It also does opendir()/readdir()/closedir() to find what's\n> available.\n> \n> If you can arrange to automount via luserfs (or whatever; we do\n> not care much about particular implementation) so that it can do\n> these filesystem operations, git will happily use it.\n> \n> Even in that case, the alternate you would tell git should look\n> like a local filesystem path --- your automounting from remote\n> specified with URL for such a path would happen outside GIT.\n\nAll this means that currently there is no clean way to publish a partial\nGIT repository, unless you place it at the same server where the base\nrepository is located (and even in that case needing to use something\nlike \"echo /pub/scm/linux/kernel/git/torvalds/linux-2.6/objects >\nobjects/info/alternates\" looks like a horrible hack).\n\nWhat if we create a mapping layer from URLs to local mirrors of\ncorresponding repositories? Store this mapping, e.g., in\n~/.git-core/mirrors, then if git reads an URL instead of an absolute\nlocal path from objects/info/alternates, it will consult that file and\neither use the specified local mirror, or complain that a local copy of\nthe base repository is not available. The partial repository will then\ncontain no system-specific paths and could be published without\nproblems.\n"},{"id":"7978","messageId":"7vek8aar3b.fsf@assigned-by-dhcp.cox.net","threadId":"1689","inReplyTo":"20050831170852.2f549318.vsu@altlinux.ru","subject":"Re: please pull ppc64-2.6.git","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-08-31T18:39:20Z","receivedAt":"2005-08-31T18:39:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sergey Vlasov writes:\n\n> All this means that currently there is no clean way to publish a partial\n> GIT repository, unless you place it at the same server where the base\n> repository is located (and even in that case needing to use something\n> like \"echo /pub/scm/linux/kernel/git/torvalds/linux-2.6/objects >\n> objects/info/alternates\" looks like a horrible hack).\n\nThere has never been a way to publish a partial GIT repository.\nI personally am not convinced it is even a good idea for people\nto be able to do so.\n\n$GIT_OBJECT_DIRECTORY/info/alternates is a mechanism to solve\ncompletely different issue -- borrowing objects locally from\ndifferent object store to save space. This is only a local\n'repository organization' issue, just like packing objects or\nleaving them unpacked in a repository _should_ not make any\ndifference to people who are interacting with it.\n\nThe use of info/alternates is internal to the git aware server\nside, be it git-daemon and git-upload-pack when somebody fetches\nfrom it, or git-receive-pack when somebody pushes into it. The\nother end _should_ never have to care if the repository uses\ninfo/alternates, just like it _shouldn't_ matter if the\nrepository is packed or unpacked. Most importantly, for people\nwho are interacting with it, the repository has _everything_ it\nclaims to have by having pointers to head commits under refs/\nhierarchy, and is _not_ partial at all.\n\nYes, you could peek into the remote repository filesystem by\nother means, and even slurp the info/alternates file via rsync,\nbut as stated many times before, a lot of things that rsync\ndoes are by accident not by design.\n\nSome historical explanation may be helpful. info/alternates is\nmerely an improvement for existing ALTERNATE_OBJECT_DIRECTORIES\nmechanism. The latter required individual processes to tell git\nwhat other object stores to consult to find missing objects when\nworking on a repository whose object store is a partial object\ndatabase. This was clearly not per-process information; it was\nspecific to that partial object store, and that was the reason\ninfo/alternates was invented. The processes do not have to have\nthat environment variable; instead the necessary information is\nrecorded in the partial object store itself.\n\nI kept saying '_should_' because commit walkers would not\ncurrently understand alternates, just like they had trouble with\npacked repositories earlier. This _is_ a defect, but I\npersonally feel that this is a problem not worth solving.\n\nEarlier, when commit walkers did not understand packed\nrepositories, the only workaround was 'then do not pack your\npublic repository'. This was unacceptable from storage\nconsumption point of view, and it had to be solved in some way.\nDaniel solved it for 0.99.4.\n\nBut what info/alternates does is different. If you care about\ncommit walkers (and I certainly do), there are other means to\nsave space without using info/alternates; first try packing, and\nthen hard linking object files with whichever other repository\nyou would have placed in info/alternates, and you are done.\n\nNow you may argue that being able to publish truly partial\nrepository that does not have all the objects and depends on\nsome other repository, very likely to be on completely different\nserver (otherwise you would not be advocating for a non-local\npath such as URL) a worthwhile goal. I personally do not even\nbelieve that a repository relying on a foreign object store is a\ngood idea, and much less so if you are making that repository\npublicly available.\n\nI like your local-mirror-mapping idea very much, but I do not\nthink it is something that is specific to git. If you have such\na mechanism to map remote URL into local filesystem namespace\nand automount/mirror them on demand, I'd like to be able to use\nthat when I am browsing files in my Emacs (yes, there is\n'ange-ftp') or even 'cat' files from it. Its more useful that\nway than limiting its use to only when git wants to find missing\nobject files from its alternate object store.\n"},{"id":"7998","messageId":"7vpsrstj65.fsf@assigned-by-dhcp.cox.net","threadId":"1689","inReplyTo":"20050829184510.A20605@flint.arm.linux.org.uk","subject":"Re: please pull ppc64-2.6.git","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-09-02T00:20:02Z","receivedAt":"2005-09-02T00:20:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Russell King writes:\n\n> Is the expected filesystem layout documented somewhere online (_external_\n> to the source code) ?\n\nThere already was a sketchy description in git(7), at\n\n\nI've updated it a bit to describe the current status; please\nlook for \"File/Directory Structure\" section of the above URL.\n"}]}