{"thread":{"id":"8609","subject":"Feature request: thin checkout","startedAt":"2007-06-15T08:53:46Z","lastAt":"2007-06-15T10:44:47Z","messageCount":3,"participants":["linux@horizon.com","Rogan Dawes","Johannes Sixt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"45127","messageId":"20070615085346.8027.qmail@science.horizon.com","threadId":"8609","inReplyTo":null,"subject":"Feature request: thin checkout","fromName":"","fromEmail":"linux@horizon.com","sentAt":"2007-06-15T08:53:46Z","receivedAt":"2007-06-15T08:53:46Z","isPatch":false,"sender":{"key":"linux@horizon.com","avatar":null},"body":"Git packs so well that it's very common for the unpacked source to be much\nlarger than the history in .git.  The linux-kernel archive is a prime example.\n\nI've also started using git-svn (awesome tool, BTW) and have discovered\nthe impressive disk space costs associated with SVN's tags/ directories\nif I actually want to download the full history.\n\nIf you have multiple cloned repositories on one system, git can share\nthe history, but the working directory problem is exacerbated.\n(Disk is cheap, but the RAM to cache it is limited.)\n\nThis got me thinking...\nWouldn't it be nice if there were a way to tell git-update-index and\ngit-checkout index that certain directories are not in the working\ndirectory, but don't worry.  Just pretend they exist and match the index.\n\nThen I could mark much of arch/* as \"don't bother\" and save a pile of\ndisk space per working directory.\n\nThis would be a little bit annoying if I tried to merge two branches with\nconflicts in a \"masked\" part of the tree (well, it would create the index\nentries, but I'd have no way to resolve the conflict), but I think that's\na matter of Don't Do That.\n\nA slightly more flexible (but confusing?) option would be to mark parts\nof the tree as \"don't commit deletion\".  That is, within named sections\nof the tree:\n- Missing files in the working directory are assumed unchanged from\n  the index.  (Perhaps unless you explicitly git-add them.)\n- Files that don't already exist aren't checked out from the index.\n  (Unless explicitly named in a git-checkout operation.)\n... but you could have a \"selective checkout\" in some directories.\nE.g. in the kernel, you could include a stub Makefile, but omit\nthe .c files for file systems you don't need.\n\n(And if we're really sneaky, teach the linux kernel Makefile how to check\nout code when features are enabled.  That would address a longstanding\ncomplaint about the size of the linux kernel source tree.  It's a\nbit trickier than default make rules for getting <foo> from RCS/foo,v\nbecause got doesn't provide a signle file that make(1) can look for,\nbut something's probably possible.)\n\nThat could also handle merges... the \"check out the file with conflict\nmarkers\" operation could be unconditional if there are conflicts.\n\n\nThe multiple-git-repositories issue could be handled by hard-linking\nthe working directory files together (assuming your editor knows how to\nunlink when changing them) using information easily available in the\nindex files.  Git could even detect and complain if a two files that\nmismatched their index entries were hard-linked together.\n\nBut for the git-svn case where you have a tags/ directory full of old\ncopies of files, hard-linking is of limited use if most files changed\nbetween tags.  Here, just being able to say \"don't bother populating\nthat part of the working directory\" would be very nice.\n\n\nDoes this make sense to anyone else?\n"},{"id":"45128","messageId":"4672608C.7090908@dawes.za.net","threadId":"8609","inReplyTo":"20070615085346.8027.qmail@science.horizon.com","subject":"Re: Feature request: thin checkout","fromName":"Rogan Dawes","fromEmail":"lists@dawes.za.net","sentAt":"2007-06-15T09:49:00Z","receivedAt":"2007-06-15T09:49:00Z","isPatch":false,"sender":{"key":"lists@dawes.za.net","avatar":null},"body":"linux@horizon.com wrote:\n> Git packs so well that it's very common for the unpacked source to be much\n> larger than the history in .git.  The linux-kernel archive is a prime example.\n> \n> I've also started using git-svn (awesome tool, BTW) and have discovered\n> the impressive disk space costs associated with SVN's tags/ directories\n> if I actually want to download the full history.\n> \n> If you have multiple cloned repositories on one system, git can share\n> the history, but the working directory problem is exacerbated.\n> (Disk is cheap, but the RAM to cache it is limited.)\n> \n> This got me thinking...\n> Wouldn't it be nice if there were a way to tell git-update-index and\n> git-checkout index that certain directories are not in the working\n> directory, but don't worry.  Just pretend they exist and match the index.\n> \n\nI think that update-index is able to do (some of) this already:\n\n$ man git-update-index\n\nSYNOPSIS\n        git-update-index\n\n                     [--cacheinfo <mode> <object> <file>]*\n\n\n        --cacheinfo <mode> <object> <path>\n               Directly insert the specified info into the index.\n\n\nUSING --CACHEINFO OR --INFO-ONLY\n        --cacheinfo is used to register a file that is not in the current\n        working directory. This is useful for minimum-checkout merging.\n\n        To pretend you have a file with mode and sha1 at path, say:\n\n        $ git-update-index --cacheinfo mode sha1 path\n        --info-only is used to register files without placing them in the\n        object database. This is useful for status-only repositories.\n\n        Both --cacheinfo and --info-only behave similarly: the index is \nupdated\n        but the object database isn't. --cacheinfo is useful when the \nobject is\n        in the database but the file isn't available locally. --info-only is\n        useful when the file is available, but you do not wish to update the\n        object database.\n\nAt any rate, it looks like some of the infrastructure is existing \nalready, even if the complete solution doesn't exist.\n\nI *guess* it might even be as simple as maintaining a list of \n\"uncheckedout files with mode and sha\" in the .git directory, and \nmerging that with what has actually been checked out when updating the \nindex.\n\ni.e.\n\n$ git checkout master:src/drivers\n\nGet the <tree> object for master. Step through each entry. If the \nrequested path falls under the entry, recurse into it, checking out the \nrequired files, otherwise write the <tree/file> info into \n.git/partialcheckout.\n\nHack, hack, hack in src/drivers.\n\nWhen you want to check what part of the tree is dirty, check if \n.git/partialcheckouts exists. If it does, read through each entry, \ncomparing them to the index. Then, for the entries that are not in \npartialcheckout, but are in the index, actually go to the filesystem to \ncheck stat for each file.\n\nNot quite sure how to handle something like:\n\n$ git checkout master:src/drivers/scsi\n$ git checkout master:src/drivers/usb\n\nI guess one would have to trim entries from .git/partialcheckout as they \nare actually fully checked out.\n\nRogan\n"},{"id":"45130","messageId":"46726D9F.C4801AA@eudaptics.com","threadId":"8609","inReplyTo":"20070615085346.8027.qmail@science.horizon.com","subject":"Re: Feature request: thin checkout","fromName":"Johannes Sixt","fromEmail":"j.sixt@eudaptics.com","sentAt":"2007-06-15T10:44:47Z","receivedAt":"2007-06-15T10:44:47Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"linux@horizon.com wrote:\n> Wouldn't it be nice if there were a way to tell git-update-index and\n> git-checkout index that certain directories are not in the working\n> directory, but don't worry.  Just pretend they exist and match the index.\n> \n> Then I could mark much of arch/* as \"don't bother\" and save a pile of\n> disk space per working directory.\n\nCurrently, directories are not registered in the index. For this reason,\nempty directories cannot be versioned. One day, will want to support\nthis, and for this purpose, directories must also be registered in the\nindex. Once this infrastructure is in place, the \"don't bother\" flag\nwill be a no-brainer, methinks.\n\n-- Hannes\n"}]}