{"thread":{"id":"3134","subject":"Caching directories","startedAt":"2006-01-24T04:20:23Z","lastAt":"2006-01-25T07:29:32Z","messageCount":3,"participants":["Pavel Roskin","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"15077","messageId":"1138076423.15105.36.camel@dv","threadId":"3134","inReplyTo":null,"subject":"Caching directories","fromName":"Pavel Roskin","fromEmail":"proski@gnu.org","sentAt":"2006-01-24T04:20:23Z","receivedAt":"2006-01-24T04:20:23Z","isPatch":false,"sender":{"key":"proski@gnu.org","avatar":null},"body":"Hello!\n\nI'm thinking of moving cg-clean functionality to git.  After having\nswitched to StGIT, it's the last cogito command I'm still using.  I\nthink git can go it much better, since it's a recursive command\ntraversing the whole repository.\n\nTo be safe and useful, the new command should distinguish between\ntracked and untracked directories.  Untracked files in tracked\ndirectories are usually the first target for cleaning, as they are\nmostly automatic backups and temporary files.  Untracked directories are\nmore likely candidates to be preserved, as they can hold external\nsources, build output, extensive test data etc.\n\ncg-clean considers a directory untracked if it has no cached files in\nit.  This carries a significant speed penalty, even if not coded in\nbash.\n\nMaybe it's time to start caching directories in git?  I mean,\ndirectories corresponding to tree objects could have their stats\nrecorded in the cache.  This would allow to distinguish between tracked\nand untracked directories without scanning them recursively.\n\n-- \nRegards,\nPavel Roskin\n"},{"id":"15102","messageId":"7vvew8zwfe.fsf@assigned-by-dhcp.cox.net","threadId":"3134","inReplyTo":"1138076423.15105.36.camel@dv","subject":"Re: Caching directories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-25T05:52:21Z","receivedAt":"2006-01-25T05:52:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Pavel Roskin <proski@gnu.org> writes:\n\n> Maybe it's time to start caching directories in git?  I mean,\n> directories corresponding to tree objects could have their stats\n> recorded in the cache.  This would allow to distinguish between tracked\n> and untracked directories without scanning them recursively.\n\nI do not understand the above logic.  \n\nGiven a directory path, finding out if the directory has\nsomething tracked in it is an O(log n) operation in the current\nindex that does not \"cache directory\".  Your message implies\nthat you feel we could use the index file to list \"untracked\ndirectories\" without recursively scanning the directory tree,\nbut to me, the only way to do that is to record a new directory\nin the index file every time somebody (either Makefile or the\nuser) creates a junk directory.  That does not make much sense\nto me, so I am probably misreading what you really meant.\n\nI've been meaning to explore the possibility of recording 0{40}\nSHA1 in the index file to mean \"I do not want to place anything\non this path when I write the index out to a tree yet, but keep\nan eye on the path in the working tree for me\".\n\nYou can consider this as an \"intent to add\"; for example, with\nsuch an index file, you could do something like this:\n\n\t$ git update-index --intent-to-add foo\n\nThis would record 0^{40} SHA1 with the 0 mode in the index at\n\"foo\".  Then:\n\n        $ git diff-files -p\n        diff --git a/foo b/foo\n        new file mode 100644\n        index 0000000..6690023\n        --- /dev/null\n        +++ b/foo\n        @@ -0,0 +1,24 @@\n        +...\n        +....\n        ...\n\nThe index has heard about it, but does not actually have it, so\nit reports an addition.  Since we currently do not have such,\nafter a \"git add\", the index not just has heard about it, but\nactually has it, and as a consequence, there is no way to get\n\"new file\" out of diff-files.\n\n\t$ git diff-index --cached HEAD ;# nothing\n\nThe index has heard about it, but does not have it.  If the HEAD\ncommit did not have it, diff-index --cached would report\nnothing.\n\n        $ git diff-index HEAD\n        diff --git a/foo b/foo\n        new file mode 100644\n        index 0000000..6690023\n        --- /dev/null\n        +++ b/foo\n        @@ -0,0 +1,24 @@\n        +...\n        +....\n        ...\n\nThe index has heard about it, and without --cached it uses the\nworking tree file, so if HEAD did not have it you would see \"new\nfile\" out of diff-index.  If the comparison were with a tree\nthat has \"foo\" in it, diff-index using an index that does not\nhave \"foo\" would not say anything in the current system, but\nwith \"intent to add\", it would say \"Oh, your index knows about\nit so let me look in the working tree; ah, you have something\nthere.  Let me compare it with the version in the tree in\nquestion\".\n\nOne interesting thing the \"intent to add\" entries would do is\nthis:\n\n        $ git diff-files --abbrev foo\n        :000000 100644 0000... 0000...\n\nNote that two \"0^{40}\" mean quite different things.  The one on\nthe LHS means \"we've heard about it but we do not have it\".  On\nthe other hand, the one on the RHS means \"we do not cache the\nSHA1 --- go look at the working tree file\".\n\nWe might want to represent the existence of a tree that does not\nhave anything under using 0^{40} as well.  Or it might be better\nkept out of the main index entries list, and become extra data\njust like we have been discussing how to store \"bind\" entries in\nthe \"Subprojects\" thread.  I dunno.\n\nI have no idea what 'clean' does, so would not comment on that\npart of your message.\n"},{"id":"15106","messageId":"1138174172.27366.116.camel@dv","threadId":"3134","inReplyTo":"7vvew8zwfe.fsf@assigned-by-dhcp.cox.net","subject":"Re: Caching directories","fromName":"Pavel Roskin","fromEmail":"proski@gnu.org","sentAt":"2006-01-25T07:29:32Z","receivedAt":"2006-01-25T07:29:32Z","isPatch":false,"sender":{"key":"proski@gnu.org","avatar":null},"body":"On Tue, 2006-01-24 at 21:52 -0800, Junio C Hamano wrote:\n> Pavel Roskin <proski@gnu.org> writes:\n> \n> > Maybe it's time to start caching directories in git?  I mean,\n> > directories corresponding to tree objects could have their stats\n> > recorded in the cache.  This would allow to distinguish between tracked\n> > and untracked directories without scanning them recursively.\n> \n> I do not understand the above logic.  \n> \n> Given a directory path, finding out if the directory has\n> something tracked in it is an O(log n) operation in the current\n> index that does not \"cache directory\".  Your message implies\n> that you feel we could use the index file to list \"untracked\n> directories\" without recursively scanning the directory tree,\n> but to me, the only way to do that is to record a new directory\n> in the index file every time somebody (either Makefile or the\n> user) creates a junk directory.  That does not make much sense\n> to me, so I am probably misreading what you really meant.\n\nSorry, it looks like my post was based on incorrect assumptions.  The\nnew --directory option to git-ls-files seems to be exactly what I want.\nIt allowed me to simplify cg-clean immensely.  Further simplification\nwill be possible once the support for .gitignore in parent directories\nis fixed.\n\n> I have no idea what 'clean' does, so would not comment on that\n> part of your message.\n\nIt means removing untracked files and directories.\n\n-- \nRegards,\nPavel Roskin\n"}]}