{"thread":{"id":"16392","subject":"git and mtime","startedAt":"2008-11-19T11:37:52Z","lastAt":"2008-11-20T19:24:00Z","messageCount":33,"participants":["Roger Leigh","Matthias Kestenholz","Johannes Schindelin","Jakub Narebski","Arafangion","Matthieu Moy","Christian MICHON","Andreas Ericsson","Randal L. Schwartz","martin f krafft","Kyle Moffett","Samuel Tardieu","Daniel Barkalow","Joey Hess"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"96140","messageId":"20081119113752.GA13611@ravenclaw.codelibre.net","threadId":"16392","inReplyTo":null,"subject":"git and mtime","fromName":"Roger Leigh","fromEmail":"rleigh@codelibre.net","sentAt":"2008-11-19T11:37:52Z","receivedAt":"2008-11-19T11:37:52Z","isPatch":false,"sender":{"key":"rleigh@codelibre.net","avatar":null},"body":"Hi folks,\n\nI'm using git to store some generated files, as well as their sources.\n(This is in the context of Debian package development, where entire\nupstream release tarballs are injected into an upstream branch, with\nDebian releases merging the upstream branch, and adding the Debian\npackaging files.)\n\nThe upstream release tarballs contains files such as\n- yacc/lex code, and the corresponding generated sources\n- Docbook/XML code, and corresponding HTML/PDF documentation\n\nThese are provided by upstream so that end users don't need these tools\ninstalled (particularly docbook, since the toolchain is so flaky on\ndifferent systems). However, the fact that git isn't storing the\nmtime of the files confuses make, so it then tries to regenerate these\n(already up-to-date) files, and fails in the process since the tools\naren't available.\n\nWould it be possible for git to store the mtime of files in the tree?\n\nThis would make it possible to do this type of work in git, since it's\ncurrently a bit random as to whether it works or not. This only\nstarted when I upgraded to an amd64 architecture from powerpc32,\nI guess it's maybe using high-resolution timestamps.\n\n\nThanks,\nRoger\n\n\nP.S. The repo I'm working on here is at\n git://git.debian.org/git/collab-maint/gutenprint.git\n\n-- \n .''`. Roger Leigh\n : :' : Debian GNU/Linux http://people.debian.org/~rleigh/\n `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/\n `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.\n"},{"id":"96142","messageId":"C33ABF98-2E52-4928-BF79-CB3B6A8460DB@feinheit.ch","threadId":"16392","inReplyTo":"20081119113752.GA13611@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Matthias Kestenholz","fromEmail":"mk@feinheit.ch","sentAt":"2008-11-19T12:22:34Z","receivedAt":"2008-11-19T12:22:34Z","isPatch":false,"sender":{"key":"mk@feinheit.ch","avatar":"https://gravatar.com/avatar/f4f02a5336cf0e3d40b05498959e997f023cb5d8c83ab41545a3272268c67949?d=mp&s=160"},"body":"Hi,\n\nOn 19.11.2008, at 12:37, Roger Leigh wrote:\n\n> Hi folks,\n>\n> I'm using git to store some generated files, as well as their sources.\n> (This is in the context of Debian package development, where entire\n> upstream release tarballs are injected into an upstream branch, with\n> Debian releases merging the upstream branch, and adding the Debian\n> packaging files.)\n>\n> The upstream release tarballs contains files such as\n> - yacc/lex code, and the corresponding generated sources\n> - Docbook/XML code, and corresponding HTML/PDF documentation\n>\n> These are provided by upstream so that end users don't need these \n> tools\n> installed (particularly docbook, since the toolchain is so flaky on\n> different systems). However, the fact that git isn't storing the\n> mtime of the files confuses make, so it then tries to regenerate these\n> (already up-to-date) files, and fails in the process since the tools\n> aren't available.\n>\n> Would it be possible for git to store the mtime of files in the tree?\n>\n\nThis subject comes up from time to time, but the answer always\nstays the same: No. The trees are purely defined by their content, and\nthat's by design.\n\nIf you do not want to regenerate files that are already up-to-date,\nyou need multiple checkouts of the same repository.\n\n\n\nThanks,\nMatthias\n"},{"id":"96143","messageId":"alpine.DEB.1.00.0811191327210.30769@pacific.mpi-cbg.de","threadId":"16392","inReplyTo":"20081119113752.GA13611@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-11-19T12:31:57Z","receivedAt":"2008-11-19T12:31:57Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 19 Nov 2008, Roger Leigh wrote:\n\n> Would it be possible for git to store the mtime of files in the tree?\n\nNo, since this would wreck people's workflows:\n\n\t- compile in branch \"master\"\n\t- switch to branch \"topic\"\n\t- compile\n\t- switch back to branch \"master\"\n\nNow you _want_ files in \"master\" that were changed in \"topic\" to be \nrecompiled.\n\nThis is a quite common case.\n\nHowever, nothing hinders you having your own \".gitmtimes\" in the tree, and \na script people can use as a hook, which applies the mtimes to the files.\n\nCiao,\nDscho\n"},{"id":"96162","messageId":"1227098252.11370.8.camel@therock.nsw.bigpond.net.au","threadId":"16392","inReplyTo":"20081119113752.GA13611@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Arafangion","fromEmail":"thestar@fussycoder.id.au","sentAt":"2008-11-19T12:37:32Z","receivedAt":"2008-11-19T12:37:32Z","isPatch":false,"sender":{"key":"thestar@fussycoder.id.au","avatar":null},"body":"On Wed, 2008-11-19 at 11:37 +0000, Roger Leigh wrote:\n\n> different systems). However, the fact that git isn't storing the\n> mtime of the files confuses make, so it then tries to regenerate these\n> (already up-to-date) files, and fails in the process since the tools\n> aren't available.\n\nUnless I'm mistaken, I was under the impression that the reason why git\ndoesn't, and shouldn't do this is _because_ it confuses make.\n\nSuppose you've got two branches, and you check out the other branch,\nresulting in changes in 3 files. Should git go and modify the mtime for\nevery single file, and remove any file that isn't part of the repo (Such\nas generated object files)?\n\nIf it modifies the dates on every file, but doesn't remove the generated\nobject files, how does make handle that, as it'll likely generate some\nof the object files, but not all of them.\n\nIf it doesn't, but touches the files that changed, and the dates are now\nolder than the corresponding object files, make would fail to recompile\nthe project properly!\n\nThe only way this could work is if you never switch branches, which is\nquite limiting for git, and never check out an older revision, which is\nquite limiting for the RCS systems in general.\n\nYou should probably fix your build script, or add a hook script that\nsets the dates on the files in question manually, but the former\nsolution would be much better.\n"},{"id":"96153","messageId":"m3abbvlu9a.fsf@localhost.localdomain","threadId":"16392","inReplyTo":"20081119113752.GA13611@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-11-19T13:29:43Z","receivedAt":"2008-11-19T13:29:43Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Roger Leigh writes:\n\n> Would it be possible for git to store the mtime of files in the tree?\n> \n> This would make it possible to do this type of work in git, since it's\n> currently a bit random as to whether it works or not. This only\n> started when I upgraded to an amd64 architecture from powerpc32,\n> I guess it's maybe using high-resolution timestamps.\n\nI don't think it would be done as in core change at all, or at least\nsoon.\n\nYou can use Metastore, or some custom clean/smudge gitattribute\nfilters with something like Metastore (or etckeeper) to store extra\nmetadata about files in your tree.\n\nSee http://git.or.cz/gitwiki/InterfacesFrontendsAndTools\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"96164","messageId":"vpqhc63zrz2.fsf@bauges.imag.fr","threadId":"16392","inReplyTo":"1227098252.11370.8.camel@therock.nsw.bigpond.net.au","subject":"Re: git and mtime","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2008-11-19T14:54:25Z","receivedAt":"2008-11-19T14:54:25Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Arafangion writes:\n\n> You should probably fix your build script,\n\nccache should help:\n\nhttp://ccache.samba.org/\n\n-- \nMatthieu\n"},{"id":"96171","messageId":"46d6db660811190818r3aa2a392pda9106ac4a579cf0@mail.gmail.com","threadId":"16392","inReplyTo":"20081119113752.GA13611@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Christian MICHON","fromEmail":"christian.michon@gmail.com","sentAt":"2008-11-19T16:18:16Z","receivedAt":"2008-11-19T16:18:16Z","isPatch":false,"sender":{"key":"christian.michon@gmail.com","avatar":"https://gravatar.com/avatar/8a7c327b21187fbcab5c27640a49450eec72e0355dc292501197f27a5a744ec4?d=mp&s=160"},"body":"On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n> Would it be possible for git to store the mtime of files in the tree?\n>\n> This would make it possible to do this type of work in git, since it's\n> currently a bit random as to whether it works or not. This only\n> started when I upgraded to an amd64 architecture from powerpc32,\n> I guess it's maybe using high-resolution timestamps.\n>\n\nbeside the obvious answer it comes back often as a request, it is\npossible in theory to create a shell script which, for each file\npresent in the sandbox in the current branch, would find the mtime of\nthe last commit on that file (quite an expensive operation) and apply\nit.\n\nI had a need for this once, then lost interest since using git as it\nis is so much better than trying to mimic behaviour of old scm tools\nand makefiles.\n\nYou should store mostly content of source files. You should do a make\nin your first cloned repo at least once before committing anything to\nthe repo. That's what I did and I saved days...\n\n-- \nChristian\n--\nhttp://detaolb.sourceforge.net/, a linux distribution for Qemu with Git inside !\n"},{"id":"96203","messageId":"49252204.2070906@op5.se","threadId":"16392","inReplyTo":"C33ABF98-2E52-4928-BF79-CB3B6A8460DB@feinheit.ch","subject":"Re: git and mtime","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-20T08:38:28Z","receivedAt":"2008-11-20T08:38:28Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Matthias Kestenholz wrote:\n> Hi,\n> \n> On 19.11.2008, at 12:37, Roger Leigh wrote:\n> \n>> Hi folks,\n>>\n>> I'm using git to store some generated files, as well as their sources.\n>> (This is in the context of Debian package development, where entire\n>> upstream release tarballs are injected into an upstream branch, with\n>> Debian releases merging the upstream branch, and adding the Debian\n>> packaging files.)\n>>\n>> The upstream release tarballs contains files such as\n>> - yacc/lex code, and the corresponding generated sources\n>> - Docbook/XML code, and corresponding HTML/PDF documentation\n>>\n>> These are provided by upstream so that end users don't need these tools\n>> installed (particularly docbook, since the toolchain is so flaky on\n>> different systems). However, the fact that git isn't storing the\n>> mtime of the files confuses make, so it then tries to regenerate these\n>> (already up-to-date) files, and fails in the process since the tools\n>> aren't available.\n>>\n>> Would it be possible for git to store the mtime of files in the tree?\n>>\n> \n> This subject comes up from time to time, but the answer always\n> stays the same: No. The trees are purely defined by their content, and\n> that's by design.\n> \n> If you do not want to regenerate files that are already up-to-date,\n> you need multiple checkouts of the same repository.\n> \n\nOr a make-rule that touches the files you know are up to date. Since you\ncontrol the build environment, that's probably the simplest solution.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"96204","messageId":"49252256.7020001@op5.se","threadId":"16392","inReplyTo":"vpqhc63zrz2.fsf@bauges.imag.fr","subject":"Re: git and mtime","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-20T08:39:50Z","receivedAt":"2008-11-20T08:39:50Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Matthieu Moy wrote:\n> Arafangion writes:\n> \n>> You should probably fix your build script,\n> \n> ccache should help:\n> \n> http://ccache.samba.org/\n> \n\nNot for docbook/flex/yacc stuff, which is what was causing trouble.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"96213","messageId":"alpine.DEB.1.00.0811201132220.30769@pacific.mpi-cbg.de","threadId":"16392","inReplyTo":"vpqhc63zrz2.fsf@bauges.imag.fr","subject":"Re: git and mtime","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-11-20T10:34:08Z","receivedAt":"2008-11-20T10:34:08Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 19 Nov 2008, Matthieu Moy wrote:\n\n> Arafangion writes:\n> \n> > You should probably fix your build script,\n> \n> ccache should help:\n\nOnly if you _allow_ the problem that makes ccache necessary. Which we \ndon't.\n\nCiao,\nDscho\n\nP.S.: reminds me -- once again -- of the complicator's glove.\n"},{"id":"96214","messageId":"alpine.DEB.1.00.0811201134570.30769@pacific.mpi-cbg.de","threadId":"16392","inReplyTo":"46d6db660811190818r3aa2a392pda9106ac4a579cf0@mail.gmail.com","subject":"Re: git and mtime","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-11-20T10:35:45Z","receivedAt":"2008-11-20T10:35:45Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 19 Nov 2008, Christian MICHON wrote:\n\n> On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n> > Would it be possible for git to store the mtime of files in the tree?\n> >\n> > This would make it possible to do this type of work in git, since it's \n> > currently a bit random as to whether it works or not. This only \n> > started when I upgraded to an amd64 architecture from powerpc32, I \n> > guess it's maybe using high-resolution timestamps.\n> >\n> \n> beside the obvious answer it comes back often as a request, it is \n> possible in theory to create a shell script which, for each file present \n> in the sandbox in the current branch, would find the mtime of the last \n> commit on that file (quite an expensive operation) and apply it.\n> \n> I had a need for this once, then lost interest since using git as it is \n> is so much better than trying to mimic behaviour of old scm tools and \n> makefiles.\n\nI had a need like this, too, and solved it by teaching the build process \nto fall back to generated files if the tool to generate them was not \navailable.\n\nCiao,\nDscho\n"},{"id":"96217","messageId":"vpqwseypt1n.fsf@bauges.imag.fr","threadId":"16392","inReplyTo":"alpine.DEB.1.00.0811201132220.30769@pacific.mpi-cbg.de","subject":"Re: git and mtime","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2008-11-20T10:53:40Z","receivedAt":"2008-11-20T10:53:40Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Johannes Schindelin writes:\n\n> Hi,\n>\n> On Wed, 19 Nov 2008, Matthieu Moy wrote:\n>\n>> Arafangion writes:\n>> \n>> > You should probably fix your build script,\n>> \n>> ccache should help:\n>\n> Only if you _allow_ the problem that makes ccache necessary. Which we \n> don't.\n\nYou do, for some definition of \"problem\".\n\nmake\ngit checkout whatever\nmake\ngit checkout where-you-were-before\nmake # <--- this one\n\nThe last make is correct with plain git and plain make, but it can be\nslow. With a cache in your build system, it can just reuse the objects\ncreated during the first \"make\". It doesn't change correctness, only\nperformance.\n\n-- \nMatthieu\n"},{"id":"96227","messageId":"20081120112051.GB22787@ravenclaw.codelibre.net","threadId":"16392","inReplyTo":"49252204.2070906@op5.se","subject":"Re: git and mtime","fromName":"Roger Leigh","fromEmail":"rleigh@codelibre.net","sentAt":"2008-11-20T11:20:51Z","receivedAt":"2008-11-20T11:20:51Z","isPatch":false,"sender":{"key":"rleigh@codelibre.net","avatar":null},"body":"On Thu, Nov 20, 2008 at 09:38:28AM +0100, Andreas Ericsson wrote:\n> Matthias Kestenholz wrote:\n>> Hi,\n>>\n>> On 19.11.2008, at 12:37, Roger Leigh wrote:\n>>\n>>> I'm using git to store some generated files, as well as their sources.\n>>> (This is in the context of Debian package development, where entire\n>>> upstream release tarballs are injected into an upstream branch, with\n>>> Debian releases merging the upstream branch, and adding the Debian\n>>> packaging files.)\n>>>\n>>> The upstream release tarballs contains files such as\n>>> - yacc/lex code, and the corresponding generated sources\n>>> - Docbook/XML code, and corresponding HTML/PDF documentation\n>>>\n>>> Would it be possible for git to store the mtime of files in the tree?\n>>\n>> This subject comes up from time to time, but the answer always\n>> stays the same: No. The trees are purely defined by their content, and\n>> that's by design.\n>>\n>> If you do not want to regenerate files that are already up-to-date,\n>> you need multiple checkouts of the same repository.\n>\n> Or a make-rule that touches the files you know are up to date. Since you\n> control the build environment, that's probably the simplest solution.\n\nThis is the approach I'm currently taking, since it's simple and\ndoesn't require any tool changes. Ideally, I'd like to avoid\nsuch hackiness, though.\n\nI understand all the arguments I've seen in favour of not using the\nmtime of the files when checking out. They make sense. However,\nin some situations (such as this), they do not--git is breaking\nsomething that was previously working. In my case, I'm\ninjecting *release tarballs* into git, and the timestamps on the\nfiles really do matter. Regarding issues with branching and branch\nswitching, I always do builds from clean in this case.\n\nIf an option was added to git-checkout to restore mtimes, it need\nnot be the default, but git could record them on commit and then\nrestore them if asked /explicitly/.\n\nFor this, and some other uses I have in mind for git, it would be\ngreat if git could store some more components of the inode\nmetadata in the tree, such as:\n- mtime\n- user\n- group\n- full permissions\n- and also allow storage of the full range of file types (i.e.\n block, character, pipe, etc.)\n\nThis would allow git to be used as the basis for a complete\nfunctional versioned filesystem (which I'd like to use for my\nlightweight virtualisation tool, schroot, which currently\nuses LVM snapshots for this purpose).\n\n\nRegards,\nRoger\n\n-- \n .''`. Roger Leigh\n : :' : Debian GNU/Linux http://people.debian.org/~rleigh/\n `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/\n `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.\n"},{"id":"96226","messageId":"20081120112708.GC22787@ravenclaw.codelibre.net","threadId":"16392","inReplyTo":"46d6db660811190818r3aa2a392pda9106ac4a579cf0@mail.gmail.com","subject":"Re: git and mtime","fromName":"Roger Leigh","fromEmail":"rleigh@codelibre.net","sentAt":"2008-11-20T11:27:08Z","receivedAt":"2008-11-20T11:27:08Z","isPatch":false,"sender":{"key":"rleigh@codelibre.net","avatar":null},"body":"On Wed, Nov 19, 2008 at 05:18:16PM +0100, Christian MICHON wrote:\n> On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n> > Would it be possible for git to store the mtime of files in the tree?\n> >\n> > This would make it possible to do this type of work in git, since it's\n> > currently a bit random as to whether it works or not. This only\n> > started when I upgraded to an amd64 architecture from powerpc32,\n> > I guess it's maybe using high-resolution timestamps.\n> >\n> \n> beside the obvious answer it comes back often as a request, it is\n> possible in theory to create a shell script which, for each file\n> present in the sandbox in the current branch, would find the mtime of\n> the last commit on that file (quite an expensive operation) and apply\n> it.\n\nSurely this is only expensive because you're not already storing the\ninformation in the tree; if it was there, it would be (relatively)\ncheap? You could even compare the old and new trees to see if you\nneeded to touch a file at all.\n\n> You should store mostly content of source files. You should do a make\n> in your first cloned repo at least once before committing anything to\n> the repo. That's what I did and I saved days...\n\nExcept in this case I'm storing the content of *tarballs* (along with\npristine-tar). I'm committing exactly what's in the tarball with\nno changes (this is a requirement). I can't change the source prior\nto commit.\n\n\nRegards,\nRoger\n\n-- \n .''`. Roger Leigh\n : :' : Debian GNU/Linux http://people.debian.org/~rleigh/\n `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/\n `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.\n"},{"id":"96228","messageId":"49255CBA.4030709@op5.se","threadId":"16392","inReplyTo":"20081120112051.GB22787@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-20T12:48:58Z","receivedAt":"2008-11-20T12:48:58Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Roger Leigh wrote:\n> On Thu, Nov 20, 2008 at 09:38:28AM +0100, Andreas Ericsson wrote:\n>> Matthias Kestenholz wrote:\n>>> Hi,\n>>>\n>>> On 19.11.2008, at 12:37, Roger Leigh wrote:\n>>>\n>>>> I'm using git to store some generated files, as well as their sources.\n>>>> (This is in the context of Debian package development, where entire\n>>>> upstream release tarballs are injected into an upstream branch, with\n>>>> Debian releases merging the upstream branch, and adding the Debian\n>>>> packaging files.)\n>>>>\n>>>> The upstream release tarballs contains files such as\n>>>> - yacc/lex code, and the corresponding generated sources\n>>>> - Docbook/XML code, and corresponding HTML/PDF documentation\n>>>>\n>>>> Would it be possible for git to store the mtime of files in the tree?\n>>> This subject comes up from time to time, but the answer always\n>>> stays the same: No. The trees are purely defined by their content, and\n>>> that's by design.\n>>>\n>>> If you do not want to regenerate files that are already up-to-date,\n>>> you need multiple checkouts of the same repository.\n>> Or a make-rule that touches the files you know are up to date. Since you\n>> control the build environment, that's probably the simplest solution.\n> \n> This is the approach I'm currently taking, since it's simple and\n> doesn't require any tool changes. Ideally, I'd like to avoid\n> such hackiness, though.\n> \n> I understand all the arguments I've seen in favour of not using the\n> mtime of the files when checking out. They make sense. However,\n> in some situations (such as this), they do not--git is breaking\n> something that was previously working. In my case, I'm\n> injecting *release tarballs* into git, and the timestamps on the\n> files really do matter. Regarding issues with branching and branch\n> switching, I always do builds from clean in this case.\n> \n> If an option was added to git-checkout to restore mtimes, it need\n> not be the default, but git could record them on commit and then\n> restore them if asked /explicitly/.\n> \n\nYou can. The way to ask explicitly right now is to write hooks\nthat implement the functionality you want. It's not as easy as\nsetting a config value, but since you'd have to write the patch\nto do that anyways (and it's likely it will get dropped), you'd\nbe better off writing some hooks and submitting them as contrib\nstuff.\n\n> For this, and some other uses I have in mind for git, it would be\n> great if git could store some more components of the inode\n> metadata in the tree, such as:\n> - mtime\n> - user\n> - group\n> - full permissions\n> - and also allow storage of the full range of file types (i.e.\n> block, character, pipe, etc.)\n> \n> This would allow git to be used as the basis for a complete\n> functional versioned filesystem (which I'd like to use for my\n> lightweight virtualisation tool, schroot, which currently\n> uses LVM snapshots for this purpose).\n> \n\nI believe someone else has done some work along the way of\nturning git into complete-with-metadata backupsystem before.\nGoogle might prove beneficial.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"96231","messageId":"492560C5.5070308@op5.se","threadId":"16392","inReplyTo":"20081120112708.GC22787@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-20T13:06:13Z","receivedAt":"2008-11-20T13:06:13Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Roger Leigh wrote:\n> On Wed, Nov 19, 2008 at 05:18:16PM +0100, Christian MICHON wrote:\n>> On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n>>> Would it be possible for git to store the mtime of files in the tree?\n>>>\n>>> This would make it possible to do this type of work in git, since it's\n>>> currently a bit random as to whether it works or not. This only\n>>> started when I upgraded to an amd64 architecture from powerpc32,\n>>> I guess it's maybe using high-resolution timestamps.\n>>>\n>> beside the obvious answer it comes back often as a request, it is\n>> possible in theory to create a shell script which, for each file\n>> present in the sandbox in the current branch, would find the mtime of\n>> the last commit on that file (quite an expensive operation) and apply\n>> it.\n> \n> Surely this is only expensive because you're not already storing the\n> information in the tree; if it was there, it would be (relatively)\n> cheap?\n\nNo, it's because git is *snapshot* based and doesn't care about anything\nbut contents. Storing filestate information in the tree would be a\nbackwards incompatible change that would require a major version change.\n\nCaring about meta-data the way you mean it would mean that\n\n git add foo.c; git commit -m \"kapooie\"; touch foo.c; git status\n\nwould show \"foo.c\" as modified. How sane is that? Or should we introduce\na new concept for altered metadata only? \"metafied\"? So what do we do\nwhen the next user whizzes along and wants support for full acl's? And\nwhat do we do when Windows (or some other bizarre system) add some sort\nof extension so we have to have different types of ACL support on both\nsystems? Kablooie and welcome to interoperability hell.\n\n> You could even compare the old and new trees to see if you\n> needed to touch a file at all.\n> \n\nWe already do that by matching the SHA1 hash for the index entries.\nOnly content that is actually different between to branches are altered\nupon checkout (which is why it's so damn fast when you're using topic-\nbranches properly).\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"96234","messageId":"86iqqizgng.fsf@blue.stonehenge.com","threadId":"16392","inReplyTo":"20081120112708.GC22787@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Randal L. Schwartz","fromEmail":"merlyn@stonehenge.com","sentAt":"2008-11-20T13:11:15Z","receivedAt":"2008-11-20T13:11:15Z","isPatch":false,"sender":{"key":"merlyn@stonehenge.com","avatar":"https://gravatar.com/avatar/dc528d210743ff0333e6213f9ee7b33b23f1b7bc1f3c5a8c2d819074ecd7ab19?d=mp&s=160"},"body":">>>>> \"Roger\" == Roger Leigh writes:\n\nRoger> Except in this case I'm storing the content of *tarballs* (along with\nRoger> pristine-tar). I'm committing exactly what's in the tarball with\nRoger> no changes (this is a requirement). I can't change the source prior\nRoger> to commit.\n\nIf you're not doing distributed source code development, why are you using\ngit? It's hard to be angry at a screwdriver for not pounding in nails\nproperly.\n\nSounds like you want rsync or something.\n\n-- \nRandal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n \nSmalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.\nSee http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion\n"},{"id":"96235","messageId":"49256223.1090206@op5.se","threadId":"16392","inReplyTo":"49255CBA.4030709@op5.se","subject":"Re: git and mtime","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-20T13:12:03Z","receivedAt":"2008-11-20T13:12:03Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Andreas Ericsson wrote:\n> Roger Leigh wrote:\n>> For this, and some other uses I have in mind for git, it would be\n>> great if git could store some more components of the inode\n>> metadata in the tree, such as:\n>> - mtime\n>> - user\n>> - group\n>> - full permissions\n>> - and also allow storage of the full range of file types (i.e.\n>> block, character, pipe, etc.)\n>>\n>> This would allow git to be used as the basis for a complete\n>> functional versioned filesystem (which I'd like to use for my\n>> lightweight virtualisation tool, schroot, which currently\n>> uses LVM snapshots for this purpose).\n>>\n> \n> I believe someone else has done some work along the way of\n> turning git into complete-with-metadata backupsystem before.\n> Google might prove beneficial.\n> \n\nAlthough now that I come to think of it, storing \"user\" and\n\"group\" made it near-enough totally useless for anything a\nuser had created as the repos hardly ever could be shared.\n\nI'll say it again; Hooks can be written to handle this.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"96238","messageId":"20081120132107.GA27571@piper.oerlikon.madduck.net","threadId":"16392","inReplyTo":"20081119113752.GA13611@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"martin f krafft","fromEmail":"madduck@madduck.net","sentAt":"2008-11-20T13:21:07Z","receivedAt":"2008-11-20T13:21:07Z","isPatch":false,"sender":{"key":"madduck@madduck.net","avatar":null},"body":"also sprach Roger Leigh [2008.11.19.1237 +0100]:\n> These are provided by upstream so that end users don't need these tools\n> installed (particularly docbook, since the toolchain is so flaky on\n> different systems). However, the fact that git isn't storing the\n> mtime of the files confuses make, so it then tries to regenerate these\n> (already up-to-date) files, and fails in the process since the tools\n> aren't available.\n\nI don't get it. Why are end users running make in the first place?\nWhy aren't those in the build-dependencies?\n\n-- \nmartin | http://madduck.net/ | http://two.sentenc.es/\n \nit is better to have loft and lost\nthan to never have loft at all.\n -- groucho marx\n \nspamtraps: madduck.bogus@madduck.net\n"},{"id":"96240","messageId":"20081120133546.GA6023@codelibre.net","threadId":"16392","inReplyTo":"20081120132107.GA27571@piper.oerlikon.madduck.net","subject":"Re: git and mtime","fromName":"Roger Leigh","fromEmail":"rleigh@codelibre.net","sentAt":"2008-11-20T13:35:46Z","receivedAt":"2008-11-20T13:35:46Z","isPatch":false,"sender":{"key":"rleigh@codelibre.net","avatar":null},"body":"On Thu, Nov 20, 2008 at 02:21:07PM +0100, martin f krafft wrote:\n> also sprach Roger Leigh [2008.11.19.1237 +0100]:\n> > These are provided by upstream so that end users don't need these tools\n> > installed (particularly docbook, since the toolchain is so flaky on\n> > different systems). However, the fact that git isn't storing the\n> > mtime of the files confuses make, so it then tries to regenerate these\n> > (already up-to-date) files, and fails in the process since the tools\n> > aren't available.\n> \n> I don't get it. Why are end users running make in the first place?\n> Why aren't those in the build-dependencies?\n\nBy end user, I mean person downloading and building the sources.\n\nThey are optional build depdendencies. They are provided pre-built,\nand won't be rebuilt unless they get outdated. In the release\ntarball, the timestamps are correct, ensuring this never happens.\nWhen checking out with git, the timestamps are incorrect, and it\nattempts to rebuild something that's *already built*.\n\n\nRegards,\nRoger\n\n-- \n .''`. Roger Leigh\n : :' : Debian GNU/Linux http://people.debian.org/~rleigh/\n `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/\n `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.\n"},{"id":"96241","messageId":"20081120134055.GB6023@codelibre.net","threadId":"16392","inReplyTo":"86iqqizgng.fsf@blue.stonehenge.com","subject":"Re: git and mtime","fromName":"Roger Leigh","fromEmail":"rleigh@codelibre.net","sentAt":"2008-11-20T13:40:55Z","receivedAt":"2008-11-20T13:40:55Z","isPatch":false,"sender":{"key":"rleigh@codelibre.net","avatar":null},"body":"On Thu, Nov 20, 2008 at 05:11:15AM -0800, Randal L. Schwartz wrote:\n> >>>>> \"Roger\" == Roger Leigh writes:\n> \n> Roger> Except in this case I'm storing the content of *tarballs* (along with\n> Roger> pristine-tar). I'm committing exactly what's in the tarball with\n> Roger> no changes (this is a requirement). I can't change the source prior\n> Roger> to commit.\n> \n> If you're not doing distributed source code development, why are you using\n> git? It's hard to be angry at a screwdriver for not pounding in nails\n> properly.\n\nErr, it *is* being used for distributed development... of Debian\npackaging. We track upstream releases on one branch, merge this\nperiodically onto the master branch containing the Debian packaging\ninfrastructure, and also have other bits such as a\ncontinually-rebased patches branch to generate quilt patch series\nfrom. I think you'll find we do actually need to use git.\n\n> Sounds like you want rsync or something.\n\nI think not! Perhaps if you read my original mail, you might\nunderstand the reasoning behind this (whether you consider that\nvalid reasoning or not is another matter).\n\n\nRegards,\nRoger\n\n-- \n .''`. Roger Leigh\n : :' : Debian GNU/Linux http://people.debian.org/~rleigh/\n `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/\n `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.\n"},{"id":"96248","messageId":"20081120135956.GA29789@piper.oerlikon.madduck.net","threadId":"16392","inReplyTo":"20081120133546.GA6023@codelibre.net","subject":"Re: git and mtime","fromName":"martin f krafft","fromEmail":"madduck@madduck.net","sentAt":"2008-11-20T13:59:56Z","receivedAt":"2008-11-20T13:59:56Z","isPatch":false,"sender":{"key":"madduck@madduck.net","avatar":null},"body":"also sprach Roger Leigh [2008.11.20.1435 +0100]:\n> By end user, I mean person downloading and building the sources.\n> \n> They are optional build depdendencies. They are provided pre-built,\n> and won't be rebuilt unless they get outdated. In the release\n> tarball, the timestamps are correct, ensuring this never happens.\n> When checking out with git, the timestamps are incorrect, and it\n> attempts to rebuild something that's *already built*.\n\nI know you will hate me, but I think the solution here is to fix the\ntoolchain and make those build dependencies required.\n\n-- \nmartin | http://madduck.net/ | http://two.sentenc.es/\n \n\"first get your facts; then you can distort them at your leisure.\"\n -- mark twain\n \nspamtraps: madduck.bogus@madduck.net\n"},{"id":"96247","messageId":"alpine.DEB.1.00.0811201506390.30769@pacific.mpi-cbg.de","threadId":"16392","inReplyTo":"20081120133546.GA6023@codelibre.net","subject":"Re: git and mtime","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-11-20T14:07:46Z","receivedAt":"2008-11-20T14:07:46Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 20 Nov 2008, Roger Leigh wrote:\n\n> They are optional build depdendencies. They are provided pre-built, and \n> won't be rebuilt unless they get outdated. In the release tarball, the \n> timestamps are correct, ensuring this never happens. When checking out \n> with git, the timestamps are incorrect, and it attempts to rebuild \n> something that's *already built*.\n\nI'll try just one more time. Why don't you teach your build process to \ncheck if the generated files can be generated, and if not, fall back to \nthe committed ones?\n\nCiao,\nDscho\n"},{"id":"96249","messageId":"20081120141533.GC6023@codelibre.net","threadId":"16392","inReplyTo":"492560C5.5070308@op5.se","subject":"Re: git and mtime","fromName":"Roger Leigh","fromEmail":"rleigh@codelibre.net","sentAt":"2008-11-20T14:15:33Z","receivedAt":"2008-11-20T14:15:33Z","isPatch":false,"sender":{"key":"rleigh@codelibre.net","avatar":null},"body":"On Thu, Nov 20, 2008 at 02:06:13PM +0100, Andreas Ericsson wrote:\n> Roger Leigh wrote:\n>> On Wed, Nov 19, 2008 at 05:18:16PM +0100, Christian MICHON wrote:\n>>> On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n>>>> Would it be possible for git to store the mtime of files in the tree?\n>>>>\n>>>> This would make it possible to do this type of work in git, since it's\n>>>> currently a bit random as to whether it works or not. This only\n>>>> started when I upgraded to an amd64 architecture from powerpc32,\n>>>> I guess it's maybe using high-resolution timestamps.\n>>>>\n>>> beside the obvious answer it comes back often as a request, it is\n>>> possible in theory to create a shell script which, for each file\n>>> present in the sandbox in the current branch, would find the mtime of\n>>> the last commit on that file (quite an expensive operation) and apply\n>>> it.\n>>\n>> Surely this is only expensive because you're not already storing the\n>> information in the tree; if it was there, it would be (relatively)\n>> cheap?\n>\n> No, it's because git is *snapshot* based and doesn't care about anything\n> but contents. Storing filestate information in the tree would be a\n> backwards incompatible change that would require a major version change.\n\nIt's not strictly true that it's only caring about contents. The\ncontents are of course in the blobs, but the tree is already\neffectively storing inode data, since it's a directory of\nfilenames/subtrees, just one that only cares to store the\npermissions part of the total inode data.\n\nI understand that git stored the permissions tacked onto the hash;\nwould it be feasable to tack on the other bits as well.\nIf I understand correctly, it's binary encoded in the pack format,\nand that would require updating the format to hold the additional\ndata?\n\n> Caring about meta-data the way you mean it would mean that\n>\n> git add foo.c; git commit -m \"kapooie\"; touch foo.c; git status\n>\n> would show \"foo.c\" as modified. How sane is that?\n\nI've never come close to suggesting we do anything so insane.\n\nWhat I am suggesting is that on add/commit, the inode metadata\nbe recorded in the tree (like we already store perms), so that\nit can be (**optionally**) reused/restored on checkout.\n\nWhether it's stored in the tree or not is a separate concern from\nwhether to *use* it or not. For most situations, it won't be\nuseful, as has been made quite clear from all of the replies, and I\ndon't disagree with this. However, for some, the ability to have\nthis information to hand to make use of would be invaluable.\n\n\nThere have been quite a few suggestions to look into using hooks,\nand I'll investigate this. However, I do have some concerns\nabout *where* I would store this \"extended tree\" data, since it\nis implicitly tied to a single tree object, and I wouldn't\nwant to store it directly as content.\n\n\nRegards,\nRoger\n\n-- \n .''`. Roger Leigh\n : :' : Debian GNU/Linux http://people.debian.org/~rleigh/\n `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/\n `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.\n"},{"id":"96251","messageId":"20081120142217.GD6023@codelibre.net","threadId":"16392","inReplyTo":"alpine.DEB.1.00.0811201506390.30769@pacific.mpi-cbg.de","subject":"Re: git and mtime","fromName":"Roger Leigh","fromEmail":"rleigh@codelibre.net","sentAt":"2008-11-20T14:22:17Z","receivedAt":"2008-11-20T14:22:17Z","isPatch":false,"sender":{"key":"rleigh@codelibre.net","avatar":null},"body":"On Thu, Nov 20, 2008 at 03:07:46PM +0100, Johannes Schindelin wrote:\n> Hi,\n> \n> On Thu, 20 Nov 2008, Roger Leigh wrote:\n> \n> > They are optional build depdendencies. They are provided pre-built, and \n> > won't be rebuilt unless they get outdated. In the release tarball, the \n> > timestamps are correct, ensuring this never happens. When checking out \n> > with git, the timestamps are incorrect, and it attempts to rebuild \n> > something that's *already built*.\n> \n> I'll try just one more time. Why don't you teach your build process to \n> check if the generated files can be generated, and if not, fall back to \n> the committed ones?\n\nWell, it's definitely not a good idea to try rebuilding when the tools\naren't available, and I'll update the Makefiles to only attempt a\nrebuild when this is the case. So yes, making the build a bit more\nintelligent is definitely something to do. However, this is really\na separate issue, since the repo dates back eight years, and I don't\nwant to break older stuff. This will only fix things for the future.\n\n\nRegards,\nRoger\n\n-- \n .''`. Roger Leigh\n : :' : Debian GNU/Linux http://people.debian.org/~rleigh/\n `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/\n `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.\n"},{"id":"96256","messageId":"49257949.4070308@op5.se","threadId":"16392","inReplyTo":"20081120141533.GC6023@codelibre.net","subject":"Re: git and mtime","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-20T14:50:49Z","receivedAt":"2008-11-20T14:50:49Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Roger Leigh wrote:\n> On Thu, Nov 20, 2008 at 02:06:13PM +0100, Andreas Ericsson wrote:\n>> Roger Leigh wrote:\n>>> On Wed, Nov 19, 2008 at 05:18:16PM +0100, Christian MICHON wrote:\n>>>> On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n>>>>> Would it be possible for git to store the mtime of files in the tree?\n>>>>>\n>>>>> This would make it possible to do this type of work in git, since it's\n>>>>> currently a bit random as to whether it works or not. This only\n>>>>> started when I upgraded to an amd64 architecture from powerpc32,\n>>>>> I guess it's maybe using high-resolution timestamps.\n>>>>>\n>>>> beside the obvious answer it comes back often as a request, it is\n>>>> possible in theory to create a shell script which, for each file\n>>>> present in the sandbox in the current branch, would find the mtime of\n>>>> the last commit on that file (quite an expensive operation) and apply\n>>>> it.\n>>> Surely this is only expensive because you're not already storing the\n>>> information in the tree; if it was there, it would be (relatively)\n>>> cheap?\n>> No, it's because git is *snapshot* based and doesn't care about anything\n>> but contents. Storing filestate information in the tree would be a\n>> backwards incompatible change that would require a major version change.\n> \n> It's not strictly true that it's only caring about contents. The\n> contents are of course in the blobs, but the tree is already\n> effectively storing inode data, since it's a directory of\n> filenames/subtrees, just one that only cares to store the\n> permissions part of the total inode data.\n> \n> I understand that git stored the permissions tacked onto the hash;\n> would it be feasable to tack on the other bits as well.\n\nNo, that would break backwards compatibility with cross-repo\ntransfers.\n\n> If I understand correctly, it's binary encoded in the pack format,\n> and that would require updating the format to hold the additional\n> data?\n> \n>> Caring about meta-data the way you mean it would mean that\n>>\n>> git add foo.c; git commit -m \"kapooie\"; touch foo.c; git status\n>>\n>> would show \"foo.c\" as modified. How sane is that?\n> \n> I've never come close to suggesting we do anything so insane.\n> \n> What I am suggesting is that on add/commit, the inode metadata\n> be recorded in the tree (like we already store perms), so that\n> it can be (**optionally**) reused/restored on checkout.\n> \n> Whether it's stored in the tree or not is a separate concern from\n> whether to *use* it or not. For most situations, it won't be\n> useful, as has been made quite clear from all of the replies, and I\n> don't disagree with this. However, for some, the ability to have\n> this information to hand to make use of would be invaluable.\n> \n\nThen write a hook for it. You agree that for most users this will be\ntotally insane, and yet you request that it's added in a place where\neveryone will have to pay the performance/diskspace penalty for it\nbut only a handful will get any benefits. That's patently absurd.\nEspecially since there are such easy workarounds that you can put in\nplace yourself instead.\n\n> \n> There have been quite a few suggestions to look into using hooks,\n> and I'll investigate this. However, I do have some concerns\n> about *where* I would store this \"extended tree\" data, since it\n> is implicitly tied to a single tree object, and I wouldn't\n> want to store it directly as content.\n> \n\nStore it as a blob targeted by a lightweight tag named\n\"metadata.$sha1\" and you'll have the easiest time in the world when\nwriting the hooks. Also, the tags won't be propagated by default,\nwhich is a good thing since your timestamps/uid's whatever almost\ncertainly will not work well on other developers repositories.\n\nThat's what I'd do anyways.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"96262","messageId":"20081120151925.GE6023@codelibre.net","threadId":"16392","inReplyTo":"49257949.4070308@op5.se","subject":"Re: git and mtime","fromName":"Roger Leigh","fromEmail":"rleigh@codelibre.net","sentAt":"2008-11-20T15:19:25Z","receivedAt":"2008-11-20T15:19:25Z","isPatch":false,"sender":{"key":"rleigh@codelibre.net","avatar":null},"body":"On Thu, Nov 20, 2008 at 03:50:49PM +0100, Andreas Ericsson wrote:\n> Roger Leigh wrote:\n>> On Thu, Nov 20, 2008 at 02:06:13PM +0100, Andreas Ericsson wrote:\n>>> Roger Leigh wrote:\n>>>> On Wed, Nov 19, 2008 at 05:18:16PM +0100, Christian MICHON wrote:\n>>>>> On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n>>>>>> Would it be possible for git to store the mtime of files in the tree?\n>>>>>>\n>>>>>> This would make it possible to do this type of work in git, since it's\n>>>>>> currently a bit random as to whether it works or not. This only\n>>>>>> started when I upgraded to an amd64 architecture from powerpc32,\n>>>>>> I guess it's maybe using high-resolution timestamps.\n>>>>>>\n>>> Caring about meta-data the way you mean it would mean that\n>>>\n>>> git add foo.c; git commit -m \"kapooie\"; touch foo.c; git status\n>>>\n>>> would show \"foo.c\" as modified. How sane is that?\n>>\n>> I've never come close to suggesting we do anything so insane.\n>>\n>> What I am suggesting is that on add/commit, the inode metadata\n>> be recorded in the tree (like we already store perms), so that\n>> it can be (**optionally**) reused/restored on checkout.\n>>\n>> Whether it's stored in the tree or not is a separate concern from\n>> whether to *use* it or not. For most situations, it won't be\n>> useful, as has been made quite clear from all of the replies, and I\n>> don't disagree with this. However, for some, the ability to have\n>> this information to hand to make use of would be invaluable.\n>>\n>\n> Then write a hook for it. You agree that for most users this will be\n> totally insane, and yet you request that it's added in a place where\n> everyone will have to pay the performance/diskspace penalty for it\n> but only a handful will get any benefits. That's patently absurd.\n\nThe cost is tiny. The extra space would be smaller than a single\nSHA1 hash.\n\n> Especially since there are such easy workarounds that you can put in\n> place yourself instead.\n\n\n>> There have been quite a few suggestions to look into using hooks,\n>> and I'll investigate this. However, I do have some concerns\n>> about *where* I would store this \"extended tree\" data, since it\n>> is implicitly tied to a single tree object, and I wouldn't\n>> want to store it directly as content.\n>\n> Store it as a blob targeted by a lightweight tag named\n> \"metadata.$sha1\" and you'll have the easiest time in the world when\n> writing the hooks. Also, the tags won't be propagated by default,\n> which is a good thing since your timestamps/uid's whatever almost\n> certainly will not work well on other developers repositories.\n\nAnd yet the fact that it won't propagate makes it totally useless:\nall the other people using the repo won't get the extra metadata\nthat will prevent build failures. Having the extra data locally\nis nice, but not exactly what I'd call a solution. The whole point\nof what I want is to have it as an integral part of the repo.\n\n\nRegards,\nRoger\n\n-- \n .''`. Roger Leigh\n : :' : Debian GNU/Linux http://people.debian.org/~rleigh/\n `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/\n `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.\n"},{"id":"96266","messageId":"f73f7ab80811200733u46ba63f3h11bf13cc476a5f9f@mail.gmail.com","threadId":"16392","inReplyTo":"20081120151925.GE6023@codelibre.net","subject":"Re: git and mtime","fromName":"Kyle Moffett","fromEmail":"kyle@moffetthome.net","sentAt":"2008-11-20T15:33:46Z","receivedAt":"2008-11-20T15:33:46Z","isPatch":false,"sender":{"key":"kyle@moffetthome.net","avatar":null},"body":"On Thu, Nov 20, 2008 at 10:19 AM, Roger Leigh wrote:\n> And yet the fact that it won't propagate makes it totally useless:\n> all the other people using the repo won't get the extra metadata\n> that will prevent build failures. Having the extra data locally\n> is nice, but not exactly what I'd call a solution. The whole point\n> of what I want is to have it as an integral part of the repo.\n\nEasiest way is typically something like this in the makefile:\n\ndocbook_version = $(shell docbook2man --version 2>/dev/null)\nifneq \"$docbook_version\",\"\"\n\nmymanpage.1:\n ## Real docbook build rules here\n\nelse\n\nmymanpage.1:\n if [ -e $@ ]; then \\\n echo \"No 'docbook' installed, using pregenerated man\npages\" >&2 ; \\\n else \\\n echo \"Pregenerated manpages are missing and no docbook\nfound!\" >&2 ; \\\n exit 1 ; \\\n fi\n\nendif\n\nSuch stuff will take an order of magnitude less time than trying to\npatch GIT to preserve metadata that most projects don't want\npreserved. You may also find it's easier to just comment out the\ndocumentation build rules if you are always guaranteeing that the docs\nhave been compiled.\n\nCheers,\nKyle Moffett\n"},{"id":"96267","messageId":"49258421.5090308@op5.se","threadId":"16392","inReplyTo":"20081120151925.GE6023@codelibre.net","subject":"Re: git and mtime","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-11-20T15:37:05Z","receivedAt":"2008-11-20T15:37:05Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Roger Leigh wrote:\n> On Thu, Nov 20, 2008 at 03:50:49PM +0100, Andreas Ericsson wrote:\n>> Roger Leigh wrote:\n>>> On Thu, Nov 20, 2008 at 02:06:13PM +0100, Andreas Ericsson wrote:\n>>>> Roger Leigh wrote:\n>>>>> On Wed, Nov 19, 2008 at 05:18:16PM +0100, Christian MICHON wrote:\n>>>>>> On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n>>>>>>> Would it be possible for git to store the mtime of files in the tree?\n>>>>>>>\n>>>>>>> This would make it possible to do this type of work in git, since it's\n>>>>>>> currently a bit random as to whether it works or not. This only\n>>>>>>> started when I upgraded to an amd64 architecture from powerpc32,\n>>>>>>> I guess it's maybe using high-resolution timestamps.\n>>>>>>>\n>>>> Caring about meta-data the way you mean it would mean that\n>>>>\n>>>> git add foo.c; git commit -m \"kapooie\"; touch foo.c; git status\n>>>>\n>>>> would show \"foo.c\" as modified. How sane is that?\n>>> I've never come close to suggesting we do anything so insane.\n>>>\n>>> What I am suggesting is that on add/commit, the inode metadata\n>>> be recorded in the tree (like we already store perms), so that\n>>> it can be (**optionally**) reused/restored on checkout.\n>>>\n>>> Whether it's stored in the tree or not is a separate concern from\n>>> whether to *use* it or not. For most situations, it won't be\n>>> useful, as has been made quite clear from all of the replies, and I\n>>> don't disagree with this. However, for some, the ability to have\n>>> this information to hand to make use of would be invaluable.\n>>>\n>> Then write a hook for it. You agree that for most users this will be\n>> totally insane, and yet you request that it's added in a place where\n>> everyone will have to pay the performance/diskspace penalty for it\n>> but only a handful will get any benefits. That's patently absurd.\n> \n> The cost is tiny. The extra space would be smaller than a single\n> SHA1 hash.\n> \n>> Especially since there are such easy workarounds that you can put in\n>> place yourself instead.\n> \n> \n>>> There have been quite a few suggestions to look into using hooks,\n>>> and I'll investigate this. However, I do have some concerns\n>>> about *where* I would store this \"extended tree\" data, since it\n>>> is implicitly tied to a single tree object, and I wouldn't\n>>> want to store it directly as content.\n>> Store it as a blob targeted by a lightweight tag named\n>> \"metadata.$sha1\" and you'll have the easiest time in the world when\n>> writing the hooks. Also, the tags won't be propagated by default,\n>> which is a good thing since your timestamps/uid's whatever almost\n>> certainly will not work well on other developers repositories.\n> \n> And yet the fact that it won't propagate makes it totally useless:\n> all the other people using the repo won't get the extra metadata\n> that will prevent build failures. Having the extra data locally\n> is nice, but not exactly what I'd call a solution. The whole point\n> of what I want is to have it as an integral part of the repo.\n> \n\nThen make it signed tags and ship them along.\n\nOr do this properly and simply put in your buildsystem that some\ntargets never need to be rebuilt. That's (by far) the simplest\nsolution.\n\nOn a sidenote, I fail to see how the pre-generated stuff can avoid\ngetting updated unless also the sources for that stuff was updated,\nin which case either of the following is true:\na) You really do need to rebuild, because upstream fucked up.\nb) The pre-generated stuff should *also* be checked out and get new\n timestamps.\n\nEither way, to me it sounds like your buildsystem needs some love.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"96269","messageId":"2008-11-20-16-56-33+trackit+sam@rfc1149.net","threadId":"16392","inReplyTo":"20081120135956.GA29789@piper.oerlikon.madduck.net","subject":"Re: git and mtime","fromName":"Samuel Tardieu","fromEmail":"sam@rfc1149.net","sentAt":"2008-11-20T15:56:33Z","receivedAt":"2008-11-20T15:56:33Z","isPatch":false,"sender":{"key":"sam@rfc1149.net","avatar":"https://avatars.githubusercontent.com/u/44656?v=4"},"body":">>>>> \"martin\" == martin f krafft writes:\n\nmartin> I know you will hate me, but I think the solution here is to\nmartin> fix the toolchain and make those build dependencies required.\n\nI agree with martin here. Your planned solution of not rebuilding the\nfiles if the tools are not present may lead to serious problems if the\nuser modifies the source files and happens not to have the tools\naround.\n\nMoreover, requiring the build dependencies would allow you to drop the\ngenerated files from the repository and rebuild them in your packaging\n(source or binary) process.\n\n Sam\n-- \nSamuel Tardieu -- sam@rfc1149.net -- http://www.rfc1149.net/\n"},{"id":"96276","messageId":"alpine.LNX.1.00.0811201223300.19665@iabervon.org","threadId":"16392","inReplyTo":"20081120112708.GC22787@ravenclaw.codelibre.net","subject":"Re: git and mtime","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2008-11-20T17:59:08Z","receivedAt":"2008-11-20T17:59:08Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Thu, 20 Nov 2008, Roger Leigh wrote:\n\n> On Wed, Nov 19, 2008 at 05:18:16PM +0100, Christian MICHON wrote:\n> > On Wed, Nov 19, 2008 at 12:37 PM, Roger Leigh wrote:\n> > > Would it be possible for git to store the mtime of files in the tree?\n> > >\n> > > This would make it possible to do this type of work in git, since it's\n> > > currently a bit random as to whether it works or not. This only\n> > > started when I upgraded to an amd64 architecture from powerpc32,\n> > > I guess it's maybe using high-resolution timestamps.\n> > >\n> > \n> > beside the obvious answer it comes back often as a request, it is\n> > possible in theory to create a shell script which, for each file\n> > present in the sandbox in the current branch, would find the mtime of\n> > the last commit on that file (quite an expensive operation) and apply\n> > it.\n> \n> Surely this is only expensive because you're not already storing the\n> information in the tree; if it was there, it would be (relatively)\n> cheap? You could even compare the old and new trees to see if you\n> needed to touch a file at all.\n> \n> > You should store mostly content of source files. You should do a make\n> > in your first cloned repo at least once before committing anything to\n> > the repo. That's what I did and I saved days...\n> \n> Except in this case I'm storing the content of *tarballs* (along with\n> pristine-tar). I'm committing exactly what's in the tarball with\n> no changes (this is a requirement). I can't change the source prior\n> to commit.\n\nCan you store the tarballs in the repository, instead of the contents of \nthe tarballs? The tarballs will contain the dates you want, and you can \nobviously get tar to set the timestamps the way you want. (Then you add a \nhigher-level Makefile that knows how to unpack the tarball to a directory, \nmaintaining the timestamps, patch anything you're changing, and run make \nin that directory.)\n\nThat is to say, from your perspective, the sources include the upstream \ndistributed tarballs, but the individual files in upstream tarballs aren't \nsource files for you, since you can't (by policy) modify them (within the \npristine tarball). If you want to change the sources of the packaged \nproject, you add a patch file to do it, rather than simply changing the \nsource (which, as you say, you're required not to do).\n\nGit really wants to store the inputs to your workflow, each of which might\nchange independently. That's why the files in your work tree have \ntimestamps based on when they came to be in your work tree (get set to the \ncurrent time whenever git puts different content there, and leaves them \nunchanged if their contents don't change when moving from commit to \ncommit). The \"sources\" in your workflow are a different set of files from \nthe sources in the project, and git really wants *your* repository to \nmatch *your* workflow and not the workflow of the upstream project, when \nyou're acting as a packager rather than an upstream developer.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"96278","messageId":"A12D78B0-7CC0-4877-AB91-A547AE1A7EC8@feinheit.ch","threadId":"16392","inReplyTo":"20081120151925.GE6023@codelibre.net","subject":"Re: git and mtime","fromName":"Matthias Kestenholz","fromEmail":"mk@feinheit.ch","sentAt":"2008-11-20T18:36:29Z","receivedAt":"2008-11-20T18:36:29Z","isPatch":false,"sender":{"key":"mk@feinheit.ch","avatar":"https://gravatar.com/avatar/f4f02a5336cf0e3d40b05498959e997f023cb5d8c83ab41545a3272268c67949?d=mp&s=160"},"body":"\nOn 20.11.2008, at 16:19, Roger Leigh wrote:\n\n>>\n>> Then write a hook for it. You agree that for most users this will be\n>> totally insane, and yet you request that it's added in a place where\n>> everyone will have to pay the performance/diskspace penalty for it\n>> but only a handful will get any benefits. That's patently absurd.\n>\n> The cost is tiny. The extra space would be smaller than a single\n> SHA1 hash.\n\nNo, the cost is huge. The SHA-1 for the tree with _exactly the same\ncontents_ will be different, just because f.e you applied a patch one\nsecond earlier than I did, and that's completely insane. Git is purely\na content tracker as has been said numerous times on this mailing\nlist, and that is for good reasons. If the tree entries change just\nbecause some timestamps are different, the CPU time needed to\ngenerate a diff will grow by a big amount of time.\n\nAtempts to add additional information to the basic git objects have\nfailed several times, and yours will probably fail too since there are\nnumerous reasons why you do _not_ want a timestamp in the tree\n_and_ there are several workarounds for your problem, which at\nleast in my opinion seem much less insane than adding timestamps...\n\n\n>> Especially since there are such easy workarounds that you can put in\n>> place yourself instead.\n>\n"},{"id":"96281","messageId":"20081120192400.GB25604@kodama.kitenet.net","threadId":"16392","inReplyTo":"alpine.LNX.1.00.0811201223300.19665@iabervon.org","subject":"Re: git and mtime","fromName":"Joey Hess","fromEmail":"joey@kitenet.net","sentAt":"2008-11-20T19:24:00Z","receivedAt":"2008-11-20T19:24:00Z","isPatch":false,"sender":{"key":"joey@kitenet.net","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"On Thu, 20 Nov 2008, Roger Leigh wrote:\n> Except in this case I'm storing the content of *tarballs* (along with\n> pristine-tar). I'm committing exactly what's in the tarball with\n> no changes (this is a requirement). I can't change the source prior\n> to commit.\n\nNote that pristine-tar will work no matter what the mtimes or other file\nmetadata are, none of that affects generation of deltas or regeneration\nof tarballs from them.\n\nAlso, the source you commit does not really have to be identical to\nwhat's in the tarball. (Despite what it may say in the man page. ;-)\nA larger delta will be generated if something is different.\n\nSo, three possible approaches:\n\n1. Run make or whatever you need to do before running pristine-tar,\n and put up with a larger delta.\n\n2. Before building, you could use pristine-tar to extract the original\n tarball, and then have a program examine that tarball, and reset the\n mtimes in your build tree to match the mtimes of files in it.\n (Or you could duplicate the info with metastore -m, which could be\n restored quicker.)\n\n3. Store uncompressed tarballs in git, so that they will pack\n efficiently, and use pristine-gz to regenerate the pristine .tar.gz.\n Only mentioned because this could be more space efficient than option\n #1, if the pristine-tar deltas get too large.\n\n-- \nsee shy jo\n"}]}