{"thread":{"id":"25946","subject":"git reset and ctime","startedAt":"2010-12-03T21:36:37Z","lastAt":"2010-12-07T15:14:12Z","messageCount":8,"participants":["ghazel@gmail.com","Jonathan Nieder","Junio C Hamano","Drew Northup"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"157269","messageId":"AANLkTimyYTup+PqJFJ+2g-tVwWXA2bxTT3noonEuKBSu@mail.gmail.com","threadId":"25946","inReplyTo":null,"subject":"git reset and ctime","fromName":"","fromEmail":"ghazel@gmail.com","sentAt":"2010-12-03T21:36:37Z","receivedAt":"2010-12-03T21:36:37Z","isPatch":false,"sender":{"key":"ghazel@gmail.com","avatar":null},"body":"Hi,\n\nI've encountered a strange issue where \"git reset --hard\" insists on\n\"Checking out files ...\" when all that is changed is the ctime on\nthese files. My deploy process (capistrano) maintains a cached copy of\na git repo, which it fetches, resets, and then hardlinks files from\nwhen a deploy occurs ( https://github.com/37signals/fast_remote_cache\n). The hardlinking step is meant to save the time of copying the file.\nbut hardlinking changes the ctime of the source files. That causes git\nreset to re-check out the files when the next deploy occurs, which is\nquite time-consuming. Some helpful people on #git showed that \"git\nupdate-index --refresh\" before the git reset prevents this behavior,\nbut I wonder why that is needed at all.\n\nShould git reset be performing whatever \"git update-index --refresh\"\nis doing? Certainly in this case it would result in a vast speed\nimprovement.\n\n-Greg\n"},{"id":"157285","messageId":"20101204005131.GB15906@burratino","threadId":"25946","inReplyTo":"AANLkTimyYTup+PqJFJ+2g-tVwWXA2bxTT3noonEuKBSu@mail.gmail.com","subject":"Re: git reset and ctime","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-04T00:51:32Z","receivedAt":"2010-12-04T00:51:32Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Greg,\n\nghazel@gmail.com wrote:\n\n> I've encountered a strange issue where \"git reset --hard\" insists on\n> \"Checking out files ...\" when all that is changed is the ctime\n\nThere is a performance trade-off.  Refreshing the index requires\nreading+hashing the existing file if the stat information changed;\nthis could be faster or slower than blindly overwriting depending on\nthe situation.\n\nThat said, I have no strong objection to an implicit refresh in \"git\nreset\" (performance-sensitive scripts should be using read-tree\ndirectly anyway).  Have you tried making that change to\nbuiltin/reset.c?  How does it perform in practice?\n\n>              My deploy process (capistrano) maintains a cached copy of\n> a git repo, which it fetches, resets, and then hardlinks files from\n> when a deploy occurs ( https://github.com/37signals/fast_remote_cache\n> ). The hardlinking step is meant to save the time of copying the file.\n> but hardlinking changes the ctime of the source files.\n\nInteresting.  Setting \"[core] trustctime = false\" in the repository\nconfiguration could be a good solution (no performance downside I can\nthink of).\n\nHope that helps,\nJonathan\n"},{"id":"157287","messageId":"AANLkTi=MSf+0rkBmvuJQBaSs1-G_gFahnLvt44fnwAwf@mail.gmail.com","threadId":"25946","inReplyTo":"20101204005131.GB15906@burratino","subject":"Re: git reset and ctime","fromName":"","fromEmail":"ghazel@gmail.com","sentAt":"2010-12-04T01:39:08Z","receivedAt":"2010-12-04T01:39:08Z","isPatch":false,"sender":{"key":"ghazel@gmail.com","avatar":null},"body":"On Fri, Dec 3, 2010 at 4:51 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:\n> ghazel@gmail.com wrote:\n>\n>> I've encountered a strange issue where \"git reset --hard\" insists on\n>> \"Checking out files ...\" when all that is changed is the ctime\n>\n> There is a performance trade-off.  Refreshing the index requires\n> reading+hashing the existing file if the stat information changed;\n> this could be faster or slower than blindly overwriting depending on\n> the situation.\n>\n> That said, I have no strong objection to an implicit refresh in \"git\n> reset\" (performance-sensitive scripts should be using read-tree\n> directly anyway).  Have you tried making that change to\n> builtin/reset.c?  How does it perform in practice?\n\nI did not make the modifications to reset.c, I just ran the refresh\nbefore reset:\n\nSo originally:\n\n$ time git reset --hard <rev>\nChecking out files: 100% (2772/2772), done.\n\nreal    0m5.328s\nuser    0m2.539s\nsys     0m2.542s\n\nas opposed to:\n\n$ time git update-index --refresh\n\nreal    0m1.236s\nuser    0m1.024s\nsys     0m0.201s\n\n$ time git reset --hard <rev>\n\nreal    0m0.055s\nuser    0m0.011s\nsys     0m0.041s\n\n>>              My deploy process (capistrano) maintains a cached copy of\n>> a git repo, which it fetches, resets, and then hardlinks files from\n>> when a deploy occurs ( https://github.com/37signals/fast_remote_cache\n>> ). The hardlinking step is meant to save the time of copying the file.\n>> but hardlinking changes the ctime of the source files.\n>\n> Interesting.  Setting \"[core] trustctime = false\" in the repository\n> configuration could be a good solution (no performance downside I can\n> think of).\n\nThis is a very useful suggestion. I do not see a case where ctime\nwould be valuable to me. Is it really valuable to other people? What\nis the trade-off?\n\n-Greg\n"},{"id":"157288","messageId":"20101204014751.GA18629@burratino","threadId":"25946","inReplyTo":"AANLkTi=MSf+0rkBmvuJQBaSs1-G_gFahnLvt44fnwAwf@mail.gmail.com","subject":"Re: git reset and ctime","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-04T01:47:51Z","receivedAt":"2010-12-04T01:47:51Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"ghazel@gmail.com wrote:\n> On Fri, Dec 3, 2010 at 4:51 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:\n>> ghazel@gmail.com wrote:\n\n>>>              My deploy process (capistrano) maintains a cached copy of\n>>> a git repo, which it fetches, resets, and then hardlinks files from\n>>> when a deploy occurs ( https://github.com/37signals/fast_remote_cache\n>>> ). The hardlinking step is meant to save the time of copying the file.\n>>> but hardlinking changes the ctime of the source files.\n>>\n>> Interesting.  Setting \"[core] trustctime = false\" in the repository\n>> configuration could be a good solution (no performance downside I can\n>> think of).\n>\n> This is a very useful suggestion. I do not see a case where ctime\n> would be valuable to me. Is it really valuable to other people? What\n> is the trade-off?\n\nSome reading for a rainy day :): [1] and surrounding discussion.\n\nShort answer: I think the main purpose is catching worktree corruption\n(e.g., if rsync screws up).\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/89370/focus=89993\n"},{"id":"157290","messageId":"7v62vaxoen.fsf@alter.siamese.dyndns.org","threadId":"25946","inReplyTo":"20101204005131.GB15906@burratino","subject":"Re: git reset and ctime","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-12-04T02:28:00Z","receivedAt":"2010-12-04T02:28:00Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> That said, I have no strong objection to an implicit refresh in \"git\n> reset\" (performance-sensitive scripts should be using read-tree\n> directly anyway).  Have you tried making that change to\n> builtin/reset.c?  How does it perform in practice?\n\nI would be more worried about correctness impact such a patch may make\nwhen the index contains unmerged entries.\n"},{"id":"157410","messageId":"1291657037.30271.17.camel@drew-northup.unet.maine.edu","threadId":"25946","inReplyTo":"20101204005131.GB15906@burratino","subject":"Re: git reset and ctime","fromName":"Drew Northup","fromEmail":"drew.northup@maine.edu","sentAt":"2010-12-06T17:37:17Z","receivedAt":"2010-12-06T17:37:17Z","isPatch":false,"sender":{"key":"drew.northup@maine.edu","avatar":"https://avatars.githubusercontent.com/u/18331571?v=4"},"body":"\nOn Fri, 2010-12-03 at 18:51 -0600, Jonathan Nieder wrote:\n> Hi Greg,\n> \n> ghazel@gmail.com wrote:\n> \n> > I've encountered a strange issue where \"git reset --hard\" insists on\n> > \"Checking out files ...\" when all that is changed is the ctime\n> \n> There is a performance trade-off.  Refreshing the index requires\n> reading+hashing the existing file if the stat information changed;\n> this could be faster or slower than blindly overwriting depending on\n> the situation.\n\n> >              My deploy process (capistrano) maintains a cached copy of\n> > a git repo, which it fetches, resets, and then hardlinks files from\n> > when a deploy occurs ( https://github.com/37signals/fast_remote_cache\n> > ). The hardlinking step is meant to save the time of copying the file.\n> > but hardlinking changes the ctime of the source files.\n> \n> Interesting.  Setting \"[core] trustctime = false\" in the repository\n> configuration could be a good solution (no performance downside I can\n> think of).\n\nIt is worth noting that many file-based backup systems which do \"online\"\nbackups (such as in use where I work) restore the atime by default at\nthe expense of the ctime (logic being that the atime may have had value\nand the ctime changes either way--which may or may not be true) on unix\nstyle filesystems. While many of the git command-line things I have run\nseem to figure this out ok, it drives gitk nuts. As far as I am\nconcerned this is a small price to pay for a solid daily-updated backup\nof my machine(s) to be available. I haven't yet put \"git reset\" of any\nsort to use (obviously I just haven't been breaking enough things yet),\nbut I suspect that it would react in a similar way.\n\n-- \n-Drew Northup N1XIM\n   AKA RvnPhnx on OPN\n________________________________________________\n\"As opposed to vegetable or mineral error?\"\n-John Pescatore, SANS NewsBites Vol. 12 Num. 59\n"},{"id":"157419","messageId":"20101206175102.GC6272@burratino","threadId":"25946","inReplyTo":"1291657037.30271.17.camel@drew-northup.unet.maine.edu","subject":"Re: git reset and ctime","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-06T17:51:03Z","receivedAt":"2010-12-06T17:51:03Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Drew Northup wrote:\n> On Fri, 2010-12-03 at 18:51 -0600, Jonathan Nieder wrote:\n\n>> Interesting.  Setting \"[core] trustctime = false\" in the repository\n>> configuration could be a good solution (no performance downside I can\n>> think of).\n>\n> It is worth noting that many file-based backup systems which do \"online\"\n> backups (such as in use where I work) restore the atime by default at\n> the expense of the ctime (logic being that the atime may have had value\n> and the ctime changes either way--which may or may not be true) on unix\n> style filesystems.\n\nSo have you tried putting \"[core] trustctime = false\" in /etc/gitconfig?\nThis is exactly what the setting is for, after all.\n\nIdeas for making this easier to find (FAQ on the git wiki?  advice from\nporcelain when ctime-only changes happen?) would be welcome, of course.\n"},{"id":"157486","messageId":"1291734852.3162.12.camel@drew-northup.unet.maine.edu","threadId":"25946","inReplyTo":"20101206175102.GC6272@burratino","subject":"Re: git reset and ctime","fromName":"Drew Northup","fromEmail":"drew.northup@maine.edu","sentAt":"2010-12-07T15:14:12Z","receivedAt":"2010-12-07T15:14:12Z","isPatch":false,"sender":{"key":"drew.northup@maine.edu","avatar":"https://avatars.githubusercontent.com/u/18331571?v=4"},"body":"\nOn Mon, 2010-12-06 at 11:51 -0600, Jonathan Nieder wrote:\n> Drew Northup wrote:\n> > On Fri, 2010-12-03 at 18:51 -0600, Jonathan Nieder wrote:\n> \n> >> Interesting.  Setting \"[core] trustctime = false\" in the repository\n> >> configuration could be a good solution (no performance downside I can\n> >> think of).\n> >\n> > It is worth noting that many file-based backup systems which do \"online\"\n> > backups (such as in use where I work) restore the atime by default at\n> > the expense of the ctime (logic being that the atime may have had value\n> > and the ctime changes either way--which may or may not be true) on unix\n> > style filesystems.\n> \n> So have you tried putting \"[core] trustctime = false\" in /etc/gitconfig?\n> This is exactly what the setting is for, after all.\n\nI hadn't yet, but it works like a charm.\n\n> Ideas for making this easier to find (FAQ on the git wiki?  advice from\n> porcelain when ctime-only changes happen?) would be welcome, of course.\n\nI'll have a look over that way a bit later. \n\nI'm also going to have to have a look at the src.rpm for this particular\npackaging of git and find out why it didn't create a\nskeleton /etc/gitconfig (without much in it) in the postinstall script.\n(I'm using the Dag Wieers / rpmforge one on my desktop.) It makes a lot\nmore sense to send along a patch then randomly demand that he change\nit--he may have had a decent reason for not doing so.\n\n-- \n-Drew Northup N1XIM\n   AKA RvnPhnx on OPN\n________________________________________________\n\"As opposed to vegetable or mineral error?\"\n-John Pescatore, SANS NewsBites Vol. 12 Num. 59\n"}]}