# git reset and ctime 8 messages from 2010-12-03 to 2010-12-07. Participants: ghazel@gmail.com, Jonathan Nieder, Junio C Hamano, Drew Northup. Thread: https://gitlist.dev/t/25946 ## ghazel@gmail.com, 2010-12-03 21:36 Subject: git reset and ctime Message-ID: URL: https://gitlist.dev/e/AANLkTimyYTup%2BPqJFJ%2B2g-tVwWXA2bxTT3noonEuKBSu%40mail.gmail.com ``` Hi, I've encountered a strange issue where "git reset --hard" insists on "Checking out files ..." when all that is changed is the ctime on these files. My deploy process (capistrano) maintains a cached copy of a git repo, which it fetches, resets, and then hardlinks files from when a deploy occurs ( https://github.com/37signals/fast_remote_cache ). The hardlinking step is meant to save the time of copying the file. but hardlinking changes the ctime of the source files. That causes git reset to re-check out the files when the next deploy occurs, which is quite time-consuming. Some helpful people on #git showed that "git update-index --refresh" before the git reset prevents this behavior, but I wonder why that is needed at all. Should git reset be performing whatever "git update-index --refresh" is doing? Certainly in this case it would result in a vast speed improvement. -Greg ``` ## Jonathan Nieder, 2010-12-04 00:51 Subject: Re: git reset and ctime Message-ID: <20101204005131.GB15906@burratino> URL: https://gitlist.dev/e/20101204005131.GB15906%40burratino In-Reply-To: ``` Hi Greg, ghazel@gmail.com wrote: > I've encountered a strange issue where "git reset --hard" insists on > "Checking out files ..." when all that is changed is the ctime There is a performance trade-off. Refreshing the index requires reading+hashing the existing file if the stat information changed; this could be faster or slower than blindly overwriting depending on the situation. That said, I have no strong objection to an implicit refresh in "git reset" (performance-sensitive scripts should be using read-tree directly anyway). Have you tried making that change to builtin/reset.c? How does it perform in practice? > My deploy process (capistrano) maintains a cached copy of > a git repo, which it fetches, resets, and then hardlinks files from > when a deploy occurs ( https://github.com/37signals/fast_remote_cache > ). The hardlinking step is meant to save the time of copying the file. > but hardlinking changes the ctime of the source files. Interesting. Setting "[core] trustctime = false" in the repository configuration could be a good solution (no performance downside I can think of). Hope that helps, Jonathan ``` ## ghazel@gmail.com, 2010-12-04 01:39 Subject: Re: git reset and ctime Message-ID: URL: https://gitlist.dev/e/AANLkTi%3DMSf%2B0rkBmvuJQBaSs1-G_gFahnLvt44fnwAwf%40mail.gmail.com In-Reply-To: <20101204005131.GB15906@burratino> ``` On Fri, Dec 3, 2010 at 4:51 PM, Jonathan Nieder wrote: > ghazel@gmail.com wrote: > >> I've encountered a strange issue where "git reset --hard" insists on >> "Checking out files ..." when all that is changed is the ctime > > There is a performance trade-off.  Refreshing the index requires > reading+hashing the existing file if the stat information changed; > this could be faster or slower than blindly overwriting depending on > the situation. > > That said, I have no strong objection to an implicit refresh in "git > reset" (performance-sensitive scripts should be using read-tree > directly anyway).  Have you tried making that change to > builtin/reset.c?  How does it perform in practice? I did not make the modifications to reset.c, I just ran the refresh before reset: So originally: $ time git reset --hard Checking out files: 100% (2772/2772), done. real 0m5.328s user 0m2.539s sys 0m2.542s as opposed to: $ time git update-index --refresh real 0m1.236s user 0m1.024s sys 0m0.201s $ time git reset --hard real 0m0.055s user 0m0.011s sys 0m0.041s >>              My deploy process (capistrano) maintains a cached copy of >> a git repo, which it fetches, resets, and then hardlinks files from >> when a deploy occurs ( https://github.com/37signals/fast_remote_cache >> ). The hardlinking step is meant to save the time of copying the file. >> but hardlinking changes the ctime of the source files. > > Interesting.  Setting "[core] trustctime = false" in the repository > configuration could be a good solution (no performance downside I can > think of). This is a very useful suggestion. I do not see a case where ctime would be valuable to me. Is it really valuable to other people? What is the trade-off? -Greg ``` ## Jonathan Nieder, 2010-12-04 01:47 Subject: Re: git reset and ctime Message-ID: <20101204014751.GA18629@burratino> URL: https://gitlist.dev/e/20101204014751.GA18629%40burratino In-Reply-To: ``` ghazel@gmail.com wrote: > On Fri, Dec 3, 2010 at 4:51 PM, Jonathan Nieder wrote: >> ghazel@gmail.com wrote: >>>              My deploy process (capistrano) maintains a cached copy of >>> a git repo, which it fetches, resets, and then hardlinks files from >>> when a deploy occurs ( https://github.com/37signals/fast_remote_cache >>> ). The hardlinking step is meant to save the time of copying the file. >>> but hardlinking changes the ctime of the source files. >> >> Interesting.  Setting "[core] trustctime = false" in the repository >> configuration could be a good solution (no performance downside I can >> think of). > > This is a very useful suggestion. I do not see a case where ctime > would be valuable to me. Is it really valuable to other people? What > is the trade-off? Some reading for a rainy day :): [1] and surrounding discussion. Short answer: I think the main purpose is catching worktree corruption (e.g., if rsync screws up). [1] http://thread.gmane.org/gmane.comp.version-control.git/89370/focus=89993 ``` ## Junio C Hamano, 2010-12-04 02:28 Subject: Re: git reset and ctime Message-ID: <7v62vaxoen.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7v62vaxoen.fsf%40alter.siamese.dyndns.org In-Reply-To: <20101204005131.GB15906@burratino> ``` Jonathan Nieder writes: > That said, I have no strong objection to an implicit refresh in "git > reset" (performance-sensitive scripts should be using read-tree > directly anyway). Have you tried making that change to > builtin/reset.c? How does it perform in practice? I would be more worried about correctness impact such a patch may make when the index contains unmerged entries. ``` ## Drew Northup, 2010-12-06 17:37 Subject: Re: git reset and ctime Message-ID: <1291657037.30271.17.camel@drew-northup.unet.maine.edu> URL: https://gitlist.dev/e/1291657037.30271.17.camel%40drew-northup.unet.maine.edu In-Reply-To: <20101204005131.GB15906@burratino> ``` On Fri, 2010-12-03 at 18:51 -0600, Jonathan Nieder wrote: > Hi Greg, > > ghazel@gmail.com wrote: > > > I've encountered a strange issue where "git reset --hard" insists on > > "Checking out files ..." when all that is changed is the ctime > > There is a performance trade-off. Refreshing the index requires > reading+hashing the existing file if the stat information changed; > this could be faster or slower than blindly overwriting depending on > the situation. > > My deploy process (capistrano) maintains a cached copy of > > a git repo, which it fetches, resets, and then hardlinks files from > > when a deploy occurs ( https://github.com/37signals/fast_remote_cache > > ). The hardlinking step is meant to save the time of copying the file. > > but hardlinking changes the ctime of the source files. > > Interesting. Setting "[core] trustctime = false" in the repository > configuration could be a good solution (no performance downside I can > think of). It is worth noting that many file-based backup systems which do "online" backups (such as in use where I work) restore the atime by default at the expense of the ctime (logic being that the atime may have had value and the ctime changes either way--which may or may not be true) on unix style filesystems. While many of the git command-line things I have run seem to figure this out ok, it drives gitk nuts. As far as I am concerned this is a small price to pay for a solid daily-updated backup of my machine(s) to be available. I haven't yet put "git reset" of any sort to use (obviously I just haven't been breaking enough things yet), but I suspect that it would react in a similar way. -- -Drew Northup N1XIM AKA RvnPhnx on OPN ________________________________________________ "As opposed to vegetable or mineral error?" -John Pescatore, SANS NewsBites Vol. 12 Num. 59 ``` ## Jonathan Nieder, 2010-12-06 17:51 Subject: Re: git reset and ctime Message-ID: <20101206175102.GC6272@burratino> URL: https://gitlist.dev/e/20101206175102.GC6272%40burratino In-Reply-To: <1291657037.30271.17.camel@drew-northup.unet.maine.edu> ``` Drew Northup wrote: > On Fri, 2010-12-03 at 18:51 -0600, Jonathan Nieder wrote: >> Interesting. Setting "[core] trustctime = false" in the repository >> configuration could be a good solution (no performance downside I can >> think of). > > It is worth noting that many file-based backup systems which do "online" > backups (such as in use where I work) restore the atime by default at > the expense of the ctime (logic being that the atime may have had value > and the ctime changes either way--which may or may not be true) on unix > style filesystems. So have you tried putting "[core] trustctime = false" in /etc/gitconfig? This is exactly what the setting is for, after all. Ideas for making this easier to find (FAQ on the git wiki? advice from porcelain when ctime-only changes happen?) would be welcome, of course. ``` ## Drew Northup, 2010-12-07 15:14 Subject: Re: git reset and ctime Message-ID: <1291734852.3162.12.camel@drew-northup.unet.maine.edu> URL: https://gitlist.dev/e/1291734852.3162.12.camel%40drew-northup.unet.maine.edu In-Reply-To: <20101206175102.GC6272@burratino> ``` On Mon, 2010-12-06 at 11:51 -0600, Jonathan Nieder wrote: > Drew Northup wrote: > > On Fri, 2010-12-03 at 18:51 -0600, Jonathan Nieder wrote: > > >> Interesting. Setting "[core] trustctime = false" in the repository > >> configuration could be a good solution (no performance downside I can > >> think of). > > > > It is worth noting that many file-based backup systems which do "online" > > backups (such as in use where I work) restore the atime by default at > > the expense of the ctime (logic being that the atime may have had value > > and the ctime changes either way--which may or may not be true) on unix > > style filesystems. > > So have you tried putting "[core] trustctime = false" in /etc/gitconfig? > This is exactly what the setting is for, after all. I hadn't yet, but it works like a charm. > Ideas for making this easier to find (FAQ on the git wiki? advice from > porcelain when ctime-only changes happen?) would be welcome, of course. I'll have a look over that way a bit later. I'm also going to have to have a look at the src.rpm for this particular packaging of git and find out why it didn't create a skeleton /etc/gitconfig (without much in it) in the postinstall script. (I'm using the Dag Wieers / rpmforge one on my desktop.) It makes a lot more sense to send along a patch then randomly demand that he change it--he may have had a decent reason for not doing so. -- -Drew Northup N1XIM AKA RvnPhnx on OPN ________________________________________________ "As opposed to vegetable or mineral error?" -John Pescatore, SANS NewsBites Vol. 12 Num. 59 ```