{"thread":{"id":"28194","subject":"Is there a scriptable way to update the stat-info in the index without having git open and read those files?","startedAt":"2011-08-22T22:28:26Z","lastAt":"2011-08-27T15:18:24Z","messageCount":3,"participants":["Elijah Newren","Junio C Hamano","Pete Wyckoff"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"174051","messageId":"CABPp-BEiN7ffNmvLCvEfz056M_F36j+gV9t6J9-x_=H9q0rZFw@mail.gmail.com","threadId":"28194","inReplyTo":null,"subject":"Is there a scriptable way to update the stat-info in the index without having git open and read those files?","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2011-08-22T22:28:26Z","receivedAt":"2011-08-22T22:28:26Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nI want to do something really close to\n  git update-index -q --refresh\nHowever, I want it to assume the files in the working tree are\nunmodified from the index (i.e. don't waste time opening and reading\nthe file) and simply update the stat information in the index to match\nthe current files on disk.\n\nYes, I know that would be unsafe if the files don't have the\nappropriate contents; I'm promising that they do have the appropriate\ncontents and don't want to pay the performance penalty for git to\nverify.  Is that possible?\n\n\nA little more detail, for the curious: I have a script that is, among\nother things, renaming large numbers of files.  Calling 'git mv <old>\n<new>' on each pair took forever.  So I switched to manually renaming\nthe files in the working copy myself, and using git update-index\n--index-info to do the renames in the index.  The result was _much_\nfaster, but of course that method blows away all the stat information\nfor the relevant files and causes any subsequent git operation (after\nmy script is done) to be slow.  I inserted a 'git update-index -q\n--refresh' at the end of my script to fix that, but that is much\nslower than I want since it has to re-read all the affected files to\nensure they haven't been modified (however, it isn't as slow as\nforking many git-mv processes).  I've tried to look for a way to speed\nup this update, but haven't found one.  Did I miss it?\n\nThanks,\nElijah\n"},{"id":"174054","messageId":"7vwre5t621.fsf@alter.siamese.dyndns.org","threadId":"28194","inReplyTo":"CABPp-BEiN7ffNmvLCvEfz056M_F36j+gV9t6J9-x_=H9q0rZFw@mail.gmail.com","subject":"Re: Is there a scriptable way to update the stat-info in the index without having git open and read those files?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-08-22T22:49:58Z","receivedAt":"2011-08-22T22:49:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren <newren@gmail.com> writes:\n\n> A little more detail, for the curious: I have a script that is, among\n> other things, renaming large numbers of files.  Calling 'git mv <old>\n> <new>' on each pair took forever.  So I switched to manually renaming\n> the files in the working copy myself, and using git update-index\n> --index-info to do the renames in the index.  The result was _much_\n> faster, but of course that method blows away all the stat information\n> for the relevant files and causes any subsequent git operation (after\n> my script is done) to be slow.  I inserted a 'git update-index -q\n> --refresh' at the end of my script to fix that, but that is much\n> slower than I want since it has to re-read all the affected files to\n> ensure they haven't been modified (however, it isn't as slow as\n> forking many git-mv processes).  I've tried to look for a way to speed\n> up this update, but haven't found one.\n\nSounds like a one-off thing to me, and I personally do not think it is\nworth the effort to add something even riskier than assume-unchanged to\nsolve.\n"},{"id":"174379","messageId":"20110827151824.GA1903@arf.padd.com","threadId":"28194","inReplyTo":"CABPp-BEiN7ffNmvLCvEfz056M_F36j+gV9t6J9-x_=H9q0rZFw@mail.gmail.com","subject":"Re: Is there a scriptable way to update the stat-info in the index without having git open and read those files?","fromName":"Pete Wyckoff","fromEmail":"pw@padd.com","sentAt":"2011-08-27T15:18:24Z","receivedAt":"2011-08-27T15:18:24Z","isPatch":false,"sender":{"key":"pw@padd.com","avatar":null},"body":"newren@gmail.com wrote on Mon, 22 Aug 2011 16:28 -0600:\n> I want to do something really close to\n>   git update-index -q --refresh\n> However, I want it to assume the files in the working tree are\n> unmodified from the index (i.e. don't waste time opening and reading\n> the file) and simply update the stat information in the index to match\n> the current files on disk.\n> \n> Yes, I know that would be unsafe if the files don't have the\n> appropriate contents; I'm promising that they do have the appropriate\n> contents and don't want to pay the performance penalty for git to\n> verify.  Is that possible?\n\nI have the same issue in my workflow, and agree with Junio that this\nis just too bizarre to put in the code.  Here's the script I use,\nrelying on dulwich, that you might find helpful.\n\n\t\t-- Pete\n\n\n------8<------------------\n#!/usr/bin/env python2.6\n# git-index-clone - Update index after a volume clone\n# Copyright 2010 Pete Wyckoff <pw@padd.com>\n\nimport sys\nimport os\nfrom dulwich.index import Index\n\nindex_name = \".git/index\"\n\n#\n# Debugging option:  show the index entry for just one file name,\n# e.g. git-index-clone file/name/in/tree\n#\ndef show_entry(name):\n    idx = Index(index_name)\n    if name not in idx:\n\tprint >>sys.stderr, \"No index entry\", name\n\treturn\n    print \"index\", idx[name]\n    t = update_from_stat(idx[name], name)\n    print \"stat \", t\n\n#\n# Stat the file, return the new tuple\n#\ndef update_from_stat(idx, name):\n    (ctime, mtime, dev, ino, mode, uid, gid, size, sha, flags) = idx\n\n    sb = os.lstat(name)\n\n    # times are float; dulwich converts to (sec, ns) on write\n    ctime = sb.st_ctime\n    mtime = sb.st_mtime\n    dev = sb.st_dev\n    ino = sb.st_ino\n    # assume mode unchanged\n    uid = sb.st_uid\n    gid = sb.st_gid\n    # assume size, sha, flags unchanged\n    return (ctime, mtime, dev, ino, mode, uid, gid, size, sha, flags)\n\ndef convert():\n    idx_in = Index(index_name)\n    os.unlink(index_name)\n\n    idx_out = Index(index_name)\n\n    for name in idx_in:\n\tidx_out[name] = update_from_stat(idx_in[name], name)\n\n    idx_out.write()\n    os.chmod(index_name, 0644)  # drop exec perms\n\ndef usage():\n    print >>sys.stderr, \"Usage: %s [<index entry name>]\\n\"\n    sys.exit(1)\n\ndef main():\n    if len(sys.argv) == 1:\n\tconvert()\n    elif len(sys.argv) == 2:\n\tshow_entry(sys.argv[1])\n    else:\n\tusage()\n    return 0\n\nif __name__ == \"__main__\":\n    sys.exit(main())\n"}]}