{"thread":{"id":"1624","subject":"[RFC] Removing deleted files after checkout","startedAt":"2005-08-23T16:21:56Z","lastAt":"2005-08-24T00:02:00Z","messageCount":10,"participants":["Carl Baldwin","Daniel Barkalow","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"7651","messageId":"20050823162156.GA32240@hpsvcnb.fc.hp.com","threadId":"1624","inReplyTo":null,"subject":"[RFC] Removing deleted files after checkout","fromName":"Carl Baldwin","fromEmail":"cnb@fc.hp.com","sentAt":"2005-08-23T16:21:56Z","receivedAt":"2005-08-23T16:21:56Z","isPatch":false,"sender":{"key":"cnb@fc.hp.com","avatar":null},"body":"Hello,\n\nI recently started using git to revision control the source for my\nweb-page. I wrote a post-update hook to checkout the files when I push\nto the 'live' repository.\n\nIn this particular context I decided that it was important to me to remove\ndeleted files after checking out the new HEAD. I accomplished this by running\ngit-ls-files before and after the checkout.\n\nIs there a better way? Could there be some way built into git to easily\nfind out what files dissappear when replacing the current index with one\nfrom a new tree? Is there already? The behavior of git should NOT\nchange to delete these files but I would argue that some way should\nexist to query what files disappeared if removing them is desired.\n\nHere is some code that I wrote for this. It feels a bit hackish to me but I\ncouldn't think of anything better. Comments and criticism are welcome.\n\n#!/bin/sh\n\n# HEAD changed so checkout the new HEAD deleted any files that should no longer\n# be around.\noldlist=$(tempfile)\nnewlist=$(tempfile)\nremovedlist=$(tempfile)\n\ngit-ls-files | sort -r > $oldlist\ngit-checkout-script -f\ngit-ls-files | sort -r > $newlist\n\ndiff -u $oldlist $newlist |\n tail -n +4 |\n sed -n 's/^-//p' > $removedlist\n\n# Remove each file\ncat $removedlist | xargs -rl rm -f\n# Remove the directories if empty\ncat $removedlist | xargs -rl dirname | xargs -rl rmdir -p --ignore-fail-on-non-empty\n\nrm -f $oldlist $newlist $removedlist\n\n# --- snip ---\n\nIf you are interested I attached the full post-update hook script that I\nactually use to do this. Again, comments are welcome.\n\nThanks,\nCarl\n\n-- \n- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n Carl Baldwin Systems VLSI Laboratory\n Hewlett Packard Company\n MS 88 work: 970 898-1523\n 3404 E. Harmony Rd. work: Carl.N.Baldwin@hp.com\n Fort Collins, CO 80525 home: Carl@ecBaldwin.net\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n\n\n#!/bin/sh\n\nexport PATH=/usr/local/bin:/usr/bin:/bin\n\n# cd to the root of the project directory (assume one dir up from GIT_DIR)\ncd $GIT_DIR/..\nunset GIT_DIR\n\n# Set up some temporary files and a trap to delete them\noldlist=$(tempfile)\nnewlist=$(tempfile)\nremovelist=$(tempfile)\ntrap \"rm -f $oldlist $newlist $removelist\" 0 1 2 3 15\n\n# Get list of files from the current index\ngit-ls-files | sort -r > $oldlist\n\n# Checkout the index to the working directory\ngit-checkout-script -f\n\n# Get list of files from the current (new) index\ngit-ls-files | sort -r > $newlist\n\n# Use diff to determine which files to remove from the working copy\ndiff -u $oldlist $newlist |\n tail -n +4 |\n sed -n 's/^-//p' > $removelist\n\ncat $removelist | xargs -rl rm -f\ncat $removelist | xargs -rl dirname | xargs -rl rmdir -p --ignore-fail-on-non-empty\n"},{"id":"7659","messageId":"Pine.LNX.4.63.0508231533570.23242@iabervon.org","threadId":"1624","inReplyTo":"20050823162156.GA32240@hpsvcnb.fc.hp.com","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-08-23T19:43:56Z","receivedAt":"2005-08-23T19:43:56Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 23 Aug 2005, Carl Baldwin wrote:\n\n> Hello,\n>\n> I recently started using git to revision control the source for my\n> web-page. I wrote a post-update hook to checkout the files when I push\n> to the 'live' repository.\n>\n> In this particular context I decided that it was important to me to remove\n> deleted files after checking out the new HEAD. I accomplished this by running\n> git-ls-files before and after the checkout.\n>\n> Is there a better way? Could there be some way built into git to easily\n> find out what files dissappear when replacing the current index with one\n> from a new tree? Is there already? The behavior of git should NOT\n> change to delete these files but I would argue that some way should\n> exist to query what files disappeared if removing them is desired.\n\nIf you don't use -f, git-checkout-script removes deleted files. Using -f\ntells it to ignore the old index, which means that it can't tell the\ndifference between removed files and files that weren't tracked at all.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"7663","messageId":"20050823205052.GA13311@hpsvcnb.fc.hp.com","threadId":"1624","inReplyTo":"Pine.LNX.4.63.0508231533570.23242@iabervon.org","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Carl Baldwin","fromEmail":"cnb@fc.hp.com","sentAt":"2005-08-23T20:50:52Z","receivedAt":"2005-08-23T20:50:52Z","isPatch":false,"sender":{"key":"cnb@fc.hp.com","avatar":null},"body":"On Tue, Aug 23, 2005 at 03:43:56PM -0400, Daniel Barkalow wrote:\n> On Tue, 23 Aug 2005, Carl Baldwin wrote:\n> \n> > Hello,\n> >\n> > I recently started using git to revision control the source for my\n> > web-page. I wrote a post-update hook to checkout the files when I push\n> > to the 'live' repository.\n> >\n> > In this particular context I decided that it was important to me to remove\n> > deleted files after checking out the new HEAD. I accomplished this by running\n> > git-ls-files before and after the checkout.\n> >\n> > Is there a better way? Could there be some way built into git to easily\n> > find out what files dissappear when replacing the current index with one\n> > from a new tree? Is there already? The behavior of git should NOT\n> > change to delete these files but I would argue that some way should\n> > exist to query what files disappeared if removing them is desired.\n> \n> If you don't use -f, git-checkout-script removes deleted files. Using -f\n> tells it to ignore the old index, which means that it can't tell the\n> difference between removed files and files that weren't tracked at all.\n\nMaybe I'm doing something wrong. This does not happen for me.\n\nI tried a simple test with git v0.99.4...\n\ncd\nmkdir test-git && cd test-git/\necho testing | cg-init\necho contents > file\ngit-add-script file\ngit-commit-script -m 'testing'\ncd ..\ncg-clone test-git/.git/ test-git2\ncd test-git2\ncg-rm file\ngit-commit-script -m 'testing'\nls\ncg-push\ncd ../test-git\ngit-checkout-script\nls\ngit-status-script\n\nAt this point, I want 'file' to be gone. It is, however, still there.\nThat is the situation that my code was meant to handle. Maybe you were\nthinking of something different? Maybe there is new code since 0.99.4?\n\nNow, I think it would be wrong for git-checkout-script to actually\nremove files unless an option were given to do so. So, the behavior\nthat I observe is correct in my opinion. I'm looking for a way to find\nout what files should be deleted if that is desired.\n\nThanks,\nCarl\n\n> \t-Daniel\n> *This .sig left intentionally blank*\n> \n\n-- \n- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n Carl Baldwin Systems VLSI Laboratory\n Hewlett Packard Company\n MS 88 work: 970 898-1523\n 3404 E. Harmony Rd. work: Carl.N.Baldwin@hp.com\n Fort Collins, CO 80525 home: Carl@ecBaldwin.net\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n"},{"id":"7669","messageId":"Pine.LNX.4.63.0508231713450.23242@iabervon.org","threadId":"1624","inReplyTo":"20050823205052.GA13311@hpsvcnb.fc.hp.com","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-08-23T21:27:12Z","receivedAt":"2005-08-23T21:27:12Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 23 Aug 2005, Carl Baldwin wrote:\n\n> On Tue, Aug 23, 2005 at 03:43:56PM -0400, Daniel Barkalow wrote:\n> > On Tue, 23 Aug 2005, Carl Baldwin wrote:\n> >\n> > > Hello,\n> > >\n> > > I recently started using git to revision control the source for my\n> > > web-page. I wrote a post-update hook to checkout the files when I push\n> > > to the 'live' repository.\n> > >\n> > > In this particular context I decided that it was important to me to remove\n> > > deleted files after checking out the new HEAD. I accomplished this by running\n> > > git-ls-files before and after the checkout.\n> > >\n> > > Is there a better way? Could there be some way built into git to easily\n> > > find out what files dissappear when replacing the current index with one\n> > > from a new tree? Is there already? The behavior of git should NOT\n> > > change to delete these files but I would argue that some way should\n> > > exist to query what files disappeared if removing them is desired.\n> >\n> > If you don't use -f, git-checkout-script removes deleted files. Using -f\n> > tells it to ignore the old index, which means that it can't tell the\n> > difference between removed files and files that weren't tracked at all.\n>\n> Maybe I'm doing something wrong. This does not happen for me.\n>\n> I tried a simple test with git v0.99.4...\n>\n> cd\n> mkdir test-git && cd test-git/\n> echo testing | cg-init\n> echo contents > file\n> git-add-script file\n> git-commit-script -m 'testing'\n\n[point 1]\n\n> cd ..\n> cg-clone test-git/.git/ test-git2\n> cd test-git2\n> cg-rm file\n> git-commit-script -m 'testing'\n> ls\n\n> cg-push\n> cd ../test-git\n> git-checkout-script\n\nAh, okay. I think \"push\" and \"checkout\" don't play that well together;\n\"push\" changes the ref, which \"checkout\" uses to determine what it expects\nfor the old contents, and then it's confused.\n\nWhat you probably actually want is:\n\ncd ../test-git\ngit pull ../test-git2\n\nwhich will correctly identify before and after, and remove any files that\nwere removed.\n\nAlternatively, you could do, at point 1:\n\ncp .git/refs/master .git/refs/deployed\ngit checkout deployed\n\nThen, after the push and cd:\n\ngit checkout master\ncp .git/refs/master .git/refs/deployed\ngit checkout deployed\n\nbecause checkout does remove files if you switch from a branch with them\n(e.g., deployed) to one without them (master, after the push).\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"7671","messageId":"20050823214020.GA3763@hpsvcnb.fc.hp.com","threadId":"1624","inReplyTo":"Pine.LNX.4.63.0508231713450.23242@iabervon.org","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Carl Baldwin","fromEmail":"cnb@fc.hp.com","sentAt":"2005-08-23T21:40:20Z","receivedAt":"2005-08-23T21:40:20Z","isPatch":false,"sender":{"key":"cnb@fc.hp.com","avatar":null},"body":"On Tue, Aug 23, 2005 at 05:27:12PM -0400, Daniel Barkalow wrote:\n> On Tue, 23 Aug 2005, Carl Baldwin wrote:\n> \n> > On Tue, Aug 23, 2005 at 03:43:56PM -0400, Daniel Barkalow wrote:\n> > > On Tue, 23 Aug 2005, Carl Baldwin wrote:\n> > >\n> > > > Hello,\n> > > >\n> > > > I recently started using git to revision control the source for my\n> > > > web-page. I wrote a post-update hook to checkout the files when I push\n> > > > to the 'live' repository.\n> > > >\n> > > > In this particular context I decided that it was important to me to remove\n> > > > deleted files after checking out the new HEAD. I accomplished this by running\n> > > > git-ls-files before and after the checkout.\n> > > >\n> > > > Is there a better way? Could there be some way built into git to easily\n> > > > find out what files dissappear when replacing the current index with one\n> > > > from a new tree? Is there already? The behavior of git should NOT\n> > > > change to delete these files but I would argue that some way should\n> > > > exist to query what files disappeared if removing them is desired.\n> > >\n> > > If you don't use -f, git-checkout-script removes deleted files. Using -f\n> > > tells it to ignore the old index, which means that it can't tell the\n> > > difference between removed files and files that weren't tracked at all.\n> >\n> > Maybe I'm doing something wrong. This does not happen for me.\n> >\n> > I tried a simple test with git v0.99.4...\n> >\n> > cd\n> > mkdir test-git && cd test-git/\n> > echo testing | cg-init\n> > echo contents > file\n> > git-add-script file\n> > git-commit-script -m 'testing'\n> \n> [point 1]\n> \n> > cd ..\n> > cg-clone test-git/.git/ test-git2\n> > cd test-git2\n> > cg-rm file\n> > git-commit-script -m 'testing'\n> > ls\n> \n> > cg-push\n> > cd ../test-git\n> > git-checkout-script\n> \n> Ah, okay. I think \"push\" and \"checkout\" don't play that well together;\n> \"push\" changes the ref, which \"checkout\" uses to determine what it expects\n> for the old contents, and then it's confused.\n> \n> What you probably actually want is:\n> \n> cd ../test-git\n> git pull ../test-git2\n\nThe point is to push and use a post-update hook to do the checkout. So,\nthis won't be possible.\n\n> which will correctly identify before and after, and remove any files that\n> were removed.\n> \n> Alternatively, you could do, at point 1:\n> \n> cp .git/refs/master .git/refs/deployed\n> git checkout deployed\n\nHow to get a post-update hook to do this? I suppose an update script\ncould set this up for the post-update to later use.\n\nThanks,\nCarl\n\n-- \n- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n Carl Baldwin Systems VLSI Laboratory\n Hewlett Packard Company\n MS 88 work: 970 898-1523\n 3404 E. Harmony Rd. work: Carl.N.Baldwin@hp.com\n Fort Collins, CO 80525 home: Carl@ecBaldwin.net\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n"},{"id":"7672","messageId":"7vvf1wnwtl.fsf@assigned-by-dhcp.cox.net","threadId":"1624","inReplyTo":"Pine.LNX.4.63.0508231713450.23242@iabervon.org","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-08-23T21:54:30Z","receivedAt":"2005-08-23T21:54:30Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Daniel Barkalow writes:\n\n>> > If you don't use -f, git-checkout-script removes deleted files. Using -f\n>> > tells it to ignore the old index, which means that it can't tell the\n>> > difference between removed files and files that weren't tracked at all.\n\nYes and no. \"git checkout\" assumes that the index file and the\nworking tree somewhat resembles what is in .git/HEAD commit.\nSince push operation updates .git/HEAD commit without touching\nthe index file, that assumption does not hold.\n\nThe update hook gets old commit name and new commit name, so you\nshould be able to do (untested):\n\n git-read-tree -m -u $old_commit $new_commit\n\nthere, of course after making sure that you had old_commit (the\nfirst time push happens to a new ref you would not have that one).\n"},{"id":"7674","messageId":"Pine.LNX.4.63.0508231753340.23242@iabervon.org","threadId":"1624","inReplyTo":"20050823214020.GA3763@hpsvcnb.fc.hp.com","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-08-23T22:12:09Z","receivedAt":"2005-08-23T22:12:09Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 23 Aug 2005, Carl Baldwin wrote:\n\n> The point is to push and use a post-update hook to do the checkout. So,\n> this won't be possible.\n\nYou could have the remote repository be something like\n\"~/git/website.git\", and have a hook which does: \"cd ~/www; git pull\n~/git/website.git/\". That is, have three things: the directory where you\nwork on stuff, the central storage location, and the area that the web\nserver serves, and have the storage location automatically update the web\nserver area. That's what I do with my website section that's still in CVS,\nand the general concept is good (and means that the \"real\" repository\nisn't somewhere the web server is poking around).\n\n> > which will correctly identify before and after, and remove any files that\n> > were removed.\n> >\n> > Alternatively, you could do, at point 1:\n> >\n> > cp .git/refs/master .git/refs/deployed\n> > git checkout deployed\n>\n> How to get a post-update hook to do this? I suppose an update script\n> could set this up for the post-update to later use.\n\nIf you have \"deployed\" checked out, and you push to \"master\" in the same\nrepository, having the hook do \"git resolve deployed master auto-update\"\nshould work.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"7677","messageId":"20050823222151.GA15321@hpsvcnb.fc.hp.com","threadId":"1624","inReplyTo":"7vvf1wnwtl.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Carl Baldwin","fromEmail":"cnb@fc.hp.com","sentAt":"2005-08-23T22:21:51Z","receivedAt":"2005-08-23T22:21:51Z","isPatch":false,"sender":{"key":"cnb@fc.hp.com","avatar":null},"body":"Ok, the following is what I came up with based on your response. This\nis .git/hooks/update. It mostly works in my situation. See below for\nmy discussion on what didn't work.\n\n#!/bin/sh\n\nexport PATH=/usr/local/bin:/usr/bin:/bin\n\n# cd to the root of the project directory (assume one dir up from GIT_DIR)\ncd $GIT_DIR/..\nunset GIT_DIR\n\nif expr \"$2\" : '0*$' >/dev/null; then\n git-read-tree --reset $3 &&\n git-checkout-cache -q -f -u -a\nelse\n git-read-tree -m -u $2 $3\nfi\n\nexit 0\n\n# --- snip ---\n\nThe thing that this doesn't do is remove empty directories when the last\nfile is deleted. I once expressed the opinion in a previous thread that\ndirectories should be added and removed explicitly in git. (Thus\nallowing an empty directory to be added). If this were to happen then\nthis case would get handled correctly. However, if git stays with the\nstatus quo then I think that git-read-tree -u should be changed to\nremove the empty directory. This would make it consistent.\n\nWhat do you think? Ideas?\n\nCarl\n\nOn Tue, Aug 23, 2005 at 02:54:30PM -0700, Junio C Hamano wrote:\n> Daniel Barkalow writes:\n> \n> >> > If you don't use -f, git-checkout-script removes deleted files. Using -f\n> >> > tells it to ignore the old index, which means that it can't tell the\n> >> > difference between removed files and files that weren't tracked at all.\n> \n> Yes and no. \"git checkout\" assumes that the index file and the\n> working tree somewhat resembles what is in .git/HEAD commit.\n> Since push operation updates .git/HEAD commit without touching\n> the index file, that assumption does not hold.\n> \n> The update hook gets old commit name and new commit name, so you\n> should be able to do (untested):\n> \n> git-read-tree -m -u $old_commit $new_commit\n> \n> there, of course after making sure that you had old_commit (the\n> first time push happens to a new ref you would not have that one).\n> \n\n-- \n- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n Carl Baldwin Systems VLSI Laboratory\n Hewlett Packard Company\n MS 88 work: 970 898-1523\n 3404 E. Harmony Rd. work: Carl.N.Baldwin@hp.com\n Fort Collins, CO 80525 home: Carl@ecBaldwin.net\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n"},{"id":"7679","messageId":"Pine.LNX.4.63.0508231830280.23242@iabervon.org","threadId":"1624","inReplyTo":"20050823222151.GA15321@hpsvcnb.fc.hp.com","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-08-23T22:34:30Z","receivedAt":"2005-08-23T22:34:30Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Tue, 23 Aug 2005, Carl Baldwin wrote:\n\n> The thing that this doesn't do is remove empty directories when the last\n> file is deleted. I once expressed the opinion in a previous thread that\n> directories should be added and removed explicitly in git. (Thus\n> allowing an empty directory to be added). If this were to happen then\n> this case would get handled correctly. However, if git stays with the\n> status quo then I think that git-read-tree -u should be changed to\n> remove the empty directory. This would make it consistent.\n\nI think that git-read-tree -u ought to remove a directory if it removes\nthe last file (or directory) in it.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"7682","messageId":"7vu0hgmccn.fsf@assigned-by-dhcp.cox.net","threadId":"1624","inReplyTo":"Pine.LNX.4.63.0508231830280.23242@iabervon.org","subject":"Re: [RFC] Removing deleted files after checkout","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-08-24T00:02:00Z","receivedAt":"2005-08-24T00:02:00Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Daniel Barkalow writes:\n\n> I think that git-read-tree -u ought to remove a directory if it removes\n> the last file (or directory) in it.\n\nI concur and do not have much objections to a patch that would\ndo so.\n"}]}