{"thread":{"id":"43207","subject":"Re: git-add fails after file type change","startedAt":"2006-12-16T18:16:00Z","lastAt":"2006-12-17T09:11:54Z","messageCount":11,"participants":["Steven Grimm","Junio C Hamano","Jakub Narebski","A Large Angry SCM"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"295588","messageId":"458437E0.1050501@midwinter.com","threadId":"43207","inReplyTo":null,"subject":"git-add fails after file type change","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2006-12-16T18:16:00Z","receivedAt":"2006-12-16T18:16:00Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"In the course of experimenting with using git for my snapshot backups, I \nran into what looks like a bug in git-add: it croaks when it tries to \nadd a file whose type has changed, specifically when a directory gets \nmoved and a symbolic link is put in the old location pointing to the new \none. Here's a simple test case:\n\n$ git init-db\ndefaulting to local storage area\n$ mkdir dir\n$ echo foo > dir/file\n$ git add .\n$ git commit -m \"initial commit\" -a\nCommitting initial tree f4bc9c50d08b041f5e096fa68e243c34170f1cd8\n create mode 100644 dir/file\n$ mv dir dir.real\n$ ln -s dir.real dir\n$ git add .\nfatal: unable to add dir to index\n\nIs \"git add .\" the wrong thing to do here? I have been using it as a \ngeneric \"pick up all the files I haven't added yet\" command. Or is this \na bug?\n\nFor what it's worth, \"git update-index dir\" and \"git update-index --add \ndir\" both fail too.\n\n"},{"id":"297062","messageId":"em1e1v$lse$1@sea.gmane.org","threadId":"43207","inReplyTo":"458437E0.1050501@midwinter.com","subject":"Re: git-add fails after file type change","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-12-16T18:31:29Z","receivedAt":"2006-12-16T18:31:29Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Steven Grimm wrote:\n\n> In the course of experimenting with using git for my snapshot backups, I \n> ran into what looks like a bug in git-add: it croaks when it tries to \n> add a file whose type has changed, specifically when a directory gets \n> moved and a symbolic link is put in the old location pointing to the new \n> one. Here's a simple test case:\n> \n> $ git init-db\n> defaulting to local storage area\n> $ mkdir dir\n> $ echo foo > dir/file\n> $ git add .\n> $ git commit -m \"initial commit\" -a\n> Committing initial tree f4bc9c50d08b041f5e096fa68e243c34170f1cd8\n> create mode 100644 dir/file\n> $ mv dir dir.real\n> $ ln -s dir.real dir\n> $ git add .\n> fatal: unable to add dir to index\n\nWorks if you use \"git mv dir dir.real\".\n\n$ git init-db\ndefaulting to local storage area\n$ mkdir dir\n$ echo foo > dir/file\n$ git add .\n$ git commit -m \"initial commit\" -a\nCommitting initial tree f4bc9c50d08b041f5e096fa68e243c34170f1cd8\n$ git mv dir dir.real\n$ ln -s dir.real dir\n$ git add .\n$ git commit -m \"second commit\" -a\n$ git ls-tree HEAD\n120000 blob e05f72eddb14362b836c3612c13d441b097a065a dir\n040000 tree 4a1c03029e7407c0afe9fc0320b3258e188b115e dir.real\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"297637","messageId":"45843C5A.8020501@gmail.com","threadId":"43207","inReplyTo":"458437E0.1050501@midwinter.com","subject":"Re: git-add fails after file type change","fromName":"A Large Angry SCM","fromEmail":"gitzilla@gmail.com","sentAt":"2006-12-16T18:35:06Z","receivedAt":"2006-12-16T18:35:06Z","isPatch":false,"sender":{"key":"gitzilla@gmail.com","avatar":"https://gravatar.com/avatar/354625c442439908ff3dd99757dee330e29e9df7847472384faf7a00add247fb?d=mp&s=160"},"body":"Steven Grimm wrote:\n> In the course of experimenting with using git for my snapshot backups, I \n> ran into what looks like a bug in git-add: it croaks when it tries to \n> add a file whose type has changed, specifically when a directory gets \n> moved and a symbolic link is put in the old location pointing to the new \n> one. Here's a simple test case:\n> \n> $ git init-db\n> defaulting to local storage area\n> $ mkdir dir\n> $ echo foo > dir/file\n> $ git add .\n> $ git commit -m \"initial commit\" -a\n> Committing initial tree f4bc9c50d08b041f5e096fa68e243c34170f1cd8\n> create mode 100644 dir/file\n> $ mv dir dir.real\n> $ ln -s dir.real dir\n> $ git add .\n> fatal: unable to add dir to index\n> \n> Is \"git add .\" the wrong thing to do here? I have been using it as a \n> generic \"pick up all the files I haven't added yet\" command. Or is this \n> a bug?\n> \n> For what it's worth, \"git update-index dir\" and \"git update-index --add \n> dir\" both fail too.\n\n"},{"id":"294032","messageId":"45843E95.2000101@midwinter.com","threadId":"43207","inReplyTo":"em1e1v$lse$1@sea.gmane.org","subject":"Re: git-add fails after file type change","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2006-12-16T18:44:37Z","receivedAt":"2006-12-16T18:44:37Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"Jakub Narebski wrote:\n> Works if you use \"git mv dir dir.real\".\n> \n\nThis came up during my testing of snapshot backups. The *real* sequence \nis more like\n\n$ git checkout -b new-snapshot\n$ rsync /live/directory .\n$ git add .\n\nIn other words, I don't know in advance that there's a rename or (short \nof turning on verbose rsync output and parsing it) which parts of the \ntree have changed at all. So I can't easily use git-mv here.\n\nIt still feels like a bug that \"git add\" can fail with no useful \ndiagnostic. It actually took me a fair while to figure out what was \ngoing on here -- at first I thought it was having trouble with symlinks \nin general, then with absolute-path symlinks (which the actual symlink \nin question is), then I thought maybe it was a corrupt index. It wasn't \nuntil I went back and looked at the previous snapshot that I was \noverlaying this one on top of that I realized there used to be a \ndirectory where that symlink lives now.\n\n"},{"id":"296124","messageId":"7vmz5nejav.fsf@assigned-by-dhcp.cox.net","threadId":"43207","inReplyTo":"458437E0.1050501@midwinter.com","subject":"Re: git-add fails after file type change","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-12-16T19:23:52Z","receivedAt":"2006-12-16T19:23:52Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Steven Grimm writes:\n\n> $ mkdir dir\n> $ echo foo > dir/file\n> $ git add .\n> ...\n> Committing initial tree f4bc9c50d08b041f5e096fa68e243c34170f1cd8\n> create mode 100644 dir/file\n> $ mv dir dir.real\n> $ ln -s dir.real dir\n> $ git add .\n> fatal: unable to add dir to index\n\nYou didn't tell the index to remove dir/file. You cannot have\ndir and dir/file at the same time.\n"},{"id":"296285","messageId":"7vodq3cxl7.fsf@assigned-by-dhcp.cox.net","threadId":"43207","inReplyTo":"45843C5A.8020501@gmail.com","subject":"Re: git-add fails after file type change","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-12-16T21:58:12Z","receivedAt":"2006-12-16T21:58:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"A Large Angry SCM writes:\n\n> Steven Grimm wrote:\n>> In the course of experimenting with using git for my snapshot\n>> backups, I ran into what looks like a bug in git-add: it croaks when\n>> it tries to add a file whose type has changed, specifically when a\n>> directory gets moved and a symbolic link is put in the old location\n>> pointing to the new one. Here's a simple test case:\n>>\n>> $ git init-db\n>> defaulting to local storage area\n>> $ mkdir dir\n>> $ echo foo > dir/file\n>> $ git add .\n>> $ git commit -m \"initial commit\" -a\n>> Committing initial tree f4bc9c50d08b041f5e096fa68e243c34170f1cd8\n>> create mode 100644 dir/file\n>> $ mv dir dir.real\n>> $ ln -s dir.real dir\n>> $ git add .\n>> fatal: unable to add dir to index\n>>\n>> Is \"git add .\" the wrong thing to do here? I have been using it as a\n>> generic \"pick up all the files I haven't added yet\" command. Or is\n>> this a bug?\n>>\n>> For what it's worth, \"git update-index dir\" and \"git update-index\n>> --add dir\" both fail too.\n>\n> Did you try \"git-update-index --replace dir\"?\n\nGood point. I've forgotten about \"--replace\" codepath, although\nthat is all my code (May 7, 2005).\n\nMaybe \"git add\" should internally use ADD_CACHE_OK_TO_REPLACE\n(or error out and have an option to enable it)?\n\nIn any case, the error message could be made a bit more helpful,\nlike this.\n\n---\n\ndiff --git a/read-cache.c b/read-cache.c\nindex eae4745..a602010 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -609,7 +609,7 @@ int add_cache_entry(struct cache_entry *ce, int option)\n \tif (!skip_df_check &&\n \t check_file_directory_conflict(ce, pos, ok_to_replace)) {\n \t\tif (!ok_to_replace)\n-\t\t\treturn -1;\n+\t\t\treturn error(\"'%s' appears as both a file and as a directory\", ce->name);\n \t\tpos = cache_name_pos(ce->name, ntohs(ce->ce_flags));\n \t\tpos = -pos-1;\n \t}\n"},{"id":"295262","messageId":"458475DF.6040809@midwinter.com","threadId":"43207","inReplyTo":"7vodq3cxl7.fsf@assigned-by-dhcp.cox.net","subject":"Re: git-add fails after file type change","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2006-12-16T22:40:31Z","receivedAt":"2006-12-16T22:40:31Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"Junio C Hamano wrote:\n> Maybe \"git add\" should internally use ADD_CACHE_OK_TO_REPLACE\n> (or error out and have an option to enable it)?\n> \n\nThat sounds right to me. Maybe there are cases where this isn't true, \nbut -- especially given the recent move toward making \"git add\" the \nsuggested porcelain command for updating the index -- I want \"git add \nfoo\" to stick the current contents of \"foo\" into the index, regardless \nof what might have been there under that name previously. I'll grant \nthat it's not a super-common use case to rename or delete a directory \nand put a file in its place, but it's IMO not an unreasonable thing to \nwant git to track without undue hassle, given that it claims to support \nrenames. I'm fine with having to supply an extra option to get that \nautomatic behavior.\n\nBut barring that, the patch would at least have saved me a lot of \nhead-scratching.\n\n-Steve\n"},{"id":"298757","messageId":"45848CF8.4000704@midwinter.com","threadId":"43207","inReplyTo":"45843C5A.8020501@gmail.com","subject":"Re: git-add fails after file type change","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2006-12-17T00:19:04Z","receivedAt":"2006-12-17T00:19:04Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"A Large Angry SCM wrote:\n> Did you try \"git-update-index --replace dir\"?\n\nTurns out that doesn't work. It gives me the same error I get without \nthe --replace option:\n\nerror: dir: cannot add to the index - missing --add option?\nfatal: Unable to process file dir\n\nHowever, \"git rm\" followed by \"git add\" does seem to work. So for now it \nlooks like that'll be the best bet when I run into this problem (should \nbe possible to automate it, even.) The better error message from Junio's \npatch will at least make the failure less mysterious.\n\nIMO it'd still be nice if the porcelain could provide a \"make the index \nlook like the working directory\" operation that worked even in the face \nof changes like this, adding and/or removing files as needed. But it's \npossible that my crazy git-as-backup-device setup is the only place \nwhere this is even an issue; typical git users can use \"git mv\" and \nnever run into the problem.\n\n-Steve\n"},{"id":"295244","messageId":"7vvekb73jh.fsf@assigned-by-dhcp.cox.net","threadId":"43207","inReplyTo":"45848CF8.4000704@midwinter.com","subject":"Re: git-add fails after file type change","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-12-17T00:46:10Z","receivedAt":"2006-12-17T00:46:10Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Steven Grimm writes:\n\n> A Large Angry SCM wrote:\n>> Did you try \"git-update-index --replace dir\"?\n>\n> Turns out that doesn't work. It gives me the same error I get without\n> the --replace option:\n>\n> error: dir: cannot add to the index - missing --add option?\n> fatal: Unable to process file dir\n\n\"update-index --replace --add\" would be the way.\n\n $ git ls-files -s\n 100644 fa457baf8abbf5dd3bb4cbfab0c5a4cf0523d7f8 0\t1/2\n 100644 00750edc07d6415dcc07ae0351e9397b0222b7ba 0\t3\n $ ls -F\n ./ ../ 1/ 3\t.git/\n\nThere is file 1/2 in directory 1.\n\n $ mv 1 tmp ; mv 3 1 ; mv tmp 3\n $ ls -F\n ./ ../ 1 3/\t.git/\n\nI just swapped them.\n\n $ git update-index --replace --add 1\n $ git ls-files -s\n 100644 00750edc07d6415dcc07ae0351e9397b0222b7ba 0\t1\n 100644 00750edc07d6415dcc07ae0351e9397b0222b7ba 0\t3\n\nYou are allowing update-index to 'add' things so you would need\nto say --add regardless of --replace (--replace is only to allow\nremoval of conflicting entries while adding). In the hindsight,\nwe could have implied --add with --replace, but that is the way\nit is, and update-index is not a Porcelain so there is not much\npoint fixing it now.\n\nBut I think you helped me to spot a bug ;-).\n\n $ git update-index --replace --add 3/2\n $ git ls-files -s\n 100644 00750edc07d6415dcc07ae0351e9397b0222b7ba 0\t1\n\t100644 00750edc07d6415dcc07ae0351e9397b0222b7ba 0\t3\n 100644 fa457baf8abbf5dd3bb4cbfab0c5a4cf0523d7f8 0\t3/2\n\nThe entry '3' should have been removed when we did --replace.\nThis index cannot be written out as a tree:\n\n\t$ git write-tree\n You have both 3 and 3/2\n fatal: git-write-tree: error building trees\n\nCurrently we need to remove '3' by hand X-<.\n\n $ git update-index --remove 3\n $ git ls-files -s\n 100644 00750edc07d6415dcc07ae0351e9397b0222b7ba 0\t1\n 100644 fa457baf8abbf5dd3bb4cbfab0c5a4cf0523d7f8 0\t3/2\n $ git write-tree\n 77be0dd800d74913a90662e35215ee648815fc17\n\n"},{"id":"294080","messageId":"7vk60r7139.fsf@assigned-by-dhcp.cox.net","threadId":"43207","inReplyTo":"7vvekb73jh.fsf@assigned-by-dhcp.cox.net","subject":"Re: git-add fails after file type change","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-12-17T01:39:06Z","receivedAt":"2006-12-17T01:39:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"When replacing an existing file A with a directory B that has a\nfile B/C in it in the index, 'update-index --replace --add B/C'\ndid not properly remove the file to make room for the new\ndirectory.\n\nThere was a trivial logic error, most likely a cut & paste one,\ndating back to quite early days of git.\n\nSigned-off-by: Junio C Hamano \n---\n\n Junio C Hamano writes:\n\n > But I think you helped me to spot a bug ;-).\n >\n > $ git update-index --replace --add 3/2\n > $ git ls-files -s\n > 100644 00750edc07d6415dcc07ae0351e9397b0222b7ba 0 1\n > 100644 00750edc07d6415dcc07ae0351e9397b0222b7ba 0 3\n > 100644 fa457baf8abbf5dd3bb4cbfab0c5a4cf0523d7f8 0 3/2\n >\n > The entry '3' should have been removed when we did --replace.\n > This index cannot be written out as a tree:\n >\n > $ git write-tree\n > You have both 3 and 3/2\n > fatal: git-write-tree: error building trees\n\n And this fixes it.\n\n read-cache.c | 2 +-\n t/t0000-basic.sh | 9 +++++++++\n 2 files changed, 10 insertions(+), 1 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex a602010..e856a2e 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -517,7 +517,7 @@ static int has_dir_name(const struct cache_entry *ce, int pos, int ok_to_replace\n \t\tpos = cache_name_pos(name, ntohs(create_ce_flags(len, stage)));\n \t\tif (pos >= 0) {\n \t\t\tretval = -1;\n-\t\t\tif (ok_to_replace)\n+\t\t\tif (!ok_to_replace)\n \t\t\t\tbreak;\n \t\t\tremove_cache_entry_at(pos);\n \t\t\tcontinue;\ndiff --git a/t/t0000-basic.sh b/t/t0000-basic.sh\nindex 3260d1d..0cd1c41 100755\n--- a/t/t0000-basic.sh\n+++ b/t/t0000-basic.sh\n@@ -272,4 +272,13 @@ test_expect_success \\\n \t wc -l) &&\n test $numparent = 1'\n \n+test_expect_success 'update-index D/F conflict' '\n+\tmv path0 tmp &&\n+\tmv path2 path0 &&\n+\tmv tmp path2 &&\n+\tgit update-index --add --replace path2 path0/file2 &&\n+\tnumpath0=$(git ls-files path0 | wc -l) &&\n+\ttest $numpath0 = 1\n+'\n+\n test_done\n"},{"id":"295333","messageId":"7v3b7e7up1.fsf_-_@assigned-by-dhcp.cox.net","threadId":"43207","inReplyTo":"7vk60r7139.fsf@assigned-by-dhcp.cox.net","subject":"[PATCH] git-add: remove conflicting entry when adding.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-12-17T09:11:54Z","receivedAt":"2006-12-17T09:11:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"When replacing an existing file A with a directory A that has a\nfile A/B in it in the index, 'git add' did not succeed because\nit forgot to pass the allow-replace flag to add_cache_entry().\n\nIt might be safer to leave this as an error and require the user\nto explicitly remove the existing A first before adding A/B\nsince it is an unusual case, but doing that automatically is\nmuch easier to use.\n\nSigned-off-by: Junio C Hamano \n---\n read-cache.c | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex e856a2e..b8d83cc 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -358,7 +358,7 @@ int add_file_to_index(const char *path, int verbose)\n \n \tif (index_path(ce->sha1, path, &st, 1))\n \t\tdie(\"unable to index file %s\", path);\n-\tif (add_cache_entry(ce, ADD_CACHE_OK_TO_ADD))\n+\tif (add_cache_entry(ce, ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE))\n \t\tdie(\"unable to add %s to index\",path);\n \tif (verbose)\n \t\tprintf(\"add '%s'\\n\", path);\n-- \n1.4.4.2.g83c5\n\n"}]}