{"thread":{"id":"46631","subject":"Git *accepts* a branch name, it can't identity in the future?","startedAt":"2017-08-20T07:50:40Z","lastAt":"2017-08-20T10:00:10Z","messageCount":5,"participants":["Kaartic Sivaraam","Johannes Sixt","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"326798","messageId":"1503215489.5824.4.camel@gmail.com","threadId":"46631","inReplyTo":null,"subject":"Git *accepts* a branch name, it can't identity in the future?","fromName":"Kaartic Sivaraam","fromEmail":"kaarticsivaraam91196@gmail.com","sentAt":"2017-08-20T07:51:29Z","receivedAt":"2017-08-20T07:50:40Z","isPatch":false,"sender":{"key":"kaartic.sivaraam@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12448084?v=4"},"body":"Hello all,\n\nFirst of all, I would like to tell that this happened completely by\naccident and it's partly my mistake. Here's what happened.\n\nI recently started creating 'feature branches' a lot for the few\npatches that I sent to this mailing list. To identify the status of the\npatch corresponding to that branch I prefixed them with special unicode\ncharacters like ✓, ˅ etc. instead of using conventional hierarchical\nnames like, 'done/', 'archived/'.\n\nThen I started finding it difficult to distinguish these unicode-\nprefixed names probably because they had only one unicode character in\ncommon. So, I thought of switching to the conventional way of using\nscoped branch names (old is gold, you see). I wrote a tiny script to\nrename the branches by replacing a specific unicode prefix with a\ncorresponding hierachy. For example, the script would convert a branch\nnamed '✓doc-fix' to 'done/doc-fix'.\n\nI made a small assumption in the script which turned out to be false. I\nthought the unicode prefixes I used corresponded to only two bytes.\nThis lead to the issue. The unicode character '✓' corresponds to three\ncharacters and as a result instead of removing it, my script replaced\nit with the unknown character '�'. So, the branch named '✓doc-fix'\nbecame 'done/�doc-fix'. Here's the issue. I couldn't use \n\n    $ git branch -m done/�doc-fix done/dic-fix \n\nto rename the branch. Nor could I refer to it in anyway. Git simply\nsays,\n\n    error: pathspec 'done/�doc-fix' did not match any file(s) known to git.\n\nIt's not a big issue as I haven't lost anything out of it. The branches\nhave been merged into 'master'.\n\nI just wanted to know why git accepted a branch name which it can't\nidentify later?\n\nIf it had rejected that name in the first place it would have been\nbetter. In case you would like to know how I got that weird name,\nhere's a way to get that\n\n    $ echo '✓doc-fix' | cut -c3-100\n\n-- \nKaartic\n"},{"id":"326800","messageId":"829ac9ed-857d-84bf-2374-6a36b1d0aff1@kdbg.org","threadId":"46631","inReplyTo":"1503215489.5824.4.camel@gmail.com","subject":"Re: Git *accepts* a branch name, it can't identity in the future?","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2017-08-20T08:20:47Z","receivedAt":"2017-08-20T08:20:55Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 20.08.2017 um 09:51 schrieb Kaartic Sivaraam:\n> I made a small assumption in the script which turned out to be false. I\n> thought the unicode prefixes I used corresponded to only two bytes.\n> This lead to the issue. The unicode character '✓' corresponds to three\n> characters and as a result instead of removing it, my script replaced\n> it with the unknown character '�'. So, the branch named '✓doc-fix'\n> became 'done/�doc-fix'. Here's the issue. I couldn't use\n> \n>      $ git branch -m done/�doc-fix done/dic-fix\n> \n> to rename the branch. Nor could I refer to it in anyway. Git simply\n> says,\n> \n>      error: pathspec 'done/�doc-fix' did not match any file(s) known to git.\n> \n> It's not a big issue as I haven't lost anything out of it. The branches\n> have been merged into 'master'.\n> \n> I just wanted to know why git accepted a branch name which it can't\n> identify later?\n> \n> If it had rejected that name in the first place it would have been\n> better. In case you would like to know how I got that weird name,\n> here's a way to get that\n> \n>      $ echo '✓doc-fix' | cut -c3-100\n> \n\nSee, these two are different:\n\n$ echo '✓doc-fix' | cut -c3-100 | od -t x1\n0000000 93 64 6f 63 2d 66 69 78 0a\n0000011\n$ echo '�doc-fix' | od -t x1\n0000000 64 6f bd 64 6f 63 2d 66 69 78 0a\n0000013\n\nIt is not Git's fault that your terminal converts an invalid UTF-8 \nsequence (that your script produces) to �. Nor is it when you paste that \ncharacter onto the command line, that it is passed as a (correct) UTF-8 \ncharacter.\n\nPerhaps this helps (untested):\n\n$ git branch -m done/$(printf '\\x93')doc-fix done/dic-fix\n\nIn Git's database, branch names are just sequences of bytes. It is \noutside the scope to verify that all input is encoded correctly.\n\n-- Hannes\n"},{"id":"326801","messageId":"20170820083347.mi3bmczkpvy6qwys@sigill.intra.peff.net","threadId":"46631","inReplyTo":"1503215489.5824.4.camel@gmail.com","subject":"Re: Git *accepts* a branch name, it can't identity in the future?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-08-20T08:33:47Z","receivedAt":"2017-08-20T08:33:53Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Aug 20, 2017 at 01:21:29PM +0530, Kaartic Sivaraam wrote:\n\n> I made a small assumption in the script which turned out to be false. I\n> thought the unicode prefixes I used corresponded to only two bytes.\n> This lead to the issue. The unicode character '✓' corresponds to three\n> characters and as a result instead of removing it, my script replaced\n> it with the unknown character '�'. So, the branch named '✓doc-fix'\n> became 'done/�doc-fix'. Here's the issue. I couldn't use \n> \n>     $ git branch -m done/�doc-fix done/dic-fix \n> \n> to rename the branch. Nor could I refer to it in anyway. Git simply\n> says,\n> \n>     error: pathspec 'done/�doc-fix' did not match any file(s) known to git.\n\nWhat does \"git for-each-ref\" say about which branches you _do_ have?\n\nAlso, what platform are you on?\n\nI'm wondering specifically if you have a filesystem (like HFS+ on MacOS)\nthat silently rewrites invalid unicode in filenames we create. That\nwould mean your branches are still there, but probably with some funny\nfilename like \"done/%xxdoc-fix\". Git wouldn't know that name because the\nfilesystem rewriting happened behinds its back (though I'd think that a\nfurther open() call would find the same file, so maybe this is barking\nup the wrong tree).\n\nAnother line of thinking: are you sure the � you are writing on the\ncommand line is identical to the one generated by the corruption (and if\nyou cut and paste, is perhaps a generic glyph placed in the buffer by\nyour terminal to replace an invalid codepoint, rather than the actual\nbytes)?\n\n> I just wanted to know why git accepted a branch name which it can't\n> identify later?\n> \n> If it had rejected that name in the first place it would have been\n> better. In case you would like to know how I got that weird name,\n> here's a way to get that\n> \n>     $ echo '✓doc-fix' | cut -c3-100\n\n  [a few defines to make it easy to prod git]\n  $ check=$(printf '\\342\\234\\223')\n  $ broken=$(printf '\\223')\n\n  [this is your starting state, a branch with the unicode name]\n  $ git branch ${check}doc-fix\n\n  [you didn't say how your script works, so let's use git to rename]\n  $ git branch -m ${check}doc-fix ${broken}doc-fix\n\n  [my terminal doesn't show the unknown-character glyph, but we\n   can see the funny character with \"cat -A\"]:\n  $ git for-each-ref --format='%(refname)' | cat -A\n  refs/heads/master$\n  refs/heads/M-^Sdoc-fix$\n\n  [and we can rename it using that knowledge]\n  $ git branch ${broken}doc-fix doc-fix\n\n-Peff\n"},{"id":"326805","messageId":"1503220271.5824.9.camel@gmail.com","threadId":"46631","inReplyTo":"829ac9ed-857d-84bf-2374-6a36b1d0aff1@kdbg.org","subject":"Re: Git *accepts* a branch name, it can't identity in the future?","fromName":"Kaartic Sivaraam","fromEmail":"kaarticsivaraam91196@gmail.com","sentAt":"2017-08-20T09:11:11Z","receivedAt":"2017-08-20T09:10:22Z","isPatch":false,"sender":{"key":"kaartic.sivaraam@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12448084?v=4"},"body":"On Sun, 2017-08-20 at 10:20 +0200, Johannes Sixt wrote:\n> It is not Git's fault that your terminal converts an invalid UTF-8 \n> sequence (that your script produces) to �. Nor is it when you paste that \n> character onto the command line, that it is passed as a (correct) UTF-8 \n> character.\n> \n\nYou're right. I just now realise how I missed the line between \"what's\nseen by us\" and \"what's seen by the program\".\n\n\n> Perhaps this helps (untested):\n> \n> $ git branch -m done/$(printf '\\x93')doc-fix done/dic-fix\n> \n\nThis one helped, thanks.\n\n-- \nKaartic\n"},{"id":"326810","messageId":"1503223257.2116.2.camel@gmail.com","threadId":"46631","inReplyTo":"20170820083347.mi3bmczkpvy6qwys@sigill.intra.peff.net","subject":"Re: Git *accepts* a branch name, it can't identity in the future?","fromName":"Kaartic Sivaraam","fromEmail":"kaarticsivaraam91196@gmail.com","sentAt":"2017-08-20T10:00:57Z","receivedAt":"2017-08-20T10:00:10Z","isPatch":false,"sender":{"key":"kaartic.sivaraam@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12448084?v=4"},"body":"Thanks, but Johannes has already found the issue and given a solution.\nRegardless, replying to the questions just for the note.\n\nOn Sun, 2017-08-20 at 04:33 -0400, Jeff King wrote:\n> What does \"git for-each-ref\" say about which branches you _do_ have?\n> \n> Also, what platform are you on?\n> \n\nI use a \"Debian GNU/Linux buster/sid 64-bit\"\n\n> I'm wondering specifically if you have a filesystem (like HFS+ on MacOS)\n> that silently rewrites invalid unicode in filenames we create. That\n> would mean your branches are still there, but probably with some funny\n> filename like \"done/%xxdoc-fix\". Git wouldn't know that name because the\n> filesystem rewriting happened behinds its back (though I'd think that a\n> further open() call would find the same file, so maybe this is barking\n> up the wrong tree).\n> \n\nThat sounds dangerous!\n\n\n> Another line of thinking: are you sure the � you are writing on the\n> command line is identical to the one generated by the corruption (and if\n> you cut and paste, is perhaps a generic glyph placed in the buffer by\n> your terminal to replace an invalid codepoint, rather than the actual\n> bytes)?\n> \n\nThis was the issue. I wasn't providing git with the actual bytes that\nresulted as a consequence of the sloppy script.\n\n\n>   [you didn't say how your script works, so let's use git to rename]\n\nI know of no other way to rename a branch, so I didn't mention it :)\n\n\n>   $ broken=$(printf '\\223')\n> \n>   [and we can rename it using that knowledge]\n>   $ git branch ${broken}doc-fix doc-fix\n> \n\nJohannes has already given a solution, this one works too.\n\n\n-- \nKaartic\n"}]}