{"thread":{"id":"62593","subject":"infelicities in git hash-object --stdin-paths with special characters","startedAt":"2024-12-03T18:59:06Z","lastAt":"2024-12-05T09:45:12Z","messageCount":2,"participants":["Joey Hess","Patrick Steinhardt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"508551","messageId":"Z03xM9AvbUpqXpkI@kitenet.net","threadId":"62593","inReplyTo":null,"subject":"infelicities in git hash-object --stdin-paths with special characters","fromName":"Joey Hess","fromEmail":"id@joeyh.name","sentAt":"2024-12-02T17:41:07Z","receivedAt":"2024-12-03T18:59:06Z","isPatch":false,"sender":{"key":"id@joeyh.name","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"Apparently \"Icon\\r\" is a common filename on OSX, anyway it's a legal\nunix filename. It seems that sending a line containing that filename to\ngit hash-object --stdin-paths triggers some DOS-style CRLF handling.\nHere I am running git version 2.45.2 on Linux.\n\n$ touch Icon^M\n$ printf 'Icon\\r\\n' | git hash-object --stdin-paths\nfatal: could not open 'Icon' for reading: No such file or directory\n\n$ echo 'wrong file!' > Icon\n$ printf 'Icon\\r\\n' | git hash-object --stdin-paths\n1c43b74a7787621318ee7442eb5a36e32476f326\n\nWhile looking at builtin/hash-object.c to see why it might do this, I quickly\nnoticed another odd behavior:\n\n$ touch '\"foo\"'\n$ printf '\"foo\"\\n' | git hash-object --stdin-paths\nfatal: could not open 'foo' for reading: No such file or directory\n\n$ touch '\"foo'\n$ printf '\"foo\\n' | git hash-object --stdin-paths\nfatal: line is badly quoted\n\nThe documentation does not seem to mention that quoted lines in\n--stdin-paths are at all special. Of course, quoting would be one way to\nwork around the CRLF problem, if it were documented.\n\nIt seems that some parts of git that read filenames from stdin use\nstrbuf_getline_lf and others use strbuf_getdelim_strip_crlf. There does\nnot seem to be any consistency, and my impression is any user is best\noff using -z, when the command supports it, to avoid the mess.\n\nGiven all that, maybe adding -z to hash-object would be a good \"fix\".\n\n-- \nsee shy jo\n"},{"id":"508662","messageId":"Z1F2F58coUV7hpak@pks.im","threadId":"62593","inReplyTo":"Z03xM9AvbUpqXpkI@kitenet.net","subject":"Re: infelicities in git hash-object --stdin-paths with special characters","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-12-05T09:44:55Z","receivedAt":"2024-12-05T09:45:12Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Dec 02, 2024 at 01:41:07PM -0400, Joey Hess wrote:\n> Apparently \"Icon\\r\" is a common filename on OSX, anyway it's a legal\n> unix filename. It seems that sending a line containing that filename to\n> git hash-object --stdin-paths triggers some DOS-style CRLF handling.\n> Here I am running git version 2.45.2 on Linux.\n> \n> $ touch Icon^M\n> $ printf 'Icon\\r\\n' | git hash-object --stdin-paths\n> fatal: could not open 'Icon' for reading: No such file or directory\n> \n> $ echo 'wrong file!' > Icon\n> $ printf 'Icon\\r\\n' | git hash-object --stdin-paths\n> 1c43b74a7787621318ee7442eb5a36e32476f326\n> \n> While looking at builtin/hash-object.c to see why it might do this, I quickly\n> noticed another odd behavior:\n> \n> $ touch '\"foo\"'\n> $ printf '\"foo\"\\n' | git hash-object --stdin-paths\n> fatal: could not open 'foo' for reading: No such file or directory\n> \n> $ touch '\"foo'\n> $ printf '\"foo\\n' | git hash-object --stdin-paths\n> fatal: line is badly quoted\n> \n> The documentation does not seem to mention that quoted lines in\n> --stdin-paths are at all special. Of course, quoting would be one way to\n> work around the CRLF problem, if it were documented.\n\nIndeed -- the documentation does not meniton quoting at all, but we do\nuse `unquote_c_style()` to parse paths. So the following works:\n\n    $ echo foobar >\"$(printf 'something\\n\\rsomething')\"\n    $ printf 'something\\n\\rsomething' | git hash-object --stdin-paths\n    fatal: could not open 'something' for reading: No such file or directory\n    $ printf '\"something\\\\n\\\\rsomething\"' | git hash-object --stdin-paths\n    323fae03f4606ea9991df8befbb2fca795e648fa\n\nNote that you have to escape both \"\\n\" and \"\\r\", and then Git handles\nunquoting for you. This really needs documentation though.\n\n> It seems that some parts of git that read filenames from stdin use\n> strbuf_getline_lf and others use strbuf_getdelim_strip_crlf. There does\n> not seem to be any consistency, and my impression is any user is best\n> off using -z, when the command supports it, to avoid the mess.\n> \n> Given all that, maybe adding -z to hash-object would be a good \"fix\".\n\nI think this is a good idea regardless of whether we document the\nquoting behaviour or not. It is way easier for programs to embed NUL\ncharacters than having to handle the quoting rules implemented by Git.\n\nPatrick\n"}]}