{"thread":{"id":"40685","subject":"[PATCH v2] ls-files: Add eol diagnostics","startedAt":"2015-10-31T10:12:05Z","lastAt":"2015-11-01T20:17:43Z","messageCount":7,"participants":["Torsten Bögershausen","Matthieu Moy","Sebastian Schuberth","Junio C Hamano"],"isPatch":true,"patchVersion":2,"patchTotal":null},"messages":[{"id":"272613","messageId":"52B3B0AC-037A-4872-9E0E-CDF6BE7A9695@web.de","threadId":"40685","inReplyTo":null,"subject":"[PATCH v2] ls-files: Add eol diagnostics","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2015-10-31T10:12:05Z","receivedAt":"2015-10-31T10:12:05Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"When working in a cross-platform environment, a user wants to\ncheck if text files are stored normalized in the repository and if\n.gitattributes are set appropriately.\n\nMake it possible to let Git show the line endings in the cache and\nin the working tree.\n\nFiles which are treated as binary by Git, are shown as \"binary\",\notherwise the end of lines (\"eolinfo\") are shown:\n\n\"text-no-eol\" text file without any EOL (or empty files)\n\"text-lf\" text file with LF\n\"text-crlf\" text file with CRLF\n\"text-crlf-lf\" text file with mixed line endings.\n\ngit ls-files --eol gives an output like this:\n\nca:text-no-eol wt:text-no-eol t/t5100/empty\nca:binary wt:binary t/test-binary-2.png\nca:text-lf wt:text-lf t/t5100/rfc2047-info-0007\nca:text-lf wt:text-crlf doit.bat\nca:text-crlf-lf wt:text-crlf-lf locale/XX.po\n\nNote that the output is meant to be human-readable and may change.\nWhen e.g. a file is deleted from the working tree and another file\nis a soft link, the output may look like this:\n\nca:text-lf wt:text-lf Documentation/RelNotes/2.6.1.txt\nca:text-lf Documentation/RelNotes/2.7.0.txt\nRelNotes\n\nSigned-off-by: Torsten Bögershausen \n---\nThe main motivation of this feature is human inspection, not shell processing.\n\nFor that reason I couldn't motivate to create a new command like git check-eol\nor git get-eol.\n\nTest cases are missing, they will be in V3,\nwhen we know that this concept makes sense at all.\n\nChanges since v1:\n- Don't analyze the contents of softlinks\n- Have only one option, --eol.\n --eol works together with -s, -o or other, but there may be combintaion which\n don't make sense.\n- Updated the documentation\n\n\nDocumentation/git-ls-files.txt | 22 ++++++++++++++++++++++\nbuiltin/ls-files.c | 22 ++++++++++++++++++++++\nconvert.c | 40 ++++++++++++++++++++++++++++++++++++++++\nconvert.h | 2 ++\n4 files changed, 86 insertions(+)\n\ndiff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\nindex e26f01f..4b02912 100644\n--- a/Documentation/git-ls-files.txt\n+++ b/Documentation/git-ls-files.txt\n@@ -12,6 +12,7 @@ SYNOPSIS\n'git ls-files' [-z] [-t] [-v]\n\t\t(--[cached|deleted|others|ignored|stage|unmerged|killed|modified])*\n\t\t(-[c|d|o|i|s|u|k|m])*\n+\t\t[--eol]\n\t\t[-x |--exclude=]\n\t\t[-X |--exclude-from=]\n\t\t[--exclude-per-directory=]\n@@ -147,6 +148,15 @@ a space) at the start of each line:\n\tpossible for manual inspection; the exact format may change at\n\tany time.\n\n+--eol::\n+\tShow line endings (eolinfo) of files for manual inspection:\n+\t\"text-no-eol\", \"text-lf\", \"text-crlf\", \"text-crlf-lf\" or \"binary\".\n+\tBoth the cached content and the content in the working tree are shown for\n+\tfor regular files, if available.\n+\t\n+\tNote:\n+ `git ls-files --eol | grep ca:text-crlf` will show not-normailzed text files\n+\n\\--::\n\tDo not interpret any more arguments as options.\n\n@@ -161,6 +171,18 @@ which case it outputs:\n\n [ ] \n\n+'git ls-files --eol' will show\n+ ca: wt: \n+\n+'git ls-files --eol -d' will show\n+ ca: \n+\n+'git ls-files --eol -o' will show\n+ wt: \n+\n+'git ls-files --eol -s' will show\n+[ ] ca: wt: \n+\n'git ls-files --unmerged' and 'git ls-files --stage' can be used to examine\ndetailed information on unmerged paths.\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex b6a7cb0..bdd0fd7 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -27,6 +27,7 @@ static int show_killed;\nstatic int show_valid_bit;\nstatic int line_terminator = '\\n';\nstatic int debug_mode;\n+static int show_eol;\n\nstatic const char *prefix;\nstatic int max_prefix_len;\n@@ -47,6 +48,21 @@ static const char *tag_modified = \"\";\nstatic const char *tag_skip_worktree = \"\";\nstatic const char *tag_resolve_undo = \"\";\n\n+static void write_wt_convert_stats_ascii(const char *path)\n+{\n+\tif (show_eol && !show_deleted) {\n+\t\tstruct stat st;\n+\t\tif (!lstat(path, &st) && (S_ISREG(st.st_mode)))\n+\t\t\tprintf(\"wt:%-13s \", get_wt_convert_stats_ascii(path));\n+\t}\n+}\n+\n+static void write_ca_convert_stats_ascii(const struct cache_entry *ce)\n+{\n+\tif (show_eol && S_ISREG(ce->ce_mode))\n+\t\tprintf(\"ca:%-13s \", get_cached_convert_stats_ascii(ce->name));\n+}\n+\nstatic void write_name(const char *name)\n{\n\t/*\n@@ -68,6 +84,7 @@ static void show_dir_entry(const char *tag, struct dir_entry *ent)\n\t\treturn;\n\n\tfputs(tag, stdout);\n+\twrite_wt_convert_stats_ascii(ent->name);\n\twrite_name(ent->name);\n}\n\n@@ -170,6 +187,9 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n\t\t find_unique_abbrev(ce->sha1,abbrev),\n\t\t ce_stage(ce));\n\t}\n+\twrite_ca_convert_stats_ascii(ce);\n+\twrite_wt_convert_stats_ascii(ce->name);\n+\t\n\twrite_name(ce->name);\n\tif (debug_mode) {\n\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n@@ -206,6 +226,7 @@ static void show_ru_info(void)\n\t\t\tprintf(\"%s%06o %s %d\\t\", tag_resolve_undo, ui->mode[i],\n\t\t\t find_unique_abbrev(ui->sha1[i], abbrev),\n\t\t\t i + 1);\n+\t\t\twrite_wt_convert_stats_ascii(path);\n\t\t\twrite_name(path);\n\t\t}\n\t}\n@@ -433,6 +454,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n\t\tOPT_BIT(0, \"directory\", &dir.flags,\n\t\t\tN_(\"show 'other' directories' names only\"),\n\t\t\tDIR_SHOW_OTHER_DIRECTORIES),\n+\t\tOPT_BOOL(0, \"eol\", &show_eol, N_(\"show line endings of files\")),\n\t\tOPT_NEGBIT(0, \"empty-directory\", &dir.flags,\n\t\t\tN_(\"don't show empty directories\"),\n\t\t\tDIR_HIDE_EMPTY_DIRECTORIES),\ndiff --git a/convert.c b/convert.c\nindex 814e814..291f869 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -95,6 +95,46 @@ static int is_binary(unsigned long size, struct text_stat *stats)\n\treturn 0;\n}\n\n+\n+const char *gather_convert_stats_ascii(const char *data, unsigned long size)\n+{\n+\tstruct text_stat stats;\n+\tif (!data)\n+\t\treturn \"\";\n+\tgather_stats(data, size, &stats);\n+\tif (is_binary(size, &stats) || stats.cr != stats.crlf)\n+\t\treturn(\"binary\");\n+\telse if (stats.crlf && (stats.crlf == stats.lf))\n+\t\treturn(\"text-crlf\");\n+\telse if (stats.crlf && stats.lf)\n+\t\treturn(\"text-crlf-lf\");\n+\telse if (stats.lf)\n+\t\treturn(\"text-lf\");\n+\telse\n+\t\treturn(\"text-no-eol\");\n+}\n+\n+const char *get_cached_convert_stats_ascii(const char *path)\n+{\n+\tconst char *ret;\n+\tunsigned long sz;\n+\tvoid *data = read_blob_data_from_cache(path, &sz);\n+\tret = gather_convert_stats_ascii(data, sz);\n+\tfree(data);\n+\treturn ret;\n+}\n+\n+const char *get_wt_convert_stats_ascii(const char *path)\n+{\n+\tconst char *ret = \"\";\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tif (strbuf_read_file(&sb, path, 0) < 0)\n+\t\treturn \"error\";\n+\tret = gather_convert_stats_ascii(sb.buf, sb.len);\n+\tstrbuf_release(&sb);\n+\treturn ret;\n+}\n+\nstatic enum eol output_eol(enum crlf_action crlf_action)\n{\n\tswitch (crlf_action) {\ndiff --git a/convert.h b/convert.h\nindex d9d853c..a1671e9 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -32,6 +32,8 @@ enum eol {\n};\n\nextern enum eol core_eol;\n+extern const char *get_cached_convert_stats_ascii(const char *path);\n+extern const char *get_wt_convert_stats_ascii(const char *path);\n\n/* returns 1 if *dst was used */\nextern int convert_to_git(const char *path, const char *src, size_t len,\n-- \n2.5.0\n"},{"id":"272614","messageId":"vpqbnbfpc6p.fsf@anie.imag.fr","threadId":"40685","inReplyTo":"52B3B0AC-037A-4872-9E0E-CDF6BE7A9695@web.de","subject":"Re: [PATCH v2] ls-files: Add eol diagnostics","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2015-10-31T10:25:18Z","receivedAt":"2015-10-31T10:25:18Z","isPatch":true,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Torsten Bögershausen writes:\n\n> ca:text-no-eol wt:text-no-eol t/t5100/empty\n> ca:binary wt:binary t/test-binary-2.png\n> ca:text-lf wt:text-lf t/t5100/rfc2047-info-0007\n> ca:text-lf wt:text-crlf doit.bat\n> ca:text-crlf-lf wt:text-crlf-lf locale/XX.po\n\nI would spell the first \"in\" or \"idx\" (for \"index\"), not \"ca\" (for\n\"cache\"). I think we avoid talking about \"the cache\" these days even\nthough the doc sometimes says \"cached in the index\" (i.e. use \"cache\" as\na verb, not a noun).\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"272652","messageId":"5635D0B1.9040800@gmail.com","threadId":"40685","inReplyTo":"vpqbnbfpc6p.fsf@anie.imag.fr","subject":"Re: [PATCH v2] ls-files: Add eol diagnostics","fromName":"Sebastian Schuberth","fromEmail":"sschuberth@gmail.com","sentAt":"2015-11-01T08:43:29Z","receivedAt":"2015-11-01T08:43:29Z","isPatch":true,"sender":{"key":"sschuberth@gmail.com","avatar":"https://avatars.githubusercontent.com/u/349154?v=4"},"body":"On 31.10.2015 11:25, Matthieu Moy wrote:\n\n>> ca:text-no-eol wt:text-no-eol t/t5100/empty\n>> ca:binary wt:binary t/test-binary-2.png\n>> ca:text-lf wt:text-lf t/t5100/rfc2047-info-0007\n>> ca:text-lf wt:text-crlf doit.bat\n>> ca:text-crlf-lf wt:text-crlf-lf locale/XX.po\n> \n> I would spell the first \"in\" or \"idx\" (for \"index\"), not \"ca\" (for\n> \"cache\"). I think we avoid talking about \"the cache\" these days even\n> though the doc sometimes says \"cached in the index\" (i.e. use \"cache\" as\n> a verb, not a noun).\n\nGood point, I'd prefer \"idx\" over ca\", too.\n\nHowever, the commit message says \"to check if text files are stored normalized in the *repository*\", yet the output refers to the index / cache. Is there a (potential) difference between line endings in the index and repo? AFAIK there is not. Any I find it a bit confusing to refer to the index where, as e.g. for a freshly cloned repo the index should be empty, yet you do have specific line endings in the repo.\n\nLong story short, how about consistently talking about line endings in the repo, and also using \"repo\" instead of \"ca\" here?\n\n-- \nSebastian Schuberth\n"},{"id":"272685","messageId":"xmqqtwp5h95i.fsf@gitster.mtv.corp.google.com","threadId":"40685","inReplyTo":"vpqbnbfpc6p.fsf@anie.imag.fr","subject":"Re: [PATCH v2] ls-files: Add eol diagnostics","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-11-01T18:22:33Z","receivedAt":"2015-11-01T18:22:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy writes:\n\n> Torsten Bögershausen writes:\n>\n>> ca:text-no-eol wt:text-no-eol t/t5100/empty\n>> ca:binary wt:binary t/test-binary-2.png\n>> ca:text-lf wt:text-lf t/t5100/rfc2047-info-0007\n>> ca:text-lf wt:text-crlf doit.bat\n>> ca:text-crlf-lf wt:text-crlf-lf locale/XX.po\n>\n> I would spell the first \"in\" or \"idx\" (for \"index\"), not \"ca\" (for\n> \"cache\"). I think we avoid talking about \"the cache\" these days even\n> though the doc sometimes says \"cached in the index\" (i.e. use \"cache\" as\n> a verb, not a noun).\n\ni/ and w/ have been used to denote the \"i\"ndex and \"w\"orktree\nversions for the past 7 years with diff.mnemonicprefix option,\nwhich you may want to match.\n"},{"id":"272689","messageId":"vpqmvuxzhot.fsf@anie.imag.fr","threadId":"40685","inReplyTo":"5635D0B1.9040800@gmail.com","subject":"Re: [PATCH v2] ls-files: Add eol diagnostics","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2015-11-01T18:40:50Z","receivedAt":"2015-11-01T18:40:50Z","isPatch":true,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Sebastian Schuberth writes:\n\n> However, the commit message says \"to check if text files are stored\n> normalized in the *repository*\", yet the output refers to the index /\n> cache. Is there a (potential) difference between line endings in the\n> index and repo?\n\nThere is when you have staged changes that are not commited yet.\n\n> Any I find it a bit confusing to refer to the index where, as e.g. for\n> a freshly cloned repo the index should be empty,\n\nNo it is not. The index is a complete snapshot of your working tree.\nWhen you have no uncommited staged changes, the index contains all files\nthat are in HEAD. Most commands show you _changes_ in the index (wrt\nHEAD or wrt the working tree), but the index itself contain all files.\n\n> Long story short, how about consistently talking about line endings in\n> the repo, and also using \"repo\" instead of \"ca\" here?\n\nI don't think this is a good idea. One typical use-case for the feature\nwould probably be:\n\n1) wtf, there's something wrong with my line endings, let's fix this.\n\n2) tweak .gitattributes, try to get everything right\n\n3) prepare a commit to apply the new settings to the repository, play\n with \"git add\", \"dos2unix\" and friends.\n\n4) check that it's OK\n\n5) \"git commit\"\n\nAt stage 4), you really want to see the content of the index, because\nyour HEAD is still broken.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"272690","messageId":"vpqio5lzhnt.fsf@anie.imag.fr","threadId":"40685","inReplyTo":"xmqqtwp5h95i.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v2] ls-files: Add eol diagnostics","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2015-11-01T18:41:26Z","receivedAt":"2015-11-01T18:41:26Z","isPatch":true,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Junio C Hamano writes:\n\n> i/ and w/ have been used to denote the \"i\"ndex and \"w\"orktree\n> versions for the past 7 years with diff.mnemonicprefix option,\n> which you may want to match.\n\nIndeed.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"272702","messageId":"CAHGBnuOicaQEYB21usR3pnVQOE_d+k5mYadxVmqociLFZZtAfg@mail.gmail.com","threadId":"40685","inReplyTo":"vpqmvuxzhot.fsf@anie.imag.fr","subject":"Re: [PATCH v2] ls-files: Add eol diagnostics","fromName":"Sebastian Schuberth","fromEmail":"sschuberth@gmail.com","sentAt":"2015-11-01T20:17:43Z","receivedAt":"2015-11-01T20:17:43Z","isPatch":true,"sender":{"key":"sschuberth@gmail.com","avatar":"https://avatars.githubusercontent.com/u/349154?v=4"},"body":"On Sun, Nov 1, 2015 at 7:40 PM, Matthieu Moy\n wrote:\n\n>> Any I find it a bit confusing to refer to the index where, as e.g. for\n>> a freshly cloned repo the index should be empty,\n>\n> No it is not. The index is a complete snapshot of your working tree.\n> When you have no uncommited staged changes, the index contains all files\n> that are in HEAD. Most commands show you _changes_ in the index (wrt\n> HEAD or wrt the working tree), but the index itself contain all files.\n\nThanks for the info.\n\n> At stage 4), you really want to see the content of the index, because\n> your HEAD is still broken.\n\nOk, I'm convinced. Thanks again!\n\n-- \nSebastian Schuberth\n"}]}