{"thread":{"id":"42727","subject":"What's cooking in git.git (Jun 2016, #09; Mon, 27)","startedAt":"2016-06-27T23:22:14Z","lastAt":"2016-07-12T19:55:10Z","messageCount":24,"participants":["Junio C Hamano","tboegi@web.de","Torsten Bögershausen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"290333","messageId":"xmqq37nyb4kp.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":null,"subject":"What's cooking in git.git (Jun 2016, #09; Mon, 27)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-06-27T23:21:58Z","receivedAt":"2016-06-27T23:22:14Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Here are the topics that have been cooking. Commits prefixed with\n'-' are only in 'pu' (proposed updates) while commits prefixed with\n'+' are in 'next'. The ones marked with '.' do not appear in any of\nthe integration branches, but I am still holding onto them.\n\nThe first batch for this cycle has been merged to 'master', the tip\nof 'next' has been rewound and rebuilt, while a few topics got\ntemporarily ejected from 'next'.\n\nYou can find the changes described here in the integration branches\nof the repositories listed at\n\n http://git-blame.blogspot.com/p/git-public-repositories.html\n\n--------------------------------------------------\n[Graduated to \"master\"]\n\n* ap/git-svn-propset-doc (2016-06-15) 1 commit\n (merged to 'next' on 2016-06-22 at 5a34f7d)\n + git-svn: document the 'git svn propset' command\n\n \"git svn propset\" subcommand that was added in 2.3 days is\n documented now.\n\n\n* cc/apply-introduce-state (2016-06-06) 50 commits\n (merged to 'next' on 2016-06-20 at 4f205b8)\n + builtin/apply: remove misleading comment on lock_file field\n + builtin/apply: move 'newfd' global into 'struct apply_state'\n + builtin/apply: add 'lock_file' pointer into 'struct apply_state'\n + builtin/apply: move applying patches into apply_all_patches()\n + builtin/apply: move 'state' check into check_apply_state()\n + builtin/apply: move 'symlink_changes' global into 'struct apply_state'\n + builtin/apply: move 'fn_table' global into 'struct apply_state'\n + builtin/apply: move 'state_linenr' global into 'struct apply_state'\n + builtin/apply: move 'max_change' and 'max_len' into 'struct apply_state'\n + builtin/apply: move 'ws_ignore_action' into 'struct apply_state'\n + builtin/apply: move 'ws_error_action' into 'struct apply_state'\n + builtin/apply: move 'applied_after_fixing_ws' into 'struct apply_state'\n + builtin/apply: move 'squelch_whitespace_errors' into 'struct apply_state'\n + builtin/apply: remove whitespace_option arg from set_default_whitespace_mode()\n + builtin/apply: move 'whitespace_option' into 'struct apply_state'\n + builtin/apply: move 'whitespace_error' global into 'struct apply_state'\n + builtin/apply: move 'root' global into 'struct apply_state'\n + builtin/apply: move 'p_value_known' global into 'struct apply_state'\n + builtin/apply: move 'p_value' global into 'struct apply_state'\n + builtin/apply: move 'has_include' global into 'struct apply_state'\n + builtin/apply: move 'limit_by_name' global into 'struct apply_state'\n + builtin/apply: move 'patch_input_file' global into 'struct apply_state'\n + builtin/apply: move 'apply' global into 'struct apply_state'\n + builtin/apply: move 'p_context' global into 'struct apply_state'\n + builtin/apply: move 'fake_ancestor' global into 'struct apply_state'\n + builtin/apply: move 'line_termination' global into 'struct apply_state'\n + builtin/apply: move 'unsafe_paths' global into 'struct apply_state'\n + builtin/apply: move 'no_add' global into 'struct apply_state'\n + builtin/apply: move 'threeway' global into 'struct apply_state'\n + builtin/apply: move 'summary' global into 'struct apply_state'\n + builtin/apply: move 'numstat' global into 'struct apply_state'\n + builtin/apply: move 'diffstat' global into 'struct apply_state'\n + builtin/apply: move 'cached' global into 'struct apply_state'\n + builtin/apply: move 'allow_overlap' global into 'struct apply_state'\n + builtin/apply: move 'update_index' global into 'struct apply_state'\n + builtin/apply: move 'apply_verbosely' global into 'struct apply_state'\n + builtin/apply: move 'apply_with_reject' global into 'struct apply_state'\n + builtin/apply: move 'apply_in_reverse' global into 'struct apply_state'\n + builtin/apply: move 'check_index' global into 'struct apply_state'\n + builtin/apply: move 'check' global into 'struct apply_state'\n + builtin/apply: move 'unidiff_zero' global into 'struct apply_state'\n + builtin/apply: move 'state' init into init_apply_state()\n + builtin/apply: introduce 'struct apply_state' to start libifying\n + builtin/apply: move 'read_stdin' global into cmd_apply()\n + builtin/apply: move 'options' variable into cmd_apply()\n + builtin/apply: extract line_by_line_fuzzy_match() from match_fragment()\n + builtin/apply: avoid local variable shadowing 'len' parameter\n + builtin/apply: avoid parameter shadowing 'linenr' global\n + builtin/apply: avoid parameter shadowing 'p_value' global\n + builtin/apply: make gitdiff_verify_name() return void\n\n Originally merged to 'next' on 2016-06-06\n\n The \"git apply\" standalone program is being libified; this is the\n first step to move many state variables into a structure that can\n be explicitly (re)initialized to make the machinery callable more\n than once.\n\n The next step that moves some remaining state variables into the\n structure and turns die()s into an error return that propagates up\n to the caller is not queued yet but in flight. It would be good to\n review the above first and give the remainder of the series a solid\n base to build on.\n\n\n* dn/gpg-doc (2016-06-16) 1 commit\n (merged to 'next' on 2016-06-22 at f467355)\n + Documentation: GPG capitalization\n\n The documentation tries to consistently spell \"GPG\"; when\n referring to the specific program name, \"gpg\" is used.\n\n\n* em/newer-freebsd-shells-are-fine-with-returns (2016-06-17) 1 commit\n (merged to 'next' on 2016-06-22 at dbee33c)\n + rebase: update comment about FreeBSD /bin/sh\n\n Comments about misbehaving FreeBSD shells have been clarified with\n the version number (9.x and before are broken, newer ones are OK).\n\n\n* et/add-chmod-x (2016-06-07) 1 commit\n (merged to 'next' on 2016-06-22 at 71d65a0)\n + add: add --chmod=+x / --chmod=-x options\n\n \"git update-index --add --chmod=+x file\" may be usable as an escape\n hatch, but not a friendly thing to force for people who do need to\n use it regularly. \"git add --chmod=+x file\" can be used instead.\n\n\n* jc/deref-tag (2016-06-14) 1 commit\n (merged to 'next' on 2016-06-22 at 1075713)\n + blame, line-log: do not loop around deref_tag()\n\n Code clean-up.\n\n\n* jk/avoid-unbounded-alloca (2016-06-07) 1 commit\n (merged to 'next' on 2016-06-22 at 93feb23)\n + tree-diff: avoid alloca for large allocations\n\n A codepath that used alloca(3) to place an unbounded amount of data\n on the stack has been updated to avoid doing so.\n\n\n* jk/fetch-prune-doc (2016-06-14) 1 commit\n (merged to 'next' on 2016-06-22 at 6563376)\n + fetch: document that pruning happens before fetching\n\n Minor doc update.\n\n\n* lf/receive-pack-auto-gc-to-client (2016-06-06) 1 commit\n (merged to 'next' on 2016-06-22 at 92162f5)\n + receive-pack: send auto-gc output over sideband 2\n\n Messages that are generated by auto gc during \"git push\" on the\n receiving end are now passed back to the sending end in such a way\n that they are shown with \"remote: \" prefix to avoid confusing the\n users.\n\n\n* lv/status-say-working-tree-not-directory (2016-06-09) 1 commit\n (merged to 'next' on 2016-06-22 at c65c7c1)\n + Use \"working tree\" instead of \"working directory\" for git status\n\n \"git status\" used to say \"working directory\" when it meant \"working\n tree\".\n\n\n* mg/cherry-pick-multi-on-unborn (2016-06-06) 1 commit\n (merged to 'next' on 2016-06-22 at 183295b)\n + cherry-pick: allow to pick to unborn branches\n\n \"git cherry-pick A\" worked on an unborn branch, but \"git\n cherry-pick A..B\" didn't.\n\n\n* nb/gnome-keyring-build (2016-06-14) 1 commit\n (merged to 'next' on 2016-06-22 at 0dfb90c)\n + gnome-keyring: Don't hard-code pkg-config executable\n\n Build improvements for gnome-keyring (in contrib/)\n\n\n* pb/strbuf-read-file-doc (2016-06-14) 1 commit\n (merged to 'next' on 2016-06-22 at 10e4b4f)\n + strbuf: describe the return value of strbuf_read_file\n\n Minor doc update.\n\n\n* pc/occurred (2016-06-10) 2 commits\n (merged to 'next' on 2016-06-22 at ce0b944)\n + config.c: fix misspelt \"occurred\" in an error message\n + refs.h: fix misspelt \"occurred\" in a comment\n\n Typofix.\n\n\n* rj/compat-regex-size-max-fix (2016-06-06) 1 commit\n (merged to 'next' on 2016-06-22 at 376c5b1)\n + regex: fix a SIZE_MAX macro redefinition warning\n\n A compilation fix.\n\n\n* sg/reflog-past-root (2016-06-06) 1 commit\n (merged to 'next' on 2016-06-22 at c5d4e29)\n + reflog: continue walking the reflog past root commits\n\n \"git reflog\" stopped upon seeing an entry that denotes a branch\n creation event (aka \"unborn\"), which made it appear as if the\n reflog was truncated.\n\n\n* tb/complete-status (2016-06-10) 3 commits\n (merged to 'next' on 2016-06-22 at 44ae68f)\n + completion: add git status\n + completion: add __git_get_option_value helper\n + completion: factor out untracked file modes into a variable\n\n The completion script (in contrib/) learned to complete \"git\n status\" options.\n\n\n* tr/doc-tt (2016-06-08) 4 commits\n (merged to 'next' on 2016-06-22 at dc6df3b)\n + doc: change configuration variables format\n + doc: more consistency in environment variables format\n + doc: change environment variables format\n + doc: clearer rule about formatting literals\n\n The documentation set has been updated so that literal commands,\n configuration variables and environment variables are consistently\n typeset in fixed-width font and bold in manpages.\n\n\n* vs/prompt-avoid-unset-variable (2016-06-06) 1 commit\n (merged to 'next' on 2016-06-22 at 8bf21d3)\n + git-prompt.sh: Don't error on null ${ZSH,BASH}_VERSION, $short_sha\n\n The git-prompt scriptlet (in contrib/) was not friendly with those\n who uses \"set -u\", which has been fixed.\n\n--------------------------------------------------\n[New Topics]\n\n* jk/test-match-signal (2016-06-24) 4 commits\n - t/lib-git-daemon: use test_match_signal\n - test_must_fail: use test_match_signal\n - t0005: use test_match_signal as appropriate\n - tests: factor portable signal check out of t0005\n\n The test framework learned a new helper test_match_signal to check\n an exit code from getting killed by an expected signal.\n\n Will merge to 'next'.\n\n\n* ah/unpack-trees-advice-messages (2016-06-27) 1 commit\n - unpack-trees: fix English grammar in do-this-before-that messages\n\n Will merge to 'next'.\n\n\n* ew/gc-auto-pack-limit-fix (2016-06-27) 1 commit\n - gc: fix off-by-one error with gc.autoPackLimit\n\n \"gc.autoPackLimit\" when set to 1 should not trigger a repacking\n when there is only one pack, but the code counted poorly and did\n so.\n\n Will merge to 'next'.\n\n\n* nd/connect-ssh-command-config (2016-06-27) 1 commit\n - connect: read $GIT_SSH_COMMAND from config file\n\n A new configuration variable core.sshCommand to specify what value\n for GIT_SSH_COMMAND to use per repository.\n\n\n* nd/doc-new-command (2016-06-27) 1 commit\n - new-command.txt: correct the command description file\n\n Will merge to 'next'.\n\n\n* po/range-doc (2016-06-27) 3 commits\n - doc: give headings for the two and three dot notations\n - doc: show the actual left, right, and boundary marks\n - doc: use 'symmetric difference' consistently\n\n\n* sb/submodule-parallel-fetch (2016-06-27) 2 commits\n - xwrite: poll on non-blocking FDs\n - xread: retry after poll on EAGAIN/EWOULDBLOCK\n\n Fix a recently introduced codepaths that are involved in parallel\n submodule operations, which gave up on reading too early, and\n could have wasted CPU while attempting to write under a corner case\n condition.\n\n Will merge to 'next'.\n\n\n* mm/doc-tt (2016-06-27) 6 commits\n - doc: typeset HEAD and variants as litteral\n - CodingGuidelines: formatting HEAD in documentation\n - doc: typeset long options with argument as litteral\n - doc: typeset -- as litteral\n - doc: typeset long command-line options as literal\n - doc: typeset short command-line options as literal\n\n More mark-up updates to typeset strings that are expected to\n literally typed by the end user in fixed-width font.\n\n\n* nd/fetch-ref-summary (2016-06-27) 5 commits\n - fetch: reduce duplicate in ref update status lines with placeholder\n - fetch: align all \"remote -> local\" output\n - fetch: change flag code for displaying tag update and deleted ref\n - fetch: refactor ref update status formatting code\n - git-fetch.txt: document fetch output\n\n Improve the look of the way \"git fetch\" reports what happened to\n each ref that was fetched.\n\n Still being discussed.\n\n--------------------------------------------------\n[Stalled]\n\n* tb/convert-peek-in-index (2016-06-07) 3 commits\n - correct ce_compare_data() in a middle of a merge\n - read-cache: factor out get_sha1_from_index() helper\n - convert: unify the \"auto\" handling of CRLF\n (this branch is used by jh/clean-smudge-annex.)\n\n Needs review.\n\n\n* sb/bisect (2016-04-15) 22 commits\n - SQUASH???\n - bisect: get back halfway shortcut\n - bisect: compute best bisection in compute_relevant_weights()\n - bisect: use a bottom-up traversal to find relevant weights\n - bisect: prepare for different algorithms based on find_all\n - bisect: rename count_distance() to compute_weight()\n - bisect: make total number of commits global\n - bisect: introduce distance_direction()\n - bisect: extract get_distance() function from code duplication\n - bisect: use commit instead of commit list as arguments when appropriate\n - bisect: replace clear_distance() by unique markers\n - bisect: use struct node_data array instead of int array\n - bisect: get rid of recursion in count_distance()\n - bisect: make algorithm behavior independent of DEBUG_BISECT\n - bisect: make bisect compile if DEBUG_BISECT is set\n - bisect: plug the biggest memory leak\n - bisect: add test for the bisect algorithm\n - t6030: generalize test to not rely on current implementation\n - t: use test_cmp_rev() where appropriate\n - t/test-lib-functions.sh: generalize test_cmp_rev\n - bisect: allow 'bisect run' if no good commit is known\n - bisect: write about `bisect next` in documentation\n\n The internal algorithm used in \"git bisect\" to find the next commit\n to check has been optimized greatly.\n\n Expecting a reroll.\n ($gmane/291163)\n\n\n* sg/completion-updates (2016-02-28) 21 commits\n . completion: cache the path to the repository\n . completion: extract repository discovery from __gitdir()\n . completion: don't guard git executions with __gitdir()\n . completion: consolidate silencing errors from git commands\n . completion: don't use __gitdir() for git commands\n . completion: respect 'git -C '\n . completion: fix completion after 'git -C '\n . completion: don't offer commands when 'git --opt' needs an argument\n . rev-parse: add '--absolute-git-dir' option\n . completion: list short refs from a remote given as a URL\n . completion: don't list 'HEAD' when trying refs completion outside of a repo\n . completion: list refs from remote when remote's name matches a directory\n . completion: respect 'git --git-dir=' when listing remote refs\n . completion: fix most spots not respecting 'git --git-dir='\n . completion: ensure that the repository path given on the command line exists\n . completion tests: add tests for the __git_refs() helper function\n . completion tests: check __gitdir()'s output in the error cases\n . completion tests: consolidate getting path of current working directory\n . completion tests: make the $cur variable local to the test helper functions\n . completion tests: don't add test cruft to the test repository\n . completion: improve __git_refs()'s in-code documentation\n\n Will be rerolled.\n ($gmane/287839)\n\n\n* ec/annotate-deleted (2015-11-20) 1 commit\n - annotate: skip checking working tree if a revision is provided\n\n Usability fix for annotate-specific \" \" syntax with deleted\n files.\n\n Waiting for review.\n\n\n* dg/subtree-rebase-test (2016-01-19) 1 commit\n - contrib/subtree: Add a test for subtree rebase that loses commits\n\n Reviewed up to v5.\n Will be rerolled.\n ($gmane/284426)\n\n\n* dk/gc-more-wo-pack (2016-01-13) 4 commits\n - gc: clean garbage .bitmap files from pack dir\n - t5304: ensure non-garbage files are not deleted\n - t5304: test .bitmap garbage files\n - prepare_packed_git(): find more garbage\n\n Follow-on to dk/gc-idx-wo-pack topic, to clean up stale\n .bitmap and .keep files.\n\n Waiting for a reroll.\n ($gmane/284368).\n\n\n* jc/diff-b-m (2015-02-23) 5 commits\n . WIPWIP\n . WIP: diff-b-m\n - diffcore-rename: allow easier debugging\n - diffcore-rename.c: add locate_rename_src()\n - diffcore-break: allow debugging\n\n \"git diff -B -M\" produced incorrect patch when the postimage of a\n completely rewritten file is similar to the preimage of a removed\n file; such a resulting file must not be expressed as a rename from\n other place.\n\n The fix in this patch is broken, unfortunately.\n Will discard.\n\n--------------------------------------------------\n[Cooking]\n\n* ak/t7800-wo-readlink (2016-06-21) 1 commit\n - t7800: readlink may not be available\n\n One among four invocations of readlink(1) in our test suite has\n been rewritten so that the test can run on systems without the\n command (others are in valgrind test framework and t9802).\n\n Will merge to 'next'.\n\n\n* js/perf-on-apple (2016-06-21) 1 commit\n - perf: accommodate for MacOSX\n\n t/perf needs /usr/bin/time with GNU extension; the invocation of it\n is updated to \"gtime\" on Darwin.\n\n Will merge to 'next'.\n\n\n* sb/t5614-modernize (2016-06-21) 1 commit\n - t5614: don't use subshells\n (this branch uses sb/clone-shallow-passthru.)\n\n Test clean-up.\n\n Will merge to 'next'.\n\n\n* ao/p4-has-branch-prefix-fix (2016-06-22) 1 commit\n - git-p4: correct hasBranchPrefix verbose output\n\n A bug, which caused \"git p4\" while running under verbose mode to\n report paths that are omitted due to branch prefix incorrectly, has\n been fixed; the command said \"Ignoring file outside of prefix\" for\n paths that are _inside_.\n\n Will merge to 'next'.\n\n\n* cb/t7810-test-label-fix (2016-06-21) 1 commit\n - t7810: fix duplicated test title\n\n Test clean-up.\n\n Will merge to 'next'.\n\n\n* jc/t2300-setup (2016-06-22) 1 commit\n - t2300: \"git --exec-path\" is not usable in $PATH on Windows as-is\n\n Portability fix for Windows.\n\n Will merge to 'next'.\n\n\n* jk/perf-any-version (2016-06-22) 2 commits\n - p4211: explicitly disable renames in no-rename test\n - t/perf: fix regression in testing older versions of git\n\n Allow t/perf framework to use the features from the most recent\n version of Git even when testing an older installed version.\n\n Will merge to 'next'.\n\n\n* jn/preformatted-doc-url (2016-06-22) 1 commit\n - doc: git-htmldocs.googlecode.com is no more\n\n Will merge to 'next'.\n\n\n* ex/deprecate-empty-pathspec-as-match-all (2016-06-22) 1 commit\n - pathspec: warn on empty strings as pathspec\n\n An empty string used as a pathspec element has always meant\n 'everything matches', but it is too easy to write a script that\n finds a path to remove in $path and run 'git rm \"$paht\"', which\n ends up removing everything. Start warning about this use of an\n empty string used for 'everything matches' and ask users to use a\n more explicit '.' for that instead.\n\n The hope is that existing users will not mind this change, and\n eventually the warning can be turned into a hard error, upgrading\n the deprecation into removal of this (mis)feature.\n\n Will wait for further comments for a bit before merging to 'next'.\n\n\n* jk/ansi-color (2016-06-23) 7 commits\n - color: support strike-through attribute\n - color: support \"italic\" attribute\n - color: allow \"no-\" for negating attributes\n - color: refactor parse_attr\n - add skip_prefix_mem helper\n - doc: refactor description of color format\n - color: fix max-size comment\n\n The output coloring scheme learned two new attributes, italic and\n strike, in addition to existing bold, reverse, etc.\n\n Will merge to 'next'.\n\n\n* nd/icase (2016-06-27) 13 commits\n - SQUASH???\n - grep.c: reuse \"icase\" variable\n - diffcore-pickaxe: support case insensitive match on non-ascii\n - diffcore-pickaxe: Add regcomp_or_die()\n - grep/pcre: support utf-8\n - gettext: add is_utf8_locale()\n - grep/pcre: prepare locale-dependent tables for icase matching\n - grep: rewrite an if/else condition to avoid duplicate expression\n - grep/icase: avoid kwsset when -F is specified\n - grep/icase: avoid kwsset on literal non-ascii strings\n - test-regex: expose full regcomp() to the command line\n - test-regex: isolate the bug test code\n - grep: break down an \"if\" stmt in preparation for next changes\n\n \"git grep -i\" has been taught to fold case in non-ascii locales\n correctly.\n\n Modulo minor possible nits, this round looked mostly sensible.\n\n\n* mj/log-show-signature-conf (2016-06-24) 3 commits\n - log: add log.showSignature configuration variable\n - log: add \"--no-show-signature\" command line option\n - t4202: refactor test\n\n \"git log\" learns log.showSignature configuration variable, and a\n command line option \"--no-show-signature\" to countermand it.\n\n Will merge to 'next'.\n\n\n* mg/signature-doc (2016-06-17) 4 commits\n - Documentation/technical: signed merge tag format\n - Documentation/technical: signed commit format\n - Documentation/technical: signed tag format\n - Documentation/technical: describe signature formats\n\n Formats of the various data (and how to validate them) where we use\n GPG signature have been documented.\n\n Will merge to 'next'.\n\n\n* jk/string-list-static-init (2016-06-13) 2 commits\n (merged to 'next' on 2016-06-27 at 3d4b2fa)\n + use string_list initializer consistently\n + Merge branch 'jk/parseopt-string-list' into jk/string-list-static-init\n (this branch uses jk/parseopt-string-list.)\n\n Instead of taking advantage of a struct string_list that is\n allocated with all NULs happens to be STRING_LIST_INIT_NODUP kind,\n initialize them explicitly as such, to document their behaviour\n better.\n\n Will merge to 'master'.\n\n\n* mh/ref-store (2016-06-20) 38 commits\n - refs: implement iteration over only per-worktree refs\n - refs: make lock generic\n - refs: add method to rename refs\n - refs: add methods to init refs db\n - refs: make delete_refs() virtual\n - refs: add method for initial ref transaction commit\n - refs: add methods for reflog\n - refs: add method iterator_begin\n - files_ref_iterator_begin(): take a ref_store argument\n - split_symref_update(): add a files_ref_store argument\n - lock_ref_sha1_basic(): add a files_ref_store argument\n - lock_ref_for_update(): add a files_ref_store argument\n - commit_ref_update(): add a files_ref_store argument\n - lock_raw_ref(): add a files_ref_store argument\n - repack_without_refs(): add a files_ref_store argument\n - refs: make peel_ref() virtual\n - refs: make create_symref() virtual\n - refs: make pack_refs() virtual\n - refs: make verify_refname_available() virtual\n - refs: make read_raw_ref() virtual\n - resolve_gitlink_ref(): rename path parameter to submodule\n - resolve_gitlink_ref(): avoid memory allocation in many cases\n - resolve_gitlink_ref(): implement using resolve_ref_recursively()\n - resolve_ref_recursively(): new function\n - read_raw_ref(): take a (struct ref_store *) argument\n - resolve_gitlink_packed_ref(): remove function\n - resolve_packed_ref(): rename function from resolve_missing_loose_ref()\n - refs: reorder definitions\n - refs: add a transaction_commit() method\n - {lock,commit,rollback}_packed_refs(): add files_ref_store arguments\n - resolve_missing_loose_ref(): add a files_ref_store argument\n - get_packed_ref(): add a files_ref_store argument\n - add_packed_ref(): add a files_ref_store argument\n - refs: create a base class \"ref_store\" for files_ref_store\n - refs: add a backend method structure\n - refs: rename struct ref_cache to files_ref_store\n - rename_ref_available(): add docstring\n - resolve_gitlink_ref(): eliminate temporary variable\n (this branch uses mh/ref-iterators and mh/split-under-lock; is tangled with mh/update-ref-errors.)\n\n The ref-store abstraction was introduced to the refs API so that we\n can plug in different backends to store references.\n\n Rebased on top of mh/split-under-lock.\n\n\n* mh/update-ref-errors (2016-06-20) 6 commits\n - lock_ref_for_update(): avoid a symref resolution\n - lock_ref_for_update(): make error handling more uniform\n - t1404: add more tests of update-ref error handling\n - t1404: document function test_update_rejected\n - t1404: remove \"prefix\" argument to test_update_rejected\n - t1404: rename file to t1404-update-ref-errors.sh\n (this branch uses mh/split-under-lock; is tangled with mh/ref-iterators and mh/ref-store.)\n\n Error handling in the codepaths that updates refs has been\n improved.\n\n Rebased on top of mh/split-under-lock.\n\n\n* jh/clean-smudge-annex (2016-06-22) 10 commits\n - SQUASH???\n - use smudgeToFile filter in recursive merge\n - use smudgeToFile filter in git am\n - better recovery from failure of smudgeToFile filter\n - warn on unusable smudgeToFile/cleanFromFile config\n - use smudgeToFile in git checkout etc\n - use cleanFromFile in git add\n - add smudgeToFile and cleanFromFile filter configs\n - clarify %f documentation\n - Merge branch 'tb/convert-peek-in-index' into jh/clean-smudge-annex\n (this branch uses tb/convert-peek-in-index.)\n\n The interface to \"clean/smudge\" filters require Git to feed the\n whole contents via pipe, which is suboptimal for some\n applications. \"cleanFromFile/smudgeToFile\" commands are the moral\n equilvalents for these filters but they interact with the files on\n the filesystem directly.\n\n\n* lc/shell-default-value-noexpand (2016-06-19) 1 commit\n - sh-setup: enclose setting of ${VAR=default} in double-quotes\n\n Fix unnecessarily waste in the idiomatic use of ': ${VAR=default}'\n to set the default value, without enclosing it in double quotes.\n\n Will merge to 'next'.\n\n\n* sb/clone-shallow-passthru (2016-06-20) 1 commit\n - clone: do not let --depth imply --shallow-submodules\n (this branch is used by sb/t5614-modernize.)\n\n Fix an unintended regression in v2.9 that breaks \"clone --depth\"\n that recurses down to submodules by forcing the submodules to also\n be cloned shallowly, which many server instances that host upstream\n of the submodules are not prepared for.\n\n Will merge to 'next'.\n\n\n* js/find-commit-subject-ignore-leading-blanks (2016-06-22) 2 commits\n - commit.c: make find_commit_subject() more robust\n - pretty: make the skip_blank_lines() function public\n\n A helper function that takes the contents of a commit object and\n finds its subject line did not ignore leading blank lines, as is\n commonly done by other codepaths. Make it ignore leading blank\n lines to match.\n\n Will merge to 'next'.\n\n\n* js/log-to-diffopt-file (2016-06-24) 10 commits\n - t4211: ensure that log respects --output=\n - shortlog: respect the --output= setting\n - format-patch: use stdout directly\n - format-patch: avoid freopen()\n - format-patch: explicitly switch off color when writing to files\n - shortlog: support outputting to streams other than stdout\n - graph: respect the diffopt.file setting\n - line-log: respect diffopt's configured output file stream\n - log-tree: respect diffopt's configured output file stream\n - log: prepare log/log-tree to reuse the diffopt.close_file attribute\n\n The commands in the \"log/diff\" family had an FILE* pointer in the\n data structure they pass around for a long time, but some codepaths\n used to always write to the standard output. As a preparatory step\n to make \"git format-patch\" available to the internal callers, these\n codepaths have been updated to consistently write into that FILE*\n instead.\n\n\n* js/mingw-parameter-less-c-functions (2016-06-20) 1 commit\n - mingw: let the build succeed with DEVELOPER=1\n\n Some platform-specific code had non-ANSI strict declarations of C\n functions that do not take any parameters, which has been\n corrected.\n\n Will merge to 'next'.\n\n\n* bc/cocci (2016-06-20) 8 commits\n - merge-recursive: convert merge_recursive_generic to object_id\n - merge-recursive: convert leaf functions to use struct object_id\n - merge-recursive: convert struct merge_file_info to object_id\n - merge-recursive: convert struct stage_data to use object_id\n - diff: rename struct diff_filespec's sha1_valid member\n - diff: convert struct diff_filespec to struct object_id\n - coccinelle: apply object_id Coccinelle transformations\n - contrib/coccinelle: add basic Coccinelle transforms\n\n Conversion from unsigned char sha1[20] to struct object_id\n continues.\n\n\n* jk/tzoffset-fix (2016-06-20) 3 commits\n - local_tzoffset: detect errors from tm_to_time_t\n - t0006: test various date formats\n - t0006: rename test-date's \"show\" to \"relative\"\n\n The internal code used to show local timezone offset is not\n prepared to handle timestamps beyond year 2100, and gave a\n bogus offset value to the caller. Use a more benign looking\n +0000 instead and let \"git log\" going in such a case, instead\n of aborting.\n\n Will merge to 'next'.\n\n\n* jk/add-i-diff-compact-heuristics (2016-06-16) 1 commit\n (merged to 'next' on 2016-06-27 at 568f892)\n + add--interactive: respect diff.compactionHeuristic\n\n \"git add -i/-p\" learned to honor diff.compactionHeuristic\n experimental knob, so that the user can work on the same hunk split\n as \"git diff\" output.\n\n Will merge to 'master'.\n\n\n* jk/big-and-future-archive-tar (2016-06-21) 2 commits\n - archive-tar: write extended headers for far-future mtime\n - archive-tar: write extended headers for file sizes >= 8GB\n\n \"git archive\" learned to handle files that are larger than 8GB and\n commits far in the future than expressible by the traditional US-TAR\n format.\n\n Expecting a reroll.\n ($gmane/298119)\n\n\n* jk/gpg-interface-cleanup (2016-06-17) 7 commits\n - gpg-interface: check gpg signature creation status\n - sign_buffer: use pipe_command\n - verify_signed_buffer: use pipe_command\n - run-command: add pipe_command helper\n - verify_signed_buffer: use tempfile object\n - verify_signed_buffer: drop pbuf variable\n - gpg-interface: use child_process.args\n\n A new run-command API function pipe_command() is introduced to\n sanely feed data to the standard input while capturing data from\n the standard output and the standard error of an external process,\n which is cumbersome to hand-roll correctly without deadlocking.\n\n The codepath to sign data in a prepared buffer with GPG has been\n updated to use this API to read from the status-fd to check for\n errors (instead of relying on GPG's exit status).\n\n Will merge to 'next'.\n\n\n* lf/sideband-returns-void (2016-06-16) 2 commits\n (merged to 'next' on 2016-06-27 at 558c781)\n + upload-pack.c: make send_client_data() return void\n + sideband.c: make send_sideband() return void\n\n A small internal API cleanup.\n\n Will merge to 'master'.\n\n\n* nd/graph-width-padded (2016-06-16) 2 commits\n - pretty.c: support |() forms\n - pretty: pass graph width to pretty formatting for use in '%>|(N)'\n\n \"log --graph --format=\" learned that \"%>|(N)\" specifies the width\n relative to the terminal's left edge, not relative to the area to\n draw text that is to the right of the ancestry-graph section. It\n also now accepts negative N that means the column limit is relative\n to the right border.\n\n Will merge to 'next'.\n\n\n* jk/bisect-show-tree (2016-06-16) 1 commit\n (merged to 'next' on 2016-06-27 at 6970f87e)\n + bisect: always call setup_revisions after init_revisions\n\n \"git bisect\" makes an internal call to \"git diff-tree\" when\n bisection finds the culprit, but this call did not initialize the\n data structure to pass to the diff-tree API correctly.\n\n Will merge to 'master'.\n\n\n* jk/parseopt-string-list (2016-06-13) 3 commits\n (merged to 'next' on 2016-06-27 at 27462e6)\n + blame,shortlog: don't make local option variables static\n + interpret-trailers: don't duplicate option strings\n + parse_opt_string_list: stop allocating new strings\n (this branch is used by jk/string-list-static-init.)\n\n The command line argument parsing that uses OPT_STRING_LIST() often\n made a copy of the argv[] element, which was unnecessary.\n\n Will merge to 'master'.\n\n\n* jk/repack-keep-unreachable (2016-06-14) 3 commits\n - repack: extend --keep-unreachable to loose objects\n - repack: add --keep-unreachable option\n - repack: document --unpack-unreachable option\n\n \"git repack\" learned the \"--keep-unreachable\" option, which sends\n loose unreachable objects to a pack instead of leaving them loose.\n This helps heuristics based on the number of loose objects\n (e.g. \"gc --auto\").\n\n Will merge to 'next'.\n\n\n* lf/recv-sideband-cleanup (2016-06-13) 1 commit\n - sideband.c: refactor recv_sideband()\n\n Code simplification. It however loses the atomicity of the output\n 9ac13ec9 (atomic write for sideband remote messages, 2006-10-11)\n tried to add to an once-much-simpler codebase.\n\n Expecting a reroll.\n\n\n* nd/test-lib-httpd-show-error-log-in-verbose (2016-06-13) 1 commit\n (merged to 'next' on 2016-06-27 at 9793d81)\n + lib-httpd.sh: print error.log on error\n\n Debugging aid.\n\n Will merge to 'master'.\n\n\n* sb/submodule-clone-retry (2016-06-13) 2 commits\n - submodule update: continue when a clone fails\n - submodule--helper: initial clone learns retry logic\n\n \"git submodule update\" that drives many \"git clone\" could\n eventually hit flaky servers/network conditions on one of the\n submodules; the command learned to retry the attempt.\n\n Will merge to 'next'.\n\n\n* jc/blame-reverse (2016-06-14) 2 commits\n - blame: dwim \"blame --reverse OLD\" as \"blame --reverse OLD..\"\n - blame: improve diagnosis for \"--reverse NEW\"\n\n It is a common mistake to say \"git blame --reverse OLD path\",\n expecting that the command line is dwimmed as if asking how lines\n in path in an old revision OLD have survived up to the current\n commit.\n\n Any supporters? Otherwise will drop.\n\n\n* km/fetch-do-not-free-remote-name (2016-06-14) 1 commit\n (merged to 'next' on 2016-06-27 at 4bc34c4)\n + builtin/fetch.c: don't free remote->name after fetch\n\n The ownership rule for the piece of memory that hold references to\n be fetched in \"git fetch\" was screwy, which has been cleaned up.\n\n Will merge to 'master'.\n\n\n* nd/shallow-deepen (2016-06-13) 27 commits\n - fetch, upload-pack: --deepen=N extends shallow boundary by N commits\n - upload-pack: add get_reachable_list()\n - upload-pack: split check_unreachable() in two, prep for get_reachable_list()\n - t5500, t5539: tests for shallow depth excluding a ref\n - clone: define shallow clone boundary with --shallow-exclude\n - fetch: define shallow boundary with --shallow-exclude\n - upload-pack: support define shallow boundary by excluding revisions\n - refs: add expand_ref()\n - t5500, t5539: tests for shallow depth since a specific date\n - clone: define shallow clone boundary based on time with --shallow-since\n - fetch: define shallow boundary with --shallow-since\n - upload-pack: add deepen-since to cut shallow repos based on time\n - shallow.c: implement a generic shallow boundary finder based on rev-list\n - fetch-pack: use a separate flag for fetch in deepening mode\n - fetch-pack.c: mark strings for translating\n - fetch-pack: use a common function for verbose printing\n - fetch-pack: use skip_prefix() instead of starts_with()\n - upload-pack: move rev-list code out of check_non_tip()\n - upload-pack: make check_non_tip() clean things up on error\n - upload-pack: tighten number parsing at \"deepen\" lines\n - upload-pack: use skip_prefix() instead of starts_with()\n - upload-pack: move \"unshallow\" sending code out of deepen()\n - upload-pack: remove unused variable \"backup\"\n - upload-pack: move \"shallow\" sending code out of deepen()\n - upload-pack: move shallow deepen code out of receive_needs()\n - transport-helper.c: refactor set_helper_option()\n - remote-curl.c: convert fetch_git() to use argv_array\n\n The existing \"git fetch --depth=\" option was hard to use\n correctly when making the history of an existing shallow clone\n deeper. A new option, \"--deepen=\", has been added to make this\n easier to use. \"git clone\" also learned \"--shallow-since=\"\n and \"--shallow-exclude=\" options to make it easier to specify\n \"I am interested only in the recent N months worth of history\" and\n \"Give me only the history since that version\".\n\n Rerolled.\n Needs review. What this topic attempts to achieve is worthwhile, I\n would think.\n\n\n* jk/send-pack-stdio (2016-06-10) 2 commits\n (merged to 'next' on 2016-06-27 at 2cfb0ff)\n + write_or_die: remove the unused write_or_whine() function\n + send-pack: use buffered I/O to talk to pack-objects\n\n Code clean-up.\n\n Will merge to 'master'.\n\n\n* pb/commit-editmsg-path (2016-06-09) 1 commit\n (merged to 'next' on 2016-06-27 at 0f01ce1)\n + builtin/commit.c: memoize git-path for COMMIT_EDITMSG\n\n Code clean-up.\n\n Will merge to 'master'.\n\n\n* jc/attr-more (2016-06-09) 8 commits\n - attr.c: outline the future plans by heavily commenting\n - attr.c: always pass check[] to collect_some_attrs()\n - attr.c: introduce empty_attr_check_elems()\n - attr.c: correct ugly hack for git_all_attrs()\n - attr.c: rename a local variable check\n - fixup! d5ad6c13\n - attr.c: pass struct git_attr_check down the callchain\n - attr.c: add push_stack() helper\n (this branch uses jc/attr; is tangled with sb/pathspec-label and sb/submodule-default-paths.)\n\n The beginning of long and tortuous journey to clean-up attribute\n subsystem implementation.\n\n Needs to be redone.\n\n\n* mh/ref-iterators (2016-06-20) 13 commits\n - for_each_reflog(): reimplement using iterators\n - dir_iterator: new API for iterating over a directory tree\n - for_each_reflog(): don't abort for bad references\n - do_for_each_ref(): reimplement using reference iteration\n - refs: introduce an iterator interface\n - ref_resolves_to_object(): new function\n - entry_resolves_to_object(): rename function from ref_resolves_to_object()\n - get_ref_cache(): only create an instance if there is a submodule\n - remote rm: handle symbolic refs correctly\n - delete_refs(): add a flags argument\n - refs: use name \"prefix\" consistently\n - do_for_each_ref(): move docstring to the header file\n - refs: remove unnecessary \"extern\" keywords\n (this branch is used by mh/ref-store; uses mh/split-under-lock; is tangled with mh/update-ref-errors.)\n\n The API to iterate over all the refs (i.e. for_each_ref(), etc.)\n has been revamped.\n\n Rebased on top of mh/split-under-lock.\n\n\n* ew/mboxrd-format-am (2016-06-06) 3 commits\n - am: support --patch-format=mboxrd\n - mailsplit: support unescaping mboxrd messages\n - pretty: support \"mboxrd\" output format\n\n Teach format-patch and mailsplit (hence \"am\") how a line that\n happens to begin with \"From \" in the e-mail message is quoted with\n \">\", so that these lines can be restored to their original shape.\n\n Will merge to 'next'.\n\n\n* mh/connect (2016-06-06) 10 commits\n - connect: [host:port] is legacy for ssh\n - connect: move ssh command line preparation to a separate function\n - connect: actively reject git:// urls with a user part\n - connect: change the --diag-url output to separate user and host\n - connect: make parse_connect_url() return the user part of the url as a separate value\n - connect: group CONNECT_DIAG_URL handling code\n - connect: make parse_connect_url() return separated host and port\n - connect: re-derive a host:port string from the separate host and port variables\n - connect: call get_host_and_port() earlier\n - connect: document why we sometimes call get_port after get_host_and_port\n\n Rewrite Git-URL parsing routine (hopefully) without changing any\n behaviour.\n\n\n* va/i18n-even-more (2016-06-17) 38 commits\n - i18n: branch: mark comment when editing branch description for translation\n - i18n: unmark die messages for translation\n - i18n: submodule: escape shell variables inside eval_gettext\n - i18n: submodule: join strings marked for translation\n - i18n: init-db: join message pieces\n - i18n: remote: allow translations to reorder message\n - i18n: remote: mark URL fallback text for translation\n - i18n: standardise messages\n - i18n: sequencer: add period to error message\n - i18n: merge: change command option help to lowercase\n - i18n: merge: mark messages for translation\n - i18n: notes: mark options for translation\n - i18n: notes: mark strings for translation\n - i18n: transport-helper.c: change N_() call to _()\n - i18n: bisect: mark strings for translation\n - t5523: use test_i18ngrep for negation\n - t4153: fix negated test_i18ngrep call\n - t9003: become resilient to GETTEXT_POISON\n - tests: unpack-trees: update to use test_i18n* functions\n - tests: use test_i18n* functions to suppress false positives\n - i18n: setup: mark strings for translation\n - i18n: rebase-interactive: mark comments of squash for translation\n - i18n: rebase-interactive: mark here-doc strings for translation\n - i18n: rebase-interactive: mark strings for translation\n - i18n: git-sh-setup.sh: mark strings for translation\n - t6030: update to use test_i18ncmp\n - i18n: bisect: simplify error message for i18n\n - i18n: rebase: mark placeholder for translation\n - i18n: rebase: fix marked string to use eval_gettext variant\n - merge-octopus: use die shell function from git-sh-setup.sh\n - i18n: merge-octopus: mark messages for translation\n - i18n: sequencer: mark string for translation\n - i18n: sequencer: mark entire sentences for translation\n - i18n: transport: mark strings for translation\n - i18n: advice: internationalize message for conflicts\n - i18n: advice: mark string about detached head for translation\n - i18n: builtin/remote.c: fix mark for translation\n - Merge branch 'jc/t2300-setup' into HEAD\n\n More markings of messages for i18n, with updates to various tests\n to pass GETTEXT_POISON tests.\n\n One patch from the original submission dropped due to conflicts\n with jk/upload-pack-hook, which is still in flux.\n\n Will merge to 'next'.\n\n\n* dk/blame-move-no-reason-for-1-line-context (2016-05-29) 1 commit\n - blame: require 0 context lines while finding moved lines with -M\n\n \"git blame -M\" missed a single line that was moved within the file.\n\n We may want to squash a test or two to this commit. Volunteers?\n\n\n* nd/worktree-lock (2016-06-13) 6 commits\n - worktree.c: find_worktree() search by path suffix\n - worktree: add \"unlock\" command\n - worktree: add \"lock\" command\n - worktree.c: add is_worktree_locked()\n - worktree.c: add is_main_worktree()\n - worktree.c: add find_worktree()\n (this branch uses nd/worktree-cleanup-post-head-protection.)\n\n \"git worktree prune\" protected worktrees that are marked as\n \"locked\" by creating a file in a known location. \"git worktree\"\n command learned a dedicated command pair to create and remoev such\n a file, so that the users do not have to do this with editor.\n\n Is everybody happy with this version?\n If so, will merge to 'next'.\n\n\n* jk/upload-pack-hook (2016-06-02) 7 commits\n - upload-pack: provide a hook for running pack-objects\n - t1308: do not get fooled by symbolic links to the source tree\n - config: add a notion of \"scope\"\n - config: return configset value for current_config_ functions\n - config: set up config_source for command-line config\n - git_config_parse_parameter: refactor cleanup code\n - git_config_with_options: drop \"found\" counting\n\n Allow a custom \"git upload-pack\" replacement to respond to\n \"fetch/clone\" request.\n\n Will merge to 'next'.\n\n\n* sb/submodule-default-paths (2016-06-20) 5 commits\n - completion: clone can recurse into submodules\n - clone: add --init-submodule= switch\n - submodule update: add `--init-default-path` switch\n - Merge branch 'sb/pathspec-label' into sb/submodule-default-paths\n - Merge branch 'jc/attr' into sb/submodule-default-paths\n (this branch uses jc/attr and sb/pathspec-label; is tangled with jc/attr-more.)\n\n Allow specifying the set of submodules the user is interested in on\n the command line of \"git clone\" that clones the superproject.\n\n\n* ep/http-curl-trace (2016-05-24) 2 commits\n (merged to 'next' on 2016-06-27 at c290515)\n + imap-send.c: introduce the GIT_TRACE_CURL enviroment variable\n + http.c: implement the GIT_TRACE_CURL environment variable\n\n HTTP transport gained an option to produce more detailed debugging\n trace.\n\n Will merge to 'master'.\n\n\n* jc/attr (2016-05-25) 18 commits\n - attr: support quoting pathname patterns in C style\n - attr: expose validity check for attribute names\n - attr: add counted string version of git_attr()\n - attr: add counted string version of git_check_attr()\n - attr: retire git_check_attrs() API\n - attr: convert git_check_attrs() callers to use the new API\n - attr: convert git_all_attrs() to use \"struct git_attr_check\"\n - attr: (re)introduce git_check_attr() and struct git_attr_check\n - attr: rename function and struct related to checking attributes\n - attr.c: plug small leak in parse_attr_line()\n - attr.c: tighten constness around \"git_attr\" structure\n - attr.c: simplify macroexpand_one()\n - attr.c: mark where #if DEBUG ends more clearly\n - attr.c: complete a sentence in a comment\n - attr.c: explain the lack of attr-name syntax check in parse_attr()\n - attr.c: update a stale comment on \"struct match_attr\"\n - attr.c: use strchrnul() to scan for one line\n - commit.c: use strchrnul() to scan for one line\n (this branch is used by jc/attr-more, sb/pathspec-label and sb/submodule-default-paths.)\n\n The attributes API has been updated so that it can later be\n optimized using the knowledge of which attributes are queried.\n\n I wanted to polish this topic further to make the attribute\n subsystem thread-ready, but because other topics depend on this\n topic and they do not (yet) need it to be thread-ready, let's merge\n this early part together with the dependent topics to 'next', and\n back-burner the threading enhancement to another day.\n\n\n* pb/bisect (2016-06-27) 9 commits\n - bisect--helper: `bisect_write` shell function in C\n - bisect--helper: `is_expected_rev` & `check_expected_revs` shell function in C\n - bisect--helper: `bisect_reset` shell function in C\n - wrapper: move is_empty_file() and rename it as is_empty_or_missing_file()\n - t6030: explicitly test for bisection cleanup\n - bisect--helper: `bisect_clean_state` shell function in C\n - bisect--helper: `write_terms` shell function in C\n - bisect: rewrite `check_term_format` shell function in C\n - bisect--helper: use OPT_CMDMODE instead of OPT_BOOL\n\n GSoC \"bisect\" topic.\n\n\n* sb/pathspec-label (2016-06-03) 6 commits\n - pathspec: disable preload-index when attribute pathspec magic is in use\n - pathspec: allow escaped query values\n - pathspec: allow querying for attributes\n - pathspec: move prefix check out of the inner loop\n - pathspec: move long magic parsing out of prefix_pathspec\n - Documentation: fix a typo\n (this branch is used by sb/submodule-default-paths; uses jc/attr; is tangled with jc/attr-more.)\n\n The pathspec mechanism learned \":(attr:X)$pattern\" pathspec magic\n to limit paths that match $pattern further by attribute settings.\n The preload-index mechanism is disabled when the new pathspec magic\n is in use (at least for now), because the attribute subsystem is\n not thread-ready.\n\n\n* nd/worktree-cleanup-post-head-protection (2016-05-24) 6 commits\n - worktree: simplify prefixing paths\n - worktree: avoid 0{40}, too many zeroes, hard to read\n - worktree.c: use is_dot_or_dotdot()\n - git-worktree.txt: keep subcommand listing in alphabetical order\n - worktree.c: rewrite mark_current_worktree() to avoid strbuf\n - completion: support git-worktree\n (this branch is used by nd/worktree-lock.)\n\n Further preparatory clean-up for \"worktree\" feature.\n\n Will merge to 'next'.\n\n\n* mh/split-under-lock (2016-06-13) 33 commits\n - lock_ref_sha1_basic(): only handle REF_NODEREF mode\n - commit_ref_update(): remove the flags parameter\n - lock_ref_for_update(): don't resolve symrefs\n - lock_ref_for_update(): don't re-read non-symbolic references\n - refs: resolve symbolic refs first\n - ref_transaction_update(): check refname_is_safe() at a minimum\n - unlock_ref(): move definition higher in the file\n - lock_ref_for_update(): new function\n - add_update(): initialize the whole ref_update\n - verify_refname_available(): adjust constness in declaration\n - refs: don't dereference on rename\n - refs: allow log-only updates\n - delete_branches(): use resolve_refdup()\n - ref_transaction_commit(): correctly report close_ref() failure\n - ref_transaction_create(): disallow recursive pruning\n - refs: make error messages more consistent\n - lock_ref_sha1_basic(): remove unneeded local variable\n - read_raw_ref(): move docstring to header file\n - read_raw_ref(): improve docstring\n - read_raw_ref(): rename symref argument to referent\n - read_raw_ref(): clear *type at start of function\n - read_raw_ref(): rename flags argument to type\n - ref_transaction_commit(): remove local variables n and updates\n - rename_ref(): remove unneeded local variable\n - commit_ref_update(): write error message to *err, not stderr\n - refname_is_safe(): insist that the refname already be normalized\n - refname_is_safe(): don't allow the empty string\n - refname_is_safe(): use skip_prefix()\n - remove_dir_recursively(): add docstring\n - safe_create_leading_directories(): improve docstring\n - read_raw_ref(): don't get confused by an empty directory\n - commit_ref(): if there is an empty dir in the way, delete it\n - t1404: demonstrate a bug resolving references\n (this branch is used by mh/ref-iterators, mh/ref-store and mh/update-ref-errors.)\n\n Further preparatory work on the refs API before the pluggable\n backend series can land.\n\n Replaced with a rerolled version.\n\n\n* jc/send-email-skip-backup (2016-04-12) 1 commit\n - send-email: detect and offer to skip backup files\n\n A careless invocation of \"git send-email directory/\" after editing\n 0001-change.patch with an editor often ends up sending both\n 0001-change.patch and its backup file, 0001-change.patch~, causing\n embarrassment and a minor confusion. Detect such an input and\n offer to skip the backup files when sending the patches out.\n\n Will merge to 'next'.\n Perhaps people will enhance it more once it gains more visibility.\n\n\n* kn/ref-filter-branch-list (2016-05-17) 17 commits\n - branch: implement '--format' option\n - branch: use ref-filter printing APIs\n - branch, tag: use porcelain output\n - ref-filter: allow porcelain to translate messages in the output\n - ref-filter: add `:dir` and `:base` options for ref printing atoms\n - ref-filter: make remote_ref_atom_parser() use refname_atom_parser_internal()\n - ref-filter: introduce symref_atom_parser() and refname_atom_parser()\n - ref-filter: introduce refname_atom_parser_internal()\n - ref-filter: make \"%(symref)\" atom work with the ':short' modifier\n - ref-filter: add support for %(upstream:track,nobracket)\n - ref-filter: make %(upstream:track) prints \"[gone]\" for invalid upstreams\n - ref-filter: introduce format_ref_array_item()\n - ref-filter: move get_head_description() from branch.c\n - ref-filter: modify \"%(objectname:short)\" to take length\n - ref-filter: implement %(if:equals=) and %(if:notequals=)\n - ref-filter: include reference to 'used_atom' within 'atom_value'\n - ref-filter: implement %(if), %(then), and %(else) atoms\n\n The code to list branches in \"git branch\" has been consolidated\n with the more generic ref-filter API.\n\n Rerolled.\n This also really needs review.\n\n\n* dt/index-helper (2016-06-27) 21 commits\n - unix-socket.c: add stub implementation when unix sockets are not supported\n - index-helper: indexhelper.exitafter config\n - trace: measure where the time is spent in the index-heavy operations\n - index-helper: optionally automatically run\n - index-helper: autorun mode\n - index-helper: don't run if already running\n - index-helper: kill mode\n - watchman: add a config option to enable the extension\n - unpack-trees: preserve index extensions\n - update-index: enable/disable watchman support\n - index-helper: use watchman to avoid refreshing index with lstat()\n - watchman: support watchman to reduce index refresh cost\n - read-cache: add watchman 'WAMA' extension\n - index-helper: log warnings\n - index-helper: add --detach\n - daemonize(): set a flag before exiting the main process\n - index-helper: add --strict\n - index-helper: new daemon for caching index and related stuff\n - pkt-line: add gentle version of packet_write\n - read-cache: allow to keep mmap'd memory after reading\n - read-cache.c: fix constness of verify_hdr()\n\n A new \"index-helper\" daemon has been introduced to give newly\n spawned Git process a quicker access to the data in the index, and\n optionally interface with the watchman daemon to further reduce the\n refresh cost.\n\n Expecting a reroll.\n ($gmane/298331, $gmane/298319).\n\n\n* jc/bundle (2016-03-03) 6 commits\n - index-pack: --clone-bundle option\n - Merge branch 'jc/index-pack' into jc/bundle\n - bundle v3: the beginning\n - bundle: keep a copy of bundle file name in the in-core bundle header\n - bundle: plug resource leak\n - bundle doc: 'verify' is not about verifying the bundle\n\n The beginning of \"split bundle\", which could be one of the\n ingredients to allow \"git clone\" traffic off of the core server\n network to CDN.\n\n While I think it would make it easier for people to experiment and\n build on if the topic is merged to 'next', I am at the same time a\n bit reluctant to merge an unproven new topic that introduces a new\n file format, which we may end up having to support til the end of\n time. Comments?\n\n\n* jc/merge-drop-old-syntax (2015-04-29) 1 commit\n - merge: drop 'git merge HEAD ' syntax\n\n Stop supporting \"git merge HEAD \" syntax that has\n been deprecated since October 2007, and issues a deprecation\n warning message since v2.5.0.\n\n It has been reported that git-gui still uses the deprecated syntax,\n which needs to be fixed before this final step can proceed.\n ($gmane/282594)\n"},{"id":"290341","messageId":"1467100871-2686-1-git-send-email-tboegi@web.de","threadId":"42727","inReplyTo":"xmqq37nyb4kp.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v3 0/3] unified auto CRLF handling, V3","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-06-28T08:01:11Z","receivedAt":"2016-06-28T07:53:39Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nUnified auto CRLF handling, V3\nI did the review, some minor changes: against v2:\n - SAFE_CRLF_RENORMALIZE is now 3, not 4 so that enum safe_crlf\n uses 0..3\n - V2 had unintentionally commented out the ident TC, fixed that\n\nWhat more may be useful to bring this series forward?\n\n\nTorsten Bögershausen (3):\n convert: unify the \"auto\" handling of CRLF\n read-cache: factor out get_sha1_from_index() helper\n correct ce_compare_data() in a middle of a merge\n\n Documentation/config.txt | 12 +++----\n Documentation/gitattributes.txt | 15 +++++----\n builtin/apply.c | 3 +-\n builtin/blame.c | 2 +-\n cache.h | 4 +++\n combine-diff.c | 3 +-\n convert.c | 65 +++++++++++++++++++++++-------------\n convert.h | 18 +++++++---\n diff.c | 3 +-\n dir.c | 2 +-\n read-cache.c | 33 +++++++++++-------\n sha1_file.c | 12 +++++--\n t/t0025-crlf-auto.sh | 4 +--\n t/t0027-auto-crlf.sh | 29 ++++++++--------\n t/t6038-merge-text-auto.sh | 74 ++++++++++++++++++++++++-----------------\n 15 files changed, 170 insertions(+), 109 deletions(-)\n\n-- \n2.0.0.rc1.6318.g0c2c796\n\n"},{"id":"290342","messageId":"1467100873-2725-1-git-send-email-tboegi@web.de","threadId":"42727","inReplyTo":"xmqq37nyb4kp.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v3 1/3] convert: unify the \"auto\" handling of CRLF","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-06-28T08:01:13Z","receivedAt":"2016-06-28T07:53:44Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nBefore this change,\n$ echo \"* text=auto\" >.gitattributes\n$ echo \"* eol=crlf\" >>.gitattributes\n\nwould have the same effect as\n$ echo \"* text\" >.gitattributes\n$ git config core.eol crlf\n\nSince the 'eol' attribute had higher priority than 'text=auto', this may\ncorrupt binary files and is not what most users expect to happen.\n\nMake the 'eol' attribute to obey 'text=auto' and now\n$ echo \"* text=auto\" >.gitattributes\n$ echo \"* eol=crlf\" >>.gitattributes\nbehaves the same as\n$ echo \"* text=auto\" >.gitattributes\n$ git config core.eol crlf\n\nIn other words,\n$ echo \"* text=auto eol=crlf\" >.gitattributes\nhas the same effect as\n$ git config core.autocrlf true\n\nand\n$ echo \"* text=auto eol=lf\" >.gitattributes\nhas the same effect as\n$ git config core.autocrlf input\n\nSigned-off-by: Torsten Bögershausen \n---\n Documentation/config.txt | 12 +++++-------\n Documentation/gitattributes.txt | 15 +++++++++------\n convert.c | 42 +++++++++++++++++++++--------------------\n convert.h | 3 ++-\n t/t0025-crlf-auto.sh | 4 ++--\n t/t0027-auto-crlf.sh | 29 ++++++++++++++--------------\n t/t6038-merge-text-auto.sh | 23 ++++++++++++++--------\n 7 files changed, 69 insertions(+), 59 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 4a27ad4..62ade8b 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -389,13 +389,11 @@ file with mixed line endings would be reported by the `core.safecrlf`\n mechanism.\n \n core.autocrlf::\n-\tSetting this variable to \"true\" is almost the same as setting\n-\tthe `text` attribute to \"auto\" on all files except that text\n-\tfiles are not guaranteed to be normalized: files that contain\n-\t`CRLF` in the repository will not be touched. Use this\n-\tsetting if you want to have `CRLF` line endings in your\n-\tworking directory even though the repository does not have\n-\tnormalized line endings. This variable can be set to 'input',\n+\tSetting this variable to \"true\" is the same as setting\n+\tthe `text` attribute to \"auto\" on all files and core.eol to \"crlf\".\n+\tSet to true if you want to have `CRLF` line endings in your\n+\tworking directory and the repository has LF line endings.\n+\tThis variable can be set to 'input',\n \tin which case no output conversion is performed.\n \n core.symlinks::\ndiff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt\nindex e3b1de8..d7a124b 100644\n--- a/Documentation/gitattributes.txt\n+++ b/Documentation/gitattributes.txt\n@@ -115,6 +115,7 @@ text file is normalized, its line endings are converted to LF in the\n repository. To control what line ending style is used in the working\n directory, use the `eol` attribute for a single file and the\n `core.eol` configuration variable for all text files.\n+Note that `core.autocrlf` overrides `core.eol`\n \n Set::\n \n@@ -130,8 +131,9 @@ Unset::\n Set to string value \"auto\"::\n \n \tWhen `text` is set to \"auto\", the path is marked for automatic\n-\tend-of-line normalization. If Git decides that the content is\n-\ttext, its line endings are normalized to LF on checkin.\n+\tend-of-line conversion. If Git decides that the content is\n+\ttext, its line endings are converted to LF on checkin.\n+\tWhen the file has been commited with CRLF, no conversion is done.\n \n Unspecified::\n \n@@ -146,7 +148,7 @@ unspecified.\n ^^^^^\n \n This attribute sets a specific line-ending style to be used in the\n-working directory. It enables end-of-line normalization without any\n+working directory. It enables end-of-line conversion without any\n content checks, effectively setting the `text` attribute.\n \n Set to string value \"crlf\"::\n@@ -186,9 +188,10 @@ the working directory, and prevent .jpg files from being normalized\n regardless of their content.\n \n ------------------------\n+* text=auto\n *.txt\t\ttext\n-*.vcproj\teol=crlf\n-*.sh\t\teol=lf\n+*.vcproj\ttext eol=crlf\n+*.sh\t\ttext eol=lf\n *.jpg\t\t-text\n ------------------------\n \n@@ -198,7 +201,7 @@ normalization in Git.\n \n If you simply want to have CRLF line endings in your working directory\n regardless of the repository you are working with, you can set the\n-config variable \"core.autocrlf\" without changing any attributes.\n+config variable \"core.autocrlf\" without using any attributes.\n \n ------------------------\n [core]\ndiff --git a/convert.c b/convert.c\nindex b1614bf..67d69b5 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -176,7 +176,9 @@ static enum eol output_eol(enum crlf_action crlf_action)\n \t\treturn EOL_LF;\n \tcase CRLF_UNDEFINED:\n \tcase CRLF_AUTO_CRLF:\n+\t\treturn EOL_CRLF;\n \tcase CRLF_AUTO_INPUT:\n+\t\treturn EOL_LF;\n \tcase CRLF_TEXT:\n \tcase CRLF_AUTO:\n \t\t/* fall through */\n@@ -254,17 +256,15 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \tif (crlf_action == CRLF_AUTO || crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n \t\tif (convert_is_binary(len, &stats))\n \t\t\treturn 0;\n-\n-\t\tif (crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n-\t\t\t/*\n-\t\t\t * If the file in the index has any CR in it, do not convert.\n-\t\t\t * This is the new safer autocrlf handling.\n-\t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n-\t\t\t\treturn 0;\n-\t\t}\n+\t\t/*\n+\t\t * If the file in the index has any CR in it, do not convert.\n+\t\t * This is the new safer autocrlf handling.\n+\t\t */\n+\t\tif (checksafe == SAFE_CRLF_RENORMALIZE)\n+\t\t\tchecksafe = SAFE_CRLF_FALSE;\n+\t\telse if (has_cr_in_index(path))\n+\t\t\treturn 0;\n \t}\n-\n \tcheck_safe_crlf(path, crlf_action, &stats, checksafe);\n \n \t/* Optimization: No CRLF? Nothing to convert, regardless. */\n@@ -320,12 +320,10 @@ static int crlf_to_worktree(const char *path, const char *src, size_t len,\n \t\treturn 0;\n \n \tif (crlf_action == CRLF_AUTO || crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n-\t\tif (crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n-\t\t\t/* If we have any CR or CRLF line endings, we do not touch it */\n-\t\t\t/* This is the new safer autocrlf-handling */\n-\t\t\tif (stats.lonecr || stats.crlf )\n-\t\t\t\treturn 0;\n-\t\t}\n+\t\t/* If we have any CR or CRLF line endings, we do not touch it */\n+\t\t/* This is the new safer autocrlf-handling */\n+\t\tif (stats.lonecr || stats.crlf )\n+\t\t\treturn 0;\n \n \t\tif (convert_is_binary(len, &stats))\n \t\t\treturn 0;\n@@ -786,7 +784,11 @@ static void convert_attrs(struct conv_attrs *ca, const char *path)\n \t\tca->drv = git_path_check_convert(ccheck + 2);\n \t\tif (ca->crlf_action != CRLF_BINARY) {\n \t\t\tenum eol eol_attr = git_path_check_eol(ccheck + 3);\n-\t\t\tif (eol_attr == EOL_LF)\n+\t\t\tif (ca->crlf_action == CRLF_AUTO && eol_attr == EOL_LF)\n+\t\t\t\tca->crlf_action = CRLF_AUTO_INPUT;\n+\t\t\telse if (ca->crlf_action == CRLF_AUTO && eol_attr == EOL_CRLF)\n+\t\t\t\tca->crlf_action = CRLF_AUTO_CRLF;\n+\t\t\telse if (eol_attr == EOL_LF)\n \t\t\t\tca->crlf_action = CRLF_TEXT_INPUT;\n \t\t\telse if (eol_attr == EOL_CRLF)\n \t\t\t\tca->crlf_action = CRLF_TEXT_CRLF;\n@@ -845,9 +847,9 @@ const char *get_convert_attr_ascii(const char *path)\n \tcase CRLF_AUTO:\n \t\treturn \"text=auto\";\n \tcase CRLF_AUTO_CRLF:\n-\t\treturn \"text=auto eol=crlf\"; /* This is not supported yet */\n+\t\treturn \"text=auto eol=crlf\";\n \tcase CRLF_AUTO_INPUT:\n-\t\treturn \"text=auto eol=lf\"; /* This is not supported yet */\n+\t\treturn \"text=auto eol=lf\";\n \t}\n \treturn \"\";\n }\n@@ -949,7 +951,7 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_FALSE);\n+\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE);\n }\n \n /*****************************************************************\ndiff --git a/convert.h b/convert.h\nindex ccf436b..82871a1 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -7,7 +7,8 @@\n enum safe_crlf {\n \tSAFE_CRLF_FALSE = 0,\n \tSAFE_CRLF_FAIL = 1,\n-\tSAFE_CRLF_WARN = 2\n+\tSAFE_CRLF_WARN = 2,\n+\tSAFE_CRLF_RENORMALIZE = 3\n };\n \n extern enum safe_crlf safe_crlf;\ndiff --git a/t/t0025-crlf-auto.sh b/t/t0025-crlf-auto.sh\nindex c164b46..d0bee08 100755\n--- a/t/t0025-crlf-auto.sh\n+++ b/t/t0025-crlf-auto.sh\n@@ -114,7 +114,7 @@ test_expect_success 'autocrlf=true does not normalize CRLF files' '\n \ttest -z \"$LFonlydiff\" -a -z \"$CRLFonlydiff\" -a -z \"$LFwithNULdiff\"\n '\n \n-test_expect_success 'text=auto, autocrlf=true _does_ normalize CRLF files' '\n+test_expect_success 'text=auto, autocrlf=true does not normalize CRLF files' '\n \n \trm -f .gitattributes tmp LFonly CRLFonly LFwithNUL &&\n \tgit config core.autocrlf true &&\n@@ -126,7 +126,7 @@ test_expect_success 'text=auto, autocrlf=true _does_ normalize CRLF files' '\n \tLFonlydiff=$(git diff LFonly) &&\n \tCRLFonlydiff=$(git diff CRLFonly) &&\n \tLFwithNULdiff=$(git diff LFwithNUL) &&\n-\ttest -z \"$LFonlydiff\" -a -n \"$CRLFonlydiff\" -a -z \"$LFwithNULdiff\"\n+\ttest -z \"$LFonlydiff\" -a -z \"$CRLFonlydiff\" -a -z \"$LFwithNULdiff\"\n '\n \n test_expect_success 'text=auto, autocrlf=true does not normalize binary files' '\ndiff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\nindex 9372589..2860d2d 100755\n--- a/t/t0027-auto-crlf.sh\n+++ b/t/t0027-auto-crlf.sh\n@@ -175,8 +175,8 @@ attr_ascii () {\n \ttext,lf) echo \"text eol=lf\" ;;\n \ttext,crlf) echo \"text eol=crlf\" ;;\n \tauto,) echo \"text=auto\" ;;\n-\tauto,lf) echo \"text eol=lf\" ;;\n-\tauto,crlf) echo \"text eol=crlf\" ;;\n+\tauto,lf) echo \"text=auto eol=lf\" ;;\n+\tauto,crlf) echo \"text=auto eol=crlf\" ;;\n \tlf,) echo \"text eol=lf\" ;;\n \tcrlf,) echo \"text eol=crlf\" ;;\n \t,) echo \"\" ;;\n@@ -397,10 +397,9 @@ commit_chk_wrnNNO \"\" \"\" false \"\" \"\" \"\" \"\"\n commit_chk_wrnNNO \"\" \"\" true LF_CRLF \"\" \"\" \"\" \"\"\n commit_chk_wrnNNO \"\" \"\" input \"\" \"\" \"\" \"\" \"\"\n \n-commit_chk_wrnNNO \"auto\" \"\" false \"$WILC\" \"$WICL\" \"$WAMIX\" \"\" \"\"\n-commit_chk_wrnNNO \"auto\" \"\" true LF_CRLF \"\" LF_CRLF \"\" \"\"\n-commit_chk_wrnNNO \"auto\" \"\" input \"\" CRLF_LF CRLF_LF \"\" \"\"\n-\n+commit_chk_wrnNNO \"auto\" \"\" false \"$WILC\" \"\" \"\" \"\" \"\"\n+commit_chk_wrnNNO \"auto\" \"\" true LF_CRLF \"\" \"\" \"\" \"\"\n+commit_chk_wrnNNO \"auto\" \"\" input \"\" \"\" \"\" \"\" \"\"\n for crlf in true false input\n do\n \tcommit_chk_wrnNNO -text \"\" $crlf \"\" \"\" \"\" \"\" \"\"\n@@ -408,8 +407,8 @@ do\n \tcommit_chk_wrnNNO -text crlf $crlf \"\" \"\" \"\" \"\" \"\"\n \tcommit_chk_wrnNNO \"\" lf $crlf \"\" CRLF_LF CRLF_LF \"\" CRLF_LF\n \tcommit_chk_wrnNNO \"\" crlf $crlf LF_CRLF \"\" LF_CRLF LF_CRLF \"\"\n-\tcommit_chk_wrnNNO auto lf \t$crlf \"\" CRLF_LF CRLF_LF \"\" CRLF_LF\n-\tcommit_chk_wrnNNO auto crlf \t$crlf LF_CRLF \"\" LF_CRLF LF_CRLF \"\"\n+\tcommit_chk_wrnNNO auto lf \t$crlf \"\" \"\" \"\" \"\" \"\"\n+\tcommit_chk_wrnNNO auto crlf \t$crlf LF_CRLF \"\" \"\" \"\" \"\"\n \tcommit_chk_wrnNNO text lf \t$crlf \"\" CRLF_LF CRLF_LF \"\" CRLF_LF\n \tcommit_chk_wrnNNO text crlf \t$crlf LF_CRLF \"\" LF_CRLF LF_CRLF \"\"\n done\n@@ -454,9 +453,9 @@ do\n \tcheck_in_repo_NNO -text \"\" $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n \tcheck_in_repo_NNO -text lf $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n \tcheck_in_repo_NNO -text crlf $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n-\tcheck_in_repo_NNO auto \"\" $crlf LF LF LF LF_mix_CR CRLF_nul\n-\tcheck_in_repo_NNO auto lf $crlf LF LF LF LF_mix_CR LF_nul\n-\tcheck_in_repo_NNO auto crlf $crlf LF LF LF LF_mix_CR LF_nul\n+\tcheck_in_repo_NNO auto \"\" $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n+\tcheck_in_repo_NNO auto lf $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n+\tcheck_in_repo_NNO auto crlf $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n \tcheck_in_repo_NNO text \"\" $crlf LF LF LF LF_mix_CR LF_nul\n \tcheck_in_repo_NNO text lf $crlf LF LF LF LF_mix_CR LF_nul\n \tcheck_in_repo_NNO text crlf $crlf LF LF LF LF_mix_CR LF_nul\n@@ -509,7 +508,7 @@ do\n \t\t\tcheckout_files text \"$id\" \"crlf\" \"$crlf\" \"$ceol\" CRLF CRLF CRLF CRLF_mix_CR CRLF_nul\n \t\t\t# currently the same as text, eol=XXX\n \t\t\tcheckout_files auto \"$id\" \"lf\" \"$crlf\" \"$ceol\" LF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n-\t\t\tcheckout_files auto \"$id\" \"crlf\" \"$crlf\" \"$ceol\" CRLF CRLF CRLF CRLF_mix_CR CRLF_nul\n+\t\t\tcheckout_files auto \"$id\" \"crlf\" \"$crlf\" \"$ceol\" CRLF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n \t\tdone\n \n \t\t# core.autocrlf false, different core.eol\n@@ -517,7 +516,7 @@ do\n \t\t# core.autocrlf true\n \t\tcheckout_files \"\" \"$id\" \"\" true \"$ceol\" CRLF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n \t\t# text: core.autocrlf = true overrides core.eol\n-\t\tcheckout_files auto \"$id\" \"\" true \"$ceol\" CRLF CRLF CRLF LF_mix_CR LF_nul\n+\t\tcheckout_files auto \"$id\" \"\" true \"$ceol\" CRLF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n \t\tcheckout_files text \"$id\" \"\" true \"$ceol\" CRLF CRLF CRLF CRLF_mix_CR CRLF_nul\n \t\t# text: core.autocrlf = input overrides core.eol\n \t\tcheckout_files text \"$id\" \"\" input \"$ceol\" LF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n@@ -531,8 +530,8 @@ do\n \tcheckout_files text \"$id\" \"\" false \"\" $NL CRLF $MIX_CRLF_LF $MIX_LF_CR $LFNUL\n \tcheckout_files text \"$id\" \"\" false native $NL CRLF $MIX_CRLF_LF $MIX_LF_CR $LFNUL\n \t# auto: core.autocrlf=false and core.eol unset(or native) uses native eol\n-\tcheckout_files auto \"$id\" \"\" false \"\" $NL CRLF $MIX_CRLF_LF LF_mix_CR LF_nul\n-\tcheckout_files auto \"$id\" \"\" false native $NL CRLF $MIX_CRLF_LF LF_mix_CR LF_nul\n+\tcheckout_files auto \"$id\" \"\" false \"\" $NL CRLF CRLF_mix_LF LF_mix_CR LF_nul\n+\tcheckout_files auto \"$id\" \"\" false native $NL CRLF CRLF_mix_LF LF_mix_CR LF_nul\n done\n \n # Should be the last test case: remove some files from the worktree\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 85c10b0..33b77ee 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -16,6 +16,13 @@ test_description='CRLF merge conflict across text=auto change\n \n test_have_prereq SED_STRIPS_CR && SED_OPTIONS=-b\n \n+compare_files () {\n+\ttr '\\015\\000' QN <\"$1\" >\"$1\".expect &&\n+\ttr '\\015\\000' QN <\"$2\" >\"$2\".actual &&\n+\ttest_cmp \"$1\".expect \"$2\".actual &&\n+\trm \"$1\".expect \"$2\".actual\n+}\n+\n test_expect_success setup '\n \tgit config core.autocrlf false &&\n \n@@ -30,7 +37,7 @@ test_expect_success setup '\n \tgit branch side &&\n \n \techo \"* text=auto\" >.gitattributes &&\n-\ttouch file &&\n+\techo first line >file &&\n \tgit add .gitattributes file &&\n \ttest_tick &&\n \tgit commit -m \"normalize file\" &&\n@@ -81,7 +88,7 @@ test_expect_success 'Merge after setting text=auto' '\n \trm -f .gitattributes &&\n \tgit reset --hard a &&\n \tgit merge b &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_success 'Merge addition of text=auto' '\n@@ -99,7 +106,7 @@ test_expect_success 'Merge addition of text=auto' '\n \trm -f .gitattributes &&\n \tgit reset --hard b &&\n \tgit merge a &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n@@ -121,7 +128,7 @@ test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n \tgit reset --hard a &&\n \ttest_must_fail git merge b &&\n \tfuzz_conflict file >file.fuzzy &&\n-\ttest_cmp expected file.fuzzy\n+\tcompare_files expected file.fuzzy\n '\n \n test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n@@ -143,7 +150,7 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \tgit reset --hard b &&\n \ttest_must_fail git merge a &&\n \tfuzz_conflict file >file.fuzzy &&\n-\ttest_cmp expected file.fuzzy\n+\tcompare_files expected file.fuzzy\n '\n \n test_expect_failure 'checkout -m after setting text=auto' '\n@@ -158,7 +165,7 @@ test_expect_failure 'checkout -m after setting text=auto' '\n \tgit reset --hard initial &&\n \tgit checkout a -- . &&\n \tgit checkout -m b &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_failure 'checkout -m addition of text=auto' '\n@@ -173,7 +180,7 @@ test_expect_failure 'checkout -m addition of text=auto' '\n \tgit reset --hard initial &&\n \tgit checkout b -- . &&\n \tgit checkout -m a &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_failure 'cherry-pick patch from after text=auto was added' '\n@@ -187,7 +194,7 @@ test_expect_failure 'cherry-pick patch from after text=auto was added' '\n \tgit reset --hard b &&\n \ttest_must_fail git cherry-pick a >err 2>&1 &&\n \tgrep \"[Nn]othing added\" err &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_success 'Test delete/normalize conflict' '\n-- \n2.0.0.rc1.6318.g0c2c796\n\n"},{"id":"290343","messageId":"1467100875-2764-1-git-send-email-tboegi@web.de","threadId":"42727","inReplyTo":"xmqq37nyb4kp.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v3 2/3] read-cache: factor out get_sha1_from_index() helper","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-06-28T08:01:15Z","receivedAt":"2016-06-28T07:53:47Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nFactor out the retrieval of the sha1 for a given path in\nread_blob_data_from_index() into the function get_sha1_from_index().\n\nThis will be used in the next commit, when convert.c can do the\nanalyze for \"text=auto\" without slurping the whole blob into memory\nat once.\n\nAdd a wrapper definition get_sha1_from_cache().\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 3 +++\n read-cache.c | 29 ++++++++++++++++++-----------\n 2 files changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex b829410..bd1210a 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n #define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at)\n #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec)\n #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n+#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n #endif\n \n enum object_type {\n@@ -1008,6 +1009,8 @@ static inline void *read_sha1_file(const unsigned char *sha1, enum object_type *\n \treturn read_sha1_file_extended(sha1, type, size, LOOKUP_REPLACE_OBJECT);\n }\n \n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path);\n+\n /*\n * This internal function is only declared here for the benefit of\n * lookup_replace_object(). Please do not call it directly.\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..a3ef967 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2263,13 +2263,27 @@ int index_name_is_other(const struct index_state *istate, const char *name,\n \n void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size)\n {\n-\tint pos, len;\n+\tconst unsigned char *sha1;\n \tunsigned long sz;\n \tenum object_type type;\n \tvoid *data;\n \n-\tlen = strlen(path);\n-\tpos = index_name_pos(istate, path, len);\n+\tsha1 = get_sha1_from_index(istate, path);\n+\tif (!sha1)\n+\t\treturn NULL;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\tif (size)\n+\t\t*size = sz;\n+\treturn data;\n+}\n+\n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path)\n+{\n+\tint pos = index_name_pos(istate, path, strlen(path));\n \tif (pos < 0) {\n \t\t/*\n \t\t * We might be in the middle of a merge, in which\n@@ -2285,14 +2299,7 @@ void *read_blob_data_from_index(struct index_state *istate, const char *path, un\n \t}\n \tif (pos < 0)\n \t\treturn NULL;\n-\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n-\tif (!data || type != OBJ_BLOB) {\n-\t\tfree(data);\n-\t\treturn NULL;\n-\t}\n-\tif (size)\n-\t\t*size = sz;\n-\treturn data;\n+\treturn (istate->cache[pos]->sha1);\n }\n \n void stat_validity_clear(struct stat_validity *sv)\n-- \n2.0.0.rc1.6318.g0c2c796\n\n"},{"id":"290344","messageId":"1467100876-2803-1-git-send-email-tboegi@web.de","threadId":"42727","inReplyTo":"xmqq37nyb4kp.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-06-28T08:01:16Z","receivedAt":"2016-06-28T07:53:49Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nThe following didn't work as expected:\n\n - In a middle of a merge\n - merge.renormalize is true,\n - .gitattributes = \"* text=auto\"\n - core.eol = crlf\n\nMerge a blob with CRLF \"first line\\r\\nsame line\\r\\n\" and a blob\nwith LF \"first line\\nsame line\\n\".\n\nThe expected result of the merge is \"first line\\nsame line\\n\".\n\nThe content in the working tree is \"first line\\r\\nsame line\\r\\n\",\nand ce_compare_data() should find that the content is clean and return 0.\n\nDeep down crlf_to_git() is invoked, to check if CRLF are converted or not.\n\nThe \"new safer autocrlf handling\" calls blob_has_cr().\n\nInstead of using the sha1 of the blob, (CRLF in this example),\nthe function get_sha1_from_index() is invoked.\nget_sha1_from_index() decides to return \"ours\" when in the middle of\nthe merge, which is LF.\n\nAs a result, the CRLF in the worktree are converted into LF before\nthe comparison.\nThe contents of LF and CRLF don't match any more.\n\nThe problem is that ce_compare_data() has ce->sha1, but the sha1 is lost\non it's way into blob_has_cr().\n\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthat blob_has_cr() looks at the appropriate blob.\n\nAdd a new parameter index_blob_sha1 to convert_to_git(), and forward the\nsha1 from ce_compare_data() into convert_to_git(). Other callers use NULL\nfor index_blob_sha1, and the sha1 is determined from path\nusing get_sha1_from_cache(path). This is the same handling as before.\n\nIn the same spirit, forward the sha1 into would_convert_to_git().\n\nWhile at it, rename has_cr_in_index() into blob_has_cr()\nand replace 0 with SAFE_CRLF_FALSE.\n\nAdd a TC in t6038 to have a test coverage under Linux.\n\nSigned-off-by: Torsten Bögershausen \n---\n builtin/apply.c | 3 ++-\n builtin/blame.c | 2 +-\n cache.h | 1 +\n combine-diff.c | 3 ++-\n convert.c | 43 ++++++++++++++++++++++++++------------\n convert.h | 15 ++++++++++----\n diff.c | 3 ++-\n dir.c | 2 +-\n read-cache.c | 4 +++-\n sha1_file.c | 12 ++++++++---\n t/t6038-merge-text-auto.sh | 51 +++++++++++++++++++++++++---------------------\n 11 files changed, 90 insertions(+), 49 deletions(-)\n\ndiff --git a/builtin/apply.c b/builtin/apply.c\nindex 42c610e..f33d9cf 100644\n--- a/builtin/apply.c\n+++ b/builtin/apply.c\n@@ -2142,7 +2142,8 @@ static int read_old_data(struct stat *st, const char *path, struct strbuf *buf)\n \tcase S_IFREG:\n \t\tif (strbuf_read_file(buf, path, st->st_size) != st->st_size)\n \t\t\treturn error(_(\"unable to open or read %s\"), path);\n-\t\tconvert_to_git(path, buf->buf, buf->len, buf, 0);\n+\t\tconvert_to_git(path, buf->buf, buf->len, buf,\n+\t\t\t SAFE_CRLF_FALSE, NULL);\n \t\treturn 0;\n \tdefault:\n \t\treturn -1;\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex e982fb8..6305a1e 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -2376,7 +2376,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,\n \t\tif (strbuf_read(&buf, 0, 0) < 0)\n \t\t\tdie_errno(\"failed to read from stdin\");\n \t}\n-\tconvert_to_git(path, buf.buf, buf.len, &buf, 0);\n+\tconvert_to_git(path, buf.buf, buf.len, &buf, SAFE_CRLF_FALSE, NULL);\n \torigin->file.ptr = buf.buf;\n \torigin->file.size = buf.len;\n \tpretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_sha1);\ndiff --git a/cache.h b/cache.h\nindex bd1210a..42b74b6 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_USE_SHA_NOT_PATH 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/combine-diff.c b/combine-diff.c\nindex 0e1d4b0..c4fa884 100644\n--- a/combine-diff.c\n+++ b/combine-diff.c\n@@ -1053,7 +1053,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,\n \t\t\tif (is_file) {\n \t\t\t\tstruct strbuf buf = STRBUF_INIT;\n \n-\t\t\t\tif (convert_to_git(elem->path, result, len, &buf, safe_crlf)) {\n+\t\t\t\tif (convert_to_git(elem->path, result, len,\n+\t\t\t\t\t\t &buf, safe_crlf, NULL)) {\n \t\t\t\t\tfree(result);\n \t\t\t\t\tresult = strbuf_detach(&buf, &len);\n \t\t\t\t\tresult_size = len;\ndiff --git a/convert.c b/convert.c\nindex 67d69b5..802ee7c 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -219,23 +219,28 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int blob_has_cr(const unsigned char *index_blob_sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n+\tint has_cr = 0;\n+\tenum object_type type;\n+\tif (!index_blob_sha1)\n+\t\treturn 0;\n+\tdata = read_sha1_file(index_blob_sha1, &type, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\tif (type == OBJ_BLOB)\n+\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n \tfree(data);\n \treturn has_cr;\n }\n \n static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t struct strbuf *buf,\n-\t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n+\t\t enum crlf_action crlf_action, enum safe_crlf checksafe,\n+\t\t const unsigned char *index_blob_sha1)\n {\n \tstruct text_stat stats;\n \tchar *dst;\n@@ -256,14 +261,23 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \tif (crlf_action == CRLF_AUTO || crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n \t\tif (convert_is_binary(len, &stats))\n \t\t\treturn 0;\n+\n \t\t/*\n \t\t * If the file in the index has any CR in it, do not convert.\n \t\t * This is the new safer autocrlf handling.\n \t\t */\n \t\tif (checksafe == SAFE_CRLF_RENORMALIZE)\n \t\t\tchecksafe = SAFE_CRLF_FALSE;\n-\t\telse if (has_cr_in_index(path))\n-\t\t\treturn 0;\n+\t\telse {\n+\t\t\t/*\n+\t\t\t * If the file in the index has any CR in it, do not convert.\n+\t\t\t * This is the new safer autocrlf handling.\n+\t\t\t */\n+\t\t\tif (!index_blob_sha1)\n+\t\t\t\tindex_blob_sha1 = get_sha1_from_cache(path);\n+\t\t\tif (blob_has_cr(index_blob_sha1))\n+\t\t\t\treturn 0;\n+\t\t}\n \t}\n \tcheck_safe_crlf(path, crlf_action, &stats, checksafe);\n \n@@ -855,7 +869,8 @@ const char *get_convert_attr_ascii(const char *path)\n }\n \n int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+\t\t struct strbuf *dst, enum safe_crlf checksafe,\n+\t\t const unsigned char *index_blob_sha1)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -876,7 +891,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe, index_blob_sha1);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -885,7 +900,8 @@ int convert_to_git(const char *path, const char *src, size_t len,\n }\n \n void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n-\t\t\t enum safe_crlf checksafe)\n+\t\t\t enum safe_crlf checksafe,\n+\t\t\t const unsigned char *index_blob_sha1)\n {\n \tstruct conv_attrs ca;\n \tconvert_attrs(&ca, path);\n@@ -896,7 +912,8 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action,\n+\t\t checksafe, index_blob_sha1);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \n@@ -951,7 +968,7 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE);\n+\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE, NULL);\n }\n \n /*****************************************************************\ndiff --git a/convert.h b/convert.h\nindex 82871a1..97d4272 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -39,19 +39,26 @@ extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+\t\t\t struct strbuf *dst, enum safe_crlf checksafe,\n+\t\t\t const unsigned char *index_blob_sha1);\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n \t\t\t struct strbuf *dst);\n-static inline int would_convert_to_git(const char *path)\n+static inline int would_convert_to_git(const char *path,\n+\t\t\t\t const unsigned char *index_blob_sha1)\n {\n-\treturn convert_to_git(path, NULL, 0, NULL, 0);\n+\treturn convert_to_git(path, NULL, 0, NULL, SAFE_CRLF_FALSE,\n+\t\t\t index_blob_sha1);\n }\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n extern void convert_to_git_filter_fd(const char *path, int fd,\n \t\t\t\t struct strbuf *dst,\n-\t\t\t\t enum safe_crlf checksafe);\n+\t\t\t\t enum safe_crlf checksafe,\n+\t\t\t\t const unsigned char *index_blob_sha1);\n+\n extern int would_convert_to_git_filter_fd(const char *path);\n \n /*****************************************************************\ndiff --git a/diff.c b/diff.c\nindex 059123c..b973469 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2800,7 +2800,8 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)\n \t\t/*\n \t\t * Convert from working tree format to canonical git format\n \t\t */\n-\t\tif (convert_to_git(s->path, s->data, s->size, &buf, crlf_warn)) {\n+\t\tif (convert_to_git(s->path, s->data, s->size, &buf,\n+\t\t\t\t crlf_warn, NULL)) {\n \t\t\tsize_t size = 0;\n \t\t\tmunmap(s->data, s->size);\n \t\t\ts->should_munmap = 0;\ndiff --git a/dir.c b/dir.c\nindex a4a9d9f..8709fc0 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -720,7 +720,7 @@ static int add_excludes(const char *fname, const char *base, int baselen,\n \t\t\t\t (pos = cache_name_pos(fname, strlen(fname))) >= 0 &&\n \t\t\t\t !ce_stage(active_cache[pos]) &&\n \t\t\t\t ce_uptodate(active_cache[pos]) &&\n-\t\t\t\t !would_convert_to_git(fname))\n+\t\t\t\t !would_convert_to_git(fname, NULL))\n \t\t\t\thashcpy(sha1_stat->sha1, active_cache[pos]->sha1);\n \t\t\telse\n \t\t\t\thash_sha1_file(buf, size, \"blob\", sha1_stat->sha1);\ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..c109b6d 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0f2aa0..52e5c6f 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3285,7 +3286,8 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n \t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE,\n+\t\t\t\t valid_sha1 ? sha1 : NULL)) {\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3313,13 +3315,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n \tconvert_to_git_filter_fd(path, fd, &sbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n+\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE,\n+\t\t\t\t valid_sha1 ? sha1 : NULL);\n \n \tif (write_object)\n \t\tret = write_sha1_file(sbuf.buf, sbuf.len, typename(OBJ_BLOB),\n@@ -3396,6 +3400,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_USE_SHA_NOT_PATH ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3406,7 +3412,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 33b77ee..5e8d5fa 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -91,16 +91,13 @@ test_expect_success 'Merge after setting text=auto' '\n \tcompare_files expected file\n '\n \n-test_expect_success 'Merge addition of text=auto' '\n+test_expect_success 'Merge addition of text=auto eol=LF' '\n+\tgit config core.eol lf &&\n \tcat <<-\\EOF >expected &&\n \tfirst line\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -109,17 +106,31 @@ test_expect_success 'Merge addition of text=auto' '\n \tcompare_files expected file\n '\n \n+test_expect_success 'Merge addition of text=auto eol=CRLF' '\n+\tgit config core.eol crlf &&\n+\tcat <<-\\EOF >expected &&\n+\tfirst line\n+\tsame line\n+\tEOF\n+\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n+\tgit config merge.renormalize true &&\n+\tgit rm -fr . &&\n+\trm -f .gitattributes &&\n+\tgit reset --hard b &&\n+\techo >&2 \"After git reset --hard b\" &&\n+\tgit ls-files -s --eol >&2 &&\n+\tgit merge a &&\n+\tcompare_files expected file\n+'\n+\n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n+\tgit config core.eol native &&\n \techo \"<<<<<<<\" >expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected &&\n-\t\techo ======= | append_cr >>expected\n-\telse\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected &&\n-\t\techo ======= >>expected\n-\tfi &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n+\techo ======= >>expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n@@ -135,15 +146,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \techo \"<<<<<<<\" >expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo ======= | append_cr >>expected &&\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected\n-\telse\n-\t\techo ======= >>expected &&\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected\n-\tfi &&\n+\techo ======= >>expected &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n \techo \">>>>>>>\" >>expected &&\n \tgit config merge.renormalize false &&\n \trm -f .gitattributes &&\n-- \n2.0.0.rc1.6318.g0c2c796\n\n"},{"id":"290506","messageId":"xmqqh9cc55wm.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"1467100876-2803-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-06-29T16:14:17Z","receivedAt":"2016-06-29T16:15:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen \n>\n> The following didn't work as expected:\n\nSorry for being slow (not in response but in understanding), but\nlet's examine the expectation first.\n\n> - In a middle of a merge\n> - merge.renormalize is true,\n\ngitattributes(5) tells us that this would make a \"virtual check-out\nand check-in of all stages\", i.e. each of the three blob objects\ninvolved is run through convert_to_working_tree(), and the result is\nrun through convert_to_git(). Now, what are these two operations\ntold to do?\n\n> - .gitattributes = \"* text=auto\"\n> - core.eol = crlf\n\ngit-config(1) tells us that a text file will be checked out with\nCRLF line endings, so convert_to_working_tree() would work that\nway. Even though core.eol entry in that manual page does not tell\nus anything, presumably convert_to_git() is expected to turn it back\nto LF line endings.\n\n> Merge a blob with CRLF \"first line\\r\\nsame line\\r\\n\" and a blob\n> with LF \"first line\\nsame line\\n\".\n\nThe former, when renormalized, should (we are discussing the\nexpectation, not necessarily what is done by today's code) be\nprocessed like this:\n\n * Pretend as if blob \"first line\\r\\nsame line\\r\\n\" is in the stage\n #0 of the index at the path;\n * Pretend as if the user said \"git checkout\" that path;\n * Pretend as if the user then said \"git add\" that path.\n\nThe checkout process hopefully does not blindly turn LF into CRLF,\nmaking it \"first line \\r\\r\\nsame line\\r\\rn\". Instead the (virtual)\nworking tree file will have \"first line\\r\\nsame line\\r\\n\".\n\nThen \"git add\" should turn that CRLF into LF, which would give us\n\"first line\\nsame line\\n\", but because the \"safer autocrlf\" rule\nprevents it from happening. The (real) index already has CR in it\nin the blob in stage #2, so the check-in process would (this is not\ndescribing the ideal, but what is done by today's code) disable\nCRLF->LF conversion.\n\nIs that the problem you are trying to solve?\n\nIf that is the case, I do not see how \"don't use stage #2 of the\nreal index; use the blob being renormalized instead\" would help.\nThe blob being renormalized may have CR in it, triggering that\n\"safer autocrlf\" rule and cause you the same trouble, no?\n\nTo me, it appears that if you consider that the \"safer autocrlf\" is\na sensible thing, you _must_ expect that the renormalization would\nnot work at all, in the scenario you presented. Also,\n\n> The expected result of the merge is \"first line\\nsame line\\n\".\n\nif you expect this, to me, it appears that you need to reject the\n\"safer autocrlf\", at least during renormalization, as a sensible\nthing. And if you _are_ to disable the \"safer autocrlf\" thing, then\nit does not matter what is currently in the index--the conversion\ncan happen solely based on the data being converted and the\nconfiguration and attribute settings.\n\nSo I still do not see why you want to pass \"no do not use stage #0\nor stage #2; use this blob instead\". Shouldn't you be just passing\na bit \"don't do the safer autocrlf thing in this convert_to_git()\ncall\" from renormalization codepath without doing anything else?\n\n> The content in the working tree is \"first line\\r\\nsame line\\r\\n\",\n> and ce_compare_data() should find that the content is clean and return 0.\n>\n> Deep down crlf_to_git() is invoked, to check if CRLF are converted or not.\n>\n> The \"new safer autocrlf handling\" calls blob_has_cr().\n>\n> Instead of using the sha1 of the blob, (CRLF in this example),\n> the function get_sha1_from_index() is invoked.\n> get_sha1_from_index() decides to return \"ours\" when in the middle of\n> the merge, which is LF.\n>\n> As a result, the CRLF in the worktree are converted into LF before\n> the comparison.\n> The contents of LF and CRLF don't match any more.\n>\n> The problem is that ce_compare_data() has ce->sha1, but the sha1 is lost\n> on it's way into blob_has_cr().\n>\n> Forwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\n> that blob_has_cr() looks at the appropriate blob.\n>\n> Add a new parameter index_blob_sha1 to convert_to_git(), and forward the\n> sha1 from ce_compare_data() into convert_to_git(). Other callers use NULL\n> for index_blob_sha1, and the sha1 is determined from path\n> using get_sha1_from_cache(path). This is the same handling as before.\n>\n> In the same spirit, forward the sha1 into would_convert_to_git().\n>\n> While at it, rename has_cr_in_index() into blob_has_cr()\n> and replace 0 with SAFE_CRLF_FALSE.\n>\n> Add a TC in t6038 to have a test coverage under Linux.\n>\n> Signed-off-by: Torsten Bögershausen \n> ---\n> builtin/apply.c | 3 ++-\n> builtin/blame.c | 2 +-\n> cache.h | 1 +\n> combine-diff.c | 3 ++-\n> convert.c | 43 ++++++++++++++++++++++++++------------\n> convert.h | 15 ++++++++++----\n> diff.c | 3 ++-\n> dir.c | 2 +-\n> read-cache.c | 4 +++-\n> sha1_file.c | 12 ++++++++---\n> t/t6038-merge-text-auto.sh | 51 +++++++++++++++++++++++++---------------------\n> 11 files changed, 90 insertions(+), 49 deletions(-)\n>\n> diff --git a/builtin/apply.c b/builtin/apply.c\n> index 42c610e..f33d9cf 100644\n> --- a/builtin/apply.c\n> +++ b/builtin/apply.c\n> @@ -2142,7 +2142,8 @@ static int read_old_data(struct stat *st, const char *path, struct strbuf *buf)\n> \tcase S_IFREG:\n> \t\tif (strbuf_read_file(buf, path, st->st_size) != st->st_size)\n> \t\t\treturn error(_(\"unable to open or read %s\"), path);\n> -\t\tconvert_to_git(path, buf->buf, buf->len, buf, 0);\n> +\t\tconvert_to_git(path, buf->buf, buf->len, buf,\n> +\t\t\t SAFE_CRLF_FALSE, NULL);\n> \t\treturn 0;\n> \tdefault:\n> \t\treturn -1;\n> diff --git a/builtin/blame.c b/builtin/blame.c\n> index e982fb8..6305a1e 100644\n> --- a/builtin/blame.c\n> +++ b/builtin/blame.c\n> @@ -2376,7 +2376,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,\n> \t\tif (strbuf_read(&buf, 0, 0) < 0)\n> \t\t\tdie_errno(\"failed to read from stdin\");\n> \t}\n> -\tconvert_to_git(path, buf.buf, buf.len, &buf, 0);\n> +\tconvert_to_git(path, buf.buf, buf.len, &buf, SAFE_CRLF_FALSE, NULL);\n> \torigin->file.ptr = buf.buf;\n> \torigin->file.size = buf.len;\n> \tpretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_sha1);\n> diff --git a/cache.h b/cache.h\n> index bd1210a..42b74b6 100644\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n> \n> #define HASH_WRITE_OBJECT 1\n> #define HASH_FORMAT_CHECK 2\n> +#define HASH_USE_SHA_NOT_PATH 4\n> extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n> extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n> \n> diff --git a/combine-diff.c b/combine-diff.c\n> index 0e1d4b0..c4fa884 100644\n> --- a/combine-diff.c\n> +++ b/combine-diff.c\n> @@ -1053,7 +1053,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,\n> \t\t\tif (is_file) {\n> \t\t\t\tstruct strbuf buf = STRBUF_INIT;\n> \n> -\t\t\t\tif (convert_to_git(elem->path, result, len, &buf, safe_crlf)) {\n> +\t\t\t\tif (convert_to_git(elem->path, result, len,\n> +\t\t\t\t\t\t &buf, safe_crlf, NULL)) {\n> \t\t\t\t\tfree(result);\n> \t\t\t\t\tresult = strbuf_detach(&buf, &len);\n> \t\t\t\t\tresult_size = len;\n> diff --git a/convert.c b/convert.c\n> index 67d69b5..802ee7c 100644\n> --- a/convert.c\n> +++ b/convert.c\n> @@ -219,23 +219,28 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n> \t}\n> }\n> \n> -static int has_cr_in_index(const char *path)\n> +static int blob_has_cr(const unsigned char *index_blob_sha1)\n> {\n> \tunsigned long sz;\n> \tvoid *data;\n> -\tint has_cr;\n> -\n> -\tdata = read_blob_data_from_cache(path, &sz);\n> +\tint has_cr = 0;\n> +\tenum object_type type;\n> +\tif (!index_blob_sha1)\n> +\t\treturn 0;\n> +\tdata = read_sha1_file(index_blob_sha1, &type, &sz);\n> \tif (!data)\n> \t\treturn 0;\n> -\thas_cr = memchr(data, '\\r', sz) != NULL;\n> +\tif (type == OBJ_BLOB)\n> +\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n> +\n> \tfree(data);\n> \treturn has_cr;\n> }\n> \n> static int crlf_to_git(const char *path, const char *src, size_t len,\n> \t\t struct strbuf *buf,\n> -\t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n> +\t\t enum crlf_action crlf_action, enum safe_crlf checksafe,\n> +\t\t const unsigned char *index_blob_sha1)\n> {\n> \tstruct text_stat stats;\n> \tchar *dst;\n> @@ -256,14 +261,23 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n> \tif (crlf_action == CRLF_AUTO || crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n> \t\tif (convert_is_binary(len, &stats))\n> \t\t\treturn 0;\n> +\n> \t\t/*\n> \t\t * If the file in the index has any CR in it, do not convert.\n> \t\t * This is the new safer autocrlf handling.\n> \t\t */\n> \t\tif (checksafe == SAFE_CRLF_RENORMALIZE)\n> \t\t\tchecksafe = SAFE_CRLF_FALSE;\n> -\t\telse if (has_cr_in_index(path))\n> -\t\t\treturn 0;\n> +\t\telse {\n> +\t\t\t/*\n> +\t\t\t * If the file in the index has any CR in it, do not convert.\n> +\t\t\t * This is the new safer autocrlf handling.\n> +\t\t\t */\n> +\t\t\tif (!index_blob_sha1)\n> +\t\t\t\tindex_blob_sha1 = get_sha1_from_cache(path);\n> +\t\t\tif (blob_has_cr(index_blob_sha1))\n> +\t\t\t\treturn 0;\n> +\t\t}\n> \t}\n> \tcheck_safe_crlf(path, crlf_action, &stats, checksafe);\n> \n> @@ -855,7 +869,8 @@ const char *get_convert_attr_ascii(const char *path)\n> }\n> \n> int convert_to_git(const char *path, const char *src, size_t len,\n> - struct strbuf *dst, enum safe_crlf checksafe)\n> +\t\t struct strbuf *dst, enum safe_crlf checksafe,\n> +\t\t const unsigned char *index_blob_sha1)\n> {\n> \tint ret = 0;\n> \tconst char *filter = NULL;\n> @@ -876,7 +891,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n> \t\tsrc = dst->buf;\n> \t\tlen = dst->len;\n> \t}\n> -\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n> +\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe, index_blob_sha1);\n> \tif (ret && dst) {\n> \t\tsrc = dst->buf;\n> \t\tlen = dst->len;\n> @@ -885,7 +900,8 @@ int convert_to_git(const char *path, const char *src, size_t len,\n> }\n> \n> void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n> -\t\t\t enum safe_crlf checksafe)\n> +\t\t\t enum safe_crlf checksafe,\n> +\t\t\t const unsigned char *index_blob_sha1)\n> {\n> \tstruct conv_attrs ca;\n> \tconvert_attrs(&ca, path);\n> @@ -896,7 +912,8 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n> \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n> \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n> \n> -\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n> +\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action,\n> +\t\t checksafe, index_blob_sha1);\n> \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n> }\n> \n> @@ -951,7 +968,7 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str\n> \t\tsrc = dst->buf;\n> \t\tlen = dst->len;\n> \t}\n> -\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE);\n> +\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE, NULL);\n> }\n> \n> /*****************************************************************\n> diff --git a/convert.h b/convert.h\n> index 82871a1..97d4272 100644\n> --- a/convert.h\n> +++ b/convert.h\n> @@ -39,19 +39,26 @@ extern const char *get_convert_attr_ascii(const char *path);\n> \n> /* returns 1 if *dst was used */\n> extern int convert_to_git(const char *path, const char *src, size_t len,\n> -\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n> +\t\t\t struct strbuf *dst, enum safe_crlf checksafe,\n> +\t\t\t const unsigned char *index_blob_sha1);\n> +\n> extern int convert_to_working_tree(const char *path, const char *src,\n> \t\t\t\t size_t len, struct strbuf *dst);\n> extern int renormalize_buffer(const char *path, const char *src, size_t len,\n> \t\t\t struct strbuf *dst);\n> -static inline int would_convert_to_git(const char *path)\n> +static inline int would_convert_to_git(const char *path,\n> +\t\t\t\t const unsigned char *index_blob_sha1)\n> {\n> -\treturn convert_to_git(path, NULL, 0, NULL, 0);\n> +\treturn convert_to_git(path, NULL, 0, NULL, SAFE_CRLF_FALSE,\n> +\t\t\t index_blob_sha1);\n> }\n> +\n> /* Precondition: would_convert_to_git_filter_fd(path) == true */\n> extern void convert_to_git_filter_fd(const char *path, int fd,\n> \t\t\t\t struct strbuf *dst,\n> -\t\t\t\t enum safe_crlf checksafe);\n> +\t\t\t\t enum safe_crlf checksafe,\n> +\t\t\t\t const unsigned char *index_blob_sha1);\n> +\n> extern int would_convert_to_git_filter_fd(const char *path);\n> \n> /*****************************************************************\n> diff --git a/diff.c b/diff.c\n> index 059123c..b973469 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -2800,7 +2800,8 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)\n> \t\t/*\n> \t\t * Convert from working tree format to canonical git format\n> \t\t */\n> -\t\tif (convert_to_git(s->path, s->data, s->size, &buf, crlf_warn)) {\n> +\t\tif (convert_to_git(s->path, s->data, s->size, &buf,\n> +\t\t\t\t crlf_warn, NULL)) {\n> \t\t\tsize_t size = 0;\n> \t\t\tmunmap(s->data, s->size);\n> \t\t\ts->should_munmap = 0;\n> diff --git a/dir.c b/dir.c\n> index a4a9d9f..8709fc0 100644\n> --- a/dir.c\n> +++ b/dir.c\n> @@ -720,7 +720,7 @@ static int add_excludes(const char *fname, const char *base, int baselen,\n> \t\t\t\t (pos = cache_name_pos(fname, strlen(fname))) >= 0 &&\n> \t\t\t\t !ce_stage(active_cache[pos]) &&\n> \t\t\t\t ce_uptodate(active_cache[pos]) &&\n> -\t\t\t\t !would_convert_to_git(fname))\n> +\t\t\t\t !would_convert_to_git(fname, NULL))\n> \t\t\t\thashcpy(sha1_stat->sha1, active_cache[pos]->sha1);\n> \t\t\telse\n> \t\t\t\thash_sha1_file(buf, size, \"blob\", sha1_stat->sha1);\n> diff --git a/read-cache.c b/read-cache.c\n> index a3ef967..c109b6d 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n> \n> \tif (fd >= 0) {\n> \t\tunsigned char sha1[20];\n> -\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n> +\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n> +\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n> +\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n> \t\t\tmatch = hashcmp(sha1, ce->sha1);\n> \t\t/* index_fd() closed the file descriptor already */\n> \t}\n> diff --git a/sha1_file.c b/sha1_file.c\n> index d0f2aa0..52e5c6f 100644\n> --- a/sha1_file.c\n> +++ b/sha1_file.c\n> @@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n> {\n> \tint ret, re_allocated = 0;\n> \tint write_object = flags & HASH_WRITE_OBJECT;\n> +\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n> \n> \tif (!type)\n> \t\ttype = OBJ_BLOB;\n> @@ -3285,7 +3286,8 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n> \tif ((type == OBJ_BLOB) && path) {\n> \t\tstruct strbuf nbuf = STRBUF_INIT;\n> \t\tif (convert_to_git(path, buf, size, &nbuf,\n> -\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n> +\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE,\n> +\t\t\t\t valid_sha1 ? sha1 : NULL)) {\n> \t\t\tbuf = strbuf_detach(&nbuf, &size);\n> \t\t\tre_allocated = 1;\n> \t\t}\n> @@ -3313,13 +3315,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n> {\n> \tint ret;\n> \tconst int write_object = flags & HASH_WRITE_OBJECT;\n> +\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n> \tstruct strbuf sbuf = STRBUF_INIT;\n> \n> \tassert(path);\n> \tassert(would_convert_to_git_filter_fd(path));\n> \n> \tconvert_to_git_filter_fd(path, fd, &sbuf,\n> -\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n> +\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE,\n> +\t\t\t\t valid_sha1 ? sha1 : NULL);\n> \n> \tif (write_object)\n> \t\tret = write_sha1_file(sbuf.buf, sbuf.len, typename(OBJ_BLOB),\n> @@ -3396,6 +3400,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n> \t enum object_type type, const char *path, unsigned flags)\n> {\n> \tint ret;\n> +\tconst unsigned char *sha1_ce;\n> +\tsha1_ce = flags & HASH_USE_SHA_NOT_PATH ? sha1 : NULL;\n> \n> \t/*\n> \t * Call xsize_t() only when needed to avoid potentially unnecessary\n> @@ -3406,7 +3412,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n> \telse if (!S_ISREG(st->st_mode))\n> \t\tret = index_pipe(sha1, fd, type, path, flags);\n> \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n> -\t\t (path && would_convert_to_git(path)))\n> +\t\t (path && would_convert_to_git(path,sha1_ce)))\n> \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n> \t\t\t\t flags);\n> \telse\n> diff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\n> index 33b77ee..5e8d5fa 100755\n> --- a/t/t6038-merge-text-auto.sh\n> +++ b/t/t6038-merge-text-auto.sh\n> @@ -91,16 +91,13 @@ test_expect_success 'Merge after setting text=auto' '\n> \tcompare_files expected file\n> '\n> \n> -test_expect_success 'Merge addition of text=auto' '\n> +test_expect_success 'Merge addition of text=auto eol=LF' '\n> +\tgit config core.eol lf &&\n> \tcat <<-\\EOF >expected &&\n> \tfirst line\n> \tsame line\n> \tEOF\n> \n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\tappend_cr expected.temp &&\n> -\t\tmv expected.temp expected\n> -\tfi &&\n> \tgit config merge.renormalize true &&\n> \tgit rm -fr . &&\n> \trm -f .gitattributes &&\n> @@ -109,17 +106,31 @@ test_expect_success 'Merge addition of text=auto' '\n> \tcompare_files expected file\n> '\n> \n> +test_expect_success 'Merge addition of text=auto eol=CRLF' '\n> +\tgit config core.eol crlf &&\n> +\tcat <<-\\EOF >expected &&\n> +\tfirst line\n> +\tsame line\n> +\tEOF\n> +\n> +\tappend_cr expected.temp &&\n> +\tmv expected.temp expected &&\n> +\tgit config merge.renormalize true &&\n> +\tgit rm -fr . &&\n> +\trm -f .gitattributes &&\n> +\tgit reset --hard b &&\n> +\techo >&2 \"After git reset --hard b\" &&\n> +\tgit ls-files -s --eol >&2 &&\n> +\tgit merge a &&\n> +\tcompare_files expected file\n> +'\n> +\n> test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n> +\tgit config core.eol native &&\n> \techo \"<<<<<<<\" >expected &&\n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\techo first line | append_cr >>expected &&\n> -\t\techo same line | append_cr >>expected &&\n> -\t\techo ======= | append_cr >>expected\n> -\telse\n> -\t\techo first line >>expected &&\n> -\t\techo same line >>expected &&\n> -\t\techo ======= >>expected\n> -\tfi &&\n> +\techo first line >>expected &&\n> +\techo same line >>expected &&\n> +\techo ======= >>expected &&\n> \techo first line | append_cr >>expected &&\n> \techo same line | append_cr >>expected &&\n> \techo \">>>>>>>\" >>expected &&\n> @@ -135,15 +146,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n> \techo \"<<<<<<<\" >expected &&\n> \techo first line | append_cr >>expected &&\n> \techo same line | append_cr >>expected &&\n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\techo ======= | append_cr >>expected &&\n> -\t\techo first line | append_cr >>expected &&\n> -\t\techo same line | append_cr >>expected\n> -\telse\n> -\t\techo ======= >>expected &&\n> -\t\techo first line >>expected &&\n> -\t\techo same line >>expected\n> -\tfi &&\n> +\techo ======= >>expected &&\n> +\techo first line >>expected &&\n> +\techo same line >>expected &&\n> \techo \">>>>>>>\" >>expected &&\n> \tgit config merge.renormalize false &&\n> \trm -f .gitattributes &&\n"},{"id":"290605","messageId":"62eb3d75-126e-427b-8701-d490e80e3501@web.de","threadId":"42727","inReplyTo":"xmqqh9cc55wm.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-06-30T16:52:20Z","receivedAt":"2016-06-30T16:52:39Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 29.06.16 18:14, Junio C Hamano wrote:\n> tboegi@web.de writes:\n> \n>> From: Torsten Bögershausen \n>>\n>> The following didn't work as expected:\n> \n> Sorry for being slow (not in response but in understanding), but\n> let's examine the expectation first.\n\nThanks for the patience.\nThere is one detail missing in my description:\nThe check if a file in the working tree is clean:\n\nstatic int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n{\n\tint match = -1;\n\tint fd = open(ce->name, O_RDONLY);\n\n\tif (fd >= 0) {\n\t\tunsigned char sha1[20];\n\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n------------------\nRemove the \"new feature\":\n\ngit diff read-cache.c\ndiff --git a/read-cache.c b/read-cache.c\nindex dd98b36..1f951ea 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -170,7 +170,7 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n if (fd >= 0) {\n unsigned char sha1[20];\n- unsigned flags = HASH_USE_SHA_NOT_PATH;\n+ unsigned flags = 0;//HASH_USE_SHA_NOT_PATH;\n memcpy(sha1, ce->sha1, sizeof(sha1));\n if (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n match = hashcmp(sha1, ce->sha1);\n\nThe the problem can be re-produced, and debugged with help of t6038:\n\nnot ok 5 - Merge addition of text=auto eol=CRLF\n\n\n> \n>> - In a middle of a merge\n>> - merge.renormalize is true,\n> \n> gitattributes(5) tells us that this would make a \"virtual check-out\n> and check-in of all stages\", i.e. each of the three blob objects\n> involved is run through convert_to_working_tree(), and the result is\n> run through convert_to_git(). Now, what are these two operations\n> told to do?\n> \n>> - .gitattributes = \"* text=auto\"\n>> - core.eol = crlf\n> \n> git-config(1) tells us that a text file will be checked out with\n> CRLF line endings, so convert_to_working_tree() would work that\n> way. Even though core.eol entry in that manual page does not tell\n> us anything, presumably convert_to_git() is expected to turn it back\n> to LF line endings.\n> \nYes, the git-config(1) may need improvements, but that can go as a later path.\n\n>> Merge a blob with CRLF \"first line\\r\\nsame line\\r\\n\" and a blob\n>> with LF \"first line\\nsame line\\n\".\nI may have written:\nMerge a blob with CRLF \"first line\\r\\nsame line\\r\\n\" \n(which is checked out as\n\"first line\\r\\nsame line\\r\\n\" \nin the working tree, and therefore clean)\nand a blob with LF \"first line\\nsame line\\n\".\n\n> \n> The former, when renormalized, should (we are discussing the\n> expectation, not necessarily what is done by today's code) be\n> processed like this:\n> \n> * Pretend as if blob \"first line\\r\\nsame line\\r\\n\" is in the stage\n> #0 of the index at the path;\n> * Pretend as if the user said \"git checkout\" that path;\n> * Pretend as if the user then said \"git add\" that path.\n> \n> The checkout process hopefully does not blindly turn LF into CRLF,\n> making it \"first line \\r\\r\\nsame line\\r\\rn\". Instead the (virtual)\n> working tree file will have \"first line\\r\\nsame line\\r\\n\".\nYes\n> \n> Then \"git add\" should turn that CRLF into LF, which would give us\n> \"first line\\nsame line\\n\", but because the \"safer autocrlf\" rule\n> prevents it from happening. The (real) index already has CR in it\n> in the blob in stage #2, so the check-in process would (this is not\n> describing the ideal, but what is done by today's code) disable\n> CRLF->LF conversion.\n> \n> Is that the problem you are trying to solve?\nNot sure, if that problem isn't already solved.\n\n> \n> If that is the case, I do not see how \"don't use stage #2 of the\n> real index; use the blob being renormalized instead\" would help.\n> The blob being renormalized may have CR in it, triggering that\n> \"safer autocrlf\" rule and cause you the same trouble, no?\n> \n> To me, it appears that if you consider that the \"safer autocrlf\" is\n> a sensible thing, you _must_ expect that the renormalization would\n> not work at all, in the scenario you presented. Also,\n> \n>> The expected result of the merge is \"first line\\nsame line\\n\".\n> \n> if you expect this, to me, it appears that you need to reject the\n> \"safer autocrlf\", at least during renormalization, as a sensible\n> thing. And if you _are_ to disable the \"safer autocrlf\" thing, then\n> it does not matter what is currently in the index--the conversion\n> can happen solely based on the data being converted and the\n> configuration and attribute settings.\n> \n> So I still do not see why you want to pass \"no do not use stage #0\n> or stage #2; use this blob instead\". Shouldn't you be just passing\n> a bit \"don't do the safer autocrlf thing in this convert_to_git()\n> call\" from renormalization codepath without doing anything else?\n> \n\nThis is my understanding:\n- git checkout and git add are working as expected:\n LF in blob gives CRLF in the working tree at checkout (if attr says \"auto\")\n CRLF in blob gives CRLF in the working tree at checkout.\n CRLF in the working tree gives CRLF in the blob at \"git add\", when\n the new safer CRLF handling says so.\n\n- Merges\n First the working tree is checked, if it is clean.\n I haven't digged deep enough to follow the whole code path,\n but that is what ce_compare_data() does, and it fails in a merge.\n\n Next thing is that the blobs are merged, and when a blob with CRLF\n is merged with a blob with LF, there are only conflicts :-(\n (Remember that both may have CRLF in the worktree)\n And here renormalize_buffer() comes in (only when \"merge.renormalize\" is true):\n - convert the blob with convert_to_working_tree_internal() into the working\n tree format (but do it in memory), and back to the \"blob\":\n - convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE, NULL);\n\n Note the the \"new safer autocrlf\" is suppressed here, because of SAFE_CRLF_RENORMALIZE.\n The result is, that both blobs are now normalized, having LF.\n After that, they are merged.\n\nThis is my understanding: there are 2 different things involved in t6038#5:\n- check for a clean worktree in ce_compare_data() # fails -> this is the problem to be solved\n- When merge.renormalize is true: renormalize the buffers before the blobs are merged # -> works\n\n\n\n\n"},{"id":"290715","messageId":"xmqq8txlvwip.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"62eb3d75-126e-427b-8701-d490e80e3501@web.de","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-01T22:11:42Z","receivedAt":"2016-07-01T22:11:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n>> The checkout process hopefully does not blindly turn LF into CRLF,\n>> making it \"first line \\r\\r\\nsame line\\r\\rn\". Instead the (virtual)\n>> working tree file will have \"first line\\r\\nsame line\\r\\n\".\n> Yes\n>> \n>> Then \"git add\" should turn that CRLF into LF, which would give us\n>> \"first line\\nsame line\\n\", but because the \"safer autocrlf\" rule\n>> prevents it from happening. The (real) index already has CR in it\n>> in the blob in stage #2, so the check-in process would (this is not\n>> describing the ideal, but what is done by today's code) disable\n>> CRLF->LF conversion.\n>> \n>> Is that the problem you are trying to solve?\n> Not sure, if that problem isn't already solved.\n\nOK, so in short, renormalizing three blobs in the conflicted index\nis done correcte because you do not allow safer-crlf kicks in during\nthe renormalization?\n\nSo how does ce_compare_data() judge \"first line\\r\\nsame line\\r\\n\"\nin stage #2 and working tree (both with CRLF) not to match? The\nCRLF in working tree data, when run convert_to_git(), will be kept\nas CRLF because of safer-crlf rule, and would match, no? Or do you\ndisable safer-crlf for that case, too?\n\n> This is my understanding:\n> - git checkout and git add are working as expected:\n> LF in blob gives CRLF in the working tree at checkout (if attr says \"auto\")\n> CRLF in blob gives CRLF in the working tree at checkout.\n\nOK.\n> CRLF in the working tree gives CRLF in the blob at \"git add\", when\n> the new safer CRLF handling says so.\n\nOK.\n\n> - Merges\n> First the working tree is checked, if it is clean.\n> I haven't digged deep enough to follow the whole code path,\n> but that is what ce_compare_data() does, and it fails in a merge.\n\nOK, I misunderstood what you are trying to fix. I somehow thought\nthat you wanted to tweak the renormalization of three blobs.\n\n> Next thing is that the blobs are merged, and when a blob with CRLF\n> is merged with a blob with LF, there are only conflicts :-(\n> (Remember that both may have CRLF in the worktree)\n> And here renormalize_buffer() comes in (only when \"merge.renormalize\" is true):\n> - convert the blob with convert_to_working_tree_internal() into the working\n> tree format (but do it in memory), and back to the \"blob\":\n> - convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE, NULL);\n>\n> Note the the \"new safer autocrlf\" is suppressed here, because of SAFE_CRLF_RENORMALIZE.\n\nThat makes sense.\n\n> - check for a clean worktree in ce_compare_data() # fails -> this is the problem to be solved\n\nBut it is unclear to me how that need an extra \"no, do not look at\nthe indexed blob, but look at this blob instead\"?\n\nThe relevant part is this:\n\ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..c109b6d 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\n\nI do not quite see how \"Don't use ce->sha1, but use sha1 given to\nyou\" flag affects this call, when sha1 and ce->sha1 are the same due\nto memcpy() just before the call?\n\n"},{"id":"290751","messageId":"574692d1-c8ae-9c2f-6b99-a01545b15051@telia.com","threadId":"42727","inReplyTo":"xmqq8txlvwip.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-07-02T18:41:33Z","receivedAt":"2016-07-02T18:47:40Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2016-07-02 00.11, Junio C Hamano wrote:\n[snip]\n> diff --git a/read-cache.c b/read-cache.c\n> index a3ef967..c109b6d 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n>\n> \tif (fd >= 0) {\n> \t\tunsigned char sha1[20];\n> -\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n> +\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n> +\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n> +\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n> \t\t\tmatch = hashcmp(sha1, ce->sha1);\n> \t\t/* index_fd() closed the file descriptor already */\n> \t}\n>\n> I do not quite see how \"Don't use ce->sha1, but use sha1 given to\n> you\" flag affects this call, when sha1 and ce->sha1 are the same due\n> to memcpy() just before the call?\n>\n\nLets step back a second. and try to explain whats going on.\n\nDo a merge (and renormalize the files).\n\nAt some point inside the merge, Git calls read-cache.c/ce_compare_data(),\n to check if the path named \"file\" is clean.\n According to the tree information, the path \"file\" has the sha1 99b633.\n #Note:\n #sha1 99b633 is \"first line\\nsame line\\n\"\n\nce_compare_data() starts the work:\nOK, lets run index_fd(...,ce->name,...)\n# index_fd will simulate a \"git add\" and return the sha1 (via the sha1 pointer)\n# after the hashing.\n\n# ce_compare_data() will later compare ce->sha1 with the result stored in\n# the local sha1. That's why a sha1 is in the parameter list.\n# To return the resulting hash:\n\nce_compare_data() calls index_fd(sha1, ..., ce->name, ...)\n\n#Down in index_fd():\n\nsha1_file.c/index_fd() calls index_core() (after consulting the attributes)\nindex_core() reads the file from the working tree into memory using\nread_in_full(), and after that index_mem() calls hash_sha1_file(buf)\nto calculate the sha1.\n\nBefore the hashing is done, index_mem() runs\nconvert_to_git(path, buf, size, &nbuf, SAFE_CRLF_FALSE)\nto convert \"blobs to git internal format\".\n\n\nHere, convert_to_git() consults the .gitattributes (again) to find out that\ncrlf_action is CRLF_AUTO in our case.\nThe \"new safer autocrlf handling\" says that if a blob as any CR, don't convert\nCRLFs at \"git add\".\n\nconvert.c/crlf_to_git() starts to do it's job:\n- look at buf (It has CRLF, conversion may be needed)\n- consult blob_has_cr()\n # Note: Before this patch, has_cr_in_index(path)) was used\n\n#Again, before this patch,\nhas_cr_in_index(path) calls read_blob_data_from_cache(path, &sz) to read the\nblob into memory.\n\nNow read_blob_data_from_cache() is a macro, and we end up in\nread_blob_data_from_index(&the_index, (path), (sz))\n\nread-cache.c/read_blob_data_from_index() starts its work:\n\tpos = index_name_pos(istate, path, len);\n\tif (pos < 0) {\n\t\t/*\n\t\t * We might be in the middle of a merge, in which\n\t\t * case we would read stage #2 (ours).\n\t\t */\n\n# And here, and this is important to notice, \"ours\" is sha1 ad55e2,\n# which corresponds to \"first line\\r\\nsame line\\r\\n\"\n# On our way in the callstack the information what to check had been lost:\n# blob 99b633 and nothing else.\n# read_blob_data_from_index() doesn't know the sha, but makes a guess,\n# derived from path.\n# The guess works OK for most callers: to take \"ours\" in a middle\n# of a merge, but not here.\n\n#Back to crlf_to_git():\nThe (wrong) blob has CRs in it, so crlf_to_git() decides not to convert CRLFs,\n(and this is wrong).\n\n# And now we go all the way back in the call chain:\n\nThe content in the worktree which is \"first line\\r\\nsame line\\r\\n\" is hashed:\nhash_sha1_file(buf, size, typename(type), sha1);\n#and the result is stored in the return pointer \"sha1\": ad55e2\n\n#Back to ce_compare_data():\nmatch = hashcmp(sha1, ce->sha1);\n\n#And here sha1=ad55e2 != ce->sha199b633\n\nThe whole merge is aborted:\n\n error: addinfo_cache failed for path 'file'\n file: unmerged (cbd69ec7cd12dd0989e853923867d94c8519aa52)\n file: unmerged (ad55e240aeb42e0d9a0e18d6d8b02dd82ee3e527)\n file: unmerged (99b633103c15c20cebebf821133ab526b0ff90b2)\n fatal: git write-tree failed to write a tree\n#Side note: cbd69ec is another file in the working tree.\n\n#######################\n\nWith this patch,\nce_compare_data() feeds the sha1 99b633 into\nblob_has_cr(const unsigned char *index_blob_sha1).\n\ncrlf_to_git() says:\n\"first line\\r\\nsame line\\r\\n\" in the worktree needs to be converted\ninto\n\"first line\\nsame line\\n\" as the \"new\" blob.\n\nThe new (converted) blob \"first line\\nsame line\\n\"\nis hashed into 99b633, so ce_compare_data() says\nfine, converting \"first line\\r\\nsame line\\r\\n\" will give sha1 99b633,\nlets continue with the merge, and do the normalization.\n\n# Note1:\n# The renormalization is outside what this patch fixes.\n# But I didn't manage to construct a test case, which triggered\n# \"fatal: git write-tree failed to write a tree\" without using\n# merge.renormalize\n\n# Note2:\n# ce_compare_data() has (if I understand things right)\n# no knowledge that a merge is going on\n# ce_compare_data() has now knowledge that a merge\n# with a merge with merge.renormalize=true is going on either.\n\n#Note3:\n# I hope that this explanation makes it more clear,\n# why ce->sha1 must be forwarded into blob_has_cr() (to check the right blob)\n# and that ce->name must be forwarded as path into crlf_to_git()\n# (to check the .gitattributes)\n\n"},{"id":"290877","messageId":"xmqqa8huvmpv.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"574692d1-c8ae-9c2f-6b99-a01545b15051@telia.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-06T14:57:00Z","receivedAt":"2016-07-06T14:57:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n> At some point inside the merge, Git calls read-cache.c/ce_compare_data(),\n> to check if the path named \"file\" is clean.\n> According to the tree information, the path \"file\" has the sha1 99b633.\n> #Note:\n> #sha1 99b633 is \"first line\\nsame line\\n\"\n\nI thought your scenario was that our side had CRLF both in the blob\nand in the working tree. So this is a different example? Our side\nhas LF in the blob that is checked out with CRLF in the working tree,\nand their side has CRLF in the blob?\n\n> ce_compare_data() starts the work:\n> OK, lets run index_fd(...,ce->name,...)\n> # index_fd will simulate a \"git add\" and return the sha1 (via the sha1 pointer)\n> # after the hashing.\n>\n> # ce_compare_data() will later compare ce->sha1 with the result stored in\n> # the local sha1. That's why a sha1 is in the parameter list.\n> # To return the resulting hash:\n>\n> ce_compare_data() calls index_fd(sha1, ..., ce->name, ...)\n>\n> #Down in index_fd():\n>\n> sha1_file.c/index_fd() calls index_core() (after consulting the attributes)\n> index_core() reads the file from the working tree into memory using\n> read_in_full(), and after that index_mem() calls hash_sha1_file(buf)\n> to calculate the sha1.\n>\n> Before the hashing is done, index_mem() runs\n> convert_to_git(path, buf, size, &nbuf, SAFE_CRLF_FALSE)\n> to convert \"blobs to git internal format\".\n>\n>\n> Here, convert_to_git() consults the .gitattributes (again) to find out that\n> crlf_action is CRLF_AUTO in our case.\n> The \"new safer autocrlf handling\" says that if a blob as any CR, don't convert\n> CRLFs at \"git add\".\n>\n> convert.c/crlf_to_git() starts to do it's job:\n> - look at buf (It has CRLF, conversion may be needed)\n> - consult blob_has_cr()\n> # Note: Before this patch, has_cr_in_index(path)) was used\n>\n> #Again, before this patch,\n> has_cr_in_index(path) calls read_blob_data_from_cache(path, &sz) to read the\n> blob into memory.\n>\n> Now read_blob_data_from_cache() is a macro, and we end up in\n> read_blob_data_from_index(&the_index, (path), (sz))\n>\n> read-cache.c/read_blob_data_from_index() starts its work:\n> \tpos = index_name_pos(istate, path, len);\n> \tif (pos < 0) {\n> \t\t/*\n> \t\t * We might be in the middle of a merge, in which\n> \t\t * case we would read stage #2 (ours).\n> \t\t */\n>\n> # And here, and this is important to notice, \"ours\" is sha1 ad55e2,\n> # which corresponds to \"first line\\r\\nsame line\\r\\n\"\n\nWhere did 99b633 come from then? There still is something missing\nin this description.\n\nPuzzled...\n"},{"id":"290966","messageId":"c36fe487-b8dc-9767-7fae-bee513dac0b2@web.de","threadId":"42727","inReplyTo":"xmqqa8huvmpv.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-07-07T17:16:09Z","receivedAt":"2016-07-07T17:20:18Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2016-07-06 16.57, Junio C Hamano wrote:\n> Torsten Bögershausen writes:\n> \n>> At some point inside the merge, Git calls read-cache.c/ce_compare_data(),\n>> to check if the path named \"file\" is clean.\n>> According to the tree information, the path \"file\" has the sha1 99b633.\n>> #Note:\n>> #sha1 99b633 is \"first line\\nsame line\\n\"\n> \n> I thought your scenario was that our side had CRLF both in the blob\n> and in the working tree. So this is a different example? Our side\n> has LF in the blob that is checked out with CRLF in the working tree,\n> and their side has CRLF in the blob?\n> \nThat was probably to confuse myself, and the rest of the world,\nsorry for confusion.\n\nThis is the callstack:\n\nmerge-recursive.c/add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n\t\tconst char *path, int stage, int refresh, int options)\n{\n\tstruct cache_entry *ce;\n\tce = make_cache_entry\n\tif (!ce)\n\t\treturn error(_(\"addinfo_cache failed for path '%s'\"), path);\n\treturn add_cache_entry(ce, options);\n}\n#Calls\nread-cache.c/make_cache_entry(path=file sha1=0x99b633 stage=0)\n\n\nstruct cache_entry *make_cache_entry(unsigned int mode,\n\t\tconst unsigned char *sha1, const char *path, int stage,\n\t\tunsigned int refresh_options)\n{\n [snip]\n\tret = refresh_cache_entry(ce, refresh_options);\n\tif (ret != ce)\n\t\tfree(ce);\n\treturn ret;\n}\n\n#Calls\nrefresh_cache_ent(&the_index, ce, options, NULL, NULL);\n\n#Calls\nie_modified()\n\n#Calls\nread-cache.c/ie_match_stat()\n\n#Calls\nchanged |= ce_modified_check_fs(ce, st);\n\n#Calls\nce_compare_data(path=file sha1=0x99b633)\n\n#Calls\nindex_fd(..., ..., ce->name, ...)\n#Note the sha is lost here, the parameter sha is the output.\n\nDeep down, has_cr_in_index(path) will look at ad55e2 instead,\nwhich is wrong.\n\n\n>> ce_compare_data() starts the work:\n>> OK, lets run index_fd(...,ce->name,...)\n>> # index_fd will simulate a \"git add\" and return the sha1 (via the sha1 pointer)\n>> # after the hashing.\n>>\n>> # ce_compare_data() will later compare ce->sha1 with the result stored in\n>> # the local sha1. That's why a sha1 is in the parameter list.\n>> # To return the resulting hash:\n>>\n>> ce_compare_data() calls index_fd(sha1, ..., ce->name, ...)\n>>\n>> #Down in index_fd():\n>>\n>> sha1_file.c/index_fd() calls index_core() (after consulting the attributes)\n>> index_core() reads the file from the working tree into memory using\n>> read_in_full(), and after that index_mem() calls hash_sha1_file(buf)\n>> to calculate the sha1.\n>>\n>> Before the hashing is done, index_mem() runs\n>> convert_to_git(path, buf, size, &nbuf, SAFE_CRLF_FALSE)\n>> to convert \"blobs to git internal format\".\n>>\n>>\n>> Here, convert_to_git() consults the .gitattributes (again) to find out that\n>> crlf_action is CRLF_AUTO in our case.\n>> The \"new safer autocrlf handling\" says that if a blob as any CR, don't convert\n>> CRLFs at \"git add\".\n>>\n>> convert.c/crlf_to_git() starts to do it's job:\n>> - look at buf (It has CRLF, conversion may be needed)\n>> - consult blob_has_cr()\n>> # Note: Before this patch, has_cr_in_index(path)) was used\n>>\n>> #Again, before this patch,\n>> has_cr_in_index(path) calls read_blob_data_from_cache(path, &sz) to read the\n>> blob into memory.\n>>\n>> Now read_blob_data_from_cache() is a macro, and we end up in\n>> read_blob_data_from_index(&the_index, (path), (sz))\n>>\n>> read-cache.c/read_blob_data_from_index() starts its work:\n>> \tpos = index_name_pos(istate, path, len);\n>> \tif (pos < 0) {\n>> \t\t/*\n>> \t\t * We might be in the middle of a merge, in which\n>> \t\t * case we would read stage #2 (ours).\n>> \t\t */\n>>\n>> # And here, and this is important to notice, \"ours\" is sha1 ad55e2,\n>> # which corresponds to \"first line\\r\\nsame line\\r\\n\"\n> \n> Where did 99b633 come from then? There still is something missing\n> in this description.\n> \n> Puzzled...\nThis is an unfinished attempt for a commit message:\n--------------------------------------------------\ncorrect ce_compare_data() at the end of a merge\n\nThe following didn't work as expected:\n\n - At the end of a merge\n - merge.renormalize is true,\n - .gitattributes = \"* text=auto\"\n - core.eol = crlf\n\nMerge a blob with CRLF \"first line\\r\\nsame line\\r\\n\" and a blob\nwith LF \"first line\\nsame line\\n\".\n\nThe expected result of the merge is \"first line\\nsame line\\n\".\n\nThe content in the working tree is \"first line\\r\\nsame line\\r\\n\",\nand ce_compare_data() should find that the content is clean and return 0.\n\nThe following callstack does not work:\nmerge-recursive.c/add_cacheinfo(path=file sha1=0x99b633)\n#Calls\nrefresh_cache_ent(&the_index, ce, options, NULL, NULL);\n\n#Calls\nie_modified()\n\n#Calls\nread-cache.c/ie_match_stat()\n\n#Calls\nchanged |= ce_modified_check_fs(ce, st);\n\n#Calls\nce_compare_data(path=file sha1=0x99b633)\n\n#Calls\nindex_fd(..., ..., ce->name, ...)\n#Note the sha is lost here.\n\n#Calls\nindex_core()\n\nindex_core() reads the file from the working tree into memory using\nread_in_full(), and after that index_mem() calls hash_sha1_file(buf)\nto calculate the sha1.\nBefore the hashing is done, index_mem() runs\nconvert_to_git(path, buf, size, &nbuf, SAFE_CRLF_FALSE)\nto convert \"blobs to git internal format\".\n\nHere, convert_to_git() consults the .gitattributes (again) to find out that\ncrlf_action is CRLF_AUTO in our case.\nThe \"new safer autocrlf handling\" says that if a blob as any CR, don't convert\nCRLFs at \"git add\".\n\nconvert.c/crlf_to_git() starts to do it's job:\n- look at buf (It has CRLF, conversion may be needed)\n- consult blob_has_cr()\n # Note: Before this patch, has_cr_in_index(path)) was used\n\n#Before this patch,\nhas_cr_in_index(path)\n#Calls\nread_blob_data_from_cache(path, &sz) to read the blob into memory.\n\nNow read_blob_data_from_cache() is a macro, and we end up in\nread_blob_data_from_index(&the_index, (path), (sz))\n\nread-cache.c/read_blob_data_from_index() starts its work:\n\tpos = index_name_pos(istate, path, len);\n\tif (pos < 0) {\n\t\t/*\n\t\t * We might be in the middle of a merge, in which\n\t\t * case we would read stage #2 (ours).\n\t\t */\n\n# And here, and this is important to notice, \"ours\" is sha1 ad55e2,\n# which corresponds to \"first line\\r\\nsame line\\r\\n\"\n\nThe result of the check is that the blob 99b633 doesn't seem\nto match the working tree, and the whold merge is aborted:\n error: addinfo_cache failed for path 'file'\n\nSolution:\nMake sure that ce_compare_data() forwards the source sha1 into crlf_to_git().\n\nReplace has_cr_in_index(path) with blob_has_cr(sha1), and forward\nthe source sha1 from ce_compare_data() into blob_has_cr(sha1).\n\nWhile at it, rename has_cr_in_index() into blob_has_cr()\nand replace 0 with SAFE_CRLF_FALSE.\n\nAdd a TC in t6038 to have a test coverage under Linux.\n\n\n\n\n\n\n\n\n\n"},{"id":"290969","messageId":"xmqqr3b5p9v0.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"c36fe487-b8dc-9767-7fae-bee513dac0b2@web.de","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-07T18:43:31Z","receivedAt":"2016-07-07T18:43:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n> This is the callstack:\n>\n> merge-recursive.c/add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n> \t\tconst char *path, int stage, int refresh, int options)\n> {\n> \tstruct cache_entry *ce;\n> \tce = make_cache_entry\n> \tif (!ce)\n> \t\treturn error(_(\"addinfo_cache failed for path '%s'\"), path);\n> \treturn add_cache_entry(ce, options);\n> }\n> #Calls\n> read-cache.c/make_cache_entry(path=file sha1=0x99b633 stage=0)\n>\n>\n> struct cache_entry *make_cache_entry(unsigned int mode,\n> \t\tconst unsigned char *sha1, const char *path, int stage,\n> \t\tunsigned int refresh_options)\n> {\n> [snip]\n> \tret = refresh_cache_entry(ce, refresh_options);\n> \tif (ret != ce)\n> \t\tfree(ce);\n> \treturn ret;\n> }\n>\n> #Calls\n> refresh_cache_ent(&the_index, ce, options, NULL, NULL);\n>\n> #Calls\n> ie_modified()\n>\n> #Calls\n> read-cache.c/ie_match_stat()\n>\n> #Calls\n> changed |= ce_modified_check_fs(ce, st);\n>\n> #Calls\n> ce_compare_data(path=file sha1=0x99b633)\n>\n> #Calls\n> index_fd(..., ..., ce->name, ...)\n> #Note the sha is lost here, the parameter sha is the output.\n>\n> Deep down, has_cr_in_index(path) will look at ad55e2 instead,\n> which is wrong.\n\nThe call to add_cacheinfo() is made with 99b633 to record the 3-way\nmerge result to the index in this callchain:\n\n merge_trees()\n -> git_merge_trees()\n -> process_renames() # does nothing for this case\n -> process_entry()\n -> merge_content()\n -> merge_file_special_markers()\n -> merge_file_1()\n -> merge_3way()\n -> ll_merge() # correctly handles the renormalization\n -> write_sha1_file() # this is what gives us 99b633\n -> update_file() # this is fed the 99b633 write_sha1_file() computed\n -> update_file_flags()\n -> read_sha1_file() # reads 99b633\n -> convert_to_working_tree()\n -> write_in_full() # updates the working tree\n -> add_cacheinfo() # to record 99b633 at \"file\" in the index\n\nSo refresh() may incorrectly find that 99b633 does not match what is\nin the filesystem if it looks at _any_ entry in the index. We know\nwe have written the file ourselves, so by definition it should match\n;-) Even though I am not sure why that should affect the overall\ncorrectness of the program (because we have written the correct\nresult to both working tree and to the index), this needs to be\nfixed.\n\nI am however not convinced passing the full SHA-1 is a good\nsolution. The make_cache_entry() function may be creating a cache\nentry to stuff in a non-default index (i.e. not \"the_index\"), but\nits caller does not have a way to tell it which index the resulting\ncache entry will go, and calls refresh_cache_entry(), which always\nassumes that the caller is interested in \"the_index\", so what\nadd_cacheinfo() does by calling make_cache_entry() feels wrong in\nthe first place. Also, the call from update_file_flags() knows that\nthe working tree is in sync with the resulting cache entry.\n\nPerhaps update_file_flags() should not even call add_cacheinfo()\nthere in the first place? I wonder if it should just call\nadd_file_to_index()---after all, even though the current code\nalready knows that 99b633 _is_ the result it wants in the index, it\nstill goes to the filesystem and rehashes.\n\nI dunno. I really do not like that extra sha1 argument added all\nover the callchain by this patch.\n\nOr did you mean other calls to add_cacheinfo()?\n"},{"id":"290999","messageId":"xmqqd1mpnl9v.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"xmqqr3b5p9v0.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-07T22:19:56Z","receivedAt":"2016-07-07T22:20:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> I am however not convinced passing the full SHA-1 is a good\n> solution. The make_cache_entry() function may be creating a cache\n> entry to stuff in a non-default index (i.e. not \"the_index\"), but\n> its caller does not have a way to tell it which index the resulting\n> cache entry will go, and calls refresh_cache_entry(), which always\n> assumes that the caller is interested in \"the_index\", so what\n> add_cacheinfo() does by calling make_cache_entry() feels wrong in\n> the first place. Also, the call from update_file_flags() knows that\n> the working tree is in sync with the resulting cache entry.\n>\n> Perhaps update_file_flags() should not even call add_cacheinfo()\n> there in the first place? I wonder if it should just call\n> add_file_to_index()---after all, even though the current code\n> already knows that 99b633 _is_ the result it wants in the index, it\n> still goes to the filesystem and rehashes.\n>\n> I dunno. I really do not like that extra sha1 argument added all\n> over the callchain by this patch.\n\nWhile the above still stands (i.e. I really want to see us find a\nsolution without the extra argument all over), I do not think\nadd_file_to_index() would work well here, as the stage #2 of the\nindex is contaminated with CRLF in this case, and getting rid of it\nis the whole point of renormalization, I would think. Of course,\nyou will get a normalized result if you merged in the other\ndirection, as your stage #2 would have normalized 99b633 when you\ncleanly merge the CRLF version from the other side with\nrenormalization, and \"git add\" of the cleanly merged result would\nkeep the normalized version.\n\nPerhaps that is an acceptable downside. If your side is not\nnormalized, and if the merge result is the same as what you already\nhave, perhaps you deserve to keep that unnormalized result, and\nyou'd be better off normalizing your tree before doing a merge, if\nyou do not like the fact that your indexed blobs have CRLF.\n\n> Or did you mean other calls to add_cacheinfo()?\n"},{"id":"291012","messageId":"2cbf12a6-2dca-8180-323b-f79638aa03bd@web.de","threadId":"42727","inReplyTo":"xmqqr3b5p9v0.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-07-08T07:52:30Z","receivedAt":"2016-07-08T07:53:03Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"\n\nOn 07/07/16 20:43, Junio C Hamano wrote:\n> Torsten Bögershausen writes:\n>\n>> This is the callstack:\n>>\n>> merge-recursive.c/add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n>> \t\tconst char *path, int stage, int refresh, int options)\n>> {\n>> \tstruct cache_entry *ce;\n>> \tce = make_cache_entry\n>> \tif (!ce)\n>> \t\treturn error(_(\"addinfo_cache failed for path '%s'\"), path);\n>> \treturn add_cache_entry(ce, options);\n>> }\n>> #Calls\n>> read-cache.c/make_cache_entry(path=file sha1=0x99b633 stage=0)\n>>\n>>\n>> struct cache_entry *make_cache_entry(unsigned int mode,\n>> \t\tconst unsigned char *sha1, const char *path, int stage,\n>> \t\tunsigned int refresh_options)\n>> {\n>> [snip]\n>> \tret = refresh_cache_entry(ce, refresh_options);\n>> \tif (ret != ce)\n>> \t\tfree(ce);\n>> \treturn ret;\n>> }\n>>\n>> #Calls\n>> refresh_cache_ent(&the_index, ce, options, NULL, NULL);\n>>\n>> #Calls\n>> ie_modified()\n>>\n>> #Calls\n>> read-cache.c/ie_match_stat()\n>>\n>> #Calls\n>> changed |= ce_modified_check_fs(ce, st);\n>>\n>> #Calls\n>> ce_compare_data(path=file sha1=0x99b633)\n>>\n>> #Calls\n>> index_fd(..., ..., ce->name, ...)\n>> #Note the sha is lost here, the parameter sha is the output.\n>>\n>> Deep down, has_cr_in_index(path) will look at ad55e2 instead,\n>> which is wrong.\n> The call to add_cacheinfo() is made with 99b633 to record the 3-way\n> merge result to the index in this callchain:\n>\n> merge_trees()\n> -> git_merge_trees()\n> -> process_renames() # does nothing for this case\n> -> process_entry()\n> -> merge_content()\n> -> merge_file_special_markers()\n> -> merge_file_1()\n> -> merge_3way()\n> -> ll_merge() # correctly handles the renormalization\n> -> write_sha1_file() # this is what gives us 99b633\n> -> update_file() # this is fed the 99b633 write_sha1_file() computed\n> -> update_file_flags()\n> -> read_sha1_file() # reads 99b633\n> -> convert_to_working_tree()\n> -> write_in_full() # updates the working tree\n> -> add_cacheinfo() # to record 99b633 at \"file\" in the index\n>\n> So refresh() may incorrectly find that 99b633 does not match what is\n> in the filesystem if it looks at _any_ entry in the index. We know\n> we have written the file ourselves, so by definition it should match\n> ;-) Even though I am not sure why that should affect the overall\n> correctness of the program (because we have written the correct\n> result to both working tree and to the index), this needs to be\n> fixed.\n>\n> I am however not convinced passing the full SHA-1 is a good\n> solution.\nIt seems at least that this is the correct way to do it.\n> The make_cache_entry() function may be creating a cache\n> entry to stuff in a non-default index (i.e. not \"the_index\"), but\n> its caller does not have a way to tell it which index the resulting\n> cache entry will go, and calls refresh_cache_entry(), which always\n> assumes that the caller is interested in \"the_index\", so what\n> add_cacheinfo() does by calling make_cache_entry() feels wrong in\n> the first place. Also, the call from update_file_flags() knows that\n> the working tree is in sync with the resulting cache entry.\n>\n> Perhaps update_file_flags() should not even call add_cacheinfo()\n> there in the first place? I wonder if it should just call\n> add_file_to_index()---after all, even though the current code\n> already knows that 99b633 _is_ the result it wants in the index, it\n> still goes to the filesystem and rehashes.\n>\n> I dunno. I really do not like that extra sha1 argument added all\n> over the callchain by this patch.\n>\n> Or did you mean other calls to add_cacheinfo()?\nI didn't mean too much - the whole call chain touches code where I am not able to\ncomment on details.\nI'm happy to test other implementations, if someone suggests a path, so to say.\n\n\n\n\n"},{"id":"291030","messageId":"30eb6711-a559-155b-d068-c7ab94f32871@web.de","threadId":"42727","inReplyTo":"xmqqr3b5p9v0.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-07-08T15:00:39Z","receivedAt":"2016-07-08T15:01:26Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"\n\nOn 07/07/16 20:43, Junio C Hamano wrote:\n > Torsten Bögershausen writes:\n >\n >> This is the call stack:\n >>\n >> merge-recursive.c/add_cacheinfo(unsigned int mode, const unsigned \nchar *sha1,\n >> \t\tconst char *path, int stage, int refresh, int options)\n >> {\n >> \tstruct cache_entry *ce;\n >> \tce = make_cache_entry\n >> \tif (!ce)\n >> \t\treturn error(_(\"addinfo_cache failed for path '%s'\"), path);\n >> \treturn add_cache_entry(ce, options);\n >> }\n >> #Calls\n >> read-cache.c/make_cache_entry(path=file sha1=0x99b633 stage=0)\n >>\n >>\n >> struct cache_entry *make_cache_entry(unsigned int mode,\n >> \t\tconst unsigned char *sha1, const char *path, int stage,\n >> \t\tunsigned int refresh_options)\n >> {\n >> [snip]\n >> \tret = refresh_cache_entry(ce, refresh_options);\n >> \tif (ret != ce)\n >> \t\tfree(ce);\n >> \treturn ret;\n >> }\n >>\n >> #Calls\n >> refresh_cache_ent(&the_index, ce, options, NULL, NULL);\n >>\n >> #Calls\n >> ie_modified()\n >>\n >> #Calls\n >> read-cache.c/ie_match_stat()\n >>\n >> #Calls\n >> changed |= ce_modified_check_fs(ce, st);\n >>\n >> #Calls\n >> ce_compare_data(path=file sha1=0x99b633)\n >>\n >> #Calls\n >> index_fd(..., ..., ce->name, ...)\n >> #Note the sha is lost here, the parameter sha is the output.\n >>\n >> Deep down, has_cr_in_index(path) will look at ad55e2 instead,\n >> which is wrong.\n > The call to add_cacheinfo() is made with 99b633 to record the 3-way\n > merge result to the index in this callchain:\n >\n > merge_trees()\n > -> git_merge_trees()\n > -> process_renames() # does nothing for this case\n > -> process_entry()\n > -> merge_content()\n > -> merge_file_special_markers()\n > -> merge_file_1()\n > -> merge_3way()\n > -> ll_merge() # correctly handles the renormalization\n > -> write_sha1_file() # this is what gives us 99b633\n > -> update_file() # this is fed the 99b633 write_sha1_file() computed\n > -> update_file_flags()\n > -> read_sha1_file() # reads 99b633\n > -> convert_to_working_tree()\n > -> write_in_full() # updates the working tree\n > -> add_cacheinfo() # to record 99b633 at \"file\" in the index\n >\n > So refresh() may incorrectly find that 99b633 does not match what is\n > in the filesystem if it looks at _any_ entry in the index. We know\n > we have written the file ourselves, so by definition it should match\n > ;-)\n\nDoes it, always ?\nThere was a lengthy discussion around January, if\ngit checkout -f\nshould always give a clean workspace.\n(My suggestion was yes, please), but the conclusion\nwas to always check the other way around:\nWhat would \"git add\" say ?\nIf the smudge/clean don't provide\na round trip, then the worktree should not be clean.\n\nMy understanding is, that after merge, the filters must be round trip\n(and all other conversions), otherwise the merge willl fail.\n\nIn this case the merge fails because of a bug in Git.\n"},{"id":"291035","messageId":"xmqqmvlsm6hu.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"2cbf12a6-2dca-8180-323b-f79638aa03bd@web.de","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-08T16:36:45Z","receivedAt":"2016-07-08T16:37:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n>> I dunno. I really do not like that extra sha1 argument added all\n>> over the callchain by this patch.\n>>\n>> Or did you mean other calls to add_cacheinfo()?\n>\n> I didn't mean too much - the whole call chain touches code where I\n> am not able to comment on details.\n> I'm happy to test other implementations, if someone suggests a\n> path, so to say.\n\nI did a bit of experiment.\n\nWhen 1/3 alone is applied, and then only changes for t/t6038 from\n3/3 is picked, (i.e. we do not add the extra \"don't look at index,\ncheck this contents\"), your \"Merge addition of text=auto eol=CRLF\"\ntest would fail.\n\nAnd then with this further on top:\n\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex b880ae5..628c8ed 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -202,6 +202,9 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n \t\tconst char *path, int stage, int refresh, int options)\n {\n \tstruct cache_entry *ce;\n+\n+\tif (!stage)\n+\t\tremove_file_from_cache(path);\n \tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage,\n \t\t\t (refresh ? (CE_MATCH_REFRESH |\n \t\t\t\t\t CE_MATCH_IGNORE_MISSING) : 0 ));\n\nI get the renomalized result registered to the index.\n\nThe root cause of the \"bug\", I think, is that the \"safer crlf\" is\nfundamentally a flawed mechanism and does not mesh well with the\nrest of the system. If you start from an index without the path and\nadd a CRLF file in the working tree, you get LF blob (as expected),\nbut if you happen to already have CRLF file for the path in the\nindex, you get CRLF blob (as claimed to be \"safer\"), which\nessentially means that with that mechanism in place, you do not know\nblob with what object name you would get, given the same working\ntree file with the same configuration setting.\n\nIn this codepath, we already have three stages in the index, and the\nstage #2 happens to be a CRLF blob in your test. make_cache_entry()\nwants to refresh, which involves comparing the working tree file to\nsee if it hashes down to the same object name as ce->sha1, which is\nthe result of the \"normalizing\" merge that uses LF. But if you\nmerge the other way, the stage #2 would have LF blob, and that would\nnot prevent the working tree file with CRLF written from the result\nof the \"normalizing\" merge to be cleaned again to LF blob to\nround-trip.\n\nThe \"this is a minimum workaround\" patch above disables the \"safer\ncrlf\" conversion by removing the stages that will get in the way.\nWhen we are recording the cleanly resolved result of a merge, we\nknow what the resulting ce->sha1 should be, and we also know the\nhigher stages for the path will be removed, so there is no point\nkeeping the higher stages in the index and get them looked at by the\n\"safer crlf\" codepath.\n\n"},{"id":"291040","messageId":"f78fa94d-abd2-05a2-c411-15e2ffdb7dae@web.de","threadId":"42727","inReplyTo":"xmqqmvlsm6hu.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-07-08T17:13:40Z","receivedAt":"2016-07-08T17:14:34Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"\n\nOn 07/08/2016 06:36 PM, Junio C Hamano wrote:\n> Torsten Bögershausen writes:\n>\n>>> I dunno. I really do not like that extra sha1 argument added all\n>>> over the callchain by this patch.\n>>>\n>>> Or did you mean other calls to add_cacheinfo()?\n>>\n>> I didn't mean too much - the whole call chain touches code where I\n>> am not able to comment on details.\n>> I'm happy to test other implementations, if someone suggests a\n>> path, so to say.\n>\n> I did a bit of experiment.\n>\n> When 1/3 alone is applied, and then only changes for t/t6038 from\n> 3/3 is picked, (i.e. we do not add the extra \"don't look at index,\n> check this contents\"), your \"Merge addition of text=auto eol=CRLF\"\n> test would fail.\n>\n> And then with this further on top:\n>\n> diff --git a/merge-recursive.c b/merge-recursive.c\n> index b880ae5..628c8ed 100644\n> --- a/merge-recursive.c\n> +++ b/merge-recursive.c\n> @@ -202,6 +202,9 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n> \t\tconst char *path, int stage, int refresh, int options)\n> {\n> \tstruct cache_entry *ce;\n> +\n> +\tif (!stage)\n> +\t\tremove_file_from_cache(path);\n> \tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage,\n> \t\t\t (refresh ? (CE_MATCH_REFRESH |\n> \t\t\t\t\t CE_MATCH_IGNORE_MISSING) : 0 ));\n>\nThanks :-)\nDid that experiment made it to a branch somewhere ?\n"},{"id":"291044","messageId":"xmqq1t34m47z.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"f78fa94d-abd2-05a2-c411-15e2ffdb7dae@web.de","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-08T17:25:52Z","receivedAt":"2016-07-08T17:26:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n>> And then with this further on top:\n>>\n>> diff --git a/merge-recursive.c b/merge-recursive.c\n>> index b880ae5..628c8ed 100644\n>> --- a/merge-recursive.c\n>> +++ b/merge-recursive.c\n>> @@ -202,6 +202,9 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n>> \t\tconst char *path, int stage, int refresh, int options)\n>> {\n>> \tstruct cache_entry *ce;\n>> +\n>> +\tif (!stage)\n>> +\t\tremove_file_from_cache(path);\n>> \tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage,\n>> \t\t\t (refresh ? (CE_MATCH_REFRESH |\n>> \t\t\t\t\t CE_MATCH_IGNORE_MISSING) : 0 ));\n>>\n> Thanks :-)\n> Did that experiment made it to a branch somewhere ?\n\nNot yet. As I called it \"experiment\", it was merely to demonstrate\nthat there are less intrusive ways to kill the \"safer crlf\" we may\nwant to consider first before passing an extra blob object name\naround.\n"},{"id":"291046","messageId":"xmqqwpkwko3w.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"xmqq1t34m47z.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-08T17:59:15Z","receivedAt":"2016-07-08T17:59:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Not yet. As I called it \"experiment\", it was merely to demonstrate\n> that there are less intrusive ways to kill the \"safer crlf\" we may\n> want to consider first before passing an extra blob object name\n> around.\n\nHere is another approach, that probably is less intrusive. It\ncontains the test updates from your 3/3, and you can apply it\ndirectly on top of 65237284 (convert: unify the \"auto\" handling of\nCRLF, 2016-06-28), which is the result of applying your 1/3 on top\nof v2.9.0-rc0~11^2 (convert.c: ident + core.autocrlf didn't work,\n2016-04-25).\n\nInstead of letting add_cacheinfo() to refresh individual cache\nentries as they are added, we just leave them as they are, which\nwould bypass the make_cache_entry() -> refresh_cache_entry() ->\n... -> ce_compare_data() -> \"safer crlf\" callchain. After we are\ndone all (in builtin/merge.c that directly calls merge_recursive(),\nand also merge_recursive_generic()), we let refresh_cache() to take\ncare of the refreshing of the stat information--by the time this\nhappens, the stale cache entries that had CRLF in stage #2 that were\ncarried over before the renormalizing merge started will all be gone\nand will not interfere.\n\n\n builtin/merge.c | 3 ++-\n merge-recursive.c | 20 +++++++++---------\n t/t6038-merge-text-auto.sh | 51 +++++++++++++++++++++++++---------------------\n 3 files changed, 40 insertions(+), 34 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 101ffef..d5bf68d 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -678,7 +678,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n \n \t\thold_locked_index(&lock, 1);\n \t\tclean = merge_recursive(&o, head,\n-\t\t\t\tremoteheads->item, reversed, &result);\n+\t\t\t\t\tremoteheads->item, reversed, &result);\n+\t\trefresh_cache(CE_MATCH_REFRESH | CE_MATCH_IGNORE_MISSING);\n \t\tif (active_cache_changed &&\n \t\t write_locked_index(&the_index, &lock, COMMIT_LOCK))\n \t\t\tdie (_(\"unable to write %s\"), get_index_file());\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex b880ae5..8aaf1b5 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -199,14 +199,14 @@ static void output_commit_title(struct merge_options *o, struct commit *commit)\n }\n \n static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n-\t\tconst char *path, int stage, int refresh, int options)\n+\t\t\t const char *path, int stage, int options)\n {\n \tstruct cache_entry *ce;\n-\tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage,\n-\t\t\t (refresh ? (CE_MATCH_REFRESH |\n-\t\t\t\t\t CE_MATCH_IGNORE_MISSING) : 0 ));\n+\n+\tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage, 0);\n \tif (!ce)\n \t\treturn error(_(\"addinfo_cache failed for path '%s'\"), path);\n+\n \treturn add_cache_entry(ce, options);\n }\n \n@@ -552,13 +552,13 @@ static int update_stages(const char *path, const struct diff_filespec *o,\n \t\tif (remove_file_from_cache(path))\n \t\t\treturn -1;\n \tif (o)\n-\t\tif (add_cacheinfo(o->mode, o->sha1, path, 1, 0, options))\n+\t\tif (add_cacheinfo(o->mode, o->sha1, path, 1, options))\n \t\t\treturn -1;\n \tif (a)\n-\t\tif (add_cacheinfo(a->mode, a->sha1, path, 2, 0, options))\n+\t\tif (add_cacheinfo(a->mode, a->sha1, path, 2, options))\n \t\t\treturn -1;\n \tif (b)\n-\t\tif (add_cacheinfo(b->mode, b->sha1, path, 3, 0, options))\n+\t\tif (add_cacheinfo(b->mode, b->sha1, path, 3, options))\n \t\t\treturn -1;\n \treturn 0;\n }\n@@ -804,7 +804,7 @@ static void update_file_flags(struct merge_options *o,\n \t}\n update_index:\n \tif (update_cache)\n-\t\tadd_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);\n+\t\tadd_cacheinfo(mode, sha, path, 0, ADD_CACHE_OK_TO_ADD);\n }\n \n static void update_file(struct merge_options *o,\n@@ -1638,8 +1638,7 @@ static int merge_content(struct merge_options *o,\n \t\t */\n \t\tpath_renamed_outside_HEAD = !path2 || !strcmp(path, path2);\n \t\tif (!path_renamed_outside_HEAD) {\n-\t\t\tadd_cacheinfo(mfi.mode, mfi.sha, path,\n-\t\t\t\t 0, (!o->call_depth), 0);\n+\t\t\tadd_cacheinfo(mfi.mode, mfi.sha, path, 0, 0);\n \t\t\treturn mfi.clean;\n \t\t}\n \t} else\n@@ -2019,6 +2018,7 @@ int merge_recursive_generic(struct merge_options *o,\n \thold_locked_index(lock, 1);\n \tclean = merge_recursive(o, head_commit, next_commit, ca,\n \t\t\tresult);\n+\trefresh_cache(CE_MATCH_REFRESH | CE_MATCH_IGNORE_MISSING);\n \tif (active_cache_changed &&\n \t write_locked_index(&the_index, lock, COMMIT_LOCK))\n \t\treturn error(_(\"Unable to write index.\"));\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 33b77ee..5e8d5fa 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -91,16 +91,13 @@ test_expect_success 'Merge after setting text=auto' '\n \tcompare_files expected file\n '\n \n-test_expect_success 'Merge addition of text=auto' '\n+test_expect_success 'Merge addition of text=auto eol=LF' '\n+\tgit config core.eol lf &&\n \tcat <<-\\EOF >expected &&\n \tfirst line\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -109,17 +106,31 @@ test_expect_success 'Merge addition of text=auto' '\n \tcompare_files expected file\n '\n \n+test_expect_success 'Merge addition of text=auto eol=CRLF' '\n+\tgit config core.eol crlf &&\n+\tcat <<-\\EOF >expected &&\n+\tfirst line\n+\tsame line\n+\tEOF\n+\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n+\tgit config merge.renormalize true &&\n+\tgit rm -fr . &&\n+\trm -f .gitattributes &&\n+\tgit reset --hard b &&\n+\techo >&2 \"After git reset --hard b\" &&\n+\tgit ls-files -s --eol >&2 &&\n+\tgit merge a &&\n+\tcompare_files expected file\n+'\n+\n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n+\tgit config core.eol native &&\n \techo \"<<<<<<<\" >expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected &&\n-\t\techo ======= | append_cr >>expected\n-\telse\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected &&\n-\t\techo ======= >>expected\n-\tfi &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n+\techo ======= >>expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n@@ -135,15 +146,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \techo \"<<<<<<<\" >expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo ======= | append_cr >>expected &&\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected\n-\telse\n-\t\techo ======= >>expected &&\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected\n-\tfi &&\n+\techo ======= >>expected &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n \techo \">>>>>>>\" >>expected &&\n \tgit config merge.renormalize false &&\n \trm -f .gitattributes &&\n\n\n\n"},{"id":"291051","messageId":"xmqqoa68kl7n.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"f78fa94d-abd2-05a2-c411-15e2ffdb7dae@web.de","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-08T19:01:48Z","receivedAt":"2016-07-08T19:01:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n> Did that experiment made it to a branch somewhere ?\n\nI've tentatively queued the following (it's the same as \"Here is\nanother approach.\" I sent earlier, but with a real log message) on\n'pu'.\n\n-- >8 --\nSubject: [PATCH] merge: avoid \"safer crlf\" during recording of merge results\n\nWhen merge_recursive() decides what the correct blob object merge\nresult for a path should be, it uses update_file_flags() helper\nfunction to write it out to a working tree file and then calls\nadd_cacheinfo(). The add_cacheinfo() function in turn calls\nmake_cache_entry() to create a new cache entry to replace the\nhigher-stage entries for the path that represents the conflict.\n\nThe make_cache_entry() function calls refresh_cache_entry() to fill\nin the cached stat information. To mark a cache entry as\nup-to-date, the data is re-read from the file in the working tree,\nand goes through convert_to_git() conversion to be compared with the\nblob object name the new cache entry records.\n\nIt is important to note that this happens while the higher-stage\nentries, which are going to be replaced with the new entry, are\nstill in the index. Unfortunately, the convert_to_git() conversion\nhas a misguided \"safer crlf\" mechanism baked in, and looks at the\nexisting cache entry for the path to decide how to convert the\ncontents in the working tree file. If our side (i.e. stage#2)\nrecords a text blob with CRLF in it, even when the system is\nconfigured to record LF in blobs and convert them to CRLF upon\ncheckout (and back to LF upon checkin), the \"safer crlf\" mechanism\nstops us doing so.\n\nThis especially poses a problem during a renormalizing merge, where\nthe merge result for the path is computed by first \"normalizing\" the\nblobs involved in the merge by using convert_to_working_tree()\nfollowed by convert_to_git() with \"safer crlf\" disabled. The merge\nresult that is computed correctly and fed to add_cacheinfo() via\nupdate_file_flags() does _not_ match what refresh_cache_entry() sees\nby converting the working tree file via convert_to_git().\n\nWe can work this around by not refreshing the individual cache\nentries in make_cache_entry() called by add_cacheinfo(). After we\nare finished mergeing, we call refresh_cache() to take care of the\nrefreshing of the stat information. By the time this refreshing\nhappens, the stale cache entries that had CRLF in stage #2 that were\ncarried over before the renormalizing merge started will all be gone\nand will not interfere with the correct recording of the result.\n\nThe test update was taken from a series by Torsten Bögershausen\nthat attempted to fix this with a different approach (which was a\nlot more intrusive).\n\nSigned-off-by: Junio C Hamano \n---\n builtin/merge.c | 3 ++-\n merge-recursive.c | 20 +++++++++---------\n t/t6038-merge-text-auto.sh | 51 +++++++++++++++++++++++++---------------------\n 3 files changed, 40 insertions(+), 34 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 101ffef..d5bf68d 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -678,7 +678,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n \n \t\thold_locked_index(&lock, 1);\n \t\tclean = merge_recursive(&o, head,\n-\t\t\t\tremoteheads->item, reversed, &result);\n+\t\t\t\t\tremoteheads->item, reversed, &result);\n+\t\trefresh_cache(CE_MATCH_REFRESH | CE_MATCH_IGNORE_MISSING);\n \t\tif (active_cache_changed &&\n \t\t write_locked_index(&the_index, &lock, COMMIT_LOCK))\n \t\t\tdie (_(\"unable to write %s\"), get_index_file());\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex b880ae5..8aaf1b5 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -199,14 +199,14 @@ static void output_commit_title(struct merge_options *o, struct commit *commit)\n }\n \n static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n-\t\tconst char *path, int stage, int refresh, int options)\n+\t\t\t const char *path, int stage, int options)\n {\n \tstruct cache_entry *ce;\n-\tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage,\n-\t\t\t (refresh ? (CE_MATCH_REFRESH |\n-\t\t\t\t\t CE_MATCH_IGNORE_MISSING) : 0 ));\n+\n+\tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage, 0);\n \tif (!ce)\n \t\treturn error(_(\"addinfo_cache failed for path '%s'\"), path);\n+\n \treturn add_cache_entry(ce, options);\n }\n \n@@ -552,13 +552,13 @@ static int update_stages(const char *path, const struct diff_filespec *o,\n \t\tif (remove_file_from_cache(path))\n \t\t\treturn -1;\n \tif (o)\n-\t\tif (add_cacheinfo(o->mode, o->sha1, path, 1, 0, options))\n+\t\tif (add_cacheinfo(o->mode, o->sha1, path, 1, options))\n \t\t\treturn -1;\n \tif (a)\n-\t\tif (add_cacheinfo(a->mode, a->sha1, path, 2, 0, options))\n+\t\tif (add_cacheinfo(a->mode, a->sha1, path, 2, options))\n \t\t\treturn -1;\n \tif (b)\n-\t\tif (add_cacheinfo(b->mode, b->sha1, path, 3, 0, options))\n+\t\tif (add_cacheinfo(b->mode, b->sha1, path, 3, options))\n \t\t\treturn -1;\n \treturn 0;\n }\n@@ -804,7 +804,7 @@ static void update_file_flags(struct merge_options *o,\n \t}\n update_index:\n \tif (update_cache)\n-\t\tadd_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);\n+\t\tadd_cacheinfo(mode, sha, path, 0, ADD_CACHE_OK_TO_ADD);\n }\n \n static void update_file(struct merge_options *o,\n@@ -1638,8 +1638,7 @@ static int merge_content(struct merge_options *o,\n \t\t */\n \t\tpath_renamed_outside_HEAD = !path2 || !strcmp(path, path2);\n \t\tif (!path_renamed_outside_HEAD) {\n-\t\t\tadd_cacheinfo(mfi.mode, mfi.sha, path,\n-\t\t\t\t 0, (!o->call_depth), 0);\n+\t\t\tadd_cacheinfo(mfi.mode, mfi.sha, path, 0, 0);\n \t\t\treturn mfi.clean;\n \t\t}\n \t} else\n@@ -2019,6 +2018,7 @@ int merge_recursive_generic(struct merge_options *o,\n \thold_locked_index(lock, 1);\n \tclean = merge_recursive(o, head_commit, next_commit, ca,\n \t\t\tresult);\n+\trefresh_cache(CE_MATCH_REFRESH | CE_MATCH_IGNORE_MISSING);\n \tif (active_cache_changed &&\n \t write_locked_index(&the_index, lock, COMMIT_LOCK))\n \t\treturn error(_(\"Unable to write index.\"));\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 33b77ee..5e8d5fa 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -91,16 +91,13 @@ test_expect_success 'Merge after setting text=auto' '\n \tcompare_files expected file\n '\n \n-test_expect_success 'Merge addition of text=auto' '\n+test_expect_success 'Merge addition of text=auto eol=LF' '\n+\tgit config core.eol lf &&\n \tcat <<-\\EOF >expected &&\n \tfirst line\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -109,17 +106,31 @@ test_expect_success 'Merge addition of text=auto' '\n \tcompare_files expected file\n '\n \n+test_expect_success 'Merge addition of text=auto eol=CRLF' '\n+\tgit config core.eol crlf &&\n+\tcat <<-\\EOF >expected &&\n+\tfirst line\n+\tsame line\n+\tEOF\n+\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n+\tgit config merge.renormalize true &&\n+\tgit rm -fr . &&\n+\trm -f .gitattributes &&\n+\tgit reset --hard b &&\n+\techo >&2 \"After git reset --hard b\" &&\n+\tgit ls-files -s --eol >&2 &&\n+\tgit merge a &&\n+\tcompare_files expected file\n+'\n+\n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n+\tgit config core.eol native &&\n \techo \"<<<<<<<\" >expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected &&\n-\t\techo ======= | append_cr >>expected\n-\telse\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected &&\n-\t\techo ======= >>expected\n-\tfi &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n+\techo ======= >>expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n@@ -135,15 +146,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \techo \"<<<<<<<\" >expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo ======= | append_cr >>expected &&\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected\n-\telse\n-\t\techo ======= >>expected &&\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected\n-\tfi &&\n+\techo ======= >>expected &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n \techo \">>>>>>>\" >>expected &&\n \tgit config merge.renormalize false &&\n \trm -f .gitattributes &&\n-- \n2.9.0-512-gbea6568\n\n"},{"id":"291056","messageId":"xmqqk2gvlur5.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"xmqqoa68kl7n.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-08T20:50:22Z","receivedAt":"2016-07-08T20:50:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> I've tentatively queued the following (it's the same as \"Here is\n> another approach.\" I sent earlier, but with a real log message) on\n> 'pu'.\n\nAnd here is yet another approach, which probably is even less\nintrusive.\n\nWhatever alternative we would end up picking, in the longer term, I\nthink we should just get rid of the \"safer crlf\" hack from the\ncodebase, instead of adding the conditional disabling to work it\naround. The \"safe crlf\" that inspects the contents being added and\ndecides that it is not a good idea to turn CRLF in the contents\nbeing added into LF was sensible. The \"safer crlf\" that inspects\nwhat happens to be at the path in the index and decides that it\nshould not turn CRLF in the contents being added into LF, only\nbecause the blob in the index has CRLF in it, is simply broken. If\na project wants to make sure blobs in some paths are recorded with\nCRLF (perhaps because they are test data to compare with and the\nprogram is supposed to always produce CRLF data), they can and\nshould be use the gitattributes mechanism to mark those paths as\nsuch.\n\nAnd that is why I really didn't like the approach to pass extra\nsha1's around, churning codepaths that are at quite a low level,\nonly to feed it to the \"safer crlf\" mechanism. If removing \"safer\ncrlf\" is the right thing to do in the longer term (and I think it\nis), we shouldn't be making a big interface changes that we'd need\nto revert when it goes away.\n\n-- >8 --\nSubject: [PATCH] merge: avoid \"safer crlf\" during recording of merge results\n\nWhen merge_recursive() decides what the correct blob object merge\nresult for a path should be, it uses update_file_flags() helper\nfunction to write it out to a working tree file and then calls\nadd_cacheinfo(). The add_cacheinfo() function in turn calls\nmake_cache_entry() to create a new cache entry to replace the\nhigher-stage entries for the path that represents the conflict.\n\nThe make_cache_entry() function calls refresh_cache_entry() to fill\nin the cached stat information. To mark a cache entry as\nup-to-date, the data is re-read from the file in the working tree,\nand goes through convert_to_git() conversion to be compared with the\nblob object name the new cache entry records.\n\nIt is important to note that this happens while the higher-stage\nentries, which are going to be replaced with the new entry, are\nstill in the index. Unfortunately, the convert_to_git() conversion\nhas a misguided \"safer crlf\" mechanism baked in, and looks at the\nexisting cache entry for the path to decide how to convert the\ncontents in the working tree file. If our side (i.e. stage#2)\nrecords a text blob with CRLF in it, even when the system is\nconfigured to record LF in blobs and convert them to CRLF upon\ncheckout (and back to LF upon checkin), the \"safer crlf\" mechanism\nstops us doing so.\n\nThis especially poses a problem during a renormalizing merge, where\nthe merge result for the path is computed by first \"normalizing\" the\nblobs involved in the merge by using convert_to_working_tree()\nfollowed by convert_to_git() with \"safer crlf\" disabled. The merge\nresult that is computed correctly and fed to add_cacheinfo() via\nupdate_file_flags() does _not_ match what refresh_cache_entry() sees\nby converting the working tree file via convert_to_git().\n\nWe can work this around by not refreshing the new cache entry in\nmake_cache_entry() called by add_cacheinfo(). After add_cacheinfo()\nadds the new entry, we can call refresh_cache_entry() on that,\nknowing that addition of this new cache entry would have removed the\nstale cache entries that had CRLF in stage #2 that were carried over\nbefore the renormalizing merge started and will not interfere with\nthe correct recording of the result.\n\nThe test update was taken from a series by Torsten Bögershausen\nthat attempted to fix this with a different approach (which was a\nlot more intrusive).\n\nSigned-off-by: Junio C Hamano \n---\n cache.h | 1 +\n merge-recursive.c | 17 ++++++++++++----\n read-cache.c | 5 +----\n t/t6038-merge-text-auto.sh | 51 +++++++++++++++++++++++++---------------------\n 4 files changed, 43 insertions(+), 31 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex b829410..b33cb54 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -632,6 +632,7 @@ extern void fill_stat_cache_info(struct cache_entry *ce, struct stat *st);\n #define REFRESH_IGNORE_SUBMODULES\t0x0010\t/* ignore submodules */\n #define REFRESH_IN_PORCELAIN\t0x0020\t/* user friendly output, not \"needs update\" */\n extern int refresh_index(struct index_state *, unsigned int flags, const struct pathspec *pathspec, char *seen, const char *header_msg);\n+extern struct cache_entry *refresh_cache_entry(struct cache_entry *, unsigned int);\n \n extern void update_index_if_able(struct index_state *, struct lock_file *);\n \ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex b880ae5..de37e51 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -202,12 +202,21 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n \t\tconst char *path, int stage, int refresh, int options)\n {\n \tstruct cache_entry *ce;\n-\tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage,\n-\t\t\t (refresh ? (CE_MATCH_REFRESH |\n-\t\t\t\t\t CE_MATCH_IGNORE_MISSING) : 0 ));\n+\tint ret;\n+\n+\tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage, 0);\n \tif (!ce)\n \t\treturn error(_(\"addinfo_cache failed for path '%s'\"), path);\n-\treturn add_cache_entry(ce, options);\n+\n+\tret = add_cache_entry(ce, options);\n+\tif (refresh) {\n+\t\tstruct cache_entry *nce;\n+\n+\t\tnce = refresh_cache_entry(ce, CE_MATCH_REFRESH | CE_MATCH_IGNORE_MISSING);\n+\t\tif (nce != ce)\n+\t\t\tret = add_cache_entry(nce, options);\n+\t}\n+\treturn ret;\n }\n \n static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..6af409a 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -19,9 +19,6 @@\n #include \"split-index.h\"\n #include \"utf8.h\"\n \n-static struct cache_entry *refresh_cache_entry(struct cache_entry *ce,\n-\t\t\t\t\t unsigned int options);\n-\n /* Mask for the name length in ce_flags in the on-disk index */\n \n #define CE_NAMEMASK (0x0fff)\n@@ -1254,7 +1251,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \treturn has_errors;\n }\n \n-static struct cache_entry *refresh_cache_entry(struct cache_entry *ce,\n+struct cache_entry *refresh_cache_entry(struct cache_entry *ce,\n \t\t\t\t\t unsigned int options)\n {\n \treturn refresh_cache_ent(&the_index, ce, options, NULL, NULL);\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 33b77ee..5e8d5fa 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -91,16 +91,13 @@ test_expect_success 'Merge after setting text=auto' '\n \tcompare_files expected file\n '\n \n-test_expect_success 'Merge addition of text=auto' '\n+test_expect_success 'Merge addition of text=auto eol=LF' '\n+\tgit config core.eol lf &&\n \tcat <<-\\EOF >expected &&\n \tfirst line\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -109,17 +106,31 @@ test_expect_success 'Merge addition of text=auto' '\n \tcompare_files expected file\n '\n \n+test_expect_success 'Merge addition of text=auto eol=CRLF' '\n+\tgit config core.eol crlf &&\n+\tcat <<-\\EOF >expected &&\n+\tfirst line\n+\tsame line\n+\tEOF\n+\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n+\tgit config merge.renormalize true &&\n+\tgit rm -fr . &&\n+\trm -f .gitattributes &&\n+\tgit reset --hard b &&\n+\techo >&2 \"After git reset --hard b\" &&\n+\tgit ls-files -s --eol >&2 &&\n+\tgit merge a &&\n+\tcompare_files expected file\n+'\n+\n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n+\tgit config core.eol native &&\n \techo \"<<<<<<<\" >expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected &&\n-\t\techo ======= | append_cr >>expected\n-\telse\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected &&\n-\t\techo ======= >>expected\n-\tfi &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n+\techo ======= >>expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n@@ -135,15 +146,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \techo \"<<<<<<<\" >expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo ======= | append_cr >>expected &&\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected\n-\telse\n-\t\techo ======= >>expected &&\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected\n-\tfi &&\n+\techo ======= >>expected &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n \techo \">>>>>>>\" >>expected &&\n \tgit config merge.renormalize false &&\n \trm -f .gitattributes &&\n-- \n2.9.0-512-gbea6568\n\n"},{"id":"291197","messageId":"xmqqinwc7xbb.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"xmqqk2gvlur5.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-11T20:07:52Z","receivedAt":"2016-07-11T20:08:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Subject: [PATCH] merge: avoid \"safer crlf\" during recording of merge results\n> ...\n> We can work this around by not refreshing the new cache entry in\n> make_cache_entry() called by add_cacheinfo(). After add_cacheinfo()\n> adds the new entry, we can call refresh_cache_entry() on that,\n> knowing that addition of this new cache entry would have removed the\n> stale cache entries that had CRLF in stage #2 that were carried over\n> before the renormalizing merge started and will not interfere with\n> the correct recording of the result.\n>\n> The test update was taken from a series by Torsten Bögershausen\n> that attempted to fix this with a different approach (which was a\n> lot more intrusive).\n>\n> Signed-off-by: Junio C Hamano \n> ---\n\nHow do things look at this point? This version is what I ended up\nqueuing in 'pu', but I took your \"Thanks\" in $gmane/299120 to only\nmean \"Thanks for feeding some ideas to help me move forward\", not\nnecessarily \"Tnanks that looks like the right approach.\" yet, so\nright now both topics are stalled and waiting for an action from\nyou.\n\nThanks.\n\n> cache.h | 1 +\n> merge-recursive.c | 17 ++++++++++++----\n> read-cache.c | 5 +----\n> t/t6038-merge-text-auto.sh | 51 +++++++++++++++++++++++++---------------------\n> 4 files changed, 43 insertions(+), 31 deletions(-)\n\n[no comment below this line; the contents kept as reference]\n\n> diff --git a/cache.h b/cache.h\n> index b829410..b33cb54 100644\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -632,6 +632,7 @@ extern void fill_stat_cache_info(struct cache_entry *ce, struct stat *st);\n> #define REFRESH_IGNORE_SUBMODULES\t0x0010\t/* ignore submodules */\n> #define REFRESH_IN_PORCELAIN\t0x0020\t/* user friendly output, not \"needs update\" */\n> extern int refresh_index(struct index_state *, unsigned int flags, const struct pathspec *pathspec, char *seen, const char *header_msg);\n> +extern struct cache_entry *refresh_cache_entry(struct cache_entry *, unsigned int);\n> \n> extern void update_index_if_able(struct index_state *, struct lock_file *);\n> \n> diff --git a/merge-recursive.c b/merge-recursive.c\n> index b880ae5..de37e51 100644\n> --- a/merge-recursive.c\n> +++ b/merge-recursive.c\n> @@ -202,12 +202,21 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,\n> \t\tconst char *path, int stage, int refresh, int options)\n> {\n> \tstruct cache_entry *ce;\n> -\tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage,\n> -\t\t\t (refresh ? (CE_MATCH_REFRESH |\n> -\t\t\t\t\t CE_MATCH_IGNORE_MISSING) : 0 ));\n> +\tint ret;\n> +\n> +\tce = make_cache_entry(mode, sha1 ? sha1 : null_sha1, path, stage, 0);\n> \tif (!ce)\n> \t\treturn error(_(\"addinfo_cache failed for path '%s'\"), path);\n> -\treturn add_cache_entry(ce, options);\n> +\n> +\tret = add_cache_entry(ce, options);\n> +\tif (refresh) {\n> +\t\tstruct cache_entry *nce;\n> +\n> +\t\tnce = refresh_cache_entry(ce, CE_MATCH_REFRESH | CE_MATCH_IGNORE_MISSING);\n> +\t\tif (nce != ce)\n> +\t\t\tret = add_cache_entry(nce, options);\n> +\t}\n> +\treturn ret;\n> }\n> \n> static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)\n> diff --git a/read-cache.c b/read-cache.c\n> index d9fb78b..6af409a 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -19,9 +19,6 @@\n> #include \"split-index.h\"\n> #include \"utf8.h\"\n> \n> -static struct cache_entry *refresh_cache_entry(struct cache_entry *ce,\n> -\t\t\t\t\t unsigned int options);\n> -\n> /* Mask for the name length in ce_flags in the on-disk index */\n> \n> #define CE_NAMEMASK (0x0fff)\n> @@ -1254,7 +1251,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n> \treturn has_errors;\n> }\n> \n> -static struct cache_entry *refresh_cache_entry(struct cache_entry *ce,\n> +struct cache_entry *refresh_cache_entry(struct cache_entry *ce,\n> \t\t\t\t\t unsigned int options)\n> {\n> \treturn refresh_cache_ent(&the_index, ce, options, NULL, NULL);\n> diff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\n> index 33b77ee..5e8d5fa 100755\n> --- a/t/t6038-merge-text-auto.sh\n> +++ b/t/t6038-merge-text-auto.sh\n> @@ -91,16 +91,13 @@ test_expect_success 'Merge after setting text=auto' '\n> \tcompare_files expected file\n> '\n> \n> -test_expect_success 'Merge addition of text=auto' '\n> +test_expect_success 'Merge addition of text=auto eol=LF' '\n> +\tgit config core.eol lf &&\n> \tcat <<-\\EOF >expected &&\n> \tfirst line\n> \tsame line\n> \tEOF\n> \n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\tappend_cr expected.temp &&\n> -\t\tmv expected.temp expected\n> -\tfi &&\n> \tgit config merge.renormalize true &&\n> \tgit rm -fr . &&\n> \trm -f .gitattributes &&\n> @@ -109,17 +106,31 @@ test_expect_success 'Merge addition of text=auto' '\n> \tcompare_files expected file\n> '\n> \n> +test_expect_success 'Merge addition of text=auto eol=CRLF' '\n> +\tgit config core.eol crlf &&\n> +\tcat <<-\\EOF >expected &&\n> +\tfirst line\n> +\tsame line\n> +\tEOF\n> +\n> +\tappend_cr expected.temp &&\n> +\tmv expected.temp expected &&\n> +\tgit config merge.renormalize true &&\n> +\tgit rm -fr . &&\n> +\trm -f .gitattributes &&\n> +\tgit reset --hard b &&\n> +\techo >&2 \"After git reset --hard b\" &&\n> +\tgit ls-files -s --eol >&2 &&\n> +\tgit merge a &&\n> +\tcompare_files expected file\n> +'\n> +\n> test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n> +\tgit config core.eol native &&\n> \techo \"<<<<<<<\" >expected &&\n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\techo first line | append_cr >>expected &&\n> -\t\techo same line | append_cr >>expected &&\n> -\t\techo ======= | append_cr >>expected\n> -\telse\n> -\t\techo first line >>expected &&\n> -\t\techo same line >>expected &&\n> -\t\techo ======= >>expected\n> -\tfi &&\n> +\techo first line >>expected &&\n> +\techo same line >>expected &&\n> +\techo ======= >>expected &&\n> \techo first line | append_cr >>expected &&\n> \techo same line | append_cr >>expected &&\n> \techo \">>>>>>>\" >>expected &&\n> @@ -135,15 +146,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n> \techo \"<<<<<<<\" >expected &&\n> \techo first line | append_cr >>expected &&\n> \techo same line | append_cr >>expected &&\n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\techo ======= | append_cr >>expected &&\n> -\t\techo first line | append_cr >>expected &&\n> -\t\techo same line | append_cr >>expected\n> -\telse\n> -\t\techo ======= >>expected &&\n> -\t\techo first line >>expected &&\n> -\t\techo same line >>expected\n> -\tfi &&\n> +\techo ======= >>expected &&\n> +\techo first line >>expected &&\n> +\techo same line >>expected &&\n> \techo \">>>>>>>\" >>expected &&\n> \tgit config merge.renormalize false &&\n> \trm -f .gitattributes &&\n"},{"id":"291235","messageId":"51079140-98c8-ee0c-72d2-af5c348c19fd@web.de","threadId":"42727","inReplyTo":"xmqqinwc7xbb.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-07-12T02:23:47Z","receivedAt":"2016-07-12T02:24:34Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":">\n> How do things look at this point? This version is what I ended up\n> queuing in 'pu', but I took your \"Thanks\" in $gmane/299120 to only\n> mean \"Thanks for feeding some ideas to help me move forward\", not\n> necessarily \"Tnanks that looks like the right approach.\" yet, so\n> right now both topics are stalled and waiting for an action from\n> you.\nYes, the code looks good to me.\nAnd the commit message does explain what is going on.\n\nFor my taste, these 3 lines don't explain too much,may be remove them ?\n > The test update was taken from a series by Torsten Bögershausen\n > that attempted to fix this with a different approach (which was a\n > lot more intrusive).\nSo thanks for your efforts, ack from my side.\n"},{"id":"291301","messageId":"xmqqa8hm4ooh.fsf@gitster.mtv.corp.google.com","threadId":"42727","inReplyTo":"51079140-98c8-ee0c-72d2-af5c348c19fd@web.de","subject":"Re: [PATCH v3 3/3] correct ce_compare_data() in a middle of a merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-12T19:54:54Z","receivedAt":"2016-07-12T19:55:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n>> How do things look at this point? This version is what I ended up\n>> queuing in 'pu', but I took your \"Thanks\" in $gmane/299120 to only\n>> mean \"Thanks for feeding some ideas to help me move forward\", not\n>> necessarily \"Tnanks that looks like the right approach.\" yet, so\n>> right now both topics are stalled and waiting for an action from\n>> you.\n> Yes, the code looks good to me.\n> And the commit message does explain what is going on.\n>\n> For my taste, these 3 lines don't explain too much,may be remove them ?\n>> The test update was taken from a series by Torsten Bögershausen\n>> that attempted to fix this with a different approach (which was a\n>> lot more intrusive).\n\nOK. I wanted to make sure the resulting log message not only gives\nyou credit for the test portion of the change, but also for the fact\nthat you thought long and hard about the issue. I'll tone it down\nby removing \"(which was...)\" part.\n\n> So thanks for your efforts, ack from my side.\n\nThanks.\n"}]}