{"thread":{"id":"45840","subject":"What's cooking in git.git (May 2017, #01; Mon, 1)","startedAt":"2017-05-01T05:43:41Z","lastAt":"2017-05-09T14:46:54Z","messageCount":36,"participants":["Junio C Hamano","Ævar Arnfjörð Bjarmason","Brandon Williams","Jeff King","Johannes Schindelin","Kevin Daudt","Duy Nguyen","Ramsay Jones","brian m. carlson","demerphq"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"318352","messageId":"xmqqefw9gmvq.fsf@gitster.mtv.corp.google.com","threadId":"45840","inReplyTo":null,"subject":"What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-01T05:35:37Z","receivedAt":"2017-05-01T05:43:41Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Here are the topics that have been cooking. Commits prefixed with\n'-' are only in 'pu' (proposed updates) while commits prefixed with\n'+' are in 'next'. The ones marked with '.' do not appear in any of\nthe integration branches, but I am still holding onto them.\n\n2.13-rc1 was tagged and we are now in a pre-release freeze. The\ntopics marked to be merged to 'next' in the list below may be merged\nto 'next' as usual, but the ones marked for 'master' may stay in\n'next' until the final release, unless they are small bug/doc fixes\nor hotfixes to topics that are already in -rc1 from now on.\n\nYou can find the changes described here in the integration branches\nof the repositories listed at\n\n http://git-blame.blogspot.com/p/git-public-repositories.html\n\n--------------------------------------------------\n[Graduated to \"master\"]\n\n* ab/align-perf-descriptions (2017-04-23) 1 commit\n (merged to 'next' on 2017-04-26 at 7d66e7019e)\n + t/perf: correctly align non-ASCII descriptions in output\n\n Output from perf tests have been updated to align their titles.\n\n\n* jk/complete-checkout-sans-dwim-remote (2017-04-23) 1 commit\n (merged to 'next' on 2017-04-26 at 0c69cff69f)\n + completion: optionally disable checkout DWIM\n\n Completion for \"git checkout \" that auto-creates the branch\n out of a remote tracking branch can now be disabled, as this\n completion often gets in the way when completing to checkout an\n existing local branch that happens to share the same prefix with\n bunch of remote tracking branches.\n\n\n* jk/prio-queue-avoid-swap-with-self (2017-04-24) 1 commit\n (merged to 'next' on 2017-04-26 at 6a55996734)\n + prio_queue_reverse: don't swap elements with themselves\n\n Code clean-up.\n\n\n* jk/submodule-init-segv-fix (2017-04-24) 1 commit\n (merged to 'next' on 2017-04-26 at 9a6eaaff89)\n + submodule_init: die cleanly on submodules without url defined\n\n Fix a segv in 'submodule init' when url is not given for a submodule.\n\n\n* ls/travis-stricter-linux32-builds (2017-04-27) 1 commit\n (merged to 'next' on 2017-04-28 at b0a56aec83)\n + travis-ci: set DEVELOPER knob for Linux32 build\n\n 32-bit Linux build on Travis CI uses stricter compilation options.\n\n\n* ls/travis-win-fix-status (2017-04-26) 1 commit\n (merged to 'next' on 2017-04-28 at 5c5e4d3278)\n + travis-ci: printf $STATUS as string\n\n Relaying status from Windows build by Travis CI was done with an\n unsafe invocation of printf.\n\n\n* sh/rebase-i-reread-todo-after-exec (2017-04-27) 1 commit\n (merged to 'next' on 2017-04-28 at 1ea7e62026)\n + rebase -i: reread the todo list if `exec` touched it\n\n \"git rebase -i\" failed to re-read the todo list file when the\n command specified with the `exec` instruction updated it.\n\n--------------------------------------------------\n[New Topics]\n\n* bw/submodule-with-bs-path (2017-05-01) 1 commit\n - t7400: add !CYGWIN prerequisite to 'add with \\\\ in path'\n\n A hotfix to a topic that is already in 'master'.\n\n Wait for a few days for Acks and then merge to 'master'.\n\n\n* jk/am-leakfix (2017-04-27) 3 commits\n (merged to 'next' on 2017-04-30 at 78becd7a96)\n + am: shorten ident_split variable name in get_commit_info()\n + am: simplify allocations in get_commit_info()\n + am: fix commit buffer leak in get_commit_info()\n\n The codepath in \"git am\" that is used when running \"git rebase\"\n leaked memory held for the log message of the commits being rebased.\n\n Will cook in 'next'.\n\n\n* js/plug-leaks (2017-04-27) 13 commits\n - checkout: fix memory leak\n - cat-file: fix memory leak\n - mailinfo & mailsplit: check for EOF while parsing mails\n - status: close file descriptor after reading git-rebase-todo\n - difftool: close file descriptors after reading\n - http-backend: avoid memory leaks\n - SQUASH???\n - get_mail_commit_oid(): avoid resource leak\n - git_config_rename_section_in_file(): avoid resource leak\n - add_commit_patch_id(): avoid allocating memory unnecessarily\n - winansi: avoid buffer overrun\n - winansi: avoid use of uninitialized value\n - mingw: avoid memory leak when splitting PATH\n\n An attempt to fix memory leaks pointed out by Coverity.\n\n A reroll exists and needs to be picked up. What's queued here is\n just an early parts of the first iteration.\n\n\n* sk/status-short-branch-color-config (2017-04-28) 2 commits\n (merged to 'next' on 2017-04-30 at 2a7eb05d7b)\n + status: add color config slots for branch info in \"--short --branch\"\n + status: fix missing newline when comment chars are disabled\n\n The colors in which \"git status --short --branch\" showed the names\n of the current branch and its remote-tracking branch are now\n configurable.\n\n Will cook in 'next'.\n\n\n* jh/close-index-before-stat (2017-04-28) 1 commit\n (merged to 'next' on 2017-04-30 at 918d4f3378)\n + read-cache: close index.lock in do_write_index\n\n The timestamp of the index file is now taken after the file is\n closed, to help Windows, on which a stale timestamp is reported by\n fstat() on a file that is opened for writing and data was written\n but not yet closed.\n\n Will cook in 'next'.\n\n\n* bw/submodule-has-commits-update (2017-05-01) 6 commits\n - submodule: refactor logic to determine changed submodules\n - submodule: improve submodule_has_commits\n - submodule: change string_list changed_submodule_paths\n - submodule: remove add_oid_to_argv()\n - submodule: rename free_submodules_sha1s()\n - submodule: rename add_sha1_to_array()\n\n\n* ja/i18n-cleanup (2017-05-01) 2 commits\n (merged to 'next' on 2017-04-30 at 8002e53820)\n + i18n: read-cache: typofix\n + i18n: remove i18n from tag reflog message\n\n Will merge to 'master'.\n\n\n* ls/travis-relays-for-windows-ci (2017-05-01) 2 commits\n - travis-ci: handle Git for Windows CI status \"failed\" explicitly\n - travis-ci: retry if Git for Windows CI returns HTTP error 502 or 503\n\n Will merge to 'next' after a few CI run in 'pu'.\n\n\n* mb/diff-default-to-indent-heuristics (2017-05-01) 5 commits\n - add--interactive: drop diff.indentHeuristic handling\n - SQUASH\n - diff: enable indent heuristic by default\n - diff: have the diff-* builtins configure diff before initializing revisions\n - diff: make the indent heuristic part of diff's basic configuration\n\n Make the \"indent\" heuristics the default in \"diff\" and diff.indentHeuristics\n configuration variable an escape hatch for those who do no want it.\n\n The fix-up needs to be squashed in.\n\n\n* rg/doc-pull-typofix (2017-05-01) 1 commit\n (merged to 'next' on 2017-04-30 at c2edb4d813)\n + doc: git-pull.txt use US spelling, fix minor typo\n\n Will merge to 'master'.\n\n\n* rg/doc-submittingpatches-wordfix (2017-05-01) 1 commit\n (merged to 'next' on 2017-04-30 at 4065036588)\n + doc: update SubmittingPatches\n\n Will merge to 'master'.\n\n\n* sr/hooks-cwd-doc (2017-05-01) 1 commit\n (merged to 'next' on 2017-04-30 at 063dd5cb02)\n + githooks.txt: clarify push hooks are always executed in $GIT_DIR\n\n Will merge to 'master'.\n\n\n* tb/dedup-crlf-tests (2017-05-01) 1 commit\n - t0027: Some tests are not expensive\n\n Will be rerolled.\n\n--------------------------------------------------\n[Stalled]\n\n* ja/doc-l10n (2017-03-20) 3 commits\n - SQUASH???\n - l10n: add git-add.txt to localized man pages\n - l10n: introduce framework for localizing man pages\n\n A proposal to use po4a to localize our manual pages.\n\n\n* sg/clone-refspec-from-command-line-config (2017-02-27) 1 commit\n - clone: respect configured fetch respecs during initial fetch\n\n Expecting a reroll.\n cf. <20170227211217.73gydlxb2qu2sp3m@sigill.intra.peff.net>\n cf. \n Some nits but looks ok.\n\n\n* sk/dash-is-previous (2017-03-01) 5 commits\n - revert.c: delegate handling of \"-\" shorthand to setup_revisions\n - sha1_name.c: teach get_sha1_1 \"-\" shorthand for \"@{-1}\"\n - revision.c: args starting with \"-\" might be a revision\n - revision.c: swap if/else blocks\n - revision.c: do not update argv with unknown option\n\n A dash \"-\" can be written to mean \"the branch that was previously\n checked out\" in more places.\n\n Needs review.\n cf. <1488007487-12965-1-git-send-email-kannan.siddharth12@gmail.com>\n\n\n* ls/filter-process-delayed (2017-03-06) 1 commit\n - convert: add \"status=delayed\" to filter process protocol\n\n What's the status of this one???\n cf. \n\n\n* pb/bisect (2017-02-18) 28 commits\n . fixup! bisect--helper: `bisect_next_check` & bisect_voc shell function in C\n . bisect--helper: remove the dequote in bisect_start()\n . bisect--helper: retire `--bisect-auto-next` subcommand\n . bisect--helper: retire `--bisect-autostart` subcommand\n . bisect--helper: retire `--bisect-write` subcommand\n . bisect--helper: `bisect_replay` shell function in C\n . bisect--helper: `bisect_log` shell function in C\n . bisect--helper: retire `--write-terms` subcommand\n . bisect--helper: retire `--check-expected-revs` subcommand\n . bisect--helper: `bisect_state` & `bisect_head` shell function in C\n . bisect--helper: `bisect_autostart` shell function in C\n . bisect--helper: retire `--next-all` subcommand\n . bisect--helper: retire `--bisect-clean-state` subcommand\n . bisect--helper: `bisect_next` and `bisect_auto_next` shell function in C\n . t6030: no cleanup with bad merge base\n . bisect--helper: `bisect_start` shell function partially in C\n . bisect--helper: `get_terms` & `bisect_terms` shell function in C\n . bisect--helper: `bisect_next_check` & bisect_voc shell function in C\n . bisect--helper: `check_and_set_terms` shell function in C\n . bisect--helper: `bisect_write` shell function in C\n . bisect--helper: `is_expected_rev` & `check_expected_revs` shell function in C\n . bisect--helper: `bisect_reset` shell function in C\n . wrapper: move is_empty_file() and rename it as is_empty_or_missing_file()\n . t6030: explicitly test for bisection cleanup\n . bisect--helper: `bisect_clean_state` shell function in C\n . bisect--helper: `write_terms` shell function in C\n . bisect: rewrite `check_term_format` shell function in C\n . bisect--helper: use OPT_CMDMODE instead of OPT_BOOL\n\n Move more parts of \"git bisect\" to C.\n\n Ejected from 'pu', as it overlaps and gets in the way of bc/object-id topic.\n Expecting a reroll.\n cf. \n cf. \n cf. \n\n\n* sh/grep-tree-obj-tweak-output (2017-01-20) 2 commits\n - grep: use '/' delimiter for paths\n - grep: only add delimiter if there isn't one already\n\n \"git grep\", when fed a tree-ish as an input, shows each hit\n prefixed with \":::\". As is\n almost always either a commit or a tag that points at a commit, the\n early part of the output \":\" can be used as the\n name of the blob and given to \"git show\". When is a\n tree given in the extended SHA-1 syntax (e.g. \":\", or\n \":\"), however, this results in a string that does not\n name a blob (e.g. \"::\" or \"::\").\n \"git grep\" has been taught to be a bit more intelligent about these\n cases and omit a colon (in the former case) or use slash (in the\n latter case) to produce \":\" and\n \":/\" that can be used as the name of a blob.\n\n Expecting a reroll? Is this good enough with known limitations?\n\n\n* jc/diff-b-m (2015-02-23) 5 commits\n . WIPWIP\n . WIP: diff-b-m\n - diffcore-rename: allow easier debugging\n - diffcore-rename.c: add locate_rename_src()\n - diffcore-break: allow debugging\n\n \"git diff -B -M\" produced incorrect patch when the postimage of a\n completely rewritten file is similar to the preimage of a removed\n file; such a resulting file must not be expressed as a rename from\n other place.\n\n The fix in this patch is broken, unfortunately.\n\n Will discard.\n\n--------------------------------------------------\n[Cooking]\n\n* jc/repack-threads (2017-04-27) 1 commit\n - repack: accept --threads= and pass it down to pack-objects\n\n \"git repack\" learned to accept the --threads= option and pass it\n to pack-objects.\n\n\n* jh/verify-index-checksum-only-in-fsck (2017-04-27) 1 commit\n (merged to 'next' on 2017-04-28 at afb4c70061)\n + t1450: avoid use of \"sed\" on the index, which is a binary file\n\n Update an unportable constructin a new test.\n\n Will cook in 'next'.\n\n\n* dt/gc-ignore-old-gc-logs (2017-04-24) 3 commits\n (merged to 'next' on 2017-04-26 at 4f4cab8368)\n + test-lib: retire $remove_trash variable\n + test-lib.sh: do not barf under --debug at the end of the test\n + test-lib: abort when can't remove trash directory\n\n An attempt to allow us notice \"fishy\" situation where we fail to\n remove the temporary directory used during the test.\n\n Will cook in 'next'.\n\n\n* nd/fopen-errors (2017-04-23) 17 commits\n - warn_failure_to_open_read_optional_path(): a new helper\n - t1308: add a test case on open a config directory\n - config.c: handle error on failing to fopen()\n - xdiff-interface.c: report errno on failure to stat() or fopen()\n - wt-status.c: report error on failure to fopen()\n - server-info: report error on failure to fopen()\n - sequencer.c: report error on failure to fopen()\n - rerere.c: report correct errno\n - rerere.c: report error on failure to fopen()\n - remote.c: report error on failure to fopen()\n - commit.c: report error on failure to fopen() the graft file\n - log: report errno on failure to fopen() a file\n - clone: use xfopen() instead of fopen()\n - blame: report error on open if graft_file is a directory\n - bisect: report on fopen() error\n - config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD\n - git_fopen: fix a sparse 'not declared' warning\n\n We often try to open a file for reading whose existence is\n optional, and silently ignore errors from open/fopen; report such\n errors if they are not due to missing files.\n\n Expecting a reroll that would be much simplified thanks to a higher\n level helper.\n\n cf. \n cf. \n\n\n* dt/raise-core-packed-git-limit (2017-04-20) 1 commit\n (merged to 'next' on 2017-04-26 at c72dd8c62f)\n + Increase core.packedGitLimit\n\n Raise the default packed-git limit value on larger platforms to\n avoid \"git fetch\" from a (recoverable) failure while \"gc\" is\n running in parallel.\n\n Will cook in 'next'.\n\n\n* js/larger-timestamps (2017-04-27) 8 commits\n (merged to 'next' on 2017-04-28 at b56a0d38cd)\n + use uintmax_t for timestamps\n + date.c: abort if the system time cannot handle one of our timestamps\n + timestamp_t: a new data type for timestamps\n + PRItime: introduce a new \"printf format\" for timestamps\n + parse_timestamp(): specify explicitly where we parse timestamps\n + t0006 & t5000: skip \"far in the future\" test when time_t is too limited\n + t0006 & t5000: prepare for 64-bit timestamps\n + ref-filter: avoid using `unsigned long` for catch-all data type\n\n Some platforms have ulong that is smaller than time_t, and our\n historical use of ulong for timestamp would mean they cannot\n represent some timestamp that the platform allows. Invent a\n separate and dedicated timestamp_t (so that we can distingiuish\n timestamps and a vanilla ulongs, which along is already a good\n move), and then declare uintmax_t is the type to be used as the\n timestamp_t\n\n Will cook in 'next'.\n\n\n* ab/grep-pcre-v2 (2017-04-25) 20 commits\n - SQUASH???\n - Makefile & configure: make PCRE v2 the default PCRE implementation\n - grep: remove support for concurrent use of both PCRE v1 & v2\n - grep: add support for PCRE v2\n - grep: add support for the PCRE v1 JIT API\n - perf: add a performance comparison test of grep -E and -P\n - grep: change the internal PCRE code & header names to be PCRE1\n - grep: change the internal PCRE macro names to be PCRE1\n - test-lib: rename the LIBPCRE prerequisite to PCRE\n - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n - grep & rev-list doc: stop promising libpcre for --perl-regexp\n - log: add -P as a synonym for --perl-regexp\n - log: add exhaustive tests for pattern style options & config\n - grep: add a test for backreferences in PCRE patterns\n - Makefile & configure: reword outdated comment about PCRE\n - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n - grep: remove redundant regflags assignment under PCRE\n - grep: submodule-related case statements should die if new fields are added\n - grep: add tests for grep pattern types being passed to submodules\n - grep: amend submodule recursion test in preparation for rx engine testing\n\n PCRE2, which has an API different from and incompatible with PCRE,\n can now be chosen to support \"grep -P -e ''\" and friends.\n\n\n* bc/object-id (2017-04-23) 53 commits\n - object: convert parse_object* to take struct object_id\n - tree: convert parse_tree_indirect to struct object_id\n - sequencer: convert do_recursive_merge to struct object_id\n - diff-lib: convert do_diff_cache to struct object_id\n - builtin/ls-tree: convert to struct object_id\n - merge: convert checkout_fast_forward to struct object_id\n - sequencer: convert fast_forward_to to struct object_id\n - builtin/ls-files: convert overlay_tree_on_cache to object_id\n - builtin/read-tree: convert to struct object_id\n - sha1_name: convert internals of peel_onion to object_id\n - upload-pack: convert remaining parse_object callers to object_id\n - revision: convert remaining parse_object callers to object_id\n - revision: rename add_pending_sha1 to add_pending_oid\n - http-push: convert process_ls_object and descendants to object_id\n - refs/files-backend: convert many internals to struct object_id\n - refs: convert struct ref_update to use struct object_id\n - ref-filter: convert some static functions to struct object_id\n - Convert struct ref_array_item to struct object_id\n - Convert the verify_pack callback to struct object_id\n - Convert lookup_tag to struct object_id\n - log-tree: convert to struct object_id\n - Convert lookup_tree to struct object_id\n - builtin/reflog: convert tree_is_complete to take struct object_id\n - tree: convert read_tree_1 to use struct object_id internally\n - Convert lookup_blob to struct object_id\n - Convert remaining callers of lookup_blob to object_id\n - builtin/unpack-objects: convert to struct object_id\n - pack: convert struct pack_idx_entry to struct object_id\n - Convert lookup_commit* to struct object_id\n - Convert remaining callers of lookup_commit_reference* to object_id\n - builtin/tag: convert to struct object_id\n - sequencer: convert some functions to struct object_id\n - shallow: convert shallow registration functions to object_id\n - revision: convert prepare_show_merge to struct object_id\n - notes-utils: convert internals to struct object_id\n - http-push: convert some static functions to struct object_id\n - tag: convert parse_tag_buffer to struct object_id\n - builtin/verify-commit: convert to struct object_id\n - reflog_expire: convert to struct object_id\n - parse-options-cb: convert to struct object_id\n - notes-cache: convert to struct object_id\n - submodule: convert merge_submodule to use struct object_id\n - fast-import: convert to struct object_id\n - fast-import: convert internal structs to struct object_id\n - builtin/rev-parse: convert to struct object_id\n - builtin/blame: convert static function to struct object_id\n - branch: convert to struct object_id\n - bundle: convert to struct object_id\n - builtin/prune: convert to struct object_id\n - builtin/name-rev: convert to struct object_id\n - Convert struct cache_tree to use struct object_id\n - Clean up outstanding object_id transforms.\n - fetch-pack: convert to struct object_id\n\n Conversion from uchar[20] to struct object_id continues.\n\n Reroll exists and needs to be picked up.\n\n\n* jk/no-null-sha1-in-cache-tree (2017-04-23) 1 commit\n (merged to 'next' on 2017-04-26 at 45fbe9d57d)\n + cache-tree: reject entries with null sha1\n\n Tighten code to update cache-tree so that we won't accidentally\n write out any 0{40} entry in the tree object.\n\n Will cook in 'next'.\n\n\n* rg/a-the-typo (2017-05-01) 1 commit\n (merged to 'next' on 2017-04-30 at a648cf2961)\n + fix minor typos\n\n Typofix.\n\n Will merge to 'master'.\n\n\n* rs/large-zip (2017-05-01) 7 commits\n - t5004: require 64-bit support for big ZIP tests\n - archive-zip: set version field for big files correctly\n (merged to 'next' on 2017-04-26 at a6beab60f2)\n + archive-zip: support files bigger than 4GB\n + archive-zip: support archives bigger than 4GB\n + archive-zip: write ZIP dir entry directly to strbuf\n + archive-zip: use strbuf for ZIP directory\n + archive-zip: add tests for big ZIP archives\n\n \"git archive --format=zip\" learned to use zip64 extension when\n necessary to go beyond the 4GB limit.\n\n Will cook in 'next'.\n\n\n* jc/checkout-working-tree-only (2017-04-27) 1 commit\n - checkout: add --working-tree-only option\n\n \"git checkout \" learned a variant that does\n not update the index when doing its thing.\n\n\n* js/rebase-i-final (2017-04-27) 9 commits\n - rebase -i: rearrange fixup/squash lines using the rebase--helper\n - t3415: test fixup with wrapped oneline\n - rebase -i: skip unnecessary picks using the rebase--helper\n - rebase -i: check for missing commits in the rebase--helper\n - t3404: relax rebase.missingCommitsCheck tests\n - rebase -i: also expand/collapse the SHA-1s via the rebase--helper\n - rebase -i: do not invent onelines when expanding/collapsing SHA-1s\n - rebase -i: remove useless indentation\n - rebase -i: generate the script via rebase--helper\n\n The final batch to \"git rebase -i\" updates to move more code from\n the shell script to C.\n\n Needs review.\n\n\n* jt/use-trailer-api-in-commands (2017-04-26) 1 commit\n (merged to 'next' on 2017-04-30 at 006c8d7ebd)\n + sequencer: add newline before adding footers\n\n \"git cherry-pick\" and other uses of the sequencer machinery\n mishandled a trailer block whose last line is an incomplete line.\n This has been fixed so that an additional sign-off etc. are added\n after completing the existing incomplete line.\n\n Will cook in 'next'.\n\n\n* nd/worktree-move (2017-04-20) 6 commits\n - worktree remove: new command\n - worktree move: refuse to move worktrees with submodules\n - worktree move: accept destination as directory\n - worktree move: new command\n - worktree.c: add update_worktree_location()\n - worktree.c: add validate_worktree()\n\n \"git worktree\" learned move and remove subcommands.\n\n Expecting a reroll.\n cf. <20170420101024.7593-1-pclouds@gmail.com>\n cf. <20170421145916.mknekgqzhxffu7di@sigill.intra.peff.net>\n cf. \n\n\n* jk/parse-options-no-no-no (2017-04-25) 2 commits\n - SQUASH???\n - parse-options: disallow double-negations of options starting with no-\n\n Command line options that begin with \"--no-\" (e.g. \"--no-checkout\"\n option of \"git clone\") can be negated by removing \"--no-\"; we\n historically also allowed prefixing an extra \"no\" to the option\n (e.g. \"--no-no-checkout\"), which made the command line look ugly\n and unusual. This proposes to forbid it.\n\n While I agree there is no need to support \"--no-no-checkout\", this\n looks more like \"if it looks ugly and unusual, you do not have to\n use it\". Perhaps we can drop it?\n\n Will discard.\n\n\n* df/dir-iter-remove-subtree (2017-04-19) 5 commits\n - remove_subtree(): reimplement using iterators\n - dir_iterator: rewrite state machine model\n - dir_iterator: refactor dir_iterator_advance\n - remove_subtree(): test removing nested directories\n - dir_iterator: add tests for dir_iterator API\n\n Update the dir-iterator API and use it to reimplement\n remove_subtree().\n\n A reroll exists that is based on the updated 'master', but I ran\n out of time trying to get it to work with other topics in flight\n in 'pu'.\n GSoC microproject.\n\n\n* ab/clone-no-tags (2017-05-01) 3 commits\n (merged to 'next' on 2017-04-30 at 601649896a)\n + tests: rename a test having to do with shallow submodules\n + clone: add a --no-tags option to clone without tags\n + tests: change \"cd ... && git fetch\" to \"cd &&\\n\\tgit fetch\"\n\n \"git clone\" learned the \"--no-tags\" option not to fetch all tags\n initially, and also set up the tagopt not to follow any tags in\n subsequent fetches.\n\n Will cook in 'next'.\n\n\n* bw/forking-and-threading (2017-04-25) 13 commits\n (merged to 'next' on 2017-04-26 at 89c495e37f)\n + run-command: restrict PATH search to executable files\n + run-command: expose is_executable function\n (merged to 'next' on 2017-04-23 at 7754b5ebc3)\n + run-command: block signals between fork and execve\n + run-command: add note about forking and threading\n + run-command: handle dup2 and close errors in child\n + run-command: eliminate calls to error handling functions in child\n + run-command: don't die in child when duping /dev/null\n + run-command: prepare child environment before forking\n + string-list: add string_list_remove function\n + run-command: use the async-signal-safe execv instead of execvp\n + run-command: prepare command before forking\n + t0061: run_command executes scripts without a #! line\n + t5550: use write_script to generate post-update hook\n\n The \"run-command\" API implementation has been made more robust\n against dead-locking in a threaded environment.\n\n Will cook in 'next'.\n\n\n* sb/reset-recurse-submodules (2017-04-23) 4 commits\n - builtin/reset: add --recurse-submodules switch\n - submodule.c: submodule_move_head works with broken submodules\n - submodule.c: uninitialized submodules are ignored in recursive commands\n - entry.c: submodule recursing: respect force flag correctly\n\n \"git reset\" learned \"--recurse-submodules\" option.\n\n\n* ls/travis-doc-asciidoctor (2017-04-26) 4 commits\n - travis-ci: check AsciiDoc/AsciiDoctor stderr output\n (merged to 'next' on 2017-04-19 at 359c32953b)\n + travis-ci: unset compiler for jobs that do not need one\n + travis-ci: parallelize documentation build\n + travis-ci: build documentation with AsciiDoc and Asciidoctor\n\n Have Travis CI format the documentation with both AsciiDoc and\n AsciiDoctor.\n\n Will merge to 'next' after a few CI run in 'pu'.\n\n\n* mg/status-in-progress-info (2017-04-14) 1 commit\n - status: show in-progress info for short status\n\n \"git status\" learns an option to report various operations\n (e.g. \"merging\") that the user is in the middle of.\n\n It is still unclear if the participants decided that it is OK to\n spell \"--inprogress\" as a single word.\n\n\n* ab/grep-threading-cleanup (2017-04-16) 8 commits\n - grep: given --threads with NO_PTHREADS=YesPlease, warn\n - pack-objects: fix buggy warning about threads under NO_PTHREADS=YesPlease\n - pack-object & index-pack: add test for --threads warning under NO_PTHREADS\n - tests: add a PTHREADS prerequisite\n - grep: skip pthreads overhead when using one thread\n - grep: don't redundantly compile throwaway patterns under threading\n - grep: add tests for --threads=N and grep.threads\n - grep: assert that threading is enabled when calling grep_{lock,unlock}\n\n Code cleanup.\n\n Needs review.\n\n\n* bp/sub-process-convert-filter (2017-04-23) 8 commits\n - convert: update subprocess_read_status() to not die on EOF\n - sub-process: move sub-process functions into separate files\n - convert: rename reusable sub-process functions\n - convert: update generic functions to only use generic data structures\n - convert: separate generic structures and variables from the filter specific ones\n - convert: split start_multi_file_filter into two separate functions\n - convert: move packet_write_list() into pkt-line as packet_writel()\n - pkt-line: add packet_read_line_gently()\n\n Code from \"conversion using external process\" codepath has been\n extracted to a separate sub-process.[ch] module.\n\n Reroll exists but needs a bit more work.\n cf. <20170407120354.17736-1-benpeart@microsoft.com>\n\n\n* mg/name-rev-debug (2017-03-31) 4 commits\n - describe: pass --debug down to name-rev\n - name-rev: provide debug output\n - name-rev: favor describing with tags and use committer date to tiebreak\n - name-rev: refactor logic to see if a new candidate is a better name\n\n \"git describe --debug --contains\" did not add any meaningful\n information, even though without \"--contains\" it did.\n\n Expecting a reroll of the tip two.\n cf. \n\n\n* nd/prune-in-worktree (2017-04-24) 12 commits\n - rev-list: expose and document --single-worktree\n - revision.c: --reflog add HEAD reflog from all worktrees\n - files-backend: make reflog iterator go through per-worktree reflog\n - revision.c: --all adds HEAD from all worktrees\n - refs: remove dead for_each_*_submodule()\n - revision.c: use refs_for_each*() instead of for_each_*_submodule()\n - refs: add refs_head_ref()\n - refs: move submodule slash stripping code to get_submodule_ref_store\n - refs.c: refactor get_submodule_ref_store(), share common free block\n - revision.c: --indexed-objects add objects from all worktrees\n - revision.c: refactor add_index_objects_to_pending()\n - revision.h: new flag in struct rev_info wrt. worktree-related refs\n (this branch uses nd/worktree-kill-parse-ref.)\n\n \"git gc\" and friends when multiple worktrees are used off of a\n single repository did not consider the index and per-worktree refs\n of other worktrees as the root for reachability traversal, making\n objects that are in use only in other worktrees to be subject to\n garbage collection.\n\n Expecting a reroll.\n Waiting for nd/worktree-kill-parse-ref to settle.\n cf. \n\n\n* nd/worktree-kill-parse-ref (2017-04-24) 6 commits\n (merged to 'next' on 2017-04-26 at b8e40da709)\n + refs: kill set_worktree_head_symref()\n + worktree.c: kill parse_ref() in favor of refs_resolve_ref_unsafe()\n + refs: introduce get_worktree_ref_store()\n + refs: add REFS_STORE_ALL_CAPS\n + refs.c: make submodule ref store hashmap generic\n + environment.c: fix potential segfault by get_git_common_dir()\n (this branch is used by nd/prune-in-worktree.)\n\n \"git gc\" did not interact well with \"git worktree\"-managed\n per-worktree refs.\n\n Will cook in 'next'.\n\n\n* jc/bundle (2016-03-03) 6 commits\n - index-pack: --clone-bundle option\n - Merge branch 'jc/index-pack' into jc/bundle\n - bundle v3: the beginning\n - bundle: keep a copy of bundle file name in the in-core bundle header\n - bundle: plug resource leak\n - bundle doc: 'verify' is not about verifying the bundle\n\n The beginning of \"split bundle\", which could be one of the\n ingredients to allow \"git clone\" traffic off of the core server\n network to CDN.\n"},{"id":"318371","messageId":"CACBZZX4ty91vh=ykMtpF0pV8Ru3BKWBXyQYsFRdPkKkxMd8pzw@mail.gmail.com","threadId":"45840","inReplyTo":"xmqqefw9gmvq.fsf@gitster.mtv.corp.google.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-01T14:25:14Z","receivedAt":"2017-05-01T14:25:41Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Mon, May 1, 2017 at 7:35 AM, Junio C Hamano wrote:\n> * ab/clone-no-tags (2017-05-01) 3 commits\n> (merged to 'next' on 2017-04-30 at 601649896a)\n> + tests: rename a test having to do with shallow submodules\n> + clone: add a --no-tags option to clone without tags\n> + tests: change \"cd ... && git fetch\" to \"cd &&\\n\\tgit fetch\"\n>\n> \"git clone\" learned the \"--no-tags\" option not to fetch all tags\n> initially, and also set up the tagopt not to follow any tags in\n> subsequent fetches.\n>\n> Will cook in 'next'.\n\nThanks for trimming off the top 2 patches. I've dropped those myself,\nif someone (Brandon || Stefan) more interested in working on\nsubmodules wants to pick them up that would be neat, but I don't need\nit myself & doing it differently than the existing submodule options\nwould take too much of my time.\n\n> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n> - SQUASH???\n> - Makefile & configure: make PCRE v2 the default PCRE implementation\n> - grep: remove support for concurrent use of both PCRE v1 & v2\n> - grep: add support for PCRE v2\n> - grep: add support for the PCRE v1 JIT API\n> - perf: add a performance comparison test of grep -E and -P\n> - grep: change the internal PCRE code & header names to be PCRE1\n> - grep: change the internal PCRE macro names to be PCRE1\n> - test-lib: rename the LIBPCRE prerequisite to PCRE\n> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n> - log: add -P as a synonym for --perl-regexp\n> - log: add exhaustive tests for pattern style options & config\n> - grep: add a test for backreferences in PCRE patterns\n> - Makefile & configure: reword outdated comment about PCRE\n> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n> - grep: remove redundant regflags assignment under PCRE\n> - grep: submodule-related case statements should die if new fields are added\n> - grep: add tests for grep pattern types being passed to submodules\n> - grep: amend submodule recursion test in preparation for rx engine testing\n>\n> PCRE2, which has an API different from and incompatible with PCRE,\n> can now be chosen to support \"grep -P -e ''\" and friends.\n\nThat squash looks good to me.\n\n> * ab/grep-threading-cleanup (2017-04-16) 8 commits\n> - grep: given --threads with NO_PTHREADS=YesPlease, warn\n> - pack-objects: fix buggy warning about threads under NO_PTHREADS=YesPlease\n> - pack-object & index-pack: add test for --threads warning under NO_PTHREADS\n> - tests: add a PTHREADS prerequisite\n> - grep: skip pthreads overhead when using one thread\n> - grep: don't redundantly compile throwaway patterns under threading\n> - grep: add tests for --threads=N and grep.threads\n> - grep: assert that threading is enabled when calling grep_{lock,unlock}\n>\n> Code cleanup.\n>\n> Needs review.\n\nBetween these two series there's 27 patches, and I understand it's a\nbit of a PITA to review/get comments on it.\n\nAnything I should be doing differently here other than just waiting\nfor 2.13 to come out so they can be cooked further & merged down to\nnext & then master if there's no objections?\n"},{"id":"318374","messageId":"20170501162132.GA39135@google.com","threadId":"45840","inReplyTo":"CACBZZX4ty91vh=ykMtpF0pV8Ru3BKWBXyQYsFRdPkKkxMd8pzw@mail.gmail.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-05-01T16:21:32Z","receivedAt":"2017-05-01T16:21:56Z","isPatch":false,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 05/01, Ævar Arnfjörð Bjarmason wrote:\n> On Mon, May 1, 2017 at 7:35 AM, Junio C Hamano wrote:\n> > * ab/clone-no-tags (2017-05-01) 3 commits\n> > (merged to 'next' on 2017-04-30 at 601649896a)\n> > + tests: rename a test having to do with shallow submodules\n> > + clone: add a --no-tags option to clone without tags\n> > + tests: change \"cd ... && git fetch\" to \"cd &&\\n\\tgit fetch\"\n> >\n> > \"git clone\" learned the \"--no-tags\" option not to fetch all tags\n> > initially, and also set up the tagopt not to follow any tags in\n> > subsequent fetches.\n> >\n> > Will cook in 'next'.\n> \n> Thanks for trimming off the top 2 patches. I've dropped those myself,\n> if someone (Brandon || Stefan) more interested in working on\n> submodules wants to pick them up that would be neat, but I don't need\n> it myself & doing it differently than the existing submodule options\n> would take too much of my time.\n\nYeah we can add it to our backlog, though I'm not sure how quickly we'll\nbe able to get to it. If you end up needing this sooner just let me\nknow.\n\n-- \nBrandon Williams\n"},{"id":"318381","messageId":"CACBZZX6FfLEQGaCWj=vsMrOsPwKSEc2uW+bXDu4e3rhe18_VaQ@mail.gmail.com","threadId":"45840","inReplyTo":"20170501162132.GA39135@google.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-01T17:44:50Z","receivedAt":"2017-05-01T17:46:06Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Mon, May 1, 2017 at 6:21 PM, Brandon Williams wrote:\n> On 05/01, Ęvar Arnfjörš Bjarmason wrote:\n>> On Mon, May 1, 2017 at 7:35 AM, Junio C Hamano wrote:\n>> > * ab/clone-no-tags (2017-05-01) 3 commits\n>> > (merged to 'next' on 2017-04-30 at 601649896a)\n>> > + tests: rename a test having to do with shallow submodules\n>> > + clone: add a --no-tags option to clone without tags\n>> > + tests: change \"cd ... && git fetch\" to \"cd &&\\n\\tgit fetch\"\n>> >\n>> > \"git clone\" learned the \"--no-tags\" option not to fetch all tags\n>> > initially, and also set up the tagopt not to follow any tags in\n>> > subsequent fetches.\n>> >\n>> > Will cook in 'next'.\n>>\n>> Thanks for trimming off the top 2 patches. I've dropped those myself,\n>> if someone (Brandon || Stefan) more interested in working on\n>> submodules wants to pick them up that would be neat, but I don't need\n>> it myself & doing it differently than the existing submodule options\n>> would take too much of my time.\n>\n> Yeah we can add it to our backlog, though I'm not sure how quickly we'll\n> be able to get to it. If you end up needing this sooner just let me\n> know.\n\nI don't need it at all, I just thought it was prudent to add the\nsubmodule part of passing clone options / config when adding a new\noption since there were some existing options that did that.\n\nSince there's outstanding debate about the interface for that sort of\nthing I'll just leave that to people who care more about submodules if\nthey'd like to pursue it.\n"},{"id":"318402","messageId":"20170501192942.lhed5ygwj3micjz4@sigill.intra.peff.net","threadId":"45840","inReplyTo":"CACBZZX4ty91vh=ykMtpF0pV8Ru3BKWBXyQYsFRdPkKkxMd8pzw@mail.gmail.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-05-01T19:29:43Z","receivedAt":"2017-05-01T19:33:46Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 01, 2017 at 04:25:14PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> > * ab/grep-pcre-v2 (2017-04-25) 20 commits\n> [...]\n> > * ab/grep-threading-cleanup (2017-04-16) 8 commits\n> [...]\n> >\n> > Needs review.\n> \n> Between these two series there's 27 patches, and I understand it's a\n> bit of a PITA to review/get comments on it.\n> \n> Anything I should be doing differently here other than just waiting\n> for 2.13 to come out so they can be cooked further & merged down to\n> next & then master if there's no objections?\n\nThese are both on my todo list to review. That's not a promise of\ntimeliness, but at least they didn't get dropped into a void. :) If you\nhaven't heard anything post-2.13, I think reposting them then would be\nreasonable.\n\n-Peff\n"},{"id":"318421","messageId":"xmqqa86wgo33.fsf@gitster.mtv.corp.google.com","threadId":"45840","inReplyTo":"CACBZZX4ty91vh=ykMtpF0pV8Ru3BKWBXyQYsFRdPkKkxMd8pzw@mail.gmail.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-01T23:21:52Z","receivedAt":"2017-05-01T23:22:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n>> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n>> - SQUASH???\n>> - Makefile & configure: make PCRE v2 the default PCRE implementation\n>> - grep: remove support for concurrent use of both PCRE v1 & v2\n>> - grep: add support for PCRE v2\n>> - grep: add support for the PCRE v1 JIT API\n>> - perf: add a performance comparison test of grep -E and -P\n>> - grep: change the internal PCRE code & header names to be PCRE1\n>> - grep: change the internal PCRE macro names to be PCRE1\n>> - test-lib: rename the LIBPCRE prerequisite to PCRE\n>> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n>> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n>> - log: add -P as a synonym for --perl-regexp\n>> - log: add exhaustive tests for pattern style options & config\n>> - grep: add a test for backreferences in PCRE patterns\n>> - Makefile & configure: reword outdated comment about PCRE\n>> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n>> - grep: remove redundant regflags assignment under PCRE\n>> - grep: submodule-related case statements should die if new fields are added\n>> - grep: add tests for grep pattern types being passed to submodules\n>> - grep: amend submodule recursion test in preparation for rx engine testing\n>>\n>> PCRE2, which has an API different from and incompatible with PCRE,\n>> can now be chosen to support \"grep -P -e ''\" and friends.\n>\n> That squash looks good to me.\n\nThanks.\n\nThat is not a particulary helpful comment, by the way. I can help\ntopics by contributors by queuing emergency fix at the tip to make\nones that do not build correctly buildable and testable (which is\nwhat the \"SQUASH???\" commits are about), but I'd rather not see me\nforced to find among 19 commits which one is broken and needs the\nhotfix squashed in myself.\n\n>> * ab/grep-threading-cleanup (2017-04-16) 8 commits\n>> - grep: given --threads with NO_PTHREADS=YesPlease, warn\n>> - pack-objects: fix buggy warning about threads under NO_PTHREADS=YesPlease\n>> - pack-object & index-pack: add test for --threads warning under NO_PTHREADS\n>> - tests: add a PTHREADS prerequisite\n>> - grep: skip pthreads overhead when using one thread\n>> - grep: don't redundantly compile throwaway patterns under threading\n>> - grep: add tests for --threads=N and grep.threads\n>> - grep: assert that threading is enabled when calling grep_{lock,unlock}\n>>\n>> Code cleanup.\n>>\n>> Needs review.\n>\n> Between these two series there's 27 patches, and I understand it's a\n> bit of a PITA to review/get comments on it.\n>\n> Anything I should be doing differently here other than just waiting\n> for 2.13 to come out so they can be cooked further & merged down to\n> next & then master if there's no objections?\n\nThere are topics that need fresh eyes to be reviewed by other\ncontributors, so perhaps you can help unblock them by reviewing,\nwhile they pick lints from yours?\n"},{"id":"318475","messageId":"CACBZZX5N0L_M9b4RTRLSvcA2u69H=s=S-VEc-Nr71wW6Scyjvw@mail.gmail.com","threadId":"45840","inReplyTo":"xmqqa86wgo33.fsf@gitster.mtv.corp.google.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-02T08:23:46Z","receivedAt":"2017-05-02T08:24:18Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 2, 2017 at 1:21 AM, Junio C Hamano wrote:\n> Ævar Arnfjörð Bjarmason writes:\n>\n>>> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n>>> - SQUASH???\n>>> - Makefile & configure: make PCRE v2 the default PCRE implementation\n>>> - grep: remove support for concurrent use of both PCRE v1 & v2\n>>> - grep: add support for PCRE v2\n>>> - grep: add support for the PCRE v1 JIT API\n>>> - perf: add a performance comparison test of grep -E and -P\n>>> - grep: change the internal PCRE code & header names to be PCRE1\n>>> - grep: change the internal PCRE macro names to be PCRE1\n>>> - test-lib: rename the LIBPCRE prerequisite to PCRE\n>>> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n>>> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n>>> - log: add -P as a synonym for --perl-regexp\n>>> - log: add exhaustive tests for pattern style options & config\n>>> - grep: add a test for backreferences in PCRE patterns\n>>> - Makefile & configure: reword outdated comment about PCRE\n>>> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n>>> - grep: remove redundant regflags assignment under PCRE\n>>> - grep: submodule-related case statements should die if new fields are added\n>>> - grep: add tests for grep pattern types being passed to submodules\n>>> - grep: amend submodule recursion test in preparation for rx engine testing\n>>>\n>>> PCRE2, which has an API different from and incompatible with PCRE,\n>>> can now be chosen to support \"grep -P -e ''\" and friends.\n>>\n>> That squash looks good to me.\n>\n> Thanks.\n>\n> That is not a particulary helpful comment, by the way. I can help\n> topics by contributors by queuing emergency fix at the tip to make\n> ones that do not build correctly buildable and testable (which is\n> what the \"SQUASH???\" commits are about), but I'd rather not see me\n> forced to find among 19 commits which one is broken and needs the\n> hotfix squashed in myself.\n\nI'm happy to change what I'm doing to be more helpful, but it's not\nclear to me from this & the context what that would be.\n\n* I sent a v4 that had this bug in <20170425210548.24612-6-avarab@gmail.com>\n* You pointed out that initialization bug in response\n* I sent a v5 of just that patch (not the rest of the series) in\nresponse to that in <20170426074856.29903-1-avarab@gmail.com>\n* You replied in in a\nreply I (probably mis-)read as \"no worries, I'll just squash the fix\nin\"\n\nSo now a ~week later in WCIG you've taken the series but added that\nsquash instead of using my v5 of that one patch, that looks good to me\n(i.e. your hotfix does the same thing as my v5) but you don't think\nthat's a helpful comment.\n\nSo what would you like to have happen instead? If it's easier I could\njust re-sent a v6 of the whole thing and we could do away with this\nsquash/replace-one-patch dance.\n\n>>> * ab/grep-threading-cleanup (2017-04-16) 8 commits\n>>> - grep: given --threads with NO_PTHREADS=YesPlease, warn\n>>> - pack-objects: fix buggy warning about threads under NO_PTHREADS=YesPlease\n>>> - pack-object & index-pack: add test for --threads warning under NO_PTHREADS\n>>> - tests: add a PTHREADS prerequisite\n>>> - grep: skip pthreads overhead when using one thread\n>>> - grep: don't redundantly compile throwaway patterns under threading\n>>> - grep: add tests for --threads=N and grep.threads\n>>> - grep: assert that threading is enabled when calling grep_{lock,unlock}\n>>>\n>>> Code cleanup.\n>>>\n>>> Needs review.\n>>\n>> Between these two series there's 27 patches, and I understand it's a\n>> bit of a PITA to review/get comments on it.\n>>\n>> Anything I should be doing differently here other than just waiting\n>> for 2.13 to come out so they can be cooked further & merged down to\n>> next & then master if there's no objections?\n>\n> There are topics that need fresh eyes to be reviewed by other\n> contributors, so perhaps you can help unblock them by reviewing,\n> while they pick lints from yours?\n\n*nod*\n"},{"id":"318479","messageId":"xmqqr307d2jc.fsf@gitster.mtv.corp.google.com","threadId":"45840","inReplyTo":"CACBZZX5N0L_M9b4RTRLSvcA2u69H=s=S-VEc-Nr71wW6Scyjvw@mail.gmail.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-02T09:35:35Z","receivedAt":"2017-05-02T09:35:45Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n>>> That squash looks good to me.\n>>\n>> Thanks.\n>>\n>> That is not a particulary helpful comment, by the way. I can help\n>> topics by contributors by queuing emergency fix at the tip to make\n>> ones that do not build correctly buildable and testable (which is\n>> what the \"SQUASH???\" commits are about), but I'd rather not see me\n>> forced to find among 19 commits which one is broken and needs the\n>> hotfix squashed in myself.\n>\n> I'm happy to change what I'm doing to be more helpful, but it's not\n> clear to me from this & the context what that would be.\n>\n> * I sent a v4 that had this bug in <20170425210548.24612-6-avarab@gmail.com>\n> * You pointed out that initialization bug in response\n> * I sent a v5 of just that patch (not the rest of the series) in\n> response to that in <20170426074856.29903-1-avarab@gmail.com>\n> * You replied in in a\n> reply I (probably mis-)read as \"no worries, I'll just squash the fix\n> in\"\n\nSorry, I completely forgot about our exchange around your v5. If\nyour comment were \"squash is good but you've seen a replacement sent\nas v5 that is not there yet\", I wouldn't have made such a silly\ncomment, but given that I've already responded to your v5 saying\nI'll handle it, that is asking too much from you.\n\nWhat I pushed out a few hours ago should already have the fix in.\nThanks for clarifying the situation, and sorry again.\n\n\n\n"},{"id":"318483","messageId":"alpine.DEB.2.20.1705021310290.3480@virtualbox","threadId":"45840","inReplyTo":"xmqqefw9gmvq.fsf@gitster.mtv.corp.google.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-02T11:11:55Z","receivedAt":"2017-05-02T11:12:07Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Sun, 30 Apr 2017, Junio C Hamano wrote:\n\n> * js/rebase-i-final (2017-04-27) 9 commits\n> - rebase -i: rearrange fixup/squash lines using the rebase--helper\n> - t3415: test fixup with wrapped oneline\n> - rebase -i: skip unnecessary picks using the rebase--helper\n> - rebase -i: check for missing commits in the rebase--helper\n> - t3404: relax rebase.missingCommitsCheck tests\n> - rebase -i: also expand/collapse the SHA-1s via the rebase--helper\n> - rebase -i: do not invent onelines when expanding/collapsing SHA-1s\n> - rebase -i: remove useless indentation\n> - rebase -i: generate the script via rebase--helper\n> \n> The final batch to \"git rebase -i\" updates to move more code from\n> the shell script to C.\n\nThis is not the final batch, hopefully. It is the last batch of my\noriginal work to accelerate rebase -i, and it is a blocker for more work\ntoward making rebase -i a builtin. Once this patch series is in, I plan on\ndoing more work on that front.\n\nCiao,\nDscho\n"},{"id":"318486","messageId":"alpine.DEB.2.20.1705021406510.3480@virtualbox","threadId":"45840","inReplyTo":"xmqqefw9gmvq.fsf@gitster.mtv.corp.google.com","subject":"PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-02T12:09:00Z","receivedAt":"2017-05-02T12:09:34Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Sun, 30 Apr 2017, Junio C Hamano wrote:\n\n> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n> - SQUASH???\n> - Makefile & configure: make PCRE v2 the default PCRE implementation\n> - grep: remove support for concurrent use of both PCRE v1 & v2\n> - grep: add support for PCRE v2\n> - grep: add support for the PCRE v1 JIT API\n> - perf: add a performance comparison test of grep -E and -P\n> - grep: change the internal PCRE code & header names to be PCRE1\n> - grep: change the internal PCRE macro names to be PCRE1\n> - test-lib: rename the LIBPCRE prerequisite to PCRE\n> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n> - log: add -P as a synonym for --perl-regexp\n> - log: add exhaustive tests for pattern style options & config\n> - grep: add a test for backreferences in PCRE patterns\n> - Makefile & configure: reword outdated comment about PCRE\n> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n> - grep: remove redundant regflags assignment under PCRE\n> - grep: submodule-related case statements should die if new fields are added\n> - grep: add tests for grep pattern types being passed to submodules\n> - grep: amend submodule recursion test in preparation for rx engine testing\n> \n> PCRE2, which has an API different from and incompatible with PCRE,\n> can now be chosen to support \"grep -P -e ''\" and friends.\n\nFWIW for quite a couple of recent builds, `pu` fails on Windows with a\nvariation of this error:\n\n\t CC blob.o\n\tIn file included from revision.h:5:0,\n\t\t\t from bisect.c:4:\n\tgrep.h:16:19: fatal error: pcre2.h: No such file or directory\n\t #include \n\t\t\t ^\n\tcompilation terminated.\n\nMaybe this can be fixed before hitting `next`?\n\nCiao,\nDscho"},{"id":"318488","messageId":"CACBZZX5M1Pnvw01wP8id75Ja9NJ3nwVfydsX6g0Ys_QD72r6dQ@mail.gmail.com","threadId":"45840","inReplyTo":"alpine.DEB.2.20.1705021406510.3480@virtualbox","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-02T12:27:08Z","receivedAt":"2017-05-02T12:27:34Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 2, 2017 at 2:09 PM, Johannes Schindelin\n wrote:\n> Hi Ævar,\n>\n> On Sun, 30 Apr 2017, Junio C Hamano wrote:\n>\n>> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n>> - SQUASH???\n>> - Makefile & configure: make PCRE v2 the default PCRE implementation\n>> - grep: remove support for concurrent use of both PCRE v1 & v2\n>> - grep: add support for PCRE v2\n>> - grep: add support for the PCRE v1 JIT API\n>> - perf: add a performance comparison test of grep -E and -P\n>> - grep: change the internal PCRE code & header names to be PCRE1\n>> - grep: change the internal PCRE macro names to be PCRE1\n>> - test-lib: rename the LIBPCRE prerequisite to PCRE\n>> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n>> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n>> - log: add -P as a synonym for --perl-regexp\n>> - log: add exhaustive tests for pattern style options & config\n>> - grep: add a test for backreferences in PCRE patterns\n>> - Makefile & configure: reword outdated comment about PCRE\n>> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n>> - grep: remove redundant regflags assignment under PCRE\n>> - grep: submodule-related case statements should die if new fields are added\n>> - grep: add tests for grep pattern types being passed to submodules\n>> - grep: amend submodule recursion test in preparation for rx engine testing\n>>\n>> PCRE2, which has an API different from and incompatible with PCRE,\n>> can now be chosen to support \"grep -P -e ''\" and friends.\n>\n> FWIW for quite a couple of recent builds, `pu` fails on Windows with a\n> variation of this error:\n>\n> CC blob.o\n> In file included from revision.h:5:0,\n> from bisect.c:4:\n> grep.h:16:19: fatal error: pcre2.h: No such file or directory\n> #include \n> ^\n> compilation terminated.\n>\n> Maybe this can be fixed before hitting `next`?\n\nThis will be due to a combination of the build machine not having pcre\nv2 (but having v1) & my \"Makefile & configure: make PCRE v2 the\ndefault PCRE implementation\" patch, which makes v2 the default for\nUSE_LIBPCRE=YesPlease.\n\nIs it easy to install v2 on these build machines? Alternatively that\npatch could be ejected out of pu, or you could USE_LIBPCRE1=YesPlease\nto use v1, but as explained in that commit I think it makes sense to\nmake v2 the default.\n"},{"id":"318495","messageId":"CACBZZX53Y50CbmkPb1USmXF4E+BzBbyo_Sr=H+Q55rujF9+ZvA@mail.gmail.com","threadId":"45840","inReplyTo":"xmqqr307d2jc.fsf@gitster.mtv.corp.google.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-02T12:31:59Z","receivedAt":"2017-05-02T12:32:32Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 2, 2017 at 11:35 AM, Junio C Hamano wrote:\n> Ævar Arnfjörð Bjarmason writes:\n>\n>>>> That squash looks good to me.\n>>>\n>>> Thanks.\n>>>\n>>> That is not a particulary helpful comment, by the way. I can help\n>>> topics by contributors by queuing emergency fix at the tip to make\n>>> ones that do not build correctly buildable and testable (which is\n>>> what the \"SQUASH???\" commits are about), but I'd rather not see me\n>>> forced to find among 19 commits which one is broken and needs the\n>>> hotfix squashed in myself.\n>>\n>> I'm happy to change what I'm doing to be more helpful, but it's not\n>> clear to me from this & the context what that would be.\n>>\n>> * I sent a v4 that had this bug in <20170425210548.24612-6-avarab@gmail.com>\n>> * You pointed out that initialization bug in response\n>> * I sent a v5 of just that patch (not the rest of the series) in\n>> response to that in <20170426074856.29903-1-avarab@gmail.com>\n>> * You replied in in a\n>> reply I (probably mis-)read as \"no worries, I'll just squash the fix\n>> in\"\n>\n> Sorry, I completely forgot about our exchange around your v5. If\n> your comment were \"squash is good but you've seen a replacement sent\n> as v5 that is not there yet\", I wouldn't have made such a silly\n> comment, but given that I've already responded to your v5 saying\n> I'll handle it, that is asking too much from you.\n>\n> What I pushed out a few hours ago should already have the fix in.\n> Thanks for clarifying the situation, and sorry again.\n\nOkey, no worries. Was just a bit confused & wondering if I could make\nsomething easier for you in the future.\n"},{"id":"318538","messageId":"alpine.DEB.2.20.1705021756530.3480@virtualbox","threadId":"45840","inReplyTo":"CACBZZX5M1Pnvw01wP8id75Ja9NJ3nwVfydsX6g0Ys_QD72r6dQ@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-02T16:05:13Z","receivedAt":"2017-05-02T16:05:24Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Tue, 2 May 2017, Ævar Arnfjörð Bjarmason wrote:\n\n> On Tue, May 2, 2017 at 2:09 PM, Johannes Schindelin\n> wrote:\n> >\n> > On Sun, 30 Apr 2017, Junio C Hamano wrote:\n> >\n> >> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n> >> - SQUASH???\n> >> - Makefile & configure: make PCRE v2 the default PCRE implementation\n> >> - grep: remove support for concurrent use of both PCRE v1 & v2\n> >> - grep: add support for PCRE v2\n> >> - grep: add support for the PCRE v1 JIT API\n> >> - perf: add a performance comparison test of grep -E and -P\n> >> - grep: change the internal PCRE code & header names to be PCRE1\n> >> - grep: change the internal PCRE macro names to be PCRE1\n> >> - test-lib: rename the LIBPCRE prerequisite to PCRE\n> >> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n> >> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n> >> - log: add -P as a synonym for --perl-regexp\n> >> - log: add exhaustive tests for pattern style options & config\n> >> - grep: add a test for backreferences in PCRE patterns\n> >> - Makefile & configure: reword outdated comment about PCRE\n> >> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n> >> - grep: remove redundant regflags assignment under PCRE\n> >> - grep: submodule-related case statements should die if new fields are added\n> >> - grep: add tests for grep pattern types being passed to submodules\n> >> - grep: amend submodule recursion test in preparation for rx engine testing\n> >>\n> >> PCRE2, which has an API different from and incompatible with PCRE,\n> >> can now be chosen to support \"grep -P -e ''\" and friends.\n> >\n> > FWIW for quite a couple of recent builds, `pu` fails on Windows with a\n> > variation of this error:\n> >\n> > CC blob.o\n> > In file included from revision.h:5:0,\n> > from bisect.c:4:\n> > grep.h:16:19: fatal error: pcre2.h: No such file or directory\n> > #include \n> > ^\n> > compilation terminated.\n> >\n> > Maybe this can be fixed before hitting `next`?\n> \n> This will be due to a combination of the build machine not having pcre\n> v2 (but having v1) & my \"Makefile & configure: make PCRE v2 the\n> default PCRE implementation\" patch, which makes v2 the default for\n> USE_LIBPCRE=YesPlease.\n> \n> Is it easy to install v2 on these build machines? Alternatively that\n> patch could be ejected out of pu, or you could USE_LIBPCRE1=YesPlease\n> to use v1, but as explained in that commit I think it makes sense to\n> make v2 the default.\n\nLet me put it this way: Installing PCRE v1 in MSYS2 is as easy as\n\n\tpacman -Sy mingw-w64-x86_64-pcre\n\nTo install PCRE v2, you would have to copy-edit\nhttps://github.com/Alexpux/MINGW-packages/blob/master/mingw-w64-pcre/PKGBUILD,\nmake sure that it builds by running it through\n\n\tmakepkg-mingw -s\n\npossibly initializing a Git repository in the\nmingw-w64-pcre/src/${_realname}-${pkgver}/ directory, patching the source\nuntil it builds, committing the changes, adding them as patch files to the\nsame directory as the new PKGBUILD, adjusting the PKGBUILD file to list\nthe patch files with their checksums and to add the commands to apply\nthem.\n\nThen (and this is a one time cost, fortunately) initializing two packages\non BinTray (which we use to serve the Pacman packages of Git for Windows),\nthen build and upload the packages.\n\nIn short, PCRE v2 would be slightly (ahem, ahem) more involved than PCRE\nv1.\n\nI cannot imagine that MSYS2 is the only environment with that issue.\n\nCiao,\nDscho"},{"id":"318546","messageId":"20170502174329.GB181268@google.com","threadId":"45840","inReplyTo":"CACBZZX5M1Pnvw01wP8id75Ja9NJ3nwVfydsX6g0Ys_QD72r6dQ@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-05-02T17:43:29Z","receivedAt":"2017-05-02T17:43:39Z","isPatch":false,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 05/02, Ævar Arnfjörð Bjarmason wrote:\n> On Tue, May 2, 2017 at 2:09 PM, Johannes Schindelin\n> wrote:\n> > Hi Ævar,\n> >\n> > On Sun, 30 Apr 2017, Junio C Hamano wrote:\n> >\n> >> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n> >> - SQUASH???\n> >> - Makefile & configure: make PCRE v2 the default PCRE implementation\n> >> - grep: remove support for concurrent use of both PCRE v1 & v2\n> >> - grep: add support for PCRE v2\n> >> - grep: add support for the PCRE v1 JIT API\n> >> - perf: add a performance comparison test of grep -E and -P\n> >> - grep: change the internal PCRE code & header names to be PCRE1\n> >> - grep: change the internal PCRE macro names to be PCRE1\n> >> - test-lib: rename the LIBPCRE prerequisite to PCRE\n> >> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n> >> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n> >> - log: add -P as a synonym for --perl-regexp\n> >> - log: add exhaustive tests for pattern style options & config\n> >> - grep: add a test for backreferences in PCRE patterns\n> >> - Makefile & configure: reword outdated comment about PCRE\n> >> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n> >> - grep: remove redundant regflags assignment under PCRE\n> >> - grep: submodule-related case statements should die if new fields are added\n> >> - grep: add tests for grep pattern types being passed to submodules\n> >> - grep: amend submodule recursion test in preparation for rx engine testing\n> >>\n> >> PCRE2, which has an API different from and incompatible with PCRE,\n> >> can now be chosen to support \"grep -P -e ''\" and friends.\n> >\n> > FWIW for quite a couple of recent builds, `pu` fails on Windows with a\n> > variation of this error:\n> >\n> > CC blob.o\n> > In file included from revision.h:5:0,\n> > from bisect.c:4:\n> > grep.h:16:19: fatal error: pcre2.h: No such file or directory\n> > #include \n> > ^\n> > compilation terminated.\n> >\n> > Maybe this can be fixed before hitting `next`?\n> \n> This will be due to a combination of the build machine not having pcre\n> v2 (but having v1) & my \"Makefile & configure: make PCRE v2 the\n> default PCRE implementation\" patch, which makes v2 the default for\n> USE_LIBPCRE=YesPlease.\n> \n> Is it easy to install v2 on these build machines? Alternatively that\n> patch could be ejected out of pu, or you could USE_LIBPCRE1=YesPlease\n> to use v1, but as explained in that commit I think it makes sense to\n> make v2 the default.\n\nShouldn't the Makefile check for the existence of PCREv2 before making\nit the default?\n\n-- \nBrandon Williams\n"},{"id":"318547","messageId":"CACBZZX5_45KnU7qzW2ojJiznmfkef44YGL8-CYkHFLOvhLSASg@mail.gmail.com","threadId":"45840","inReplyTo":"20170502174329.GB181268@google.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-02T17:49:53Z","receivedAt":"2017-05-02T17:50:19Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 2, 2017 at 7:43 PM, Brandon Williams wrote:\n> On 05/02, Ævar Arnfjörð Bjarmason wrote:\n>> On Tue, May 2, 2017 at 2:09 PM, Johannes Schindelin\n>> wrote:\n>> > Hi Ævar,\n>> >\n>> > On Sun, 30 Apr 2017, Junio C Hamano wrote:\n>> >\n>> >> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n>> >> - SQUASH???\n>> >> - Makefile & configure: make PCRE v2 the default PCRE implementation\n>> >> - grep: remove support for concurrent use of both PCRE v1 & v2\n>> >> - grep: add support for PCRE v2\n>> >> - grep: add support for the PCRE v1 JIT API\n>> >> - perf: add a performance comparison test of grep -E and -P\n>> >> - grep: change the internal PCRE code & header names to be PCRE1\n>> >> - grep: change the internal PCRE macro names to be PCRE1\n>> >> - test-lib: rename the LIBPCRE prerequisite to PCRE\n>> >> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n>> >> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n>> >> - log: add -P as a synonym for --perl-regexp\n>> >> - log: add exhaustive tests for pattern style options & config\n>> >> - grep: add a test for backreferences in PCRE patterns\n>> >> - Makefile & configure: reword outdated comment about PCRE\n>> >> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n>> >> - grep: remove redundant regflags assignment under PCRE\n>> >> - grep: submodule-related case statements should die if new fields are added\n>> >> - grep: add tests for grep pattern types being passed to submodules\n>> >> - grep: amend submodule recursion test in preparation for rx engine testing\n>> >>\n>> >> PCRE2, which has an API different from and incompatible with PCRE,\n>> >> can now be chosen to support \"grep -P -e ''\" and friends.\n>> >\n>> > FWIW for quite a couple of recent builds, `pu` fails on Windows with a\n>> > variation of this error:\n>> >\n>> > CC blob.o\n>> > In file included from revision.h:5:0,\n>> > from bisect.c:4:\n>> > grep.h:16:19: fatal error: pcre2.h: No such file or directory\n>> > #include \n>> > ^\n>> > compilation terminated.\n>> >\n>> > Maybe this can be fixed before hitting `next`?\n>>\n>> This will be due to a combination of the build machine not having pcre\n>> v2 (but having v1) & my \"Makefile & configure: make PCRE v2 the\n>> default PCRE implementation\" patch, which makes v2 the default for\n>> USE_LIBPCRE=YesPlease.\n>>\n>> Is it easy to install v2 on these build machines? Alternatively that\n>> patch could be ejected out of pu, or you could USE_LIBPCRE1=YesPlease\n>> to use v1, but as explained in that commit I think it makes sense to\n>> make v2 the default.\n>\n> Shouldn't the Makefile check for the existence of PCREv2 before making\n> it the default?\n\nWe can't check for the existence of anything in the Makefile, that's\nwhat the configure script is for.\n\nThe configure script does the \"right\" thing with pcre v2.\n\nThe \"right\" thing being the bizarro pre-existing behavior of \"oh you\nasked for PCRE? Well let me check if it's on the system, if it's not\nI'll just silently undo what you asked for\". Which is crazy, but was\nthere before my patches, so I didn't change it.\n\nI.e. instead of the more sane behavior of opting the user into an\noptional library if it's found on the system, and producing an error\nif you supply --with-that-library but don't have thatlibrary.so.\n"},{"id":"318555","messageId":"CACBZZX6-qZLEGob6CEwpJ7jtEBG6WLPdHQsO4DsbkNZ8di5mjg@mail.gmail.com","threadId":"45840","inReplyTo":"alpine.DEB.2.20.1705021756530.3480@virtualbox","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-02T18:52:21Z","receivedAt":"2017-05-02T18:52:47Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 2, 2017 at 6:05 PM, Johannes Schindelin\n wrote:\n> Hi Ævar,\n>\n> On Tue, 2 May 2017, Ævar Arnfjörð Bjarmason wrote:\n>\n>> On Tue, May 2, 2017 at 2:09 PM, Johannes Schindelin\n>> wrote:\n>> >\n>> > On Sun, 30 Apr 2017, Junio C Hamano wrote:\n>> >\n>> >> * ab/grep-pcre-v2 (2017-04-25) 20 commits\n>> >> - SQUASH???\n>> >> - Makefile & configure: make PCRE v2 the default PCRE implementation\n>> >> - grep: remove support for concurrent use of both PCRE v1 & v2\n>> >> - grep: add support for PCRE v2\n>> >> - grep: add support for the PCRE v1 JIT API\n>> >> - perf: add a performance comparison test of grep -E and -P\n>> >> - grep: change the internal PCRE code & header names to be PCRE1\n>> >> - grep: change the internal PCRE macro names to be PCRE1\n>> >> - test-lib: rename the LIBPCRE prerequisite to PCRE\n>> >> - grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n>> >> - grep & rev-list doc: stop promising libpcre for --perl-regexp\n>> >> - log: add -P as a synonym for --perl-regexp\n>> >> - log: add exhaustive tests for pattern style options & config\n>> >> - grep: add a test for backreferences in PCRE patterns\n>> >> - Makefile & configure: reword outdated comment about PCRE\n>> >> - grep: remove redundant `regflags &= ~REG_EXTENDED` assignments\n>> >> - grep: remove redundant regflags assignment under PCRE\n>> >> - grep: submodule-related case statements should die if new fields are added\n>> >> - grep: add tests for grep pattern types being passed to submodules\n>> >> - grep: amend submodule recursion test in preparation for rx engine testing\n>> >>\n>> >> PCRE2, which has an API different from and incompatible with PCRE,\n>> >> can now be chosen to support \"grep -P -e ''\" and friends.\n>> >\n>> > FWIW for quite a couple of recent builds, `pu` fails on Windows with a\n>> > variation of this error:\n>> >\n>> > CC blob.o\n>> > In file included from revision.h:5:0,\n>> > from bisect.c:4:\n>> > grep.h:16:19: fatal error: pcre2.h: No such file or directory\n>> > #include \n>> > ^\n>> > compilation terminated.\n>> >\n>> > Maybe this can be fixed before hitting `next`?\n>>\n>> This will be due to a combination of the build machine not having pcre\n>> v2 (but having v1) & my \"Makefile & configure: make PCRE v2 the\n>> default PCRE implementation\" patch, which makes v2 the default for\n>> USE_LIBPCRE=YesPlease.\n>>\n>> Is it easy to install v2 on these build machines? Alternatively that\n>> patch could be ejected out of pu, or you could USE_LIBPCRE1=YesPlease\n>> to use v1, but as explained in that commit I think it makes sense to\n>> make v2 the default.\n>\n> Let me put it this way: Installing PCRE v1 in MSYS2 is as easy as\n>\n> pacman -Sy mingw-w64-x86_64-pcre\n>\n> To install PCRE v2, you would have to copy-edit\n> https://github.com/Alexpux/MINGW-packages/blob/master/mingw-w64-pcre/PKGBUILD,\n> make sure that it builds by running it through\n>\n> makepkg-mingw -s\n>\n> possibly initializing a Git repository in the\n> mingw-w64-pcre/src/${_realname}-${pkgver}/ directory, patching the source\n> until it builds, committing the changes, adding them as patch files to the\n> same directory as the new PKGBUILD, adjusting the PKGBUILD file to list\n> the patch files with their checksums and to add the commands to apply\n> them.\n>\n> Then (and this is a one time cost, fortunately) initializing two packages\n> on BinTray (which we use to serve the Pacman packages of Git for Windows),\n> then build and upload the packages.\n>\n> In short, PCRE v2 would be slightly (ahem, ahem) more involved than PCRE\n> v1.\n>\n> I cannot imagine that MSYS2 is the only environment with that issue.\n\nI think it's worth it to copy/paste the commit message where I made\nthis change, since we're having this discussion in this thread:\n\n Makefile & configure: make PCRE v2 the default PCRE implementation\n\n Change the USE_LIBPCRE=YesPlease & --with-libpcre flags to the\n Makefile & configure script, respectively, to mean use PCRE v2, not\n PCRE v1.\n\n The legacy library previously available via those options is still\n available on request via USE_LIBPCRE1=YesPlease or\n --with-libpcre1. The existing USE_LIBPCRE2=YesPlease & --with-libpcre2\n still explicitly ask for v2.\n\n The v2 PCRE is stable & end-user compatible, all this change does is\n change the default. Someone building a new git is likely to also have\n packaged PCRE v2 sometime in the last 2 years since it was released.\n If not they can choose to use the legacy v2 library by making the\n trivial s/USE_LIBPCRE/USE_LIBPCRE1/ change, or package up PCRE v2.\n\n New releases of PCRE v2 are already faster than PCRE v1, and not only\n is all significant development is happening on v2, but bugs in\n reported against v1 have started getting WONTFIX'd asking users to\n just upgrade to v2.\n\n So it makes sense to give our downstream distributors a nudge to\n switch over to it.\n\nSo that's why I did it. I also looked at several Linux distros & BSDs\n/ Solaris and they have it packaged, although some not for their\ncurrent stable release but for their \"next\" release, but that's likely\nwhat they're going to be building a new git for.\n\nBut, problems with this:\n\n * Debian testing has ~600 projects that rdepend on pcre1, with ~15\nthat rdepend on pcre2. Git would definitely be the most prominent\nproject using pcre2, it seems 2 years after release few have bothered\nto switch over.\n\n Thus even though it came out 2 years ago it's likely that a few\nplatforms like MSYS2 simply haven't bothered to package it yet\n(although packaging it is trivial if you already have a recipe for\nv1).\n\n * There's no set of compilation flags that work before & after my\npatch to request v1. I.e. before we only have v1 via USE_LIBPCRE=1,\nnow we have USE_LIBPCRE1=1 for v1, and either (or both) USE_LIBPCRE=1\n& USE_LIBPCRE2=1 for v2.\n\n I can't think of a good way out of this that doesn't both make sense\nMakefile UI-wise and allows the same CI script to Just Work as this\nchange migrates from pu->next->master.\n\n * Due to the bizarro existing semantics of the configure script noted\nupthread if you have a git build script that does --with-libpcre & you\nhave libpcre1 installed, it'll link to it, but now since\n--with-libpcre defaults to libpcre2 it'll silently skip linking to it\nif you don't have it installed.\n\n Arguably the first part of this series should be to change this\nbehavior so we don't silently subject distributors to this bad\nbehavior when and if we migrate to v2 by default, i.e. we should\ndetect & use a PCRE we find if it's there, and if the user explicitly\nasks for --with-libpcre and we can't find whatever the default is we\nshould error out.\n"},{"id":"318566","messageId":"20170502205138.GB17703@alpha.vpn.ikke.info","threadId":"45840","inReplyTo":"CACBZZX6-qZLEGob6CEwpJ7jtEBG6WLPdHQsO4DsbkNZ8di5mjg@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Kevin Daudt","fromEmail":"me@ikke.info","sentAt":"2017-05-02T20:51:38Z","receivedAt":"2017-05-02T20:51:45Z","isPatch":false,"sender":{"key":"me@ikke.info","avatar":"https://avatars.githubusercontent.com/u/135698?v=4"},"body":"On Tue, May 02, 2017 at 08:52:21PM +0200, Ævar Arnfjörð Bjarmason wrote:\n> \n> * Due to the bizarro existing semantics of the configure script noted\n> upthread if you have a git build script that does --with-libpcre & you\n> have libpcre1 installed, it'll link to it, but now since\n> --with-libpcre defaults to libpcre2 it'll silently skip linking to it\n> if you don't have it installed.\n> \n\nCase in point: The Archlinux git-git aur package[0] (community maintained,\nlatest git version) does run ./configure without --with-libpcre, but\n requests it from make with USE_LIBPCRE=1.\n\nI noticed when trying git grep -P which then failed.\n\n[0]: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=git-git\n"},{"id":"318567","messageId":"CACBZZX5ani4DLj4-63yZFPaJnWXquuFb+pXKvyHL61kja6YghA@mail.gmail.com","threadId":"45840","inReplyTo":"20170502205138.GB17703@alpha.vpn.ikke.info","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-02T21:16:22Z","receivedAt":"2017-05-02T21:16:48Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 2, 2017 at 10:51 PM, Kevin Daudt wrote:\n> On Tue, May 02, 2017 at 08:52:21PM +0200, Ęvar Arnfjörš Bjarmason wrote:\n>>\n>> * Due to the bizarro existing semantics of the configure script noted\n>> upthread if you have a git build script that does --with-libpcre & you\n>> have libpcre1 installed, it'll link to it, but now since\n>> --with-libpcre defaults to libpcre2 it'll silently skip linking to it\n>> if you don't have it installed.\n>>\n>\n> Case in point: The Archlinux git-git aur package[0] (community maintained,\n> latest git version) does run ./configure without --with-libpcre, but\n> requests it from make with USE_LIBPCRE=1.\n>\n> I noticed when trying git grep -P which then failed.\n>\n> [0]: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=git-git\n\nWhatever's going on there is unrelated to the issue I'm talking about,\nbut if that's producing a package where -P doesn't work that looks\nlike a bug in the Makefile.\n\nThe way --with-libpcre works now is that it second guesses you, i.e.\nit'll put USE_LIBPCRE=YesPlease into config.mak.autogen (sourced by\nthe Makefile) only if you supply --with-libpcre *and* it can find a\nworking libpcre on the system, otherwise it silently ignores you.\n\nWhatever you supply to the configure script it'll be overridden by\nMakefile arguments if present, but even if there was another bug where\n./configure arguments took precedence over Makefile arguments I don't\nsee how it would apply here, in this case nothing libpcre related will\nbe written to config.mak.autogen.\n\nSo I must be missing something. I don't see how that package build\ndoesn't either fail at compile time because it doesn't have pcre, or\nwork, in which case the -P option will work.\n"},{"id":"318612","messageId":"alpine.DEB.2.20.1705031139090.3480@virtualbox","threadId":"45840","inReplyTo":"CACBZZX6-qZLEGob6CEwpJ7jtEBG6WLPdHQsO4DsbkNZ8di5mjg@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-03T09:45:39Z","receivedAt":"2017-05-03T09:45:59Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Tue, 2 May 2017, Ævar Arnfjörð Bjarmason wrote:\n\n> On Tue, May 2, 2017 at 6:05 PM, Johannes Schindelin\n> wrote:\n>\n> > [... complaint about the abrupt change that makes Git for Windows fail\n> > to build because pcre2.h is missing ...]\n>\n> I think it's worth it to copy/paste the commit message where I made\n> this change, since we're having this discussion in this thread:\n> \n> Makefile & configure: make PCRE v2 the default PCRE implementation\n> \n> Change the USE_LIBPCRE=YesPlease & --with-libpcre flags to the\n> Makefile & configure script, respectively, to mean use PCRE v2, not\n> PCRE v1.\n> \n> The legacy library previously available via those options is still\n> available on request via USE_LIBPCRE1=YesPlease or\n> --with-libpcre1. The existing USE_LIBPCRE2=YesPlease & --with-libpcre2\n> still explicitly ask for v2.\n> \n> The v2 PCRE is stable & end-user compatible, all this change does is\n> change the default. Someone building a new git is likely to also have\n> packaged PCRE v2 sometime in the last 2 years since it was released.\n> If not they can choose to use the legacy v2 library by making the\n> trivial s/USE_LIBPCRE/USE_LIBPCRE1/ change, or package up PCRE v2.\n> \n> New releases of PCRE v2 are already faster than PCRE v1, and not only\n> is all significant development is happening on v2, but bugs in\n> reported against v1 have started getting WONTFIX'd asking users to\n> just upgrade to v2.\n> \n> So it makes sense to give our downstream distributors a nudge to\n> switch over to it.\n\nI see where you are coming from.\n\nAt this point, I feel that someone should recall into our collective\nmemory what happened when we made a change similar in nature that broke\nexisting build setups: by requiring REG_STARTEND all of a sudden (\"you can\neasily flip the NO_REGEX switch\", as if everybody should know about those\nMakefile flags we have).\n\nI hate to be that someone, but it has to be said: this is a disruptive\nchange, and it would be a lot better to make it an opt-in at first, and\nwhen the dust settled about this option and many distributions have opted\nin already because of the benefits and tested this thoroughly in practice,\nthen we can think about making this an opt-out option, potentially putting\nsome serious thought into trying to make it painless to upgrade for\nusers casually compiling Git (and not, woohoo, being subscribed to the Git\nmailing list, let alone knowing about all those Makefile knobs).\n\nIn short: it would be a mistake to force PCRE v2 support on by default.\n\nCiao,\nDscho"},{"id":"318643","messageId":"CACsJy8C1vaNFint17-ASy=RQnQswEJf-goQ-wiRCFdkeh+punw@mail.gmail.com","threadId":"45840","inReplyTo":"alpine.DEB.2.20.1705031139090.3480@virtualbox","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2017-05-03T11:15:55Z","receivedAt":"2017-05-03T11:16:31Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, May 3, 2017 at 4:45 PM, Johannes Schindelin\n wrote:\n>> So it makes sense to give our downstream distributors a nudge to\n>> switch over to it.\n\nSome contributor (i.e. me) was not happy with this nudging though. The\nother day I switched to some branch (probably 'pu') and the build\nfailed because, guess what, I didn't have pcre2 installed. If I set\nUSE_LIBPCRE1 then I lose pcre support when switching to other\nbranches. And no, I don't want to install libpcre2, not when I'm\nforced to this way.\n\n188 packages on Gentoo optionally depend on libpcre, 6 packages on\nlibpcre2. Chances that a Gentoo user has libpcre2 already are rather\nlow. I'll revisit my installation when the level of libpcre2 support\ngrows a bit more than that. You can nudge distributors directly,\nprobably more efficient too.\n\n> ...\n>\n> I hate to be that someone, but it has to be said: this is a disruptive\n> change, and it would be a lot better to make it an opt-in at first, and\n> when the dust settled about this option and many distributions have opted\n> in already because of the benefits and tested this thoroughly in practice,\n\nAgreed.\n-- \nDuy\n"},{"id":"318656","messageId":"CACBZZX6_5krLp93PmsW639-N4f1efUT5rPnN+5im=d9-66=QbQ@mail.gmail.com","threadId":"45840","inReplyTo":"alpine.DEB.2.20.1705031139090.3480@virtualbox","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-03T15:10:33Z","receivedAt":"2017-05-03T15:10:59Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"[Just replying to you & Duy in the same mail, easier]\n\nOn Wed, May 3, 2017 at 11:45 AM, Johannes Schindelin\n wrote:\n> Hi Ævar,\n>\n> On Tue, 2 May 2017, Ævar Arnfjörð Bjarmason wrote:\n>\n>> On Tue, May 2, 2017 at 6:05 PM, Johannes Schindelin\n>> wrote:\n>>\n>> > [... complaint about the abrupt change that makes Git for Windows fail\n>> > to build because pcre2.h is missing ...]\n>>\n>> I think it's worth it to copy/paste the commit message where I made\n>> this change, since we're having this discussion in this thread:\n>>\n>> Makefile & configure: make PCRE v2 the default PCRE implementation\n>>\n>> Change the USE_LIBPCRE=YesPlease & --with-libpcre flags to the\n>> Makefile & configure script, respectively, to mean use PCRE v2, not\n>> PCRE v1.\n>>\n>> The legacy library previously available via those options is still\n>> available on request via USE_LIBPCRE1=YesPlease or\n>> --with-libpcre1. The existing USE_LIBPCRE2=YesPlease & --with-libpcre2\n>> still explicitly ask for v2.\n>>\n>> The v2 PCRE is stable & end-user compatible, all this change does is\n>> change the default. Someone building a new git is likely to also have\n>> packaged PCRE v2 sometime in the last 2 years since it was released.\n>> If not they can choose to use the legacy v2 library by making the\n>> trivial s/USE_LIBPCRE/USE_LIBPCRE1/ change, or package up PCRE v2.\n>>\n>> New releases of PCRE v2 are already faster than PCRE v1, and not only\n>> is all significant development is happening on v2, but bugs in\n>> reported against v1 have started getting WONTFIX'd asking users to\n>> just upgrade to v2.\n>>\n>> So it makes sense to give our downstream distributors a nudge to\n>> switch over to it.\n>\n> I see where you are coming from.\n>\n> At this point, I feel that someone should recall into our collective\n> memory what happened when we made a change similar in nature that broke\n> existing build setups: by requiring REG_STARTEND all of a sudden (\"you can\n> easily flip the NO_REGEX switch\", as if everybody should know about those\n> Makefile flags we have).\n\nAnd as a result grep/log -G got faster by default, and more\nimportantly since v2.10.1 which includes your 2f8952250a and made a\nREG_STARTEND engine a hard requirement nobody using git is\nmysteriously going to miss grep results because of some stray \\0 in\nthe string being matched.\n\nI agree that I should drop the patch to make v2 the default from this\nseries for now. Clearly it's controversial, and can be considered on\nits own merits once the supporting code is in. I'll do that in the\nnext submission, which I'm planning to send after v2.13.0 comes out.\n\nBut aside from that, I must say I completely disagree with this line\nof argument. I think to the extent that we did anything wrong with\nNO_REGEX it's that I should have noticed that we could use\nREG_STARTEND after I imported gawk's engine in compat/regex &\nsubmitted a patch like yours back in 2010. Instead we got 6 years of\ngit releases where\n\nSimilarly, if PCRE v2 is less buggy & faster then we're looking at man\ndays/weeks and possibly months/years saved given how many people use\ngit v.s. how many are developing it & write build scripts for it.\n\nLeaving user benefits on the table to optimize for reducing the\none-time inconvenience of packagers is the wrong move.\n\n> [...]\nOn Wed, May 3, 2017 at 1:15 PM, Duy Nguyen wrote:\n> On Wed, May 3, 2017 at 4:45 PM, Johannes Schindelin\n> wrote:\n>>> So it makes sense to give our downstream distributors a nudge to\n>>> switch over to it.\n>\n> Some contributor (i.e. me) was not happy with this nudging though. The\n> other day I switched to some branch (probably 'pu') and the build\n> failed because, guess what, I didn't have pcre2 installed. If I set\n> USE_LIBPCRE1 then I lose pcre support when switching to other\n> branches. And no, I don't want to install libpcre2, not when I'm\n> forced to this way.\n\nAssuming that we'd want to change the default to v2, can you think of\na better way to handle this transition? As pointed out in\n I\ncan't.\n\nPerhaps the best way would be to migrate this change directly to\nmaster if there's consensus to move to v2 by default, that would avoid\nthe pain for developers related to switching between concurrent\nbranches while the change is cooking.\n\n> 188 packages on Gentoo optionally depend on libpcre, 6 packages on\n> libpcre2. Chances that a Gentoo user has libpcre2 already are rather\n> low.\n\nIt sounds like two things are being conflated here, surely \"a Gentoo\nuser\", i.e. someone who's not actively contributing to git.git will\njust get this via `emerge git` if this change were released, since the\nebuild will depend on libpcre2?\n\n> I'll revisit my installation when the level of libpcre2 support\n> grows a bit more than that.\n\nThere being few existing reverse dependencies seems like a strange\nreason not to run `sudo emerge libpcre2`, but whatever, your dev env,\nand we won't be breaking v1 support any time soon.\n\n> You can nudge distributors directly, probably more efficient too.\n\nMy assumption in making this change this way is that there's a\ndecreasing amount of git users who'll get a change if:\n\n* 1. It's made mandatory to compile git (100%)\n* 2. It's made a default flag that must be turned off (e.g. our HTTP\nsupport via curl)\n* 3. It's something users actually notice being missing, e.g. PCRE\nsupport in general via -P\n* 4. It's the default mode of some opt-in flag, e.g. this change where\nUSE_LIBPCRE=YesPlease gives you v2, not v1\n* 5. it's advertised prominently in release notes / the installation\nguide / on-list.\n\nYour suggestion is some variant of doing #5, whereas I'm currently\ntrying to aim for #4 (and in the future I think we should probably go\nfor #2 for PCRE, but that's another discussion).\n\nYou can E-Mail some distributors directly, but there's a big long tail\nof git builders who maintain build scripts for their company, e.g. I\nmaintain one of those. If it's just advertised as #5 I'd probably miss\nit unless I was paying close attention, particularly since there's no\n#3 since I'd be building with v1 already.\n\nWhich is why I think if we're going to nudge distributors to move to\nv2 it makes sense to do so as this patch is doing it, i.e. if they're\nasking for PCRE already make the build fail until they declare\nexplicitly whether they'd like v1 or v2.\n"},{"id":"318734","messageId":"alpine.DEB.2.21.1.1705041104070.4905@virtualbox","threadId":"45840","inReplyTo":"CACBZZX6_5krLp93PmsW639-N4f1efUT5rPnN+5im=d9-66=QbQ@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-04T09:11:56Z","receivedAt":"2017-05-04T09:12:29Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Wed, 3 May 2017, Ævar Arnfjörð Bjarmason wrote:\n\n> [Just replying to you & Duy in the same mail, easier]\n\nIt makes it harder on everybody else, though, as two slightly different\ndiscussion points are conflated now. Also, no single online mail archive\nwill be able to render the thread correctly (assuming that you edited in\nthe In-Reply-To header to loop back to Duy's mail).\n\n> On Wed, May 3, 2017 at 11:45 AM, Johannes Schindelin\n> wrote:\n> >\n> > At this point, I feel that someone should recall into our collective\n> > memory what happened when we made a change similar in nature that\n> > broke existing build setups: by requiring REG_STARTEND all of a sudden\n> > (\"you can easily flip the NO_REGEX switch\", as if everybody should\n> > know about those Makefile flags we have).\n> \n> And as a result grep/log -G got faster by default,\n\nSure. For those developers where the build was not broken.\n\nSoftware maintenance is always a trade-off, and with software as popular\nas Git, maintainers bear a special responsibility to *not* break builds\neasily, as it is more likely than not that anybody who wants to build Git\nis *unfamiliar* with the specifics.\n\nThat is the main reason why we have a configure, even if we try hard to\nmake things work with a straight `make`: people who are happily oblivious\nof our discussions on this here high-volume mailing list will be able to\nbuild Git without even consulting the documentation, just by doing what\nthey would do with any Unix-based software: ./configure && make\n\n> and more importantly since v2.10.1 which includes your 2f8952250a and\n> made a REG_STARTEND engine a hard requirement nobody using git is\n> mysteriously going to miss grep results because of some stray \\0 in the\n> string being matched.\n\nThat is a misinterpretation of what the REG_STARTEND flag is supposed to\ndo. In *some* implementations, REG_STARTEND allows NULs in the haystack.\nSome other implementations do not allow that. It is ill-defined.\n\n> I agree that I should drop the patch to make v2 the default from this\n> series for now. Clearly it's controversial, and can be considered on\n> its own merits once the supporting code is in. I'll do that in the\n> next submission, which I'm planning to send after v2.13.0 comes out.\n\nGood. I am really glad that we agree that the move to v2 should be a\ntwo-step process, with the uncontroversial \"optionally use PCRE v2 for\nspeed and robustness\" first.\n\nOnce enough users like it (and speaking for myself, once I heard from\nenough users how good it is so that I can justify to set aside enough time\nto support PCRE v2 in MSYS2), you will find it much smoother sailing to go\nto phase 2.\n\nCiao,\nDscho"},{"id":"318753","messageId":"CACBZZX5bPN3vZhE=0TSQNdRvKYuV3635=VCQAppAfcZ_tuGpvg@mail.gmail.com","threadId":"45840","inReplyTo":"alpine.DEB.2.21.1.1705041104070.4905@virtualbox","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-04T10:24:42Z","receivedAt":"2017-05-04T10:25:09Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Thu, May 4, 2017 at 11:11 AM, Johannes Schindelin\n wrote:\n> Hi Ævar,\n>\n> On Wed, 3 May 2017, Ævar Arnfjörð Bjarmason wrote:\n>\n>> [Just replying to you & Duy in the same mail, easier]\n>\n> It makes it harder on everybody else, though, as two slightly different\n> discussion points are conflated now. Also, no single online mail archive\n> will be able to render the thread correctly (assuming that you edited in\n> the In-Reply-To header to loop back to Duy's mail).\n>\n>> On Wed, May 3, 2017 at 11:45 AM, Johannes Schindelin\n>> wrote:\n>> >\n>> > At this point, I feel that someone should recall into our collective\n>> > memory what happened when we made a change similar in nature that\n>> > broke existing build setups: by requiring REG_STARTEND all of a sudden\n>> > (\"you can easily flip the NO_REGEX switch\", as if everybody should\n>> > know about those Makefile flags we have).\n>>\n>> And as a result grep/log -G got faster by default,\n>\n> Sure. For those developers where the build was not broken.\n>\n> Software maintenance is always a trade-off, and with software as popular\n> as Git, maintainers bear a special responsibility to *not* break builds\n> easily, as it is more likely than not that anybody who wants to build Git\n> is *unfamiliar* with the specifics.\n>\n> That is the main reason why we have a configure, even if we try hard to\n> make things work with a straight `make`: people who are happily oblivious\n> of our discussions on this here high-volume mailing list will be able to\n> build Git without even consulting the documentation, just by doing what\n> they would do with any Unix-based software: ./configure && make\n\nAnd they still will. That sort of invocation will not build with\nPCRE[1]. This change to v2 by default only impacts builders who are\nalready going out of their way to customize their build with\nUSE_LIBPCRE=YesPlease.\n\nI don't think your comments make sense in that context. We're not even\ntalking about the advanced user/newbie developer who wants to build\ngit for the first time, we're talking about the user who's already\ninvolved enough to start scouring the Makefile & tweaking the various\nflags there.\n\nChanging the current advice from \"if you want PCRE enable this\" to \"if\nyou want PCRE enable this, we use v2 then, turn this other thing on if\nyou want v1\" is a *very* small imposition on people already deeply\ncustomizing their build to the extent of turning on the non-default\nPCRE in the first place.\n\nYes that's a trade-off, and I'm not denying that some advanced users\nwill notice / be annoyed by that change. I just think the end-user\nbenefit of bringing v2 to the attention of distributors who are doing\nsuch customized builds outweighs that.\n\n>> and more importantly since v2.10.1 which includes your 2f8952250a and\n>> made a REG_STARTEND engine a hard requirement nobody using git is\n>> mysteriously going to miss grep results because of some stray \\0 in the\n>> string being matched.\n>\n> That is a misinterpretation of what the REG_STARTEND flag is supposed to\n> do. In *some* implementations, REG_STARTEND allows NULs in the haystack.\n> Some other implementations do not allow that. It is ill-defined.\n\nWe have a test in t7008-grep-binary.sh that'll fail if REG_STARTEND\ndoesn't behave as I described. Actually I think I got the history a\nbit wrong here, I marked that test as succeeding in commit 7e36de5859\n(\"t/t7008-grep-binary.sh: un-TODO a test that needs REG_STARTEND\",\n2010-08-17).\n\nSo I think since 2010 either distributors without REG_STARTEND have\nbeen enabling NO_REGEX=YesPlease giving users the right behavior with\na \\0 in their haystack, or their tests have been consistently failing\nfor 7 years, hopefully the former is the case.\n\n>> I agree that I should drop the patch to make v2 the default from this\n>> series for now. Clearly it's controversial, and can be considered on\n>> its own merits once the supporting code is in. I'll do that in the\n>> next submission, which I'm planning to send after v2.13.0 comes out.\n>\n> Good. I am really glad that we agree that the move to v2 should be a\n> two-step process, with the uncontroversial \"optionally use PCRE v2 for\n> speed and robustness\" first.\n>\n> Once enough users like it (and speaking for myself, once I heard from\n> enough users how good it is so that I can justify to set aside enough time\n> to support PCRE v2 in MSYS2), you will find it much smoother sailing to go\n> to phase 2.\n\nThe v2 API is not a user-visible change. It'll help performance, and\nover the long term subject users to fewer internal PCRE bugs in v1\nthat'll never get fixed.\n\nThat's pretty much a textbook example of the sort of thing users will\nnever even notice, but which is a good thing to do anyway, users don't\nnotice most of the incremental performance improvements made to git,\nbut they matter to them in the aggregate.\n\nSo I think if your criteria for working on integrating v2 is users\nnoticing it elsewhere and asking you for it you'll likely never switch\nto it. I think that's a bit unfortunate & a strange way to evaluate\nwhether you should turn on incremental performance improvements, but\nwhatever, your build & your users.\n\nRegardless of that, I think it's a different question whether most\nother distributors of git feel the same way. I'd hope that most of\nthem would be happy to take a performance/reliability improvement like\nthat, most of them likely have v2 packaged already so it would be\ntrivial for them, and if not they'd either be happy to package it up\nfor such an improvement, or alternatively change USE_LIBPCRE=Y to\nUSE_LIBPCRE1=Y.\n\n1. Which I think is a bug as noted upthread in\n,\nbut this behavior is what users use/rely on now.\n"},{"id":"318757","messageId":"alpine.DEB.2.21.1.1705041328190.4905@virtualbox","threadId":"45840","inReplyTo":"CACBZZX5bPN3vZhE=0TSQNdRvKYuV3635=VCQAppAfcZ_tuGpvg@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-04T11:32:59Z","receivedAt":"2017-05-04T11:33:10Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Thu, 4 May 2017, Ævar Arnfjörð Bjarmason wrote:\n\n> So I think if your criteria for working on integrating v2 is users\n> noticing it elsewhere and asking you for it you'll likely never switch\n> to it.\n\nSpeaking for myself, my biggest problem with your patch series is to find\nthe time to work on packaging PCRE v2 as an MSYS2 package.\n\nIf you force me to make that time *now*, by forcing the switch to v2 in\n`pu`, you will disimprove my mood. That's all.\n\nIf, however, you are gently with people like me, offering this as an\nopt-in, so that I can play with it when I find some time to build PCRE v2\nin MSYS2's context, I will be pretty happy, and of course will ship Git\nfor Windows with the faster PCRE support as soon as I am satisfied that it\nworks correctly.\n\nThat should make you happy, too, as you will get quite a bit of testers\nthat way. Gently.\n\nCiao,\nDscho"},{"id":"318758","messageId":"CACBZZX7DKOvcCRCHgkxiCOVx4XH=B1C+0=Q_ykjm3Odk-aWEWw@mail.gmail.com","threadId":"45840","inReplyTo":"alpine.DEB.2.21.1.1705041328190.4905@virtualbox","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-04T11:53:27Z","receivedAt":"2017-05-04T11:53:53Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Thu, May 4, 2017 at 1:32 PM, Johannes Schindelin\n wrote:\n> Hi Ævar,\n>\n> On Thu, 4 May 2017, Ævar Arnfjörð Bjarmason wrote:\n>\n>> So I think if your criteria for working on integrating v2 is users\n>> noticing it elsewhere and asking you for it you'll likely never switch\n>> to it.\n>\n> Speaking for myself, my biggest problem with your patch series is to find\n> the time to work on packaging PCRE v2 as an MSYS2 package.\n>\n> If you force me to make that time *now*, by forcing the switch to v2 in\n> `pu`, you will disimprove my mood. That's all.\n>\n> If, however, you are gently with people like me, offering this as an\n> opt-in, so that I can play with it when I find some time to build PCRE v2\n> in MSYS2's context, I will be pretty happy, and of course will ship Git\n> for Windows with the faster PCRE support as soon as I am satisfied that it\n> works correctly.\n>\n> That should make you happy, too, as you will get quite a bit of testers\n> that way. Gently.\n\nSure, all makes sense. As noted I'll keep it from being the default for now.\n\nThe only reason I even replied & kept this thread going now was\nbecause I noticed \"make\" not enabling it, REG_STARTEND making \\0\nsearches work in practice (even if not specified by the NetBSD\nextension) etc.\n\nAll stuff I would surely forget by the time we have the discussion\nagain down the road, so I wanted to get that E-Mail out now for future\nreference.\n"},{"id":"318815","messageId":"4fe7bb75-2d4d-37b2-910c-e63cfdfa5282@ramsayjones.plus.com","threadId":"45840","inReplyTo":"xmqqefw9gmvq.fsf@gitster.mtv.corp.google.com","subject":"Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2017-05-05T01:12:23Z","receivedAt":"2017-05-05T01:12:33Z","isPatch":false,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 01/05/17 06:35, Junio C Hamano wrote:\n> \n> * nd/fopen-errors (2017-04-23) 17 commits\n> - warn_failure_to_open_read_optional_path(): a new helper\n> - t1308: add a test case on open a config directory\n> - config.c: handle error on failing to fopen()\n> - xdiff-interface.c: report errno on failure to stat() or fopen()\n> - wt-status.c: report error on failure to fopen()\n> - server-info: report error on failure to fopen()\n> - sequencer.c: report error on failure to fopen()\n> - rerere.c: report correct errno\n> - rerere.c: report error on failure to fopen()\n> - remote.c: report error on failure to fopen()\n> - commit.c: report error on failure to fopen() the graft file\n> - log: report errno on failure to fopen() a file\n> - clone: use xfopen() instead of fopen()\n> - blame: report error on open if graft_file is a directory\n> - bisect: report on fopen() error\n> - config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD\n> - git_fopen: fix a sparse 'not declared' warning\n> \n> We often try to open a file for reading whose existence is\n> optional, and silently ignore errors from open/fopen; report such\n> errors if they are not due to missing files.\n> \n> Expecting a reroll that would be much simplified thanks to a higher\n> level helper.\n\nSince this \"What's cooking\", this has been re-rolled, but the\n\"git_fopen: fix a sparse 'not declared' warning\" patch has been\ndropped, so I'm seeing the sparse warning again.\n\nAlso, this series removes every external caller of the\n'warn_on_inaccessible()' function, so this could be marked\nstatic in wrapper.c. (I would move the definition of that\nfunction above/before warn_on_fopen_errors(), in order not\nto require a forward declaration).\n\nATB,\nRamsay Jones\n\n\n"},{"id":"319061","messageId":"CACBZZX75nvfQft-gjUG+YP0Y-e-=Knm3bMeDWHSUF=Juz9Ua5Q@mail.gmail.com","threadId":"45840","inReplyTo":"alpine.DEB.2.21.1.1705041328190.4905@virtualbox","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-08T06:30:56Z","receivedAt":"2017-05-08T06:31:22Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Thu, May 4, 2017 at 1:32 PM, Johannes Schindelin\n wrote:\n> Hi Ævar,\n>\n> On Thu, 4 May 2017, Ævar Arnfjörð Bjarmason wrote:\n>\n>> So I think if your criteria for working on integrating v2 is users\n>> noticing it elsewhere and asking you for it you'll likely never switch\n>> to it.\n>\n> Speaking for myself, my biggest problem with your patch series is to find\n> the time to work on packaging PCRE v2 as an MSYS2 package.\n\nThis won't be in my next PCRE submission, but I have a path locally to\nsimply import PCRE into git.git as compat/pcre2, so it can be compiled\nwith NO_PCRE=Y similar to how NO_REGEX=Y works.\n\nThis will hopefully address your concerns partially, i.e. when you do\nwant to try it out it'll be easier.\n"},{"id":"319066","messageId":"xmqqpofj7rim.fsf@gitster.mtv.corp.google.com","threadId":"45840","inReplyTo":"CACBZZX75nvfQft-gjUG+YP0Y-e-=Knm3bMeDWHSUF=Juz9Ua5Q@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-08T07:10:41Z","receivedAt":"2017-05-08T07:10:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> This won't be in my next PCRE submission, but I have a path locally to\n> simply import PCRE into git.git as compat/pcre2, so it can be compiled\n> with NO_PCRE=Y similar to how NO_REGEX=Y works.\n>\n> This will hopefully address your concerns partially, i.e. when you do\n> want to try it out it'll be easier.\n\nEek, please don't. \n\nUntil pcre2 becomes _so_ stable that all reasonable distros give\nchoice to the end-users to install it easily in a packaged form,\nsuch a \"not a fork, but a copy\" will be a moving target that I'd\nrather not to have in compat/. IOW, our compat/$pkg should be a\nlast resort to help those on distros that are so hard to convince to\ncarry the version/variant of $pkg we would like to use.\n\n\n\n\n"},{"id":"319127","messageId":"20170508233224.udpuuzlygjpsjogt@genre.crustytoothpaste.net","threadId":"45840","inReplyTo":"xmqqpofj7rim.fsf@gitster.mtv.corp.google.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2017-05-08T23:32:24Z","receivedAt":"2017-05-08T23:32:37Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Mon, May 08, 2017 at 04:10:41PM +0900, Junio C Hamano wrote:\n> Ævar Arnfjörð Bjarmason writes:\n> \n> > This won't be in my next PCRE submission, but I have a path locally to\n> > simply import PCRE into git.git as compat/pcre2, so it can be compiled\n> > with NO_PCRE=Y similar to how NO_REGEX=Y works.\n> >\n> > This will hopefully address your concerns partially, i.e. when you do\n> > want to try it out it'll be easier.\n> \n> Eek, please don't.\n> \n> Until pcre2 becomes _so_ stable that all reasonable distros give\n> choice to the end-users to install it easily in a packaged form,\n> such a \"not a fork, but a copy\" will be a moving target that I'd\n> rather not to have in compat/. IOW, our compat/$pkg should be a\n> last resort to help those on distros that are so hard to convince to\n> carry the version/variant of $pkg we would like to use.\n\nPCRE and PCRE2 also tend to have a lot of security updates, so I would\nprefer if we didn't import them into the tree. It is far better for\nusers to use their distro's packages for PCRE, as it means they get\nautomatic security updates even if they're using an old Git.\n\nWe shouldn't consider shipping anything with a remotely frequent history\nof security updates in our tree, since people very frequently run old or\nancient versions of Git.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"319128","messageId":"CACBZZX4G_ThE55Gi53QJt1=9K4jQXqJ3QL8JSGpiSSSDRrKeNA@mail.gmail.com","threadId":"45840","inReplyTo":"20170508233224.udpuuzlygjpsjogt@genre.crustytoothpaste.net","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-09T00:00:18Z","receivedAt":"2017-05-09T00:00:45Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 9, 2017 at 1:32 AM, brian m. carlson\n wrote:\n> On Mon, May 08, 2017 at 04:10:41PM +0900, Junio C Hamano wrote:\n>> Ævar Arnfjörð Bjarmason writes:\n>>\n>> > This won't be in my next PCRE submission, but I have a path locally to\n>> > simply import PCRE into git.git as compat/pcre2, so it can be compiled\n>> > with NO_PCRE=Y similar to how NO_REGEX=Y works.\n>> >\n>> > This will hopefully address your concerns partially, i.e. when you do\n>> > want to try it out it'll be easier.\n>>\n>> Eek, please don't.\n>>\n>> Until pcre2 becomes _so_ stable that all reasonable distros give\n>> choice to the end-users to install it easily in a packaged form,\n>> such a \"not a fork, but a copy\" will be a moving target that I'd\n>> rather not to have in compat/. IOW, our compat/$pkg should be a\n>> last resort to help those on distros that are so hard to convince to\n>> carry the version/variant of $pkg we would like to use.\n\nThe reason I'm looking into this is because the WIP part of my PCRE\nbranch has changes which start to use PCRE as a general matching\nengine in git, even. I.e.:\n\n* git grep -F will be powered by it rather than kwset (which'll be git rm'd)\n* Long standing limitations with \\0s in regexes go away.\n* grep -G and -E will use PCRE via a WIP POSIX -> PCRE pattern\ntranslator (https://bugs.exim.org/show_bug.cgi?id=2106)\n* Perhaps we can remove compat/regex/ entirely & use PCRE via its\nPOSIX emulation mode + pattern translator (we use regcomp/regexec a\nlot for non-grep/log), I'm not sure yet.\n\nI have messy but working code for this in a WIP branch, here's the\nperformance improvement against linux.git:\n\nTest v2.13.0-rc2 HEAD\n---------------------------------------------------------------------------------------\n7820.1: basic grep how.to 0.31(1.20+0.46)\n0.19(0.33+0.55) -38.7%\n7820.2: extended grep how.to 0.31(1.19+0.46)\n0.19(0.33+0.55) -38.7%\n7820.3: perl grep how.to 0.30(1.12+0.46)\n0.19(0.28+0.62) -36.7%\n7820.5: basic grep ^how to 0.31(1.24+0.39)\n0.19(0.32+0.56) -38.7%\n7820.6: extended grep ^how to 0.30(1.18+0.44)\n0.19(0.22+0.66) -36.7%\n7820.7: perl grep ^how to 0.55(2.68+0.41)\n0.19(0.32+0.56) -65.5%\n7820.9: basic grep [how] to 0.47(2.17+0.44)\n0.22(0.39+0.54) -53.2%\n7820.10: extended grep [how] to 0.47(2.21+0.40)\n0.22(0.39+0.55) -53.2%\n7820.11: perl grep [how] to 0.53(2.64+0.39)\n0.21(0.37+0.58) -60.4%\n7820.13: basic grep \\(e.t[^ ]*\\|v.ry\\) rare 0.63(3.16+0.48)\n0.21(0.48+0.53) -66.7%\n7820.14: extended grep (e.t[^ ]*|v.ry) rare 0.64(3.28+0.38)\n0.21(0.45+0.57) -67.2%\n7820.15: perl grep (e.t[^ ]*|v.ry) rare 1.00(5.77+0.37)\n0.21(0.50+0.53) -79.0%\n7820.17: basic grep m\\(ú\\|u\\)ult.b\\(æ\\|y\\)te 0.31(1.23+0.51)\n0.19(0.30+0.58) -38.7%\n7820.18: extended grep m(ú|u)ult.b(æ|y)te 0.32(1.26+0.47)\n0.19(0.27+0.61) -40.6%\n7820.19: perl grep m(ú|u)ult.b(æ|y)te 0.36(1.61+0.37)\n0.19(0.30+0.57) -47.2%\n7821.1: fixed grep int 0.52(1.71+0.64)\n0.43(1.11+0.72) -17.3%\n7821.2: basic grep int 0.54(1.62+0.70)\n0.42(1.14+0.62) -22.2%\n7821.3: extended grep int 0.53(1.67+0.64)\n0.51(1.17+0.62) -3.8%\n7821.4: perl grep int 0.53(1.71+0.59)\n0.72(1.13+0.63) +35.8%\n7821.6: fixed grep -i int 0.58(1.86+0.67)\n0.47(1.32+0.62) -19.0%\n7821.7: basic grep -i int 0.62(1.94+0.61)\n0.57(1.25+0.72) -8.1%\n7821.8: extended grep -i int 0.82(1.86+0.68)\n0.50(1.41+0.56) -39.0%\n7821.9: perl grep -i int 0.70(1.88+0.68)\n0.56(1.25+0.70) -20.0%\n7821.11: fixed grep æ 0.33(1.30+0.43)\n0.19(0.22+0.64) -42.4%\n7821.12: basic grep æ 0.33(1.35+0.38)\n0.18(0.26+0.59) -45.5%\n7821.13: extended grep æ 0.33(1.20+0.52)\n0.18(0.32+0.53) -45.5%\n7821.14: perl grep æ 0.33(1.31+0.40)\n0.18(0.28+0.56) -45.5%\n7821.16: fixed grep -i æ 0.25(0.87+0.50)\n0.18(0.24+0.60) -28.0%\n7821.17: basic grep -i æ 0.26(0.88+0.48)\n0.18(0.24+0.60) -30.8%\n7821.18: extended grep -i æ 0.26(0.92+0.44)\n0.18(0.24+0.61) -30.8%\n7821.19: perl grep -i æ 0.25(0.79+0.45)\n0.19(0.32+0.56) -24.0%\n\nIn case that comes out misformatted it's also available at\nhttps://github.com/avar/git/commit/ee5b2040e2c697e22a73c7b5f07f1b1e591f07e3\n\nI.e. grepping is sped up by 50% and more in many cases, even for -G\nand -E patterns which now get translated internally into PCRE\npatterns.\n\n> PCRE and PCRE2 also tend to have a lot of security updates, so I would\n> prefer if we didn't import them into the tree. It is far better for\n> users to use their distro's packages for PCRE, as it means they get\n> automatic security updates even if they're using an old Git.\n>\n> We shouldn't consider shipping anything with a remotely frequent history\n> of security updates in our tree, since people very frequently run old or\n> ancient versions of Git.\n\nI'm aware of its security record[1], but I wonder what threat model\nyou have in mind here. I'm not aware of any parts of git (except maybe\ngitweb?) where we take regexes from untrusted sources.\n\nI.e. yes there have been DoS's & even some overflow bugs leading code\nexecution in PCRE, but in the context of powering git-grep & git-log\nwith PCRE this falls into the \"stop hitting yourself\" category.\n\n1. https://www.cvedetails.com/vendor/3265/Pcre.html\n"},{"id":"319130","messageId":"20170509003714.ylwn5ezvu5h36kj7@genre.crustytoothpaste.net","threadId":"45840","inReplyTo":"CACBZZX4G_ThE55Gi53QJt1=9K4jQXqJ3QL8JSGpiSSSDRrKeNA@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2017-05-09T00:37:14Z","receivedAt":"2017-05-09T00:37:25Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, May 09, 2017 at 02:00:18AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> On Tue, May 9, 2017 at 1:32 AM, brian m. carlson\n> wrote:\n> > PCRE and PCRE2 also tend to have a lot of security updates, so I would\n> > prefer if we didn't import them into the tree. It is far better for\n> > users to use their distro's packages for PCRE, as it means they get\n> > automatic security updates even if they're using an old Git.\n> >\n> > We shouldn't consider shipping anything with a remotely frequent history\n> > of security updates in our tree, since people very frequently run old or\n> > ancient versions of Git.\n> \n> I'm aware of its security record[1], but I wonder what threat model\n> you have in mind here. I'm not aware of any parts of git (except maybe\n> gitweb?) where we take regexes from untrusted sources.\n> \n> I.e. yes there have been DoS's & even some overflow bugs leading code\n> execution in PCRE, but in the context of powering git-grep & git-log\n> with PCRE this falls into the \"stop hitting yourself\" category.\n\nJust because you don't drive Git with untrusted regexes doesn't mean\nother people don't. It's not a good idea to require a stronger security\nmodel than we absolutely have to, since people can and will violate it.\nThink how devastating Shellshock was even though technically nobody\nshould provide insecure environment variables to the shell.\n\nAnd, yes, gitweb does in fact call git grep. That means that git grep\nmust in fact be secure against untrusted regexes, or you have a remote\ncode execution vulnerability.\n\nFurthermore, at work we distribute Git with all releases of our product.\nWe normally only do non-security updates to the last couple of releases,\nbut we provide security updates to all supported versions. I'm not\ncomfortable shipping the entirety of PCRE or PCRE2 to customers without\nproviding security updates, so you're going to make my job (and my\ncoworkers') a lot harder by shipping it. Please don't.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"319178","messageId":"alpine.DEB.2.21.1.1705091234210.146734@virtualbox","threadId":"45840","inReplyTo":"20170509003714.ylwn5ezvu5h36kj7@genre.crustytoothpaste.net","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-09T10:40:40Z","receivedAt":"2017-05-09T10:41:09Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 9 May 2017, brian m. carlson wrote:\n\n> On Tue, May 09, 2017 at 02:00:18AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> > On Tue, May 9, 2017 at 1:32 AM, brian m. carlson\n> > wrote:\n> > > PCRE and PCRE2 also tend to have a lot of security updates, so I\n> > > would prefer if we didn't import them into the tree. It is far\n> > > better for users to use their distro's packages for PCRE, as it\n> > > means they get automatic security updates even if they're using an\n> > > old Git.\n> > >\n> > > We shouldn't consider shipping anything with a remotely frequent\n> > > history of security updates in our tree, since people very\n> > > frequently run old or ancient versions of Git.\n> > \n> > I'm aware of its security record[1], but I wonder what threat model\n> > you have in mind here. I'm not aware of any parts of git (except maybe\n> > gitweb?) where we take regexes from untrusted sources.\n> > \n> > I.e. yes there have been DoS's & even some overflow bugs leading code\n> > execution in PCRE, but in the context of powering git-grep & git-log\n> > with PCRE this falls into the \"stop hitting yourself\" category.\n> \n> Just because you don't drive Git with untrusted regexes doesn't mean\n> other people don't.\n\nOr other applications.\n\n> It's not a good idea to require a stronger security model than we\n> absolutely have to, since people can and will violate it. Think how\n> devastating Shellshock was even though technically nobody should provide\n> insecure environment variables to the shell.\n> \n> And, yes, gitweb does in fact call git grep. That means that git grep\n> must in fact be secure against untrusted regexes, or you have a remote\n> code execution vulnerability.\n\nAnd not only grep is affected. Think HEAD^{/}. There are plenty of\nsites where you are allowed to specify revs in a freer form than SHA-1s.\n\nHaving said that, I do like the prospect of a faster git grep.\n\nHopefully there will be a way to make use of PCRE that can be switched\noff? Like, a compile-time replacement of the regex API backed by PCRE v2\n*iff* PCRE v2 is used for building?\n\nCiao,\nDscho"},{"id":"319181","messageId":"CACBZZX6V8qbnrZAdhRvPthy5Z91iEG8rrJ=Sf9tdkOt52M9j1Q@mail.gmail.com","threadId":"45840","inReplyTo":"20170509003714.ylwn5ezvu5h36kj7@genre.crustytoothpaste.net","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-09T11:12:41Z","receivedAt":"2017-05-09T11:13:08Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 9, 2017 at 2:37 AM, brian m. carlson\n wrote:\n> On Tue, May 09, 2017 at 02:00:18AM +0200, Ævar Arnfjörð Bjarmason wrote:\n>> On Tue, May 9, 2017 at 1:32 AM, brian m. carlson\n>> wrote:\n>> > PCRE and PCRE2 also tend to have a lot of security updates, so I would\n>> > prefer if we didn't import them into the tree. It is far better for\n>> > users to use their distro's packages for PCRE, as it means they get\n>> > automatic security updates even if they're using an old Git.\n>> >\n>> > We shouldn't consider shipping anything with a remotely frequent history\n>> > of security updates in our tree, since people very frequently run old or\n>> > ancient versions of Git.\n>>\n>> I'm aware of its security record[1], but I wonder what threat model\n>> you have in mind here. I'm not aware of any parts of git (except maybe\n>> gitweb?) where we take regexes from untrusted sources.\n>>\n>> I.e. yes there have been DoS's & even some overflow bugs leading code\n>> execution in PCRE, but in the context of powering git-grep & git-log\n>> with PCRE this falls into the \"stop hitting yourself\" category.\n>\n> Just because you don't drive Git with untrusted regexes doesn't mean\n> other people don't. It's not a good idea to require a stronger security\n> model than we absolutely have to, since people can and will violate it.\n> Think how devastating Shellshock was even though technically nobody\n> should provide insecure environment variables to the shell.\n\nYes this is definitely something we should keep in mind. I see my\ncomment could be read as dismissing your concerns, I didn't mean that.\nThis is definitely something to be concerned about.\n\nI was trying to solicit feedback on whether there were any other parts\nof stock git that could take external input as regexes than gitweb,\nI'm not aware of any.\n\nI thought that gitweb wouldn't take regexes by default, but I see now\nthat the 'search' feature is on by default, and that allows you to\ngrep / pickaxe with regexes. Either -E or -F for grep, or\n--pickaxe-regex (which implies -E) for log.\n\n> And, yes, gitweb does in fact call git grep. That means that git grep\n> must in fact be secure against untrusted regexes, or you have a remote\n> code execution vulnerability.\n\nIndeed, but it's not as clear-cut as turning on on PCRE making you vulnerable.\n\n* gitweb is vulnerable to CPU DoS now in its default configuration.\nIt's easy to provide an ERE that ends up slurping up 100% CPU for\nseveral seconds on any non-trivial sized repo, do that in parallel &\nyou have a DoS vector.\n\n* Obviously something that's 2x as fast or more (which my WIP code is)\nmakes this better.\n\n* PCRE tends to be worse at pathological patterns, but this is because\nit has really large limits by default and will try rather insanely\nhard to match your pattern.\n\n -DHEAP_LIMIT=20000000 \\\n -DMATCH_LIMIT=10000000 \\\n -DMATCH_LIMIT_DEPTH=10000000 \\\n -DMAX_NAME_COUNT=10000 \\\n -DMAX_NAME_SIZE=32 \\\n -DPARENS_NEST_LIMIT=250 \\\n\nThis can be reduced dynamically via the API (and the patterns can't\nchange it, except to reduce it). For example on my system 2.11.0\n(before any of my changes) on linux.git:\n\n $ time git grep -E -- '-?-?-?-?-?-?-?-?-?-?-?-----------$' |wc -l\n 12434\n real 0m0.444s\n $ time git grep -P -- '-?-?-?-?-?-?-?-?-?-?-?-----------$' | wc -l\n 12434\n real 1m20.849s\n\nWith my JIT changes:\n\n $ time ~/g/git/git --exec-path=/home/avar/g/git grep -P --\n'-?-?-?-?-?-?-?-?-?-?-?-----------$' | wc -l\n 12434\n real 0m5.334s\n\nHowever for user-supplied patterns we can just turn on really\nconservative settings:\n\n $ time ~/g/git/git --exec-path=/home/avar/g/git grep -P --\n'(*LIMIT_RECURSION=1000)(*LIMIT_MATCH=1000)-?-?-?-?-?-?-?-?-?-?-?-----------$'\n| wc -l\n fatal: pcre2_jit_match failed with error code -47: match limit exceeded\n 0\n real 0m0.021s\n\nWhen I locally compile git with something like this:\n\n- -DMATCH_LIMIT=10000000 \\\n- -DMATCH_LIMIT_DEPTH=10000000 \\\n- -DMATCH_LIMIT_RECURSION=10000000 \\\n- -DMAX_NAME_COUNT=10000 \\\n+ -DMATCH_LIMIT=1000 \\\n+ -DMATCH_LIMIT_DEPTH=1000 \\\n+ -DMATCH_LIMIT_RECURSION=1000 \\\n+ -DMAX_NAME_COUNT=100 \\\n -DMAX_NAME_SIZE=32 \\\n- -DPARENS_NEST_LIMIT=250 \\\n+ -DPARENS_NEST_LIMIT=10 \\\n\nAll tests still pass, and from my own testing I can't find any\nnon-pathological patterns this would break. So we might actually\nconsider turning this down for git by default.\n\n* Much of the rest of the patterns PCRE has CVEs for can similarly be\nmitigated by simply turning various features off.\n\n> Furthermore, at work we distribute Git with all releases of our product.\n> We normally only do non-security updates to the last couple of releases,\n> but we provide security updates to all supported versions. I'm not\n> comfortable shipping the entirety of PCRE or PCRE2 to customers without\n> providing security updates, so you're going to make my job (and my\n> coworkers') a lot harder by shipping it. Please don't.\n\nI'm not going to make your job harder. This WIP patch I have for\nembedding PCRE2 as compat/pcre2 is for use by those who'd want to get\na faster grep, but for whatever reason don't have a handy PCRE v2\npackage already, or wouldn't mind supporting a git that comes bundled\nwith it.\n\nMy comments about \"git rm\"-ing kwset & compat/regex were meant in the\nsense of \"I have a branch where we don't use that anymore, and it's\nfaster\", not that you couldn't compile a git without PCRE for the\nforseeable future.\n\nMy WIP branch does remove much of the non-PCRE code, but that's just\nbecause it was easier to hack up like that and not support a bunch of\nif pcre/else branches, I'll amend that and keep PCRE optional when I\nend up submitting those patches.\n"},{"id":"319183","messageId":"CACBZZX6=Uu-tdSMzkyvFGZvbLohSrrqsw1f-NavECs-saX1L6A@mail.gmail.com","threadId":"45840","inReplyTo":"alpine.DEB.2.21.1.1705091234210.146734@virtualbox","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-09T11:29:57Z","receivedAt":"2017-05-09T11:30:23Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 9, 2017 at 12:40 PM, Johannes Schindelin\n wrote:\n> Hi,\n>\n> On Tue, 9 May 2017, brian m. carlson wrote:\n>\n>> On Tue, May 09, 2017 at 02:00:18AM +0200, Ævar Arnfjörð Bjarmason wrote:\n>> > On Tue, May 9, 2017 at 1:32 AM, brian m. carlson\n>> > wrote:\n>> > > PCRE and PCRE2 also tend to have a lot of security updates, so I\n>> > > would prefer if we didn't import them into the tree. It is far\n>> > > better for users to use their distro's packages for PCRE, as it\n>> > > means they get automatic security updates even if they're using an\n>> > > old Git.\n>> > >\n>> > > We shouldn't consider shipping anything with a remotely frequent\n>> > > history of security updates in our tree, since people very\n>> > > frequently run old or ancient versions of Git.\n>> >\n>> > I'm aware of its security record[1], but I wonder what threat model\n>> > you have in mind here. I'm not aware of any parts of git (except maybe\n>> > gitweb?) where we take regexes from untrusted sources.\n>> >\n>> > I.e. yes there have been DoS's & even some overflow bugs leading code\n>> > execution in PCRE, but in the context of powering git-grep & git-log\n>> > with PCRE this falls into the \"stop hitting yourself\" category.\n>>\n>> Just because you don't drive Git with untrusted regexes doesn't mean\n>> other people don't.\n>\n> Or other applications.\n>\n>> It's not a good idea to require a stronger security model than we\n>> absolutely have to, since people can and will violate it. Think how\n>> devastating Shellshock was even though technically nobody should provide\n>> insecure environment variables to the shell.\n>>\n>> And, yes, gitweb does in fact call git grep. That means that git grep\n>> must in fact be secure against untrusted regexes, or you have a remote\n>> code execution vulnerability.\n>\n> And not only grep is affected. Think HEAD^{/}. There are plenty of\n> sites where you are allowed to specify revs in a freer form than SHA-1s.\n\nThat will still use reg(comp|exec) for the foreseeable future. We have\nplenty of manual use of that all over the place:\n\n $ git grep 'reg(comp|exec)\\(' *.[ch] builtin/*.[ch]\n\nAnd the ^{/rx} feature is powered by the one in sha1_name.c\n\n> Having said that, I do like the prospect of a faster git grep.\n>\n> Hopefully there will be a way to make use of PCRE that can be switched\n> off? Like, a compile-time replacement of the regex API backed by PCRE v2\n> *iff* PCRE v2 is used for building?\n\nYup, see my just-sent\n.\nIt'll be optional for now, as it's been for a while.\n\nAside from that I do think given these numbers it's worth considering\nmaking PCRE a default dependency, and possibly getting rid of stuff\nlike kwset because a) it reduces the many codepaths we have now of\neither doing fixed/basic/extended/pcre into one b) since the numbers\nsuggest pcre can support all of that faster that seems like a sensible\nthing to do.\n\nBut anything like that will be a few patch series's down the road, for\nnow I'm just making it all optional.\n"},{"id":"319202","messageId":"CANgJU+Vc2EC6o9CQ8O_Q52u0Q=3rkQDWafYWs9DzsFACBzX3Lw@mail.gmail.com","threadId":"45840","inReplyTo":"CACBZZX6V8qbnrZAdhRvPthy5Z91iEG8rrJ=Sf9tdkOt52M9j1Q@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"demerphq","fromEmail":"demerphq@gmail.com","sentAt":"2017-05-09T14:22:22Z","receivedAt":"2017-05-09T14:22:32Z","isPatch":false,"sender":{"key":"demerphq@gmail.com","avatar":null},"body":"On 9 May 2017 at 13:12, Ævar Arnfjörð Bjarmason wrote:\n> On Tue, May 9, 2017 at 2:37 AM, brian m. carlson\n> wrote:\n>> On Tue, May 09, 2017 at 02:00:18AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> * gitweb is vulnerable to CPU DoS now in its default configuration.\n> It's easy to provide an ERE that ends up slurping up 100% CPU for\n> several seconds on any non-trivial sized repo, do that in parallel &\n> you have a DoS vector.\n\nDoes one need an ERE? Can't one do that now to many parts of git just\nwith a glob?\n\nYves\n"},{"id":"319205","messageId":"CACBZZX6svPkny9Cy14fdPQNaTV_d0tEUWLvO49fdbwuH=_e9tg@mail.gmail.com","threadId":"45840","inReplyTo":"CANgJU+Vc2EC6o9CQ8O_Q52u0Q=3rkQDWafYWs9DzsFACBzX3Lw@mail.gmail.com","subject":"Re: PCRE v2 compile error, was Re: What's cooking in git.git (May 2017, #01; Mon, 1)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-09T14:46:28Z","receivedAt":"2017-05-09T14:46:54Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 9, 2017 at 4:22 PM, demerphq wrote:\n> On 9 May 2017 at 13:12, Ævar Arnfjörð Bjarmason wrote:\n>> On Tue, May 9, 2017 at 2:37 AM, brian m. carlson\n>> wrote:\n>>> On Tue, May 09, 2017 at 02:00:18AM +0200, Ævar Arnfjörð Bjarmason wrote:\n>> * gitweb is vulnerable to CPU DoS now in its default configuration.\n>> It's easy to provide an ERE that ends up slurping up 100% CPU for\n>> several seconds on any non-trivial sized repo, do that in parallel &\n>> you have a DoS vector.\n>\n> Does one need an ERE? Can't one do that now to many parts of git just\n> with a glob?\n\nin practice I don't think so because:\n\n1) I'm now aware of any place where we expose globbing over the wire.\n\n2) AFAICT for the issue detailed in [1] to trigger you also need a\npathological filename in the repo, e.g. I can't get git-ls-files to go\nquadratic on either git.git or linux.git, whereas it's pretty easy to\ncome up with a really expensive regex since there's more content to\nchoose from when matching file content than filenames.\n\n1. https://public-inbox.org/git/20170424211249.28553-1-avarab@gmail.com/\n"}]}