{"thread":{"id":"40967","subject":"What's cooking in git.git (Dec 2015, #03; Thu, 10)","startedAt":"2015-12-10T22:46:40Z","lastAt":"2015-12-14T19:37:05Z","messageCount":13,"participants":["Junio C Hamano","Stefan Beller","John Keeping","Johannes Sixt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"274250","messageId":"xmqqmvthhqgf.fsf@gitster.mtv.corp.google.com","threadId":"40967","inReplyTo":null,"subject":"What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-12-10T22:46:40Z","receivedAt":"2015-12-10T22:46:40Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Here are the topics that have been cooking.  Commits prefixed with\n'-' are only in 'pu' (proposed updates) while commits prefixed with\n'+' are in 'next'.  The ones marked with '.' do not appear in any of\nthe integration branches, but I am still holding onto them.\n\nAn early preview v2.7.0-rc0 has been tagged.  I notice that quite a\nfew topics have been in \"waiting for review\" state without getting\nanybody helping the review process, leaving them in 'pu'--they will\nnot have enough time to be part of 2.7 at this rate, but perhaps\nthat is unavoidable, as this is historically a slow season.\n\nYou can find the changes described here in the integration branches of the\nrepositories listed at\n\n    http://git-blame.blogspot.com/p/git-public-repositories.html\n\n--------------------------------------------------\n[Graduated to \"master\"]\n\n* bc/object-id (2015-11-20) 12 commits\n  (merged to 'next' on 2015-12-04 at 2d3f5f2)\n + remote: convert functions to struct object_id\n + Remove get_object_hash.\n + Convert struct object to object_id\n + Add several uses of get_object_hash.\n + object: introduce get_object_hash macro.\n + ref_newer: convert to use struct object_id\n + push_refs_with_export: convert to struct object_id\n + get_remote_heads: convert to struct object_id\n + parse_fetch: convert to use struct object_id\n + add_sought_entry_mem: convert to struct object_id\n + Convert struct ref to use object_id.\n + sha1_file: introduce has_object_file helper.\n (this branch is used by bc/format-patch-null-from-line.)\n\n More transition from \"unsigned char[40]\" to \"struct object_id\".\n\n This needed a few merge fixups, but is mostly disentangled from other\n topics.\n\n\n* dt/fsck-verify-pack-error (2015-12-01) 1 commit\n  (merged to 'next' on 2015-12-04 at fbd82d1)\n + verify_pack: do not ignore return value of verification function\n\n The exit code of git-fsck didnot reflect some types of errors found\n in packed objects, which has been corrected.\n\n\n* dt/refs-backend-pre-vtable (2015-11-20) 10 commits\n  (merged to 'next' on 2015-11-24 at 8fd7293)\n + refs: break out ref conflict checks\n + files_log_ref_write: new function\n + initdb: make safe_create_dir public\n + refs: split filesystem-based refs code into a new file\n + refs/refs-internal.h: new header file\n + refname_is_safe(): improve docstring\n + pack_if_possible_fn(): use ref_type() instead of is_per_worktree_ref()\n + copy_msg(): rename to copy_reflog_msg()\n + verify_refname_available(): new function\n + verify_refname_available(): rename function\n (this branch is used by dt/refs-backend-lmdb.)\n\n Code preparation for pluggable ref backends.\n\n\n* ep/ident-with-getaddrinfo (2015-11-28) 1 commit\n  (merged to 'next' on 2015-12-01 at 0775d4c)\n + ident.c: add support for IPv6\n\n A build without NO_IPv6 used to use gethostbyname() when guessing\n user's hostname, instead of getaddrinfo() that is used in other\n codepaths in such a build.\n\n\n* ls/test-must-fail-sigpipe (2015-11-28) 2 commits\n  (merged to 'next' on 2015-12-01 at d374686)\n + add \"ok=sigpipe\" to test_must_fail and use it to fix flaky tests\n + implement test_might_fail using a refactored test_must_fail\n\n Fix some racy client/server tests by treating SIGPIPE the same as a\n normal non-zero exit.\n\n\n* ls/travis-yaml (2015-11-28) 1 commit\n  (merged to 'next' on 2015-12-04 at e7852e1)\n + Add Travis CI support\n\n The necessary infrastructure to build topics using the free Travis\n CI has been added. Developers forking from this topic (and enabling\n Travis) can do their own builds, and we can turn on auto-builds for\n git/git (including build-status for pull requests that people\n open).\n\n\n* nd/doc-check-ref-format-typo (2015-12-04) 1 commit\n  (merged to 'next' on 2015-12-08 at 1ec8a6b)\n + git-check-ref-format.txt: typo, s/avoids/avoid/\n\n\n* rs/status-detached-head-memcmp (2015-11-28) 1 commit\n  (merged to 'next' on 2015-12-04 at 60232db)\n + wt-status: correct and simplify check for detached HEAD\n\n Fix some string-matching corner cases when digging in the reflog for\n \"git status\".\n\n--------------------------------------------------\n[New Topics]\n\n* bc/format-patch-null-from-line (2015-12-07) 2 commits\n - format-patch: add an option to suppress commit hash\n - Introduce a null_oid constant.\n\n \"format-patch\" has learned a new option to zero-out the commit\n object name on the mbox \"From \" line.\n\n The bottom patch needs to be retitled but otherwise OK.  The top\n one names the new option --no-hash that is doubly undesirable.\n\n Waiting for a reroll.\n ($gmane/282111)\n\n\n* sn/null-pointer-arith-in-mark-tree-uninteresting (2015-12-07) 1 commit\n  (merged to 'next' on 2015-12-08 at 91b14e2)\n + revision.c: fix possible null pointer arithmetic\n\n mark_tree_uninteresting() has code to handle the case where it gets\n passed a NULL pointer in its 'tree' parameter, but the function had\n 'object = &tree->object' assignment before checking if tree is\n NULL.  This gives a compiler an excuse to declare that tree will\n never be NULL and apply a wrong optimization.  Avoid it.\n\n Will merge to 'master'.\n\n\n* dt/refs-backend-lmdb (2015-12-04) 16 commits\n - refs: tests for lmdb backend\n - refs: add LMDB refs backend\n - refs: allow ref backend to be set for clone\n - init: allow alternate backends to be set for new repos\n - refs: always handle non-normal refs in files backend\n - refs: move duplicate check to common code\n - refs: make lock generic\n - refs: add method to rename refs\n - refs: add methods to init refs backend and db\n - refs: add method for delete_refs\n - refs: add method for initial ref transaction commit\n - refs: add methods for reflog\n - refs: add do_for_each_per_worktree_ref\n - refs: add methods for the ref iterators\n - refs: add methods for misc ref operations\n - refs: add a backend method structure with transaction functions\n\n Building on top of a few refs-backend preparatory series, LMDB\n based refs backend has been plugged into the system.\n\n Waiting for review.\n\n\n* dw/subtree-split-do-not-drop-merge (2015-12-10) 1 commit\n - contrib/subtree: fix \"subtree split\" skipped-merge bug\n\n The \"split\" subcommand of \"git subtree\" (in contrib/) incorrectly\n skipped merges when it shouldn't, which was corrected.\n\n Waiting for review from 'subtree' folks.\n\n\n* ls/p4-keep-empty-commits (2015-12-10) 1 commit\n - git-p4: add option to keep empty commits\n\n \"git p4\" used to import Perforce CLs that touch only paths outside\n the client spec as empty commits.  It has been corrected to ignore\n them instead, with a new configuration git-p4.keepEmptyCommits as a\n backward compatibility knob.\n\n Will merge to 'next'.\n\n--------------------------------------------------\n[Stalled]\n\n* vl/grep-configurable-threads (2015-12-04) 1 commit\n . grep: add --threads=<num> option and grep.threads configuration\n\n \"git grep\" can now be configured (or told from the command line)\n how many threads to use when searching in the working tree files.\n\n There was some review from Eric.  Also this seems to break a test.\n\n Waiting for a reroll, or at least a response.\n ($gmane/281160)\n\n\n* kf/http-proxy-auth-methods (2015-11-04) 3 commits\n . SQUASH???\n . http: use credential API to handle proxy authentication\n . http: allow selection of proxy authentication method\n\n New http.proxyAuthMethod configuration variable can be used to\n specify what authentication method to use, as a way to work around\n proxies that do not give error response expected by libcurl when\n CURLAUTH_ANY is used.  Also, the codepath for proxy authentication\n has been taught to use credential API to store the authentication\n material in user's keyrings.\n\n I ejected this from pu for the moment, as it conflicts with the\n pt/http-socks-proxy topic. That is now in master, so it can\n be re-rolled on top.\n\n Still being worked on.\n ($gmane/280925)\n\n\n* nd/ita-cleanup (2015-09-06) 6 commits\n - grep: make it clear i-t-a entries are ignored\n - checkout(-index): do not checkout i-t-a entries\n - apply: make sure check_preimage() does not leave empty file on error\n - apply: fix adding new files on i-t-a entries\n - add and use a convenience macro ce_intent_to_add()\n - blame: remove obsolete comment\n\n Paths that have been told the index about with \"add -N\" are not yet\n in the index, but various commands behaved as if they already are.\n\n Some commits need better explanation.\n\n Waiting for a reroll.\n\n\n* mg/httpd-tests-update-for-apache-2.4 (2015-04-08) 2 commits\n - t/lib-git-svn: check same httpd module dirs as lib-httpd\n - t/lib-httpd: load mod_unixd\n\n This is the first two commits in a three-patch series $gmane/266962\n\n Becoming tired of waiting for a reroll.\n with updated log message ($gmane/268061).\n\n\n* wp/sha1-name-negative-match (2015-06-08) 2 commits\n - sha1_name.c: introduce '^{/!-<negative pattern>}' notation\n - test for '!' handling in rev-parse's named commits\n\n Introduce \"branch^{/!-<pattern>}\" notation to name a commit\n reachable from branch that does not match the given pattern.\n\n Becoming tired of waiting for a reroll.\n ($gmane/271213).\n\n\n* ak/format-patch-odir-config (2015-06-19) 1 commit\n - format-patch: introduce format.outputDirectory configuration\n\n Reroll exists but didn't pick it up as it seemed to be still\n collecting review comments.\n\n Becoming tired of waiting for a reroll.\n ($gmane/272180).\n\n\n* jc/diff-b-m (2015-02-23) 5 commits\n . WIPWIP\n . WIP: diff-b-m\n - diffcore-rename: allow easier debugging\n - diffcore-rename.c: add locate_rename_src()\n - diffcore-break: allow debugging\n\n \"git diff -B -M\" produced incorrect patch when the postimage of a\n completely rewritten file is similar to the preimage of a removed\n file; such a resulting file must not be expressed as a rename from\n other place.\n\n The fix in this patch is broken, unfortunately.\n\n--------------------------------------------------\n[Cooking]\n\n* cb/t3404-shellquote (2015-12-04) 1 commit\n  (merged to 'next' on 2015-12-08 at 1a01305)\n + t3404: fix quoting of redirect for some versions of bash\n\n Will merge to 'master'.\n\n\n* nd/clear-gitenv-upon-use-of-alias (2015-12-07) 3 commits\n . git.c: make sure we do not leak GIT_* to alias scripts\n - setup.c: re-fix d95138e (setup: set env $GIT_WORK_TREE when ..\n - git.c: make it clear save_env() is for alias handling only\n\n d95138e6 (setup: set env $GIT_WORK_TREE when work tree is set, like\n $GIT_DIR, 2015-06-26) attempted to work around a glitch in alias\n handling by overwriting GIT_WORK_TREE environment variable to\n affect subprocesses when set_git_work_tree() gets called, which\n resulted in a rather unpleasant regression to \"clone\" and \"init\".\n Try to address the same issue by always restoring the environment\n and respawning the real underlying command when handling alias.\n\n The tip one seems to break tests under \"prove\".\n\n Waiting for review.\n\n\n* sb/doc-submodule-sync-recursive (2015-12-04) 1 commit\n  (merged to 'next' on 2015-12-08 at 7c24ee5)\n + document submodule sync --recursive\n\n Will merge to 'master'.\n\n\n* kn/ref-filter-atom-parsing (2015-12-01) 10 commits\n - ref-filter: introduce objectname_atom_parser()\n - ref-filter: introduce contents_atom_parser()\n - ref-filter: introduce remote_ref_atom_parser()\n - ref-filter: introduce align_atom_parser()\n - strbuf: introduce strbuf_split_str_without_term()\n - ref-filter: introduce color_atom_parser()\n - ref-filter: skip deref specifier in match_atom_name()\n - ref-fitler: bump match_atom() name to the top\n - ref-filter: introduce struct used_atom\n - ref-filter: introduce a parsing function for each atom in valid_atom\n\n Refactoring of ref-filter's format-parsing code, in preparation\n for \"branch --format\" and friends.\n\n This replaces (for now) kn/for-each-ref-remainder, which will be built\n on top.\n\n Waiting for review.\n\n\n* sg/lock-file-commit-error (2015-12-01) 1 commit\n  (merged to 'next' on 2015-12-08 at ef53cd6)\n + Make error message after failing commit_lock_file() less confusing\n\n Cosmetic improvement to lock-file error messages.\n\n Will merge to 'master'.\n\n\n* bb/merge-marker-crlf (2015-11-24) 1 commit\n - merge-file: consider core.crlf when writing merge markers\n\n Write out merge markers using system end-of-line convention.\n\n Waiting for a re-roll to handle gitattributes.\n ($gmane/281701)\n\n\n* dk/gc-more-wo-pack (2015-11-24) 3 commits\n - gc: Clean garbage .bitmap files from pack dir\n - t5304: Add test for .bitmap garbage files\n - prepare_packed_git(): find more garbage\n\n Follow-on to dk/gc-idx-wo-pack topic, to clean up stale\n .bitmap and .keep files.\n\n Waiting for review.\n\n\n* jk/send-email-ssl-errors (2015-11-24) 1 commit\n - send-email: enable SSL level 1 debug output\n\n Improve error reporting when SMTP TLS fails.\n\n Waiting for a reroll.\n ($gmane/281693)\n\n\n* ps/rebase-keep-empty (2015-11-24) 2 commits\n - rebase: fix preserving commits with --keep-empty\n - rebase: test broken behavior with --keep-empty\n\n Keep duplicate commits via rebase --keep-empty.\n\n I'm not sure if I agree with this interpretation of the \"rebase\n --keep-empty\" documentation, but I haven't thought too hard about it.\n Comments welcome.\n\n Waiting for review.\n\n\n* rm/subtree-unwrap-tags (2015-11-24) 1 commit\n - contrib/subtree: unwrap tag refs\n\n Waiting for review from subtree folks.\n\n\n* sg/sh-require-clean-orphan (2015-11-24) 2 commits\n - sh-setup: make require_clean_work_tree() work on orphan branches\n - Add tests for git-sh-setup's require_clean_work_tree()\n\n Allow users of git-sh-setup to handle orphan branch state.\n\n This series takes the conservative route of requiring scripts to opt\n into the looser behavior, at the expense of carrying around a new\n option-flag forever. I'm not sure if we need to do so. Comments\n welcome.\n\n\n* tb/ls-files-eol (2015-11-28) 2 commits\n - convert.c: mark a file-local function static\n - ls-files: Add eol diagnostics\n\n Add options to ls-files to help diagnose end-of-line problems.\n\n This latest round hasn't gotten any review yet.\n\n Waiting for review.\n\n\n* ec/annotate-deleted (2015-11-20) 1 commit\n - annotate: skip checking working tree if a revision is provided\n\n Usability fix for annotate-specific \"<file> <rev>\" syntax with deleted\n files.\n\n Waiting for review.\n\n\n* mc/push-recurse-submodules-config (2015-12-04) 3 commits\n  (merged to 'next' on 2015-12-08 at 1b81967)\n + push: follow the \"last one wins\" convention for --recurse-submodules\n + push: test that --recurse-submodules on command line overrides config\n  (merged to 'next' on 2015-11-24 at 3644d4b)\n + push: add recurseSubmodules config option\n\n Add new config to avoid typing \"--recurse-submodules\" on each push.\n\n Waiting for review from submodule folks.\n\n\n* sb/submodule-parallel-update (2015-11-20) 27 commits\n - clone: allow an explicit argument for parallel submodule clones\n - submodule update: expose parallelism to the user\n - git submodule update: have a dedicated helper for cloning\n - fetching submodules: respect `submodule.jobs` config option\n - submodule-config: introduce parse_generic_submodule_config\n - submodule-config: remove name_and_item_from_var\n - submodule-config: drop check against NULL\n - submodule-config: keep update strategy around\n - run_processes_parallel: delimit intermixed task output\n - Merge branch 'rs/daemon-plug-child-leak' into sb/submodule-parallel-update\n - Merge branch 'sb/submodule-parallel-fetch' into sb/submodule-parallel-update\n  (merged to 'next' on 2015-11-20 at 89fc723)\n + strbuf: update documentation for strbuf_read_once()\n + run-command: remove set_nonblocking()\n  (merged to 'next' on 2015-10-23 at 8f04bbd)\n + run-command: fix missing output from late callbacks\n + test-run-command: increase test coverage\n + test-run-command: test for gracefully aborting\n + run-command: initialize the shutdown flag\n + run-command: clear leftover state from child_process structure\n + run-command: fix early shutdown\n  (merged to 'next' on 2015-10-15 at df63590)\n + submodules: allow parallel fetching, add tests and documentation\n + fetch_populated_submodules: use new parallel job processing\n + run-command: add an asynchronous parallel child processor\n + sigchain: add command to pop all common signals\n + strbuf: add strbuf_read_once to read without blocking\n + xread_nonblock: add functionality to read from fds without blocking\n + xread: poll on non blocking fds\n + submodule.c: write \"Fetching submodule <foo>\" to stderr\n (this branch is tangled with sb/submodule-parallel-fetch.)\n\n Builds on top of the \"fetch --recurse-submodules\" work to introduce\n parallel downloading into multiple submodules for \"submodule update\".\n\n Waiting for sb/submodule-parallel-fetch to stabilize.\n\n It would be the cleanest to rebuild sb/submodule-parallel-fetch on\n top of 2.7.0 once it ships and then build this directly on top;\n that way, we do not have to have merges in this topic that\n distracting (besides, some part of the other topic can be updated\n in-place instead of this follow-up topic tweaking them as past\n mistakes and inflexibilities).\n\n I picked up v4 from the list, but it needs review.\n\n\n* jc/strbuf-gets (2015-10-28) 17 commits\n - test-sha1-array: read command stream with strbuf_gets()\n - grep: read -f file with strbuf_gets()\n - send-pack: read list of refs with strbuf_gets()\n - column: read lines with strbuf_gets()\n - cat-file: read batch stream with strbuf_gets()\n - transport-helper: read helper response with strbuf_gets()\n - clone/sha1_file: read info/alternates with strbuf_gets()\n - remote.c: read $GIT_DIR/remotes/* with strbuf_gets()\n - ident.c: read /etc/mailname with strbuf_gets()\n - rev-parse: read parseopt spec with strbuf_gets()\n - revision: read --stdin with strbuf_gets()\n - hash-object: read --stdin-paths with strbuf_gets()\n - mktree: read textual tree representation with strbuf_gets()\n - update-index: read list of paths with strbuf_gets() under --stdin\n - update-index: read --index-info with strbuf_gets()\n - check-attr, check-ignore, checkout-index: read paths with strbuf_gets()\n - strbuf: add strbuf_gets()\n\n Teach codepaths that communicate with users by reading text files\n to be more lenient to editors that write CRLF-terminated lines.\n Note that this is only about communication with Git, like feeding\n list of object names from the standard input instead of from the\n command line, and does not involve files in the working tree.\n\n Waiting for review.\n\n\n* mh/notes-allow-reading-treeish (2015-10-08) 3 commits\n  (merged to 'next' on 2015-10-23 at 8a697f0)\n + notes: allow treeish expressions as notes ref\n + Merge branch 'jk/notes-dwim-doc' into next\n + Merge branch 'jc/merge-drop-old-syntax' into next\n (this branch uses jc/merge-drop-old-syntax.)\n\n Some \"git notes\" operations, e.g. \"git log --notes=<note>\", should\n be able to read notes from any tree-ish that is shaped like a notes\n tree, but the notes infrastructure required that the argument must\n be a ref under refs/notes/.  Loosen it to require a valid ref only\n when the operation would update the notes (in which case we must\n have a place to store the updated notes tree, iow, a ref).\n\n As the patch was done on top of the 'drop old-syntax from merge',\n this has to wait until that other topic can graduate, unfortunately.\n It can be redone in a way that does not depend on that topic after\n this cycle, though.\n\n Will keep in 'next'.\n\n\n* jc/mailinfo (2015-10-21) 1 commit\n - mailinfo: ignore in-body header that we do not care about\n\n Some people write arbitrary garbage at the beginning of a piece of\n e-mail (or after -- >8 -- scissors -- >8 -- line) in the commit log\n message and expect them to be discarded, even though \"From:\" and\n \"Subject:\" are the only documented in-body headers that you are\n supposed to have there.  Allow some garbage (specifically, what may\n look like RFC2822 headers like \"MIME-Version: ...\") to be there and\n ignore them.\n\n I have a feeling that that this is a step in a wrong direction.\n Comments?\n\n\n* js/am-3-merge-recursive-direct (2015-10-12) 2 commits\n  (merged to 'next' on 2015-10-23 at dc631e5)\n + am: make a direct call to merge_recursive\n + merge_recursive_options: introduce the \"gently\" flag\n\n The merge_recursive_generic() function has been made a bit safer to\n call from inside a process.  \"git am -3\" was taught to make a direct\n call to the function when falling back to three-way merge.\n\n Being able to make a direct call would be good in general, but as a\n performance thing, we would want to see it backed up by numbers.\n\n I haven't gone through the \"gently\" change with fine toothed comb;\n I can see that the change avoids calling die(), but I haven't made\n sure that the program states (e.g. what's in the in-core index) are\n adjusted sensibly when it returns to the caller instead of dying,\n or the codepaths that used to die() are free of resource leaks.\n The original code certainly did not care the program states at the\n point of dying exactly because it knew it is going to exit, but now\n they have to care, and they need to be audited.\n\n Will keep in 'next'.\n\n\n* sg/pretty-more-date-mode-format (2015-10-07) 1 commit\n - pretty: add format specifiers for short and raw date formats\n\n Introduce \"%as\" and \"%aR\" placeholders for \"log --format\" to show\n the author date in the short and raw formats.\n\n I have a feeling that that this is a step in a wrong direction.\n Comments?\n\n\n* kn/for-each-branch-remainder (2015-10-02) 9 commits\n . branch: implement '--format' option\n . branch: use ref-filter printing APIs\n . ref-filter: make %(upstream:track) prints \"[gone]\" for invalid upstreams\n . ref-filter: introduce format_ref_array_item()\n . ref-filter: adopt get_head_description() from branch.c\n . ref-filter: modify \"%(objectname:short)\" to take length\n . ref-filter: add support for %(path) atom\n . ref-filter: implement %(if:equals=<string>) and %(if:notequals=<string>)\n . ref-filter: implement %(if), %(then), and %(else) atoms\n\n More unification among \"branch -l\", \"tag -l\" and \"for-each-ref --format\".\n\n Ejected from pu for now, as a re-roll should come on top of\n kn/ref-filter-atom-parsing.\n\n Expecting a reroll.\n ($gmane/278926)\n\n\n* jk/graph-format-padding (2015-09-14) 1 commit\n - pretty: pass graph width to pretty formatting for use in '%>|(N)'\n\n Redefine the way '%>|(N)' padding and the \"--graph\" option\n interacts.  It has been that the available columns to display the\n log message was measured from the edge of the area the graph ended,\n but with this it becomes the beginning of the entire output.\n\n I have a suspicion that 50% of the users would appreciate this\n change, and the remainder see this break their expectation.  If\n that is the case, we might need to introduce a similar but\n different alignment operator so that this new behaviour is\n available to those who want to use it, without negatively affecting\n existing uses.\n\n Undecided.\n ($gmane/278326)\n\n\n* sb/submodule-parallel-fetch (2015-11-24) 17 commits\n  (merged to 'next' on 2015-12-04 at 2c5ea47)\n + run-command: detect finished children by closed pipe rather than waitpid\n  (merged to 'next' on 2015-11-20 at 89fc723)\n + strbuf: update documentation for strbuf_read_once()\n + run-command: remove set_nonblocking()\n  (merged to 'next' on 2015-10-23 at 8f04bbd)\n + run-command: fix missing output from late callbacks\n + test-run-command: increase test coverage\n + test-run-command: test for gracefully aborting\n + run-command: initialize the shutdown flag\n + run-command: clear leftover state from child_process structure\n + run-command: fix early shutdown\n  (merged to 'next' on 2015-10-15 at df63590)\n + submodules: allow parallel fetching, add tests and documentation\n + fetch_populated_submodules: use new parallel job processing\n + run-command: add an asynchronous parallel child processor\n + sigchain: add command to pop all common signals\n + strbuf: add strbuf_read_once to read without blocking\n + xread_nonblock: add functionality to read from fds without blocking\n + xread: poll on non blocking fds\n + submodule.c: write \"Fetching submodule <foo>\" to stderr\n (this branch is tangled with sb/submodule-parallel-update.)\n\n Add a framework to spawn a group of processes in parallel, and use\n it to run \"git fetch --recurse-submodules\" in parallel.\n\n Waiting for review.\n\n\n* ad/cygwin-wants-rename (2015-08-07) 1 commit\n - config.mak.uname: Cygwin needs OBJECT_CREATION_USES_RENAMES\n\n Will hold.\n ($gmane/275680).\n\n\n* jc/rerere-multi (2015-09-14) 7 commits\n - rerere: do use multiple variants\n - t4200: rerere a merge with two identical conflicts\n - rerere: allow multiple variants to exist\n - rerere: delay the recording of preimage\n - rerere: handle leftover rr-cache/$ID directory and postimage files\n - rerere: scan $GIT_DIR/rr-cache/$ID when instantiating a rerere_id\n - rerere: split conflict ID further\n\n \"git rerere\" can encounter two or more files with the same conflict\n signature that have to be resolved in different ways, but there was\n no way to record these separate resolutions.\n\n\n* jc/merge-drop-old-syntax (2015-04-29) 1 commit\n  (merged to 'next' on 2015-10-07 at 50fed71)\n + merge: drop 'git merge <message> HEAD <commit>' syntax\n (this branch is used by mh/notes-allow-reading-treeish.)\n\n Originally merged to 'next' on 2015-05-28\n\n Stop supporting \"git merge <message> HEAD <commit>\" syntax that has\n been deprecated since October 2007.  It has been reported that\n git-gui still uses the deprecated syntax, which needs to be fixed\n before this final step can proceed.\n\n Will keep in 'next'.\n"},{"id":"274254","messageId":"CAGZ79kaA6Lo1W-SudX6v5styyGrX-igGC7i=u5AYOvFYK0DOGA@mail.gmail.com","threadId":"40967","inReplyTo":"xmqqmvthhqgf.fsf@gitster.mtv.corp.google.com","subject":"Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-12-10T23:25:46Z","receivedAt":"2015-12-10T23:25:46Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Dec 10, 2015 at 2:46 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> * sb/submodule-parallel-update (2015-11-20) 27 commits\n...\n>\n>  Waiting for sb/submodule-parallel-fetch to stabilize.\n>\n>  It would be the cleanest to rebuild sb/submodule-parallel-fetch on\n>  top of 2.7.0 once it ships and then build this directly on top;\n>  that way, we do not have to have merges in this topic that\n>  distracting (besides, some part of the other topic can be updated\n>  in-place instead of this follow-up topic tweaking them as past\n>  mistakes and inflexibilities).\n>\n>  I picked up v4 from the list, but it needs review.\n\n\n\n\n>\n> * sb/submodule-parallel-fetch (2015-11-24) 17 commits\n>   (merged to 'next' on 2015-12-04 at 2c5ea47)\n>  + run-command: detect finished children by closed pipe rather than waitpid\n>   (merged to 'next' on 2015-11-20 at 89fc723)\n>  + strbuf: update documentation for strbuf_read_once()\n>  + run-command: remove set_nonblocking()\n>   (merged to 'next' on 2015-10-23 at 8f04bbd)\n>  + run-command: fix missing output from late callbacks\n>  + test-run-command: increase test coverage\n>  + test-run-command: test for gracefully aborting\n>  + run-command: initialize the shutdown flag\n>  + run-command: clear leftover state from child_process structure\n>  + run-command: fix early shutdown\n>   (merged to 'next' on 2015-10-15 at df63590)\n>  + submodules: allow parallel fetching, add tests and documentation\n>  + fetch_populated_submodules: use new parallel job processing\n>  + run-command: add an asynchronous parallel child processor\n>  + sigchain: add command to pop all common signals\n>  + strbuf: add strbuf_read_once to read without blocking\n>  + xread_nonblock: add functionality to read from fds without blocking\n>  + xread: poll on non blocking fds\n>  + submodule.c: write \"Fetching submodule <foo>\" to stderr\n>  (this branch is tangled with sb/submodule-parallel-update.)\n>\n>  Add a framework to spawn a group of processes in parallel, and use\n>  it to run \"git fetch --recurse-submodules\" in parallel.\n>\n>  Waiting for review.\n\nWhat kind of review do you wait for?\nEach patch had lots of discussion, both submodule people (Jens)\nand Windows people (various) seem to be happy with this.\n\nI assume you plan on merging this after 2.7 settled and then we can\nalso get the above sb/submodule-parallel-update going again.\n"},{"id":"274256","messageId":"xmqqd1udhoa2.fsf@gitster.mtv.corp.google.com","threadId":"40967","inReplyTo":"CAGZ79kaA6Lo1W-SudX6v5styyGrX-igGC7i=u5AYOvFYK0DOGA@mail.gmail.com","subject":"Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-12-10T23:33:41Z","receivedAt":"2015-12-10T23:33:41Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n>> * sb/submodule-parallel-fetch (2015-11-24) 17 commits\n>>   (merged to 'next' on 2015-12-04 at 2c5ea47)\n>>  + run-command: detect finished children by closed pipe rather than waitpid\n>>...\n>>  Waiting for review.\n>\n> What kind of review do you wait for?\n\nThis I think was inherited from previous issues.\n\nThe cleanest response that would move these topics forward is for\narea experts to step up and say \"This has been discussed and Acks\nare given here (cf. $URL)\" ;-)\n\nThanks.\n"},{"id":"274261","messageId":"xmqqvb85g8v9.fsf@gitster.mtv.corp.google.com","threadId":"40967","inReplyTo":"CAGZ79kaA6Lo1W-SudX6v5styyGrX-igGC7i=u5AYOvFYK0DOGA@mail.gmail.com","subject":"Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-12-10T23:51:54Z","receivedAt":"2015-12-10T23:51:54Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n>> * sb/submodule-parallel-fetch (2015-11-24) 17 commits\n>> ...\n>\n> I assume you plan on merging this after 2.7 settled and then we can\n> also get the above sb/submodule-parallel-update going again.\n\nYeah, thanks for reminding me.  I think that would be a good plan\nthat gives us an opportunity to clean up this topic, some parts of\nwhich are have \"an early patch that was too hastily merged to 'next'\nhad to be tweaked by an 'oops' follow-up patch in the topic\"\npattern, e.g. \"make waitpid the secondary and closed pipe the\nprimary way to monitor children\".\n\nThanks.\n"},{"id":"274262","messageId":"CAGZ79kY8M-Xv1s4-s7HnjxZ_X19SR4PsWMtQ3yogqN=vjDiMwQ@mail.gmail.com","threadId":"40967","inReplyTo":"xmqqvb85g8v9.fsf@gitster.mtv.corp.google.com","subject":"Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-12-10T23:55:34Z","receivedAt":"2015-12-10T23:55:34Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Dec 10, 2015 at 3:51 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>>> * sb/submodule-parallel-fetch (2015-11-24) 17 commits\n>>> ...\n>>\n>> I assume you plan on merging this after 2.7 settled and then we can\n>> also get the above sb/submodule-parallel-update going again.\n>\n> Yeah, thanks for reminding me.  I think that would be a good plan\n> that gives us an opportunity to clean up this topic, some parts of\n> which are have \"an early patch that was too hastily merged to 'next'\n> had to be tweaked by an 'oops' follow-up patch in the topic\"\n> pattern, e.g. \"make waitpid the secondary and closed pipe the\n> primary way to monitor children\".\n>\n> Thanks.\n\nThis makes it sound as if you would drop it from next once 2.7 is out,\nexpecting a complete reroll, which does the right thing from the beginning?\nThat was not was I was expecting, but thanks for clarifying.\n"},{"id":"274263","messageId":"20151211001815.GB1581@serenity.lan","threadId":"40967","inReplyTo":"xmqqmvthhqgf.fsf@gitster.mtv.corp.google.com","subject":"Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2015-12-11T00:18:15Z","receivedAt":"2015-12-11T00:18:15Z","isPatch":false,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Thu, Dec 10, 2015 at 02:46:40PM -0800, Junio C Hamano wrote:\n> * jk/send-email-ssl-errors (2015-11-24) 1 commit\n>  - send-email: enable SSL level 1 debug output\n> \n>  Improve error reporting when SMTP TLS fails.\n> \n>  Waiting for a reroll.\n>  ($gmane/281693)\n\nIt looks like this got lost in the noise:\n\nhttp://article.gmane.org/gmane.comp.version-control.git/281975\n"},{"id":"274277","messageId":"xmqqr3isg9zu.fsf@gitster.mtv.corp.google.com","threadId":"40967","inReplyTo":"CAGZ79kY8M-Xv1s4-s7HnjxZ_X19SR4PsWMtQ3yogqN=vjDiMwQ@mail.gmail.com","subject":"Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-12-11T17:39:49Z","receivedAt":"2015-12-11T17:39:49Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> On Thu, Dec 10, 2015 at 3:51 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Stefan Beller <sbeller@google.com> writes:\n>>\n>>>> * sb/submodule-parallel-fetch (2015-11-24) 17 commits\n>>>> ...\n>>>\n>>> I assume you plan on merging this after 2.7 settled and then we can\n>>> also get the above sb/submodule-parallel-update going again.\n>>\n>> Yeah, thanks for reminding me.  I think that would be a good plan\n>> that gives us an opportunity to clean up this topic, some parts of\n>> which are have \"an early patch that was too hastily merged to 'next'\n>> had to be tweaked by an 'oops' follow-up patch in the topic\"\n>> pattern, e.g. \"make waitpid the secondary and closed pipe the\n>> primary way to monitor children\".\n>>\n>> Thanks.\n>\n> This makes it sound as if you would drop it from next once 2.7 is out,\n> expecting a complete reroll, which does the right thing from the beginning?\n\nYes, what is still in 'next' when a new release is made has the\nchance to (re)do the right thing from the beginning, and it also can\nlose merges from other topics in the middle of the topic if they\nhave graduated to 'master'.\n\nA topic that did the right thing from this cycle already, but needs\nto stay in 'next' only because the area it touches is so important\nand deserves more real world testing by those who run 'next', may\nnot have to reroll.\n\nI think two sb/submodule-parallel-* topics fall into the former\ncategory, i.e. ones that can take advantage of the chance to escape\nthe rigidity of 'next' at the release cycle boundary, and I think\nwe should grab that opportunity to clean these series up.\n\nThanks.\n"},{"id":"274278","messageId":"xmqqmvtgg9ue.fsf@gitster.mtv.corp.google.com","threadId":"40967","inReplyTo":"20151211001815.GB1581@serenity.lan","subject":"Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-12-11T17:43:05Z","receivedAt":"2015-12-11T17:43:05Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"John Keeping <john@keeping.me.uk> writes:\n\n> On Thu, Dec 10, 2015 at 02:46:40PM -0800, Junio C Hamano wrote:\n>> * jk/send-email-ssl-errors (2015-11-24) 1 commit\n>>  - send-email: enable SSL level 1 debug output\n>> \n>>  Improve error reporting when SMTP TLS fails.\n>> \n>>  Waiting for a reroll.\n>>  ($gmane/281693)\n>\n> It looks like this got lost in the noise:\n>\n> http://article.gmane.org/gmane.comp.version-control.git/281975\n\nIndeed it was; thanks for a pointer.\n"},{"id":"274294","messageId":"566B4207.8020009@kdbg.org","threadId":"40967","inReplyTo":"CAGZ79kY8M-Xv1s4-s7HnjxZ_X19SR4PsWMtQ3yogqN=vjDiMwQ@mail.gmail.com","subject":"sb/submodule-parallel-fetch, was: Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2015-12-11T21:37:11Z","receivedAt":"2015-12-11T21:37:11Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 11.12.2015 um 00:55 schrieb Stefan Beller:\n> On Thu, Dec 10, 2015 at 3:51 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Stefan Beller <sbeller@google.com> writes:\n>>\n>>>> * sb/submodule-parallel-fetch (2015-11-24) 17 commits\n>>>> ...\n>>>\n>>> I assume you plan on merging this after 2.7 settled and then we can\n>>> also get the above sb/submodule-parallel-update going again.\n>>\n>> Yeah, thanks for reminding me.  I think that would be a good plan\n>> that gives us an opportunity to clean up this topic, some parts of\n>> which are have \"an early patch that was too hastily merged to 'next'\n>> had to be tweaked by an 'oops' follow-up patch in the topic\"\n>> pattern, e.g. \"make waitpid the secondary and closed pipe the\n>> primary way to monitor children\".\n>>\n>> Thanks.\n> \n> This makes it sound as if you would drop it from next once 2.7 is out,\n> expecting a complete reroll, which does the right thing from the beginning?\n> That was not was I was expecting, but thanks for clarifying.\n\nAlso, rebuilding the topic such that it takes the direct route to its\ncurrent state would help bisectability on Windows.\n\nGenerally, I'm already quite satisfied with the state of the\ninfrastructure at the tip of the branch.\n\nNevertheless, I have a few niggles.\n\nThe primary one is that we are using a global variable of type\nstruct parallel_processes to keep track of the processes. Fortunately,\nmost functions already take a pointer (I gather you did anticipate an\nobject oriented use of the functions). The only exception is pp_init():\nIt returns a pointer to the global object, which is then passed around\nto the other functions. This does not conform to our usual style,\nhowever, where the initializer function takes a pointer to the object,\ntoo.\n\nAfter converting pp_init, we can have a nicely object oriented\ncollection of functions and get rid of the global object ... almost.\n\nWe still need a global variable for the signal handler. But since\nsignals and their handlers are a global resource, it is not that bad to\nhave a global variable that is dedicated to signal handling.\n\nAnother small nit is that I found it confusing that the closure\nparameters are arranged differently in the callback functions. Granted,\nin get_next_task one of them is an out parameter, but that is actually\nan argument to move it to the end of the parameter list, IMHO.\n\nOn top of that I found an error or two in the documentation, and I have\na few suggestions for improvements.\n\nAll this is summarized in the patch below.\n\ndiff --git a/run-command.c b/run-command.c\nindex db4d916..f3addb9 100644\n--- a/run-command.c\n+++ b/run-command.c\n@@ -864,7 +864,7 @@ enum child_state {\n \tGIT_CP_WAIT_CLEANUP,\n };\n \n-static struct parallel_processes {\n+struct parallel_processes {\n \tvoid *data;\n \n \tint max_processes;\n@@ -890,7 +890,7 @@ static struct parallel_processes {\n \n \tint output_owner;\n \tstruct strbuf buffered_output; /* of finished children */\n-} parallel_processes_struct;\n+};\n \n static int default_start_failure(struct child_process *cp,\n \t\t\t\t struct strbuf *err,\n@@ -933,23 +933,23 @@ static void kill_children(struct parallel_processes *pp, int signo)\n \t\t\tkill(pp->children[i].process.pid, signo);\n }\n \n+static struct parallel_processes *pp_for_signal;\n+\n static void handle_children_on_signal(int signo)\n {\n-\tstruct parallel_processes *pp = &parallel_processes_struct;\n-\n-\tkill_children(pp, signo);\n+\tkill_children(pp_for_signal, signo);\n \tsigchain_pop(signo);\n \traise(signo);\n }\n \n-static struct parallel_processes *pp_init(int n,\n-\t\t\t\t\t  get_next_task_fn get_next_task,\n-\t\t\t\t\t  start_failure_fn start_failure,\n-\t\t\t\t\t  task_finished_fn task_finished,\n-\t\t\t\t\t  void *data)\n+static void pp_init(struct parallel_processes *pp,\n+\t\t    int n,\n+\t\t    get_next_task_fn get_next_task,\n+\t\t    start_failure_fn start_failure,\n+\t\t    task_finished_fn task_finished,\n+\t\t    void *data)\n {\n \tint i;\n-\tstruct parallel_processes *pp = &parallel_processes_struct;\n \n \tif (n < 1)\n \t\tn = online_cpus();\n@@ -976,8 +976,9 @@ static struct parallel_processes *pp_init(int n,\n \t\tpp->pfd[i].events = POLLIN | POLLHUP;\n \t\tpp->pfd[i].fd = -1;\n \t}\n+\n+\tpp_for_signal = pp;\n \tsigchain_push_common(handle_children_on_signal);\n-\treturn pp;\n }\n \n static void pp_cleanup(struct parallel_processes *pp)\n@@ -1019,10 +1020,10 @@ static int pp_start_one(struct parallel_processes *pp)\n \tif (i == pp->max_processes)\n \t\tdie(\"BUG: bookkeeping is hard\");\n \n-\tcode = pp->get_next_task(&pp->children[i].data,\n-\t\t\t\t &pp->children[i].process,\n+\tcode = pp->get_next_task(&pp->children[i].process,\n \t\t\t\t &pp->children[i].err,\n-\t\t\t\t pp->data);\n+\t\t\t\t pp->data,\n+\t\t\t\t &pp->children[i].data);\n \tif (!code) {\n \t\tstrbuf_addbuf(&pp->buffered_output, &pp->children[i].err);\n \t\tstrbuf_reset(&pp->children[i].err);\n@@ -1153,35 +1154,35 @@ int run_processes_parallel(int n,\n \tint i, code;\n \tint output_timeout = 100;\n \tint spawn_cap = 4;\n-\tstruct parallel_processes *pp;\n+\tstruct parallel_processes pp;\n \n-\tpp = pp_init(n, get_next_task, start_failure, task_finished, pp_cb);\n+\tpp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb);\n \twhile (1) {\n \t\tfor (i = 0;\n-\t\t    i < spawn_cap && !pp->shutdown &&\n-\t\t    pp->nr_processes < pp->max_processes;\n+\t\t    i < spawn_cap && !pp.shutdown &&\n+\t\t    pp.nr_processes < pp.max_processes;\n \t\t    i++) {\n-\t\t\tcode = pp_start_one(pp);\n+\t\t\tcode = pp_start_one(&pp);\n \t\t\tif (!code)\n \t\t\t\tcontinue;\n \t\t\tif (code < 0) {\n-\t\t\t\tpp->shutdown = 1;\n-\t\t\t\tkill_children(pp, -code);\n+\t\t\t\tpp.shutdown = 1;\n+\t\t\t\tkill_children(&pp, -code);\n \t\t\t}\n \t\t\tbreak;\n \t\t}\n-\t\tif (!pp->nr_processes)\n+\t\tif (!pp.nr_processes)\n \t\t\tbreak;\n-\t\tpp_buffer_stderr(pp, output_timeout);\n-\t\tpp_output(pp);\n-\t\tcode = pp_collect_finished(pp);\n+\t\tpp_buffer_stderr(&pp, output_timeout);\n+\t\tpp_output(&pp);\n+\t\tcode = pp_collect_finished(&pp);\n \t\tif (code) {\n-\t\t\tpp->shutdown = 1;\n+\t\t\tpp.shutdown = 1;\n \t\t\tif (code < 0)\n-\t\t\t\tkill_children(pp, -code);\n+\t\t\t\tkill_children(&pp, -code);\n \t\t}\n \t}\n \n-\tpp_cleanup(pp);\n+\tpp_cleanup(&pp);\n \treturn 0;\n }\ndiff --git a/run-command.h b/run-command.h\nindex 414cc81..a4745b0 100644\n--- a/run-command.h\n+++ b/run-command.h\n@@ -126,8 +126,6 @@ int finish_async(struct async *async);\n  * pp_cb is the callback cookie as passed to run_processes_parallel.\n  * You can store a child process specific callback cookie in pp_task_cb.\n  *\n- * You MUST set stdout_to_stderr.\n- *\n  * Even after returning 0 to indicate that there are no more processes,\n  * this function will be called again until there are no more running\n  * child processes.\n@@ -135,12 +133,12 @@ int finish_async(struct async *async);\n  * Return 1 if the next child is ready to run.\n  * Return 0 if there are currently no more tasks to be processed.\n  * To send a signal to other child processes for abortion,\n- * return negative signal code.\n+ * return the negative signal number.\n  */\n-typedef int (*get_next_task_fn)(void **pp_task_cb,\n-\t\t\t\tstruct child_process *cp,\n+typedef int (*get_next_task_fn)(struct child_process *cp,\n \t\t\t\tstruct strbuf *err,\n-\t\t\t\tvoid *pp_cb);\n+\t\t\t\tvoid *pp_cb,\n+\t\t\t\tvoid **pp_task_cb);\n \n /**\n  * This callback is called whenever there are problems starting\n@@ -155,7 +153,7 @@ typedef int (*get_next_task_fn)(void **pp_task_cb,\n  *\n  * Return 0 to continue the parallel processing. To abort return non zero.\n  * To send a signal to other child processes for abortion, return\n- * negative signal code.\n+ * the negative signal number.\n  */\n typedef int (*start_failure_fn)(struct child_process *cp,\n \t\t\t\tstruct strbuf *err,\n@@ -174,7 +172,7 @@ typedef int (*start_failure_fn)(struct child_process *cp,\n  *\n  * Return 0 to continue the parallel processing.  To abort return non zero.\n  * To send a signal to other child processes for abortion, return\n- * negative signal code.\n+ * the negative signal number.\n  */\n typedef int (*task_finished_fn)(int result,\n \t\t\t\tstruct child_process *cp,\n@@ -184,14 +182,14 @@ typedef int (*task_finished_fn)(int result,\n \n /**\n  * Runs up to n processes at the same time. Whenever a process can be\n- * started, the callback `get_next_task` is called to obtain the data\n- * fed to the child process.\n+ * started, the callback get_next_task_fn is called to obtain the data\n+ * required to start another child process.\n  *\n- * The children started via this function run in parallel and their output\n- * to stderr is buffered, while one of the children will directly output\n- * to stderr.\n+ * The children started via this function run in parallel. Their output\n+ * (both stdout and stderr) is routed to stderr in a manner that output\n+ * from different tasks does not interleave.\n  *\n- * If start_failure_fn and return_value_fn are NULL, default handlers\n+ * If start_failure_fn or task_finished_fn are NULL, default handlers\n  * will be used. The default handlers will print an error message on\n  * error without issuing an emergency stop.\n  */\ndiff --git a/submodule.c b/submodule.c\nindex 281bccd..00b7c4a 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -627,8 +627,8 @@ struct submodule_parallel_fetch {\n };\n #define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0}\n \n-static int get_next_submodule(void **task_cb, struct child_process *cp,\n-\t\t\t      struct strbuf *err, void *data)\n+static int get_next_submodule(struct child_process *cp,\n+\t\t\t      struct strbuf *err, void *data, void **task_cb)\n {\n \tint ret = 0;\n \tstruct submodule_parallel_fetch *spf = data;\ndiff --git a/test-run-command.c b/test-run-command.c\nindex b1f04d1..f964507 100644\n--- a/test-run-command.c\n+++ b/test-run-command.c\n@@ -16,10 +16,10 @@\n #include <errno.h>\n \n static int number_callbacks;\n-static int parallel_next(void** task_cb,\n-\t\t\t struct child_process *cp,\n+static int parallel_next(struct child_process *cp,\n \t\t\t struct strbuf *err,\n-\t\t\t void *cb)\n+\t\t\t void *cb,\n+\t\t\t void** task_cb)\n {\n \tstruct child_process *d = cb;\n \tif (number_callbacks >= 4)\n@@ -31,10 +31,10 @@ static int parallel_next(void** task_cb,\n \treturn 1;\n }\n \n-static int no_job(void** task_cb,\n-\t\t  struct child_process *cp,\n+static int no_job(struct child_process *cp,\n \t\t  struct strbuf *err,\n-\t\t  void *cb)\n+\t\t  void *cb,\n+\t\t  void** task_cb)\n {\n \tstrbuf_addf(err, \"no further jobs available\\n\");\n \treturn 0;\n"},{"id":"274298","messageId":"CAGZ79kYOtoWHCzpeHGrCEjTUuKYB3rogfV2dxL_TL5Pcu59RSg@mail.gmail.com","threadId":"40967","inReplyTo":"566B4207.8020009@kdbg.org","subject":"Re: sb/submodule-parallel-fetch, was: Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-12-11T22:52:53Z","receivedAt":"2015-12-11T22:52:53Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Dec 11, 2015 at 1:37 PM, Johannes Sixt <j6t@kdbg.org> wrote:\n>\n> Generally, I'm already quite satisfied with the state of the\n> infrastructure at the tip of the branch.\n>\n\nI was about the rework the patch series.\n\n> Nevertheless, I have a few niggles.\n\nIf you don't mind I'll split your patch into chunks and\nsquash these where appropriate and resend the series\nwith the suggestions included without the intermediate stages\nof non compiling code for Windows.\n\n>\n> The primary one is that we are using a global variable of type\n> struct parallel_processes to keep track of the processes. Fortunately,\n> most functions already take a pointer (I gather you did anticipate an\n> object oriented use of the functions). The only exception is pp_init():\n> It returns a pointer to the global object, which is then passed around\n> to the other functions. This does not conform to our usual style,\n> however, where the initializer function takes a pointer to the object,\n> too.\n\nNoted. I thought about the init function as a constructor,\nsuch as\n\n    foo *pp = new foo();\n\nin C++ would be just syntactic sugar for what I did there.\nI'll adhere to Git style then instead.\n\n>\n> After converting pp_init, we can have a nicely object oriented\n> collection of functions and get rid of the global object ... almost.\n>\n> We still need a global variable for the signal handler. But since\n> signals and their handlers are a global resource, it is not that bad to\n> have a global variable that is dedicated to signal handling.\n>\n> Another small nit is that I found it confusing that the closure\n> parameters are arranged differently in the callback functions. Granted,\n> in get_next_task one of them is an out parameter, but that is actually\n> an argument to move it to the end of the parameter list, IMHO.\n>\n> On top of that I found an error or two in the documentation, and I have\n> a few suggestions for improvements.\n>\n> All this is summarized in the patch below.\n\nThanks for the improvements!\n\nStefan\n"},{"id":"274299","messageId":"xmqqvb84d1p1.fsf@gitster.mtv.corp.google.com","threadId":"40967","inReplyTo":"CAGZ79kYOtoWHCzpeHGrCEjTUuKYB3rogfV2dxL_TL5Pcu59RSg@mail.gmail.com","subject":"Re: sb/submodule-parallel-fetch,","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-12-11T23:07:22Z","receivedAt":"2015-12-11T23:07:22Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> On Fri, Dec 11, 2015 at 1:37 PM, Johannes Sixt <j6t@kdbg.org> wrote:\n>>\n>> Generally, I'm already quite satisfied with the state of the\n>> infrastructure at the tip of the branch.\n>\n> I was about the rework the patch series.\n\nOK.\n\nI think the very early part of the series, up to 8fc3f2ee (sigchain:\nadd command to pop all common signals, 2015-09-30), may be fine\nas-is (i.e. just rebase on top of updated 'master').\n\nThe step after that, i.e. asynch processor, is the most interesting\nand important one.  Since it was written, I think the improvements\nthat we want to be rolled into it from the beginning are:\n\n - do not rely on waitpid(-1);\n\n - no need for set_nonglocking(), squashing a4433fd4a and\n   6f963a895a9 in;\n\n - correct the early-shutdown bug 79f38577 and again in 63ce47e1;\n\n - child_process_clear() in 1c53754a, which probably will become\n   unnecessary if the series is rebuilt on top of updated 'master';\n\n - follow-up fixes and enhancements to the tests in c3a5d11 and\n   74cc04d;\n\n - debugging support in 7eb93e91069 (from the other series that\n   builds on top).\n\nThat would slim down not just the total number of patches, but also\nthe amount of the code that needs to be looked at (as we would not\nadd code only to later remove or fixup).\n"},{"id":"274301","messageId":"566B5E95.9030207@kdbg.org","threadId":"40967","inReplyTo":"CAGZ79kYOtoWHCzpeHGrCEjTUuKYB3rogfV2dxL_TL5Pcu59RSg@mail.gmail.com","subject":"Re: sb/submodule-parallel-fetch, was: Re: What's cooking in git.git (Dec 2015, #03; Thu, 10)","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2015-12-11T23:39:01Z","receivedAt":"2015-12-11T23:39:01Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 11.12.2015 um 23:52 schrieb Stefan Beller:\n> If you don't mind I'll split your patch into chunks and\n> squash these where appropriate\n\nI don't mind at all; please pick what you like.\n\n-- Hannes\n"},{"id":"274400","messageId":"CAGZ79kaD5vtrofHmeWyKEDiNMSuuCfwwHHie8O03PdjQRK_V=A@mail.gmail.com","threadId":"40967","inReplyTo":"xmqqvb84d1p1.fsf@gitster.mtv.corp.google.com","subject":"Re: sb/submodule-parallel-fetch,","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-12-14T19:37:05Z","receivedAt":"2015-12-14T19:37:05Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Dec 11, 2015 at 3:07 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> On Fri, Dec 11, 2015 at 1:37 PM, Johannes Sixt <j6t@kdbg.org> wrote:\n>>>\n>>> Generally, I'm already quite satisfied with the state of the\n>>> infrastructure at the tip of the branch.\n>>\n>> I was about the rework the patch series.\n>\n> OK.\n>\n> I think the very early part of the series, up to 8fc3f2ee (sigchain:\n> add command to pop all common signals, 2015-09-30), may be fine\n> as-is (i.e. just rebase on top of updated 'master').\n\nI did that.\n\n>\n> The step after that, i.e. asynch processor, is the most interesting\n> and important one.  Since it was written, I think the improvements\n> that we want to be rolled into it from the beginning are:\n>\n>  - do not rely on waitpid(-1);\n\nand documented in the commit message why we abandoned that\nimplementation.\n\n>\n>  - no need for set_nonglocking(), squashing a4433fd4a and\n>    6f963a895a9 in;\n\n6f963a895a9 (strbuf: update documentation for strbuf_read_once())\nsquashed into a commit before your anticipated \"very early part\nwhich  may be fine as is\", but no worries.\n\n>  - correct the early-shutdown bug 79f38577 and again in 63ce47e1;\n>\n>  - child_process_clear() in 1c53754a, which probably will become\n>    unnecessary if the series is rebuilt on top of updated 'master';\n\nit did.\n\n>\n>  - follow-up fixes and enhancements to the tests in c3a5d11 and\n>    74cc04d;\n>\n>  - debugging support in 7eb93e91069 (from the other series that\n>    builds on top).\n\nnot just cherry picking that one, but also adapt the tests in this series to\nuse it.\n\n>\n> That would slim down not just the total number of patches, but also\n> the amount of the code that needs to be looked at (as we would not\n> add code only to later remove or fixup).\n"}]}