{"thread":{"id":"56747","subject":"Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20","startedAt":"2021-10-21T11:55:15Z","lastAt":"2021-11-30T10:06:23Z","messageCount":58,"participants":["Johannes Schindelin","Son Luong Ngoc","Konstantin Ryabitsev","Ævar Arnfjörð Bjarmason","Junio C Hamano","Bagas Sanjaya","martin","Jean-Noël Avila","Sergey Organov","Martin","Han-Wen Nienhuys","Philip Oakley","Eric Wong","Jeff King","Taylor Blau","Christian Couder"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"439186","messageId":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":null,"subject":"Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:55:10Z","receivedAt":"2021-10-21T11:55:15Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Team,\n\nwe held our second all-virtual Summit over the past two days. It was the\ntraditional unconference style meeting, with topics being proposed and\nvoted on right before the introduction round. It was really good to see\nthe human faces behind those email addresses.\n\n32 contributors participated, and we spanned the timezones from PST to\nIST. To make that possible, the event took place on two days, from\n1500-1900 UTC, which meant that the attendees from the US West coast had\nto get up really early, while it was past midnight in India at the end.\n\nI would like to thank all participants for accommodating the time, and in\nparticular for creating such a friendly, collaborative atmosphere.\n\nA particular shout-out to Jonathan Nieder, Emily Shaffer and Derrick\nStolee for taking notes. I am going to send out these notes in per-topic\nsubthreads, replying to this mail.\n\nDay 1 topics:\n\n* Crazy (and not so crazy) ideas\n* SHA-256 Updates\n* Server-side merge/rebase: needs and wants?\n* Submodules and how to make them worth using\n* Sparse checkout behavior and plans\n\nDay 2 topics:\n\n* The state of getting a reftable backend working in git.git\n* Documentation (translations, FAQ updates, new user-focused, general\n  improvements, etc.)\n* Let's have public Git chalk talks\n* Increasing diversity & inclusion (transition to `main`, etc)\n* Improving Git UX\n* Improving reviewer quality of life (patchwork, subsystem lists?, etc)\n\nA few topics were left for a later date (maybe as public Git chalk talks):\n\n* Making Git memory-leak free (already landed patches)\n* Scaling Git\n* Scaling ref advertisements\n* Config-based hooks (and getting there via migration ot hook.[ch] lib &\n  \"git hook run\")\n* Make git [clone|fetch] support pre-seeding via downloaded *.bundle files\n\nCiao,\nJohannes\n"},{"id":"439187","messageId":"nycvar.QRO.7.76.6.2110211144490.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] Crazy (and not so crazy) ideas","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:55:21Z","receivedAt":"2021-10-21T11:55:29Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by Elijah Newren. Supporting cast: Johannes \"Dscho\"\nSchindelin, Jonathan Tan, Jonathan \"jrnieder\" Nieder, brian m. carlson,\nJeff \"Peff\" King, Ævar Arnfjörð Bjarmason, Emily Shaffer, CB Bailey,\nTaylor Blau, and Philip Oakley.\n\nNotes:\n\n* sent my idea for rebase merges on-list\n\n* Test suite is slow. Shell scripts and process forking.\n\n   * What if we had a special shell that interpreted the commands in a\n     single process?\n\n   * Even Git commands like rev-parse and hash-object, as long as that’s\n     not the command you’re trying to test\n\n   * Dscho wants to slip in a C-based solution\n\n   * Jonathan tan commented: going back to your custom shell for tests\n     idea, one thing we could do is have a custom command that generates\n     the repo commits that we want (and that saves process spawns and\n     might make the tests simpler too)\n\n      * We could replace several “setup repo” steps with “git fast-import”\n        instead.\n\n   * Dscho measured: 0.5 sec - 30 sec in setup steps. Can use fast-import,\n     or can make a new format that helps us set up the test scenario\n\n   * Elijah: test-lib-functions helpers could be built ins\n\n* Biggest idea: there are a lot of people who version control things via\n  tarballs or .zip files per version. This prevents history from\n  compressing well. Some people check in those compressed files into Git\n  for purposes of history.\n\n   * In particular, .jar files or npm packages. Initial testing showed\n     that you can expand .jar files in a way that creates source-like\n     files.\n\n   * Jonathan Nieder points out that “pristine-tar” exists to do similar\n     ideas: https://joeyh.name/code/pristine-tar/\n\n   * Others use “git archive” for this purpose to mixed success.\n\n   * jars and npm packages compress better if you store them in expanded\n     form instead of compressed form\n\n   * So many tools are used to using the end-archive, so while it’s\n     tempting to have the build system be responsible for this, being able\n     to “git add” the archive and have the right thing happen behind the\n     scenes would be nice for ease of use\n\n   * Goal here isn’t bit-for-bit reproducibility, just semantic\n     reproducibility\n\n   * What about other file formats that use zips, such as LibreOffice?\n\n   * Git Merge 2018: Designers Git-It; A unified design system workflow\n     did something similar, except made the tool understand the “exploded”\n     file view.\n\n   * Jonathan Tan mentions that smudge/clean filters can help, except this\n     is about tree<->blob instead of blob<->blob\n\n   * brian m. carlson mentions “git archive” output isn’t stable across\n     Git versions. Should we have a canonical tar format that provides\n     reproducibility?\n\n   * Peff: tree<->blob filters can get confusing in the\n     tree<->index<->worktree mapping. Possible, but requires careful\n     thought about the details about when each spot\n\n   * Old suggestion of a “blob-tree” type that allows storing a single\n     index entry that corresponds to multiple trees and blobs in the\n     background, possibly.\n\n   * One long-term dream (inspired by Avery Pennarun’s “bup” tool) is to\n     store large binary files in a tree-structured way that can store\n     common regions as deltas, improve random access, parallelized\n     hashing. Involves a consistent way to split the file into stable\n     pieces, like --rsyncable uses (based on a rolling hash being zero).\n\n   * Peff: you can do that at the object model layer or at the storage\n     layer. The latter is less invasive.\n\n   * jrnieder: The benefits of blobtree are greater at the object model\n     layer --- e.g. not having to transmit chunks over the wire that you\n     already have. I think the main obstacle has been that the benefits\n     haven’t been enough to be worth the complexity. If that changes, we\n     can imagine bundling it with some other object format changes, e.g.\n     putting blob sizes in tree objects, and rolling it out as a new\n     object-format.\n\n   * Ævar: can we do this in a simpler manner, without deep technical\n     changes? (Context: was thinking about this in the context of some\n     $id$ questions.) Clean/smudge filters have some significant UX\n     drawbacks. Has experience helping users trying to commit .jar files.\n     Some simple advice saying “maybe you don’t want to commit this file\n     type, here are some ways to expand it to a committable format…” based\n     on patterns such as .gitignore or .gitattributes. We don’t have ways\n     to indicate “this repo uses Git LFS, but you don’t have the plugin.”\n\n* Emily: If I could rewrite the commit object format, I would change some\n  things\n\n   * Allow multiple authors\n\n   * Add a layer of indirection to author name\n\n      * brian has thought about this too: replace name with email address\n        + some ssh key or something and use something mailmap-like to map\n        it. Could be a backward-compatible approach\n\n   * CB has been thinking about these problems in the background. Could\n     randomly generate an identifier when you commit your first patch, an\n     @example.com address to avoid conflicting with any real address.\n     Mailmap can be a blob maintained by the project\n\n      * In the process can get first-class multiple authors\n\n      * If I have this id representing this particular pair of authors,\n        can update what the id points to\n\n      * Cool stuff but gets complicated\n\n   * Just getting mailmap applied to trailers in “git log” would be huge\n\n      * CB: main reason I don’t put myself in mailmap is that it’s not\n        worth bothering without that feature\n\n      * Ævar: “git log --author” would want the mapping, too. (and ‘git\n        shortlog --group’) Do we do this only at the presentation layer or\n        if we do it at a lower layer do we get such things for free?\n\n      * If anyone’s interested, I might know where the dragons are hiding,\n        happy to give advice\n\n      * Peff: “git shortlog” already knows how to parse it out so this\n        seems very possible\n\n      * Taylor:\n        https://lore.kernel.org/git/YW8A5FznqLYs7MqH@coredump.intra.peff.net/T/\n\n   * Generation number was discussed ~2011(?)\n\n   * Ævar: does this really need a format change? Two “author” fields\n     would break things, but could have “author” and “x-author” header\n\n   * General principle when changing formats: teasing apart where it’s\n     possible to achieve what you want backward compatibility\n\n* Philip Oakley would like a commit id referring to an unborn branch as a\n  proper id\n\n   * brian: empty tree works for what you’re talking about when you want a\n     diff\n\n   * Philip: motivating example was “first parent is going nowhere, but\n     you have a second parent”\n\n   * jrnieder: I see, you want the --first-parent history of your\n     published branch to match the reflog. As a workaround, you’re able to\n     use an empty initial commit and use --no-ff merges whenever you pull\n     things in, but you’re referring to wishing you didn’t have to make\n     that empty initial commit\n\n   * Ævar: reminds me of the discussion in\n     https://www.fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki of\n     commit/branch relationships\n"},{"id":"439188","messageId":"nycvar.QRO.7.76.6.2110211147250.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] SHA-256 Updates","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:55:58Z","receivedAt":"2021-10-21T11:56:03Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by brian m. carlson. Supporting cast: Jonathan\n\"jrnieder\" Nieder, Derrick Stolee, Johannes \"Dscho\" Schindelin, Toon\nClaes, and Ævar Arnfjörð Bjarmason.\n\nNotes:\n\n 1.  Summarizing where we are with what merged:\n\n     1. We have full SHA256 support \\o/\n\n     2. Some minor glitches, updated the docs to reflect that\n\n     3. It works. 2.30 is a good state.\n\n     4. None of the major forges support it yet but that will come\n\n 2.  Interop between SHA256 repositories and SHA1 repositories\n\n     1. Take each object we receive over the wire or create locally and give it\n        a sha1 value as well\n\n     2. We have a giant loose object index that maps sha256-id and sha1-id\n        values. Hashmap\n\n     3. Will be changed to some tree to allow prefix mapping\n\n     4. index-pack has to take two passes over the pack, because you can’t map\n        a commit before you’ve mapped the tree it points to (or more generally\n        can’t map an object before the objects it references)\n\n     5. Fortunately blobs don’t point to any other objects so this is\n        relatively quick\n\n 3.  Submodules are tricky\n\n     1. They come from a different repository so we don’t have anything to map\n        to\n\n     2. What I’m currently doing is requiring that the submodule be present\n        locally and storing the mapping separately in the superproject\n\n     3. The mapping isn’t sent over the wire. That could create some complexity\n        around malicious histories\n\n 4.  For the same reason we don’t have partial clone working\n\n     1. Might require an on-disk format bump\n\n     2. jrnieder: taking a step back, the hash verifies the full history via\n        the Merkle tree property.\n\n     3. However, with partial clone we already relax this: it is no longer\n        verifiable, locally.\n\n     4. Therefore, we place a lot of trust on the server.\n\n     5. The server could tell us more information about the edge commits, e.g.\n        SHA-1<->SHA-256 mapping\n\n     6. Stolee: if I am sha256 client, that’s what I want, you kind of decide\n        up front what you want\n\n     7. jrnieder: at $DAYJOB common partial clone scenario is triangular\n        workflow\n\n     8. Stolee: how likely are the multiple hosts not homogenous (all SHA-1, or\n        all SHA-256)?\n\n     9. brian: Valuable to be able to work in SHA256 and refer in input+output\n        to SHA-1. If someone refers to a SHA-1, you still want to be able to\n        see what they’re referring to, to interact with other people, even\n        though SHA-1 is insecure\n\n 5.  Multi-pack index: doesn’t work, but won’t be hard to fix\n\n 6.  We write signatures for both objects. When you “git commit --gpg-sign”, it\n     can sign in both formats\n\n     1. Verifies in current format\n\n 7.  Timeframe for hosting providers moving to SHA256\n\n     1. Dscho: should we have a multi provider meeting and coordinate that?\n        Could be everyone waiting for others\n\n     2. brian: cgit supports SHA256 already, allows self-hosting\n\n     3. jrnieder: with interop, individuals can use SHA256 against servers that\n        only support SHA-1. Then that creates pressure for the servers to\n        support SHA256 for performance reasons\n\n     4. brian: interop doesn’t exist yet. If GitHub decides I work on that for\n        the next two months, I think I could do it. But requires the code\n        getting written.\n\n     5. Toon: we at gitlab have sha256 on our radar, but with a very low prio\n        https://gitlab.com/groups/gitlab-org/-/epics/794\n\n 8.  jrnieder: Signing: very old Git versions won’t know to invalidate them\n     when I commit --amend. How old is “very old”?\n\n     1. brian: somewhere between 2.20 and 2.28. In 2.20 started treating\n        everything with “gpgsig” at start as a potential signature.\n\n     2. There were a couple of bugs I fixed in 2.30, working on signature\n        interoperability. Tested with sha256.\n\n     3. Updated the transition plan: in tags, the trailing signature is always\n        the current signature, other ones go in the header.\n\n 9.  Updating other hosting provider glue to support sha256\n\n     1. jrnieder: e.g. GitHub API, UIs, …. Is it hard, similar to the Git part,\n        or a little easier?\n\n     2. brian: hardest part is libgit2. Lots of hardcoded oids in its testsuite\n\n     3. Libraries tend to be the hardest piece --- e.g. Gerrit will need JGit\n        updates\n\n     4. Dscho: gitk also has some references to hardcoded 40-length\n\n     5. Ævar: some patches on the mailing list for gitk and git-gui to adapt\n        them, from Carlos\n\n     6. brian: hopefully the ecosystem learns from this experience and doesn’t\n        just hardcode 64 here :)\n\n 10. Interop code only supports 2 algorithms. Hopefully finish this transition\n     before we need the next one :)\n"},{"id":"439189","messageId":"nycvar.QRO.7.76.6.2110211147490.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] Server-side merge/rebase: needs and wants?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:56:06Z","receivedAt":"2021-10-21T11:56:12Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by Elijah Newren. Supporting cast: Christian Couder,\nJonathan \"jrnieder\" Nieder, brian m. carlson, Toon Claes, Orgad Shaneh,\nJohannes \"Dscho\" Schindelin, Derrick Stolee, Philip Oakley, Jeff \"Peff\"\nKing, CB Bailey, Ævar Arnfjörð Bjarmason, and Phillip Wood.\n\nNotes:\n\n 1.  https://github.com/git/git/pull/1114\n\n 2.  Not about exposing merge over Git protocol, but about providing plumbing\n     for a server to use to run a merge\n\n 3.  Not only merge/rebase, but also cherry-pick, revert\n\n 4.  merge-ORT makes things a bit better, as it doesn’t have some of the\n     problems of the recursive algorithm.\n\n 5.  The challenge is not necessarily the technical challenges, but the UX for\n     server tools that live “above” the git executable.\n\n     1. What kind of output is needed? Machine-readable error messages?\n\n     2. What Git objects must be created: a tree? A commit?\n\n     3. How to handle, report, and store conflicts? Index is not typically\n        available on the server.\n\n 6.  Use case?\n\n     1.  Currently servers use libgit2 to do these algorithms instead of git\n         itself.\n\n     2.  What would it take for us to move the servers off of libgit2 and onto\n         Git?\n\n     3.  This would help with a lot of compatibility issues (sha-256, new data\n         formats)\n\n     4.  Server cares about the exit code to record the success of the\n         operation, including some details around which conflicts happened.\n\n     5.  MUST NOT WRITE A REF in-process (because of replication), so must be\n         at a deep plumbing level.\n\n     6.  How to restart the merge once a user has submitted conflict\n         resolutions?\n\n     7.  Christian: GitLab also uses libgit2, would like to use C Git. Want to\n         not need a worktree for scratch space.\n\n     8.  jrnieder: Write tree with conflict markers and report the conflict\n         (just a boolean), at least as an optional mode (JGit does this and\n         Gerrit relies on it)\n\n         1. Including when merging binary files, rename conflicts, etc where\n            there’s no place to put the conflict markers\n\n     9.  brian: fail-fast mode. Present that a conflict happens very quickly,\n         allow conflict marker computation to be done later, upon user request\n         or as a background job.\n\n     10. Toon: GitLab would love to use merge ORT, and collaborate on it\n\n     11. Orgad: In case you do have conflicts, does a mergetool-style frontend\n         want the three competing versions?\n\n 7.  Dscho: there’s a little-used “git merge-tree” plumbing command\n\n     1. jrnieder: it’s a low-level doesn’t-resolve-conflicts thing, but nothing\n        forces us to keep it that way. Intriguing idea\n\n 8.  Difference between rebase and cherry-pick not all that big, apart from\n     looking at HEAD (which does not make sense on the server-side)\n\n 9.  --onto already strains the concept of the rebase, should maybe not be\n     implicit.\n\n 10. Stolee: Think about future extensibility: e.g., servers might want to\n     support --autosquash\n\n 11. It would be nice to rebase multiple, interconnected branches at the same\n     time. But how to specify that?\n\n 12. Dscho: I have this problem quite often with my many stacked patch series\n\n 13. I use --recreate-merges (uses “label” command), create refs along the way\n\n 14. Philip: I also rebase with merges and then run a script after the fact to\n     update refs\n\n 15. Peff: I do something lower-tech. When I have branches depending on each\n     other, I set the upstream config. By doing rebases in the right order, the\n     right thing happens.\n\n 16. CB: This feature sounds really exciting, often develops parallel,\n     semi-independent changes that only come together in an octopus merge at\n     the end\n\n 17. Jonathan: Newcomers sometimes put commits that don’t belong together on\n     the same branch; I wish there were a smooth way for them to just “drag\n     over” a commit, which we don’t currently have because it involves multiple\n     branches. Cheering you on.\n\n 18. cherry-pick in the middle of an interrupted rebase\n\n 19. If we unify them, then this gets messy\n\n 20. Dscho: I’m a strong proponent of being able to cherry-pick while you’re\n     rebasing. But I’m also missing the ability to do an interactive rebase in\n     the middle of an interactive rebase. I implemented a nested interactive\n     rebase in the tooling for Git for Windows, which works by prepending the\n     current interactive rebase’s todo\n\n 21. Peff: That works in that context, but is not fully generic (no way to\n     --abort / --quit). Would want a stack of operations. I have a command\n     called “git continue” that continues whatever operation is in progress.\n\n 22. Once we have every high-level operation pushing / popping like this, that\n     kind of thing becomes possible.\n\n 23. Toon: I have that too, also “git abort”\n\n 24. CB: “git abort ” is slightly terrifying, we started with git shell and now\n     we have git forth :)\n\n 25. Dscho: could standardize on the git-rebase-todo script and add support for\n     other operations, tricky bit would be how to implemented nested commands\n     in an abortable fashion\n\n 26. Ævar: would be nice if these are pushable/sharable\n\n 27. Is rebase the right top-level command?\n\n 28. Phillip Wood: for refactoring history, would like a different abstraction\n     from rebase\n\n 29. I have a script that does that which works well\n\n 30. jrnieder: https://github.com/arxanas/git-branchless has some non rebase\n     based history manipulation helpers as well, can be useful for inspiration\n\n 31. Elijah: I’m thinking of a “git replay” command\n"},{"id":"439190","messageId":"nycvar.QRO.7.76.6.2110211148060.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] Submodules and how to make them worth using","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:56:13Z","receivedAt":"2021-10-21T11:56:18Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by Emily Shaffer. Supporting cast: brian m. carlson,\nOrgad Shaneh, Jonathan \"jrnieder\" Nieder, Jeff Hostetler, and Philip\nOakley.\n\nNotes:\n\n 1. https://lore.kernel.org/git/YHofmWcIAidkvJiD@google.com/\n\n    1. Internally at Google, a lot of use of “repo”\n\n    2. Isn’t great, but not much alternative available\n\n    3. Submodules are also not great, let’s make them better\n\n    4. Some prior work: --recurse-submodules options\n\n       1. I can run “git branch” with and without --recurse-submodules\n\n    5. Being in recurse mode gives us a chance to be opinionated\n\n    6. Don’t want to have a million options and create a lot of complexity\n\n    7. Branches\n\n       1. Superproject thinks “main” points to one set of states in submodules\n\n       2. Submodules have “main” pointing elsewhere\n\n       3. Which is right? The superproject is right, “git status” can show the\n          difference\n\n    8. Not trying to eliminate all complexity. There is some inherent\n       complexity in stitching repositories together. But I want to make it\n       predictable\n\n    9. For specifics, see the RFC linked to above\n\n 2. brian: Interested in current status, what’s been implemented\n\n    1. Emily: workflow git clone / git branch / git commit / git push, all\n       using submodule.recurse, worked well\n\n    2. Intern Mahi Kolla sent a patch to recurse by default once you’ve done a\n       --recurse-submodules clone\n\n    3. Ran demo for an internal team, feedback was positive\n\n    4. Used a hacky remote helper to map “git push” to “git push origin\n       HEAD:refs/for/main”, we have plans for not needing that :)\n\n    5. Partial clone with submodules is close to done, is another important\n       part of this\n\n    6. Glen and Josh have done some work on branching + setting tracking info.\n       That’s key for making recursive push work in an intuitive way, because\n       the branch you want to push to in each submodule is not always the same\n\n    7. I also pushed a series storing a path in each submodule’s git directory\n       to its superproject’s git directory. Use that as another phase in config\n       parsing, inherited-from-superproject config. That combines well with\n       config-based hooks (thanks Ævar for the help with that)\n\n    8. Next steps are around fast-forward merges and rebases\n\n    9. Specifics are in the doc linked to\n\n 3. Interaction with Gerrit\n\n    1. Orgad: when you push to a submodule and superproject, at merge time the\n       submodule commit changes, what do you do in the superproject to handle\n       this?\n\n    2. jrnieder: This comes up in any review flow, not just Gerrit --- ideally\n       you’d want to review the superproject and submodule changes together as\n       one unit. There’s some work happening in Gerrit on “multi-change\n       review”.\n\n    3. What works today: Gerrit’s submodule subscription feature has the\n       ability to update a superproject. If you have a set of submodule changes\n       and a superproject change that are submitted together, then at submit\n       time Gerrit will rewrite the superproject change to reflect what\n       happened in the submodules.\n\n    4. In the Android workflow the superproject only contains pointers to\n       submodules so we don’t push changes for review to the superproject at\n       all. So we handle this with submodule subscription.\n\n    5. Emily: analogy to auto-generated merge commits\n\n 4. Jeff Hostetler: back in 2014 Microsoft considered submodules, hit a can of\n    worms\n\n    1. Coordinating changes between submodule and superproject, this requires\n       server-side locks to prevent edge cases\n\n    2. Was hard enough that we abandoned it\n\n    3. jrnieder: we’re viewing submodules as not a replacement for the\n       monorepo, but as a separate thing for when components have an\n       independent existence. Microsoft made the right choice by not using\n       submodules artificially in the creation of the Windows monorepo.\n\n 5. Jeff: do you want to support sub-sub-sub-submodules?\n\n    1. Emily: we ruled that out.\n\n    2. jrnieder: nested submodules already work well in Git, we’re not breaking\n       that\n\n       1. Philip Oakley: good; if that changes, please make docs + config clear\n          about it\n\n    3. As a matter of project hygiene, we encourage people to put their\n       submodules in the top-level directly. That way, you know what code\n       you’re pulling in.\n\n    4. That said, there are unusual use cases e.g. around a build that pulls\n       together multiple versions of the full Android codebase. So we actually\n       do take advantage of nested submodules for those niche cases\n\n 6. Please read the design doc, and expect lotsa patches over the next 3-6\n    months\n"},{"id":"439191","messageId":"nycvar.QRO.7.76.6.2110211148230.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] Sparse checkout behavior and plans","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:56:26Z","receivedAt":"2021-10-21T11:56:31Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by Derrick Stolee. Supporting cast: Jonathan\n\"jrnieder\" Nieder, Elijah Newren, Jeff Hostetler, Jeff \"Peff\" King,\nJohannes \"Dscho\" Schindelin, Ævar Arnfjörð Bjarmason, Emily Shaffer,\nVictoria Dye, brian m. carlson, and CB Bailey.\n\nNotes:\n\n 1.  Cone mode has stabilized\n\n 2.  jrnieder: would sparse index without cone mode support be welcome?\n\n     1. Stolee: you’re welcome to try ;-)\n\n     2. Elijah: main theme: performance. Cone mode allows reasonable\n        performance due to fewer rules to check\n\n     3. Stolee: directory-level lookups mean lookups can have sublinear cost,\n        since you can skip sparse rules (no need to check them in order to\n        figure out whether or not a file is excluded or not)\n\n 3.  Elijah: interested in “sparse clones”, i.e. clones that download\n     everything related to a specified cone\n\n     1.  Would be nice not having to download extra objects when already having\n         specified a cone of interest\n\n     2.  Jeff: the original partial clone had code to restrict to a cone\n\n     3.  Peff: we still have the code, but turned it off, you can have bitmaps\n         with that (too heavy on the server)\n\n     4.  Stolee: also, how can the cone be updated if things change? Never\n         solved that problem\n\n     5.  Stolee: but the extra blob downloads turned out not to be too big of a\n         problem\n\n     6.  Stolee: got a feature request to restrict git log to the current cone,\n         git grep already does that (thanks Matheus)\n\n     7.  Elijah: “git grep” without revision arguments is restricted to\n         worktree, so it respects the sparse checkout. When you pass a\n         revision, though, it searches the whole tree\n\n     8.  Many commands want to examine the whole tree, makes sense to figure\n         out the UX (configuration, etc) of them together\n\n     9.  Peff: Is diff code on someone’s radar?\n\n     10. Stolee: I’d view that as part of the same story as “git log”, “git log\n         -p”.\n\n     11. Sparse index means we can avoid faulting in trees outside of HEAD, so\n         it helps unlock this\n\n 4.  Sparse index: Victoria and Lessley are taking lead on the number of\n     commands supporting sparse index\n\n     1. update-index, diff, blame, clean, stash, sparse-checkout itself so far\n        supported only in the Microsoft fork of Git\n\n     2. Enabled by default internally so helps us gather data\n\n     3. Elijah: awesome that you’re working on this, sorry I haven’t been as\n        responsive as I’d like on reviews\n\n     4. I’m interested in “clean” in particular --- isn’t that about untracked\n        files?\n\n     5. Stolee: It uses the index to find what is tracked, want to avoid\n        expanding the in-memory index. If there are files outside the sparse\n        checkout area then it does expand.\n\n 5.  jrnieder: question about failure modes\n\n     1. When I convert a command, I make sure my code path doesn’t assume the\n        cache array contains all entries. Then I turn off\n        command_requires_full_index. What happens if I missed a spot?\n\n     2. Stolee: I put ensure_full_index() in front of everything that assumes a\n        full index, but if there’s a loop that we missed, there’s no extra\n        protection.\n\n     3. Example: cache-tree was calling itself, invalidating points,\n        segfaulted.\n\n     4. More worrying failure mode would be if commands proceed with bad data.\n        Segfaulting is the good case!\n\n     5. jrnieder is not too worried since we’re pretty far along and soon\n        enough we’ll have converted all commands and these questions would be\n        moot\n\n        1. Stolee: goal isn’t to get 100% coverage, so point of questions being\n           moot isn’t coming soon\n\n        2. jrnieder: Thanks! Okay, I’ll take a look.\n\n     6. http://sweng.the-davies.net/Home/rustys-api-design-manifesto\n\n     7. Stolee is less worried because we have sufficient ensure_full_index\n        calls.\n\n 6.  One optimization we’re considering: not expanding the full index when\n     anything outside the cone is needed (we’d like to maybe expand just the\n     part that needs expanding)\n\n     1. Elijah: we would still keep cone mode, but it’s a bit weird because the\n        cone mode does not match what we have in the index\n\n     2. Stolee: we might actually not need this\n\n 7.  Stolee: in the process of this work, found D/F conflict issue, made a test\n     illustrating it\n\n 8.  Elijah: atomicitiy\n\n     1.  checkout is a non-atomic operation. ^C makes a mess\n\n     2.  “git sparse-checkout disable” is non-atomic. Takes a while, people ^C,\n         and the very last step is updating the sparsity files. Leaves the\n         worktree with a bunch of files they don’t need but commands ignore\n         them\n\n     3.  We run into problems because then they can check out a different\n         branch, do a bunch of other work, then update the sparse-checkout and\n         it will see these precious files it doesn’t want to overwrite\n\n     4.  Should “git status” show them?\n\n     5.  Dscho: We could set a flag on disk when you’re about to disable, then\n         if we were interrupted print an error message to get the user to sort\n         things out\n\n     6.  Peff: I was going to suggest something similar. FS doesn’t make\n         transactions easy, but we can at least do a rollback (signal handler),\n         not foolproof, but it works pretty well and covers your ^C case.\n\n     7.  Stolee: coming in 2.34: sparse-checkout reapply will delete ignored\n         (and tracked?) files. Helps with these leftover files.\n\n     8.  Elijah: no current way to get out of that state, thank you for making\n         sparse-checkout reapply do that\n\n     9.  Stolee: noticed during experimental release to people from Office.\n         Everything was slow because they had run build and left behind ignored\n         files\n\n     10. jrnieder: Piggy-backing on Dscho’s comment, there’s a database\n         analogy: record intent (in the database case, that’s a transaction\n         journal) before the non-atomic steps the act on that intent. Suggests\n         maybe we should be updating the sparsity pattern before the checkout\n         step\n\n 9.  That’s it, that’s the status update what’s currently on the list.\n\n 10. We have more plans, though.\n\n 11. Idea: use git.git itself\n\n     1. Tried it, but had to have 97% files to still be workable\n\n     2. Could change the Makefile to accept that, say, po/ is missing\n\n     3. Ævar: creates a lot of complexity for the build\n\n     4. jrnieder: as VCS provider, what is our recommendation to build authors?\n        Do we want them querying sparse checkout, do we want builds that Just\n        Work in cone mode, do we want to treat sparse checkout as a thing that\n        builds don’t need to support?\n\n     5. Stolee: want build system to be able to tell Git about what needs to be\n        checked out. “In-tree sparse checkout” (see below)\n\n 12. Emily: we’re interested in sparse-checkout affecting the set of active\n     submodules, just mentioning this as a heads-up\n\n 13. [PATCH 00/10] [RFC] In-tree sparse-checkout definitions - Derrick Stolee\n     via GitGitGadget\n     (https://lore.kernel.org/git/pull.627.git.1588857462.gitgitgadget@gmail.com/)\n\n 14. Victoria: today when you switch gears and work on something else you have\n     to update the sparse checkout pattern\n\n 15. Proposal here is to have in-tree sparse checkout definitions, e.g. a\n     .gitdependencies file that lists, for the directories you’re working with,\n     what other subdirectories they depend on\n\n 16. That way, you get exactly the folders you need\n\n 17. Stolee: office has their own tool “scoper” that figures out dependencies\n     and runs “git sparse-checkout set” for the user. Is confusing when you\n     rebase and need to remember to run it\n\n 18. Currently lives in a hook, custom and built for one engineering system,\n     want to generalize and make a standard feature\n\n 19. Victoria: being built in to Git would make sense because it’s general\n     enough to work in most monorepo environments.\n\n 20. Involves two pieces: having git understand the dependencies and assemble\n     your sparse checkout cone using them, and having the build system maintain\n     and use sparse checkout correctly.\n\n 21. Some build setups tolerate missing directories reasonably well. If we make\n     .gitdependencies more of a first-class concept then we could go further\n     and make build systems handle missing directories as something that would\n     be expected\n\n 22. C# .proj files link to dependencies on other .proj files with relative\n     path. But in a solution file collecting all .proj files, it lists all of\n     them and you need to have them all present. If a subdirectory isn’t\n     present, proposal is to build what is there instead of everything.\n\n 23. Tried another prototype on how to do this in Bazel. It has a rigorous\n     definition of inputs and outputs, and based on that you could translate to\n     a .gitdependencies file or sparse-checkout pattern.\n\n 24. Microsoft’s buildxl has similar properties\n\n 25. Victoria asks: how general is the above?\n\n 26. brian: Many monorepos has multiple microservices. A cone can represent\n     what a particular service needs to run.\n\n 27. If you’re building one coherent product like Windows, you’re going to need\n     some prebuilt artifacts that you pull down.\n\n 28. jrnieder: Large monorepos often have strong remote build. Not everything\n     you depend on is things that you need to have in source form locally\n\n 29. CB: My team at Bloomberg has a teamwide “monorepo” (not Bloomberg-wide).\n     We’re cmake based. Sparse checkout would be interesting for us. We’re\n     experimenting with what’s called workspace builds: you have a thing you\n     can build (a subdirectory), that you pull into the toplevel CMakeLists.txt\n     as a single thing.\n\n 30. With cmake you can declare a dependency with target_link_libraries. A\n     dependency name can either be a cmake defined target in the codebase\n     you’re building it, or it can be a pre-built library pulled in another\n     way, e.g. importing via a pkg-config file.\n\n 31. At build time if I decide I want to change that library, I’ll expand my\n     sparse-checkout region, and rerun cmake to have it understand the newly\n     available source.\n\n 32. Optionality: I don’t have to have that source checked out, but when it’s\n     present I want to use it.\n\n 33. Victoria: sounds like in-tree sparse checkout is more of an intermediate\n     step. Sometimes you want the source, sometimes you want to pull in an\n     external artifact.\n\n 34. Elijah: we have a monorepo, about the size of the Linux kernel. Multiple\n     separate services, interconnected pieces. Using sparse-checkout required\n     some code changes, refactoring that wasn’t just around the build system.\n     We created a tool before the sparse-checkout command existed, using older\n     mechanisms, and then switched to sparse-checkout when it came out. We\n     track our dependencies ourselves --- you need this set of modules (3 or 4)\n     or the modules relevant to a particular team, and it then computes the\n     relevant directories to get. We had to make some changes to adopt cone\n     mode but I like it and the changes it led to. Then you run the build\n     system --- you have files that declare the dependencies, are they newer\n     than .git/info/sparse-checkout? If not then recompute them again.\n\n 35. Potentially would want to rerun the dependency generation after you run a\n     rebase as well…\n\n 36. If we track it in-tree, there are some interesting cases we’ll run into\n     (merge conflicts on this generated file).\n\n 37. Also, tracking dependencies in two places can result in difficulty, skew.\n     Maybe can generate one from the other.\n\n 38. Our sparse checkout tends to be build oriented “what do I need for this\n     build”. But testing inverts the dependency graph, want to see what tests\n     depend on this code. We encourage them to test in the cloud but not\n     everyone does that, leads fewer people to use sparse checkout.\n\n 39. There’s some remote build, mixing-and-matching pieces built remotely and\n     locally.\n\n 40. Part of working in a monorepo is you need strong tool hygiene enforcement.\n     Without that, you get a ball of mud of dependencies. Adopting sparse\n     checkout drove modularity.\n\n 41. Ævar: I’d be interested in a summary\n\n 42. Git’s lack of support for sparse checkout was unusual, so I think this\n     topic is well explored by previous version control systems\n"},{"id":"439192","messageId":"nycvar.QRO.7.76.6.2110211148400.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] The state of getting a reftable backend working in git.git","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:56:33Z","receivedAt":"2021-10-21T11:56:38Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by Ævar Arnfjörð Bjarmason (on behalf of Han-Wen\nNienhuys, the driving force behind the reftable patches, who did not\nattend the Summit). Supporting cast: Jonathan \"jrnieder\" Nieder, Johannes\n\"Dscho\" Schindelin, Philip Oakley, Jeff \"Peff\" King, and Junio Hamano.\n\nNotes:\n\n 1.  Ævar: helping Han-Wen with reviewing\n\n 2.  Was split into multiple patch series\n\n 3.  Han-Wen implemented the reftable library, has been kicking on the mailing\n     list for over a year\n\n 4.  Before reftable, need to merge some preliminaries\n\n 5.  Odd cases:\n\n     1. slightly different semantics of reflog\n\n     2. Probably more things that haven’t cropped up yet\n\n     3. Some tests are still broken, question is still: are the tests wrong, or\n        the code\n\n 6.  Plan is to get reftable library and underlying fixes in place, and then do\n     the process of actual reftable-as-ref-backend afterward\n\n     1. Jrnieder: sounds like you’re alluding to a mailing list thread. Do you\n        have a link?\n\n     2. Ævar: there are multiple, as the initial patch series has been split\n        multiple times\n\n     3. “Reftable plan”:\n        https://lore.kernel.org/git/87h7jqz7k5.fsf@evledraar.gmail.com/\n\n     4. Also alluded to in Han-Wen’s later rerolls\n\n 7.  What is reftable?\n\n     1. It’s a custom way to store refs\n\n     2. Instead of writing individual files per ref, it’s a single file (or\n        multiple files when updating the refs)\n\n 8.  Dscho: three issues that were outstanding when I reviewed it\n\n     1. That said, I clashed with Han-Wen\n\n     2. 1. licensing / contribution model\n\n        2. Ævar: through the Software Freedom Conservancy we have good access\n           to legal advice. I got advice there about how to document this well,\n           will be getting us into an end state that will hopefully satisfy\n           everybody\n\n        3. To be clear, we already have some code in-tree that is under\n           different licenses. xdiff is LGPL code used by libgit2, there’s\n           contrib/ + compat/ code under various licenses\n\n        4. For legal purposes want to make sure this is clear and unambiguous\n           to everyone\n\n        5. jrnieder: about contribution model, there is on-list discussion\n           about this, taking patches in the normal way to this directory in\n           git@vger.kernel.org is where I thought that ended up\n\n        6. Ævar: yes, git.git as source-of-truth. Not like gitk where there’s a\n           separate upstream repo\n\n     3. 2. coding style consistency, + not using git core data structures\n           enough\n\n        3. Ævar: still substantially true. Integrating into git.git means any\n           stylistic or structural changes to fit well into git are fair game.\n           Carlo has been helping with that\n\n     4. 3. I forgot the third :)\n\n 9.  Philip Oakley: debugging when things go wrong\n\n     1.  When reftable arrives, will people be unable to look behind the scenes\n         at what’s going on when issues happen?\n\n     2.  Especially for people who don’t understand refs as well\n\n     3.  Jonathan: format =\n         www.kernel.org/pub/software/scm/git/docs/technical/reftable.html\n         [http://www.kernel.org/pub/software/scm/git/docs/technical/reftable.html]\n\n     4.  Ævar: That’s a fair summary. It’s as though we didn’t have packfiles\n         and only had loose files and then switched to using packfiles. Can’t\n         just “cat” any more. Switching to a binary format\n\n     5.  That said, you get advantages out of that. Situations where people end\n         up needing to examine the low-level details are\n\n     6.  Not a fully fair comparison, but we have this problem already with\n         packed-refs, having to look in two places\n\n     7.  Philip: An inspection tool to export as a directory tree might be\n         handy, as an inspection tool\n\n     8.  Peff: We have pretty good inspection tools that look at the whole ref\n         database\n\n     9.  Reftable has a set of files that go together. May want debugging tool\n         to dump the content of a binary reftable file. But we can\n         incrementally add those\n\n     10. As we discover bugs, I expect to have to build tooling\n\n     11. Dscho: We also have a .git/index file and don’t have tooling to\n         interact with it other than the standard Git tools\n\n     12. Ævar: To be clear, once these patches land it would still be optional,\n         would not be the default ref backend\n\n     13. Even if it’s 100% bug free, we still have concern for users in the\n         wild that make it not so easy to just flip the switch\n\n     14. Not going to be the default backend any time soon\n\n     15. jrnieder: makes sense to wait for a while to make it the default, even\n         once it is robust, since we have to pay attention to what Git versions\n         + implementations are out there in the wild\n\n     16. Ævar: When you run “git init”, it currently still creates a branches/\n         directory. Dscho tried to get rid of it before\n\n     17. jrnieder: I think that previous attempt was getting rid of read\n         capability, too\n\n     18. Dscho: don’t remember the details, has been a couple years\n\n     19. Junio: I do not think it is a bad idea to drop branches from template.\n\n 10. jrnieder: Question about how to handle this kind of large contribution\n\n 11. At some point does it make sense to take it, mark as experimental, and\n     improve in place?\n\n 12. Hoping the previous discussion will help me think about that\n\n 13. Ævar: I agree about importing the bulk of the code as-is and iterating\n     from there\n\n 14. At that point it’s still not accessible to users but we get portability,\n     testing, etc\n\n 15. Dscho: Agreed, that makes sense to me\n"},{"id":"439193","messageId":"nycvar.QRO.7.76.6.2110211149000.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] Documentation (translations, FAQ updates, new user-focused, general improvements, etc.)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:56:43Z","receivedAt":"2021-10-21T11:56:51Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by brian m. carlson. Supporting cast: Jeff \"Peff\"\nKing, Ævar Arnfjörð Bjarmason, Taylor Blau, Philip Oakley, Emily Shaffer,\nCB Bailey, and Jonathan \"jrnieder\" Nieder.\n\nNotes:\n\n 1. Background: answering on StackOverflow, other avenues for user questions,\n    even users from very large companies\n\n 2. How can we improve documentation?\n\n 3. Maybe even think about translating docs such as FAQs\n\n 4. Peff: there’s an effort to translate manpages\n\n    1. brian: Saw an announcement, haven’t seen what came of it\n\n    2. Peff: Some translated pages are live on git-scm.com (a github repo with\n       translations)\n\n    3. Ævar: It uses a third-party tool (po4a) that uses gettext by making each\n       paragraph a translated string. So it’s the same workflow as translating\n       code changes\n\n    4. Taylor: https://github.com/jnavila/git-manpages-l10n\n\n 5. Philip Oakley: I see manpages used as reference material instead of\n    educational documents\n\n    1.  Audience often already knows what they’re looking up\n\n    2.  That approach makes it harder to bring people in. Examples are of the\n        difficult things instead of how to get started, workable examples that\n        can be copy/pasted straight into the shell and tell you how things go\n\n    3.  Emily: We have the two-part Git tutorial (“git help tutorial”) which is\n        part of manpages, but I think it’s pretty dated. It starts with how to\n        convert your zipfile-based software distribution to Git which is not\n        where most people start these days\n\n    4.  Philip: user manual also is not accessible as part of manual\n\n    5.  CB: I wonder if this is even where people look. A lot of new users will\n        hit Google and find git-scm.com/book which historically has been a very\n        good introduction\n\n    6.  Slightly misleading calling it Pro Git because it has good\n        introductions\n\n    7.  Philip: maybe the Git project wants to state: we don’t make great\n        documentation, look elsewhere\n\n    8.  jrnieder: thank you for the perspective. It’s not quite the intent,\n        though, we might just not do a good enough job. For example, when\n        examples are too complex, that’s worth improving\n\n    9.  Used to have active contributors who maintained documentation better\n        (e.g., Jon Loeliger)\n\n    10. A part of the problem is the format. Pro Git can include diagrams, the\n        Git user manual can’t (or at least doesn’t)\n\n    11. brian: likes Pro Git, but maybe not the best for new folks (it assumes\n        some familiarity with source code management)\n\n    12. In stackoverflow you can see how people answer questions, how much less\n        existing background they assume\n\n    13. Ævar: One issue with the Pro Git book is that it is not under a free\n        software license (though it is free of charge). That means it can’t be\n        included in free software distributions.\n\n    14. I want to close the gap between output we emit and providing backlinks\n        to relevant documentation. E.g. sometimes when we emit advice output,\n        we say what config variable is involved and sometimes we don’t\n\n    15. Having documentation distributed with Git is also helpful for having\n        something that’s up to date and matches the code people are using\n\n    16. Philip Oakley: Google Season of Docs is a place we can help\n\n    17. brian: Mining stackoverflow has been very helpful for FAQs, helps avoid\n        having to give the same answer again and again\n\n    18. Goal is to have a good FAQ in git/git, to be able to link to from\n        StackOverflow\n\n    19. Perl approach of including references in error messages is very useful\n        for people being able to solve their own problems\n\n    20. Ævar: “git help git” landing page is not so helpful. I’d prefer\n        something like the perl manpage that gives an overview and table of\n        contents and nothing else, instead of incorporating reference\n        documentation about common options\n\n    21. brian: I’d like to see both in the toplevel manpage. “How to invoke\n        git” is something people expect to see when they run “man git”\n\n    22. Ævar: agreed about synopsis, as long as it focuses on the commonly used\n        options\n\n    23. Peff: every time I want to look up perl commandline options, I run “man\n        perl”, get annoyed, and then run “man perlrun”. I think “man git” does\n        the things you’re describing but organized poorly. Even “git help”\n        output does a better job of organizing. I also wouldn’t be sad to see\n        the options section coming after.\n\n    24. CB: dashed commands should not be listed\n\n 6. Emily: Side topic: the state of git help on stackoverflow is abysmal\n\n    1. Doesn’t have much Git project presence, devrel teams focus on\n       company-specific things instead of Git basics.\n\n    2. A lot of answers are just wrong\n\n    3. Someone spending some 20% time on that could improve things a lot and in\n       the process would see where people are struggling, which can help us\n       make Git more intuitive and make better intuitive tutorial documentation\n\n    4. Ævar: Having a commonly cited FAQ used in stackoverflow can be great\n\n 7. Philip: there are commands that are (at least almost) undocumented, e.g.\n    git rerere\n\n    1. brian: Have seen occasions where people struggled with commands like\n       this\n\n    2. Ævar: have seen undocumented patches\n\n    3. Tried to improve documentation e.g. git fetch --prune, sometimes\n       phrasing is too concise to be helpful\n\n    4. Emily: getting confused e.g. when notes are transported via operations\n       such as rebase\n\n    5. Philip: may go in hand with the lack of good examples\n\n 8. Lots of good ideas for contributions!\n"},{"id":"439194","messageId":"nycvar.QRO.7.76.6.2110211149530.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] Increasing diversity & inclusion (transition to `main`, etc)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:56:59Z","receivedAt":"2021-10-21T11:57:03Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by Johannes \"Dscho\" Schindelin. Supporting cast:\nbrian \"Bmc\" carlson, Jeff \"Peff\" King, Taylor Blau, CB Bailey, Ævar\nArnfjörð \"Avarab\" Bjarmason, Jonathan \"Jrnieder\" Nieder, Derrick Stolee,\nLessley Dennington, Glen Choo, Philip Oakley, Victoria Dye, and Jonathan\n\"Jonathantanmy\" Tan.\n\nNotes:\n\n 1.  Dscho: Background. If you look into archives, I myself have made a change\n     in communication style. Used to do a lot of judgmental code reviews, but\n     noticed that interactions which are respectful and collaborative can be\n     much more productive and enjoyable. I learned this from management help at\n     $dayjob; it helps a lot to strengthen the team & product. Two people with\n     different points of view working together make a better result than one\n     person\n\n 2.  Lately noticing more places Git can improve, for example default branch\n     name ‘master’, which has huge impact on a lot of population, esp. USA.\n     ‘main’ is a very good alternative adopted by most hosters (GitLab, GitHub,\n     BitBucket, etc)\n\n 3.  There is still work to do in documentation, other places. Some of these\n     changes can be wide-reaching so dscho trying to contribute them in a less\n     painful way; soon hopefully we can switch the default without this option\n     workaround. A lot of work, but the impact is worth it\n\n 4.  Still, a tiny first step - let’s go further, both with Git the tool and\n     Git the community/list. Easy to forget there is someone on the other end\n     of the email address who could feel discouraged\n\n 5.  Git the community is still overwhelmingly male, white - what can we do?\n\n 6.  Bmc: the note about conversational tone is important. I’ve heard from\n     others, especially women that a confrontational review style is a turnoff;\n     that’s true for me too. No longer interested in a 20-email debate about\n     something. Let’s look for a more collaborative approach on list; when I\n     point this out I don’t see a bunch of help backing me up, usually.\n\n 7.  Peff: I like that bmc points out when folks aren’t behaving well; I often\n     consciously don’t jump in to support you, because a pile-on is a bad\n     experience for someone who’s usually ok, and if it’s someone who’s truly\n     awful, I don’t want to feed the troll - they’re prone to escalate instead.\n     Understandable that you could wonder on your end how your pushback is\n     being received - we should be better about showing support.\n\n 8.  Bmc: the goal is to make the list more positive; if I should do something\n     differently please tell me\n\n 9.  Taylor: I’ve had conversations off-list with people who are speaking up\n     on-list. Stolee: +1, I reply (not reply-all) to say “yes thank you”\n\n 10. Dscho: Firstly, bmc, you’re a great example and I appreciate that you push\n     back when you do. When I see just bmc’s reply and nobody else, it seems\n     the offender is feeling validated and continues behaving that way (because\n     nobody agreed with bmc publicly).\n\n 11. Dscho: maybe we could be quicker to ask for a video chat when someone is\n     being harsh on list. Often when I point it out the quick response is “well\n     I thought it was fine” and escalates.\n\n 12. CB: I would be intimidated being invited into a private video call with\n     someone on the mailing list when I’m a brand new contributor. I have\n     experience helping moderate the #include<c++> discord which tries harder\n     to moderate aggressively and make inclusive environment. Moderation takes\n     a lot of load, but it might be a better alternative than instant dogpiling\n     from all the senior Gitters. Maybe a happy medium to coordinate off-list\n     first before everybody jumps on someone communicating rudely.\n\n 13. Avarab: there are some instructions in the CoC about\n     enforcement/escalation. We tend to take these reports off-list, so\n     something does happen but it’s not so visible. Usually these resolve\n     happily, but the list is left hanging, so it’s easy to get the impression\n     that nothing happened. At the same time, there are negatives to making the\n     whole thing public.\n\n 14. Philip: often common words mean different things between nationalities,\n     this can get in the way\n\n 15. Dscho: Thank you for taking on the task to be on the escalation list, it’s\n     hard. It can be really soul-draining, e.g. the GfW “how can we make Git\n     more inclusion” issue. I had to stay away from the GfW fork entirely\n     because I was so tired from trolling on that issue. From my point of view,\n     the CoC is there to protect folks with less standing/representation. I\n     thought for sure the CoC would never be invoked by a white male, but we\n     saw it happen\n\n 16. Dscho: for this session, hoping to find strategies to turn the tone around\n     on list and avoid issues from the beginning, throughout the\n     project/community. Recently read Nonviolent Communication, has tips to\n     turn around the conversation even if it started poorly\n\n 17. Jrnieder: are you saying someone in a position of power should never make\n     use of the CoC? Dscho: I figured it was not there for me, it was there for\n     people who don’t have the privilege I do. Jrnieder: A few things - CoC\n     sets expectations regardless of who interactions are directed to;\n     somewhere unwelcoming to established contributors but welcoming to newbies\n     is still not an appealing place for some newcomers to invest in joining\n     because they know they will one day be an established contributor.\n     Secondly, if I have a dispute, having a guide for turning a potentially\n     problematic event into a productive event is really important. A process\n     that moves in that direction of nonviolent communication. Easier said than\n     done. But that’s part of making a friendly environment, and overall that\n     points to not treating the CoC as “only for some people”.\n\n 18. Bmc: the goal of the CoC is to produce and preserve the community we want\n     to have. It should produce a place where everybody can participate fully;\n     if we have an environment that’s unproductive or toxic we lose\n     contributors. I’ve left projects over a poor contributor experience\n     before, because it wasn’t worth my time to deal with the overhead.\n     Conversely, having a great and safe experience is a good way to attract\n     diverse contributor base.\n\n 19. Stolee: When we moved from MS to GH, we received quick feedback that we\n     weren’t communicating well - too direct and unemotional. Maybe Git\n     community communicates that way, but that’s not how most people interact;\n     that makes me think that our “efficient and effective” communication is\n     actually too aggressive, and easily interpreted as attacks on\n     contributors. Basically… let’s all lighten up? :)\n\n 20. Taylor: Yep, my “talking to GitHubbers at GitHub” voice is different from\n     my “talking to Gitters on Git list” voice. New contributors, are we on the\n     right track here?\n\n 21. Lessley: I made first contribution recently, and had been warned about the\n     list, papers about the Linux kernel, and that open source contribution\n     could be a little contentious. I was really nervous and put it off, but in\n     practice it was fine; I broke ‘seen’ which was embarrassing but was ok in\n     general. Maybe I’d have more input if I had a larger contribution. I do\n     wish I had gotten more review faster.\n\n 22. Glen: I’m also a pretty new contributor. The communication style is a lot\n     more direct than what I’m used to on the outside. But that doesn’t mean\n     it’s unhelpful or unconstructive… but it takes some getting used to. I had\n     to put in a lot of effort to trust that folks meant well and were trying\n     to help, and that’s just how we communicate here. But I could imagine it\n     being really intimidating if people aren’t used to that kind of review.\n\n 23. Taylor: To emphasize, I also remember when I started, I felt like people\n     were disappointed/upset by my contributions. Took me a while to\n     internalize that people were trying to help me make my contributions\n     better. So we should A) be careful to remember that new contributor\n     experience, and B) be careful to set an example even in reviews with\n     veteran contributors.\n\n 24. Philip: Often different nations have different writing style. “Thanks.” at\n     the end of the email means “Thanks but no thanks” in British English, but\n     that’s not usually how it’s meant on Git list. I also noticed it’s not\n     well explained why something is a problem. Reviewer thinks everybody knows\n     why they’re making some comment, but the person who proposed the patch\n     really didn’t see the issue and won’t understand. We should give a little\n     more background when pointing out an issue.\n\n 25. CB: We’re not the only project that won’t land things that aren’t\n     technically excellent. It’s important on my team too, or else the whole\n     company falls over next week. We’ve had lots of internal events and talks\n     about inclusive code reviews. The few extra sentences - “Thanks for\n     submitting, it looks great, the direction is good, I’ve got comments\n     because xyz” - go a really long way. We recently had a new team member’s\n     “good first issue” turn into a 2 week ordeal, and taking the extra time to\n     say “this is good progress, it’s shaping up well” was helpful to keep from\n     discouraging our new teammate.\n\n 26. Jrnieder: Chromium project has had a problem with this too, having very\n     high standards. So first Cr contribution would often just feel like a\n     hazing ritual. The focus should be on helpfulness, not “demonstrate how\n     much you care about the project by putting up with us”\n\n 27. https://chromium.googlesource.com/chromium/src/+/main/docs/cr_respect.md\n\n 28. ^ This covers a lot of what we were talking about; a good reference for\n     better/respectful code reviews. Timeframe, tone, stating\n     goals/expectations, etc. Should we adopt something similar in Git?\n\n 29. 26. Bmc: It can be frustrating to spend a lot of time on a series and then\n         immediately get a lot of technical feedback, without any assurance\n         that the direction is good. We should work harder to say “I’m glad to\n         see this patch, looking forward to seeing it land” instead of just\n         pointing out things to fix. Or “the way this patch looks vs. the last\n         version is really great”\n\n 30. Dscho: One thing my manager does well is to lead by asking, to give space\n     for me to reflect on what I just said and think about more perspectives.\n     This doesn’t put me on the defensive right away. I’ve made the mistake\n     before by assuming any reply at all implies “I’m interested in this\n     feature” - that’s not obvious, and instead my review comes out as “You’re\n     doing this wrong, go away” :( and I don’t know how many people felt that\n     and didn’t say anything, because they left.\n\n 31. Victoria: Given the unique nature of mailing list reviews, even though\n     there are a ton of resources on how to give respectful reviews, it’d be\n     useful to do a more specific guide for Git, discussing how to structure\n     review reply, how to follow up, etc.\n\n 32. Stolee: We have discussed a “guide to reviewing” in Documentation/, along\n     with SubmittingPatches and CodingGuidelines. We avoided it because it’s a\n     lot of work, and I’m also worried about the review of the review doc.\n     Would be a productive discussion….but a lot :)\n\n 33. Jonathantanmy: Yep, I’m thinking of doing one like that, hopefully in a\n     few weeks we can discuss it on list.\n\n 34. Avarab: I think it’s a good thing to work on; we need to be really careful\n     about what guidelines we pick and choose. Need to ensure an easy path for\n     new contributors so they don’t need to read hours of documentation for a\n     typo fix. Plus we need to ensure that this doc is accessible for folks who\n     have different first language than English.\n\n 35. Bmc: on git-lfs we have a contributor with very little English, so when we\n     did the review I’d offer an alternative text, and we would work together.\n     That process was useful to come up with readable documentation in a\n     helpful way. That is, proposing a solution instead of pointing out the\n     problem and saying “fix it” can help a lot in scenarios like this.\n\n 36. Dscho: Yep, this is important and will help us be more accessible to\n     contributors whose English is not super top notch Cambridge exam :)\n"},{"id":"439195","messageId":"nycvar.QRO.7.76.6.2110211150290.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] Improving Git UX","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:57:05Z","receivedAt":"2021-10-21T11:57:09Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by Johannes \"Dscho\" Schindelin. Supporting cast:\nJonathan \"jrnieder\" Nieder, brian m. carlson, Ævar Arnfjörð Bjarmason, CB\nBailey, Phillip Wood, and Emily Shaffer.\n\nNotes:\n\n 1.  A serious problem! ~½ of blog posts about Git start by ridiculing the\n     command-line interface of Git\n\n 2.  Is very flexible, and that flexibility is reflected in the user interface\n\n 3.  On the other hand, it could be a lot easier to use. Example: GitHub CLI in\n     Go, tries not to supplant Git but gives a really good user experience\n     interacting with GitHub-specific entities like PRs and Issues. Everything\n     you can do day-to-day in the web UI you can do in the command line, and\n     it’s scriptable\n\n 4.  Excellent discoverability. Never needed to check the manual. Well designed\n     interface, good command line completion.\n\n 5.  What would it take to revamp Git’s user interface?\n\n 6.  “git restore” example\n\n     1. Dscho doesn’t like it, feels wrong\n\n     2. Designed by a software engineer, not a designer\n\n     3. jrnieder: The manpage is pretty clear about “this is experimental,\n        we’re willing to modify it based on feedback”. Is there information we\n        can gather and work with a designer to improve it?\n\n     4. brian: “I don’t know how to use it” is valuable feedback. Maybe the\n        documentation is failing? Etc\n\n     5. Can be a sign of excessive complexity or of it relying on too much\n        previous knowledge to use\n\n 7.  Ævar: On switch/restore in particular, there was a recent discussion.\n\n     1. Ultimately came down to inconsistency with other commands in the same\n        area\n\n     2. I gave some suggestions\n\n     3. Some patches started, there was some trepidation about making changes,\n        though\n\n 8.  Dscho: Is working one command at a time too incremental?\n\n     1. Inconsistencies between different commands\n\n     2. “git add -A” exists, “git commit -A” doesn’t\n\n     3. Are those the most pressing problems? I don’t even know that\n\n 9.  Want guidance from user interface experts that work on the command line\n\n 10. gh command involved contractors that are no longer with GitHub\n\n 11. CB: The “pip” project is doing UX research. I don’t know who commissioned\n     it. Got UX experts to design study:\n     https://pip.pypa.io/en/latest/ux_research_design/\n\n 12. Phillip: The inconsistency between 'checkout -b' and 'switch -c' was\n     deliberate in the hope that '-c' for 'create' would be easier for users to\n     understand but ends up being confusing.\n\n 13. jrnieder: we should not expect an “angel” to swoop in and solve all our\n     problems for us. It’s more about how do we build this skill within the Git\n     project (by improving our own skills or attracting new contributors)\n\n 14. We should also consider that there are many people on the mailing list\n     with plenty of backgrounds, we might just need to band together to get it\n     done\n\n 15. Emily: maybe we can get some training (maybe the SFC could fund it, or\n     others in the Git ecosystem)\n\n 16. jrnieder: training is easier to fund than a permanent engagement\n\n 17. brian: if we had this expertise, we could probably make better decisions\n     in the future\n\n 18. Ævar: Conservancy could potentially find someone, but funding a different\n     matter\n\n 19. jrnieder: neutrality not all that important in this context, finding\n     funding at Google or GitHub should be easy\n\n 20. Ævar: often comes down to these consistencies. Getting anywhere with that\n     might just be a long slog.\n\n 21. Phillip Wood: checkout -b vs switch -c inconsistency was deliberate, in\n     that “-c” for create is meant to be easier for a new user\n\n 22. Dscho: Good first step is getting the UX design basics (training), I’ll\n     look for funding\n\n 23. Example: when I just run “git” with no other options, can that output be\n     more helpful? “gh” has a nice overview when I run it.\n\n 24. Ævar: That in particular was improved years ago\n\n     1. Dscho: Oh! Good. Might be possible to keep improving along those lines.\n\n     2. Dscho: Sounds like we have a good path forward. \\o/\n\n 25. hallway conversation\n\n     1. the index as a UI concept, what if we didn’t have it?\n\n     2. learning curve design space\n\n     3. how much does telemetry help us?\n\n     4. popcon-style telemetry\n\n     5. statistical rigor in surveys\n"},{"id":"439196","messageId":"nycvar.QRO.7.76.6.2110211150460.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"[Summit topic] Improving reviewer quality of life (patchwork, subsystem lists?, etc)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-21T11:57:11Z","receivedAt":"2021-10-21T11:57:15Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This session was led by Jonathan \"jrnieder\" Nieder. Supporting cast: brian\n\"bmc\" carlson, Emily Shaffer, Johannes \"Dscho\" Schindelin, CB Bailey,\nJunio Hamano, and Matthias Aßauer.\n\nNotes:\n\n 1. Been interested in this for a long time\n\n 2. Dscho said he’s not able to follow everything on the mailing list\n\n    1. if you have just one patch you send, reply-all works okay\n\n    2. mailing list works reasonably well if you’re someone like Junio, working\n       on it full time, has good mail filters, keeps up to date with everything\n\n    3. If you’re in-between, does not work well\n\n 3. Lessley mentioned in the Diversity & Inclusion context:\n\n    1. you send your patch,\n\n    2. want to have a solid review\n\n    3. want guidance where to go from here\n\n    4. timely feedback\n\n    5. want to know where the patch stands\n\n    6. What happens: guilt-based workflow, where reviewers reply much later\n       after being prodded\n\n 4. bmc: I want some way to track patches\n\n    1. What have I reviewed before and what have I not reviewed since last\n       time?\n\n    2. Emily: most of this exists in patchwork. Our intern Raxel Gutierrez did\n       work on that this summer. Alas, that doesn’t show up on\n       patchwork.kernel.org because it’s using patchwork 2.x and the features\n       are in 3.x\n\n    3. https://youtu.be/24dL8yqhYNg\n\n 5. bmc: I want some kind of bug tracking system\n\n    1.  In git-lfs when I need a git feature, people are happy to send a patch,\n        there’s no point of coordination for this. Don’t know where to send a\n        patch, don’t know where to send a bug report\n\n    2.  debbugs works okay, has a huge spam problem, but it works fine; email\n        based\n\n    3.  Emily: Every time this comes up I go oh $&!& because this is\n        perennially a source of dispute. I don’t care what tracker we use, just\n        want one\n\n    4.  Dscho: everyone else is caught in the crossfire between jrnieder and\n        me.\n\n    5.  CB: Is there an option that makes you both equally miserable?\n\n    6.  bmc: Could we get kernel.org to host something?\n\n    7.  jrnieder: there’s a bugzilla instance at bugzilla.kernel.org, which\n        might satisfy CB’s criterion\n\n    8.  bmc: I want to have whatever we use send out to the list. That would\n        avoid conversations going on without people in the mailing list centric\n        workflow being aware of it. If we are all using a GitHub/GitLab based\n        workflow then that’s not required\n\n    9.  Emily: +1, great point\n\n    10. jrnieder: Sounds like we have some common ground so seems worth\n        starting a mailing list thread\n\n    11. Junio: As long as I’m not the person operating the bug tracker, I’m\n        happy :)\n\n    12. Dscho: Is it important to you that it sends things to the mailing list?\n\n    13. Junio: Not really. The extra tracking conversations are not as\n        important to me. I think it’s a feature that if someone requests a\n        feature and nothing happens for a while that it no longer produces\n        overhead for people is a useful feature. That kind of old filtering\n        feature is sometimes valuable.\n\n    14. jrnieder: in a bug tracker, triage + common sense of priorities is very\n        useful. Experiences in JGit bugzilla vs the Debian bugtracker (the\n        latter is better curated)\n\n    15. brian: I’m happy to volunteer to do some triage on the bugtracker. If\n        other people will help out and contribute, happy to do that\n\n    16. I’m also happy to work with kernel.org admins to get this set up for us\n        if that’s what we want\n\n    17. people would expected to be kind+helpful in interactions there, can’t\n        expect it to devolve into a cesspit\n\n    18. Matthias: I’m happy to help with triage too\n"},{"id":"439199","messageId":"CAL3xRKck=cbdo=7gqv0q=MLjn+5J1ES6fbzHP1mEj90pLdUdAA@mail.gmail.com","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211144490.56@tvgsbejvaqbjf.bet","subject":"Re: [Summit topic] Crazy (and not so crazy) ideas","fromName":"Son Luong Ngoc","fromEmail":"sluongng@gmail.com","sentAt":"2021-10-21T12:30:50Z","receivedAt":"2021-10-21T12:31:04Z","isPatch":false,"sender":{"key":"sluongng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/26684313?v=4"},"body":"Hi,\n\nOn Thu, Oct 21, 2021 at 1:56 PM Johannes Schindelin\n<Johannes.Schindelin@gmx.de> wrote:\n>\n> This session was led by Elijah Newren. Supporting cast: Johannes \"Dscho\"\n> Schindelin, Jonathan Tan, Jonathan \"jrnieder\" Nieder, brian m. carlson,\n> Jeff \"Peff\" King, Ævar Arnfjörð Bjarmason, Emily Shaffer, CB Bailey,\n> Taylor Blau, and Philip Oakley.\n>\n> Notes:\n>\n\n...\n\n>\n> * Biggest idea: there are a lot of people who version control things via\n>   tarballs or .zip files per version. This prevents history from\n>   compressing well. Some people check in those compressed files into Git\n>   for purposes of history.\n>\n\n...\n\n>\n>    * Old suggestion of a “blob-tree” type that allows storing a single\n>      index entry that corresponds to multiple trees and blobs in the\n>      background, possibly.\n>\n>    * One long-term dream (inspired by Avery Pennarun’s “bup” tool) is to\n>      store large binary files in a tree-structured way that can store\n>      common regions as deltas, improve random access, parallelized\n>      hashing. Involves a consistent way to split the file into stable\n>      pieces, like --rsyncable uses (based on a rolling hash being zero).\n>\n>    * Peff: you can do that at the object model layer or at the storage\n>      layer. The latter is less invasive.\n>\n>    * jrnieder: The benefits of blobtree are greater at the object model\n>      layer --- e.g. not having to transmit chunks over the wire that you\n>      already have. I think the main obstacle has been that the benefits\n>      haven’t been enough to be worth the complexity. If that changes, we\n>      can imagine bundling it with some other object format changes, e.g.\n>      putting blob sizes in tree objects, and rolling it out as a new\n>      object-format.\n>\n\nI think this was implemented as 'Blob Ref' in Yandex's vcs named Arc.\nI was suggesting this to Gitlab folks earlier (1) as a possible solution to\nlarge file storage.\n\nVery glad to hear that it was brought up during the summit.\n\nCheers,\nSon Luong.\n\n(1): https://gitlab.com/gitlab-org/git/-/issues/93\n"},{"id":"439200","messageId":"CAL3xRKe6Ewps2n54KED7kX=8=Nk7RWHvTkhoB2X-Y1-ZjKEizw@mail.gmail.com","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211149530.56@tvgsbejvaqbjf.bet","subject":"Re: [Summit topic] Increasing diversity & inclusion (transition to `main`, etc)","fromName":"Son Luong Ngoc","fromEmail":"sluongng@gmail.com","sentAt":"2021-10-21T12:55:45Z","receivedAt":"2021-10-21T12:55:59Z","isPatch":false,"sender":{"key":"sluongng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/26684313?v=4"},"body":"Hi,\n\nOn Thu, Oct 21, 2021 at 1:57 PM Johannes Schindelin\n<Johannes.Schindelin@gmx.de> wrote:\n>\n> This session was led by Johannes \"Dscho\" Schindelin. Supporting cast:\n> brian \"Bmc\" carlson, Jeff \"Peff\" King, Taylor Blau, CB Bailey, Ævar\n> Arnfjörð \"Avarab\" Bjarmason, Jonathan \"Jrnieder\" Nieder, Derrick Stolee,\n> Lessley Dennington, Glen Choo, Philip Oakley, Victoria Dye, and Jonathan\n> \"Jonathantanmy\" Tan.\n>\n> Notes:\n>\n\n...\n\n>\n>  5.  Git the community is still overwhelmingly male, white - what can we do?\n>\n\n...\n\n>\n>  19. Stolee: When we moved from MS to GH, we received quick feedback that we\n>      weren’t communicating well - too direct and unemotional. Maybe Git\n>      community communicates that way, but that’s not how most people interact;\n>      that makes me think that our “efficient and effective” communication is\n>      actually too aggressive, and easily interpreted as attacks on\n>      contributors. Basically… let’s all lighten up? :)\n>\n>  20. Taylor: Yep, my “talking to GitHubbers at GitHub” voice is different from\n>      my “talking to Gitters on Git list” voice. New contributors, are we on the\n>      right track here?\n>\n\n...\n\n>\n>  34. Avarab: I think it’s a good thing to work on; we need to be really careful\n>      about what guidelines we pick and choose. Need to ensure an easy path for\n>      new contributors so they don’t need to read hours of documentation for a\n>      typo fix. Plus we need to ensure that this doc is accessible for folks who\n>      have different first language than English.\n>\n>  35. Bmc: on git-lfs we have a contributor with very little English, so when we\n>      did the review I’d offer an alternative text, and we would work together.\n>      That process was useful to come up with readable documentation in a\n>      helpful way. That is, proposing a solution instead of pointing out the\n>      problem and saying “fix it” can help a lot in scenarios like this.\n>\n>  36. Dscho: Yep, this is important and will help us be more accessible to\n>      contributors whose English is not super top notch Cambridge exam :)\n\nYes, thanks for mentioning the non-English speaking community.\n\nI have been an avid reader of the Git Mailing List for the past years and can't\nhelp but notice contributions from folks working in Alibaba(China) have been\ntaking a lot more iterations to get to final reviews than usual contributions.\n\nI would recommend, on top of having a guideline document, to have a\nValve check (1) setup as a commit-msg hook and run it as part of\nGitGitGadget CI to help folks shorten the feedback loops in some basic cases.\n\nCheers,\nSon Luong.\n\n(1): https://docs.errata.ai/vale/styles\n"},{"id":"439204","messageId":"20211021134105.ziqmcknnpdsg6cvc@meerkat.local","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211150460.56@tvgsbejvaqbjf.bet","subject":"Re: [Summit topic] Improving reviewer quality of life (patchwork, subsystem lists?, etc)","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2021-10-21T13:41:05Z","receivedAt":"2021-10-21T13:41:10Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Thu, Oct 21, 2021 at 01:57:11PM +0200, Johannes Schindelin wrote:\n>  2. Dscho said he’s not able to follow everything on the mailing list\n> \n>     1. if you have just one patch you send, reply-all works okay\n> \n>     2. mailing list works reasonably well if you’re someone like Junio, working\n>        on it full time, has good mail filters, keeps up to date with everything\n> \n>     3. If you’re in-between, does not work well\n\nThis is a problem that's not actually unique to mailing lists. If you have any\nproject that is popular enough, at some point it reaches critical mass where\ndeveloper/user feedback becomes too much for anyone to keep up. Github\nprojects aren't immune to this either, but they do have a benefit of providing\nan easy interface for someone to apply categorization to issues/discussions.\n\nOne of the efforts currently under way at public-inbox is the \"lei\" tool that\nshould allow similar workflows for mailing-list based interactions. At some\npoint we will be able to provide both topical and search-based subscriptions\nto subsets of the mailing list traffic that you're interested in. Search-based\nsubscriptions will allow you to monitor the list for discussions relevant to\nyour interest (e.g. patches touching functions/files/keywords that you are\nworking on). Topical subscriptions are a bit more complicated and would\nrequire someone to actively categorize mailing list discussions by keywords\n(e.g. bugs, suggestions, security), which would allow others to monitor just\nthose aspects of mailing list discussion. The latter requires someone's active\ninvolvement and dedication from the project side, not unlike for categorizing\nissues reported at github or any other issue tracker.\n\nIf you're curious, you can see my presentation to Linux Plumbers last month,\nwhich is here:\nyoutube: https://www.youtube.com/watch?v=mF10hgVIx9o&t=1490s\nslides: https://linuxplumbersconf.org/event/11/contributions/983/attachments/759/1421/Doing%20more%20with%20lore%20and%20b4.pdf\n\n>  4. bmc: I want some way to track patches\n> \n>     1. What have I reviewed before and what have I not reviewed since last\n>        time?\n> \n>     2. Emily: most of this exists in patchwork. Our intern Raxel Gutierrez did\n>        work on that this summer. Alas, that doesn’t show up on\n>        patchwork.kernel.org because it’s using patchwork 2.x and the features\n>        are in 3.x\n\n3.x is a bit new still, but chances are we'll be running it in a couple of\nmonths. Unfortunately, our previous experiences with major patchwork upgrades\nhave been a bit thorny, so I'm trying to approach this carefully in order not\nto impact other projects relying on it. (Not a dig at patchwork folks, just an\nobservation.)\n\n>     7.  jrnieder: there’s a bugzilla instance at bugzilla.kernel.org, which\n>         might satisfy CB’s criterion\n> \n>     8.  bmc: I want to have whatever we use send out to the list. That would\n>         avoid conversations going on without people in the mailing list centric\n>         workflow being aware of it. If we are all using a GitHub/GitLab based\n>         workflow then that’s not required\n\nBugzilla's mail integration is fairly good and list-friendly. We have several\nprojects that largely interact with their bugzilla via mailing lists\n(two-way). Note, that someone still has to do things like closing and\nrecategorizing bugs through the website.\n\nNote, that the initial bug report must come in through the bugzilla web\ninterface. There's a way to create bugs via incoming mail, but it works very\npoorly.\n\n>     13. Junio: Not really. The extra tracking conversations are not as\n>         important to me. I think it’s a feature that if someone requests a\n>         feature and nothing happens for a while that it no longer produces\n>         overhead for people is a useful feature. That kind of old filtering\n>         feature is sometimes valuable.\n\nI find that if there's no mailing list integration, then bugzilla generally\nrots after the initial person getting the bug reports moves on. Then bugs\nreported via bugzilla just sit there without anyone paying attention. At least\nwhen bug reports get sent to the list, the ensuing discussions get reflected\nin both the list archives and in bugzilla.\n\n>     16. I’m also happy to work with kernel.org admins to get this set up for us\n>         if that’s what we want\n\nConsider this part done. :)\n\n-K\n"},{"id":"439257","messageId":"211021.86wnm6l1ip.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211150290.56@tvgsbejvaqbjf.bet","subject":"changing the experimental 'git switch' (was: [Summit topic] Improving Git UX)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-21T16:45:44Z","receivedAt":"2021-10-21T17:40:36Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Oct 21 2021, Johannes Schindelin wrote:\n\n>  7.  Ævar: On switch/restore in particular, there was a recent discussion.\n>\n>      1. Ultimately came down to inconsistency with other commands in the same\n>         area\n>\n>      2. I gave some suggestions\n\nThose suggestions are at:\nhttps://lore.kernel.org/git/877dkdwgfe.fsf@evledraar.gmail.com/ copying\nthe most relevant part from that:\n\nIn summary, I think it should be changed to act like this:\n    \n    |---------------------------+------------------------+---------------------------|\n    | What                      | Now                    | New                       |\n    |---------------------------+------------------------+---------------------------|\n    | Switch                    | git switch existing    | git switch existing       |\n    | Error                     | git switch nonexisting | <no change (errors)>      |\n    | Switch with --merge       | git switch -m branch   | git switch --merge branch |\n    | Create                    | git switch -c new      | git switch -n new         |\n    | Create from existing      | N/A                    | git switch -c new [<old>] |\n    | Move & switch to existing | N/A                    | git switch -m new [<old>] |\n    |---------------------------+------------------------+---------------------------|\n\n>      3. Some patches started, there was some trepidation about making changes,\n>         though\n\nI was thinking of this patch, i.e. it implements the \"-n\" option for\n\"git\nswitch\": https://lore.kernel.org/git/20210709174310.94209-1-felipe.contreras@gmail.com/\n\nWe could then add the same to \"git branch\", i.e. \"git branch foo\" could\nalso be invoked as \"git branch -n foo\".\n\nWe'd then need to have a hard change in the semantics of the\n(experimental) \"git switch\" commant to make \"-c\" mean \"copy\" (like in\n\"git branch\").\n\nWe'd then reach an end-state where these two commands would behave in\nthe same way for these common options, with the difference being that\n\"branch\".\n\nWhatever anyone thinks of my specific suggestions there I think that in\ngeneral we should be trying to aim more towards that in git's UI, even\nto the point of slowly phasing in deprecations for non-experimental\ncommands. E.g. the \"-n\" option to \"git fetch\" comes to mind, which isn't\na synonym for \"--dry-run\", as in most other places.\n\nI realize that doing that is hard, e.g. Josh Steadmon has a patch\non-list now to add a configurable \"inherit\" mode to \"git branch[1].\n\nI noted in a similar vein as the table above that it would leave us with\nanother inconsistency between \"branch\" and \"checkout\"/\"switch\" in [2].\n\nDoes that mean we shouldn't take that patch and others like it until\nsuch UX inconsistencies are addressed?\n\nI really don't know, but I do think that the most viable path to a\nbetter UX for git is to consider its UX more holistically.\n\nTo the extent that our UX is a mess I think it's mainly because we've\nended up with an accumulation of behavior that made sense in isolation\nat the time, but which when combined presents bad or inconsistent UX to\nthe user.\n\n1. https://lore.kernel.org/git/9628d145881cb875f8e284967e10f587b9f686f9.1631126999.git.steadmon@google.com/\n2. https://lore.kernel.org/git/87a6j6tbsv.fsf@gmgdl.gmail.com/\n"},{"id":"439302","messageId":"xmqq7de6htfg.fsf@gitster.g","threadId":"56747","inReplyTo":"211021.86wnm6l1ip.gmgdl@evledraar.gmail.com","subject":"Re: changing the experimental 'git switch'","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-10-21T23:03:31Z","receivedAt":"2021-10-21T23:03:39Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n\n>     | Switch with --merge       | git switch -m branch   | git switch --merge branch |\n>     | Create                    | git switch -c new      | git switch -n new         |\n>     | Create from existing      | N/A                    | git switch -c new [<old>] |\n\nI agree that adding a way to \"clone\", which is missing from\n\"checkout/switch/branch\", is a good idea.  I do not necessarily\nthink it is a good idea to say \"-n\" is \"new\" or \"-c\" is not \"create\"\nbut is \"clone/copy\".  As you said yourself in a later paragraph,\n\"-n\" sometimes is \"--dry-run\", and as we can see here \"-c\" in the\ncontext of a command that can create and clone (with two verbs\nbehaving differently) is ambiguous.\n\nStarting with a spelled out --copy vs --create without muddying the\nwater with -n may be a sensible way forward.\n"},{"id":"439325","messageId":"bdf47d51-9cf6-046d-fd97-aa35299daadd@gmail.com","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211147490.56@tvgsbejvaqbjf.bet","subject":"Re: [Summit topic] Server-side merge/rebase: needs and wants?","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-10-22T03:06:28Z","receivedAt":"2021-10-22T03:06:37Z","isPatch":false,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 21/10/21 18.56, Johannes Schindelin wrote:\n>   5.  The challenge is not necessarily the technical challenges, but the UX for\n>       server tools that live “above” the git executable.\n> \n>       1. What kind of output is needed? Machine-readable error messages?\n> \n>       2. What Git objects must be created: a tree? A commit?\n> \n>       3. How to handle, report, and store conflicts? Index is not typically\n>          available on the server.\n\n1) I prefer human-readable (i.e. l10n-able) output, because the output \nmessages for server-side merge/rebase are user-facing.\n\n2) Same as when doing merge/rebase on local machine (merge commit if \nnon-ff).\n\n3) I think because on the server-side we have bare repo (instead of \nnormal repo), we need to create temporary index just for merge/rebase. \nFor conflicts, the users need to resolve them locally, then notify the \nserver that they have been resolved, and continue merging process.\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"439329","messageId":"dc479498-428e-7cd4-cad5-da19875a9ad8@gmail.com","threadId":"56747","inReplyTo":"211021.86wnm6l1ip.gmgdl@evledraar.gmail.com","subject":"Re: changing the experimental 'git switch' (was: [Summit topic] Improving Git UX)","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-10-22T03:33:43Z","receivedAt":"2021-10-22T03:33:48Z","isPatch":false,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 21/10/21 23.45, Ævar Arnfjörð Bjarmason wrote:\n> In summary, I think it should be changed to act like this:\n>      \n>      |---------------------------+------------------------+---------------------------|\n>      | What                      | Now                    | New                       |\n>      |---------------------------+------------------------+---------------------------|\n>      | Switch                    | git switch existing    | git switch existing       |\n>      | Error                     | git switch nonexisting | <no change (errors)>      |\n>      | Switch with --merge       | git switch -m branch   | git switch --merge branch |\n>      | Create                    | git switch -c new      | git switch -n new         |\n>      | Create from existing      | N/A                    | git switch -c new [<old>] |\n>      | Move & switch to existing | N/A                    | git switch -m new [<old>] |\n>      |---------------------------+------------------------+---------------------------|\n> \n\nFor switch with --merge case, it seems like adding long-option variant \nof -m (--merge), right?\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"439347","messageId":"nycvar.QRO.7.76.6.2110221000480.62@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"Missing notes, was Re: Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-22T08:02:50Z","receivedAt":"2021-10-22T08:02:55Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 21 Oct 2021, Johannes Schindelin wrote:\n\n> Team,\n>\n> we held our second all-virtual Summit over the past two days. It was the\n> traditional unconference style meeting, with topics being proposed and\n> voted on right before the introduction round. It was really good to see\n> the human faces behind those email addresses.\n>\n> 32 contributors participated, and we spanned the timezones from PST to\n> IST. To make that possible, the event took place on two days, from\n> 1500-1900 UTC, which meant that the attendees from the US West coast had\n> to get up really early, while it was past midnight in India at the end.\n>\n> I would like to thank all participants for accommodating the time, and in\n> particular for creating such a friendly, collaborative atmosphere.\n>\n> A particular shout-out to Jonathan Nieder, Emily Shaffer and Derrick\n> Stolee for taking notes. I am going to send out these notes in per-topic\n> subthreads, replying to this mail.\n>\n> Day 1 topics:\n>\n> * Crazy (and not so crazy) ideas\n> * SHA-256 Updates\n> * Server-side merge/rebase: needs and wants?\n> * Submodules and how to make them worth using\n> * Sparse checkout behavior and plans\n>\n> Day 2 topics:\n>\n> * The state of getting a reftable backend working in git.git\n> * Documentation (translations, FAQ updates, new user-focused, general\n>   improvements, etc.)\n> * Let's have public Git chalk talks\n\nYou might wonder why I did not send out the notes for this talk.\n\nBut that is not true! I sent it 6 times already, in various variations,\nand it never came through (but I did get two nastygrams telling me that my\nmessage was rejected because it apparently triggered a filter).\n\nI shall keep trying, but my hopes are pretty low by now.\n\nCiao,\nJohannes\n\n> * Increasing diversity & inclusion (transition to `main`, etc)\n> * Improving Git UX\n> * Improving reviewer quality of life (patchwork, subsystem lists?, etc)\n>\n> A few topics were left for a later date (maybe as public Git chalk talks):\n>\n> * Making Git memory-leak free (already landed patches)\n> * Scaling Git\n> * Scaling ref advertisements\n> * Config-based hooks (and getting there via migration ot hook.[ch] lib &\n>   \"git hook run\")\n> * Make git [clone|fetch] support pre-seeding via downloaded *.bundle files\n>\n> Ciao,\n> Johannes\n>\n"},{"id":"439348","messageId":"nycvar.QRO.7.76.6.2110221020570.62@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110221000480.62@tvgsbejvaqbjf.bet","subject":"Re: Missing notes, was Re: Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-22T08:22:42Z","receivedAt":"2021-10-22T08:22:47Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Team,\n\nI tried to reply with the full notes, which failed. So I'll try again,\nthis time in chunks.\n\nOn Fri, 22 Oct 2021, Johannes Schindelin wrote:\n\n> Hi,\n>\n> On Thu, 21 Oct 2021, Johannes Schindelin wrote:\n>\n> > Team,\n> >\n> > we held our second all-virtual Summit over the past two days. It was the\n> > traditional unconference style meeting, with topics being proposed and\n> > voted on right before the introduction round. It was really good to see\n> > the human faces behind those email addresses.\n> >\n> > 32 contributors participated, and we spanned the timezones from PST to\n> > IST. To make that possible, the event took place on two days, from\n> > 1500-1900 UTC, which meant that the attendees from the US West coast had\n> > to get up really early, while it was past midnight in India at the end.\n> >\n> > I would like to thank all participants for accommodating the time, and in\n> > particular for creating such a friendly, collaborative atmosphere.\n> >\n> > A particular shout-out to Jonathan Nieder, Emily Shaffer and Derrick\n> > Stolee for taking notes. I am going to send out these notes in per-topic\n> > subthreads, replying to this mail.\n> >\n> > Day 1 topics:\n> >\n> > * Crazy (and not so crazy) ideas\n> > * SHA-256 Updates\n> > * Server-side merge/rebase: needs and wants?\n> > * Submodules and how to make them worth using\n> > * Sparse checkout behavior and plans\n> >\n> > Day 2 topics:\n> >\n> > * The state of getting a reftable backend working in git.git\n> > * Documentation (translations, FAQ updates, new user-focused, general\n> >   improvements, etc.)\n> > * Let's have public Git chalk talks\n>\n> You might wonder why I did not send out the notes for this talk.\n>\n> But that is not true! I sent it 6 times already, in various variations,\n> and it never came through (but I did get two nastygrams telling me that my\n> message was rejected because it apparently triggered a filter).\n\nThis session was led by Emily Shaffer. Supporting cast: Ævar Arnfjörð\nBjarmason, brian m. carlson, CB Bailey, and Junio Hamano.\n\nNotes:\n\n 1.  What’s a public chalk talk?\n\n     1.  At Google, once a week, the team meets up with no particular topic in\n         mind, or a couple topics, very informal\n\n     2.  One person’s turn each week to give an informal talk with a white\n         board (not using chalk)\n\n     3.  Topic should be technical and of interest to the presenter\n\n     4.  For example: how does protocol v2 work\n\n     5.  Collaborative, interactive user session\n\n     6.  Helps by learning about things\n\n     7.  Helps by honing skills like presentation skills\n\n     8.  A lot of (good) humility involved. For example, colleagues who have\n         been familiar with the project for a long time admitting they don’t\n         know, or have been wrong about things. Makes others feel more\n         comfortable with their perceived lack of knowledge\n\n     9.  Could be good for everybody on the Git mailing list, might foster less\n         combative communication on the list\n\n     10. Might be a way to attract new people by presenting “old timers” as\n         humble\n\n 2.  Does that appeal to anybody else?\n\nto be continued...\n\n>\n> I shall keep trying, but my hopes are pretty low by now.\n>\n> Ciao,\n> Johannes\n>\n> > * Increasing diversity & inclusion (transition to `main`, etc)\n> > * Improving Git UX\n> > * Improving reviewer quality of life (patchwork, subsystem lists?, etc)\n> >\n> > A few topics were left for a later date (maybe as public Git chalk talks):\n> >\n> > * Making Git memory-leak free (already landed patches)\n> > * Scaling Git\n> > * Scaling ref advertisements\n> > * Config-based hooks (and getting there via migration ot hook.[ch] lib &\n> >   \"git hook run\")\n> > * Make git [clone|fetch] support pre-seeding via downloaded *.bundle files\n> >\n> > Ciao,\n> > Johannes\n> >\n>\n"},{"id":"439349","messageId":"nycvar.QRO.7.76.6.2110221028090.62@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110221020570.62@tvgsbejvaqbjf.bet","subject":"Re: Missing notes, was Re: Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-22T08:30:23Z","receivedAt":"2021-10-22T08:30:28Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Continuing... (2nd try, with redactions)\n\nOn Fri, 22 Oct 2021, Johannes Schindelin wrote:\n\n> I tried to reply with the full notes, which failed. So I'll try again,\n> this time in chunks.\n>\n> On Fri, 22 Oct 2021, Johannes Schindelin wrote:\n>\n> > On Thu, 21 Oct 2021, Johannes Schindelin wrote:\n> >\n> > > Team,\n> > >\n> > > we held our second all-virtual Summit over the past two days. It was the\n> > > traditional unconference style meeting, with topics being proposed and\n> > > voted on right before the introduction round. It was really good to see\n> > > the human faces behind those email addresses.\n> > >\n> > > 32 contributors participated, and we spanned the timezones from PST to\n> > > IST. To make that possible, the event took place on two days, from\n> > > 1500-1900 UTC, which meant that the attendees from the US West coast had\n> > > to get up really early, while it was past midnight in India at the end.\n> > >\n> > > I would like to thank all participants for accommodating the time, and in\n> > > particular for creating such a friendly, collaborative atmosphere.\n> > >\n> > > A particular shout-out to Jonathan Nieder, Emily Shaffer and Derrick\n> > > Stolee for taking notes. I am going to send out these notes in per-topic\n> > > subthreads, replying to this mail.\n> > >\n> > > Day 1 topics:\n> > >\n> > > * Crazy (and not so crazy) ideas\n> > > * SHA-256 Updates\n> > > * Server-side merge/rebase: needs and wants?\n> > > * Submodules and how to make them worth using\n> > > * Sparse checkout behavior and plans\n> > >\n> > > Day 2 topics:\n> > >\n> > > * The state of getting a reftable backend working in git.git\n> > > * Documentation (translations, FAQ updates, new user-focused, general\n> > >   improvements, etc.)\n> > > * Let's have public Git chalk talks\n> >\n> > You might wonder why I did not send out the notes for this talk.\n> >\n> > But that is not true! I sent it 6 times already, in various variations,\n> > and it never came through (but I did get two nastygrams telling me that my\n> > message was rejected because it apparently triggered a filter).\n>\n> This session was led by Emily Shaffer. Supporting cast: Ævar Arnfjörð\n> Bjarmason, brian m. carlson, CB Bailey, and Junio Hamano.\n>\n> Notes:\n>\n>  1.  What’s a public chalk talk?\n>\n>      1.  At Google, once a week, the team meets up with no particular topic in\n>          mind, or a couple topics, very informal\n>\n>      2.  One person’s turn each week to give an informal talk with a white\n>          board (not using chalk)\n>\n>      3.  Topic should be technical and of interest to the presenter\n>\n>      4.  For example: how does protocol v2 work\n>\n>      5.  Collaborative, interactive user session\n>\n>      6.  Helps by learning about things\n>\n>      7.  Helps by honing skills like presentation skills\n>\n>      8.  A lot of (good) humility involved. For example, colleagues who have\n>          been familiar with the project for a long time admitting they don’t\n>          know, or have been wrong about things. Makes others feel more\n>          comfortable with their perceived lack of knowledge\n>\n>      9.  Could be good for everybody on the Git mailing list, might foster less\n>          combative communication on the list\n>\n>      10. Might be a way to attract new people by presenting “old timers” as\n>          humble\n>\n>  2.  Does that appeal to anybody else?\n\n[redacting a word I suspect to have triggered vger's filter: it is a word\nstarting with \"T\" and continuing with \"witch\". Whenever you read \"[itch]\",\nthat's what I substitued for the culprit]\n\n 3.  Ævar: I think it would be great, has been a long time we’ve seen each\n     other, and already feels different\n\n 4.  One thing to keep in mind: it’s hard to program on a white board :-)\n\n 5.  Emily: some challenges:\n\n     1. How often?\n\n     2. What time?\n\n     3. Probably move things around (because we’re global)\n\n     4. Tech to use? Jitsi? [itch]? ([itch] seems to be particularly popular to\n        teach programming)\n\n     5. Figure out what topics to present\n\n 6.  Ævar: does not matter what tech to use\n\n 7.  Emily: some difference may make it matter: on [itch], you can record, and\n     they host recordings\n\n 8.  One thing to worry about recording: people might be reticent to make\n     public mistakes\n\n 9.  It’s possible to do a [itch] stream, and not record it\n\nto be continued...\n>\n> >\n> > I shall keep trying, but my hopes are pretty low by now.\n> >\n> > Ciao,\n> > Johannes\n> >\n> > > * Increasing diversity & inclusion (transition to `main`, etc)\n> > > * Improving Git UX\n> > > * Improving reviewer quality of life (patchwork, subsystem lists?, etc)\n> > >\n> > > A few topics were left for a later date (maybe as public Git chalk talks):\n> > >\n> > > * Making Git memory-leak free (already landed patches)\n> > > * Scaling Git\n> > > * Scaling ref advertisements\n> > > * Config-based hooks (and getting there via migration ot hook.[ch] lib &\n> > >   \"git hook run\")\n> > > * Make git [clone|fetch] support pre-seeding via downloaded *.bundle files\n> > >\n> > > Ciao,\n> > > Johannes\n> > >\n> >\n"},{"id":"439357","messageId":"1M4JmN-1me7VT1yCn-000Mey@mail.gmx.net","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110221028090.62@tvgsbejvaqbjf.bet","subject":"Re: Missing notes, was Re: Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-22T09:07:37Z","receivedAt":"2021-10-22T09:07:42Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Team,\n\nafter 10 failed attempts to send more notes, this might start to get a bit\nannoying on my side.\n\nOn Fri, 22 Oct 2021, Johannes Schindelin wrote:\n\n> On Fri, 22 Oct 2021, Johannes Schindelin wrote:\n>\n> > On Fri, 22 Oct 2021, Johannes Schindelin wrote:\n> >\n> > > On Thu, 21 Oct 2021, Johannes Schindelin wrote:\n> > >\n> > > > * Let's have public Git chalk talks\n> > >\n> > > You might wonder why I did not send out the notes for this talk.\n> > >\n> > > But that is not true! I sent it 6 times already, in various variations,\n> > > and it never came through (but I did get two nastygrams telling me that my\n> > > message was rejected because it apparently triggered a filter).\n> >\n> > This session was led by Emily Shaffer. Supporting cast: Ævar Arnfjörð\n> > Bjarmason, brian m. carlson, CB Bailey, and Junio Hamano.\n> >\n> > Notes:\n> >\n> >  1.  What’s a public chalk talk?\n> >\n> >      1.  At Google, once a week, the team meets up with no particular topic in\n> >          mind, or a couple topics, very informal\n> >\n> >      2.  One person’s turn each week to give an informal talk with a white\n> >          board (not using chalk)\n> >\n> >      3.  Topic should be technical and of interest to the presenter\n> >\n> >      4.  For example: how does protocol v2 work\n> >\n> >      5.  Collaborative, interactive user session\n> >\n> >      6.  Helps by learning about things\n> >\n> >      7.  Helps by honing skills like presentation skills\n> >\n> >      8.  A lot of (good) humility involved. For example, colleagues who have\n> >          been familiar with the project for a long time admitting they don’t\n> >          know, or have been wrong about things. Makes others feel more\n> >          comfortable with their perceived lack of knowledge\n> >\n> >      9.  Could be good for everybody on the Git mailing list, might foster less\n> >          combative communication on the list\n> >\n> >      10. Might be a way to attract new people by presenting “old timers” as\n> >          humble\n> >\n> >  2.  Does that appeal to anybody else?\n>\n> [redacting a word I suspect to have triggered vger's filter: it is a word\n> starting with \"T\" and continuing with \"witch\". Whenever you read \"[itch]\",\n> that's what I substitued for the culprit]\n>\n>  3.  Ævar: I think it would be great, has been a long time we’ve seen each\n>      other, and already feels different\n>\n>  4.  One thing to keep in mind: it’s hard to program on a white board :-)\n>\n>  5.  Emily: some challenges:\n>\n>      1. How often?\n>\n>      2. What time?\n>\n>      3. Probably move things around (because we’re global)\n>\n>      4. Tech to use? Jitsi? [itch]? ([itch] seems to be particularly popular to\n>         teach programming)\n>\n>      5. Figure out what topics to present\n>\n>  6.  Ævar: does not matter what tech to use\n>\n>  7.  Emily: some difference may make it matter: on [itch], you can record, and\n>      they host recordings\n>\n>  8.  One thing to worry about recording: people might be reticent to make\n>      public mistakes\n>\n>  9.  It’s possible to do a [itch] stream, and not record it\n\nThe brian m. carlson offered the idea to be considerate of reservations by\nparticipants, but also accommodate Git contributors who would have loved\nto see the presentation but were unable to attend due to timezones, time\nconflicts, etc: offer it for viewing only for a short while.\n\nto be continued\n\n> > >\n> > > I shall keep trying, but my hopes are pretty low by now.\n> > >\n> > > Ciao,\n> > > Johannes\n> > >\n> > > > * Increasing diversity & inclusion (transition to `main`, etc)\n> > > > * Improving Git UX\n> > > > * Improving reviewer quality of life (patchwork, subsystem lists?, etc)\n> > > >\n> > > > A few topics were left for a later date (maybe as public Git chalk talks):\n> > > >\n> > > > * Making Git memory-leak free (already landed patches)\n> > > > * Scaling Git\n> > > > * Scaling ref advertisements\n> > > > * Config-based hooks (and getting there via migration ot hook.[ch] lib &\n> > > >   \"git hook run\")\n> > > > * Make git [clone|fetch] support pre-seeding via downloaded *.bundle files\n> > > >\n> > > > Ciao,\n> > > > Johannes\n> > > >\n> > >\n>\n"},{"id":"439360","messageId":"nycvar.QRO.7.76.6.2110221143240.62@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211129130.56@tvgsbejvaqbjf.bet","subject":"Let's have public Git chalk talks, was Re: Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-22T09:44:46Z","receivedAt":"2021-10-22T09:44:51Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Team,\n\nOn Thu, 21 Oct 2021, Johannes Schindelin wrote:\n\n> * Let's have public Git chalk talks\n\nOkay, I give up on the mailing list. I tried some 20 times to send the\nnotes out in one form or another, and it simply is not working, and the\ntime I spent trying was definitely lost time.\n\nSo here is a link:\nhttps://gist.github.com/dscho/003a0e112058e5794b5e08e84d34092d\n\nCiao,\nJohannes\n"},{"id":"439362","messageId":"nycvar.QRO.7.76.6.2110221155460.62@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"bdf47d51-9cf6-046d-fd97-aa35299daadd@gmail.com","subject":"Re: [Summit topic] Server-side merge/rebase: needs and wants?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-22T10:01:09Z","receivedAt":"2021-10-22T10:01:14Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Bagas,\n\nOn Fri, 22 Oct 2021, Bagas Sanjaya wrote:\n\n> On 21/10/21 18.56, Johannes Schindelin wrote:\n> >   5.  The challenge is not necessarily the technical challenges, but the UX\n> >   for\n> >       server tools that live “above” the git executable.\n> >\n> >       1. What kind of output is needed? Machine-readable error messages?\n> >\n> >       2. What Git objects must be created: a tree? A commit?\n> >\n> >       3. How to handle, report, and store conflicts? Index is not typically\n> >          available on the server.\n>\n> 1) I prefer human-readable (i.e. l10n-able) output, because the output\n> messages for server-side merge/rebase are user-facing.\n\nFor server-side usage, a human-readable output _by Git_ would not make\nsense. It would be the responsibility of the server-side caller (which is\nusually a web application) to present the result, potentially translated,\ndefinitely prettified.\n\nSo while I agree with you that the result should be made pretty on the\nserver side, I disagree that this is Git's job. Instead, Git should\nproduce something eminently machine-parseable in this context.\n\n> 2) Same as when doing merge/rebase on local machine (merge commit if non-ff).\n\nLocal usage is _totally_ different.\n\n> 3) I think because on the server-side we have bare repo (instead of normal\n> repo), we need to create temporary index just for merge/rebase.\n\nMerge ORT does not need a temporary index. That's the reason it is so much\nfaster than the regular merge-recursive.\n\n> For conflicts, the users need to resolve them locally, then notify the\n> server that they have been resolved, and continue merging process.\n\nIt is already possible e.g. on GitHub to resolve merge conflicts in the\nweb UI. That is very convenient, and I think we all agreed at the Summit\nthat this is a scenario Git should support as well as it can. We did not\ncome to any concrete conclusion how that should look like (read: what\noutput format Git could use to support server-side consumption better),\nthough, and I think it basically comes down to experimenting with a couple\napproaches.\n\nCiao,\nDscho\n"},{"id":"439363","messageId":"nycvar.QRO.7.76.6.2110221201290.62@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"CAL3xRKe6Ewps2n54KED7kX=8=Nk7RWHvTkhoB2X-Y1-ZjKEizw@mail.gmail.com","subject":"vale check, was Re: [Summit topic] Increasing diversity & inclusion (transition to `main`, etc)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-22T10:02:14Z","receivedAt":"2021-10-22T10:02:18Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Bagas,\n\nOn Thu, 21 Oct 2021, Son Luong Ngoc wrote:\n\n> I would recommend, on top of having a guideline document, to have a\n> Valve check (1) setup as a commit-msg hook and run it as part of\n> GitGitGadget CI to help folks shorten the feedback loops in some basic\n> cases.\n>\n> [...]\n>\n> (1): https://docs.errata.ai/vale/styles\n\nHow about setting this up, then opening a PR?\n\nCiao,\nDscho\n"},{"id":"439364","messageId":"nycvar.QRO.7.76.6.2110221202430.62@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110221201290.62@tvgsbejvaqbjf.bet","subject":"Re: vale check, was Re: [Summit topic] Increasing diversity & inclusion (transition to `main`, etc)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-10-22T10:03:14Z","receivedAt":"2021-10-22T10:03:19Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Son Luong,\n\nOn Fri, 22 Oct 2021, Johannes Schindelin wrote:\n\n> Hi Bagas,\n\nSorry about that, I meant to address _you_, not Bagas.\n\nCiao,\nDscho\n\n>\n> On Thu, 21 Oct 2021, Son Luong Ngoc wrote:\n>\n> > I would recommend, on top of having a guideline document, to have a\n> > Valve check (1) setup as a commit-msg hook and run it as part of\n> > GitGitGadget CI to help folks shorten the feedback loops in some basic\n> > cases.\n> >\n> > [...]\n> >\n> > (1): https://docs.errata.ai/vale/styles\n>\n> How about setting this up, then opening a PR?\n>\n> Ciao,\n> Dscho\n>\n"},{"id":"439370","messageId":"9c6b3041-a5c0-6fe1-860e-7bfcb292ae81@mfriebe.de","threadId":"56747","inReplyTo":"211021.86wnm6l1ip.gmgdl@evledraar.gmail.com","subject":"Re: changing the experimental 'git switch' (was: [Summit topic] Improving Git UX)","fromName":"martin","fromEmail":"test2@mfriebe.de","sentAt":"2021-10-22T14:04:23Z","receivedAt":"2021-10-22T14:09:59Z","isPatch":false,"sender":{"key":"test2@mfriebe.de","avatar":null},"body":"On 21/10/2021 18:45, Ævar Arnfjörð Bjarmason wrote:\n> E.g. the \"-n\" option to \"git fetch\" comes to mind, which isn't\n> a synonym for \"--dry-run\", as in most other places.\n>\n\n-n\nis only used very few times for dry run. I found\ngit add\ngit rm\ngit mv\n\nBut\ncherry-pick => no commit\npull => no stat\nrebase => no stat\nmerge => no stat\nfetch => no tags\nclone => no checkout\n\nIn any case, \"-n\" has always a \"no\" meaning (even dry run, mean \"no \nchanges to be recorded\").\n\nSo IMHO -n is a really bad idea for \"new\"\n\n\nAbout \"-b\" branch:\nThat does give no indication something is created. I find it highly \nconfusing for checkout already,\nbecause the word \"branch\" could also mean \"check out to existing branch\" \nrather than doing a detached checkout.\nHowever, others may be perfectly fine with -b only referring to branches \nthat will be created.\n\n-c of course is also used for config in clone.... :)\n\nIf 2 letters could be used, then -c could be given twice for \"create copy\"\n-c  => create\n-c -c  => create copy\n-cc  => create copy\n\n----------\nAlso, will move/copy for switch actually be the same as for \"git branch\"?\n\nI haven't used them, but from the docs, I take it that a \n[new/replacement] branch will be created, and this branches tip points \nto the same commit as the origin branch.\n\nBut in \"git switch\" a new commit for the top is given. So that differs.\nMaybe someone can educate me ?\n- For move, where is the diff between\n   git switch --move existing_branch  commit\n   git switch --force-create existing_branch  commit\nAfaik only that the reflog will be copied/kept?\n\nFor copy what does it mean at all?\n   git switch --copy existing_branch  commit\nDoes not make any sense at all.\nBecause \"copy\" means that \"existing_branch\" is to be kept. So copy needs \na name for the new branch.\nI see 2 possible copies\n   git switch --copy existing_branch  new_branch commit\n   git switch --copy existing_branch  target_branch\nFor the latter, it switches to the existing \"target_branch\", but \nreplaces its reflog.\n\nUnless there is more, than the copying of the reflog, wouldn't it be \nbetter to add an option \"--copy-reflog\"\nThen you could do\ngit switch --copy-reflog=branch   target_branch  # replace reflog of \nexisting target branch\ngit switch --copy-reflog=branch  -c new_branch  target_branch  # \nnew_branch will get the reflog / this is \"copy\"\ngit switch --copy-reflog=branch  -C new_branch  target_branch  # \nnew_branch will get the reflog\ngit switch --copy-reflog  -C existing_branch  target_branch  # \nexisting_branch will keep the reflog. / this is \"move\"\n\n"},{"id":"439372","messageId":"1c9adc5d-21ac-f6c6-8a87-959be5420636@free.fr","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211149000.56@tvgsbejvaqbjf.bet","subject":"Re: [Summit topic] Documentation (translations, FAQ updates, new user-focused, general improvements, etc.)","fromName":"Jean-Noël Avila","fromEmail":"jn.avila@free.fr","sentAt":"2021-10-22T14:20:27Z","receivedAt":"2021-10-22T14:20:35Z","isPatch":false,"sender":{"key":"jn.avila@free.fr","avatar":"https://avatars.githubusercontent.com/u/156172?v=4"},"body":"I'm sorry that my presence at this meeting could have helped a bit for\nsome subtopics.\n\nLe 21/10/2021 à 13:56, Johannes Schindelin a écrit :\n> This session was led by brian m. carlson. Supporting cast: Jeff \"Peff\"\n> King, Ævar Arnfjörð Bjarmason, Taylor Blau, Philip Oakley, Emily Shaffer,\n> CB Bailey, and Jonathan \"jrnieder\" Nieder.\n>\n> Notes:\n>\n>  1. Background: answering on StackOverflow, other avenues for user questions,\n>     even users from very large companies\n>\n>  2. How can we improve documentation?\n>\n>  3. Maybe even think about translating docs such as FAQs\n>\n>  4. Peff: there’s an effort to translate manpages\n>\n>     1. brian: Saw an announcement, haven’t seen what came of it\n\nThe effort is still ongoing. Unfortunately, there aren't much outputs\nfrom it, only the inclusion on git-scm.com.\n\nA proposition was sent for Debian packages.\n\nI'm open for any help in packaging what's already available for whatever\nuseful.\n\n\nFor some statistics\n\n* there are 23 po files, \"pt_BR\" fully translated, \"fr\" half translated,\n\"de\" one third; most other languages have not really started (the\nportion already translated was made automatically for unmodified strings).\n\n* not all pages are included for translation; most porcelain pages\navailable on git-scm.com are included, but for instance, not the config\nparts or the guides. That's already 10,687 source segments and 206,700\nsource words, which is a volume similar to \"Crime and Punishment\" by\nDostoyevsky. And it really looks like an punishment for most apprentice\ntranslators willing to start.\n\nIn order to lower the barrier to translators, the project is relying on\nweblate: https://hosted.weblate.org/projects/git-manpages/translations/\nwhile still retaining a \"Developer's Certificate of Origin\".\n\n\n>\n>     2. Peff: Some translated pages are live on git-scm.com (a github repo with\n>        translations)\n\nFor instance, git init manpages is already available in 8 languages.\n\n\n>\n>     3. Ævar: It uses a third-party tool (po4a) that uses gettext by making each\n>        paragraph a translated string. So it’s the same workflow as translating\n>        code changes\n\nAsciidoc support is \"co-developed\" in po4a in parallel with the\ntranslation: I fix bugs when they are found in the po files.\n\n>     4. Taylor: https://github.com/jnavila/git-manpages-l10n\n\n\nIf it looks too personal, it can be moved into the git organization.\n\n\n>\n>  5. Philip Oakley: I see manpages used as reference material instead of\n>     educational documents\n>\n>\n>     12. In stackoverflow you can see how people answer questions, how much less\n>         existing background they assume\n\nVersion control is usually already in the culture of most users\n(writers, engineers in other fields have come to use them some 10 years\nago). What their questions usually boil down to is: how can I use and\ncustomize git features for my field of expertise. When software editors\ninclude git support in their applications, it is usually with severed\nfunctions and users quickly have to get back to plain git when they want\na little more.\n\nGeneral rules can help start up with a new customization, but at some\npoint, the customization is specific to the tool. A library of\napplication oriented customizations, help files and FAQs may be of\ninterest. Some customizations already exist, sometimes with errors\n(meaning the maintainer of the customization has not fully understood\nhow git works) but they are scattered.\n\n\n\n"},{"id":"439378","messageId":"211022.86v91pjfn7.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"9c6b3041-a5c0-6fe1-860e-7bfcb292ae81@mfriebe.de","subject":"Re: changing the experimental 'git switch' (was: [Summit topic] Improving Git UX)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-22T14:24:49Z","receivedAt":"2021-10-22T14:30:42Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Oct 22 2021, martin wrote:\n\n> On 21/10/2021 18:45, Ævar Arnfjörð Bjarmason wrote:\n>> E.g. the \"-n\" option to \"git fetch\" comes to mind, which isn't\n>> a synonym for \"--dry-run\", as in most other places.\n>>\n>\n> -n\n> is only used very few times for dry run. I found\n> git add\n> git rm\n> git mv\n>\n> But\n> cherry-pick => no commit\n> pull => no stat\n> rebase => no stat\n> merge => no stat\n> fetch => no tags\n> clone => no checkout\n>\n> In any case, \"-n\" has always a \"no\" meaning (even dry run, mean \"no\n> changes to be recorded\").\n>\n> So IMHO -n is a really bad idea for \"new\"\n\nGood point. I think I've changed my mind on that, but can't think of a\ngood short flag for such a thing.\n\nFWIW one reason this would be needed is that \"switch\" intentionally did\nnot take \"git switch unknown-name\" to create \"unknown-name\", but maybe\nwe could relax that if we just e.g. printed out a notice saying a new\nbranch is created (which we probably do already...).\n\nI.e. then the worst that'll happen is that the user has to \"git switch\n-\" and \"git branch -d -\", except I think the latter doesn't work, so\n\"git branch -d <that-name>\".\n\n> About \"-b\" branch:\n> That does give no indication something is created. I find it highly\n> confusing for checkout already,\n> because the word \"branch\" could also mean \"check out to existing\n> branch\" rather than doing a detached checkout.\n> However, others may be perfectly fine with -b only referring to\n> branches that will be created.\n>\n> -c of course is also used for config in clone.... :)\n>\n> If 2 letters could be used, then -c could be given twice for \"create copy\"\n> -c  => create\n> -c -c  => create copy\n> -cc  => create copy\n\nHrm, that's interesting. But probably better to have a long-option. Some\nshort options (notable -v for --verbose) often work like that, but I\nwonder if people wouldn't just be confused by it.\n\nMaybe not.\n\n> ----------\n> Also, will move/copy for switch actually be the same as for \"git branch\"?\n>\n> I haven't used them, but from the docs, I take it that a\n> [new/replacement] branch will be created, and this branches tip points \n> to the same commit as the origin branch.\n\nBoth of them can take an optional \"copy/create from\". So I this is the\nsame for both already, aside from one not supporting \"copy\".\n\n> But in \"git switch\" a new commit for the top is given. So that differs.\n> Maybe someone can educate me ?\n> - For move, where is the diff between\n>   git switch --move existing_branch  commit\n>   git switch --force-create existing_branch  commit\n> Afaik only that the reflog will be copied/kept?\n>\n> For copy what does it mean at all?\n>   git switch --copy existing_branch  commit\n> Does not make any sense at all.\n> Because \"copy\" means that \"existing_branch\" is to be kept. So copy\n> needs a name for the new branch.\n> I see 2 possible copies\n>   git switch --copy existing_branch  new_branch commit\n>   git switch --copy existing_branch  target_branch\n> For the latter, it switches to the existing \"target_branch\", but\n> replaces its reflog.\n\nMaybe I'm being dense, but I'm not really seeing how a:\n\n    git switch [some create option] <new> <old>\n\nWould have caveats that we don't have already with:\n\n    git branch [some create option] [<old>] <new>\n\nAside from the confusing switch-around of the arguments (which is\nanother UX wart...).\n\n> Unless there is more, than the copying of the reflog, wouldn't it be\n> better to add an option \"--copy-reflog\"\n> Then you could do\n> git switch --copy-reflog=branch   target_branch  # replace reflog of\n> existing target branch\n> git switch --copy-reflog=branch  -c new_branch  target_branch  #\n> new_branch will get the reflog / this is \"copy\"\n> git switch --copy-reflog=branch  -C new_branch  target_branch  #\n> new_branch will get the reflog\n> git switch --copy-reflog  -C existing_branch  target_branch  #\n> existing_branch will keep the reflog. / this is \"move\"\n\nYes, I think \"should it copy the reflog\" is a thing that's arguably\neither a missing feature or a bug in the \"git branch\" copy mode,\ndepending on your POV.\n"},{"id":"439379","messageId":"211022.86r1cdjfe2.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"1c9adc5d-21ac-f6c6-8a87-959be5420636@free.fr","subject":"Re: [Summit topic] Documentation (translations, FAQ updates, new user-focused, general improvements, etc.)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-22T14:31:46Z","receivedAt":"2021-10-22T14:36:10Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Oct 22 2021, Jean-Noël Avila wrote:\n\n> I'm sorry that my presence at this meeting could have helped a bit for\n> some subtopics.\n>\n> Le 21/10/2021 à 13:56, Johannes Schindelin a écrit :\n>> This session was led by brian m. carlson. Supporting cast: Jeff \"Peff\"\n>> King, Ævar Arnfjörð Bjarmason, Taylor Blau, Philip Oakley, Emily Shaffer,\n>> CB Bailey, and Jonathan \"jrnieder\" Nieder.\n>>\n>> Notes:\n>>\n>>  1. Background: answering on StackOverflow, other avenues for user questions,\n>>     even users from very large companies\n>>\n>>  2. How can we improve documentation?\n>>\n>>  3. Maybe even think about translating docs such as FAQs\n>>\n>>  4. Peff: there’s an effort to translate manpages\n>>\n>>     1. brian: Saw an announcement, haven’t seen what came of it\n>\n> The effort is still ongoing. Unfortunately, there aren't much outputs\n> from it, only the inclusion on git-scm.com.\n>\n> A proposition was sent for Debian packages.\n>\n> I'm open for any help in packaging what's already available for whatever\n> useful.\n>\n>\n> For some statistics\n>\n> * there are 23 po files, \"pt_BR\" fully translated, \"fr\" half translated,\n> \"de\" one third; most other languages have not really started (the\n> portion already translated was made automatically for unmodified strings).\n>\n> * not all pages are included for translation; most porcelain pages\n> available on git-scm.com are included, but for instance, not the config\n> parts or the guides. That's already 10,687 source segments and 206,700\n> source words, which is a volume similar to \"Crime and Punishment\" by\n> Dostoyevsky. And it really looks like an punishment for most apprentice\n> translators willing to start.\n>\n> In order to lower the barrier to translators, the project is relying on\n> weblate: https://hosted.weblate.org/projects/git-manpages/translations/\n> while still retaining a \"Developer's Certificate of Origin\".\n>\n>\n>>\n>>     2. Peff: Some translated pages are live on git-scm.com (a github repo with\n>>        translations)\n>\n> For instance, git init manpages is already available in 8 languages.\n>\n>\n>>\n>>     3. Ævar: It uses a third-party tool (po4a) that uses gettext by making each\n>>        paragraph a translated string. So it’s the same workflow as translating\n>>        code changes\n>\n> Asciidoc support is \"co-developed\" in po4a in parallel with the\n> translation: I fix bugs when they are found in the po files.\n>\n>>     4. Taylor: https://github.com/jnavila/git-manpages-l10n\n>\n>\n> If it looks too personal, it can be moved into the git organization.\n>\n>\n>>\n>>  5. Philip Oakley: I see manpages used as reference material instead of\n>>     educational documents\n>>\n>>\n>>     12. In stackoverflow you can see how people answer questions, how much less\n>>         existing background they assume\n>\n> Version control is usually already in the culture of most users\n> (writers, engineers in other fields have come to use them some 10 years\n> ago). What their questions usually boil down to is: how can I use and\n> customize git features for my field of expertise. When software editors\n> include git support in their applications, it is usually with severed\n> functions and users quickly have to get back to plain git when they want\n> a little more.\n>\n> General rules can help start up with a new customization, but at some\n> point, the customization is specific to the tool. A library of\n> application oriented customizations, help files and FAQs may be of\n> interest. Some customizations already exist, sometimes with errors\n> (meaning the maintainer of the customization has not fully understood\n> how git works) but they are scattered.\n\nI'd very much support this living in-tree just as the po/* directory\nalready does. I.e. periodically pulled down.\n\nThere are many OS's that have something like \"apt install\nmanpages-<lang>\", so if we had these available they could be much more\nuseful to users.\n\nE.g. I see I can \"apt install manpages-pt\", but if you're a Portuguese\nspeaker you probably won't chase down some third-party addition of\nPortuguese manpages, and even if they're in Debian other package\nmaintainers might not add them if they're not in the \"main\" package etc.\n\nWhat's standing in the way of us treating this in the same way as the\npo/* directory, if anything?\n"},{"id":"439425","messageId":"875ytozpwh.fsf@osv.gnss.ru","threadId":"56747","inReplyTo":"211022.86v91pjfn7.gmgdl@evledraar.gmail.com","subject":"Re: changing the experimental 'git switch'","fromName":"Sergey Organov","fromEmail":"sorganov@gmail.com","sentAt":"2021-10-22T21:54:38Z","receivedAt":"2021-10-22T21:54:43Z","isPatch":false,"sender":{"key":"sorganov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8501568?v=4"},"body":"Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n\n> On Fri, Oct 22 2021, martin wrote:\n>\n\n[...]\n\n>> If 2 letters could be used, then -c could be given twice for \"create copy\"\n>> -c  => create\n>> -c -c  => create copy\n>> -cc  => create copy\n\nPlease, no!\n\n> Hrm, that's interesting.\n\nYep, Git UI is too \"interesting\" already.\n\n> But probably better to have a long-option.\n\nDefinitely.\n\n> Some short options (notable -v for --verbose) often work like that,\n> but I wonder if people wouldn't just be confused by it.\n\nI would be confused. Those options that do behave like that usually\njust increase (implicit) level of verbosity or debug level, so -vv is a\nway to say --verbose=2, and -vvv => --verbose=3.\n\nAn option that changes its semantic depending on its sequence number is\nsomething that I'd avoid like a plague.\n\nThanks,\n-- Sergey Organov\n \n"},{"id":"439429","messageId":"211023.861r4ck8jw.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"20211021134105.ziqmcknnpdsg6cvc@meerkat.local","subject":"Re: [Summit topic] Improving reviewer quality of life (patchwork, subsystem lists?, etc)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-22T22:06:06Z","receivedAt":"2021-10-22T22:18:34Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Oct 21 2021, Konstantin Ryabitsev wrote:\n\n> On Thu, Oct 21, 2021 at 01:57:11PM +0200, Johannes Schindelin wrote:\n>>  2. Dscho said he’s not able to follow everything on the mailing list\n>> \n>>     1. if you have just one patch you send, reply-all works okay\n>> \n>>     2. mailing list works reasonably well if you’re someone like Junio, working\n>>        on it full time, has good mail filters, keeps up to date with everything\n>> \n>>     3. If you’re in-between, does not work well\n>\n> This is a problem that's not actually unique to mailing lists. If you have any\n> project that is popular enough, at some point it reaches critical mass where\n> developer/user feedback becomes too much for anyone to keep up. Github\n> projects aren't immune to this either, but they do have a benefit of providing\n> an easy interface for someone to apply categorization to issues/discussions.\n\nI'd like to use this mail as a good jump-off point to link to my \"how\"\nv.s. \"what\" E-Mail from when this was last discussed. I think I\nmentioned it in passing at the recent summit:\n\nhttps://lore.kernel.org/git/87fszd3xo0.fsf@evledraar.gmail.com/\n\nEspecially as...\n\n>>     13. Junio: Not really. The extra tracking conversations are not as\n>>         important to me. I think it’s a feature that if someone requests a\n>>         feature and nothing happens for a while that it no longer produces\n>>         overhead for people is a useful feature. That kind of old filtering\n>>         feature is sometimes valuable.\n>\n> I find that if there's no mailing list integration, then bugzilla generally\n> rots after the initial person getting the bug reports moves on. Then bugs\n> reported via bugzilla just sit there without anyone paying attention. At least\n> when bug reports get sent to the list, the ensuing discussions get reflected\n> in both the list archives and in bugzilla.\n\n...it makes a passive mention to this \"forgetting as a feature\" aspect\nof not having a bug tracker.\n\n>\n>>     16. I’m also happy to work with kernel.org admins to get this set up for us\n>>         if that’s what we want\n>\n> Consider this part done. :)\n\nAnd thank you for your contribution to kernel.org infrastructure.\n"},{"id":"439443","messageId":"dc5dcc43-34d0-f4a8-93fa-6875c98e74a5@mfriebe.de","threadId":"56747","inReplyTo":"211022.86v91pjfn7.gmgdl@evledraar.gmail.com","subject":"Re: changing the experimental 'git switch' (was: [Summit topic] Improving Git UX)","fromName":"martin","fromEmail":"test2@mfriebe.de","sentAt":"2021-10-22T15:30:26Z","receivedAt":"2021-10-23T08:08:24Z","isPatch":false,"sender":{"key":"test2@mfriebe.de","avatar":null},"body":"On 22/10/2021 16:24, Ævar Arnfjörð Bjarmason wrote:\n> FWIW one reason this would be needed is that \"switch\" intentionally did\n> not take \"git switch unknown-name\" to create \"unknown-name\", but maybe\n> we could relax that if we just e.g. printed out a notice saying a new\n> branch is created (which we probably do already...).\nI think the \"required flag for create\" is a good idea and should be \nkept. (My 2 cents)\n\nhaving the \"new name\" identified, also helps the user to get the order \nof the arguments right.\nTake\n   git rebase  <upstream>   <branch>\nWhy is the target listed before the source? But whichever way round, it \nis hard to remember.\nIf rebase would only take one <upstream>, and the optional <branch> \nwould be a \"--from\", wouldn't that be easier?\n\n>> If 2 letters could be used, then -c could be given twice for \"create copy\"\n>> -c  => create\n>> -c -c  => create copy\n>> -cc  => create copy\n> Hrm, that's interesting. But probably better to have a long-option.\nWell, both: Long and short. But long is --copy or --create-copy.\nThe issue is finding a short option. -cc imho is still short.\n\n>> But in \"git switch\" a new commit for the top is given. So that differs.\n>> Maybe someone can educate me ?\n> Maybe I'm being dense, but I'm not really seeing how a:\n>\n>      git switch [some create option] <new> <old>\n>\n> Would have caveats that we don't have already with:\nI only tried to illuminate my question with some made-up examples (the \nexamples were not meant to be a solution).\n\nWhat exactly does copy/move do? Am I missing any point?\n- They create a new branch (when using switch this branch will also be \nreset to a given commit)\n- They copy the reflog\n- \"move\" deletes the old branch (or can be seen as rename)\n\nAnything else?\n\n\n"},{"id":"439444","messageId":"875ytokuy3.fsf@osv.gnss.ru","threadId":"56747","inReplyTo":"dc5dcc43-34d0-f4a8-93fa-6875c98e74a5@mfriebe.de","subject":"Re: changing the experimental 'git switch'","fromName":"Sergey Organov","fromEmail":"sorganov@gmail.com","sentAt":"2021-10-23T08:27:00Z","receivedAt":"2021-10-23T08:27:07Z","isPatch":false,"sender":{"key":"sorganov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8501568?v=4"},"body":"martin <test2@mfriebe.de> writes:\n\n> On 22/10/2021 16:24, Ævar Arnfjörð Bjarmason wrote:\n\n[...]\n\n>>> If 2 letters could be used, then -c could be given twice for \"create copy\"\n>>> -c  => create\n>>> -c -c  => create copy\n>>> -cc  => create copy\n>> Hrm, that's interesting. But probably better to have a long-option.\n> Well, both: Long and short. But long is --copy or --create-copy.\n> The issue is finding a short option. -cc imho is still short.\n\nNo -cc or --cc, please! -cc is not single option, it's -c -c in a line,\nand you will then have hard time to even describe -c.\n\n--cc would be a point of confusion as well, e.g., see \"git log --cc\".\n\nBTW, is it frequent enough operation to even demand something shorter\nthan --copy?\n\nThanks,\n-- Sergey Organov\n"},{"id":"439456","messageId":"211023.86fssrihp5.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110221155460.62@tvgsbejvaqbjf.bet","subject":"Re: [Summit topic] Server-side merge/rebase: needs and wants?","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-23T20:52:27Z","receivedAt":"2021-10-23T20:56:12Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Oct 22 2021, Johannes Schindelin wrote:\n\n> Hi Bagas,\n>\n> On Fri, 22 Oct 2021, Bagas Sanjaya wrote:\n>\n>> On 21/10/21 18.56, Johannes Schindelin wrote:\n>> >   5.  The challenge is not necessarily the technical challenges, but the UX\n>> >   for\n>> >       server tools that live “above” the git executable.\n>> >\n>> >       1. What kind of output is needed? Machine-readable error messages?\n>> >\n>> >       2. What Git objects must be created: a tree? A commit?\n>> >\n>> >       3. How to handle, report, and store conflicts? Index is not typically\n>> >          available on the server.\n>>\n>> 1) I prefer human-readable (i.e. l10n-able) output, because the output\n>> messages for server-side merge/rebase are user-facing.\n>\n> For server-side usage, a human-readable output _by Git_ would not make\n> sense. It would be the responsibility of the server-side caller (which is\n> usually a web application) to present the result, potentially translated,\n> definitely prettified.\n>\n> So while I agree with you that the result should be made pretty on the\n> server side, I disagree that this is Git's job. Instead, Git should\n> produce something eminently machine-parseable in this context.\n\nOur server-side already produces human-readable output via die()\nmessages or ERR packets.\n\nTo the extent that we need human-readable output in say protocol v2 I\nthink it makes more sense to start supporting passing over the user's\nlocale to look the appropriate thing up in our *.mo files. The\nalternative is maintaining an exhaustive catalog of unique error ID's or\nwhatever.\n\nMost things should of course have meaningful error codes etc., I'm only\nreferring to the output that has some human readable \"we've failed, and\nhere's a text explanation for why\" component, see the various places in\nprotocol v0..2 where we emit that, e.g. telling a user what went wrong\nwith the \"filter\" arguments they provided etc.\n"},{"id":"439482","messageId":"da952e81-70f9-886b-42ff-2ec850f55fa0@mfriebe.de","threadId":"56747","inReplyTo":"9c6b3041-a5c0-6fe1-860e-7bfcb292ae81@mfriebe.de","subject":"Re: changing the experimental 'git switch' (was: [Summit topic] Improving Git UX)","fromName":"Martin","fromEmail":"git@mfriebe.de","sentAt":"2021-10-24T06:54:12Z","receivedAt":"2021-10-24T06:54:19Z","isPatch":false,"sender":{"key":"git@mfriebe.de","avatar":null},"body":"On 22/10/2021 16:04, martin wrote:\n> Unless there is more, than the copying of the reflog, wouldn't it be \n> better to add an option \"--copy-reflog\"\n\nOk, I found the answer (actually 2 / see other mail)\n> The |-c| and |-C| options have the exact same semantics as |-m| and \n> |-M|, except instead of the branch being renamed, it will be copied to \n> a new name, along with its config and reflog.\nAs for the \"config\" being part depends on which part of the man-page one \nreads.\n\nBut, anyway on the topic of \"git switch\"\n\nCopying a branch (and maybe moving too) could be seen as an extension to \ncreating a new one.\nAfter all, after the copy operation there is a newly created branch. \nOnly it has some more data with it.\n\nSo one could do\ngit switch  --settings-from <branch-with-reflog-and-conf> --create \n<new-branch>   <commit>\ngit switch  -s <branch-with-reflog-and-conf>   -c <new-branch>   <commit>\n\n\"settings-from\" is just an example, there may be better names for it. \nIdeally not starting with a \"c\".\n\nAnd using a name different from \"copy\" may be more accurate, because \nunless it is created on the same one <commit> to which the \n<branch-with-reflog-and-conf> points, then its at best partially copied.\n\nOn top of that options could be brought in, to copy only reflog or only \nconfig.\n\nUsing the above with an --force-create / -C would make it a \"move branch\".\nIn this case there could be a shortcut, if <branch-with-reflog-and-conf> \nand (the old) <new-branch>  are the same.\n\n\n"},{"id":"439506","messageId":"xmqqwnm2418p.fsf@gitster.g","threadId":"56747","inReplyTo":"da952e81-70f9-886b-42ff-2ec850f55fa0@mfriebe.de","subject":"Re: changing the experimental 'git switch'","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-10-24T20:27:34Z","receivedAt":"2021-10-24T20:27:39Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Martin <git@mfriebe.de> writes:\n\n> So one could do\n> git switch  --settings-from <branch-with-reflog-and-conf> --create\n> <new-branch>   <commit>\n> git switch  -s <branch-with-reflog-and-conf>   -c <new-branch>   <commit>\n>\n> \"settings-from\" is just an example, there may be better names for\n> it. Ideally not starting with a \"c\".\n>\n> And using a name different from \"copy\" may be more accurate, because\n> unless it is created on the same one <commit> to which the \n> <branch-with-reflog-and-conf> points, then its at best partially copied.\n\nI like the \"copy the settings from this other branch when creating\nthis new branch\" as a concept.\n\nOne thing that I find iffy is the reflog.  Even with the current\n\"create a new branch NEW, pointing at the same commit, tracking the\nsame remote-tracking branch, having the same branch description, and\npretending to have come along the same trajectory, out of this\noriginal branch OLD\", I actually find that the copyng of reflog is\nutterly questionable.  Before that operation, the new branch did not\nexist, hence NEW@{4.days.ago} shouldn't say the same thing as\nOLD@{4.days.ago} for the branch NEW that was created like so just a\nminute ago.\n\nIf you generalize the operation to allow starting the new branch at\na different commit, it becomes even more strange to copy the reflog\nof the \"original\" branch, which is not even the original for this\nnew branch.\n\nAnother thing nobody seems to have brought up is the branch\ndescription.  We copy everything under branch.OLD.* to branch.NEW.*\nand end up copying it from OLD to NEW, but I think that is also a\nnonsense operation.\n\nSo, it probably makes sense to be more selective that what are\nsensibly copied and what are not.  Reflog most likely does not\nbelong to the \"sensibly copyable\" set.  Tracking info most likely\ndoes.  Among various configuration in branch.OLD.*, there may be\nthings like description that are not sensibly copyable.\n\n"},{"id":"439541","messageId":"211025.86tuh5gtcc.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"xmqqwnm2418p.fsf@gitster.g","subject":"Re: changing the experimental 'git switch'","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-25T12:48:47Z","receivedAt":"2021-10-25T12:52:07Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Oct 24 2021, Junio C Hamano wrote:\n\n> Martin <git@mfriebe.de> writes:\n>\n>> So one could do\n>> git switch  --settings-from <branch-with-reflog-and-conf> --create\n>> <new-branch>   <commit>\n>> git switch  -s <branch-with-reflog-and-conf>   -c <new-branch>   <commit>\n>>\n>> \"settings-from\" is just an example, there may be better names for\n>> it. Ideally not starting with a \"c\".\n>>\n>> And using a name different from \"copy\" may be more accurate, because\n>> unless it is created on the same one <commit> to which the \n>> <branch-with-reflog-and-conf> points, then its at best partially copied.\n>\n> I like the \"copy the settings from this other branch when creating\n> this new branch\" as a concept.\n>\n> One thing that I find iffy is the reflog.  Even with the current\n> \"create a new branch NEW, pointing at the same commit, tracking the\n> same remote-tracking branch, having the same branch description, and\n> pretending to have come along the same trajectory, out of this\n> original branch OLD\", I actually find that the copyng of reflog is\n> utterly questionable.  Before that operation, the new branch did not\n> exist, hence NEW@{4.days.ago} shouldn't say the same thing as\n> OLD@{4.days.ago} for the branch NEW that was created like so just a\n> minute ago.\n>\n> If you generalize the operation to allow starting the new branch at\n> a different commit, it becomes even more strange to copy the reflog\n> of the \"original\" branch, which is not even the original for this\n> new branch.\n>\n> Another thing nobody seems to have brought up is the branch\n> description.  We copy everything under branch.OLD.* to branch.NEW.*\n> and end up copying it from OLD to NEW, but I think that is also a\n> nonsense operation.\n>\n> So, it probably makes sense to be more selective that what are\n> sensibly copied and what are not.  Reflog most likely does not\n> belong to the \"sensibly copyable\" set.  Tracking info most likely\n> does.  Among various configuration in branch.OLD.*, there may be\n> things like description that are not sensibly copyable.\n\nIt is a bit weird, but the main problem is that we'll use it for UI such\nas @{-1} or whatever in addition to things like \"x days ago\". So if you\ncopy a branch for some ad-hoc testing, and were just running such a\ncommand you might expend it to work.\n\nFor a user it also maps nicely to the mental model you'd have if you\ncopied two directories with the \"-p\" option to \"cp\", i.e. you'll be able\nto run a \"find\" command on that checking mtime of N days ago and the\nlike.\n\nMaybe it still doesn't make sense for those cases just some thoughts on\nUX edge cases.\n"},{"id":"439542","messageId":"211025.86pmrtgsxr.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110221143240.62@tvgsbejvaqbjf.bet","subject":"Re: Let's have public Git chalk talks, was Re: Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-25T12:58:48Z","receivedAt":"2021-10-25T13:00:56Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Oct 22 2021, Johannes Schindelin wrote:\n\n> Team,\n>\n> On Thu, 21 Oct 2021, Johannes Schindelin wrote:\n>\n>> * Let's have public Git chalk talks\n>\n> Okay, I give up on the mailing list. I tried some 20 times to send the\n> notes out in one form or another, and it simply is not working, and the\n> time I spent trying was definitely lost time.\n>\n> So here is a link:\n> https://gist.github.com/dscho/003a0e112058e5794b5e08e84d34092d\n\nTrying to see if it works for me. FWIW I recieved a bounce from GMail on\nan unrelated mail of mine in this thread whose error suggests that\ngmx.de's MX's may be rate limiting recieved mails on your account in\nsome way, which may or may not have anything to do with difficulties\ninteracting with kernel.org infrastructure in general.\n\nAttempt to paste\nhttps://gist.githubusercontent.com/dscho/003a0e112058e5794b5e08e84d34092d/raw/8d825e01152d957671d5da9e7c5217cabc37afa5/gistfile1.txt\nfollows below:\n\nThis session was led by Emily Shaffer. Supporting cast: Ævar Arnfjörð\nBjarmason, brian m. carlson, CB Bailey, and Junio Hamano.\n\nNotes:\n\n 1.  What’s a public chalk talk?\n\n     1.  At Google, once a week, the team meets up with no particular topic in\n         mind, or a couple topics, very informal\n\n     2.  One person’s turn each week to give an informal talk with a white\n         board (not using chalk)\n\n     3.  Topic should be technical and of interest to the presenter\n\n     4.  For example: how does protocol v2 work\n\n     5.  Collaborative, interactive user session\n\n     6.  Helps by learning about things\n\n     7.  Helps by honing skills like presentation skills\n\n     8.  A lot of (good) humility involved. For example, colleagues who have\n         been familiar with the project for a long time admitting they don’t\n         know, or have been wrong about things. Makes others feel more\n         comfortable with their perceived lack of knowledge\n\n     9.  Could be good for everybody on the Git mailing list, might foster less\n         combative communication on the list\n\n     10. Might be a way to attract new people by presenting “old timers” as\n         humble\n\n 2.  Does that appeal to anybody else?\n\n 3.  Ævar: I think it would be great, has been a long time we’ve seen each\n     other, and already feels different\n\n 4.  One thing to keep in mind: it’s hard to program on a white board :-)\n\n 5.  Emily: some challenges:\n\n     1. How often?\n\n     2. What time?\n\n     3. Probably move things around (because we’re global)\n\n     4. Tech to use? Jitsi? Twitch? (Twitch seems to be particularly popular to\n        teach programming)\n\n     5. Figure out what topics to present\n\n 6.  Ævar: does not matter what tech to use\n\n 7.  Emily: some difference may make it matter: on Twitch, you can record, and\n     they host recordings\n\n 8.  One thing to worry about recording: people might be reticent to make\n     public mistakes\n\n 9.  It’s possible to do a Twitch stream, and not record it\n\n 10. brian: maybe record it, but not keep the recordings forever\n\n 11. People might be uncomfortable having their homes being recorded\n\n 12. At GitHub, some sessions are recorded just so people from other timezones\n     can watch later\n\n 13. CB: would be a nice way to see the other contributors\n\n 14. Really like the idea, hopefully won’t replace other things we do\n\n 15. Emily: internally, often about patch series in progress (or not even\n     started)\n\n 16. So retaining recordings for long time makes even less sense\n\n 17. Weekly might be too frequently, Monthly cadence sound more reasonable?\n\n 18. Junio: not sure we want an official schedule\n\n 19. Assumed this would be an extension of what we do on IRC\n\n 20. Remember when Linus would drop in and talk about a specific topic in\n     depth, was nice\n\n 21. Now we have video\n\n 22. Emily: I fear if we don’t schedule it, it’ll never happen\n\n 23. Ævar: would like it to be organized, maybe try some schedule and then\n     iterate?\n\n 24. brian: if it is scheduled, I can put it on my calendar, otherwise might be\n     hard to block the time\n\n 25. Every two weeks would be fine, especially when alternating timezones\n\n 26. Emily: who besides me wants to volunteer for the other timezone?\n\n 27. Ævar: if you start a schedule, I’ll see what I can do\n\n 28. CB: also interested\n\n 29. brian: can do, but Toronto is probably too close to California time\n\n 30. Junio: schedule should be put on https://tinyurl.com/gitcal\n\n 31. Emily: how about using a Google Sheet just like for the Contributors’\n     Summit?\n\n 32. One advantage to decide the topic in advance is that people can decide\n     whether to make time to attend, on the other hand people might show up\n     with a polished PowerPoint, which is not the idea\n\n 33. brian: we can try, and if it does not work, make it less formal\n\n 34. Emily: pretty much got what I need to start this\n"},{"id":"439560","messageId":"87h7d5yrxy.fsf@osv.gnss.ru","threadId":"56747","inReplyTo":"211021.86wnm6l1ip.gmgdl@evledraar.gmail.com","subject":"Re: changing the experimental 'git switch'","fromName":"Sergey Organov","fromEmail":"sorganov@gmail.com","sentAt":"2021-10-25T16:44:57Z","receivedAt":"2021-10-25T16:45:03Z","isPatch":false,"sender":{"key":"sorganov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8501568?v=4"},"body":"Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n\n[...]\n\n> I really don't know, but I do think that the most viable path to a\n> better UX for git is to consider its UX more holistically.\n>\n> To the extent that our UX is a mess I think it's mainly because we've\n> ended up with an accumulation of behavior that made sense in isolation\n> at the time, but which when combined presents bad or inconsistent UX to\n> the user.\n\nYep. Moreover, this practice of \"making sense\" being the primary\nreasoning factor doesn't work very well even in isolation, for single\nGit sub-commands. As there is no defined underlying UI model, or rules,\nor even clear guidelines of how to properly design command-line options,\nmultiple authors, all having their own sense and having no common ground\nto base their decisions on, inevitably produce some spaghetti UI.\n\nThe UI model to be defined, provided we are serious about aiming at a\ngood design, in fact has at least 2 aspects to address:\n\n1. Uniform top-level syntax of all the Git commands.\n\n2. Uniform rules to handle command-line options.\n\nBeing hard to produce simple yet flexible design by itself, the problem\nis further complicated by the need to absorb as much of the existing UI\nas reasonably possible.\n\nOnce a model is defined though, we should be able to at least ensure new\ndesigns fit the model, and then, over time, gradually replace legacy UIs\nthat currently don't fit.\n\nAs a side-note, from this standpoint, discussing deep details of \"git\nswitch\" options, or even relevancy of introducing of \"git switch\" in the\nfirst place, has still no proper ground.\n\nNot even touching (1) for now, let me put some feelers out to see if we\ncan even figure how the rules or guidelines for command-line options\ndesign may look like.\n\n1. All options are divided into 2 classes: basic options and convenience\n   options.\n\n2. Minimalism. Every basic option should tweak exactly one aspect of\n   program behavior.\n\n3. Orthogonality. Every basic option should not \"imply\" any other\n   option, nor change the behavior of any other option.\n\n4. Reversibility. Every basic option should have a way to set it to any\n   supported value at any moment, including setting it back to its\n   default value.\n\n5. Grouping for convenience. A convenience option (usually with a short\n   syntax), should be semantically equivalent to an exact sequence of\n   basic options, as if it were substituted at the place of the\n   convenience option, and should not otherwise tweak program behavior.\n   I.e., a convenience option should be simple textual synonym for\n   particular sequence of basic options.\n\nPlease notice that in the above model basic option having a short form\nis formally considered to be a short convenience option that is a\nsynonym for long basic option.\n\nThere are obviously some other useful guidelines that could be defined,\nor some alternate approach could be chosen,but the primary point is that\nif we want a consistent UI, we do need some rules, and we need\nconvenient implementation of the model agreed upon, and then ensure that\nfrom all the designs that \"make sense\", only those that fit into\nunderlying model are accepted.\n\nThanks,\n-- Sergey Organov\n"},{"id":"439566","messageId":"xmqq5ytl2fvb.fsf@gitster.g","threadId":"56747","inReplyTo":"211025.86tuh5gtcc.gmgdl@evledraar.gmail.com","subject":"Re: changing the experimental 'git switch'","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-10-25T17:06:48Z","receivedAt":"2021-10-25T17:08:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n\n>> So, it probably makes sense to be more selective that what are\n>> sensibly copied and what are not.  Reflog most likely does not\n>> belong to the \"sensibly copyable\" set.  Tracking info most likely\n>> does.  Among various configuration in branch.OLD.*, there may be\n>> things like description that are not sensibly copyable.\n>\n> It is a bit weird, but the main problem is that we'll use it for UI such\n> as @{-1} or whatever in addition to things like \"x days ago\". So if you\n> copy a branch for some ad-hoc testing, and were just running such a\n> command you might expend it to work.\n\nThe event of new branch \"creation\" onward should be recorded to the\nreflog of the newly created branch.  As of X days ago, the new\nbranch did not even exist, so that is not a good excuse to copy the\nreflog.\n\nAlso @{-1} comes from the reflog of HEAD, which is different from\nwhat we are discussing.\n\n> For a user it also maps nicely to the mental model you'd have if you\n> copied two directories with the \"-p\" option to \"cp\", i.e. you'll be able\n> to run a \"find\" command on that checking mtime of N days ago and the\n> like.\n>\n> Maybe it still doesn't make sense for those cases just some thoughts on\n> UX edge cases.\n\nTo me, it makes no sense, with these analogies.  If I make a copy of\na file one month old with timestamp copied, I may appreciate that\nthe newly created copy hasn't yet been touched by looking at the old\ntimestamp, but that does not necessarily mean that I want to pretend\nthat the new file was there from that old date, or I want to pretend\nthat the last time the new file was edited before that was at an\neven old time.\n\nIf I were renaming a branch, that is a totally different story.  In\nthe mental model, the \"identity\" of the branch did not change, only\nthe label that I use to refer to it (called \"name\") has changed.\n\nBut I do not expect copying to split and give half the identity of\nthe original to the new one.\n\n"},{"id":"439579","messageId":"CAFQ2z_NBOC5sDSL6AjCe-5mPVhU1B_guJEsHwVT3=AK1aAt8UA@mail.gmail.com","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211148400.56@tvgsbejvaqbjf.bet","subject":"Re: [Summit topic] The state of getting a reftable backend working in git.git","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-10-25T19:00:23Z","receivedAt":"2021-10-25T19:00:41Z","isPatch":false,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Oct 21, 2021 at 1:56 PM Johannes Schindelin\n<Johannes.Schindelin@gmx.de> wrote:\n>\n> This session was led by Ævar Arnfjörð Bjarmason (on behalf of Han-Wen\n> Nienhuys, the driving force behind the reftable patches, who did not\n> attend the Summit). Supporting cast: Jonathan \"jrnieder\" Nieder, Johannes\n> \"Dscho\" Schindelin, Philip Oakley, Jeff \"Peff\" King, and Junio Hamano.\n>\n\nThanks Ævar for doing this. I wanted to be there, but I took a much\nneeded 2 week computer-less vacation .\n\n>..\n>      9.  Reftable has a set of files that go together. May want debugging tool\n>          to dump the content of a binary reftable file. But we can\n>          incrementally add those\n\n\nThe patch series includes a test-tool for dumping both individual\ntables and a stack of tables. It's not super-polished, but it gets the\njob done.\n\n$ touch a ; ~/vc/git/git add a; ~/vc/git/git commit -mx\n...\n\n$  ~/vc/git/bin-wrappers/test-tool  dump-reftable -t\n.git/reftable/0x000000000002-0x000000000002-327b23c6.ref\nref{refs/heads/main(2) val 1 ab21c324503544acca84eb55f5ee7dce24b23e15}\nlog{HEAD(2) Han-Wen Nienhuys <hanwen@google.com> 1635188263 0200\n0000000000000000000000000000000000000000 =>\nab21c324503544acca84eb55f5ee7dce24b23e15\n\ncommit (initial): x\n\n}\nlog{refs/heads/main(2) Han-Wen Nienhuys <hanwen@google.com> 1635188263 0200\n0000000000000000000000000000000000000000 =>\nab21c324503544acca84eb55f5ee7dce24b23e15\n\ncommit (initial): x\n\n}\n\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"439602","messageId":"211026.86wnm021ih.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"CAFQ2z_NBOC5sDSL6AjCe-5mPVhU1B_guJEsHwVT3=AK1aAt8UA@mail.gmail.com","subject":"Re: [Summit topic] The state of getting a reftable backend working in git.git","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-25T22:09:55Z","receivedAt":"2021-10-25T22:16:58Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Oct 25 2021, Han-Wen Nienhuys wrote:\n\n> On Thu, Oct 21, 2021 at 1:56 PM Johannes Schindelin\n> <Johannes.Schindelin@gmx.de> wrote:\n>>\n>> This session was led by Ævar Arnfjörð Bjarmason (on behalf of Han-Wen\n>> Nienhuys, the driving force behind the reftable patches, who did not\n>> attend the Summit). Supporting cast: Jonathan \"jrnieder\" Nieder, Johannes\n>> \"Dscho\" Schindelin, Philip Oakley, Jeff \"Peff\" King, and Junio Hamano.\n>>\n>\n> Thanks Ævar for doing this. I wanted to be there, but I took a much\n> needed 2 week computer-less vacation .\n\nNo problem, as is perhaps clear from the notes I had to hand-wave some\nquestions away since I didn't know about those things.\n\n>>..\n>>      9.  Reftable has a set of files that go together. May want debugging tool\n>>          to dump the content of a binary reftable file. But we can\n>>          incrementally add those\n>\n>\n> The patch series includes a test-tool for dumping both individual\n> tables and a stack of tables. It's not super-polished, but it gets the\n> job done.\n>\n> $ touch a ; ~/vc/git/git add a; ~/vc/git/git commit -mx\n> ...\n>\n> $  ~/vc/git/bin-wrappers/test-tool  dump-reftable -t\n> .git/reftable/0x000000000002-0x000000000002-327b23c6.ref\n> ref{refs/heads/main(2) val 1 ab21c324503544acca84eb55f5ee7dce24b23e15}\n> log{HEAD(2) Han-Wen Nienhuys <hanwen@google.com> 1635188263 0200\n> 0000000000000000000000000000000000000000 =>\n> ab21c324503544acca84eb55f5ee7dce24b23e15\n>\n> commit (initial): x\n>\n> }\n> log{refs/heads/main(2) Han-Wen Nienhuys <hanwen@google.com> 1635188263 0200\n> 0000000000000000000000000000000000000000 =>\n> ab21c324503544acca84eb55f5ee7dce24b23e15\n>\n> commit (initial): x\n>\n> }\n\nNeat.\n\nFrom memory I think the more general concern Philip Oakley was also\nexpressing (but maybe he'll chime in) could also be addressed by a tool\nthat just un-reftable-ifies a repository.\n\nI think such a thing would be useful, and I think we don't have that\nalready. Isn't the files backend or reftable usage now an \"init\"-time\nsetting.\n\nIt would be useful if for no other reason than to give user who are\nlooking at a repository that's weird somehow the ability to quickly\nmigrate 100% away from reftable, to see if it has any impact on whatever\nthey're seeing.\n\nI wanted to implement a \"git unpack-refs\" a while ago for \"pack-refs\",\njust to simulate some performance aspects of loose-refs without writing\nan ad-hoc \"ref exploder\" one-liner again.\n\nA migration tool would surely be pretty much that, no? I.e. we'd just\ncreate a .git/refs.migrate or whatever, then hold a lock on reftable,\nand in-place move .git/refs{.migrate,} (along with top-level files like\nHEAD et al, presumably...).\n\nMaybe there's more complexity I'm not considering than just the *.lock\ndance in .git/*, but if not such a tool could also convert freely\nbetween the two backends, so you could try refable out in an existing\ncheckout.\n"},{"id":"439609","messageId":"211026.86sfwo20kr.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"87h7d5yrxy.fsf@osv.gnss.ru","subject":"Re: changing the experimental 'git switch'","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-25T22:23:29Z","receivedAt":"2021-10-25T22:37:13Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Oct 25 2021, Sergey Organov wrote:\n\n> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n>\n> [...]\n>\n>> I really don't know, but I do think that the most viable path to a\n>> better UX for git is to consider its UX more holistically.\n>>\n>> To the extent that our UX is a mess I think it's mainly because we've\n>> ended up with an accumulation of behavior that made sense in isolation\n>> at the time, but which when combined presents bad or inconsistent UX to\n>> the user.\n>\n> Yep. Moreover, this practice of \"making sense\" being the primary\n> reasoning factor doesn't work very well even in isolation, for single\n> Git sub-commands. As there is no defined underlying UI model, or rules,\n> or even clear guidelines of how to properly design command-line options,\n> multiple authors, all having their own sense and having no common ground\n> to base their decisions on, inevitably produce some spaghetti UI.\n\nYes we're definitely lacking on the documentation front here at least,\nbut I do think we have quite a bit of consistency in the form of\nparse_options() users....\n\n> The UI model to be defined, provided we are serious about aiming at a\n> good design, in fact has at least 2 aspects to address:\n>\n> 1. Uniform top-level syntax of all the Git commands.\n\nhave have e.g. hash-object but nothing like hash_object, there's that at\nleast..., but also mktag, not make-tag, so....\n\n> 2. Uniform rules to handle command-line options.\n>\n> Being hard to produce simple yet flexible design by itself, the problem\n> is further complicated by the need to absorb as much of the existing UI\n> as reasonably possible.\n>\n> Once a model is defined though, we should be able to at least ensure new\n> designs fit the model, and then, over time, gradually replace legacy UIs\n> that currently don't fit.\n>\n> As a side-note, from this standpoint, discussing deep details of \"git\n> switch\" options, or even relevancy of introducing of \"git switch\" in the\n> first place, has still no proper ground.\n>\n> Not even touching (1) for now, let me put some feelers out to see if we\n> can even figure how the rules or guidelines for command-line options\n> design may look like.\n\nHaving hacked quite a bit on parse_options() recently, including quite a\nbit of unsubmitted work I've got some opinions in this area :)\n\nThat API is as close as we get to uniform UX in this area.\n\n> 1. All options are divided into 2 classes: basic options and convenience\n>    options.\n\nAre you thinking of things like \"git config --bool\" v.s. \"git config\n--type=bool\" (let's ignore that we discourage the former for now), or\nmore like \"common\" v.s. \"obscure\" ?\n\n> 2. Minimalism. Every basic option should tweak exactly one aspect of\n>    program behavior.\n\nGenerally, although for things like \"git log\" you quickly end up with\nwanting to have pseudo-mode options imply one thing or the other,\nsometimes for the better, sometimes wfor worse.\n\n> 3. Orthogonality. Every basic option should not \"imply\" any other\n>    option, nor change the behavior of any other option.\n\nYeah, generally.\n\n> 4. Reversibility. Every basic option should have a way to set it to any\n>    supported value at any moment, including setting it back to its\n>    default value.\n\nYeah, for sure, we're generally quite good at this with parse_options(),\nbut there's exceptions (particularly with callbacks).\n\n> 5. Grouping for convenience. A convenience option (usually with a short\n>    syntax), should be semantically equivalent to an exact sequence of\n>    basic options, as if it were substituted at the place of the\n>    convenience option, and should not otherwise tweak program behavior.\n>    I.e., a convenience option should be simple textual synonym for\n>    particular sequence of basic options.\n\nI think some examples for the above in terms of current git commands\nwould be quite helpful, I'm struggling to think of examples for some of\nthese.\n\n> Please notice that in the above model basic option having a short form\n> is formally considered to be a short convenience option that is a\n> synonym for long basic option.\n>\n> There are obviously some other useful guidelines that could be defined,\n> or some alternate approach could be chosen,but the primary point is that\n> if we want a consistent UI, we do need some rules, and we need\n> convenient implementation of the model agreed upon, and then ensure that\n> from all the designs that \"make sense\", only those that fit into\n> underlying model are accepted.\n\nThere was a recent discussion about cat-file option parsing semantics at\nhttps://lore.kernel.org/git/87tuhuikhf.fsf@evledraar.gmail.com/\n\nI have this unsubmitted (and updated from that discussion) patch to make\n\"cat-file\" help friendlier:\nhttps://github.com/avar/git/commit/bd32f57cd21\n\nI wonder what you think abut that new output v.s. the old.\n\nMore generally, I've wanted to have some mode for parse_options() for a\nwhile now to label a given option X as only going with option. We have\nOPT_CMDMODE() for things that are mutually exclusive with all other\noptions, but not anything like a OPT_SUBCMDMODE() or whatever (and\nsometimes such a thing would go with N \"top-level modes\", not just one).\n\nRight now you need to do that manually, see the usage_msg_opt[f]()\nverbosity at:\nhttps://github.com/avar/git/blob/avar/cat-file-usage-and-options-handling/builtin/cat-file.c#L679-L755\n\nI thing like that would be really useful, and would go a long way\ntowards consistent UX, as you could generate the sort of \"grouped help\"\nshown in the commit link above with it, as well as have things like:\n\n    git some-command --top-level-option --op<TAB>\n\nTab-complete only those --op* options that go with that\n--top-level-option.\n\nI guess what I'm saying is that I agree with you, but just think that\nincremental changes to these UX APIs is the most viable way forward.\n"},{"id":"439631","messageId":"CAFQ2z_OfWe53zh-Eu09M=7pxD4=AMiWrNvwqv_HB7NFNRX+dhw@mail.gmail.com","threadId":"56747","inReplyTo":"211026.86wnm021ih.gmgdl@evledraar.gmail.com","subject":"Re: [Summit topic] The state of getting a reftable backend working in git.git","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2021-10-26T08:12:06Z","receivedAt":"2021-10-26T08:12:21Z","isPatch":false,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Tue, Oct 26, 2021 at 12:16 AM Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n> From memory I think the more general concern Philip Oakley was also\n> expressing (but maybe he'll chime in) could also be addressed by a tool\n> that just un-reftable-ifies a repository.\n>\n> I think such a thing would be useful, and I think we don't have that\n> already. Isn't the files backend or reftable usage now an \"init\"-time\n> setting.\n>..\n> Maybe there's more complexity I'm not considering than just the *.lock\n> dance in .git/*, but if not such a tool could also convert freely\n> between the two backends, so you could try refable out in an existing\n> checkout.\n\nI added a convert-ref-storage command to the JGit command line client\nfor exactly this,\n\n$ jgit convert-ref-storage  -h\njgit convert-ref-storage [--format VAL] [--help (-h)] [--ssh [JSCH | APACHE]]\n\n --format VAL          : Format to convert to (reftable or refdir) (default:\n                         reftable)\n --help (-h)           : display this help text (default: true)\n --ssh [JSCH | APACHE] : Selects the built-in ssh library to use, JSch or\n                         Apache MINA sshd. (default: JSCH)\n\nSee here[1] for implementation. It's not safe for concurrent use with\nother git commands, but that's hardly a common use-case.\n\n[1] https://eclipse.googlesource.com/gerrit/jgit/jgit/+/1825a2230c06e7a6cbe23c69b63c3b7ecd2ceac6/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/file/FileRepository.java#806\n\n\n-- \nHan-Wen Nienhuys - Google Munich\nI work 80%. Don't expect answers from me on Fridays.\n--\n\nGoogle Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich\n\nRegistergericht und -nummer: Hamburg, HRB 86891\n\nSitz der Gesellschaft: Hamburg\n\nGeschäftsführer: Paul Manicle, Halimah DeLaine Prado\n"},{"id":"439654","messageId":"04ef1d38-8800-e260-b852-8ca86ef44d3c@iee.email","threadId":"56747","inReplyTo":"211026.86wnm021ih.gmgdl@evledraar.gmail.com","subject":"Re: [Summit topic] The state of getting a reftable backend working in git.git","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2021-10-26T15:51:52Z","receivedAt":"2021-10-26T15:52:12Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"Hi Han-Wen,\n\nOn 25/10/2021 23:09, Ævar Arnfjörð Bjarmason wrote:\n> On Mon, Oct 25 2021, Han-Wen Nienhuys wrote:\n>\n>> On Thu, Oct 21, 2021 at 1:56 PM Johannes Schindelin\n>> <Johannes.Schindelin@gmx.de> wrote:\n>>> This session was led by Ævar Arnfjörð Bjarmason (on behalf of Han-Wen\n>>> Nienhuys, the driving force behind the reftable patches, who did not\n>>> attend the Summit). Supporting cast: Jonathan \"jrnieder\" Nieder, Johannes\n>>> \"Dscho\" Schindelin, Philip Oakley, Jeff \"Peff\" King, and Junio Hamano.\n>>>\n>> Thanks Ævar for doing this. I wanted to be there, but I took a much\n>> needed 2 week computer-less vacation .\n> No problem, as is perhaps clear from the notes I had to hand-wave some\n> questions away since I didn't know about those things.\n>\n>>> ..\n>>>      9.  Reftable has a set of files that go together. May want debugging tool\n>>>          to dump the content of a binary reftable file. But we can\n>>>          incrementally add those\n>>\n>> The patch series includes a test-tool for dumping both individual\n>> tables and a stack of tables. It's not super-polished, but it gets the\n>> job done.\n>>\n>> $ touch a ; ~/vc/git/git add a; ~/vc/git/git commit -mx\n>> ...\n>>\n>> $  ~/vc/git/bin-wrappers/test-tool  dump-reftable -t\n>> .git/reftable/0x000000000002-0x000000000002-327b23c6.ref\n>> ref{refs/heads/main(2) val 1 ab21c324503544acca84eb55f5ee7dce24b23e15}\n>> log{HEAD(2) Han-Wen Nienhuys <hanwen@google.com> 1635188263 0200\n>> 0000000000000000000000000000000000000000 =>\n>> ab21c324503544acca84eb55f5ee7dce24b23e15\n>>\n>> commit (initial): x\n>>\n>> }\n>> log{refs/heads/main(2) Han-Wen Nienhuys <hanwen@google.com> 1635188263 0200\n>> 0000000000000000000000000000000000000000 =>\n>> ab21c324503544acca84eb55f5ee7dce24b23e15\n>>\n>> commit (initial): x\n>>\n>> }\n> Neat.\n>\n> From memory I think the more general concern Philip Oakley was also\n> expressing (but maybe he'll chime in) could also be addressed by a tool\n> that just un-reftable-ifies a repository.\n\nI was remembering my early exploits with trying to understand Git and\nall the web references tended to refer to the file system implementation\nof refs, in a reverse-specification sort of way.\n\nrefs can be hard to comprehend especially when DWIMmery is also\ninvolved, and the user hasn't yet fully understood all the git commands\nthat can affect and read refs\n\n>\n> I think such a thing would be useful, and I think we don't have that\n> already. Isn't the files backend or reftable usage now an \"init\"-time\n> setting.\n>\n> It would be useful if for no other reason than to give user who are\n> looking at a repository that's weird somehow the ability to quickly\n> migrate 100% away from reftable, to see if it has any impact on whatever\n> they're seeing.\n\nI remember the usefulness of the data_dumper when I was looking at the\nearly Git Visual Studio project generators and the like.\n\nHaving a similar dumper for the refs would be useful. I can see it being\nsplit between a dumper for repos with just a few refs and one that can\ncope with the thousands of refs scaling problem (some sort of selectivity?)\n>\n> I wanted to implement a \"git unpack-refs\" a while ago for \"pack-refs\",\n> just to simulate some performance aspects of loose-refs without writing\n> an ad-hoc \"ref exploder\" one-liner again.\n>\n> A migration tool would surely be pretty much that, no? I.e. we'd just\n> create a .git/refs.migrate or whatever, then hold a lock on reftable,\n> and in-place move .git/refs{.migrate,} (along with top-level files like\n> HEAD et al, presumably...).\n\nI could see an option that puts the exploded refs 'somewhere else' just\nfor inspection by a confused user...\n>\n> Maybe there's more complexity I'm not considering than just the *.lock\n> dance in .git/*, but if not such a tool could also convert freely\n> between the two backends, so you could try refable out in an existing\n> checkout.\nPhilip\n"},{"id":"439671","messageId":"20211026201448.GA29480@dcvr","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211144490.56@tvgsbejvaqbjf.bet","subject":"scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas]","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2021-10-26T20:14:49Z","receivedAt":"2021-10-26T20:14:50Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> * Test suite is slow. Shell scripts and process forking.\n> \n>    * What if we had a special shell that interpreted the commands in a\n>      single process?\n> \n>    * Even Git commands like rev-parse and hash-object, as long as that’s\n>      not the command you’re trying to test\n\nThis is something I've wanted in a very long time as a scripter.\nfast-import has been great over the years, as is\n\"cat-file --batch(-check)\", but there's gaps should be filled\n(preferably without fragile linkage of shared libraries into a\nscript process)\n\n>    * Dscho wants to slip in a C-based solution\n> \n>    * Jonathan tan commented: going back to your custom shell for tests\n>      idea, one thing we could do is have a custom command that generates\n>      the repo commits that we want (and that saves process spawns and\n>      might make the tests simpler too)\n\nPerhaps a not-seriously-proposed patch from 2006 could be\nmodernized for our now-libified internals:\n\nhttps://yhbt.net/lore/git/Pine.LNX.4.64.0602232229340.3771@g5.osdl.org/\n\n>       * We could replace several “setup repo” steps with “git fast-import”\n>         instead.\n> \n>    * Dscho measured: 0.5 sec - 30 sec in setup steps. Can use fast-import,\n>      or can make a new format that helps us set up the test scenario\n\n0.5s - 30s across the whole suite or individual tests?\n\nHaving a way to disable fsync globally should further improve\nthings, especially for people on slower storage.  libeatmydata\nis available, but perhaps not widely available/known.\n\n>    * Elijah: test-lib-functions helpers could be built ins\n"},{"id":"439711","messageId":"88515746-6c6c-1341-993f-e079d23e67f6@free.fr","threadId":"56747","inReplyTo":"211022.86r1cdjfe2.gmgdl@evledraar.gmail.com","subject":"Re: [Summit topic] Documentation (translations, FAQ updates, new user-focused, general improvements, etc.)","fromName":"Jean-Noël Avila","fromEmail":"jn.avila@free.fr","sentAt":"2021-10-27T07:02:05Z","receivedAt":"2021-10-27T07:02:12Z","isPatch":false,"sender":{"key":"jn.avila@free.fr","avatar":"https://avatars.githubusercontent.com/u/156172?v=4"},"body":"On Fri, Oct 22 2021, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Fri, Oct 22 2021, Jean-Noël Avila wrote:\n> \n>> I'm sorry that my presence at this meeting could have helped a bit for\n>> some subtopics.\n>>\n>> Le 21/10/2021 à 13:56, Johannes Schindelin a écrit :\n>>> This session was led by brian m. carlson. Supporting cast: Jeff \"Peff\"\n>>> King, Ævar Arnfjörð Bjarmason, Taylor Blau, Philip Oakley, Emily Shaffer,\n>>> CB Bailey, and Jonathan \"jrnieder\" Nieder.\n>>>\n>>> Notes:\n>>>\n>>>  1. Background: answering on StackOverflow, other avenues for user questions,\n>>>     even users from very large companies\n>>>\n>>>  2. How can we improve documentation?\n>>>\n>>>  3. Maybe even think about translating docs such as FAQs\n>>>\n>>>  4. Peff: there’s an effort to translate manpages\n>>>\n>>>     1. brian: Saw an announcement, haven’t seen what came of it\n>>\n>> The effort is still ongoing. Unfortunately, there aren't much outputs\n>> from it, only the inclusion on git-scm.com.\n>>\n>> A proposition was sent for Debian packages.\n>>\n>> I'm open for any help in packaging what's already available for whatever\n>> useful.\n>>\n>>\n>> For some statistics\n>>\n>> * there are 23 po files, \"pt_BR\" fully translated, \"fr\" half translated,\n>> \"de\" one third; most other languages have not really started (the\n>> portion already translated was made automatically for unmodified strings).\n>>\n>> * not all pages are included for translation; most porcelain pages\n>> available on git-scm.com are included, but for instance, not the config\n>> parts or the guides. That's already 10,687 source segments and 206,700\n>> source words, which is a volume similar to \"Crime and Punishment\" by\n>> Dostoyevsky. And it really looks like an punishment for most apprentice\n>> translators willing to start.\n>>\n>> In order to lower the barrier to translators, the project is relying on\n>> weblate: https://hosted.weblate.org/projects/git-manpages/translations/\n>> while still retaining a \"Developer's Certificate of Origin\".\n>>\n>>\n>>>\n>>>     2. Peff: Some translated pages are live on git-scm.com (a github repo with\n>>>        translations)\n>>\n>> For instance, git init manpages is already available in 8 languages.\n>>\n>>\n>>>\n>>>     3. Ævar: It uses a third-party tool (po4a) that uses gettext by making each\n>>>        paragraph a translated string. So it’s the same workflow as translating\n>>>        code changes\n>>\n>> Asciidoc support is \"co-developed\" in po4a in parallel with the\n>> translation: I fix bugs when they are found in the po files.\n>>\n>>>     4. Taylor: https://github.com/jnavila/git-manpages-l10n\n>>\n>>\n>> If it looks too personal, it can be moved into the git organization.\n>>\n>>\n>>>\n>>>  5. Philip Oakley: I see manpages used as reference material instead of\n>>>     educational documents\n>>>\n>>>\n>>>     12. In stackoverflow you can see how people answer questions, how much less\n>>>         existing background they assume\n>>\n>> Version control is usually already in the culture of most users\n>> (writers, engineers in other fields have come to use them some 10 years\n>> ago). What their questions usually boil down to is: how can I use and\n>> customize git features for my field of expertise. When software editors\n>> include git support in their applications, it is usually with severed\n>> functions and users quickly have to get back to plain git when they want\n>> a little more.\n>>\n>> General rules can help start up with a new customization, but at some\n>> point, the customization is specific to the tool. A library of\n>> application oriented customizations, help files and FAQs may be of\n>> interest. Some customizations already exist, sometimes with errors\n>> (meaning the maintainer of the customization has not fully understood\n>> how git works) but they are scattered.\n> \n> I'd very much support this living in-tree just as the po/* directory\n> already does. I.e. periodically pulled down.\n> \n> There are many OS's that have something like \"apt install\n> manpages-<lang>\", so if we had these available they could be much more\n> useful to users.\n> \n> E.g. I see I can \"apt install manpages-pt\", but if you're a Portuguese\n> speaker you probably won't chase down some third-party addition of\n> Portuguese manpages, and even if they're in Debian other package\n> maintainers might not add them if they're not in the \"main\" package etc.\n> \n> What's standing in the way of us treating this in the same way as the\n> po/* directory, if anything?\n> \n\n * I'm using Asciidoctor to process manpages, because it processes\ndirectly the asciidoc source, whereas using the intermediate docbook\nstage stops when some included files are missing (very common problem\nwith such long running translations and included files are not yet\ntranslated).\n * I had understood from my initial presentation that adding this\ncontent to \"common\" Git was not desirable. The question was more to make\nthe repo appear under the git organization on GitHub, not a full\nintegration.\n\n"},{"id":"439745","messageId":"YXkS85G5ujqxVf0M@coredump.intra.peff.net","threadId":"56747","inReplyTo":"211022.86r1cdjfe2.gmgdl@evledraar.gmail.com","subject":"Re: [Summit topic] Documentation (translations, FAQ updates, new user-focused, general improvements, etc.)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-10-27T08:50:59Z","receivedAt":"2021-10-27T08:51:02Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Oct 22, 2021 at 04:31:46PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> I'd very much support this living in-tree just as the po/* directory\n> already does. I.e. periodically pulled down.\n\nJust a bit of a tangent here, since weblate was mentioned earlier.\n\nI'd caution a bit against pulling the history generated by weblate\ndirectly. It's pretty sub-optimal from a Git perspective: you have a\nbunch of big .po files and then a ton of little commits changing one or\na handful of lines.\n\nSo the \"logical\" size of the repository (the sum of the actual object\nsizes) ends up growing quite a bit. Deltas can help with the on-disk\nsize, but:\n\n  - lots of operations scale with the logical size. The client-side\n    index-pack of a clone, for instance, but also everyday stuff like\n    \"git log -S\".\n\n  - empirically we don't do a great job of finding these. See below for\n    some numbers.\n\nFor instance, take https://github.com/phpmyadmin/phpmyadmin, a\nrepository which uses weblate (I don't mean to pick on them; it's just a\nrepo whose weblate-related packing I've looked into before). A fresh\nclone is 1.3GB. If you do an aggressive repack, you can get it down to\nabout 550MB. But there's still tons of logical data. Running:\n\n  git cat-file --batch-all-objects --batch-check='%(objectsize) %(objectsize:disk)' |\n  perl -alne '\n    $logical += $F[0]; $disk += $F[1];\n    END { print \"$logical / $disk = \" . $logical / $disk }\n  '\n\nshows that there's over 70GB of logical data. It gets an impressive\n156:1 compression ratio (for comparison, \"normal\" repos like linux.git\nand git.git are around 40-60x in my experience).\n\nIf you split it up by directory, like this:\n\n  git rev-list --objects --all --no-object-names -- po |\n  git cat-file --batch-check='%(objectsize)' |\n  perl -lne '$total += $_; END { print $total }'\n\nyou'll see that po/ accounts for almost 60GB of that logical size.\n\nWe face some of that in our current po/, too. They're big files, and\nthat's the nature of the problem space. But our current ones tend to be\nedited by taking a pass over the whole file, rather than the one-liners\nthat a web-based workflow encourages.\n\nTo be clear, I'm not arguing against weblate in general. It's cool that\nit makes it easier for people to contribute to translations. But I think\nit has an outsized impact on size and performance compared to the rest\nof the repository. That's a big price to pay for carrying the history\nin-tree.\n\nObviously one option there is to squash the po/ history before pulling\nit in. The weblate commit messages themselves aren't that useful. I'm\nnot actually sure if jnavila's work so far has been using weblate. The\ncommits in his git-html-l10n are much coarser than what I see in\nphpmyadmin, for example (so maybe he's doing similar squashing already).\n\n-Peff\n"},{"id":"439809","messageId":"871r469u3x.fsf@osv.gnss.ru","threadId":"56747","inReplyTo":"211026.86sfwo20kr.gmgdl@evledraar.gmail.com","subject":"Re: changing the experimental 'git switch'","fromName":"Sergey Organov","fromEmail":"sorganov@gmail.com","sentAt":"2021-10-27T18:54:10Z","receivedAt":"2021-10-27T18:54:20Z","isPatch":false,"sender":{"key":"sorganov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8501568?v=4"},"body":"Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n\n> On Mon, Oct 25 2021, Sergey Organov wrote:\n>\n>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n>>\n>> [...]\n>>\n>>> I really don't know, but I do think that the most viable path to a\n>>> better UX for git is to consider its UX more holistically.\n>>>\n>>> To the extent that our UX is a mess I think it's mainly because we've\n>>> ended up with an accumulation of behavior that made sense in isolation\n>>> at the time, but which when combined presents bad or inconsistent UX to\n>>> the user.\n>>\n>> Yep. Moreover, this practice of \"making sense\" being the primary\n>> reasoning factor doesn't work very well even in isolation, for single\n>> Git sub-commands. As there is no defined underlying UI model, or rules,\n>> or even clear guidelines of how to properly design command-line options,\n>> multiple authors, all having their own sense and having no common ground\n>> to base their decisions on, inevitably produce some spaghetti UI.\n>\n> Yes we're definitely lacking on the documentation front here at least,\n> but I do think we have quite a bit of consistency in the form of\n> parse_options() users....\n\nYes, parse_potions() is definitely a good step in the right direction.\nIt's not the magic pill though, as even if it gives all the tools for\nproper options parsing and design, it still can't prevent all the ugly\nhacks applied to the resulting program state after options are already\nparsed.\n\n>\n>> The UI model to be defined, provided we are serious about aiming at a\n>> good design, in fact has at least 2 aspects to address:\n>>\n>> 1. Uniform top-level syntax of all the Git commands.\n>\n> have have e.g. hash-object but nothing like hash_object, there's that at\n> least..., but also mktag, not make-tag, so....\n\nNaming conventions are nice to have too, but here I was more after\nsomething like \"man git\" starting with more restrictive synopsis than it\nis now, say:\n\nSYNOPSIS\n\n  git [COMMON_OPTIONS] [OBJECT] [OBJ_OPTIONS] [COMMAND] [ARGS]\n\n  OBJECT := { branch, tag, stash, ... }\n\n  COMMAND := OBJECT-dependent\n  ...\n\nSo that, say, removal of file, tag, branch, etc., all were similar, say:\n\n    git file delete\n    git branch delete\n    git tag delete\n    git submodule delete\n\nA measures to short-cut these full forms could probably be introduced,\nso that, say, \"git delete\" is in fact parsed as \"git file delete\", and\nthen synonyms could be introduces, so that current \"git rm\" is still\naccepted and becomes \"git file delete\" after parsing.\n\nAs has been mentioned elsewhere, it'd be nice to get some expertise in\nthe field of syntax and semantics of textual UIs to work on a suitable\ndesign. I'm none of an expert in the field, just someone interested in\nthe topic.\n\n>\n>> 2. Uniform rules to handle command-line options.\n>>\n>> Being hard to produce simple yet flexible design by itself, the problem\n>> is further complicated by the need to absorb as much of the existing UI\n>> as reasonably possible.\n>>\n>> Once a model is defined though, we should be able to at least ensure new\n>> designs fit the model, and then, over time, gradually replace legacy UIs\n>> that currently don't fit.\n>>\n>> As a side-note, from this standpoint, discussing deep details of \"git\n>> switch\" options, or even relevancy of introducing of \"git switch\" in the\n>> first place, has still no proper ground.\n>>\n>> Not even touching (1) for now, let me put some feelers out to see if we\n>> can even figure how the rules or guidelines for command-line options\n>> design may look like.\n>\n> Having hacked quite a bit on parse_options() recently, including quite a\n> bit of unsubmitted work I've got some opinions in this area :)\n\nSure as hell you have! I only touched a little bit of it, and even that\nraised some opinions ;-)\n\n>\n> That API is as close as we get to uniform UX in this area.\n\nI'm all for the continuous work in this area, and thank you for taking\nso much care of it.\n\n>\n>> 1. All options are divided into 2 classes: basic options and convenience\n>>    options.\n>\n> Are you thinking of things like \"git config --bool\" v.s. \"git config\n> --type=bool\" (let's ignore that we discourage the former for now), or\n> more like \"common\" v.s. \"obscure\" ?\n\nNeither. Basic options are to cover all the needed functionality, and\nconvenience options are to have most useful combinations of basic\noptions handy in a short form.\n\n>\n>> 2. Minimalism. Every basic option should tweak exactly one aspect of\n>>    program behavior.\n>\n> Generally, although for things like \"git log\" you quickly end up with\n> wanting to have pseudo-mode options imply one thing or the other,\n> sometimes for the better, sometimes wfor worse.\n\nIn this model this is covered exactly by \"convenience options\". The\nprimary difference to the current status quo being that they don't\n\"imply\" anything, or \"defaults\" anything, whatever that might mean. They\nare simple textual synonyms for a set of basic options.\n\nUser will mostly use convenience options, turning to basic option(s)\nonly when they need to achieve something very specific and not that\nusual, or for scripting, or for aliasing.\n\n>\n>> 3. Orthogonality. Every basic option should not \"imply\" any other\n>>    option, nor change the behavior of any other option.\n>\n> Yeah, generally.\n>\n>> 4. Reversibility. Every basic option should have a way to set it to any\n>>    supported value at any moment, including setting it back to its\n>>    default value.\n>\n> Yeah, for sure, we're generally quite good at this with parse_options(),\n> but there's exceptions (particularly with callbacks).\n\nYes, parse_options() is definitely a step in the right direction. Though\ndo I already repeat myself?\n\n>\n>> 5. Grouping for convenience. A convenience option (usually with a short\n>>    syntax), should be semantically equivalent to an exact sequence of\n>>    basic options, as if it were substituted at the place of the\n>>    convenience option, and should not otherwise tweak program behavior.\n>>    I.e., a convenience option should be simple textual synonym for\n>>    particular sequence of basic options.\n>\n> I think some examples for the above in terms of current git commands\n> would be quite helpful, I'm struggling to think of examples for some of\n> these.\n\nOK, let's try something from \"git log\".\n\n      --first-parent\n           Follow only the first parent commit upon seeing a merge commit. [...]\n\n           This option also changes default diff format for merge commits to first-parent, see\n           --diff-merges=first-parent for details.\n\n\nThis violates the model as it changes both how commits are selected\n(changes program behavior) and changes some default governed by another\noption.\n\nTo adhere to the model, it might rather have been:\n\n   --follow-first-parent\n      Follow only the first parent commit upon seeing a merge commit.\n\n   --first-parent\n      Synonym for --follow-first-parent --diff-merges=first-parent\n\nHere, --follow-first-parent and --diff-merges=first-parent are both\nbasic options, while --first-parent is convenience option. Please notice\nhow --first-parent is described entirely in terms of other options,\nwithout anything else, so, saying:\n\n   git log --first-parent\n\nis *exactly* the same as saying:\n\n   git log --follow-first-parent --diff-merges=first-parent\n\nonly shorter. [ OPT_SYNONYM()? ]\n\nIn this model all the functionality is to be covered by orthogonal\ngood-behaving basic options, and then convenience options are defined to\nbe like that, convenience, adding zero essential functionality.\n\nThis would force an author of an option suitable for their current work\nat hand to think and design suitable additional *basic* option(s) first,\nand only then define new convenience option(s) in terms of basic\noption(s) as needed.\n\n>\n>> Please notice that in the above model basic option having a short form\n>> is formally considered to be a short convenience option that is a\n>> synonym for long basic option.\n>>\n>> There are obviously some other useful guidelines that could be defined,\n>> or some alternate approach could be chosen,but the primary point is that\n>> if we want a consistent UI, we do need some rules, and we need\n>> convenient implementation of the model agreed upon, and then ensure that\n>> from all the designs that \"make sense\", only those that fit into\n>> underlying model are accepted.\n>\n> There was a recent discussion about cat-file option parsing semantics at\n> https://lore.kernel.org/git/87tuhuikhf.fsf@evledraar.gmail.com/\n>\n> I have this unsubmitted (and updated from that discussion) patch to make\n> \"cat-file\" help friendlier:\n> https://github.com/avar/git/commit/bd32f57cd21\n>\n> I wonder what you think abut that new output v.s. the old.\n\n[Un]fortunately I'm not familiar with cat-file at all, and after reading\na few lines of the discussion you've referenced made me think I don't\nwant to, sorry. In general, I do think it's nice you are working hard on\nthe interfaces.\n\n>\n> More generally, I've wanted to have some mode for parse_options() for a\n> while now to label a given option X as only going with option. We have\n> OPT_CMDMODE() for things that are mutually exclusive with all other\n> options, but not anything like a OPT_SUBCMDMODE() or whatever (and\n> sometimes such a thing would go with N \"top-level modes\", not just\n> one).\n\nTo me this looks like attempts to cover as much of the existing\nuse-cases for options in Git as possible, be them good or bad. This is\nvery nice to have anyway, but otherwise is orthogonal to the task of\ndefining simple interface model that'd be targeted at eventually\nobsolete such support.\n\n>\n> Right now you need to do that manually, see the usage_msg_opt[f]()\n> verbosity at:\n> https://github.com/avar/git/blob/avar/cat-file-usage-and-options-handling/builtin/cat-file.c#L679-L755\n>\n> I thing like that would be really useful, and would go a long way\n> towards consistent UX, as you could generate the sort of \"grouped help\"\n> shown in the commit link above with it, as well as have things like:\n>\n>     git some-command --top-level-option --op<TAB>\n>\n> Tab-complete only those --op* options that go with that\n> --top-level-option.\n\nProbably this --top-level-option should better have been sub-command\nrather than option in proper UI design. An option that changes semantics\nso much that a bunch of options is to be replaced with something else\nshould probably be simply prohibited.\n\nIn general, it's pain in the ass to handle options dependencies in\nuniversal APIs such as parse_options(), so the best way of handling this\nis to (eventually) get rid of the dependencies. Yeah, as usual, that's\nsimpler to say than to do, I know.\n\n>\n> I guess what I'm saying is that I agree with you, but just think that\n> incremental changes to these UX APIs is the most viable way forward.\n\nNo objections. My primary point in this discussion is rather that\nincremental changes only work fine when there is defined target that\ngoverns the direction of the steps being taken, and as far as I can see\nwe still have no such target, or maybe even no agreement on significance\nof having one.\n\nThat said, the unified parse_options() being used everywhere will\ndefinitely simplify transition to an advanced design in the future,\nshould such transition be ever attempted.\n\nBTW, I didn't try hard, but talking about incremental changes, is it\npossible to convert part(s) of handle_revision_opt() to the\nparse_options() API? For example, diff_merges_parse_opts() is right now\nmostly isolated part of handle_revision_opt(). Is it feasible to convert\nit to use parse_options()?\n\nThanks,\n-- Sergey Organov\n"},{"id":"439871","messageId":"dac200db-fc5d-41da-668e-853a2f4f02a2@iee.email","threadId":"56747","inReplyTo":"CAFQ2z_OfWe53zh-Eu09M=7pxD4=AMiWrNvwqv_HB7NFNRX+dhw@mail.gmail.com","subject":"Re: [Summit topic] The state of getting a reftable backend working in git.git","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2021-10-28T14:17:37Z","receivedAt":"2021-10-28T14:17:41Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"On 26/10/2021 09:12, Han-Wen Nienhuys wrote:\n> On Tue, Oct 26, 2021 at 12:16 AM Ævar Arnfjörð Bjarmason\n> <avarab@gmail.com> wrote:\n>> From memory I think the more general concern Philip Oakley was also\n>> expressing (but maybe he'll chime in) could also be addressed by a tool\n>> that just un-reftable-ifies a repository.\n>>\n>> I think such a thing would be useful, and I think we don't have that\n>> already. Isn't the files backend or reftable usage now an \"init\"-time\n>> setting.\n>> ..\n>> Maybe there's more complexity I'm not considering than just the *.lock\n>> dance in .git/*, but if not such a tool could also convert freely\n>> between the two backends, so you could try refable out in an existing\n>> checkout.\n> I added a convert-ref-storage command to the JGit command line client\n> for exactly this,\n>\n> $ jgit convert-ref-storage  -h\n> jgit convert-ref-storage [--format VAL] [--help (-h)] [--ssh [JSCH | APACHE]]\n>\n>  --format VAL          : Format to convert to (reftable or refdir) (default:\n>                          reftable)\n>  --help (-h)           : display this help text (default: true)\n>  --ssh [JSCH | APACHE] : Selects the built-in ssh library to use, JSch or\n>                          Apache MINA sshd. (default: JSCH)\n>\n> See here[1] for implementation. It's not safe for concurrent use with\n> other git commands, but that's hardly a common use-case.\n>\n> [1] https://eclipse.googlesource.com/gerrit/jgit/jgit/+/1825a2230c06e7a6cbe23c69b63c3b7ecd2ceac6/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/file/FileRepository.java#806\n\nUseful to know that there is a method, though I was distinguishing the\ninspection of the data, from the conversion between (in-use) storage types.\n\nMy request was more about having an easy access ramp for on-boarding new\nusers (who are likely to visualise the file system analogy better) than\nfor experienced Git admins and developers who may need to convert and\ninteract with the data.\n\nSome aspects (in the wider domain) may be the extending of the\ndocumentation to help users with understanding the ref manipulation and\nthe meanings of the values.\n-- \n\nPhilip\n>\n\n"},{"id":"440134","messageId":"211030.86ee8246hy.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"20211026201448.GA29480@dcvr","subject":"Re: scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas]","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-30T19:58:36Z","receivedAt":"2021-10-30T20:12:14Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Oct 26 2021, Eric Wong wrote:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n>> * Test suite is slow. Shell scripts and process forking.\n>> \n>>    * What if we had a special shell that interpreted the commands in a\n>>      single process?\n>> \n>>    * Even Git commands like rev-parse and hash-object, as long as that’s\n>>      not the command you’re trying to test\n>\n> This is something I've wanted in a very long time as a scripter.\n> fast-import has been great over the years, as is\n> \"cat-file --batch(-check)\", but there's gaps should be filled\n> (preferably without fragile linkage of shared libraries into a\n> script process)\n>\n>>    * Dscho wants to slip in a C-based solution\n>> \n>>    * Jonathan tan commented: going back to your custom shell for tests\n>>      idea, one thing we could do is have a custom command that generates\n>>      the repo commits that we want (and that saves process spawns and\n>>      might make the tests simpler too)\n>\n> Perhaps a not-seriously-proposed patch from 2006 could be\n> modernized for our now-libified internals:\n\nI think something very short of a \"C-based solution\" could give us most\nof the wins here. Johannes was probably thinking of the scripting being\nslow on Windows aspect of it.\n\nBut the main benefit of hypothetical C-based testing is that you can\nconnect it to the dependency tree we have in the Makefile, and only\nre-run tests for code you needed to re-compile.\n\nSo e.g. we don't need to run tests that invoke \"git tag\" if the\ndependency graph of builtin/tag.c didn't change.\n\nWith COMPUTE_HEADER_DEPENDENCIES we've got access to that dependency\ninformation for our C code.\n\nWith trace2 we could record an initial test run, and know which built-in\ncommands are executed by which tests (even down to the sub-test level).\n\nConnecting these two means that we can find all tests that say run \"git\nfsck\", and if builtin/fsck.c is the only thing that changed in an\ninteractive rebase, that's the only tests we need to run.\n\nOf course changes to things like cache.h or t/test-lib.sh would spoil\nthat cache entirely, but pretty much the same is true for re-compiling\nthings now, so would changing say builtin/init-db.c, as almost every\ntest does a \"git init\" somewhere.\n\nBut I think that approch is viable, and should take us from a huge\nhypothetical project like \"rewrite all the tests in C\" to something\nthat's a viable weekend hacking project for someone who's interested.\n"},{"id":"440307","messageId":"nycvar.QRO.7.76.6.2111021446140.56@tvgsbejvaqbjf.bet","threadId":"56747","inReplyTo":"20211026201448.GA29480@dcvr","subject":"Re: scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas]","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-11-02T13:52:54Z","receivedAt":"2021-11-02T13:53:03Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Eric,\n\nOn Tue, 26 Oct 2021, Eric Wong wrote:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> > * Test suite is slow. Shell scripts and process forking.\n> >\n> >    * What if we had a special shell that interpreted the commands in a\n> >      single process?\n> >\n> >    * Even Git commands like rev-parse and hash-object, as long as that’s\n> >      not the command you’re trying to test\n>\n> This is something I've wanted in a very long time as a scripter.\n> fast-import has been great over the years, as is\n> \"cat-file --batch(-check)\", but there's gaps should be filled\n> (preferably without fragile linkage of shared libraries into a\n> script process)\n\nThe conclusion reached at the Summit seemed to be that we don't want to\nget into that rabbit hole. We might very well end up maintaining a\nPOSIX-compatible shell inside Git. Definitely out of scope.\n\n> >    * Dscho wants to slip in a C-based solution\n> >\n> >    * Jonathan tan commented: going back to your custom shell for tests\n> >      idea, one thing we could do is have a custom command that generates\n> >      the repo commits that we want (and that saves process spawns and\n> >      might make the tests simpler too)\n>\n> Perhaps a not-seriously-proposed patch from 2006 could be\n> modernized for our now-libified internals:\n>\n> https://yhbt.net/lore/git/Pine.LNX.4.64.0602232229340.3771@g5.osdl.org/\n\nThanks for digging that out. I had looked for it multiple times over the\nyears, but searched using the wrong search terms.\n\nHowever, as you can see, it went nowhere. Probably the (implicit)\nconclusion was the same as above.\n\n> >       * We could replace several “setup repo” steps with “git fast-import”\n> >         instead.\n> >\n> >    * Dscho measured: 0.5 sec - 30 sec in setup steps. Can use fast-import,\n> >      or can make a new format that helps us set up the test scenario\n>\n> 0.5s - 30s across the whole suite or individual tests?\n\nThat was just vague recollection, but it was for setup steps, i.e. the\ninitial test cases that do not even test Git's functionality but merely\nwant to set up a repository/worktree for the subsequent test cases to play\nwith.\n\n> Having a way to disable fsync globally should further improve\n> things, especially for people on slower storage.  libeatmydata\n> is available, but perhaps not widely available/known.\n\nWhat was missing from the notes was the crucial fact that I did this on\nWindows, i.e. a platform that is pretty darned good at multi-tasking\n(something with which Linux has historically struggled a bit), but not so\ngood at spawning wholesale processes.\n\nSo the problem really is that calling, say, `git commit` in a `for\n$(test_seq 100)` loop is ridiculously expensive.\n\nEven rewriting those setup test cases to something as verbose as a\n`fast-import` stream accelerates them like you wouldn't believe.\n\nI even thought I threw out the idea of implementing a test helper that\ncould turn the output of `git log --graph --oneline` into a branch\nreplicating that structure, but it might have gotten lost in the noise.\n\nI doubt that my test suite-centered commentary is very helpful for your\nuse cases, though.\n\nCiao,\nDscho\n"},{"id":"440363","messageId":"211103.864k8t1sma.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"211030.86ee8246hy.gmgdl@evledraar.gmail.com","subject":"test suite speedups via some not-so-crazy ideas (was: scripting speedups[...])","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-03T09:24:57Z","receivedAt":"2021-11-03T09:44:02Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sat, Oct 30 2021, Ævar Arnfjörð Bjarmason wrote:\n\n> On Tue, Oct 26 2021, Eric Wong wrote:\n>\n>> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n>>> * Test suite is slow. Shell scripts and process forking.\n>>> \n>>>    * What if we had a special shell that interpreted the commands in a\n>>>      single process?\n>>> \n>>>    * Even Git commands like rev-parse and hash-object, as long as that’s\n>>>      not the command you’re trying to test\n>>\n>> This is something I've wanted in a very long time as a scripter.\n>> fast-import has been great over the years, as is\n>> \"cat-file --batch(-check)\", but there's gaps should be filled\n>> (preferably without fragile linkage of shared libraries into a\n>> script process)\n>>\n>>>    * Dscho wants to slip in a C-based solution\n>>> \n>>>    * Jonathan tan commented: going back to your custom shell for tests\n>>>      idea, one thing we could do is have a custom command that generates\n>>>      the repo commits that we want (and that saves process spawns and\n>>>      might make the tests simpler too)\n>>\n>> Perhaps a not-seriously-proposed patch from 2006 could be\n>> modernized for our now-libified internals:\n>\n> I think something very short of a \"C-based solution\" could give us most\n> of the wins here. Johannes was probably thinking of the scripting being\n> slow on Windows aspect of it.\n>\n> But the main benefit of hypothetical C-based testing is that you can\n> connect it to the dependency tree we have in the Makefile, and only\n> re-run tests for code you needed to re-compile.\n>\n> So e.g. we don't need to run tests that invoke \"git tag\" if the\n> dependency graph of builtin/tag.c didn't change.\n>\n> With COMPUTE_HEADER_DEPENDENCIES we've got access to that dependency\n> information for our C code.\n>\n> With trace2 we could record an initial test run, and know which built-in\n> commands are executed by which tests (even down to the sub-test level).\n>\n> Connecting these two means that we can find all tests that say run \"git\n> fsck\", and if builtin/fsck.c is the only thing that changed in an\n> interactive rebase, that's the only tests we need to run.\n>\n> Of course changes to things like cache.h or t/test-lib.sh would spoil\n> that cache entirely, but pretty much the same is true for re-compiling\n> things now, so would changing say builtin/init-db.c, as almost every\n> test does a \"git init\" somewhere.\n>\n> But I think that approch is viable, and should take us from a huge\n> hypothetical project like \"rewrite all the tests in C\" to something\n> that's a viable weekend hacking project for someone who's interested.\n\nFirst to outline some goals: I think saying we'd like to speed up\nscripts is really getting into the weeds.\n\nSurely we'd like to speed up test runs, and generally speaking our test\nsuite can be parallelized, and it mostly doesn't matter if it runs on\nyour computer or other people's computers, as long as it runs your\ncode. So:\n\n 1. Even for contributors that have a slow system they could benefit from\n    the hosted CI (on GitHub or wherever else) being faster.\n\n 2. Our CI takes around 30-60m to finish.\n\n 3. That CI time is almost entirely something that could be sped up by\n    throwing hardware at it.\n\n 4. We're currently using \"Dv2 and DSv2-series\" hosted runners\n    (https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners)\n    we have quite a few people on-list who work for the\n    company/companies involved.\n\n    Is it within the realm of possibility to get more CI resources\n    assigned to git/git's organization network?\n\n 5. Or, is there willingness to host/pay for hosted runners from\n    someone?\n\n    Not wearing PLC hat I'd think that we could speed that up a lot with\n    some reasonable money spending, and if pushing to CI made CI run in\n    3-5m instead of 60m that would be worthwhile.\n\n 6. Related to #5: I've been able to setup hosted runner jobs, and\n    self-hosted runner jobs, but is there a way to do some opportunistic\n    mixture of the two? Even one where self-hosted runners could come\n    and go, and if they're present contribute resources to git/git's\n    network?\n\n 7. We run the various GIT_TEST_* etc. jobs in sequence, is there a\n    reason for why we're serializing things in GitHub CI that could be\n    parallelized?\n\n    The vs-build and vs-test tests run in parallel, any reason we're not\n    doing that trick on the ubuntu runners other than \"nobody got to\n    it?\". We seem to be trying hard to do the exact opposite there..\n\n    At the extreme end we could build git ~once, and have N tests depend\n    on that, where N ~= $(ls t/*.sh) x $number_of_test_modes). But\n    perhaps runner starting overhead starts to be the limiting factor at\n    some point.\n\n 8. To a first approximation, does anyone really care about getting an\n    exhaustive list of all failures in a run, or just that we have *a*\n    failure? You can always do an exhaustive run later.\n\n 9. On the \"no\" answer to #8: When I build/test my own git I first run\n    those tests that I modified in the relevant branches, and if any of\n    those fail I just stop.\n\n    I generally don't need to run the entirety of the rest of the test\n    suite to stop and investigate why I have a failure.\n\n    Perhaps our CI could use a similar trick, i.e. first test the set of\n    modified test files, and perhaps with some ad-hoc matching of\n    filenames, so e.g. if you modify builtin/add.c we'd run t/*add*.sh\n    in the first set, and all with --immediate per #8 above.\n\n    If we pass that we'd run the full set, minus that initial set.\n"},{"id":"440403","messageId":"xmqqbl306g80.fsf@gitster.g","threadId":"56747","inReplyTo":"211103.864k8t1sma.gmgdl@evledraar.gmail.com","subject":"Re: test suite speedups via some not-so-crazy ideas","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-03T22:12:47Z","receivedAt":"2021-11-03T22:12:54Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n\n>  8. To a first approximation, does anyone really care about getting an\n>     exhaustive list of all failures in a run, or just that we have *a*\n>     failure? You can always do an exhaustive run later.\n\nI do, not necessarily because I want to catch all failures, but\nmostly because I want to use the the number of failing tests as a\nrough sanity check.  I expect that the number is low, but not\nnecessarily zero, in the normal state, but if I see many in a run,\nthat rings different bells.  If we stop at the first failure, it\nbecomes harder to do this, and having to go there and restart with\n\"this time run the full set\" manually is not really feasible.\n\n\n"},{"id":"440669","messageId":"YYlqpuzv+bmZaFzz@nand.local","threadId":"56747","inReplyTo":"nycvar.QRO.7.76.6.2110211147490.56@tvgsbejvaqbjf.bet","subject":"Re: [Summit topic] Server-side merge/rebase: needs and wants?","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-11-08T18:21:26Z","receivedAt":"2021-11-08T18:21:29Z","isPatch":false,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"I was discussing this with Elijah today in IRC. I thought that I sent\nthe following message to the list, but somehow dropped it from the CC\nlist, and only sent it to Elijah and Johannes.\n\nHere it is in its entirety, this time copying the list.\n\nn Thu, Oct 21, 2021 at 01:56:06PM +0200, Johannes Schindelin wrote:\n>  5.  The challenge is not necessarily the technical challenges, but the UX for\n>      server tools that live “above” the git executable.\n>\n>      1. What kind of output is needed? Machine-readable error messages?\n>\n>      2. What Git objects must be created: a tree? A commit?\n>\n>      3. How to handle, report, and store conflicts? Index is not typically\n>         available on the server.\n\nI looked a little bit more into what GitHub would need in order to make\nthe switch. For background, we currently perform merges and rebases\nusing libgit2 as the backend, for the obvious reason which is that in a\npre-ORT world we could not write an intermediate result without having\nan index around.\n\n(As a fun aside, we used to expand our bare copy of a repository into a\ntemporary working directory, perform the merge there, and then delete\nthe directory. We definitely don't do that anymore ;)).\n\nIt looks like our current libgit2 usage more-or-less returns an\n(object_id, list<file>) tuple, where:\n\n  - a non-NULL object_id is the result of a successful (i.e.,\n    conflict-free) merge; specifically the oid of the resulting root\n    tree\n\n  - a NULL object_id and a non-empty list of files indicates that the\n    merge could not be completed without manual conflict resolution, and\n    the list of files indicates where the conflicts were\n\nWhen we try to process a conflicted merge, we display the list of files\nwhere conflicts were present in the web UI. We do have a UI to resolve\nconflicts, but we populate the contents of that UI by telling libgit2 to\nperform the same merge on *just that file*, and writing out the file\nwith its conflict markers as the result (and sending that result out to\na web editor).\n\nSo I think an ORT-powered server-side merge would have to be able to:\n\n  - write out the contents of a merge (with a tree, not a commit), and\n    indicate whether or not that merge was successful with an exit code\n\n  - write out the list of files that had conflicts upon failure\n\nGiven my limited knowledge of the ORT implementation, it seems like\nwriting out the conflicts themselves would be pretty easy. But GitHub\nprobably wouldn't use it, or at least not immediately, since we rely\nheavily on being able to recreate the conflicts file-by-file as they are\nneeded.\n\nAnyway, I happened to be looking into all of this during the summit, but\nnever wrote any of it down. So I figured that this might be helpful in\ncase folks are interested in pursuing this further. If so, let me know\nif there are any other questions about what GitHub might want on the\nbackend, and I'll try to answer as best I can.\n\nThanks,\nTaylor\n"},{"id":"440722","messageId":"211109.861r3qdpt8.gmgdl@evledraar.gmail.com","threadId":"56747","inReplyTo":"YYlqpuzv+bmZaFzz@nand.local","subject":"Re: [Summit topic] Server-side merge/rebase: needs and wants?","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-09T02:15:50Z","receivedAt":"2021-11-09T02:29:44Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Nov 08 2021, Taylor Blau wrote:\n\n> I was discussing this with Elijah today in IRC. I thought that I sent\n> the following message to the list, but somehow dropped it from the CC\n> list, and only sent it to Elijah and Johannes.\n>\n> Here it is in its entirety, this time copying the list.\n>\n> n Thu, Oct 21, 2021 at 01:56:06PM +0200, Johannes Schindelin wrote:\n>>  5.  The challenge is not necessarily the technical challenges, but the UX for\n>>      server tools that live “above” the git executable.\n>>\n>>      1. What kind of output is needed? Machine-readable error messages?\n>>\n>>      2. What Git objects must be created: a tree? A commit?\n>>\n>>      3. How to handle, report, and store conflicts? Index is not typically\n>>         available on the server.\n>\n> I looked a little bit more into what GitHub would need in order to make\n> the switch. For background, we currently perform merges and rebases\n> using libgit2 as the backend, for the obvious reason which is that in a\n> pre-ORT world we could not write an intermediate result without having\n> an index around.\n>\n> (As a fun aside, we used to expand our bare copy of a repository into a\n> temporary working directory, perform the merge there, and then delete\n> the directory. We definitely don't do that anymore ;)).\n>\n> It looks like our current libgit2 usage more-or-less returns an\n> (object_id, list<file>) tuple, where:\n>\n>   - a non-NULL object_id is the result of a successful (i.e.,\n>     conflict-free) merge; specifically the oid of the resulting root\n>     tree\n>\n>   - a NULL object_id and a non-empty list of files indicates that the\n>     merge could not be completed without manual conflict resolution, and\n>     the list of files indicates where the conflicts were\n>\n> When we try to process a conflicted merge, we display the list of files\n> where conflicts were present in the web UI. We do have a UI to resolve\n> conflicts, but we populate the contents of that UI by telling libgit2 to\n> perform the same merge on *just that file*, and writing out the file\n> with its conflict markers as the result (and sending that result out to\n> a web editor).\n>\n> So I think an ORT-powered server-side merge would have to be able to:\n>\n>   - write out the contents of a merge (with a tree, not a commit), and\n>     indicate whether or not that merge was successful with an exit code\n>\n>   - write out the list of files that had conflicts upon failure\n>\n> Given my limited knowledge of the ORT implementation, it seems like\n> writing out the conflicts themselves would be pretty easy. But GitHub\n> probably wouldn't use it, or at least not immediately, since we rely\n> heavily on being able to recreate the conflicts file-by-file as they are\n> needed.\n>\n> Anyway, I happened to be looking into all of this during the summit, but\n> never wrote any of it down. So I figured that this might be helpful in\n> case folks are interested in pursuing this further. If so, let me know\n> if there are any other questions about what GitHub might want on the\n> backend, and I'll try to answer as best I can.\n\nThat's very informative, thanks.\n\nNot that \"ort\" won't me much better at this, but FWIW git-merge-tree\nsort of gets most of the way-ish to what you're describing already in\nterms of a command interface.\n\nI.e. I'm not the first or last to have (not for anything serious)\nimplement a dry-run bare-repo merge with something like:\n\n    git merge-tree origin/master git-for-windows/main origin/seen >diff\n    # Better regex needed, but basically this\n    grep \"^\\+<<<<<<< \\.our$\" diff && conflict=t\n\nSo with some parsing of that command output you can get a diff with one\nside or the other applied.\n\nFrom there it's a matter of applying the patch, and from there you'd get\nblobs/trees. which is painful from just having a diff & no index, so\nit's a common use-case of libgit2 for just such basic usage.\n\nBut to the extent that we were talking about plumbing interfaces\nwouldn't basically a git-merge-tree on steroids (or extension thereof)\ndo, i.e.:\n\n * Ask it to merge X heads, returns whether it worked or not\n * ... and can return a diff with conflict markers like this\n * ... for just some <pathspec>\n * ... maybe with the conflict already \"resolved\" one way or the other?\n * ... optionally, after some markers write one/both sides, spew out the\n   relevant tree/blob OIDs\n * ... which again, could be limited by the <pathspec> above.\n\nI'm thinking of something that basically works like git for-each-ref --format=\"\". So:\n\n    git merge-tree --format=\"...\" <heads> -- <pathspec>\n\nWhere that <format> can be custom \\0-delimited (or whatever) sections of\npayload that could have whatever combination of the above you'd need. I\nthink git-for-each-ref is probably the best example we've got of a\nplumbing interface in this category, i.e. being able to extract\narbitrary payloads via format specifiers & \"path\" (well, ref)\nlimitation.\n\nElijah probably has much better ideas already, I'm just spitballing. \n\nBut if something like that worked it would be mostly a matter of\nstealing code from for-each-ref and the like, and then the <handwaiving>\nmapping that to ORT callbacks somehow.\n\nAnd then it could even learn a --batch mode, which with those formats\ncould allow calling it without paying the price for command\nre-invocation, something like the update-ref/proposed cat-file interface\ndiscussed in another thread at [1].\n\n1. https://lore.kernel.org/git/211106.86k0hmgc8q.gmgdl@evledraar.gmail.com/\n"},{"id":"442622","messageId":"CAP8UFD1LgfZ0MT9=cMvxCcox++_MBBhWG9Twf42cMiXL42AdpQ@mail.gmail.com","threadId":"56747","inReplyTo":"211109.861r3qdpt8.gmgdl@evledraar.gmail.com","subject":"Re: [Summit topic] Server-side merge/rebase: needs and wants?","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2021-11-30T10:06:09Z","receivedAt":"2021-11-30T10:06:23Z","isPatch":false,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Tue, Nov 9, 2021 at 1:18 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:\n>\n>\n> On Mon, Nov 08 2021, Taylor Blau wrote:\n>\n> > I was discussing this with Elijah today in IRC. I thought that I sent\n> > the following message to the list, but somehow dropped it from the CC\n> > list, and only sent it to Elijah and Johannes.\n> >\n> > Here it is in its entirety, this time copying the list.\n> >\n> > n Thu, Oct 21, 2021 at 01:56:06PM +0200, Johannes Schindelin wrote:\n> >>  5.  The challenge is not necessarily the technical challenges, but the UX for\n> >>      server tools that live “above” the git executable.\n> >>\n> >>      1. What kind of output is needed? Machine-readable error messages?\n> >>\n> >>      2. What Git objects must be created: a tree? A commit?\n> >>\n> >>      3. How to handle, report, and store conflicts? Index is not typically\n> >>         available on the server.\n> >\n> > I looked a little bit more into what GitHub would need in order to make\n> > the switch. For background, we currently perform merges and rebases\n> > using libgit2 as the backend, for the obvious reason which is that in a\n> > pre-ORT world we could not write an intermediate result without having\n> > an index around.\n> >\n> > (As a fun aside, we used to expand our bare copy of a repository into a\n> > temporary working directory, perform the merge there, and then delete\n> > the directory. We definitely don't do that anymore ;)).\n> >\n> > It looks like our current libgit2 usage more-or-less returns an\n> > (object_id, list<file>) tuple, where:\n> >\n> >   - a non-NULL object_id is the result of a successful (i.e.,\n> >     conflict-free) merge; specifically the oid of the resulting root\n> >     tree\n> >\n> >   - a NULL object_id and a non-empty list of files indicates that the\n> >     merge could not be completed without manual conflict resolution, and\n> >     the list of files indicates where the conflicts were\n> >\n> > When we try to process a conflicted merge, we display the list of files\n> > where conflicts were present in the web UI. We do have a UI to resolve\n> > conflicts, but we populate the contents of that UI by telling libgit2 to\n> > perform the same merge on *just that file*, and writing out the file\n> > with its conflict markers as the result (and sending that result out to\n> > a web editor).\n> >\n> > So I think an ORT-powered server-side merge would have to be able to:\n> >\n> >   - write out the contents of a merge (with a tree, not a commit), and\n> >     indicate whether or not that merge was successful with an exit code\n> >\n> >   - write out the list of files that had conflicts upon failure\n> >\n> > Given my limited knowledge of the ORT implementation, it seems like\n> > writing out the conflicts themselves would be pretty easy. But GitHub\n> > probably wouldn't use it, or at least not immediately, since we rely\n> > heavily on being able to recreate the conflicts file-by-file as they are\n> > needed.\n> >\n> > Anyway, I happened to be looking into all of this during the summit, but\n> > never wrote any of it down. So I figured that this might be helpful in\n> > case folks are interested in pursuing this further. If so, let me know\n> > if there are any other questions about what GitHub might want on the\n> > backend, and I'll try to answer as best I can.\n>\n> That's very informative, thanks.\n\nYeah, thanks!\n\n> Not that \"ort\" won't me much better at this,\n\nI think the optimizations in \"ort\" could still be useful. Wouldn't it\nbe nice if rename detection was optimized for example?\n\n> but FWIW git-merge-tree\n> sort of gets most of the way-ish to what you're describing already in\n> terms of a command interface.\n\nYeah, but if the engine is not up to date, I am not sure it's worth it\nto reuse it just for the current very limited command interface.\n\n> I.e. I'm not the first or last to have (not for anything serious)\n> implement a dry-run bare-repo merge with something like:\n>\n>     git merge-tree origin/master git-for-windows/main origin/seen >diff\n>     # Better regex needed, but basically this\n>     grep \"^\\+<<<<<<< \\.our$\" diff && conflict=t\n>\n> So with some parsing of that command output you can get a diff with one\n> side or the other applied.\n\nYeah, it looks like it would be easy to add options like --ours,\n--theirs, etc, to get only the part we are interested in. And we\nalready easily see if the merge conflicted or not from the current\noutput, as it seems to output:\n\n\"0 mode sha1 filename\"\n\nin case of a successful merge, and:\n\n\"1 mode sha1 filename\"\n\"2 mode sha1 filename\"\n\"3 mode sha1 filename\"\n\nin case of conflicts.\n\n> From there it's a matter of applying the patch, and from there you'd get\n> blobs/trees. which is painful from just having a diff & no index, so\n> it's a common use-case of libgit2 for just such basic usage.\n>\n> But to the extent that we were talking about plumbing interfaces\n> wouldn't basically a git-merge-tree on steroids (or extension thereof)\n> do, i.e.:\n>\n>  * Ask it to merge X heads, returns whether it worked or not\n>  * ... and can return a diff with conflict markers like this\n>  * ... for just some <pathspec>\n>  * ... maybe with the conflict already \"resolved\" one way or the other?\n>  * ... optionally, after some markers write one/both sides, spew out the\n>    relevant tree/blob OIDs\n>  * ... which again, could be limited by the <pathspec> above.\n>\n> I'm thinking of something that basically works like git for-each-ref --format=\"\". So:\n>\n>     git merge-tree --format=\"...\" <heads> -- <pathspec>\n>\n> Where that <format> can be custom \\0-delimited (or whatever) sections of\n> payload that could have whatever combination of the above you'd need. I\n> think git-for-each-ref is probably the best example we've got of a\n> plumbing interface in this category, i.e. being able to extract\n> arbitrary payloads via format specifiers & \"path\" (well, ref)\n> limitation.\n\nThe current synopsis is:\n\ngit merge-tree <base-tree> <branch1> <branch2>\n\nwhich is quite different from what you are proposing.\n\nGiven that it seems worth it to use a different underlying engine\n(actually the \"ort\" one) than the current one, I think that it might\nbe better to start from scratch with a new command using the \"ort\"\nengine.\n\n> Elijah probably has much better ideas already, I'm just spitballing.\n\nYeah, I'd be interested in knowing Elijah's opinion on this. Although\nmaybe I misunderstood, but I thought that Elijah had plans to send\npatches related to this to the list after v2.34.\n\n> But if something like that worked it would be mostly a matter of\n> stealing code from for-each-ref and the like, and then the <handwaiving>\n> mapping that to ORT callbacks somehow.\n\nYeah, but what would be left from the original git merge-tree then?\n\nWouldn't it make more sense to start with a new command that has\nroughly the same features as git merge-tree and a similar interface\n(though maybe not quite the same as we could anticipate some future\nextensions and maybe learn a bit from other commands), but uses \"ort\".\nThen we could grow it as we want, without being burdened by the git\nmerge-tree legacy, in the same way as \"ort\" was developed without\nbeing burdened by the recursive merge legacy?\n\n> And then it could even learn a --batch mode, which with those formats\n> could allow calling it without paying the price for command\n> re-invocation, something like the update-ref/proposed cat-file interface\n> discussed in another thread at [1].\n\nYeah, sure.\n"}]}