{"thread":{"id":"52055","subject":"[RFC/WIP] range-diff: show old/new blob OIDs in comments","startedAt":"2019-10-17T12:10:46Z","lastAt":"2019-10-25T13:43:44Z","messageCount":9,"participants":["Eric Wong","Johannes Schindelin","Konstantin Ryabitsev","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"384252","messageId":"20191017121045.GA15364@dcvr","threadId":"52055","inReplyTo":null,"subject":"[RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2019-10-17T12:10:45Z","receivedAt":"2019-10-17T12:10:46Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"(WIP, mostly stream-of-concious notes + reasoning)\n\nWhen using \"git format-patch --range-diff\", the pre and\npost-image blob OIDs are in each email, while the exact\ncommit OIDs are rarely shared via emails (only the tip\ncommit from \"git request-pull\").\n\nThese blob OIDs make it easy to search for or lookup the\nfull emails which create them, or the blob itself once\nit's fetched via git.\n\npublic-inbox indexes and allows querying specifically for blob\nOIDs via dfpre:/dfpost: since June 2017.  As of Jan 2019,\npublic-inbox also supports recreating blobs out of patch emails\n(querying internally with dfpre:/dfpost: and doing \"git apply\")\n\nSearching on these blob OIDs also makes it easier to find\nprevious versions of the patch sets using any mail search\nengine.\n\nFuture changes to public-inbox may allow generating custom\ndiffs out of any blobs it can find or recreate.\n\nMost of this is pretty public-inbox-specific and would've\nmade some future changes to public-inbox much easier....\n(if we had this from the start of range-diff).\n\nUnfortunately, it won't help with cases where range-diffs\nare already published, but range-diff isn't too old.\n\nI'm also still learning my way around git's C internals, but\nusing patch.{old,new}_oid_prefix seems alright...\n\nFIXME: tests, t3206 needs updating\n\nNot-signed-off-by: Eric Wong <e@80x24.org>\n---\n range-diff.c | 19 +++++++++++++++----\n 1 file changed, 15 insertions(+), 4 deletions(-)\n\ndiff --git a/range-diff.c b/range-diff.c\nindex 7fed5a3b4b..85d2f1f58f 100644\n--- a/range-diff.c\n+++ b/range-diff.c\n@@ -118,13 +118,24 @@ static int read_patches(const char *range, struct string_list *list)\n \t\t\t\tdie(_(\"could not parse git header '%.*s'\"), (int)len, line);\n \t\t\tstrbuf_addstr(&buf, \" ## \");\n \t\t\tif (patch.is_new > 0)\n-\t\t\t\tstrbuf_addf(&buf, \"%s (new)\", patch.new_name);\n+\t\t\t\tstrbuf_addf(&buf, \"%s (new %s)\",\n+\t\t\t\t\t\tpatch.new_name,\n+\t\t\t\t\t\tpatch.new_oid_prefix);\n \t\t\telse if (patch.is_delete > 0)\n-\t\t\t\tstrbuf_addf(&buf, \"%s (deleted)\", patch.old_name);\n+\t\t\t\tstrbuf_addf(&buf, \"%s (deleted %s)\",\n+\t\t\t\t\t\tpatch.old_name,\n+\t\t\t\t\t\tpatch.old_oid_prefix);\n \t\t\telse if (patch.is_rename)\n-\t\t\t\tstrbuf_addf(&buf, \"%s => %s\", patch.old_name, patch.new_name);\n+\t\t\t\tstrbuf_addf(&buf, \"%s => %s (%s..%s)\",\n+\t\t\t\t\t\tpatch.old_name,\n+\t\t\t\t\t\tpatch.new_name,\n+\t\t\t\t\t\tpatch.old_oid_prefix,\n+\t\t\t\t\t\tpatch.new_oid_prefix);\n \t\t\telse\n-\t\t\t\tstrbuf_addstr(&buf, patch.new_name);\n+\t\t\t\tstrbuf_addf(&buf, \"%s (%s..%s)\",\n+\t\t\t\t\t\tpatch.new_name,\n+\t\t\t\t\t\tpatch.old_oid_prefix,\n+\t\t\t\t\t\tpatch.new_oid_prefix);\n \n \t\t\tfree(current_filename);\n \t\t\tif (patch.is_delete > 0)\n\n"},{"id":"384620","messageId":"nycvar.QRO.7.76.6.1910222111430.46@tvgsbejvaqbjf.bet","threadId":"52055","inReplyTo":"20191017121045.GA15364@dcvr","subject":"Re: [RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-22T19:18:35Z","receivedAt":"2019-10-22T19:18:57Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Eric,\n\n\nOn Thu, 17 Oct 2019, Eric Wong wrote:\n\n> (WIP, mostly stream-of-concious notes + reasoning)\n>\n> When using \"git format-patch --range-diff\", the pre and\n> post-image blob OIDs are in each email, while the exact\n> commit OIDs are rarely shared via emails (only the tip\n> commit from \"git request-pull\").\n>\n> These blob OIDs make it easy to search for or lookup the\n> full emails which create them, or the blob itself once\n> it's fetched via git.\n>\n> public-inbox indexes and allows querying specifically for blob\n> OIDs via dfpre:/dfpost: since June 2017.  As of Jan 2019,\n> public-inbox also supports recreating blobs out of patch emails\n> (querying internally with dfpre:/dfpost: and doing \"git apply\")\n>\n> Searching on these blob OIDs also makes it easier to find\n> previous versions of the patch sets using any mail search\n> engine.\n>\n> Future changes to public-inbox may allow generating custom\n> diffs out of any blobs it can find or recreate.\n>\n> Most of this is pretty public-inbox-specific and would've\n> made some future changes to public-inbox much easier....\n> (if we had this from the start of range-diff).\n>\n> Unfortunately, it won't help with cases where range-diffs\n> are already published, but range-diff isn't too old.\n\nI guess your patch won't hurt.\n\nAs to recreating blobs from mails: Wow. That's quite a length you're\ngoing, and I think it is a shame that you have to. If only every\ncontribution came accompanied with a pullable branch in a public\nrepository.\n\nInstead, we will have to rely on your centralized, non-distributed\nservice...\n\nCiao,\nDscho\n\n>\n> I'm also still learning my way around git's C internals, but\n> using patch.{old,new}_oid_prefix seems alright...\n>\n> FIXME: tests, t3206 needs updating\n>\n> Not-signed-off-by: Eric Wong <e@80x24.org>\n> ---\n>  range-diff.c | 19 +++++++++++++++----\n>  1 file changed, 15 insertions(+), 4 deletions(-)\n>\n> diff --git a/range-diff.c b/range-diff.c\n> index 7fed5a3b4b..85d2f1f58f 100644\n> --- a/range-diff.c\n> +++ b/range-diff.c\n> @@ -118,13 +118,24 @@ static int read_patches(const char *range, struct string_list *list)\n>  \t\t\t\tdie(_(\"could not parse git header '%.*s'\"), (int)len, line);\n>  \t\t\tstrbuf_addstr(&buf, \" ## \");\n>  \t\t\tif (patch.is_new > 0)\n> -\t\t\t\tstrbuf_addf(&buf, \"%s (new)\", patch.new_name);\n> +\t\t\t\tstrbuf_addf(&buf, \"%s (new %s)\",\n> +\t\t\t\t\t\tpatch.new_name,\n> +\t\t\t\t\t\tpatch.new_oid_prefix);\n>  \t\t\telse if (patch.is_delete > 0)\n> -\t\t\t\tstrbuf_addf(&buf, \"%s (deleted)\", patch.old_name);\n> +\t\t\t\tstrbuf_addf(&buf, \"%s (deleted %s)\",\n> +\t\t\t\t\t\tpatch.old_name,\n> +\t\t\t\t\t\tpatch.old_oid_prefix);\n>  \t\t\telse if (patch.is_rename)\n> -\t\t\t\tstrbuf_addf(&buf, \"%s => %s\", patch.old_name, patch.new_name);\n> +\t\t\t\tstrbuf_addf(&buf, \"%s => %s (%s..%s)\",\n> +\t\t\t\t\t\tpatch.old_name,\n> +\t\t\t\t\t\tpatch.new_name,\n> +\t\t\t\t\t\tpatch.old_oid_prefix,\n> +\t\t\t\t\t\tpatch.new_oid_prefix);\n>  \t\t\telse\n> -\t\t\t\tstrbuf_addstr(&buf, patch.new_name);\n> +\t\t\t\tstrbuf_addf(&buf, \"%s (%s..%s)\",\n> +\t\t\t\t\t\tpatch.new_name,\n> +\t\t\t\t\t\tpatch.old_oid_prefix,\n> +\t\t\t\t\t\tpatch.new_oid_prefix);\n>\n>  \t\t\tfree(current_filename);\n>  \t\t\tif (patch.is_delete > 0)\n>\n>\n"},{"id":"384621","messageId":"20191022193558.GC4960@chatter.i7.local","threadId":"52055","inReplyTo":"nycvar.QRO.7.76.6.1910222111430.46@tvgsbejvaqbjf.bet","subject":"Re: [RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2019-10-22T19:35:58Z","receivedAt":"2019-10-22T19:36:04Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Tue, Oct 22, 2019 at 09:18:35PM +0200, Johannes Schindelin wrote:\n>As to recreating blobs from mails: Wow. That's quite a length you're\n>going, and I think it is a shame that you have to. If only every\n>contribution came accompanied with a pullable branch in a public\n>repository.\n\nTrouble is, those public repositories are transient and may be gone soon \nafter the pull request is performed. The goal is to be able to \nreconstruct full code history prior to when it is merged into the \nofficial repository.\n\n>Instead, we will have to rely on your centralized, non-distributed\n>service...\n\nIt's actually neither, because mirroring and distributing public-inbox \nrepositories is already easy, and we hope to make it easier in the near \nfuture.\n\n-K\n"},{"id":"384658","messageId":"20191023015629.GA15495@dcvr","threadId":"52055","inReplyTo":"nycvar.QRO.7.76.6.1910222111430.46@tvgsbejvaqbjf.bet","subject":"Re: [RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2019-10-23T01:56:29Z","receivedAt":"2019-10-23T01:56:32Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> Hi Eric,\n> \n> \n> On Thu, 17 Oct 2019, Eric Wong wrote:\n> \n> > (WIP, mostly stream-of-concious notes + reasoning)\n> >\n> > When using \"git format-patch --range-diff\", the pre and\n> > post-image blob OIDs are in each email, while the exact\n> > commit OIDs are rarely shared via emails (only the tip\n> > commit from \"git request-pull\").\n> >\n> > These blob OIDs make it easy to search for or lookup the\n> > full emails which create them, or the blob itself once\n> > it's fetched via git.\n> >\n> > public-inbox indexes and allows querying specifically for blob\n> > OIDs via dfpre:/dfpost: since June 2017.  As of Jan 2019,\n> > public-inbox also supports recreating blobs out of patch emails\n> > (querying internally with dfpre:/dfpost: and doing \"git apply\")\n> >\n> > Searching on these blob OIDs also makes it easier to find\n> > previous versions of the patch sets using any mail search\n> > engine.\n> >\n> > Future changes to public-inbox may allow generating custom\n> > diffs out of any blobs it can find or recreate.\n> >\n> > Most of this is pretty public-inbox-specific and would've\n> > made some future changes to public-inbox much easier....\n> > (if we had this from the start of range-diff).\n> >\n> > Unfortunately, it won't help with cases where range-diffs\n> > are already published, but range-diff isn't too old.\n> \n> I guess your patch won't hurt.\n\nCool, will update tests and resend.\n\n> As to recreating blobs from mails: Wow. That's quite a length you're\n> going, and I think it is a shame that you have to. If only every\n contribution came accompanied with a pullable branch in a public\n> repository.\n\nWhat Konstantin said about git repos being transient.\nIt wasn't too much work to recreate those blobs from\nscratch since git-apply has done it since 2005.\nJust add search :)\n\nWe could get around transient repos with automatic mirroring\nbots which never deletes or overwrites anything published.\nThat includes preserving pre-force-push data in case of\nforce pushes.\n\n> Instead, we will have to rely on your centralized, non-distributed\n> service...\n\nI'm curious how you came to believe that, since that's the\nopposite of what public-inbox has always been intended to be.\n\nThe only thing that's centralized and not reproducible by\nmirrors is the domain name (and I also have Tor .onion mirrors\nwith no dependency on ICAAN).  Memorable naming is a tricky\nproblem in decentralized systems, though...\n"},{"id":"384672","messageId":"xmqq5zkghzid.fsf@gitster-ct.c.googlers.com","threadId":"52055","inReplyTo":"20191023015629.GA15495@dcvr","subject":"Re: [RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-10-23T04:11:38Z","receivedAt":"2019-10-23T04:12:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong <e@80x24.org> writes:\n\n> What Konstantin said about git repos being transient.\n> It wasn't too much work to recreate those blobs from\n> scratch since git-apply has done it since 2005.\n\n;-)\n\n> We could get around transient repos with automatic mirroring\n> bots which never deletes or overwrites anything published.\n> That includes preserving pre-force-push data in case of\n> force pushes.\n>\n>> Instead, we will have to rely on your centralized, non-distributed\n>> service...\n>\n> I'm curious how you came to believe that, since that's the\n> opposite of what public-inbox has always been intended to be.\n\nI think the (mis)perception comes from the fact that the website and\nthe newsfeed you give are both too easy to use and directly attract\nend users, instead of enticing them to keep their own mirrors for\noffline use.\n\nThanks for injecting dose of sanity.\n"},{"id":"384688","messageId":"20191023100614.GA2645@dcvr","threadId":"52055","inReplyTo":"20191023015629.GA15495@dcvr","subject":"Re: [RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2019-10-23T10:06:14Z","receivedAt":"2019-10-23T10:06:16Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Eric Wong <e@80x24.org> wrote:\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> > I guess your patch won't hurt.\n> \n> Cool, will update tests and resend.\n\nTurns out the t3206 \"trivial reordering\" case is way too noisy.\nI might need to change diff.c to support this case; will update\nin a few days or week.\n"},{"id":"384799","messageId":"nycvar.QRO.7.76.6.1910250014240.46@tvgsbejvaqbjf.bet","threadId":"52055","inReplyTo":"xmqq5zkghzid.fsf@gitster-ct.c.googlers.com","subject":"Re: [RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-24T22:16:29Z","receivedAt":"2019-10-24T22:16:57Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 23 Oct 2019, Junio C Hamano wrote:\n\n> Eric Wong <e@80x24.org> writes:\n>\n> > What Konstantin said about git repos being transient.\n> > It wasn't too much work to recreate those blobs from\n> > scratch since git-apply has done it since 2005.\n>\n> ;-)\n>\n> > We could get around transient repos with automatic mirroring\n> > bots which never deletes or overwrites anything published.\n> > That includes preserving pre-force-push data in case of\n> > force pushes.\n> >\n> >> Instead, we will have to rely on your centralized, non-distributed\n> >> service...\n> >\n> > I'm curious how you came to believe that, since that's the\n> > opposite of what public-inbox has always been intended to be.\n>\n> I think the (mis)perception comes from the fact that the website and\n> the newsfeed you give are both too easy to use and directly attract\n> end users, instead of enticing them to keep their own mirrors for\n> offline use.\n>\n> Thanks for injecting dose of sanity.\n\nMaybe your dose of sanity can inject a statement about the case when\npublic-inbox.org/git differs from a mirror, and not in a\nfast-forwardable way? What is the authoritative source of truth, then?\n\nCiao,\nDscho\n"},{"id":"384819","messageId":"20191025001254.GA29496@dcvr","threadId":"52055","inReplyTo":"nycvar.QRO.7.76.6.1910250014240.46@tvgsbejvaqbjf.bet","subject":"Re: [RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2019-10-25T00:12:54Z","receivedAt":"2019-10-25T00:15:58Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> On Wed, 23 Oct 2019, Junio C Hamano wrote:\n> > Eric Wong <e@80x24.org> writes:\n> > > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> > >> Instead, we will have to rely on your centralized, non-distributed\n> > >> service...\n> > >\n> > > I'm curious how you came to believe that, since that's the\n> > > opposite of what public-inbox has always been intended to be.\n> >\n> > I think the (mis)perception comes from the fact that the website and\n> > the newsfeed you give are both too easy to use and directly attract\n> > end users, instead of enticing them to keep their own mirrors for\n> > offline use.\n> >\n> > Thanks for injecting dose of sanity.\n> \n> Maybe your dose of sanity can inject a statement about the case when\n> public-inbox.org/git differs from a mirror, and not in a\n> fast-forwardable way? What is the authoritative source of truth, then?\n\nWhy does authoritative source of truth matter?  My\nanti-authoritarian ethos is what drew me to DVCS in the first\nplace.\n\nIf senders want to attest to the integrity of their messages;\nthey can sign, and/or publish a copy/log of their sent messages\non their homepage/social media/whatever.  That's up to THEM,\nnot anybody else.\n\nIf somebody wants to fork public-inbox.org/git and run\npublic-inbox-watch from their own Maildir, they're more than\nwelcome to.\n\nIf somebody wants to write their own importers since they don't\nlike the code I write, they are more than welcome to.  There's\nalready mail-archive.com, marc.info, news.gmane.org (which\npublic-inbox.org/git forked from) and some others.\n\nGoing farther, if people want to fork entire mailing lists and\ncommunities, they should be able to do so.  I don't like mail\nsubscriber lists being centralized on any host, either.\n\nI have never, ever asked anybody to trust me or public-inbox;\nin fact, I've stated the opposite and will continue to do so.\n"},{"id":"384861","messageId":"nycvar.QRO.7.76.6.1910251531350.46@tvgsbejvaqbjf.bet","threadId":"52055","inReplyTo":"20191025001254.GA29496@dcvr","subject":"Re: [RFC/WIP] range-diff: show old/new blob OIDs in comments","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-25T13:43:16Z","receivedAt":"2019-10-25T13:43:44Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Eric,\n\nOn Fri, 25 Oct 2019, Eric Wong wrote:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> > On Wed, 23 Oct 2019, Junio C Hamano wrote:\n> > > Eric Wong <e@80x24.org> writes:\n> > > > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> > > >> Instead, we will have to rely on your centralized, non-distributed\n> > > >> service...\n> > > >\n> > > > I'm curious how you came to believe that, since that's the\n> > > > opposite of what public-inbox has always been intended to be.\n> > >\n> > > I think the (mis)perception comes from the fact that the website and\n> > > the newsfeed you give are both too easy to use and directly attract\n> > > end users, instead of enticing them to keep their own mirrors for\n> > > offline use.\n> > >\n> > > Thanks for injecting dose of sanity.\n> >\n> > Maybe your dose of sanity can inject a statement about the case when\n> > public-inbox.org/git differs from a mirror, and not in a\n> > fast-forwardable way? What is the authoritative source of truth, then?\n>\n> Why does authoritative source of truth matter?  My\n> anti-authoritarian ethos is what drew me to DVCS in the first\n> place.\n>\n> If senders want to attest to the integrity of their messages;\n> they can sign, and/or publish a copy/log of their sent messages\n> on their homepage/social media/whatever.  That's up to THEM,\n> not anybody else.\n>\n> If somebody wants to fork public-inbox.org/git and run\n> public-inbox-watch from their own Maildir, they're more than\n> welcome to.\n\nI am _more_ than happy to rely on public-inbox.org/git. And I will never\nkid myself about relying on a central service, is all.\n\n> If somebody wants to write their own importers since they don't\n> like the code I write, they are more than welcome to.  There's\n> already mail-archive.com, marc.info, news.gmane.org (which\n> public-inbox.org/git forked from) and some others.\n>\n> Going farther, if people want to fork entire mailing lists and\n> communities, they should be able to do so.  I don't like mail\n> subscriber lists being centralized on any host, either.\n>\n> I have never, ever asked anybody to trust me or public-inbox;\n> in fact, I've stated the opposite and will continue to do so.\n\nWell, too bad. I trust you, Eric. I do trust you and will probably\ncontinue to trust you because I don't expect you to do anything, ever,\nto break that trust. So far, you haven't disappointed me even a single\ntime, and we've concurrently been Git contributors for, sheesh, has it\nalready been almost 14 years? I have benefitted from your work greatly,\nmostly via `git svn` in the olden days, and I hope that I could return\nthe favor every once in a while.\n\nWithout public-inbox.org/git, GitGitGadget would not be possible. My\nscripts to map commits to mails and vice versa (mirrored to\nhttps://github.com/gitgitgadget/git as `refs/notes/mail-to-commit` and\n`commit-to-mail`) would remain a pipe dream of mine.\n\n(Yes, yes, there are holes in that mapping, but even if I only have to\nlook up manually one out of 30 mails when I want to comment on a\nspecific commit, that already saves me so much time, not to mention\nnerves.)\n\nSo please understand that I am deeply grateful that you came up with\nthese projects, in particular with public-inbox. It is a life saver. I\nmight not share all of your philosophy regarding centralized vs\ndecentralized, even so, what you did helps me multiple times every\nsingle day.\n\nTherefore: a heart-felt Thank You, I owe you more than one,\nDscho\n"}]}