{"thread":{"id":"49225","subject":"Thank you for public-inbox!","startedAt":"2018-08-27T14:25:20Z","lastAt":"2018-08-31T11:01:22Z","messageCount":13,"participants":["Johannes Schindelin","Eric Wong","Jeff King","Andrei Rybak","Ævar Arnfjörð Bjarmason","Jonathan Nieder"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"356582","messageId":"nycvar.QRO.7.76.6.1808271552580.73@tvgsbejvaqbjf.bet","threadId":"49225","inReplyTo":null,"subject":"Thank you for public-inbox!","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2018-08-27T14:25:13Z","receivedAt":"2018-08-27T14:25:20Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Eric,\n\nI would like to take five minutes to thank you for public-inbox. It is\ninvaluable for me in the meantime. And I think I will never be able to\nthank you enough for it.\n\nJust a couple of things where it is super useful to me:\n\n- Recently, my mail provider started dropping mails left and right. They\n might even be addressed to me, and they never arrive in my inbox (and\n not even in the spam folder: they just never arrive). I spent some ten\n hours this past weekend to script identifying the mails in public-inbox\n that I never received, and to weed through them.\n\n It seems I missed some 4,000 mails. Thanks to you, I now saw those mails\n (although I had to delete most of them after reading only their subject,\n in the interest of time).\n\n- Many a times when my automated builds identified a problem with the test\n suite, it was a lot quicker to use public inbox to identify the mail to\n respond to than to use my mail program.\n\n- Sometimes, I find myself in want of replying to a past patch, but back\n in the day when it was sent I thought it would not be interesting to me,\n so I deleted it. With public-inbox, I can easily get it in raw format,\n i.e. put it back into my inbox so I can reply.\n\n- I cannot tell you how many times I send a link to public-inbox to my\n colleagues rather than forwarding a mail, because the former will give\n them more context (and also semi-live updates in case somebody replied\n to said mail after I sent the link).\n\nA couple of things in the future that public-inbox will make possible for\nme:\n\n- You probably are aware of my GitGitGadget endeavor, a project similar in\n aim to SubmitGit, but a lot more integrated with the GitHub experience\n (and not requiring you to hand over your mail sending credentials to\n AWS).\n\n One particular feature I found myself really wanting in SubmitGit (but\n not possible due to its one-way design): I want my Pull Requests to be\n closed once the patches are integrated into git.git's `master` branch.\n\n While this feature is not available in GitGitGadget yet, I am well\n underway there. I already have a notes ref (`commit-to-mail`, available\n via https://github.git/gitgitgadget/git) that annotates commits in\n git.git with the Message-ID of the corresponding mail. By \"I have\", I\n mean: there is an automated task that uses public-inbox to keep that ref\n up to date.\n\n I also have an accompanying `mail-to-commit` notes ref that maps\n Message-IDs back to the corresponding commit in git.git. That notes ref\n \"annotates\" the (non-existing) blob you get when piping the Message-ID\n with a trailing newline to `git hash-object`.\n\n Again, this is information that would be absolutely unobtainable without\n public-inbox.\n\n- Related, I want to annotate the GitHub Pull Requests handled by\n GitGitGadvget with the corresponding name of the branch in\n https://github.com/gitster/git.\n\n This requires that `mail-to-commit` I mentioned in the previous bullet\n point, and therefore would not be possible without public-inbox.\n\n- A feature I plan on introducing into GitGitGadget is to attach comments\n to the GitHub Pull Request when anybody replies to the patch thread sent\n out by GitGitGadget.\n\n Also this feature would be impossible without public-inbox.\n\n- Another really useful feature I plan on introducing is to attach\n comments to those PRs whenever a What's Cooking is talking about the\n corresponding branch.\n\n Once again, would be impossible without public-inbox.\n\nSo thank you, thank you, thank you, for public-inbox!\n\nCiao,\nDscho\n\nP.S.: FWIW I added a mirror of public-inbox to\nhttps://git-for-windows.visualstudio.com/git.public-inbox, so that my\nautomated tasks, as well as my playing around, does not stress your server\ntoo much.\n\n"},{"id":"356591","messageId":"20180827173140.hgxuvxbmi7rjb75b@dcvr","threadId":"49225","inReplyTo":"nycvar.QRO.7.76.6.1808271552580.73@tvgsbejvaqbjf.bet","subject":"Re: Thank you for public-inbox!","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2018-08-27T17:31:40Z","receivedAt":"2018-08-27T17:31:42Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"\n\nYou're very welcome, Johannes. And I'm hoping to have a few\nmore goodies live this fall/winter for public-inbox :>\n"},{"id":"356812","messageId":"20180829050745.GD15274@sigill.intra.peff.net","threadId":"49225","inReplyTo":"nycvar.QRO.7.76.6.1808271552580.73@tvgsbejvaqbjf.bet","subject":"Re: Thank you for public-inbox!","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-08-29T05:07:46Z","receivedAt":"2018-08-29T05:07:49Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Aug 27, 2018 at 04:25:13PM +0200, Johannes Schindelin wrote:\n\n> I would like to take five minutes to thank you for public-inbox. It is\n> invaluable for me in the meantime. And I think I will never be able to\n> thank you enough for it.\n\nLet me echo that appreciation. I have always kept my own archive of the\nlist, but it's so valuable to have stable URLs for communicating with\nother folks (and I find the general design of public-inbox _way_ more\nuseful than gmane ever was).\n\n> P.S.: FWIW I added a mirror of public-inbox to\n> https://git-for-windows.visualstudio.com/git.public-inbox, so that my\n> automated tasks, as well as my playing around, does not stress your server\n> too much.\n\nI've thought about mirroring it to a public server as well, just for\nredundancy. But without the same domain, I'm not sure it would be all\nthat useful as a community resource.\n\nEric, let me know if there's something there that would help (e.g.,\nputting more servers in DNS round-robin).\n\n-Peff\n"},{"id":"356824","messageId":"20180829100243.GA28180@dcvr","threadId":"49225","inReplyTo":"20180829050745.GD15274@sigill.intra.peff.net","subject":"Re: Thank you for public-inbox!","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2018-08-29T10:02:43Z","receivedAt":"2018-08-29T10:02:44Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Jeff King wrote:\n> I've thought about mirroring it to a public server as well, just for\n> redundancy. But without the same domain, I'm not sure it would be all\n> that useful as a community resource.\n\nI wouldn't get too attached to the domain, \"public-inbox.org\" is\ntoo long for my tastes anyways. \"peff.net/git/$MESSAGE_ID\"\nwould actually be more user-friendly :>\n\nA generic Message-ID redirection/finding service would be good,\n(maybe some DHT thing, but... has that taken off for git blobs, yet?)\n\nAnyways I hope to teach public-inbox to auto-linkify Message-ID-looking\nstrings \"\" into URLs for domain-portability,\n(but it's ambiguous with email addresses). But yeah, I don't\nlike things being tied to domain names.\n\nI've also been considering setting up a parallel instance\non 80x24.org to use the more-scalable \"v2\" repository format\ndeveloped for https://lore.kernel.org/lkml/\n\nSpeaking of lore, Konstantin confirmed he'll be getting more\nvger lists up: https://public-inbox.org/meta/20180827205811.GA1611@chatter\n"},{"id":"356850","messageId":"3eb1c5e8-3e89-0d2e-30b1-339f38c4c703@gmail.com","threadId":"49225","inReplyTo":"20180829100243.GA28180@dcvr","subject":"Re: Thank you for public-inbox!","fromName":"Andrei Rybak","fromEmail":"rybak.a.v@gmail.com","sentAt":"2018-08-29T15:43:27Z","receivedAt":"2018-08-29T15:43:34Z","isPatch":false,"sender":{"key":"rybak.a.v@gmail.com","avatar":"https://avatars.githubusercontent.com/u/624072?v=4"},"body":"On 2018-08-29 12:02, Eric Wong wrote:\n> Anyways I hope to teach public-inbox to auto-linkify Message-ID-looking\n> strings \"\" into URLs for domain-portability,\n> (but it's ambiguous with email addresses). But yeah, I don't\n> like things being tied to domain names.\n\nThis would be very useful for people who use MUAs without\nMessage-ID lookup capabilities, even without domain-portability.\n"},{"id":"356853","messageId":"871sahdtj9.fsf@evledraar.gmail.com","threadId":"49225","inReplyTo":"3eb1c5e8-3e89-0d2e-30b1-339f38c4c703@gmail.com","subject":"Re: Thank you for public-inbox!","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-08-29T16:30:50Z","receivedAt":"2018-08-29T16:30:55Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Aug 29 2018, Andrei Rybak wrote:\n\n> On 2018-08-29 12:02, Eric Wong wrote:\n>> Anyways I hope to teach public-inbox to auto-linkify Message-ID-looking\n>> strings \"\" into URLs for domain-portability,\n>> (but it's ambiguous with email addresses). But yeah, I don't\n>> like things being tied to domain names.\n>\n> This would be very useful for people who use MUAs without\n> Message-ID lookup capabilities, even without domain-portability.\n\nFWIW Many MUAs have some hidden way to do this, for example to find the\nE-Mail I'm replying to (your E-Mail) on GMail:\nhttps://mail.google.com/mail/u/0/#search/rfc822msgid%3A3eb1c5e8-3e89-0d2e-30b1-339f38c4c703%40gmail.com\n\nI.e. rfc822msgid:\n\nConfusingly Google Groups accepts the same syntax, but will barf if you\ninclude the <>'s, but that's from memory, maybe I misrecall or they've\nfixed it.\n"},{"id":"356933","messageId":"20180830033000.GD665@sigill.intra.peff.net","threadId":"49225","inReplyTo":"20180829100243.GA28180@dcvr","subject":"Re: Thank you for public-inbox!","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-08-30T03:30:00Z","receivedAt":"2018-08-30T03:30:04Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Aug 29, 2018 at 10:02:43AM +0000, Eric Wong wrote:\n\n> Jeff King wrote:\n> > I've thought about mirroring it to a public server as well, just for\n> > redundancy. But without the same domain, I'm not sure it would be all\n> > that useful as a community resource.\n> \n> I wouldn't get too attached to the domain, \"public-inbox.org\" is\n> too long for my tastes anyways. \"peff.net/git/$MESSAGE_ID\"\n> would actually be more user-friendly :>\n> \n> A generic Message-ID redirection/finding service would be good,\n> (maybe some DHT thing, but... has that taken off for git blobs, yet?)\n\nYes, and I agree that the URL portability is one of the things I really\nlove about public-inbox (after all, I do have my own archive and now I\ncan follow people's public-inbox links into my very-fast local copy).\n\nI guess I just wonder if I set up a mirror on another domain, would\nanybody actually _use_ it? I'd think most people would just go to\npublic-inbox.org as the de facto URL.\n\n> Anyways I hope to teach public-inbox to auto-linkify Message-ID-looking\n> strings \"\" into URLs for domain-portability,\n> (but it's ambiguous with email addresses). But yeah, I don't\n> like things being tied to domain names.\n\nThat would be neat, but I think it actually makes references less useful\nin a lot of cases. URLs are universally understood, which means:\n\n - people who don't know about public-inbox can just follow the link\n (and in fact, that's how they learn how useful it is!)\n\n - even for people who do know about it, they are likely to read mails\n in their MUA. And most MUAs have some mechanism for easily following\n a URL, but won't know how to auto-linkify a message-id.\n\nSo I too dream of a world where I can say \"give me more information on\nthis identifier\" and my tools search a peer to peer distributed hash\ntable for it. But I don't think we live in that world yet.\n\nAt the very least, I think if we plan to reference without an http URL\nthat we would use something like URI-ish, like . That gives\ntools a better chance to say \"OK, I know how to find message-ids\"\n(though I still think that it's much less helpful out of the box\ncompared to an http URL).\n\n-Peff\n"},{"id":"356934","messageId":"20180830035609.GA140348@aiede.svl.corp.google.com","threadId":"49225","inReplyTo":"20180830033000.GD665@sigill.intra.peff.net","subject":"Re: Thank you for public-inbox!","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-08-30T03:56:09Z","receivedAt":"2018-08-30T03:56:15Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jeff King wrote:\n\n> I guess I just wonder if I set up a mirror on another domain, would\n> anybody actually _use_ it? I'd think most people would just go to\n> public-inbox.org as the de facto URL.\n\nIf it's faster than public-inbox.org and you don't mind the traffic I\nwould send, then I'll use it. :)\n\n[...]\n> That would be neat, but I think it actually makes references less useful\n> in a lot of cases. URLs are universally understood, which means:\n>\n> - people who don't know about public-inbox can just follow the link\n> (and in fact, that's how they learn how useful it is!)\n\nI agree: please don't stop using URLs.\n\nHaving the message-id in the URL is very useful for being able to\nmigrate to another server.\n\nIn an ideal world, we'd be habitually using URLs that reference a\nredirector, in the same spirit as https://www.kernel.org/lore.html.\nThat way, there is a very restricted syntax to use (e.g. just\nmessage-ids) that is stable and can be reconfigured to redirect to\nanother service as appropriate. In principle it also allows tricks\nlike redirecting based on geography or making the redirector go to the\nuser's preferred archive interface based on a cookie.\n\nThanks,\nJonathan\n"},{"id":"356943","messageId":"20180830072043.GA14514@dcvr.yhbt.net","threadId":"49225","inReplyTo":"20180830035609.GA140348@aiede.svl.corp.google.com","subject":"Re: Thank you for public-inbox!","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2018-08-30T07:20:43Z","receivedAt":"2018-08-30T07:20:45Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Jonathan Nieder wrote:\n> Jeff King wrote:\n> \n> > I guess I just wonder if I set up a mirror on another domain, would\n> > anybody actually _use_ it? I'd think most people would just go to\n> > public-inbox.org as the de facto URL.\n> \n> If it's faster than public-inbox.org and you don't mind the traffic I\n> would send, then I'll use it. :)\n\nIs performance a problem on public-inbox.org for you?\n\nIt should be pretty fast, but maybe there's corner-cases\nand I haven't had time to pay close attention, lately...\n\nI've also been sorta considering downgrading to a $5/month VPS\n(from a $20/month VPS) to force myself to pay more attention to\nperformance while saving myself a few bucks. But I wouldn't get\nto dogfood on SMP, anymore...\n"},{"id":"356944","messageId":"20180830072049.dgsjn3537xzo6qcm@dcvr","threadId":"49225","inReplyTo":"20180830033000.GD665@sigill.intra.peff.net","subject":"Re: Thank you for public-inbox!","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2018-08-30T07:20:49Z","receivedAt":"2018-08-30T07:20:51Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Jeff King wrote:\n> On Wed, Aug 29, 2018 at 10:02:43AM +0000, Eric Wong wrote:\n> > Anyways I hope to teach public-inbox to auto-linkify Message-ID-looking\n> > strings \"\" into URLs for domain-portability,\n> > (but it's ambiguous with email addresses). But yeah, I don't\n> > like things being tied to domain names.\n> \n> That would be neat, but I think it actually makes references less useful\n> in a lot of cases. URLs are universally understood, which means:\n> \n> - people who don't know about public-inbox can just follow the link\n> (and in fact, that's how they learn how useful it is!)\n> \n> - even for people who do know about it, they are likely to read mails\n> in their MUA. And most MUAs have some mechanism for easily following\n> a URL, but won't know how to auto-linkify a message-id.\n\nHeh, one of the (unstated?) goals of public-inbox is to educate\nthe users on how Message-IDs (and email in general) works.\nAnd to that end...\n\n> So I too dream of a world where I can say \"give me more information on\n> this identifier\" and my tools search a peer to peer distributed hash\n> table for it. But I don't think we live in that world yet.\n\n....More than dreaming, our goal should be to BUILD such a world :>\nAfter all, it was my intense dislike of centralization which\ndrew me to DVCS and git in the first place.\n\n> At the very least, I think if we plan to reference without an http URL\n> that we would use something like URI-ish, like . That gives\n> tools a better chance to say \"OK, I know how to find message-ids\"\n> (though I still think that it's much less helpful out of the box\n> compared to an http URL).\n\nThat would be awesome if somelike like could be a\nstandard and adopted (likewise with ).\n\nI haven't checked, but are there existing/similar RFCs?\nSurely somebody has tried to get \nadopted by now, right?\n"},{"id":"356951","messageId":"20180830081945.GB10224@sigill.intra.peff.net","threadId":"49225","inReplyTo":"20180830072049.dgsjn3537xzo6qcm@dcvr","subject":"Re: Thank you for public-inbox!","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-08-30T08:19:45Z","receivedAt":"2018-08-30T08:19:49Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Aug 30, 2018 at 07:20:49AM +0000, Eric Wong wrote:\n\n> > At the very least, I think if we plan to reference without an http URL\n> > that we would use something like URI-ish, like . That gives\n> > tools a better chance to say \"OK, I know how to find message-ids\"\n> > (though I still think that it's much less helpful out of the box\n> > compared to an http URL).\n> \n> That would be awesome if somelike like could be a\n> standard and adopted (likewise with ).\n> \n> I haven't checked, but are there existing/similar RFCs?\n> Surely somebody has tried to get \n> adopted by now, right?\n\nThere's https://tools.ietf.org/html/rfc2392. They even call it \"mid:\". :)\n\nI don't know of any git URI scheme, though it's similar in spirit to\nmagnet: links. Those are a bit verbose, though, because they're really a\nmeta-format for a bunch of different content-addressable schemes.\n\n-Peff\n"},{"id":"357059","messageId":"20180831011417.GA98946@aiede.svl.corp.google.com","threadId":"49225","inReplyTo":"20180830072043.GA14514@dcvr.yhbt.net","subject":"Re: Thank you for public-inbox!","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-08-31T01:14:17Z","receivedAt":"2018-08-31T01:14:23Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Eric Wong wrote:\n> Jonathan Nieder wrote:\n>> Jeff King wrote:\n\n>>> I guess I just wonder if I set up a mirror on another domain, would\n>>> anybody actually _use_ it? I'd think most people would just go to\n>>> public-inbox.org as the de facto URL.\n>>\n>> If it's faster than public-inbox.org and you don't mind the traffic I\n>> would send, then I'll use it. :)\n>\n> Is performance a problem on public-inbox.org for you?\n\nIt's pretty fast. I'm just very, very picky about latency. ;-)\n\nIt's good to know you're interested in which corner cases are bad.\nThe next time I have a noticeably slow page load, I'll contact meta@.\n\n[...]\n> I've also been sorta considering downgrading to a $5/month VPS\n> (from a $20/month VPS) to force myself to pay more attention to\n> performance while saving myself a few bucks. But I wouldn't get\n> to dogfood on SMP, anymore...\n\nSounds reasonable to me. If performance gets bad, that's just a\nreason for people to help out (either with patches or e.g. with\ndonated VMs for hosting).\n\nSpeaking of the latter: what are your current resource requirements?\nE.g. which of the dimensions in [1] do you not fit into?\n\nThanks,\nJonathan\n\n[1] https://cloud.google.com/free/docs/always-free-usage-limits#compute_name\n"},{"id":"357081","messageId":"20180831110120.vbyxnnpu2iparxck@dcvr","threadId":"49225","inReplyTo":"20180831011417.GA98946@aiede.svl.corp.google.com","subject":"Re: Thank you for public-inbox!","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2018-08-31T11:01:20Z","receivedAt":"2018-08-31T11:01:22Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Jonathan Nieder wrote:\n> Eric Wong wrote:\n> > Jonathan Nieder wrote:\n> >> Jeff King wrote:\n> \n> >>> I guess I just wonder if I set up a mirror on another domain, would\n> >>> anybody actually _use_ it? I'd think most people would just go to\n> >>> public-inbox.org as the de facto URL.\n> >>\n> >> If it's faster than public-inbox.org and you don't mind the traffic I\n> >> would send, then I'll use it. :)\n> >\n> > Is performance a problem on public-inbox.org for you?\n> \n> It's pretty fast. I'm just very, very picky about latency. ;-)\n\nBest way for good latency is to have a local mirror, but I guess\nGooglers still aren't allowed to run AGPL software?\n\n> It's good to know you're interested in which corner cases are bad.\n> The next time I have a noticeably slow page load, I'll contact meta@.\n\nAlright. It could also be a general datacenter/networking\nproblem so https://status.linode.com/ (my VPS provider) is worth\nchecking.\n\n> [...]\n> > I've also been sorta considering downgrading to a $5/month VPS\n> > (from a $20/month VPS) to force myself to pay more attention to\n> > performance while saving myself a few bucks. But I wouldn't get\n> > to dogfood on SMP, anymore...\n> \n> Sounds reasonable to me. If performance gets bad, that's just a\n> reason for people to help out (either with patches or e.g. with\n> donated VMs for hosting).\n\n> Speaking of the latter: what are your current resource requirements?\n\nNot too much; but could always be better on the software side.\n\n> E.g. which of the dimensions in [1] do you not fit into?\n\n> [1] https://cloud.google.com/free/docs/always-free-usage-limits#compute_name\n\nDunno, I'm not seeing RAM, there.\n\nDepending on traffic, it's around 200MB per-public-inbox-httpd\nworker (2 workers for 2 cores) when there's a traffic surge on\nfrom popular sites. Memory usage is the biggest disappointment\nand only happens when Varnish can't read fast enough.\nEverything in the PSGI code is is streamed if possible(*).\nMy goal is to maintain <50MB per worker process, but it could\nbe tough in Perl5. Anyways $20/month gets me 4GB RAM (so I have\nway more than I need).\n\nCPU usage isn't even noticeable (only bursts) and I do other\nstuff on that server all the time.\n\nHDDs wouldn't work well at all and I've noticed differences\nbased on SSD quality with Xapian. Storage for Xapian+SQLite is\n4-5x what's in git, so for this list, it's under 7G total\n(but more will be needed for Xapian reindexing/compact and\ngit repacking).\n\n\n\n(*) Individual messages for returning giant mboxes and threads are\n all lazily fleshed out from skeleton data structures as the\n client socket becomes writable (and quickly discarded after writing).\n Technically it's all compatible with any PSGI server, but all\n the streaming stuff is tailored to run on public-inbox-httpd.\n But there's also git-http-backend memory use which comes in\n bursts (bitmaps enabled, of course)\n"}]}