{"thread":{"id":"59373","subject":"A nice, beauty progress metter for Git Clone + a feature request","startedAt":"2023-03-10T11:38:29Z","lastAt":"2023-03-28T19:51:05Z","messageCount":5,"participants":["Sebastian Gniazdowski","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"473351","messageId":"CAKc7PVA14_mjZeuED1Ee0Mu5YoCSrR4FNNhSEKFHmnKnc-gESQ@mail.gmail.com","threadId":"59373","inReplyTo":"CAKc7PVD_65vB5+meeO3xcu4ASbqr85LBGUO8Ntb7SvbO+NNHmQ@mail.gmail.com","subject":"A nice, beauty progress metter for Git Clone + a feature request","fromName":"Sebastian Gniazdowski","fromEmail":"sgniazdowski@gmail.com","sentAt":"2023-03-10T11:37:49Z","receivedAt":"2023-03-10T11:38:29Z","isPatch":false,"sender":{"key":"sgniazdowski@gmail.com","avatar":"https://gravatar.com/avatar/4fe8402ad2382744749d089f9c26c9a9fd58c4265afcc0a945d2869cad528e39?d=mp&s=160"},"body":"Check out: https://asciinema.org/a/566216\n\nFeature request is: add a pipe={cmd} option to git, that would just\npass through the git clone --progress output into {cmd}. This is\nEXTREMELY EASY TO DO SO, ONLY\n\nFILE *pipe=popen(cmd_option_str, \"w\") ;\n\nis needed, and one can then write to `pipe` file handler.\n\n-- \nBest regards,\nSebastian Gniazdowski\n"},{"id":"473560","messageId":"ZBIIKqAIF+aF8vvx@coredump.intra.peff.net","threadId":"59373","inReplyTo":"CAKc7PVA14_mjZeuED1Ee0Mu5YoCSrR4FNNhSEKFHmnKnc-gESQ@mail.gmail.com","subject":"Re: A nice, beauty progress metter for Git Clone + a feature request","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-03-15T18:02:18Z","receivedAt":"2023-03-15T18:02:34Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 10, 2023 at 11:37:49AM +0000, Sebastian Gniazdowski wrote:\n\n> Check out: https://asciinema.org/a/566216\n> \n> Feature request is: add a pipe={cmd} option to git, that would just\n> pass through the git clone --progress output into {cmd}. This is\n> EXTREMELY EASY TO DO SO, ONLY\n> \n> FILE *pipe=popen(cmd_option_str, \"w\") ;\n> \n> is needed, and one can then write to `pipe` file handler.\n\nI think it would be neat to have options for prettier progress output.\nUnfortunately, it's not quite that easy.\n\nImagine we had a config option, progress.command or something, which\nspecified a shell command to which we'd pipe machine-readable progress,\nand it would turn it into something pretty. There are two gotchas I can\nthink of immediately:\n\n 1. Progress is often created by multiple processes. For example, in a\n clone, you might see progress output from index-pack as it receives\n the pack and resolves the delta. But then you might also see\n progress during the checkout phase, as we write out the worktree\n files.\n\n One solution is to say that each progress command is independent,\n and we'd simply run it twice, once for each context. That's easy to\n do, but restricts some fancier options that a command could do.\n\n Another is for the parent clone command to kick off a progress\n process, and then tell child processes (perhaps through the\n environment) which descriptor they can use to talk to the progress\n command.\n\n 2. Sometimes progress is shoved through a single descriptor along with\n other human-readable messages. I'm thinking specifically of the\n output you get from a clone/fetch _before_ we start receiving pack\n bytes from the server. There the server is giving progress on\n preparing the fetch. It comes over a sideband channel through Git's\n protocol, along with any regular stderr messages, and the client\n side just dumps it all to the local stderr.\n\n Obviously the remote side is not going to run our custom progress\n command. But it would be possible to add a protocol extension that\n says \"hey, please send machine-readable progress data over sideband\n 4\" or something.\n\nNow, none of that is _strictly_ necessary. We could just dump the same\nhuman-readable progress to the progress command that we dump to stderr\nnow, and it could scrape it for things that look like progress, and pass\neverything else through. But then, you can already do that with:\n\n git clone --progress ... 2>&1 | my-progress-scraper\n\n(and it looks like the asciinema you showed is probably just a\nsyntactically nicer version of that with support from the shell?).\n\n-Peff\n"},{"id":"474231","messageId":"CAKc7PVDZ4Ra7s-pyFms1-sGqjYD9b407G1u0A0hHTkQRa-MPkQ@mail.gmail.com","threadId":"59373","inReplyTo":"ZBIIKqAIF+aF8vvx@coredump.intra.peff.net","subject":"Re: A nice, beauty progress metter for Git Clone + a feature request","fromName":"Sebastian Gniazdowski","fromEmail":"sgniazdowski@gmail.com","sentAt":"2023-03-27T10:59:56Z","receivedAt":"2023-03-27T11:00:36Z","isPatch":false,"sender":{"key":"sgniazdowski@gmail.com","avatar":"https://gravatar.com/avatar/4fe8402ad2382744749d089f9c26c9a9fd58c4265afcc0a945d2869cad528e39?d=mp&s=160"},"body":"On Wed, 15 Mar 2023 at 18:02, Jeff King wrote:\n> Now, none of that is _strictly_ necessary. We could just dump the same\n> human-readable progress to the progress command that we dump to stderr\n> now, and it could scrape it for things that look like progress, and pass\n> everything else through. But then, you can already do that with:\n>\n> git clone --progress ... 2>&1 | my-progress-scraper\n>\n> (and it looks like the asciinema you showed is probably just a\n> syntactically nicer version of that with support from the shell?).\n>\n> -Peff\n\nYes, that's what the asciinema does – it pipes git clone --progress\nto a script, and that's pretty much it. That's why I thought about a\ncore.pipe option, that would hold the my-progress-scrapper command.\nThat's a very easy change, could it be added to git source code? The\nscrapper from the asciinema doesn't care if it's count, retrieve or\nresolve, etc. currently ongoing, it simply displays a gauge with the\ncurrent percentage for the current stage. After a while one discovers\nthat it's resolve that's the final stage, and the gauge is intuitive\neither before or after this revelation.\n\n\n\n\n\n-- \nBest regards,\nSebastian Gniazdowski\n"},{"id":"474336","messageId":"20230328175139.GA18558@coredump.intra.peff.net","threadId":"59373","inReplyTo":"CAKc7PVDZ4Ra7s-pyFms1-sGqjYD9b407G1u0A0hHTkQRa-MPkQ@mail.gmail.com","subject":"Re: A nice, beauty progress metter for Git Clone + a feature request","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-03-28T17:51:39Z","receivedAt":"2023-03-28T17:51:50Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 27, 2023 at 10:59:56AM +0000, Sebastian Gniazdowski wrote:\n\n> On Wed, 15 Mar 2023 at 18:02, Jeff King wrote:\n> > Now, none of that is _strictly_ necessary. We could just dump the same\n> > human-readable progress to the progress command that we dump to stderr\n> > now, and it could scrape it for things that look like progress, and pass\n> > everything else through. But then, you can already do that with:\n> >\n> > git clone --progress ... 2>&1 | my-progress-scraper\n> >\n> > (and it looks like the asciinema you showed is probably just a\n> > syntactically nicer version of that with support from the shell?).\n> \n> Yes, that's what the asciinema does – it pipes git clone --progress\n> to a script, and that's pretty much it. That's why I thought about a\n> core.pipe option, that would hold the my-progress-scrapper command.\n> That's a very easy change, could it be added to git source code?\n\nFor the reasons I gave earlier, it has to scrape all of stderr, and not\njust progress output. I'm still skeptical that this buys a lot over just\nredirecting stderr entirely via the shell, but it's at least\nconceptually simple (it's basically the same as a pager for stderr).\n\nThe implementation might have some tricky bits to think about, though,\nsince we have an extra process to handle. For real pagers, we take some\ncare to terminate them on close. Maybe we wouldn't need the same here,\nthough (since the filter isn't interactive, and we don't want Git to\nwait on exiting until it's done).\n\nSo I dunno. It might only be a few lines. It just feels like a weird\nfeature to add.\n\n-Peff\n"},{"id":"474358","messageId":"CAKc7PVDUaQfcdZggraN15bnWs1M_Wk6n6u25unW+Mo5GCaEykQ@mail.gmail.com","threadId":"59373","inReplyTo":"20230328175139.GA18558@coredump.intra.peff.net","subject":"Re: A nice, beauty progress metter for Git Clone + a feature request","fromName":"Sebastian Gniazdowski","fromEmail":"sgniazdowski@gmail.com","sentAt":"2023-03-28T20:49:43Z","receivedAt":"2023-03-28T19:51:05Z","isPatch":false,"sender":{"key":"sgniazdowski@gmail.com","avatar":"https://gravatar.com/avatar/4fe8402ad2382744749d089f9c26c9a9fd58c4265afcc0a945d2869cad528e39?d=mp&s=160"},"body":"On Tue, 28 Mar 2023 at 17:51, Jeff King wrote:\n> So I dunno. It might only be a few lines. It just feels like a weird\n> feature to add.\n\nI would perceive the feature as a nice, versatile trick instead of\nweird… Current git clone progress is quite unexpected – when I was\nstarting to use Git I've had a sensation of disappointment each time\nI've run clone – was expecting some nice progress bar and was served\nwith bunch of text lines with numbers that weren't any percentages…\nI'm not saying that the progress in the asciinema is some uber cool\nsolution, but then, it may serve as an example to draw attention to\nbeautifying the progress and the freedom of hooking any command via a\ncore.progress_pipe might result in some other nice scripts/apps to\nbloom out there.\n\n-- \nBest regards,\nSebastian Gniazdowski\n"}]}