{"thread":{"id":"30450","subject":"post-fetch, tweak-fetch hook","startedAt":"2012-05-06T20:52:53Z","lastAt":"2012-05-07T13:38:10Z","messageCount":6,"participants":["Mitar","Seth Robertson","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"190938","messageId":"CAKLmikNaqVRb=pGUhbvVQTX2tYWT0HSS2R6Ezmico3X0rMgvYQ@mail.gmail.com","threadId":"30450","inReplyTo":null,"subject":"post-fetch, tweak-fetch hook","fromName":"Mitar","fromEmail":"mmitar@gmail.com","sentAt":"2012-05-06T20:52:53Z","receivedAt":"2012-05-06T20:52:53Z","isPatch":false,"sender":{"key":"mmitar@gmail.com","avatar":null},"body":"Hi!\n\nI am writing a plugin which allows syncing of GitHub repositories with\nlocal Trac mirrored ones.\n\nhttp://trac-hacks.org/wiki/GitHubSyncPlugin\n\nIn this configuration, I have a git clone --mirror local repository of\nGitHub repository and on each push to GitHub repository, GitHub does a\nPOST notification to my Trac installation, where my Trac plugin\nreceives that notification and calls fetch on local mirror.\n\nThe problem is that I also have to notify Trac of all new revisions\n(their hashes) the fetch retrieved. If this would be push,\npost-receive hook would be a place to get this information. But as I\nread, there is no post-fetch hook. The argument is that it is not\nneeded. That locally run fetch can also run needed post-processing.\nBut I have three counter arguments to this, based on my current\nexperience:\n\nCode reuse: having same interface for both post-fetch and post-receive\nhooks would mean easier post-processing. They are similar and I can\nimagine that there exist many scenarios where same script would be\nused for both hooks. Together with this is also maintainability: you\ncould make a myfetch command doing custom post-processing, but\ninterfacing this with a known API through hooks, all in one directory,\nwould make things much easier to maintain. Not that one script is in\nhooks (post-receive) and the other is invoked in some other manner.\n\nI am a git newbie, but after few hours or reading and searching I have\nnot found a simple way to get a list of revisions retrieved by a\nfetch, so that I could call my custom post-processing. If this is\nreally so simple that there is no need for post-fetch hook, I am all\nears. FETCH_HEAD file contains only last revisions for each branch,\nnot a range (old-new, like post-receive). Furthermore, even if there\nhave been no new revisions retrieved, FETCH_HEAD file stays at its old\nstate (not for example deleting it), so some additional logic would be\nneeded. Parsing the output of git fetch also does not look like\nsomething easily parsed by a program.\n\nEven if there is a way to reconstruct data passed to post-receive (to\nbe given to post-fetch), I am concerned about race-condition of this.\nBecause in post-receive this data is tightly connected to the push\nbeing done. Even if there is another push in process at the same time,\ngit would take care post-receive is called with exactly those\nrevisions. In case of reconstructing those revisions, it could happen,\nthat two fetches overlap in such a manner that in my custom myfetch\nscript I reconstruct same revisions for both fetches (for example,\nread same version of FETCH_HEAD file which has been updated twice by\nfetch, but I would get only one version), while there were of course\ndifferent.\n\nI have found some work from Joey Hess on tweak-fetch hook and I would\nreally welcome such addition. Maybe my problems stem only from me\nbeing a git novice, but post-fetch (tweak-fetch) hook would really\nreally make things simple and intuitive also for such users.\n\nSo I would kindly ask for some advice on how to get in a safe manner\ndata similar to what is provided to post-receive.\n\n\nMitar\n"},{"id":"190947","messageId":"201205062310.q46NAHnM022630@no.baka.org","threadId":"30450","inReplyTo":"CAKLmikNaqVRb=pGUhbvVQTX2tYWT0HSS2R6Ezmico3X0rMgvYQ@mail.gmail.com","subject":"Re: post-fetch, tweak-fetch hook","fromName":"Seth Robertson","fromEmail":"in-gitvger@baka.org","sentAt":"2012-05-06T23:10:17Z","receivedAt":"2012-05-06T23:10:17Z","isPatch":false,"sender":{"key":"in-gitvger@baka.org","avatar":null},"body":"\nIn message , Mitar writes:\n\n I am a git newbie, but after few hours or reading and searching I have\n not found a simple way to get a list of revisions retrieved by a\n fetch\n\nThe output of fetch seems to do that, quite nicely.\n\n----------------------------------------------------------------------\n> git fetch\nremote: Counting objects: 24155, done.\nremote: Compressing objects: 100% (6651/6651), done.\nremote: Total 21446 (delta 15831), reused 20146 (delta 14640)\nReceiving objects: 100% (21446/21446), 6.78 MiB | 239 KiB/s, done.\nResolving deltas: 100% (15831/15831), completed with 574 local objects.\nFrom git://git.kernel.org/pub/scm/git/git\n ea2c69e..edf1412 maint -> origin/maint\n ae4479d..8275905 master -> origin/master\n + b6b16ad...8a79d96 next -> origin/next (forced update)\n + 47db9a0...30b8c95 pu -> origin/pu (forced update)\n ce29fc8..3ca5cbc todo -> origin/todo\n----------------------------------------------------------------------\n\nOK, ignoring that output:\n\n----------------------------------------------------------------------\n> git branch -r | grep -v ' -> ' | while read b; do git reflog -n 1 \"$b\"; done\nedf1412 refs/remotes/origin/maint@{0}: fetch: fast-forward\nea2c69e 8275905 refs/remotes/origin/master@{0}: fetch: fast-forward\nae4479d 8a79d96 refs/remotes/origin/next@{0}: fetch: forced-update\nb6b16ad 30b8c95 refs/remotes/origin/pu@{0}: fetch: forced-update\n47db9a0 3ca5cbc refs/remotes/origin/todo@{0}: fetch: fast-forward\n----------------------------------------------------------------------\n\nThe reflog, of course, only gives you the latest change for each\nbranch, which means that two fetches in a row will return the same\noutput if no changes were received. Of course there is the classic:\n\n----------------------------------------------------------------------\ngit for-each-ref | pcregrep 'commit\\srefs/remotes/' > /tmp/old\ngit fetch\ngit for-each-ref | pcregrep 'commits\\srefs/remotes/' > /tmp/new\ndiff /tmp/old /tmp/new\n----------------------------------------------------------------------\n\nI'm in favor of more git hooks myself, but there is a solution to your\nneeds without it.\n\n Even if there is a way to reconstruct data passed to post-receive (to\n be given to post-fetch), I am concerned about race-condition of this.\n\nIf you care about race conditions (and really, a lockfile(1) call can\ntake care of that easily enough), then parse the output of fetch which\nwill make it clear what *this* call did.\n\n\t\t\t\t\t-Seth Robertson\n"},{"id":"190950","messageId":"CAKLmikNYewaRL3DUG9+NvpH3Y6sf=oJVdOo2DQSDw4vKH+Km-Q@mail.gmail.com","threadId":"30450","inReplyTo":"201205062310.q46NAHnM022630@no.baka.org","subject":"Re: post-fetch, tweak-fetch hook","fromName":"Mitar","fromEmail":"mmitar@gmail.com","sentAt":"2012-05-06T23:54:40Z","receivedAt":"2012-05-06T23:54:40Z","isPatch":false,"sender":{"key":"mmitar@gmail.com","avatar":null},"body":"Hi!\n\nThank you for examples. They guide me in good direction. But still ...\n\nOn Mon, May 7, 2012 at 1:10 AM, Seth Robertson wrote:\n> I'm in favor of more git hooks myself, but there is a solution to your\n> needs without it.\n\nOf course there is a solution. Bash is turing-complete programming\nlanguage. But the question is how easy and fast it is to use\nsomething.\n\n> If you care about race conditions (and really, a lockfile(1) call can\n> take care of that easily enough),\n\nIf git would support post-fetch hook, I would get this for free and\nwould not have to care about race conditions.\n\n> then parse the output of fetch which will make it clear what *this* call did.\n\nYes, because it is really easy to parse it? There are so many\ndifferent things it can output and I am not sure how to find which one\nshould I take care of and which one should I ignore. Yes, I could\nlearn much more about git and all possible things which can happen\nwith fetch and its output, but I am trying to avoid this. Why? Because\nI believe it should be much easier.\n\nI am sorry if I look a bit negative, but I am quite frustrated that\nfor something so simple is so hard (in a time/knowledge meaning) to\nachieve. Of course there is a way and that if you know all git command\nthis is maybe obvious, but why it couldn't be easier, if it is really\na very simple patch to make it easier (and it was already posted to\nthis mailing list few months ago).\n\n\nMitar\n"},{"id":"190965","messageId":"20120507072934.GC19874@sigill.intra.peff.net","threadId":"30450","inReplyTo":"201205062310.q46NAHnM022630@no.baka.org","subject":"Re: post-fetch, tweak-fetch hook","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-05-07T07:29:34Z","receivedAt":"2012-05-07T07:29:34Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, May 06, 2012 at 07:10:17PM -0400, Seth Robertson wrote:\n\n> The output of fetch seems to do that, quite nicely.\n> \n> ----------------------------------------------------------------------\n> > git fetch\n> remote: Counting objects: 24155, done.\n> remote: Compressing objects: 100% (6651/6651), done.\n> remote: Total 21446 (delta 15831), reused 20146 (delta 14640)\n> Receiving objects: 100% (21446/21446), 6.78 MiB | 239 KiB/s, done.\n> Resolving deltas: 100% (15831/15831), completed with 574 local objects.\n> From git://git.kernel.org/pub/scm/git/git\n> ea2c69e..edf1412 maint -> origin/maint\n> ae4479d..8275905 master -> origin/master\n> + b6b16ad...8a79d96 next -> origin/next (forced update)\n> + 47db9a0...30b8c95 pu -> origin/pu (forced update)\n> ce29fc8..3ca5cbc todo -> origin/todo\n> ----------------------------------------------------------------------\n\nThis output is human-consumable, and is not guaranteed to remain stable\nin future versions of git. Push has a --porcelain mode for this reason,\nbut nobody has bothered to implement it for fetch.\n\n> If you care about race conditions (and really, a lockfile(1) call can\n> take care of that easily enough), then parse the output of fetch which\n> will make it clear what *this* call did.\n\nCustom locking is not sufficient, as a push could modify refs behind\nyour back. I guess you could get by with a pre-receive hook that also\ntook the lock. But that is unnecessarily crappy; git does not have a\nwhole repo lock, and there is no need for lock contention between pushes\nand fetches that are touching different refs.\n\nI would say the \"most git\" thing would be to implement \"fetch\n--porcelain\", and use its output.\n\n-Peff\n"},{"id":"190972","messageId":"CAKLmikNuUB01xKSm9Skd2chXWw3BcWDHT23hqWtNBJPJfYqDKQ@mail.gmail.com","threadId":"30450","inReplyTo":"20120507072934.GC19874@sigill.intra.peff.net","subject":"Re: post-fetch, tweak-fetch hook","fromName":"Mitar","fromEmail":"mmitar@gmail.com","sentAt":"2012-05-07T09:11:30Z","receivedAt":"2012-05-07T09:11:30Z","isPatch":false,"sender":{"key":"mmitar@gmail.com","avatar":null},"body":"Hi!\n\nOn Mon, May 7, 2012 at 9:29 AM, Jeff King wrote:\n> I would say the \"most git\" thing would be to implement \"fetch\n> --porcelain\", and use its output.\n\nYes, that would be also useful. It still makes two different\ninterfaces for probably same post-processing (after push and after\nfetch), but still better than nothing, what is current state.\n\n\nMitar\n"},{"id":"190997","messageId":"20120507133810.GA4860@sigill.intra.peff.net","threadId":"30450","inReplyTo":"CAKLmikNuUB01xKSm9Skd2chXWw3BcWDHT23hqWtNBJPJfYqDKQ@mail.gmail.com","subject":"Re: post-fetch, tweak-fetch hook","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-05-07T13:38:10Z","receivedAt":"2012-05-07T13:38:10Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 07, 2012 at 11:11:30AM +0200, Mitar wrote:\n\n> On Mon, May 7, 2012 at 9:29 AM, Jeff King wrote:\n> > I would say the \"most git\" thing would be to implement \"fetch\n> > --porcelain\", and use its output.\n> \n> Yes, that would be also useful. It still makes two different\n> interfaces for probably same post-processing (after push and after\n> fetch), but still better than nothing, what is current state.\n\nThere is nothing to say that the output from \"git fetch\" could not look\nexactly like the post-receive hook's input (in fact, that seems like a\nvery simple and sensible format). Then you could reuse the code easily.\n\nThey would still differ in that one is a hook and one is not, of course.\nBut at the same time, not being a hook leaves the caller of \"git fetch\"\nwith much more flexibility about deciding when to call the hook and when\nnot (whereas push does not have that luxury, because the code is running\non the remote side).\n\n-Peff\n"}]}