{"thread":{"id":"46923","subject":"[RFC] Reporting index properties","startedAt":"2017-10-05T23:39:31Z","lastAt":"2017-10-06T19:26:53Z","messageCount":7,"participants":["Alex Vandiver","Junio C Hamano","Jonathan Nieder"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"329824","messageId":"alpine.DEB.2.10.1710051625430.812@alexmv-linux","threadId":"46923","inReplyTo":null,"subject":"[RFC] Reporting index properties","fromName":"Alex Vandiver","fromEmail":"alexmv@dropbox.com","sentAt":"2017-10-05T23:39:20Z","receivedAt":"2017-10-05T23:39:31Z","isPatch":false,"sender":{"key":"alexmv@dropbox.com","avatar":null},"body":"Heya,\n\nAs part of gathering some automated performance statistics of git, it\nwould be useful to be able to extract some vital properties of the\nindex.  While `git update-index` has ways to _set_ the index version,\nand enable/disable various extensions, AFAICT there is no method by\nwhich one can ask for reporting about the current state of them --\nshort of re-implementing all of the index-parsing logic external to\nGit, which is not terribly appealing.\n\nI hesitate to propose adding a flag to `git update-index` which reads\nbut does no updating, as that seems to make a misnomer of the\ncommand.  But this also doesn't seem worthy of a new top-level command.\n\nDo folks have feelings about surfacing this information, and where such\nlogic should live?\n\n - Alex\n"},{"id":"329826","messageId":"xmqqzi955cnx.fsf@gitster.mtv.corp.google.com","threadId":"46923","inReplyTo":"alpine.DEB.2.10.1710051625430.812@alexmv-linux","subject":"Re: [RFC] Reporting index properties","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-10-06T00:43:14Z","receivedAt":"2017-10-06T00:45:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Alex Vandiver <alexmv@dropbox.com> writes:\n\n> Do folks have feelings about surfacing this information, and where such\n> logic should live?\n\nProbably one of the t/helper/test-dump-*.c programs, if we already\ndo not have one.\n"},{"id":"329827","messageId":"alpine.DEB.2.10.1710051745060.812@alexmv-linux","threadId":"46923","inReplyTo":"xmqqzi955cnx.fsf@gitster.mtv.corp.google.com","subject":"Re: [RFC] Reporting index properties","fromName":"Alex Vandiver","fromEmail":"alexmv@dropbox.com","sentAt":"2017-10-06T00:47:36Z","receivedAt":"2017-10-06T00:47:47Z","isPatch":false,"sender":{"key":"alexmv@dropbox.com","avatar":null},"body":"On Fri, 6 Oct 2017, Junio C Hamano wrote:\n> > Do folks have feelings about surfacing this information, and where such\n> > logic should live?\n> \n> Probably one of the t/helper/test-dump-*.c programs, if we already\n> do not have one.\n\nThe goal would be to be able to extract this information from\nrepositories, with a standard git install.  That directory only\ncontains developer tools, which aren't part of the install, no?\n\n - Alex\n"},{"id":"329836","messageId":"xmqqzi95xamw.fsf@gitster.mtv.corp.google.com","threadId":"46923","inReplyTo":"alpine.DEB.2.10.1710051745060.812@alexmv-linux","subject":"Re: [RFC] Reporting index properties","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-10-06T02:39:35Z","receivedAt":"2017-10-06T02:39:41Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Alex Vandiver <alexmv@dropbox.com> writes:\n\n> On Fri, 6 Oct 2017, Junio C Hamano wrote:\n>> > Do folks have feelings about surfacing this information, and where such\n>> > logic should live?\n>> \n>> Probably one of the t/helper/test-dump-*.c programs, if we already\n>> do not have one.\n>\n> The goal would be to be able to extract this information from\n> repositories, with a standard git install.  That directory only\n> contains developer tools, which aren't part of the install, no?\n\nYes, and I am saying that such logic should not live in standard\ninstall outside developer tools ;-)\n"},{"id":"329837","messageId":"20171006031303.GB134987@aiede.mtv.corp.google.com","threadId":"46923","inReplyTo":"alpine.DEB.2.10.1710051625430.812@alexmv-linux","subject":"Re: [RFC] Reporting index properties","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2017-10-06T03:13:03Z","receivedAt":"2017-10-06T03:13:11Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Alex Vandiver wrote:\n\n> As part of gathering some automated performance statistics of git, it\n> would be useful to be able to extract some vital properties of the\n> index.  While `git update-index` has ways to _set_ the index version,\n> and enable/disable various extensions, AFAICT there is no method by\n> which one can ask for reporting about the current state of them --\n> short of re-implementing all of the index-parsing logic external to\n> Git, which is not terribly appealing.\n\nAside: git-update-index(1) says\n\n\tVersion 4 is relatively young (first released in 1.8.0 in\n\tOctober 2012).\n\nMy first reaction is to wonder if it is time to introduce a config\noption to use index format version 4 by default, since after 5 years\nit is not relatively young any more.\n\nMy second reaction is to notice that the index.version configuration\nalready exists.  Probably git-update-index(1) ought to point to it.\n\nJGit still doesn't support index format 4, so 4 is still not a good\ndefault for `index.version` for a user that hasn't explicitly\nconfigured it, but the moment JGit gains that support I think it will\nbe. :)\n\n> I hesitate to propose adding a flag to `git update-index` which reads\n> but does no updating, as that seems to make a misnomer of the\n> command.  But this also doesn't seem worthy of a new top-level command.\n\nThe existing scripting command that inspects the index is\n\"git ls-files\".\n\nIt doesn't go into this kind of low-level detail about the index\nformat, though.  In the same spirit, I don't think we have an existing\ncommand in this spirit for analyzing packfiles, either.\n\nSo I agree with Junio that this would be a good debugging aid to put\nin t/helper/ for now, and then once we've had experience with it we'd\nend up in a better position to come up with a stable, public-facing\ninterface, if one is needed.\n\nThanks and hope that helps,\nJonathan\n"},{"id":"329840","messageId":"xmqqvajsyldn.fsf@gitster.mtv.corp.google.com","threadId":"46923","inReplyTo":"20171006031303.GB134987@aiede.mtv.corp.google.com","subject":"Re: [RFC] Reporting index properties","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-10-06T04:02:12Z","receivedAt":"2017-10-06T04:02:26Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> Aside: git-update-index(1) says\n>\n> \tVersion 4 is relatively young (first released in 1.8.0 in\n> \tOctober 2012).\n>\n> My first reaction is to wonder if it is time to introduce a config\n> option to use index format version 4 by default, since after 5 years\n> it is not relatively young any more.\n>\n> My second reaction is to notice that the index.version configuration\n> already exists.  Probably git-update-index(1) ought to point to it.\n\nBoth good points.\n\n> JGit still doesn't support index format 4, so 4 is still not a good\n> default for `index.version` for a user that hasn't explicitly\n> configured it, but the moment JGit gains that support I think it will\n> be. :)\n\nAfter reading ce_write_entry(), I tend to agree.  v4 is the first\none that loses the alignment padding, which made sense back when we\nmmap(2)ed the index file and read from it directly without the\nondisk-vs-incore conversion, but that stopped to be the case several\nyears ago.\n\nI find it interesting to find that the recently proposed design for\nreftable uses the same \"common prefix compression from the previous\nname\" trick, even though index v4 does not have any resync point\nlike reftable does.\n"},{"id":"329919","messageId":"alpine.DEB.2.10.1710061225030.18016@alexmv-linux","threadId":"46923","inReplyTo":"xmqqzi95xamw.fsf@gitster.mtv.corp.google.com","subject":"Re: [RFC] Reporting index properties","fromName":"Alex Vandiver","fromEmail":"alexmv@dropbox.com","sentAt":"2017-10-06T19:26:41Z","receivedAt":"2017-10-06T19:26:53Z","isPatch":false,"sender":{"key":"alexmv@dropbox.com","avatar":null},"body":"On Fri, 6 Oct 2017, Junio C Hamano wrote:\n> Yes, and I am saying that such logic should not live in standard\n> install outside developer tools ;-)\n\nFair enough.  It seems a little odd to me that git provides logic for\n_altering_ those bits, but not to report on the state that can be so\naltered.\n\n - Alex\n"}]}