{"thread":{"id":"13132","subject":"fsck --full is Ok, but clones are not, \"missing commits\"?!","startedAt":"2008-04-16T06:37:39Z","lastAt":"2008-05-06T11:12:51Z","messageCount":7,"participants":["Brian Foster","David Kastrup","Bryan Donlan","Johannes Sixt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"74526","messageId":"20080416063739.4B72647879@blf.utvinternet.co.uk","threadId":"13132","inReplyTo":"20080416062925.8028e952@zebulon.innova-card.com","subject":"fsck --full is Ok, but clones are not, \"missing commits\"?!","fromName":"Brian Foster","fromEmail":"brian.foster@innova-card.com","sentAt":"2008-04-16T06:37:39Z","receivedAt":"2008-04-16T06:37:39Z","isPatch":false,"sender":{"key":"brian.foster@innova-card.com","avatar":null},"body":" (many many apologies if this turns into a double post,\n there seems to have been problems with the 1st attempt?)\n\n I've recently inherited a bare git repository,\n which, as far as I can tell (I'm something of\n a newbie with git), seems Ok: `git fsck --full'\n does not report any problems. however, any\n clones I make from it are not Ok:\n\n\t$ git-fsck --full # clone (same command for bare repo is Ok)\n\tbroken link from commit dd3f3c0636cfd50719c706b030db5473b0270add\n\t\t to commit 0fed9c2eb14eee47097e1d870fe8e55a6430edeb\n\tmissing commit fb57c018d15005b60f104e57f198ff34a6035b99\n\tmissing commit f8947cb0b5fe605e6cb5f73c89f262424b64ef3c\n\tmissing commit 0fed9c2eb14eee47097e1d870fe8e55a6430edeb\n\tmissing commit dff364d8da15be0b856a174062fb785acb1c363e\n\tbroken link from commit 8700854c41a40d333e90104971c3abbbcf082e57\n\t\t to commit b8b78d1c09e7009f97a1d624f4d771d7e5bd8551\n\tmissing commit ce33a5e8c32816f38862bc41560a1d646d0803a6\n\tmissing commit 91483db9b01b547ae9cc45c8c98b217642acb40a\n\tmissing commit b8b78d1c09e7009f97a1d624f4d771d7e5bd8551\n\tbroken link from commit c87c46fe892211f8aa4fd363ccff4f667a9aaf7d\n\t\t to commit ce33a5e8c32816f38862bc41560a1d646d0803a6\n\tbroken link from commit fb5967688f7b464421cff28f266b64ad2a313a9e\n\t\t to commit f8947cb0b5fe605e6cb5f73c89f262424b64ef3c\n\tbroken link from commit e5a60f1636cceac33777bb8098a0b7a4a136a56c\n\t\t to commit fb57c018d15005b60f104e57f198ff34a6035b99\n\tbroken link from commit 2dcaaf2decd31ac9a21d616604c0a7c1fa65d5a4\n\t\t to commit 91483db9b01b547ae9cc45c8c98b217642acb40a\n\tbroken link from commit 0ff75b3afff6fb306bef221bf1823ccf5ffc568b\n\t\t to commit dff364d8da15be0b856a174062fb785acb1c363e\n\t$ \n\n the Ok(?) bare repository only has a `master' branch.\n it has numerous tags. AFAIK, it has been \"inactive\"\n for the last c.6+ months (or longer?), except (maybe)\n for browsing, probably with gitweb. there have been,\n as far as I can tell, no commits or other \"writes\"\n for the last c.6+ months.\n\n to my extreme disgust, it was never(?!) backed-up,\n except for one not entirely coincidental backup,\n also c.6+ months ago. ;-( (this mind-boggling\n silliness is being fixed ASAP!)\n\n this is happening regardless of whether the clone\n is made on the same machine or a different one,\n regardless of the transport, and for both `--bare'\n and not-bare clones. nothing fancy is being done\n during the cloning: `git clone [--bare] DIR_or_URL'\n\n a test (tried by someone else) using that one\n backup apparently produced the same results: the\n backup seems Ok, but clones made from it are not.\n\n the missing commits are linux-mips.org/kernel.org\n mergers (mostly). apologies for being vague here,\n but I don't have my notes with me. ;-(\n\n what is going on? I'm completely baffled! to-date,\n nothing terribly obvious/similar has shown up in any\n searches I've tried. (and there's nothing unusual in\n /var/log/* that I can see.)\n\n I've also no idea how to \"fix\" the problem.\n\n explanations, suggestions, questions, &tc are all\n very welcome. (please keep in mind my current level\n of knowledge/experience about git is very weak. ;-\\ )\n\n this is all(?) using git 1.5.x (where the .x varies\n depending on the machine in question); the distros\n being used include ubuntu 7.10, FC-5, and probably\n others. (I'll check the filesystem tomorrow, but\n I believe it's sane.)\n\ncheers!\n\t-blf-\n-- \n\"How many surrealists does it take to | Brian Foster\n change a lightbulb? Three. One calms | somewhere in south of France\n the warthog, and two fill the bathtub | Stop E$$o (ExxonMobil)!\n with brightly-coloured machine tools.\" | http://www.stopesso.com\n"},{"id":"74535","messageId":"86lk3ecgak.fsf@lola.quinscape.zz","threadId":"13132","inReplyTo":"20080416063739.4B72647879@blf.utvinternet.co.uk","subject":"Re: fsck --full is Ok, but clones are not, \"missing commits\"?!","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2008-04-16T09:14:43Z","receivedAt":"2008-04-16T09:14:43Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Brian Foster writes:\n\n> (many many apologies if this turns into a double post,\n> there seems to have been problems with the 1st attempt?)\n>\n> I've recently inherited a bare git repository,\n> which, as far as I can tell (I'm something of\n> a newbie with git), seems Ok: `git fsck --full'\n> does not report any problems. however, any\n> clones I make from it are not Ok:\n>\n> \t$ git-fsck --full # clone (same command for bare repo is Ok)\n> \tbroken link from commit dd3f3c0636cfd50719c706b030db5473b0270add\n> \t\t to commit 0fed9c2eb14eee47097e1d870fe8e55a6430edeb\n> \tmissing commit fb57c018d15005b60f104e57f198ff34a6035b99\n> \tmissing commit f8947cb0b5fe605e6cb5f73c89f262424b64ef3c\n> \tmissing commit 0fed9c2eb14eee47097e1d870fe8e55a6430edeb\n> \tmissing commit dff364d8da15be0b856a174062fb785acb1c363e\n\n>From the git-clone manpage\n\n\n\t--shared, -s\n\t When the repository to clone is on the local machine,\n\t instead of using hard links, automatically setup\n\t .git/objects/info/alternates to share the objects with\n\t the source repository. The resulting repository starts\n\t out without any object of its own. NOTE: this is a\n\t possibly dangerous operation; do not use it unless you\n\t understand what it does. If you clone your repository\n\t using this option, then delete branches in the source\n\t repository and then run git-gc(1) using the --prune\n\t option in the source repository, it may remove objects\n\t which are referenced by the cloned repository.\n\nDid you use something like that?\n\n-- \nDavid Kastrup\n"},{"id":"76070","messageId":"20080505042546.GA7164@shion.is.fushizen.net","threadId":"13132","inReplyTo":"20080416063739.4B72647879@blf.utvinternet.co.uk","subject":"Re: fsck --full is Ok, but clones are not, \"missing commits\"?!","fromName":"Bryan Donlan","fromEmail":"bdonlan@fushizen.net","sentAt":"2008-05-05T04:25:46Z","receivedAt":"2008-05-05T04:25:46Z","isPatch":false,"sender":{"key":"bdonlan@fushizen.net","avatar":null},"body":"On Wed, Apr 16, 2008 at 08:37:39AM +0200, Brian Foster wrote:\n> (many many apologies if this turns into a double post,\n> there seems to have been problems with the 1st attempt?)\n> \n> I've recently inherited a bare git repository,\n> which, as far as I can tell (I'm something of\n> a newbie with git), seems Ok: `git fsck --full'\n> does not report any problems. however, any\n> clones I make from it are not Ok:\n \n[snip]\n \n> explanations, suggestions, questions, &tc are all\n> very welcome. (please keep in mind my current level\n> of knowledge/experience about git is very weak. ;-\\ )\n> \n> this is all(?) using git 1.5.x (where the .x varies\n> depending on the machine in question); the distros\n> being used include ubuntu 7.10, FC-5, and probably\n> others. (I'll check the filesystem tomorrow, but\n> I believe it's sane.)\n\nIs there an info/grafts file? If the repository doesn't have sensitive\ninformation in it, it would probably be helpful to tarball it up and\nupload it somewhere, so we can take a look at things directly.\n"},{"id":"76130","messageId":"a537dd660805050744h7602e553u21c70168a621fe76@mail.gmail.com","threadId":"13132","inReplyTo":"200805051608.55200.brian.foster@innova-card.com","subject":"Re: fsck --full is Ok, but clones are not, \"missing commits\"?!","fromName":"Brian Foster","fromEmail":"brian.foster@innova-card.com","sentAt":"2008-05-05T14:44:27Z","receivedAt":"2008-05-05T14:44:27Z","isPatch":false,"sender":{"key":"brian.foster@innova-card.com","avatar":null},"body":"On Monday 05 May 2008 06:25:46 Bryan Donlan asked:\n> On Wed, Apr 16, 2008 at 08:37:39AM +0200, Brian Foster wrote:\n> > I've recently inherited a bare git repository,\n> > which, as far as I can tell (I'm something of\n> > a newbie with git), seems Ok: `git fsck --full'\n> > does not report any problems. however, any\n> > clones I make from it are not Ok [ ... ]\n>\n> Is there an info/grafts file? If the repository doesn't have sensitive\n> information in it, it would probably be helpful to tarball it up and\n> upload it somewhere, so we can take a look at things directly.\n\nBryan,\n\n Yes, the proximate cause was an info/grafts.\n\n The repository in question is for a port of Linux\n to the MIPS-based \"secure\" SoC made by the company\n I work for. The repository was very simple:\n\n master: o---o---o---o---o--...--o HEAD\n\n where each `o' was a tagged version/release of the\n Linux port. Since the chip is MIPS-based, some `o'\n were the result of merges with mips-linux baseline.\n Hence, the real history was, broadly:\n\n master: o--o--o--o--o--o-----o--...--o\n / / / /\n mips-linux: o--o--o----o------o--o--o--o--...--o\n\n where, of course, `mips-linux' has its own rather\n complicated history (not shown). As implied by\n the diagrams above, grafts were used to \"suppress\"\n mips-linux and its history. (Since the situation\n really was as trivial as drawn above, it was easy\n to recover: Add the pack representing mips-linux,\n remove grafts, and ensure all the tags exist.)\n\n What I don't know is the root-cause, that is, WHY\n this was done. It wasn't a disc-space issue, and\n I've no evidence it was a network-bandwidth issue,\n but there is some anecdotal evidence it was some\n sort of a CPU-cycles issue, albeit just what the\n performance hit was is unknown.\n\n Another possibility is the repository started life\n as CVS (ugh!) before being migrated to git. It's\n my (vague) understanding grafts is somehow useful\n as a (temporary?) aid when doing a CVS-->git move;\n and I speculate it was found so useful/simple the\n practice simply continued.\n\n The developers tended to work from tarballs (not\n clones), so the cloning problem was either unknown,\n not a concern, or mis-understood.\n\ncheers!\n\t-blf-\n\n-- \n\"How many surrealists does it take to | Brian Foster\n change a lightbulb? Three. One calms | somewhere in south of France\n the warthog, and two fill the bathtub | Stop E$$o (ExxonMobil)!\n with brightly-coloured machine tools.\" | http://www.stopesso.com\n"},{"id":"76131","messageId":"481F23D4.2090909@viscovery.net","threadId":"13132","inReplyTo":"a537dd660805050744h7602e553u21c70168a621fe76@mail.gmail.com","subject":"Re: fsck --full is Ok, but clones are not, \"missing commits\"?!","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-05-05T15:12:20Z","receivedAt":"2008-05-05T15:12:20Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Brian Foster schrieb:\n> What I don't know is the root-cause, that is, WHY\n> this was done. It wasn't a disc-space issue, and\n> I've no evidence it was a network-bandwidth issue,\n> but there is some anecdotal evidence it was some\n> sort of a CPU-cycles issue, albeit just what the\n> performance hit was is unknown.\n\nHow about this theory:\n\nWhat happens if you fire up gitk as simple as\n\n $ gitk\n\nin the history if no grafts are present? Some months ago this took ages to\ncomplete, and even today you get a *huge* list of commits in a *short*\nwindow; hence, the scrollbar thumb is tiny, and if you succeed to get hold\nof it without a magnifying glass, it scrolls way more than a page of\ncommits if you move it by only one pixel.\n\nNo wonder that $user wants to have a shorter history. So $user, being\nsmart, truncates the history at a suitable point with a graft.\n\n-- Hannes\n"},{"id":"76192","messageId":"a537dd660805060358q6e39947blda348917d5853294@mail.gmail.com","threadId":"13132","inReplyTo":"200805061231.30135.brian.foster@innova-card.com","subject":"Re: fsck --full is Ok, but clones are not, \"missing commits\"?!","fromName":"Brian Foster","fromEmail":"brian.foster@innova-card.com","sentAt":"2008-05-06T10:58:31Z","receivedAt":"2008-05-06T10:58:31Z","isPatch":false,"sender":{"key":"brian.foster@innova-card.com","avatar":null},"body":"Johannes Sixt wrote:\n> Brian Foster schrieb:\n> > What I don't know is the root-cause, that is, WHY\n> > this was done. [ ... ] there is some anecdotal\n> > evidence it was some sort of a CPU-cycles issue,\n> > albeit just what the performance hit was is unknown.\n>\n> How about this theory:\n>\n> What happens if you fire up gitk as simple as\n>\n> $ gitk\n>\n> in the history if no grafts are present? Some months ago this took ages to\n> complete, and even today you get a *huge* list of commits in a *short*\n> window; hence, the scrollbar thumb is tiny, and if you succeed to get hold\n> of it without a magnifying glass, it scrolls way more than a page of\n> commits if you move it by only one pixel.\n>\n> No wonder that $user wants to have a shorter history. So $user, being\n> smart, truncates the history at a suitable point with a graft.\n\nHannes,\n\n Unfortunately, I cannot fire up `gitk' in the exact\n same configuration anymore (that server machine is now\n being used for other purposes, albeit I'm supposed to\n get the hard disc). The git on the now-vanished server\n was v1.5.3, but that's probably not relevant, since the\n repository must have been created with a much older git\n (it goes back multiple years).\n\n All the (now-)installed gits I've seen are 1.5..\n I do not see any noticeable performance issue with 1.5.2.5\n (nor with 1.5.5)? The scrollbar is, as you say, unusable.\n\n But how important is `gitk'? Is it something that'd be\n used frequently enough for the formerly-poor performance\n to be such an issue that creating and maintaining such a\n \"truncated\" repository is worthwhile?\n\n It's an interesting and plausible hypothesis, but (in\n the absence of any actual evidence) I'd be more inclined\n to buy it if there was some frequent/critical operation\n where poor performance clearly matters.\n\ncheers!\n\t-blf-\n"},{"id":"76193","messageId":"48203D33.5020900@viscovery.net","threadId":"13132","inReplyTo":"a537dd660805060358q6e39947blda348917d5853294@mail.gmail.com","subject":"Re: fsck --full is Ok, but clones are not, \"missing commits\"?!","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-05-06T11:12:51Z","receivedAt":"2008-05-06T11:12:51Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Brian Foster schrieb:\n> Johannes Sixt wrote:\n>> What happens if you fire up gitk as simple as\n>>\n>> $ gitk\n>>\n>> in the history if no grafts are present? Some months ago this took ages to\n>> complete, and even today you get a *huge* list of commits in a *short*\n>> window; hence, the scrollbar thumb is tiny, and if you succeed to get hold\n>> of it without a magnifying glass, it scrolls way more than a page of\n>> commits if you move it by only one pixel.\n>>\n>> No wonder that $user wants to have a shorter history. So $user, being\n>> smart, truncates the history at a suitable point with a graft.\n> \n> Hannes,\n> \n> Unfortunately, I cannot fire up `gitk' in the exact\n> same configuration anymore (that server machine is now\n> being used for other purposes, albeit I'm supposed to\n> get the hard disc). The git on the now-vanished server\n> was v1.5.3, but that's probably not relevant, since the\n> repository must have been created with a much older git\n> (it goes back multiple years).\n> \n> All the (now-)installed gits I've seen are 1.5..\n> I do not see any noticeable performance issue with 1.5.2.5\n> (nor with 1.5.5)? The scrollbar is, as you say, unusable.\n\nFor me, the unusable scrollbar alone would be reason enough to truncate\nthe history. Once it is truncated, performance is no longer an issue\n(whether or not it was an issue in the first place).\n\n> But how important is `gitk'? Is it something that'd be\n> used frequently enough for the formerly-poor performance\n> to be such an issue that creating and maintaining such a\n> \"truncated\" repository is worthwhile?\n\nWell, I have gitk running all the time. So, yes, it is \"important.\" But I\n run it basically as 'gitk --all --not origin' and press F5 frequently.\nWith this set of arguments the scrollbar remains usable, and performance\nis not an issue, even on Windows.\n\n-- Hannes\n"}]}