{"thread":{"id":"48038","subject":"[git-sizer] Implications of a large commit object","startedAt":"2018-03-14T08:17:55Z","lastAt":"2018-03-14T14:56:30Z","messageCount":4,"participants":["Lars Schneider","Michael Haggerty","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"341621","messageId":"953C263C-6ECA-46AC-849C-8D1A38378654@gmail.com","threadId":"48038","inReplyTo":null,"subject":"[git-sizer] Implications of a large commit object","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2018-03-14T08:14:46Z","receivedAt":"2018-03-14T08:17:55Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"Hi,\n\nI am using Michael's fantastic Git repo analyzer tool \"git-sizer\" [*]\nand it detected a very large commit of 7.33 MiB in my repo (see chart \nbelow).\n\nThis large commit is expected. I've imported that repo from another\nversion control system but excluded all binary files (e.g. images) and\nsome 3rd party components as their history is not important [**]. I've \nreintroduced these files in the head commit again. This is where the \nlarge commit came from.\n\nThis repo is not used in production yet but I wonder if this kind of\napproach can cause trouble down the line? Are there any relevant\nimplication of a single large commit like this in history? \n\nThanks,\nLars\n\n\n [*] https://github.com/github/git-sizer\n[**] I know some of this stuff shouldn't be in the repo in the first \n place, but I am constrained in the things I can change.\n\n\n#######################################################################\n## git-sizer output\n\nProcessing blobs: 543782\nProcessing trees: 517104\nProcessing commits: 43365\nMatching commits to trees: 43365\nProcessing annotated tags: 3\nProcessing references: 123\n| Name | Value | Level of concern |\n| ---------------------------- | --------- | ------------------------------ |\n| Overall repository size | | |\n| * Blobs | | |\n| * Total size | 18.8 GiB | ** |\n| | | |\n| Biggest objects | | |\n| * Commits | | |\n| * Maximum size [1] | 7.33 MiB | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! |\n| * Trees | | |\n| * Maximum entries [2] | 6.84 k | ** |\n| | | |\n| History structure | | |\n| * Maximum tag depth [3] | 1 | * |\n| | | |\n| Biggest checkouts | | |\n| * Number of directories [4] | 21.9 k | ********** |\n| * Maximum path depth [4] | 18 | * |\n| * Maximum path length [5] | 225 B | ** |\n| * Number of files [4] | 256 k | ***** |\n| * Total size of files [6] | 2.08 GiB | ** |\n\n\n\n"},{"id":"341622","messageId":"CAMy9T_GdYjUm9DqcgrC=NxyTbZk_ep5rvAAK2J=zwhiSMyvaLA@mail.gmail.com","threadId":"48038","inReplyTo":"953C263C-6ECA-46AC-849C-8D1A38378654@gmail.com","subject":"Re: [git-sizer] Implications of a large commit object","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2018-03-14T08:33:32Z","receivedAt":"2018-03-14T08:33:45Z","isPatch":false,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On Wed, Mar 14, 2018 at 9:14 AM, Lars Schneider\n wrote:\n> I am using Michael's fantastic Git repo analyzer tool \"git-sizer\" [*]\n> and it detected a very large commit of 7.33 MiB in my repo (see chart\n> below).\n>\n> This large commit is expected. I've imported that repo from another\n> version control system but excluded all binary files (e.g. images) and\n> some 3rd party components as their history is not important [**]. I've\n> reintroduced these files in the head commit again. This is where the\n> large commit came from.\n>\n> This repo is not used in production yet but I wonder if this kind of\n> approach can cause trouble down the line? Are there any relevant\n> implication of a single large commit like this in history?\n> [...]\n>\n> #######################################################################\n> ## git-sizer output\n>\n> [...]\n> | Name | Value | Level of concern |\n> | ---------------------------- | --------- | ------------------------------ |\n> [...]\n> | Biggest objects | | |\n> | * Commits | | |\n> | * Maximum size [1] | 7.33 MiB | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! |\n> [...]\n\nThe \"commit size\" that is being referred to here is the size of the\nactual commit object; i.e., the author name, parent commits, etc plus\nthe log message. So a huge commit probably means that you have a huge\nlog message. This has nothing to do with the number or sizes of the\nfiles added by the commit.\n\nMaybe your migration tool created a huge commit message, for example\nlisting each of the files that was changed.\n\nAFAIK this won't cause Git itself any problems, but it's likely to be\ninconvenient. For example, when you type `git log` and 7 million\ncharacters page by. Or when you use some GUI tool to view your history\nand it performs badly because it wasn't built to handle such enormous\ncommit messages.\n\nMichael\n"},{"id":"341623","messageId":"BB908580-36C2-466D-AF05-6BFF7F5E4705@gmail.com","threadId":"48038","inReplyTo":"CAMy9T_GdYjUm9DqcgrC=NxyTbZk_ep5rvAAK2J=zwhiSMyvaLA@mail.gmail.com","subject":"Re: [git-sizer] Implications of a large commit object","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2018-03-14T09:40:38Z","receivedAt":"2018-03-14T09:40:48Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 14 Mar 2018, at 09:33, Michael Haggerty wrote:\n> \n> On Wed, Mar 14, 2018 at 9:14 AM, Lars Schneider\n> wrote:\n>> I am using Michael's fantastic Git repo analyzer tool \"git-sizer\" [*]\n>> and it detected a very large commit of 7.33 MiB in my repo (see chart\n>> below).\n>> \n>> This large commit is expected. I've imported that repo from another\n>> version control system but excluded all binary files (e.g. images) and\n>> some 3rd party components as their history is not important [**]. I've\n>> reintroduced these files in the head commit again. This is where the\n>> large commit came from.\n>> \n>> This repo is not used in production yet but I wonder if this kind of\n>> approach can cause trouble down the line? Are there any relevant\n>> implication of a single large commit like this in history?\n>> [...]\n>> \n>> #######################################################################\n>> ## git-sizer output\n>> \n>> [...]\n>> | Name | Value | Level of concern |\n>> | ---------------------------- | --------- | ------------------------------ |\n>> [...]\n>> | Biggest objects | | |\n>> | * Commits | | |\n>> | * Maximum size [1] | 7.33 MiB | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! |\n>> [...]\n> \n> The \"commit size\" that is being referred to here is the size of the\n> actual commit object; i.e., the author name, parent commits, etc plus\n> the log message. So a huge commit probably means that you have a huge\n> log message. This has nothing to do with the number or sizes of the\n> files added by the commit.\n> \n> Maybe your migration tool created a huge commit message, for example\n> listing each of the files that was changed.\n\n\nD'oh! Of course. I was so focused on that commit with the large number of\nfiles that I missed that. Looking at the reference [1] reveals the\nproblem. Sorry for wasting your time!\n\n\n> AFAIK this won't cause Git itself any problems, but it's likely to be\n> inconvenient. For example, when you type `git log` and 7 million\n> characters page by. Or when you use some GUI tool to view your history\n> and it performs badly because it wasn't built to handle such enormous\n> commit messages.\n\n\nThank you,\nLars\n"},{"id":"341636","messageId":"20180314145623.GB15531@sigill.intra.peff.net","threadId":"48038","inReplyTo":"CAMy9T_GdYjUm9DqcgrC=NxyTbZk_ep5rvAAK2J=zwhiSMyvaLA@mail.gmail.com","subject":"Re: [git-sizer] Implications of a large commit object","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-03-14T14:56:23Z","receivedAt":"2018-03-14T14:56:30Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Mar 14, 2018 at 09:33:32AM +0100, Michael Haggerty wrote:\n\n> Maybe your migration tool created a huge commit message, for example\n> listing each of the files that was changed.\n> \n> AFAIK this won't cause Git itself any problems, but it's likely to be\n> inconvenient. For example, when you type `git log` and 7 million\n> characters page by. Or when you use some GUI tool to view your history\n> and it performs badly because it wasn't built to handle such enormous\n> commit messages.\n\nProbably one such commit won't break the bank, but it will make history\ntraversals that cross it slower (e.g., \"--contains\", merge-bases, etc).\nWe'll load the whole 7MB object just to find its parents. If you imagine\nthe average commit object is more like 1k and that current traversals\nbottleneck on loading the commit object bytes (both of which I think are\nroughly accurate), then crossing that one commit in a traversal is\nequivalent to crossing 7000 \"normal\" commits in cost.\n\nAt least until Stolee's serialized commit graph work is merged, at which\npoint it will only be expensive if we actually try to show the commit\nmessage for that particular object.\n\n-Peff\n"}]}