I'm needing to scan through git history looking for the file sizes (looking for when a particular file shrunk drastically)
I'm not seeing an option in git log or git whatchanged that gives me the file size, am I overlooking something?
David Lang
threads / discuss / 47008
hot to get file sizes in git log output
Subject: hot to get file sizes in git log output
Jacob KellerRe: hot to get file sizes in git log output
On Fri, Oct 20, 2017 at 11:12 AM, David Lang <david@lang.hm> wrote:
Show 7 quoted lines
> I'm needing to scan through git history looking for the file sizes (looking > for when a particular file shrunk drastically) > > I'm not seeing an option in git log or git whatchanged that gives me the > file size, am I overlooking something? > > David Lang
I'm not exactly sure what you mean by size, but if you want to show how many lines were added and removed by a given commit for each file, you can use the "--stat" option to produce a diffstat. The "size" of the files in each commit isn't very meaningful to the commit itself, but a stat of how much was removed might be more accurate to what you're looking for.
Thanks, Jake
Jeff KingRe: hot to get file sizes in git log output
On Fri, Oct 20, 2017 at 01:44:36PM -0700, Jacob Keller wrote:
Show 15 quoted lines
> On Fri, Oct 20, 2017 at 11:12 AM, David Lang <david@lang.hm> wrote: > > I'm needing to scan through git history looking for the file sizes (looking > > for when a particular file shrunk drastically) > > > > I'm not seeing an option in git log or git whatchanged that gives me the > > file size, am I overlooking something? > > > > David Lang > > I'm not exactly sure what you mean by size, but if you want to show > how many lines were added and removed by a given commit for each file, > you can use the "--stat" option to produce a diffstat. The "size" of > the files in each commit isn't very meaningful to the commit itself, > but a stat of how much was removed might be more accurate to what > you're looking for.
That's a good suggestion, and hopefully could help David answer his original question.
I took the request to mean "walk through history, and for each file that a commit touches, show its size". Which is a bit harder to do, and I think you need to script a little:
git rev-list HEAD |
git diff-tree --stdin -r |
perl -lne '
# raw diff line, capture filename and post-image sha1
if (/^:\S+ \S+ \S+ (\S+) \S+\t(.*)/) {
print "$1 $commit:$2"
}
# otherwise it is a commit sha1
else {
$commit = $_;
}
' |
git cat-file --batch-check='%(objectsize) %(rest)'That should show the size of each file along with the "commit:path" in which it was introduced.
-Peff
Eric SunshineRe: hot to get file sizes in git log output
On Fri, Oct 20, 2017 at 5:43 PM, Jeff King <peff@peff.net> wrote:
Show 21 quoted lines
> On Fri, Oct 20, 2017 at 01:44:36PM -0700, Jacob Keller wrote: >> On Fri, Oct 20, 2017 at 11:12 AM, David Lang <david@lang.hm> wrote: >> > I'm needing to scan through git history looking for the file sizes (looking >> > for when a particular file shrunk drastically) >> > >> > I'm not seeing an option in git log or git whatchanged that gives me the >> > file size, am I overlooking something? >> >> I'm not exactly sure what you mean by size, but if you want to show >> how many lines were added and removed by a given commit for each file, >> you can use the "--stat" option to produce a diffstat. The "size" of >> the files in each commit isn't very meaningful to the commit itself, >> but a stat of how much was removed might be more accurate to what >> you're looking for. > > That's a good suggestion, and hopefully could help David answer his > original question. > > I took the request to mean "walk through history, and for each file that > a commit touches, show its size". Which is a bit harder to do, and I > think you need to script a little:
David's mention of "a particular file", suggests to me that something "bad" happened to one file, and he wants to know in which commit that "badness" happened. If so, then it sounds like a job for git-bisect.
Jacob KellerRe: hot to get file sizes in git log output
On Fri, Oct 20, 2017 at 2:50 PM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 26 quoted lines
> On Fri, Oct 20, 2017 at 5:43 PM, Jeff King <peff@peff.net> wrote: >> On Fri, Oct 20, 2017 at 01:44:36PM -0700, Jacob Keller wrote: >>> On Fri, Oct 20, 2017 at 11:12 AM, David Lang <david@lang.hm> wrote: >>> > I'm needing to scan through git history looking for the file sizes (looking >>> > for when a particular file shrunk drastically) >>> > >>> > I'm not seeing an option in git log or git whatchanged that gives me the >>> > file size, am I overlooking something? >>> >>> I'm not exactly sure what you mean by size, but if you want to show >>> how many lines were added and removed by a given commit for each file, >>> you can use the "--stat" option to produce a diffstat. The "size" of >>> the files in each commit isn't very meaningful to the commit itself, >>> but a stat of how much was removed might be more accurate to what >>> you're looking for. >> >> That's a good suggestion, and hopefully could help David answer his >> original question. >> >> I took the request to mean "walk through history, and for each file that >> a commit touches, show its size". Which is a bit harder to do, and I >> think you need to script a little: > > David's mention of "a particular file", suggests to me that something > "bad" happened to one file, and he wants to know in which commit that > "badness" happened. If so, then it sounds like a job for git-bisect.
Yea, if you have a simple script which can tell when the file is "bad", you could run it through git bisect run pretty easily and rapidly find the right answer.
Thanks, Jake
Re: hot to get file sizes in git log output
On Fri, 20 Oct 2017, Eric Sunshine wrote:
Show 17 quoted lines
>>> I'm not exactly sure what you mean by size, but if you want to show >>> how many lines were added and removed by a given commit for each file, >>> you can use the "--stat" option to produce a diffstat. The "size" of >>> the files in each commit isn't very meaningful to the commit itself, >>> but a stat of how much was removed might be more accurate to what >>> you're looking for. >> >> That's a good suggestion, and hopefully could help David answer his >> original question. >> >> I took the request to mean "walk through history, and for each file that >> a commit touches, show its size". Which is a bit harder to do, and I >> think you need to script a little: > > David's mention of "a particular file", suggests to me that something > "bad" happened to one file, and he wants to know in which commit that > "badness" happened. If so, then it sounds like a job for git-bisect.
In this case, I have git store a copy of the state file for chromium (and do a similar thing for firefox), so that if something bad happens and it crashes and looses all 200-400 tabs in a way that it's recovery doesn't work, I can go back to a prior version.
This is done by having the following crontab entries, along with smudge filters that change the one-line json to pretty printed json before the commit.
0 * * * * dlang cd /home/dlang/.config/chromium/Default; git add *Session *Tabs Bookmarks History ; git commit -mupdate > /dev/null 2>&1
0 0 3 * * dlang cd /home/dlang/.config/chromium/Default; git gc --aggressive > /dev/null 2>&1
0 * * * * dlang cd /home/dlang/.mozilla/firefox/bux6mwl1.default/sessionstore-backups; git add *.js ; git commit -mupdate > /dev/null 2>&1
0 0 3 * * dlang cd /home/dlang/.mozilla/firefox/bux6mwl1.default/sessionstore-backups; git gc --aggressive > /dev/null 2>&1
This has saved me many times in the past. But this time I didn't recognize when the problem happened because instead of a crash, it just closed all the tabs except the one that was open. Once I realized all my other tabs were gone, I didn't have time to mess with it for a few days. So the problem could have happened anytime in the last week or two.
I'm sure that when this happened, the files shrunk drastically (from several hundred tabs to a dozen or so will be very obvious).
But I don't have any specific line I can look at, the lines that are there change pretty regularly, and/or would not have changed at the transition.
git whatchanged shows commits like:
commit fb7e54c12ddc7c87c4862806d583f5c6abf3e731 Author: David Lang <david@lang.hm> Date: Fri Oct 20 11:00:01 2017 -0700
update
:100644 100644 1a842ca... 290e9dd... M Default/Bookmarks :100644 100644 1cd745c... 388a455... M Default/Current Session :100644 100644 51074ad... c4dce40... M Default/Current Tabs
If there was a way to add file size to this output, it would be perfect for what I'm needing.
David Lang
Re: hot to get file sizes in git log output
On Fri, 20 Oct 2017, David Lang wrote:
Show 19 quoted lines
> On Fri, 20 Oct 2017, Eric Sunshine wrote: > >>>> I'm not exactly sure what you mean by size, but if you want to show >>>> how many lines were added and removed by a given commit for each file, >>>> you can use the "--stat" option to produce a diffstat. The "size" of >>>> the files in each commit isn't very meaningful to the commit itself, >>>> but a stat of how much was removed might be more accurate to what >>>> you're looking for. >>> >>> That's a good suggestion, and hopefully could help David answer his >>> original question. >>> >>> I took the request to mean "walk through history, and for each file that >>> a commit touches, show its size". Which is a bit harder to do, and I >>> think you need to script a little: >> >> David's mention of "a particular file", suggests to me that something >> "bad" happened to one file, and he wants to know in which commit that >> "badness" happened. If so, then it sounds like a job for git-bisect.
summarizing (and removing the long explination of why I'm doing this)
for each file (or each file changed in the commit), what is the byte count of that file at the time of that commit.
git whatschanged currently reports
commit 17be1c1e1f80086e8ddda1706c8c8d6cf80d26b7 Author: David Lang <david@lang.hm> Date: Thu Oct 19 22:00:01 2017 -0700
update
:100644 100644 bb9dcd3... 8635d2b... M Default/Current Session
commit d3f94d406e0d5c6ee7b6f6bcea019adff438127c Author: David Lang <david@lang.hm> Date: Thu Oct 19 21:00:01 2017 -0700
update
:100644 100644 88ece53... bb9dcd3... M Default/Current Session
commit fea290bd235a444bbd4bc4430fa0844501ae2b8c Author: David Lang <david@lang.hm> Date: Thu Oct 19 06:00:01 2017 -0700
update
:100644 100644 ff04089... 88ece53... M Default/Current Session
what is the size of the file "Current Session" for each commit?
David Lang
Jeff KingRe: hot to get file sizes in git log output
On Fri, Oct 20, 2017 at 07:38:00PM -0700, David Lang wrote:
Show 14 quoted lines
> git whatchanged shows commits like: > > commit fb7e54c12ddc7c87c4862806d583f5c6abf3e731 > Author: David Lang <david@lang.hm> > Date: Fri Oct 20 11:00:01 2017 -0700 > > update > > :100644 100644 1a842ca... 290e9dd... M Default/Bookmarks > :100644 100644 1cd745c... 388a455... M Default/Current Session > :100644 100644 51074ad... c4dce40... M Default/Current Tabs > > If there was a way to add file size to this output, it would be perfect for > what I'm needing.
There isn't. You'll have to process the output to pass the post-image hashes to cat-file to get the size (i.e., what I showed before, though of course you could make the output prettier if you wanted to).
-Peff