{"thread":{"id":"35304","subject":"Selectively commit/publish files to GIT","startedAt":"2013-11-08T10:52:38Z","lastAt":"2013-11-08T23:50:10Z","messageCount":5,"participants":["Thomas Manson","Magnus Bäck","Keshav Kini"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"230234","messageId":"CA+PenvEDY6mc+DyjXy4ebZKdK4R=ucM28MDXGC42XKxvJ=a-pQ@mail.gmail.com","threadId":"35304","inReplyTo":null,"subject":"Selectively commit/publish files to GIT","fromName":"Thomas Manson","fromEmail":"dev.mansonthomas@gmail.com","sentAt":"2013-11-08T10:52:38Z","receivedAt":"2013-11-08T10:52:38Z","isPatch":false,"sender":{"key":"dev.mansonthomas@gmail.com","avatar":null},"body":"Hi,\n\n I've converting my Bazaar repository to GIT.\n\n I've successfully done this conversion and I now want to publish my\nsource code to github.\n\n The problem is that in Bazaar, I've commited some big files (63MB &\n173MB), but this files are no longer in my project, only in the\nrevisions files of Bazaar and now Git.\n\n I don't need this files to be pushed on Github.\n\n How can I search git history for big files and remove them, or mark\nthem to be not published ?\n\nI've tryed this solution found on the link in the error:\n\ngit filter-branch --force --index-filter 'git rm --cached\n--ignore-unmatch giant_file' --prune-empty --tag-name-filter cat --\n--all\ngit commit --amend -CHEAD\n\nThen I've tryed to Commit & Push from Github mac application and I\nhad several network error, and finally get the same error on giant\nfiles (maybe my multiple commit & push did override something, but I\nunderstood that the git rm command would remove things once for\ngood...\n\ncan anybody help me ?\nI'm blocked in my dev because of this, I can't share my project with a friend.\nI'm publishing here : https://github.com/dev-mansonthomas/crf-rdp.git\n(paying for storage is an option as I'm quite fed up loosing time for\nfilesize...)\n\nRegards,\nThomas.\n\nhere is the error I have using the GitHub application on Mac :\n(after that I intend to use Eclipse)\n\nFile Ressources/dwr/dwr-3.0.0.110.dev-src.zip is 67.69 MB; this is\nlarger than GitHub's recommended maximum file size of 50 MB\nGH001: Large files detected.\nTrace: 8f0259b29260f0c4d7ae4d4ae70e0306\nSee http://git.io/iEPt8g for more information.\nFile .bzr/repository/packs/a7bcd6ba235114ab88c80fe8a97adcfa.pack is\n178.76 MB; this exceeds GitHub's file size limit of 100 MB\n"},{"id":"230236","messageId":"20131108141329.GA26790@google.com","threadId":"35304","inReplyTo":"CA+PenvEDY6mc+DyjXy4ebZKdK4R=ucM28MDXGC42XKxvJ=a-pQ@mail.gmail.com","subject":"Re: Selectively commit/publish files to GIT","fromName":"Magnus Bäck","fromEmail":"baeck@google.com","sentAt":"2013-11-08T14:13:30Z","receivedAt":"2013-11-08T14:13:30Z","isPatch":false,"sender":{"key":"baeck@google.com","avatar":null},"body":"On Friday, November 08, 2013 at 05:52 EST,\n Thomas Manson wrote:\n\n[...]\n\n> How can I search git history for big files and remove them, or mark\n> them to be not published ?\n> \n> I've tryed this solution found on the link in the error:\n> \n> git filter-branch --force --index-filter 'git rm --cached\n> --ignore-unmatch giant_file' --prune-empty --tag-name-filter cat --\n> --all\n> git commit --amend -CHEAD\n\n[...]\n\n> here is the error I have using the GitHub application on Mac :\n> (after that I intend to use Eclipse)\n> \n> File Ressources/dwr/dwr-3.0.0.110.dev-src.zip is 67.69 MB; this is\n> larger than GitHub's recommended maximum file size of 50 MB\n> GH001: Large files detected.\n> Trace: 8f0259b29260f0c4d7ae4d4ae70e0306\n> See http://git.io/iEPt8g for more information.\n> File .bzr/repository/packs/a7bcd6ba235114ab88c80fe8a97adcfa.pack is\n> 178.76 MB; this exceeds GitHub's file size limit of 100 MB\n\nDid you actually replace the 'giant_file' placeholder with the path to\n*your* giant file? The error message indicates that you didn't, or that\nsomething else went wrong during the filtering.\n\nYou can use 'git log' to look for commits touching particular pathnames,\nmaking it easy to verify that a file has been eradicated.\n\ngit log --all -- path/to/big-honking-file.zip\n\n-- \nMagnus Bäck\nbaeck@google.com\n"},{"id":"230253","messageId":"CA+PenvG7CnkvN-XZE-uYBh42Yyqq23G5fOFRNhCU83m3V5f8sQ@mail.gmail.com","threadId":"35304","inReplyTo":"20131108141329.GA26790@google.com","subject":"Re: Selectively commit/publish files to GIT","fromName":"Thomas Manson","fromEmail":"dev.mansonthomas@gmail.com","sentAt":"2013-11-08T22:51:55Z","receivedAt":"2013-11-08T22:51:55Z","isPatch":false,"sender":{"key":"dev.mansonthomas@gmail.com","avatar":null},"body":"Hi Magnus,\n\n thanks for pointing that out. I must have overlooked the webpage,\nwhen I saw that I thought it was some kind of alias that designated\nfiles greater than a specific files (server side configurated as the\n100MB limit)\n\n I'll try again ;)\n\nThomas.\n\nOn Fri, Nov 8, 2013 at 3:13 PM, Magnus Bäck wrote:\n> On Friday, November 08, 2013 at 05:52 EST,\n> Thomas Manson wrote:\n>\n> [...]\n>\n>> How can I search git history for big files and remove them, or mark\n>> them to be not published ?\n>>\n>> I've tryed this solution found on the link in the error:\n>>\n>> git filter-branch --force --index-filter 'git rm --cached\n>> --ignore-unmatch giant_file' --prune-empty --tag-name-filter cat --\n>> --all\n>> git commit --amend -CHEAD\n>\n> [...]\n>\n>> here is the error I have using the GitHub application on Mac :\n>> (after that I intend to use Eclipse)\n>>\n>> File Ressources/dwr/dwr-3.0.0.110.dev-src.zip is 67.69 MB; this is\n>> larger than GitHub's recommended maximum file size of 50 MB\n>> GH001: Large files detected.\n>> Trace: 8f0259b29260f0c4d7ae4d4ae70e0306\n>> See http://git.io/iEPt8g for more information.\n>> File .bzr/repository/packs/a7bcd6ba235114ab88c80fe8a97adcfa.pack is\n>> 178.76 MB; this exceeds GitHub's file size limit of 100 MB\n>\n> Did you actually replace the 'giant_file' placeholder with the path to\n> *your* giant file? The error message indicates that you didn't, or that\n> something else went wrong during the filtering.\n>\n> You can use 'git log' to look for commits touching particular pathnames,\n> making it easy to verify that a file has been eradicated.\n>\n> git log --all -- path/to/big-honking-file.zip\n>\n> --\n> Magnus Bäck\n> baeck@google.com\n"},{"id":"230254","messageId":"CA+PenvHDUSBUBAQB3Lxoh7sTVfq_s=D2n2FqxnEu1_uUSvOYrw@mail.gmail.com","threadId":"35304","inReplyTo":"CA+PenvG7CnkvN-XZE-uYBh42Yyqq23G5fOFRNhCU83m3V5f8sQ@mail.gmail.com","subject":"Re: Selectively commit/publish files to GIT","fromName":"Thomas Manson","fromEmail":"dev.mansonthomas@gmail.com","sentAt":"2013-11-08T23:01:31Z","receivedAt":"2013-11-08T23:01:31Z","isPatch":false,"sender":{"key":"dev.mansonthomas@gmail.com","avatar":null},"body":"Thanks Magnus, it did solve my issue.\n\nI've noticed by the way that i've accidentally commited the whole .bzr\ndirectory (that hold all my commit in Bazaar style).\n\nI've deleted this folder as well using -r\n\ngit filter-branch --force --index-filter 'git rm -r --cached\n--ignore-unmatch .bzr' --prune-empty --tag-name-filter cat -- --all\n\n\n\n\nfor reference, an example of working run :\n\nThomass-MacBook-Pro:crf-rdp tmanson$ git filter-branch --force\n--index-filter 'git rm --cached --ignore-unmatch\nRessources/dwr/dwr-3.0.0.110.dev-src.zip' --prune-empty\n--tag-name-filter cat -- --all\nRewrite 58952de4e10bc1df2f8c324e94d249a0366f7512 (65/158)rm\n'Ressources/dwr/dwr-3.0.0.110.dev-src.zip'\nRewrite e661d08f1c33d7bd53e2c2231a3296b2ade5a2dd (66/158)rm\n'Ressources/dwr/dwr-3.0.0.110.dev-src.zip'\nRewrite 8ff3f3f9ffb901b7cdc1dba220a121bab0698121 (67/158)rm\n'Ressources/dwr/dwr-3.0.0.110.dev-src.zip'\nRewrite cfec6341565c35c410f248e5861543ac01b84f40 (158/158)\nRef 'refs/heads/master' was rewritten\nRef 'refs/tags/version-0-1_20081120' was rewritten\nRef 'refs/tags/version-0-5-1_20120512' was rewritten\n\n\nOn Fri, Nov 8, 2013 at 11:51 PM, Thomas Manson\n wrote:\n> Hi Magnus,\n>\n> thanks for pointing that out. I must have overlooked the webpage,\n> when I saw that I thought it was some kind of alias that designated\n> files greater than a specific files (server side configurated as the\n> 100MB limit)\n>\n> I'll try again ;)\n>\n> Thomas.\n>\n> On Fri, Nov 8, 2013 at 3:13 PM, Magnus Bäck wrote:\n>> On Friday, November 08, 2013 at 05:52 EST,\n>> Thomas Manson wrote:\n>>\n>> [...]\n>>\n>>> How can I search git history for big files and remove them, or mark\n>>> them to be not published ?\n>>>\n>>> I've tryed this solution found on the link in the error:\n>>>\n>>> git filter-branch --force --index-filter 'git rm --cached\n>>> --ignore-unmatch giant_file' --prune-empty --tag-name-filter cat --\n>>> --all\n>>> git commit --amend -CHEAD\n>>\n>> [...]\n>>\n>>> here is the error I have using the GitHub application on Mac :\n>>> (after that I intend to use Eclipse)\n>>>\n>>> File Ressources/dwr/dwr-3.0.0.110.dev-src.zip is 67.69 MB; this is\n>>> larger than GitHub's recommended maximum file size of 50 MB\n>>> GH001: Large files detected.\n>>> Trace: 8f0259b29260f0c4d7ae4d4ae70e0306\n>>> See http://git.io/iEPt8g for more information.\n>>> File .bzr/repository/packs/a7bcd6ba235114ab88c80fe8a97adcfa.pack is\n>>> 178.76 MB; this exceeds GitHub's file size limit of 100 MB\n>>\n>> Did you actually replace the 'giant_file' placeholder with the path to\n>> *your* giant file? The error message indicates that you didn't, or that\n>> something else went wrong during the filtering.\n>>\n>> You can use 'git log' to look for commits touching particular pathnames,\n>> making it easy to verify that a file has been eradicated.\n>>\n>> git log --all -- path/to/big-honking-file.zip\n>>\n>> --\n>> Magnus Bäck\n>> baeck@google.com\n"},{"id":"230256","messageId":"87fvr6cv2l.fsf@gmail.com","threadId":"35304","inReplyTo":"CA+PenvEDY6mc+DyjXy4ebZKdK4R=ucM28MDXGC42XKxvJ=a-pQ@mail.gmail.com","subject":"Re: Selectively commit/publish files to GIT","fromName":"Keshav Kini","fromEmail":"keshav.kini@gmail.com","sentAt":"2013-11-08T23:50:10Z","receivedAt":"2013-11-08T23:50:10Z","isPatch":false,"sender":{"key":"keshav.kini@gmail.com","avatar":"https://avatars.githubusercontent.com/u/691290?v=4"},"body":"Thomas Manson writes:\n> The problem is that in Bazaar, I've commited some big files (63MB &\n> 173MB), but this files are no longer in my project, only in the\n> revisions files of Bazaar and now Git.\n>\n> I don't need this files to be pushed on Github.\n>\n> How can I search git history for big files and remove them, or mark\n> them to be not published ?\n\nI see you already solved your problem, but for future reference, there\nis a nice tool that is more limited in functionality than git\nfilter-branch but also much faster and possibly easier to use:\n\n http://rtyley.github.io/bfg-repo-cleaner/\n\n-Keshav\n"}]}