RE: [EXTERNAL] Re: Removing data from repository

From: Naum Derzhi <naum.derzhi@halliburton.com>
Date: Jan 24, 2019, 15:27 UTC
Message-ID: <7A854577E980BE4FB29FDD041B9B75E701799D06@NP1EXMB203.corp.halliburton.com>
In-Reply-To: <20190124151739.GA26616@sigill.intra.peff.net>

Thank you very much. We'll try

 
 
Naum Derzhi
Chief Scientific Adviser, Physics
 
3000 N Sam Houston Pkwy E, Technology Center, Office T1241H
Houston, TX 77032

Office: +1 (281) 871 3278

Follow Halliburton: LinkedIn | Facebook | Twitter | YouTube | Blog

This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

-----Original Message-----
From: Jeff King <peff@peff.net> 
Sent: Thursday, January 24, 2019 9:18 AM
To: Naum Derzhi <Naum.Derzhi@halliburton.com>
Cc: git@vger.kernel.org
Subject: [EXTERNAL] Re: Removing data from repository

External Sender: Use caution with links/attachments.

On Thu, Jan 24, 2019 at 02:51:50PM +0000, Naum Derzhi wrote:

Show 10 quoted lines

> I have this problem: years ago one of our developers committed a large
> (Gigabytes) piece of binary data into our project repository. This 
> should not have been done, but it happened. (Honest, it was not me).
> We never needed this data in the repository.
>
> Using git rm removes these files from the working tree, but they are 
> still somewhere in the repository, so when we clone, we transfer 
> gigabytes of unneeded data.
>
> Is it possible to fix this problem?

You'll have to rewrite the offending history. You can use git-filter-branch. See especially these sections of the manpage:

  https://urldefense.proofpoint.com/v2/url?u=https-3A__git-2Dscm.com_docs_git-2Dfilter-2Dbranch-23-5Fexamples&d=DwIBaQ&c=PskvixtEUDK7wuWU-tIg6oKuGYBRbrMXk2FZvF0UfTo&r=osDENe9k4dELyyh8ZmVzyWvVmzDFBB9Z_NHjYAXnqMg&m=090aUSqTq9bhewM2h12aExyV9vDRW60wwTcczIuu7Tg&s=_cdkm_E9dmM7yuNx0_5fpQKrV9wtnG4LPr8uooiV3cI&e=

  https://urldefense.proofpoint.com/v2/url?u=https-3A__git-2Dscm.com_docs_git-2Dfilter-2Dbranch-23-5Fchecklist-5Ffor-5Fshrinking-5Fa-5Frepository&d=DwIBaQ&c=PskvixtEUDK7wuWU-tIg6oKuGYBRbrMXk2FZvF0UfTo&r=osDENe9k4dELyyh8ZmVzyWvVmzDFBB9Z_NHjYAXnqMg&m=090aUSqTq9bhewM2h12aExyV9vDRW60wwTcczIuu7Tg&s=iBeuLkNNZbLqcAmKNw6JI7TnSt6KDObIDaBaEgQFcDQ&e=

as well as the warning in the DESCRIPTION section:

  WARNING! The rewritten history will have different object names for
  all the objects and will not converge with the original branch. You
  will not be able to easily push and distribute the rewritten branch on
  top of the original branch. Please do not use this command if you do
  not know the full implications, and avoid using it anyway, if a simple
  single commit would suffice to fix your problem. (See the "RECOVERING
  FROM UPSTREAM REBASE" section in git-rebase(1) for further information
  about rewriting published history.)

You may also want to check out the BFG repo cleaner[1], as separate project that handles this case a little more simply (it doesn't save you from dealing with rewritten history, but it does avoid having to learn filter-branch's flexible but confusing syntax).

-Peff

[1] https://urldefense.proofpoint.com/v2/url?u=https-3A__rtyley.github.io_bfg-2Drepo-2Dcleaner_&d=DwIBaQ&c=PskvixtEUDK7wuWU-tIg6oKuGYBRbrMXk2FZvF0UfTo&r=osDENe9k4dELyyh8ZmVzyWvVmzDFBB9Z_NHjYAXnqMg&m=090aUSqTq9bhewM2h12aExyV9vDRW60wwTcczIuu7Tg&s=ADIQXtQrHUv3xXudfCN94eL00x0w0vr8uK2FQMpA2dI&e=