RE: [EXTERNAL] Re: Removing data from repository

From: Naum Derzhi <naum.derzhi@halliburton.com>
Date: Jan 24, 2019, 15:28 UTC
Message-ID: <7A854577E980BE4FB29FDD041B9B75E701799D1C@NP1EXMB203.corp.halliburton.com>
In-Reply-To: <9dce6a3d8ff2f10621ea81fbd9ac9964c10aff9c.camel@mad-scientist.net>

Thank you

 
 
Naum Derzhi
Chief Scientific Adviser, Physics
 
3000 N Sam Houston Pkwy E, Technology Center, Office T1241H
Houston, TX 77032

Office: +1 (281) 871 3278

Follow Halliburton: LinkedIn | Facebook | Twitter | YouTube | Blog

This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

-----Original Message-----
From: Paul Smith <paul@mad-scientist.net> 
Sent: Thursday, January 24, 2019 9:23 AM
To: Naum Derzhi <Naum.Derzhi@halliburton.com>; git@vger.kernel.org
Subject: [EXTERNAL] Re: Removing data from repository

External Sender: Use caution with links/attachments.

On Thu, 2019-01-24 at 14:51 +0000, Naum Derzhi wrote:

Show 12 quoted lines

> Greetings,
>
> I have this problem: years ago one of our developers committed a large 
> (Gigabytes) piece of binary data into our project repository.
> This should not have been done, but it happened. (Honest, it was not 
> me). We never needed this data in the repository.
>
> Using git rm removes these files from the working tree, but they are 
> still somewhere in the repository, so when we clone, we transfer 
> gigabytes of unneeded data.
>
> Is it possible to fix this problem?

It is possible, but it isn't pretty.  By waiting so long to try to fix it you've compounded the impacts dramatically unfortunately.  By removing that file you will in effect be rewriting the history of your repository starting with the commit that introduced the problematic file, that will be removed, all the way forward.

That means that every clone of the repository will have to be, at the very least, "hard-reset" and preferably (to avoid accidentally re- introducing the bad file) re-cloned from scratch.

As for actually removing the file, you can find information on how to do this all over the Google. Here's some reasonable advice from StackOverflow:

https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_questions_2100907_how-2Dto-2Dremove-2Ddelete-2Da-2Dlarge-2Dfile-2Dfrom-2Dcommit-2Dhistory-2Din-2Dgit-2Drepository&d=DwICaQ&c=PskvixtEUDK7wuWU-tIg6oKuGYBRbrMXk2FZvF0UfTo&r=osDENe9k4dELyyh8ZmVzyWvVmzDFBB9Z_NHjYAXnqMg&m=dmUTLEwyrdm6hRWOzlTFqBaVv4_WJoUWDed9sG4uDQ0&s=QrNGuuWuNZQqkSksRUK9sfcG5iwVZIxmE6UZcneRxHg&e=