threads / discuss / 48696

How to delete files and directories from git commit history?

Subject: How to delete files and directories from git commit history?

## tl;dr

5 messages between Jun 12, 2018 and Jun 12, 2018.

replies: 4people: 5as markdown or json

Steve Litt· Jun 12, 2018, 19:44 UTC · lore
Hi all,

I have git 2.17.1 running on Void Linux 64 bit running the Linux 4.16.9_1 kernel, not available to the public in any way (yet). I have a repository in my project's working directory, and push to a bare repository on the hard disk.

My project (call it myproject) had a directory (call it docs/propdir) that was unnecessary for the project, and I've decided I don't want to offer the files in that directory as free software. So I need to delete docs/propdir from all commits in the repository. I did the following, while in my working repository's myproject directory:

git filter-branch --tree-filter 'rm -rf docs/propdir' HEAD

After that command, I could git clone the working repo and then git checkout to a commit stage that used to have the directory and files, and they're not there. So far, so good.

But then I view all filenames from that directory that have ever been in the project, as follows:

git cat-file --buffer --batch-all-objects \
 --batch-check='%(objecttype) %(objectname)' \
 | grep ^c | cut -d " "  -f 2 \
 | xargs -n 1 git ls-tree -r | sort | uniq \
 | grep propdir

The preceding command lists directory docs/propdir and all the files it's ever contained. This makes me uneasy because if the filenames are still there, I wonder if there's a route to get the files with a git command. Second, I'd prefer that when my repo is exposed to the public, people not know this directory and these files ever existed.

What command do I do to remove all mention of doc/propdir and its files from my git history?

Thanks,
SteveT

Steve Litt June 2018 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28

Eckhard Maaß· Jun 12, 2018, 20:25 UTC · re: Steve Litt · lore

Re: How to delete files and directories from git commit history?

On Tue, Jun 12, 2018 at 03:44:13PM -0400, Steve Litt wrote:
> git filter-branch --tree-filter 'rm -rf docs/propdir' HEAD
...
> What command do I do to remove all mention of doc/propdir and its
> files from my git history?

Are you sure that you pruned all branches? I would have expected a command like "git filter-branch ... -- --all" to prune every branch from the directories.

Greetings, Eckhard

Christian Couder· Jun 12, 2018, 20:26 UTC · re: Steve Litt · lore

Re: How to delete files and directories from git commit history?

Hi,
On Tue, Jun 12, 2018 at 9:44 PM, Steve Litt <slitt@troubleshooters.com> wrote:
Show 7 quoted lines
> My project (call it myproject) had a directory (call it docs/propdir)
> that was unnecessary for the project, and I've decided I don't want to
> offer the files in that directory as free software. So I need to delete
> docs/propdir from all commits in the repository. I did the following,
> while in my working repository's myproject directory:
>
> git filter-branch --tree-filter 'rm -rf docs/propdir' HEAD
[...]
> What command do I do to remove all mention of doc/propdir and its
> files from my git history?

Did you check the "CHECKLIST FOR SHRINKING A REPOSITORY" section of the filter-branch man/help page?

Best, Christian.

Eric Sunshine· Jun 12, 2018, 20:33 UTC · re: Christian Couder · lore

Re: How to delete files and directories from git commit history?

On Tue, Jun 12, 2018 at 4:26 PM, Christian Couder <christian.couder@gmail.com> wrote:

Show 15 quoted lines
> On Tue, Jun 12, 2018 at 9:44 PM, Steve Litt <slitt@troubleshooters.com> wrote:
>
>> My project (call it myproject) had a directory (call it docs/propdir)
>> that was unnecessary for the project, and I've decided I don't want to
>> offer the files in that directory as free software. So I need to delete
>> docs/propdir from all commits in the repository. I did the following,
>> while in my working repository's myproject directory:
>>
>> git filter-branch --tree-filter 'rm -rf docs/propdir' HEAD
>
>> What command do I do to remove all mention of doc/propdir and its
>> files from my git history?
>
> Did you check the "CHECKLIST FOR SHRINKING A REPOSITORY" section of
> the filter-branch man/help page?

Also see BFG Repo Cleaner for an alternative. https://rtyley.github.io/bfg-repo-cleaner/

Junio C Hamano· Jun 12, 2018, 20:36 UTC · re: Steve Litt · lore

Re: How to delete files and directories from git commit history?

Steve Litt <slitt@troubleshooters.com> writes:
Show 11 quoted lines
> But then I view all filenames from that directory that have ever been
> in the project, as follows:
>
> git cat-file --buffer --batch-all-objects \
>  --batch-check='%(objecttype) %(objectname)' \
>  | grep ^c | cut -d " "  -f 2 \
>  | xargs -n 1 git ls-tree -r | sort | uniq \
>  | grep propdir
>
> The preceding command lists directory docs/propdir and all the files
> it's ever contained....

In which repository did you run this? As cat-file's documentation clearly states, --batch-all-objects does not *care* about the reachability, so even after rewriting the history using filter-branch, as long as you have the original objects before your history rewrite in the repository, it _will_ see those trees with propdir, even if they are now unreachable thanks to your filter-branch. If you are doing the above in the original (or in a local clone), try running "git repack -a -d && git prune --expire=now" after filter-branch.

← back to recent threads