threads / discuss / 21473

How to ensure a word has been removed from repository?

Subject: How to ensure a word has been removed from repository?

## tl;dr

3 messages between Nov 4, 2009 and Nov 4, 2009.

replies: 2people: 3as markdown or json

Patrick Higgins· Nov 4, 2009, 02:12 UTC · lore

Hi all,

I just completed a series of filter-branch commands to remove a couple of sensitive words from a repository before I publish it. The words were found in commit messages, directory names, file contents, and various other places (kind of weird, I know). I believe I have removed them all, but I would like to double check but don't know how.

Given that much of the repository is stored in compressed packs, I can't just use grep to look for the words. To get around this, I've unpacked the objects, use a Perl script (filtinf example script) to decompress them and then use grep (this has proven to be quite slow).

Is that going to find every possible occurrence if all the relevant files are plain text?

Is there an easier way to search the repository? The way I'm doing it has required some awfully deep knowledge to expire and prune everything. I feel like I must be missing something.

Thanks, Patrick

Nicolas Pitre· Nov 4, 2009, 02:49 UTC · re: Patrick Higgins · lore

Re: How to ensure a word has been removed from repository?

On Tue, 3 Nov 2009, Patrick Higgins wrote:

Show 19 quoted lines

> Hi all,
> 
> I just completed a series of filter-branch commands to remove a couple
> of sensitive words from a repository before I publish it. The words
> were found in commit messages, directory names, file contents, and
> various other places (kind of weird, I know). I believe I have removed
> them all, but I would like to double check but don't know how.
> 
> Given that much of the repository is stored in compressed packs, I
> can't just use grep to look for the words. To get around this, I've
> unpacked the objects, use a Perl script (filtinf example script) to
> decompress them and then use grep (this has proven to be quite slow).
> 
> Is that going to find every possible occurrence if all the relevant
> files are plain text?
> 
> Is there an easier way to search the repository? The way I'm doing it
> has required some awfully deep knowledge to expire and prune
> everything. I feel like I must be missing something.

An easy way to look for the presence of a particular string in all the repository data is:

	git rev-list --all --objects | cut -c -40 | \
		git cat-file --batch | grep <string>

Alternatively you can use:

	git fast-export --all --signed-tag=verbatim | grep <string>

Nicolas

Sitaram Chamarty· Nov 4, 2009, 03:35 UTC · re: Patrick Higgins · lore

Re: How to ensure a word has been removed from repository?

On Wed, Nov 4, 2009 at 7:42 AM, Patrick Higgins <patrick.allen.higgins@gmail.com> wrote:

> Given that much of the repository is stored in compressed packs, I
> can't just use grep to look for the words. To get around this, I've
> unpacked the objects, use a Perl script (filtinf example script) to
> decompress them and then use grep (this has proven to be quite slow).

> Is that going to find every possible occurrence if all the relevant
> files are plain text?

> Is there an easier way to search the repository? The way I'm doing it
> has required some awfully deep knowledge to expire and prune
> everything. I feel like I must be missing something.

Instead of expire and prune, I'd clone the repo to some other location, and search in that clone. You'd still need the commands that Nicolas gave in his reply but you wouldn't get any false positivies from missing some reflog or something kept a ref alive.