threads / discuss / 16783

is it possible filter the revision history of a single file into another repository?

Subject: is it possible filter the revision history of a single file into another repository?

## tl;dr

8 messages between Dec 18, 2008 and Dec 19, 2008.

replies: 7people: 3as markdown or json

Whit Armstrong· Dec 18, 2008, 13:51 UTC · lore

For instance, if my repository contains foo.c, and 100 other files.

I would like to create a new and separate repository containing only the revision history of foo.c.

Would someone mind pointing me at some documentation for this procedure if it exists?

Thanks very much, Whit

Thomas Jarosch· Dec 18, 2008, 14:04 UTC · re: Whit Armstrong · lore

Re: is it possible filter the revision history of a single file into another repository?

On Thursday, 18. December 2008 14:51:12 Whit Armstrong wrote:

Show 7 quoted lines

> For instance, if my repository contains foo.c, and 100 other files.
>
> I would like to create a new and separate repository containing only
> the revision history of foo.c.
>
> Would someone mind pointing me at some documentation for this
> procedure if it exists?

This worked for me:

git filter-branch --tag-name-filter cat --index-filter \
    'git ls-files -s |grep -P "\t(DIR1|DIR2)" \
    |GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
    mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' -- --all

Run "git ls-files -s" to see the output format. Replace the "DIR1|DIR2" with "foo.c".

Later on you might want to remove empty commits from the history: git filter-branch --tag-name-filter cat --commit-filter 'if [ z$1 = z`git rev-parse $3^{tree}` ]; then skip_commit "$@"; else git commit-tree "$@"; fi' "$@" -- --all

If you want to run two filter-branch commands in a row or you want to free up the space in .git afterwards:

- git for-each-ref --format='%(refname)' refs/original | xargs -i git update-ref -d {}
- git reflog expire --expire=0 --all
- git repack -a -d
- git prune

Cheers, Thomas

Whit Armstrong· Dec 18, 2008, 14:19 UTC · re: Thomas Jarosch · lore

Re: is it possible filter the revision history of a single file into another repository?

thanks, I will give this a try.

On Thu, Dec 18, 2008 at 9:04 AM, Thomas Jarosch <thomas.jarosch@intra2net.com> wrote:

Show 34 quoted lines

> On Thursday, 18. December 2008 14:51:12 Whit Armstrong wrote:
>> For instance, if my repository contains foo.c, and 100 other files.
>>
>> I would like to create a new and separate repository containing only
>> the revision history of foo.c.
>>
>> Would someone mind pointing me at some documentation for this
>> procedure if it exists?
>
> This worked for me:
>
> git filter-branch --tag-name-filter cat --index-filter \
>    'git ls-files -s |grep -P "\t(DIR1|DIR2)" \
>    |GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
>    mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' -- --all
>
> Run "git ls-files -s" to see the output format.
> Replace the "DIR1|DIR2" with "foo.c".
>
> Later on you might want to remove empty commits from the history:
> git filter-branch --tag-name-filter cat --commit-filter 'if [ z$1 = z`git rev-parse $3^{tree}` ]; then skip_commit "$@"; else git commit-tree "$@"; fi' "$@" -- --all
>
> If you want to run two filter-branch commands in a row
> or you want to free up the space in .git afterwards:
>
> - git for-each-ref --format='%(refname)' refs/original | xargs -i git update-ref -d {}
> - git reflog expire --expire=0 --all
> - git repack -a -d
> - git prune
>
> Cheers,
> Thomas
>
>

Whit Armstrong· Dec 18, 2008, 19:51 UTC · re: Whit Armstrong · lore

Re: is it possible filter the revision history of a single file into another repository?

Sorry, seem to be getting this error: `/home/whit/dvl/risk.metrics.utils/RiskMetrics/.git-rewrite/t/../index.new': No such file or directory

do I need to set up the index file first?

Is there a good site that documents this procedure?

[whit@linuxsvr RiskMetrics]$ git filter-branch --tag-name-filter cat --index-filter \

>    'git ls-files -s |grep -P "riskmetrics.rb" \
>    |GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
>    mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' -- --all

Rewrite 8f1a0eaae033d109f4a3a4b410bd8e04dd9997db (1/481)mv: cannot
stat `/home/whit/dvl/risk.metrics.utils/RiskMetrics/.git-rewrite/t/../index.new':
No such file or directory
index filter failed: git ls-files -s |grep -P "riskmetrics.rb" \
   |GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
   mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE
[whit@linuxsvr RiskMetrics]$

On Thu, Dec 18, 2008 at 9:19 AM, Whit Armstrong <armstrong.whit@gmail.com> wrote:

Show 39 quoted lines

> thanks, I will give this a try.
>
> On Thu, Dec 18, 2008 at 9:04 AM, Thomas Jarosch
> <thomas.jarosch@intra2net.com> wrote:
>> On Thursday, 18. December 2008 14:51:12 Whit Armstrong wrote:
>>> For instance, if my repository contains foo.c, and 100 other files.
>>>
>>> I would like to create a new and separate repository containing only
>>> the revision history of foo.c.
>>>
>>> Would someone mind pointing me at some documentation for this
>>> procedure if it exists?
>>
>> This worked for me:
>>
>> git filter-branch --tag-name-filter cat --index-filter \
>>    'git ls-files -s |grep -P "\t(DIR1|DIR2)" \
>>    |GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
>>    mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' -- --all
>>
>> Run "git ls-files -s" to see the output format.
>> Replace the "DIR1|DIR2" with "foo.c".
>>
>> Later on you might want to remove empty commits from the history:
>> git filter-branch --tag-name-filter cat --commit-filter 'if [ z$1 = z`git rev-parse $3^{tree}` ]; then skip_commit "$@"; else git commit-tree "$@"; fi' "$@" -- --all
>>
>> If you want to run two filter-branch commands in a row
>> or you want to free up the space in .git afterwards:
>>
>> - git for-each-ref --format='%(refname)' refs/original | xargs -i git update-ref -d {}
>> - git reflog expire --expire=0 --all
>> - git repack -a -d
>> - git prune
>>
>> Cheers,
>> Thomas
>>
>>
>

Thomas Jarosch· Dec 19, 2008, 09:44 UTC · re: Whit Armstrong · lore

Re: is it possible filter the revision history of a single file into another repository?

On Thursday, 18. December 2008 20:51:38 Whit Armstrong wrote:

Show 5 quoted lines

> Sorry, seem to be getting this error:
> `/home/whit/dvl/risk.metrics.utils/RiskMetrics/.git-rewrite/t/../index.new'
>: No such file or directory
>
> do I need to set up the index file first?

Hmm, I guess you have an empty commit in your repository like I did. This is currently a corner case in update-index, which does not create empty index files. I posted a patch a few days ago and Junio posted an updated version of that. I could send you my version for git 1.6.0.5 if you need it.

> Is there a good site that documents this procedure?

A good start is the git-filter-branch man page and the mailinglist archive.

Thomas

Whit Armstrong· Dec 19, 2008, 13:08 UTC · re: Thomas Jarosch · lore

Re: is it possible filter the revision history of a single file into another repository?

thanks, Thomas. I could definitely pull from your tree. seems like the path of least resistance to get my repo split.

Cheers, Whit

On Fri, Dec 19, 2008 at 4:44 AM, Thomas Jarosch <thomas.jarosch@intra2net.com> wrote:

Show 19 quoted lines

> On Thursday, 18. December 2008 20:51:38 Whit Armstrong wrote:
>> Sorry, seem to be getting this error:
>> `/home/whit/dvl/risk.metrics.utils/RiskMetrics/.git-rewrite/t/../index.new'
>>: No such file or directory
>>
>> do I need to set up the index file first?
>
> Hmm, I guess you have an empty commit in your repository like I did.
> This is currently a corner case in update-index, which does not create empty
> index files. I posted a patch a few days ago and Junio posted an updated
> version of that. I could send you my version for git 1.6.0.5 if you need it.
>
>> Is there a good site that documents this procedure?
>
> A good start is the git-filter-branch man page and the mailinglist archive.
>
> Thomas
>
>

Thomas Jarosch· Dec 19, 2008, 13:17 UTC · re: Whit Armstrong · lore

Re: is it possible filter the revision history of a single file into another repository?

On Friday, 19. December 2008 14:08:23 Whit Armstrong wrote:

> thanks, Thomas. I could definitely pull from your tree.  seems like
> the path of least resistance to get my repo split.

Here's the patch I use for git 1.6.0.5. According to Junio it has the small drawback of always writing out the index, even if there are no changes.

If you need an updated patch against HEAD, look for Junio's reply to my patch in the list archive.

Enjoy, Thomas

Signed-off-by: Thomas Jarosch <thomas.jarosch@intra2net.com>

--- git-1.6.0.5/builtin-update-index.c 2008-12-08 02:21:49.000000000 +0100 +++ git-1.6.0.5.index/builtin-update-index.c 2008-12-13 12:43:14.000000000 +0100

@@ -297,6 +297,8 @@ static void read_index_info(int line_ter
 	struct strbuf buf;
 	struct strbuf uq;
 
+	int found_something = 0;
+
 	strbuf_init(&buf, 0);
 	strbuf_init(&uq, 0);
 	while (strbuf_getline(&buf, stdin, line_termination) != EOF) {
@@ -307,6 +309,8 @@ static void read_index_info(int line_ter
 		unsigned long ul;
 		int stage;
 
+		found_something = 1;
+
 		/* This reads lines formatted in one of three formats:
 		 *
 		 * (1) mode         SP sha1          TAB path
@@ -382,6 +386,11 @@ static void read_index_info(int line_ter
 	bad_line:
 		die("malformed index info %s", buf.buf);
 	}
+
+	/* Force creation of empty index - needed by git filter-branch */
+	if (!found_something)
+		active_cache_changed = 1;
+
 	strbuf_release(&buf);
 	strbuf_release(&uq);
 }

Sverre Rabbelier· Dec 18, 2008, 14:15 UTC · re: Whit Armstrong · lore

Re: is it possible filter the revision history of a single file into another repository?

On Thu, Dec 18, 2008 at 14:51, Whit Armstrong <armstrong.whit@gmail.com> wrote:

Show 7 quoted lines

> For instance, if my repository contains foo.c, and 100 other files.
>
> I would like to create a new and separate repository containing only
> the revision history of foo.c.
>
> Would someone mind pointing me at some documentation for this
> procedure if it exists?

I think "git filter-branch" is what you need. Have it filter out changes to files but foo.c, and then remove all empty commits.

-- 
Cheers,

Sverre Rabbelier