threads / discuss / 2338

binary safe?

Subject: binary safe?

## tl;dr

9 messages between Nov 3, 2005 and Nov 4, 2005.

replies: 8people: 7as markdown or json

Randal L. Schwartz· Nov 3, 2005, 22:02 UTC · lore

I'm currently about to abandon CVS for my website management, replacing it with git.

What problems, if any, will I have using git to manage the binary files for my site, like the custom icons? CVS is doing that just fine now.

I presume emailing diff-patches is out of the question, but if all I'm doing is git-push and git-pull (using the shared central repository model), and if I'm stupid enough to have a merge error it's OK to just blow up on a binary file, will everything else work fine?

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Junio C Hamano· Nov 3, 2005, 22:49 UTC · re: Randal L. Schwartz · lore

Re: binary safe?

merlyn@stonehenge.com (Randal L. Schwartz) writes:
> I presume emailing diff-patches is out of the question, but if all I'm
> doing is git-push and git-pull (using the shared central repository
> model), and if I'm stupid enough to have a merge error it's OK to just
> blow up on a binary file, will everything else work fine?

It should. I trust git well enough to track some png files in my day-job project.

Martin Langhoff· Nov 3, 2005, 23:00 UTC · re: Junio C Hamano · lore

Re: binary safe?

On 11/4/05, Junio C Hamano <junkio@cox.net> wrote:
Show 9 quoted lines
> merlyn@stonehenge.com (Randal L. Schwartz) writes:
>
> > I presume emailing diff-patches is out of the question, but if all I'm
> > doing is git-push and git-pull (using the shared central repository
> > model), and if I'm stupid enough to have a merge error it's OK to just
> > blow up on a binary file, will everything else work fine?
>
> It should.  I trust git well enough to track some png files in
> my day-job project.
Yes it works, and cvsimport -k will do the right thing for you.

We are tracking projects with several binary files. The only issue as you note is that trading patches with git-format-patch and git-am doesn't quite deal with it. But you can still do it across different git heads with git-read-tree -m as long as your binary file merges are truly trivial.

cheers,
martin
David Brown· Nov 4, 2005, 16:54 UTC · re: Martin Langhoff · lore

Re: binary safe?

On Fri, Nov 04, 2005 at 12:00:54PM +1300, Martin Langhoff wrote:
> Yes it works, and cvsimport -k will do the right thing for you.

Unless it has changed from 0.99.9b, 'cvsimport -k' will very much scramble some binary files. '-k' passes the '-kk' option which causes CVS to strip the keywords down. It needs to pass -ko through if you want it to be able to handle binary files.

However, since CVS (RCS really) can remember the state of this flag, it does work to 'cvs admin -ko filename' beforehand, and then do the cvsimport without the '-k' option.

Dave
Martin Langhoff· Nov 4, 2005, 21:22 UTC · re: David Brown · lore

Re: binary safe?

On 11/5/05, David Brown <git@davidb.org> wrote:
Show 8 quoted lines
> On Fri, Nov 04, 2005 at 12:00:54PM +1300, Martin Langhoff wrote:
>
> > Yes it works, and cvsimport -k will do the right thing for you.
>
> Unless it has changed from 0.99.9b, 'cvsimport -k' will very much scramble
> some binary files.  '-k' passes the '-kk' option which causes CVS to strip
> the keywords down.  It needs to pass -ko through if you want it to be able
> to handle binary files.

There is a misunderstanding here. We don't pass '-kk' to the cvs utility -- we pass it at the protocol level. Strangely enough, when passing ko we were getting broken files, and when passing kk we got all the files correctly. I explored and tested this quite a bit when I added the flag, and explicitly tested it with files that _would_ get broken with cvs update -kk.

To recap: my main test repository has a lot of binary files, files that do get broken if I do a cvs checkout with -kk. git-cvsimport gets them right with its -k parameter. Don't ask me why, though: the cvs protocol is really messy, and I suspect that part of the -kk option is being 'implemented' on the client side.

(That being said, if you have a case where git-cvsimport is doing the wrong thing, let me know!)

> However, since CVS (RCS really) can remember the state of this flag, it
> does work to  'cvs admin -ko filename' beforehand, and then do the
> cvsimport without the '-k' option.

Yes, but a repo you don't control, where people are using keywords, means thatyou need to do -kk to kill the keywords or your imported files are going to have a horrid amount of noise in them.

cheers,
martin
David Brown· Nov 4, 2005, 21:27 UTC · re: Martin Langhoff · lore

Re: binary safe?

On Sat, Nov 05, 2005 at 10:22:29AM +1300, Martin Langhoff wrote:
Show 10 quoted lines
> (That being said, if you have a case where git-cvsimport is doing the
> wrong thing, let me know!)
> 
> > However, since CVS (RCS really) can remember the state of this flag, it
> > does work to  'cvs admin -ko filename' beforehand, and then do the
> > cvsimport without the '-k' option.
> 
> Yes, but a repo you don't control, where people are using keywords,
> means thatyou need to do -kk to kill the keywords or your imported
> files are going to have a horrid amount of noise in them.
Yes, the unpleasantness of CVS.

However, I was unable to do a proper git-cvsimport of the SourceForge 'vim' archive, with '-k'. By not giving it '-k' and using cvs admin '-ko' on the appropriate files, I was able to get the correct results.

May be the interpretation of the option depends on the particular server being used?

I can investigate later which particular file is causing the problem.
Dave
Linus Torvalds· Nov 3, 2005, 22:50 UTC · re: Randal L. Schwartz · lore

Re: binary safe?

On Thu, 3 Nov 2005, Randal L. Schwartz wrote:
Show 7 quoted lines
> 
> I'm currently about to abandon CVS for my website management,
> replacing it with git.
> 
> What problems, if any, will I have using git to manage the binary
> files for my site, like the custom icons?  CVS is doing that just fine
> now.
Git doesn't have any textual representations anywhere.
So the _only_ problem should be "git diff" (and related patch-based tools 
- git-apply etc). They'll simply not work. For similar reasons, a 
three-way merge will obviously fail.
> I presume emailing diff-patches is out of the question, but if all I'm
> doing is git-push and git-pull (using the shared central repository
> model), and if I'm stupid enough to have a merge error it's OK to just
> blow up on a binary file, will everything else work fine?

Yes. I don't think it's been heavily tested, but the very architecture of git should mean that there just shouldn't be any issues with binary files outside of the obvious ones.

The only binary file the kernel ever uses is the logo.gif thing, so it's been "tested" in the sense that binary files exist, but there's never been any changes to that file, so..

		Linus
Chris Wedgwood· Nov 4, 2005, 03:49 UTC · re: Linus Torvalds · lore

Re: binary safe?

On Thu, Nov 03, 2005 at 02:50:21PM -0800, Linus Torvalds wrote:
> So the _only_ problem should be "git diff" (and related patch-based
> tools - git-apply etc). They'll simply not work. For similar
> reasons, a three-way merge will obviously fail.

I always thought it would be nice for binary files if SCMs could export a summary of what ranges of bytes changes or even xdelta'ish details.

Given that people do stick large multi-MB binary blobs in SCMs and sometimes these update only a few bytes at a time it has some value.

Nick Hengeveld· Nov 3, 2005, 23:05 UTC · re: Randal L. Schwartz · lore

Re: binary safe?

On Thu, Nov 03, 2005 at 02:02:20PM -0800, Randal L. Schwartz wrote:
> What problems, if any, will I have using git to manage the binary
> files for my site, like the custom icons?  CVS is doing that just fine
> now.

We're now using git in production to distribute content, of which well over 90% is binary files. Works great.

-- 
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.

← back to recent threads