Re: [PATCH] git-fast-import: note 1M limit of mark number

From: Sam Vilain <sam@vilain.net>
Date: Apr 15, 2008, 21:05 UTC
Message-ID: <48051882.8000201@vilain.net>
In-Reply-To: <4804CECE.2040205@alum.mit.edu>

Michael Haggerty wrote:

Show 12 quoted lines

>> ++
>> +Note that due to current internal limitations, you may not make marks
>> +with a higher number than 1048575 (2^20-1).
>>  
>>  * A complete 40 byte or abbreviated commit SHA-1 in hex.
>>  
> 
> Oh.  Um.  That is an awkwardly small number nowadays.
> 
> cvs2svn has been used for repositories with O(2^20) distinct file
> revisions (KDE, Mozilla, NetBSD, ...).  So this limit will likely be too
> small for some users.

Right. But, if you're not making the importer you write for a conversion of that size restartable, you're insane. So, marking more than 1Mi *marks* in a single gfi session might not be so vital.

It only tripped me up because I was using a database sequence to generate the marks, which meant I hit the ceiling.

Show 6 quoted lines

> Moreover, cvs2git needs to generate marks for both file contents and for
> commits.  It generates the latter by adding 1000000000 to the small
> integer IDs that it uses internally.  If git-fast-import only allows
> 20-bit integers, this makes me wonder why this hasn't broken
> dramatically in the past.  Pure numerological good fortune, combined
> with weak range checking in git-fast-import?

Perhaps.  All I saw was that after I hit 1Mi for the mark ID, the mark
numbers in the returned file were drastically different from the ones I
put in.  I had a glance over this code and it seemed likely to be a
culprit - this docpatch is really more raising awareness of the problem.
 Obviously finding the fault and fixing it would be preferable.

> While I'm at it, let me also renew my suggestion that git-fast-import
> use separate namespaces ("markspaces", so to speak) for file content
> marks and for commit marks.  There is no reason for these distinct types
> of marks to be located in a shared space of integers.

There is a reason, it's because they're both just object IDs. Is it really that much of a drag? I know what you mean though, it meant for my code I had to keep track of which type each mark was.

Sam.