git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: fact-import: failed to apply delta

From
Daniel Barkalow <barkalow@iabervon.org>
Date
Feb 11, 2009, 18:09 UTC
Message-ID
<alpine.LNX.1.00.0902111247300.19665@iabervon.org>
In-Reply-To
<7vfxillxiu.fsf@gitster.siamese.dyndns.org>
On Tue, 10 Feb 2009, Junio C Hamano wrote:
Show 35 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > "Shawn O. Pearce" <spearce@spearce.org> writes:
> >
> >> Daniel Barkalow <barkalow@iabervon.org> wrote:
> >>> On Tue, 10 Feb 2009, Shawn O. Pearce wrote:
> >>> > 
> >>> > We should dump the cached_objects table in sha1_file.c during
> >>> > a checkpoint in fast-import.
> >>> 
> >>> No, that one's keyed by sha1, and doesn't get collisions; it's the 
> >>> delta_base_cache that's the issue; it's keyed by struct packed_git * and 
> >>> offset.
> >>
> >> Uh, yea, I realize that after I sent the message.  Does this patch
> >> fix it for you?
> >>
> >> --8<--
> >> Clear the delta base cache during fast-import checkpoint
> >>
> >> Otherwise we may reuse the same memory address for a totally
> >> different "struct packed_git", and a previously cached object from
> >> the prior occupant might be returned when trying to unpack an object
> >> from the new pack.
> >
> > Can this be made more automatic?
> >
> > For example if you do this every time a new pack is installed to
> > sha1_file(), like in add_packed_git() perhaps, wouldn't that be much less
> > error prone?
> 
> On second thought, I think fast-import is the only program that plays
> funny games of feeding a packed_git that is *not* part of the real list of
> packed_git installed in the system to unpack_entry(), so probably your
> patch is a better idea.

That's not the problem; the problem is calling free on a struct packed_git that has been given to unpack_entry(), because it raises the possibility of having the memory allocated to a different pack in the future and ending up with actively wrong entries in the delta cache, since it keys off of the pointer.

I think free_pack_by_name() also needs to drop the entries that are from the freed pack, to avoid having repack able to get the same problem, although I wouldn't be surprised if repack happened to never allocate a new pack after freeing an old pack with stale delta cache entries, or never used the delta cache after that, simply because it does one thing and then exits.

	-Daniel
*This .sig left intentionally blank*
Previous: Junio C HamanoNext: Shawn O. Pearce
Message 21 of 24 in “fact-import: failed to apply delta”
  1. Daniel BarkalowFeb 10, 2009
  2. Johannes SchindelinFeb 10, 2009
  3. Shawn O. PearceFeb 10, 2009
  4. Daniel BarkalowFeb 10, 2009
  5. Shawn O. PearceFeb 10, 2009
  6. Daniel BarkalowFeb 10, 2009
  7. Shawn O. PearceFeb 10, 2009
  8. Daniel BarkalowFeb 10, 2009
  9. Shawn O. PearceFeb 10, 2009
  10. Daniel BarkalowFeb 10, 2009
  11. Shawn O. PearceFeb 10, 2009
  12. Daniel BarkalowFeb 10, 2009
  13. Shawn O. PearceFeb 10, 2009
  14. Daniel BarkalowFeb 10, 2009
  15. Junio C HamanoFeb 10, 2009
  16. Junio C HamanoFeb 10, 2009
  17. Shawn O. PearceFeb 10, 2009
  18. Junio C HamanoFeb 10, 2009
  19. Shawn O. PearceFeb 10, 2009
  20. Junio C HamanoFeb 10, 2009
  21. Daniel BarkalowFeb 11, 2009
  22. Shawn O. PearceFeb 11, 2009
  23. Junio C HamanoFeb 11, 2009
  24. Daniel BarkalowFeb 11, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.