git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/4] fast-import: only store commit objects

From
Junio C Hamano <gitster@pobox.com>
Date
May 7, 2013, 07:12 UTC
Message-ID
<7vehdjmffp.fsf@alter.siamese.dyndns.org>
In-Reply-To
<518874A5.5050002@alum.mit.edu>
Michael Haggerty <mhagger@alum.mit.edu> writes:
Show 8 quoted lines
>>>>> CVS stores all of the revisions of a single file in a single filename,v
>>>>> file in rcsfile(5) format.  The revisions are stored as deltas ordered
>>>>> so that a single revision can be reconstructed from a single serial read
>>>>> of the file.
>>>>>
>>>>> cvs2git reads each of these files once, reconstructing *all* of the
>>>>> revisions for a file in a single go.  It then pours them into a
>>>>> git-fast-import stream as blobs and sets a mark on each blob.

This is more or less off-topic but in the bigger picture it is more interesting and important X-<.

The way you describe how cvs2git handles the blobs is the more efficient way, given that fast-import does not even attempt to bother to create good deltas. The only thing it does is to see if the current data deltifies against the last object.

IIRC, CVS's backend storage is mostly recorded in backward delta, so if you are feeding the blob data from new to old, then the resulting pack would follow Linus's law (the file generally grows over time) and would generally give you a good deltified chain of objects.

Previous: Felipe ContrerasNext: Michael Haggerty
Message 31 of 38 in “fast-export: speed improvements”
  1. 0/4 fast-export: speed improvementsFelipe Contreras, May 3, 2013
  2. 1/4 fast-{import,export}: use get_sha1_hex() directlyFelipe Contreras, May 3, 2013
  3. Junio C HamanoMay 3, 2013
  4. 2/4 fast-export: improve speed by skipping blobsFelipe Contreras, May 3, 2013
  5. Junio C HamanoMay 3, 2013
  6. 3/4 fast-export: don't parse all the commitsFelipe Contreras, May 3, 2013
  7. Junio C HamanoMay 3, 2013
  8. Felipe ContrerasMay 4, 2013
  9. Junio C HamanoMay 4, 2013
  10. 4/4 fast-import: only store commit objectsFelipe Contreras, May 3, 2013
  11. Thomas RastMay 3, 2013
  12. Felipe ContrerasMay 3, 2013
  13. Michael HaggertyMay 6, 2013
  14. Thomas RastMay 6, 2013
  15. Michael HaggertyMay 6, 2013
  16. Junio C HamanoMay 6, 2013
  17. Felipe ContrerasMay 6, 2013
  18. Felipe ContrerasMay 6, 2013
  19. Michael HaggertyMay 7, 2013
  20. Johannes SchindelinMay 7, 2013
  21. Felipe ContrerasMay 7, 2013
  22. Michael HaggertyMay 7, 2013
  23. Felipe ContrerasMay 7, 2013
  24. Felipe ContrerasMay 6, 2013
  25. Michael HaggertyMay 7, 2013
  26. Johannes SchindelinMay 7, 2013
  27. Felipe ContrerasMay 7, 2013
  28. Felipe ContrerasMay 7, 2013
  29. Michael HaggertyMay 7, 2013
  30. Felipe ContrerasMay 7, 2013
  31. Junio C HamanoMay 7, 2013
  32. Michael HaggertyMay 7, 2013
  33. Johannes SchindelinMay 6, 2013
  34. Felipe ContrerasMay 6, 2013
  35. Junio C HamanoMay 3, 2013
  36. Felipe ContrerasMay 3, 2013
  37. Junio C HamanoMay 3, 2013
  38. Felipe ContrerasMay 4, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.