git/list[1] front-page[2] threads[3] people[4] search[5] about
 

CAREFUL! No more delta object support!

From
Linus Torvalds <torvalds@osdl.org>
Date
Jun 28, 2005, 01:14 UTC
Message-ID
<Pine.LNX.4.58.0506271755140.19755@ppc970.osdl.org>

Some people may have noticed already (hopefully not the hard way) that the current git code doesn't support delta objects lying around in the object directory any more.

In other words, if you have delta objects, you need to un-deltify your repository _before_ you upgrade your git binaries, or they won't be able to read your objects any more.

The reason? The new git understands packed files natively, which ends up being a much bigger win in many many ways.

You should be very careful about using packed files (since they are a very recent addition), but what you can do to try them out is to do so in a separate repository.

Starting to use a packed repository is very simple indeed, and here's what you need to do for git, for example:

In your regular "git" directory (once you have ypdated your git to a recent version, in particular you need to have the "csum-file: fix missing buf pointer update" commit), do:

	git-rev-list --objects HEAD | git-pack-objects --window=50 --depth=50 out

which will say something like "Packing 3741 objects" and result in two new files a few seconds later:

	torvalds@ppc970:~/git> ls -lh out*
	-rw-r--r--  1 torvalds torvalds  89K Jun 27 17:59 out.idx
	-rw-r--r--  1 torvalds torvalds 1.3M Jun 27 17:59 out.pack

now, don't do anythign with those files, but instead go and create a directory somewhere else:

	cd ~
	mkdir packed-git-trial
	cd packed-git-trial
	git-init-db

you have now obviously created a totally empty repository. Now, let's populate that empty repository with _just_ the pack files:

	mkdir .git/objects/pack
	mv ~/git/out.* .git/objects/pack

and then, move over your tags, in particularly the HEAD pointer, with something like

	cat ~/git/.git/HEAD > .git/HEAD
and voila, you're done. Try "gitk", for example. Or "git log".

Now, what's even cooler is how you can just start using this packed tree: feel free to do a test-commit or something, and notice how git starts populating the empty .git/objects/xx/ subdirectories with new objects. But it still relies on the pack-file for the old history.

Now, there's still a misfeature there, which is that when you create a new object, it doesn't check whether that object already exists in the pack-file, so you'll end up with a few recent objects that you really don't need (notably tree objects), and we'll fix that eventually. But notice how you started with a 17MB .git/objects/ directory in your original tree, and you now have just a 1.3MB pack-file and a 90kB index file that replaces all that?

There are some other issues too, like the fact that "git-fsck-cache" doesn't know about the pack-files yet, so it will complain about missing objects etc. Also, please note that the pack-file _only_ packs the commits and the things reachable from them: things like tags (and your references in your .git/refs directory) need to be copied over separately.

So this is all very rough, still, but the basics do actually seem to work (ie anything that doesn't look directly at the object files - which is pretty much all of it except for fsck and the direct-filesystem-access things like "rsync" and "git-local-pull").

Maybe you might not want to switch over yet, and as mentioned, rsync then ends up not being a good way to sync (nor git-local-pull), but the "git-http/ssh-pull" family should hopefully just work.

I've used a packed kernel tree too, so this has gotten _some_ testing even on really quite big git trees.

			Linus
Next: Christopher Li
Message 1 of 38 in “CAREFUL! No more delta object support!”
  1. Linus TorvaldsJun 28, 2005
  2. Christopher LiJun 27, 2005
  3. Linus TorvaldsJun 28, 2005
  4. Junio C HamanoJun 28, 2005
  5. Christopher LiJun 28, 2005
  6. Petr BaudisJun 28, 2005
  7. Benjamin LaHaiseJun 28, 2005
  8. Petr BaudisJun 28, 2005
  9. Jan HarkesJun 28, 2005
  10. Christopher LiJun 28, 2005
  11. Linus TorvaldsJun 28, 2005
  12. Emit base objects of a delta chain when the delta is output.Junio C Hamano, Jun 29, 2005
  13. Junio C HamanoJun 28, 2005
  14. Skip writing out sha1 files for objects in packed git.Junio C Hamano, Jun 28, 2005
  15. Linus TorvaldsJun 28, 2005
  16. Junio C HamanoJun 28, 2005
  17. Linus TorvaldsJun 28, 2005
  18. Linus TorvaldsJun 28, 2005
  19. Junio C HamanoJun 28, 2005
  20. Adjust to git-init-db creating $GIT_OBJECT_DIRECTORY/packJunio C Hamano, Jun 28, 2005
  21. Linus TorvaldsJun 28, 2005
  22. Daniel BarkalowJun 28, 2005
  23. Linus TorvaldsJun 28, 2005
  24. Linus TorvaldsJun 28, 2005
  25. Daniel BarkalowJun 28, 2005
  26. Linus TorvaldsJun 28, 2005
  27. Linus TorvaldsJun 28, 2005
  28. Matthias UrlichsJun 28, 2005
  29. Matthias UrlichsJun 28, 2005
  30. Daniel BarkalowJun 28, 2005
  31. Linus TorvaldsJun 29, 2005
  32. Linus TorvaldsJun 29, 2005
  33. Daniel BarkalowJun 29, 2005
  34. Linus TorvaldsJun 29, 2005
  35. Daniel BarkalowJun 29, 2005
  36. Adjust fsck-cache to packed GIT and alternate object pool.Junio C Hamano, Jun 28, 2005
  37. Expose packed_git and alt_odb.Junio C Hamano, Jun 28, 2005
  38. 3/3 Update fsck-cache (take 2)Junio C Hamano, Jun 28, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.