git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 2/2] alloc.c: remove the redundant commit_count variable

From
Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Date
Jul 11, 2014, 09:41 UTC
Message-ID
<53BFB14C.10000@ramsay1.demon.co.uk>
In-Reply-To
<20140711083220.GA5407@sigill.intra.peff.net>
On 11/07/14 09:32, Jeff King wrote:
Show 35 quoted lines
> On Fri, Jul 11, 2014 at 01:59:53AM +0100, Ramsay Jones wrote:
> 
>>> The code you're touching here was trying to make sure that each commit
>>> gets a unique index, under the assumption that commits only get
>>> allocated via alloc_commit_node. But I think that assumption is wrong.
>>> We can also get commit objects by allocating an OBJ_NONE (e.g., via
>>> lookup_unknown_object) and then converting it into an OBJ_COMMIT when we
>>> find out what it is.
>>
>> Hmm, I don't know how the object is converted, but the object allocator
>> is actually allocating an 'union any_object', so it's allocating more
>> space than for a struct object anyway.
> 
> Right, we would generally want to avoid lookup_unknown_object where we
> can for that reason.
> 
>> If you add an 'index' field to struct object, (and remove it from
>> struct commit) it could be set in alloc_object_node(). ie _all_ node
>> types get an index field.
> 
> That was something I considered when we did the original commit-slab
> work, as it would let you do similar tricks for any set of objects, not
> just commits. The reasons against it are:
> 
>   1. It would bloat the size of blob and tree structs by at least 4
>      bytes (probably 8 for alignment). In most repos, commits make up
>      only 10-20% of the total objects (so for linux.git, we're talking
>      about 25MB extra in the working set).
> 
>   2. It makes single types sparse in the index space. In cases where you
>      do just want to keep data on commits (and that is the main use),
>      you end up allocating a slab entry per object, rather than per
>      commit. That wastes memory (much worse than 25MB if your slab items
>      are large), and reduces cache locality.
> 
Ahem, yeah I keep telling myself not to post at 2am ... ;-)

Although I haven't given this too much extra thought, I'm beginning to think that your best course would be to revert commit 969eba63 and add an convert_object_to_commit() function to commit.c which would set c->index. Then track down each place an OBJ_NONE gets converted to an OBJ_COMMIT and insert a call to convert_object_to_commit(). (which may do more than just set the index field ...)

Hmm, I've just noticed a new series in my in-box. I guess I will discover what you decided to do shortly ... ;-P

ATB, Ramsay Jones

Previous: Jeff KingNext: Jeff King
Message 5 of 35 in “alloc.c: remove the redundant commit_count variable”
  1. 2/2 alloc.c: remove the redundant commit_count variableRamsay Jones, Jul 10, 2014
  2. Jeff KingJul 11, 2014
  3. Ramsay JonesJul 11, 2014
  4. Jeff KingJul 11, 2014
  5. Ramsay JonesJul 11, 2014
  6. 0/7 ensure index is set for all OBJ_COMMIT objects variableJeff King, Jul 11, 2014
  7. 1/7 alloc.c: remove the alloc_raw_commit_node() functionJeff King, Jul 11, 2014
  8. 2/7 move setting of object->type to alloc_* functionsJeff King, Jul 11, 2014
  9. Ramsay JonesJul 12, 2014
  10. Jeff KingJul 12, 2014
  11. Jeff KingJul 13, 2014
  12. 1/8 alloc.c: remove the alloc_raw_commit_node() functionJeff King, Jul 13, 2014
  13. Junio C HamanoJul 15, 2014
  14. 2/8 alloc: write out allocator definitionsJeff King, Jul 13, 2014
  15. Junio C HamanoJul 15, 2014
  16. 3/8 move setting of object->type to alloc_* functionsJeff King, Jul 13, 2014
  17. Junio C HamanoJul 15, 2014
  18. 4/8 parse_object_buffer: do not set object typeJeff King, Jul 13, 2014
  19. 5/8 add object_as_type helper for casting objectsJeff King, Jul 13, 2014
  20. 6/8 alloc: factor out commit indexJeff King, Jul 13, 2014
  21. 7/8 object_as_type: set commit indexJeff King, Jul 13, 2014
  22. 8/8 diff-tree: avoid lookup_unknown_objectJeff King, Jul 13, 2014
  23. Ramsay JonesJul 13, 2014
  24. Jeff KingJul 14, 2014
  25. Ramsay JonesJul 14, 2014
  26. Ramsay JonesJul 12, 2014
  27. Jeff KingJul 12, 2014
  28. 3/7 parse_object_buffer: do not set object typeJeff King, Jul 11, 2014
  29. 4/7 add object_as_type helper for casting objectsJeff King, Jul 11, 2014
  30. Ramsay JonesJul 11, 2014
  31. Jeff KingJul 11, 2014
  32. 5/7 alloc: factor out commit indexJeff King, Jul 11, 2014
  33. 6/7 object_as_type: set commit indexJeff King, Jul 11, 2014
  34. 7/7 diff-tree: avoid lookup_unknown_objectJeff King, Jul 11, 2014
  35. Ramsay JonesJul 11, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.