ignoring extra bitmap file?

Hi everyone,

I'm seeing the message

   remote: warning: ignoring extra bitmap file: ./objects/pack/pack-2943dc24....pack

and indeed, there is such a thing (two, actually):

   171736188 Aug 17 08:20 pack-2943dc2477026f87b280ebcefa93fe28412688df.idx
    12662268 Aug 17 08:24 pack-2943dc2477026f87b280ebcefa93fe28412688df.bitmap
 12927989355 Aug 17 08:27 pack-2943dc2477026f87b280ebcefa93fe28412688df.pack
   164857412 Aug 17 08:33 pack-8b4a42ca7aa2aca6f354292007910de1110117b2.idx
    13164932 Aug 17 08:49 pack-8b4a42ca7aa2aca6f354292007910de1110117b2.bitmap
      281872 Aug 17 09:40 pack-bddb40f984124ba8c2a4e5c55b0d1b2804fd5817.pack
       13280 Aug 17 09:40 pack-bddb40f984124ba8c2a4e5c55b0d1b2804fd5817.idx
        7904 Aug 17 15:51 pack-0f8b1478e17174c562d9a52cf577e0e050bdb7c5.idx
     2373948 Aug 17 16:09 pack-23253e17510cacaae3bb38fb5429073b3bc59480.pack
        6980 Aug 17 16:09 pack-23253e17510cacaae3bb38fb5429073b3bc59480.idx
      144158 Aug 17 17:03 pack-0f8b1478e17174c562d9a52cf577e0e050bdb7c5.pack
 12927996484 Aug 17 19:19 pack-8b4a42ca7aa2aca6f354292007910de1110117b2.pack
      153332 Aug 17 20:17 pack-65ff13a10c29a6c1604017c50dc9a320044ee605.pack
       14036 Aug 17 20:17 pack-65ff13a10c29a6c1604017c50dc9a320044ee605.idx

But it looks like something went wrong in that repack cycle (that pack-2943dc247702 is the full repo), and it won't get removed later in the next repack in the evening.

Question: Can I safely remove the .bitmap file, and repack will then
clean up the .pack and .idx files as will?

(This is still that repo in bitbucket (latest 4.x) server with git 2.6.2, now with cg.auto=0.)

- Andreas

On Thu, Aug 17, 2017 at 09:24:36PM +0200, Andreas Krey wrote:

Only one is the extra. :) The other is doing something useful.

Basically, the bitmap code was written to a handle a single bitmap file. It would be possible to handle multiple, but it simplified the implementation greatly to only handle one. And in practice, since a bitmap can only be made for a pack which contains all of the reachable objects, you'd have only one bitmap per repo, for the one big "main" pack.

Yes, it looks like you got a full repack that failed to remove the old pack. Or more likely you had two full repacks racing with each other, each creating a new big pack.

So the extra bitmap here is harmless. Both of them contain more or less the same data, and whichever one we use will be fine (and remember that the .bitmap files are purely an optimization, so that "more or less" will only make a minor impact on the speed of operations, not on the output).

Yes, it's always safe to remove a .bitmap file (though if you remove the last one, you may expect performance to drop for some operations).

Whether there's a .bitmap doesn't impact whether .pack and .idx files are deleted. The next full repack would pack everything into a new big pack, and then delete any existing files, including .pack, .idx, and .bitmap.

-Peff

On Fri, 18 Aug 2017 02:53:34 +0000, Jeff King wrote: ...

It took a bit of patience, but the extra packs and .bitmaps finally went away without intervention.

However, now I have a few GB sitting in loose objects that refuse to vanish yet. Bryan might be interested here; how often is bitbucket 4.latest doing prune-objects (with auto-gc disabled)?

Andreas