{"thread":{"id":"39490","subject":"git torrent - sane deterministic pack files","startedAt":"2015-06-02T05:07:03Z","lastAt":"2015-06-02T05:07:03Z","messageCount":1,"participants":["Zenaan Harkness"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"262683","messageId":"CAOsGNSSK6WwVvftc1Wx14gRyZ7Mx2qnXsDxducCCA0VBvqyK8w@mail.gmail.com","threadId":"39490","inReplyTo":null,"subject":"git torrent - sane deterministic pack files","fromName":"Zenaan Harkness","fromEmail":"zen@freedbms.net","sentAt":"2015-06-02T05:07:03Z","receivedAt":"2015-06-02T05:07:03Z","isPatch":false,"sender":{"key":"zen@freedbms.net","avatar":null},"body":"Please CC me when replying, if you think of it. Thanks.\n\nLWN discussion, this particular thread starts here:\nhttps://lwn.net/Articles/646758/\n\nSome extracts:\nDeterministic packfile creation is required for parallel git downloads.\n\nExample git pack file parameter/ configuration variations:\n- compression on X number of CPUs\n- maximum packfile size\n- more trees/ branches in this repo than on that repo\n\nA way is needed to capture these packfile config variations and\ndistribute them to other git servers (perhaps on a standardized branch\nname or ??).\n---\nRelated mail from 4 years ago:\nhttp://article.gmane.org/gmane.comp.version-control.git/164643\n\nConceptually it may not be hard, but implementation is hard. By\nforcing certain object layout rules, you may have lower compression\nratio, or slower pack access, and may consume more power. Git tries to\nreuse deltas from existing packs to produce a new pack. This makes it\nquick to assemble a pack, but also underterministic. There's also\nthreads stealing jobs from one another in the above link. Resumable\nclone is a frequent request, and we still don't have it now.\n---\nDeterministic pack file parameter sets and compression can always be\ntuned over time even though they change the format - that's just local\npolicy for the authoritative git torrent server.\n\nAlso for scenarios which benefit from pack file torrents, the marginal\nreduction in compression (increase in pack file size) due to the need\nfor determinism may very well be valuable (marginal increase in local\nstorage in order to distribute downloads) - local policy strikes\nagain.\n...\nIt's up to the \"authoritative git server\" admin to make the policy\ndecision of how long to keep with a current deterministic torrentable\npack file parameter set, and when to update to a new/tuned set. This\nis always a local policy matter! \"We can't do that because it's not\nthe best policy for maximum compression\" is not the right answer\nhere...\n...\nIf I am a torrent repo mirror, the \"authoritative torrent upstream\" is\nmerely a local config.\n\nThis not only sounds easy, it is easy - even in the face of\ncompression technology changes and \"tuning\" over time - that's merely\na \"version\" increase or new parameter set provided by the\n\"authoritative\" repo server, and is local policy to that server.\n\nRegards\nZenaan\n"}]}