git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Resumable clone/Gittorrent (again)

From
LLLuke Kenneth Casson Leighton <luke.leighton@gmail.com>
Date
Jan 8, 2011, 17:21 UTC
Message-ID
<AANLkTim2A4=y=XcuPuPiYGDGZyKAUEk-yv2cZVEGhQ3C@mail.gmail.com>
In-Reply-To
<AANLkTimgn2_BWYjbGKbGoeGJ=erKundX4umfy=s16dB1@mail.gmail.com>
On Sat, Jan 8, 2011 at 2:17 AM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
Show 13 quoted lines
> On Fri, Jan 7, 2011 at 10:59 PM, Luke Kenneth Casson Leighton
> <luke.leighton@gmail.com> wrote:
>>  bottom line: my take on this is (sorry to say, nguyen) that i don't
>> believe bittorrent "pieces" map well to the chains concept, unless the
>> chains are perfectly fixed identical sizes [which they could well be?
>> am i mistaken about this?]
>
> there are a few characteristics of bittorrent pieces that i see:
> verifiable, resumable, uniquely identifiable across peers and
> reasonbly small in count.
>
> The fixed size helps peers uniquely identify any pieces by splitting
> the whole transfer equally and indexing them in 1-dimension.
 ok - you haven't answered the question: are the chains perfectly
fixed identical sizes?
 if so they can be slotted into the bittorrent protocol by simply
pre-selecting the size to match.  with the downside that if there are
a million such "chains" you now pretty much overwhelm the peers with
the amount of processing, network traffic and memory requirements to
maintain the "pieces" map.
 if not then you now need to modify the bittorrent protocol to cope
with variable-length block sizes: the protocol only allows for the
last block to be of variable-length.
 also, it's worth pointing out that the entire code-base of every
single bittorrent client that you will ever be able to find revolves
around the concept of reassembly of files from "pieces".
 bottom line: the bittorrent protocol and the available bittorrent
source code libraries, both of which save you a great deal of time in
getting something up-and-running, is _not_ the right fit for the
concept of placing the proposed "chains" into bittorrent "pieces".
 translation: if you wish to pursue the "chains" concept, either a
heavily-modified bittorrent protocol and client _or_ an entirely new
peer-to-peer protocol is far more appropriate.
 orrr, doing what i initially suggested, which is to leave the
bittorrent protocol as-is and to open one .torrent per "chain".
especially if these "chains" vary considerably in size (k to gb)
l.
Previous: Nguyen Thai Ngoc DuyNext: Nguyen Thai Ngoc Duy
Message 17 of 25 in “Resumable clone/Gittorrent (again)”
  1. Nguyen Thai Ngoc DuyJan 5, 2011
  2. Luke Kenneth Casson LeightonJan 5, 2011
  3. Thomas RastJan 5, 2011
  4. Luke Kenneth Casson LeightonJan 5, 2011
  5. Nguyen Thai Ngoc DuyJan 6, 2011
  6. Luke Kenneth Casson LeightonJan 6, 2011
  7. MaaartinJan 5, 2011
  8. Nguyen Thai Ngoc DuyJan 6, 2011
  9. Maaartin-1Jan 6, 2011
  10. Nguyen Thai Ngoc DuyJan 6, 2011
  11. Maaartin-1Jan 8, 2011
  12. Nguyen Thai Ngoc DuyJan 8, 2011
  13. Nicolas PitreJan 7, 2011
  14. Nguyen Thai Ngoc DuyJan 7, 2011
  15. Luke Kenneth Casson LeightonJan 7, 2011
  16. Nguyen Thai Ngoc DuyJan 8, 2011
  17. Luke Kenneth Casson LeightonJan 8, 2011
  18. Nguyen Thai Ngoc DuyJan 9, 2011
  19. Luke Kenneth Casson LeightonJan 9, 2011
  20. Nguyen Thai Ngoc DuyJan 9, 2011
  21. Luke Kenneth Casson LeightonJan 13, 2011
  22. Sam VilainJan 13, 2011
  23. Luke Kenneth Casson LeightonJan 14, 2011
  24. Sam VilainJan 16, 2011
  25. Sam VilainJan 10, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.