Re: How to determine when to stop receiving pack content
On 11/08/19 11:31PM, Farhan Khan wrote:
Show 47 quoted lines
> August 11, 2019 11:04 AM, "Pratyush Yadav" <me@yadavpratyush.com> wrote:
>
> > On 10/08/19 11:47PM, Farhan Khan wrote:
> >
> >> Hi,
> >>
> >> I am trying to write an implementation of git clone over ssh and am a little confused how to
> >> determine a server response has ended. Specifically, after a client sends its requested 'want', the
> >> server sends the pack content over. However, how does the client know to stop reading data? If I
> >> run a simple read() of the file descriptor:
> >>
> >> A. If I use reading blocking, the client will wait until new data is available, potentially
> >> forever.
> >> B. If I use non-blocking, the client might terminate reading for new data, when in reality new data
> >> is in transit.
> >>
> >> I do not see a mechanism to specify the size or to indicate the end of the pack content. Am I
> >> missing something?
> >
> > Well, I am not very familiar with git-clone internals, but I did some
> > digging around, and I think I know what answer to your problem is.
> >
> > Looking at Documentation/technical/protocol-v2.txt:34, the flush packet
> > indicates the end of a message. Looking in the output section of the
> > fetch command (protocol-v2.txt:342), it sends you some optional
> > sections, and then the packfile and then sends a flush packet.
> >
> > So your read should stop reading data when it sees the flush packet.
> >
> > Another way would be to look at the packfile contents. Looking at
> > Documentation/technical/pack-format.txt, the packfile contains the
> > number of objects in the packfile, and each object entry has the object
> > size. So you can stop reading after you have received the last object in
> > the packfile (plus the 20-byte trailer).
> >
> > I don't know which is the better way, but the former seems like a better
> > choice to me.
> >
> > --
> > Regards,
> > Pratyush Yadav
>
> Hi Pratyush,
>
> Thanks for your reply!
>
> Unless I am mistaken, a pack file does not end in a flush_pkt. I ran some tests and did not see the stream end in "0000". Is there is a mistake somewhere on my end?
Hm, turns out I was on the pu branch, not master when I looked at
protocol-v2.txt. The file was updated about 3 days ago (not in master
yet) (7ee4ab7e8c3310fc28d9dd7d47da26e497e73556), where it seems to imply
that flush-pkt will be sent after the packfile (see excerpt below).
--- protocol-v2.txt ---
output = acknowledgements flush-pkt |
[acknowledgments delim-pkt] [shallow-info delim-pkt]
[wanted-refs delim-pkt] [packfile-uris delim-pkt]
packfile flush-pkt
---So either something changed in the protocol with that merge, or there is a discrepancy in the documentation, because the above output format seems to imply the packfile will be followed by a flush packet. I haven't looked at the full contents of the merge, but none of the commit messages mention changing this behaviour.
Either way, you can probably parse the packfile to know how many objects you will get, and stop after the last object. Or like Junio said, just wait for an EOF.
Sorry for the wrong information.
--
Regards,
Pratyush Yadav
Farhan Khan
