Re: [RFC PATCH] Introduce git-hive

From: Casey Dahlin <cdahlin@redhat.com>
Date: Aug 31, 2010, 15:52 UTC
Message-ID: <20100831155159.GD16034@foucault.redhat.com>
In-Reply-To: <AANLkTinF1o0RZSKYEL9Qc=uwXx6fBBXh6wRx2CTULBSE@mail.gmail.com>

On Tue, Aug 31, 2010 at 04:08:03PM +0100, Luke Kenneth Casson Leighton wrote:

Show 26 quoted lines

> On Tue, Aug 31, 2010 at 3:38 PM, Casey Dahlin <cdahlin@redhat.com> wrote:
> 
> >        nguyen@host_b$ git config --add hive.uri http://myproject.org
> >        nguyen@host_b$ git hive start host_a.com:21121
> >
> > So from host_b we specify host_a's address and listen port, and we join the
> > network. From here on out anyone who also connects to host_a will get host_b's
> > (randomly selected) listen port automatically and be able to connect to it as
> > well.
> >
> > So now our two peers can see each other.
> 
>  ok - this only works if the two peers can see each other's ip
> addresses.  i.e. if the two machines are either on a local subnet or
> if the two machines are directly on the public internet.  ( or if
> you've forced people to set up a firewall rule and/or UPnP rule, but
> even then UPnP doesn't solve the entire problem - only one part of it)
> 
>  ... unless (and i haven't reviewed the code closely, i admit) you're
> using the following protocol:
> 
>  * make tcp connection
>  * send dedicated specific message "please tell me my public IP and port"
>  * far end does sockaddr lookup of the incoming socket
>  * far end returns IP and port as response to requestor
>

Its a bit more primitive right now (a bit broken even) but that's the eventual plan.

Piece you missed though is that the port on the connect end isn't the same as the listen port. Connecting back to it would do no good.

Show 5 quoted lines

> in this way, requestors can determine what the "apparent" IP address
> is as far as having been NAT'd through half a ton of ISP layers
> performing NAT, local routers performing NAT, laptops such as mine
> doing NAT sharing of a 3G connection over a netgear router and so on.
>

You can't get back through all those anyway unless they've all been set up to allow it. I don't know what magic you've seen torrent clients do, but the procedure for the ones I always used was:

1) Check to see if it can receive connections on the port it wants.
2) Bitch at user if it can't.

Bittorrent has the luxury of being able to proxy for the poor firewall-bound users since as long as there's one peer exposed to the internet you can have any two other peers connect to him and give him the data they want to exchange, to the benefit of all 3. Git won't work that way because not everyone in the swarm wants all chunks of data, so if you found a proxy node, you might have to make him carry data (possibly lots of data) that he has no personal interest in.

Show 6 quoted lines

> so, answering the question you were asking earlier: i believe that you
> really do need to consider taking the closest c-based bittorrent
> library/application apart, and use it as the basis for git-hive.  if
> you don't, you will be here forever, reinventing everything that these
> fileshare-app-writers have spent nearly a decade perfecting.
>

The thing avahi was going to provide was bootstrapping. Bittorrent DHT handles this by having a list of known-good peers stashed away somewhere (bittorrent.org hosts one). Essentially a non-p2p solution to joining the p2p network. That's pretty much the only WAN solution. Local networks on the same subnet have the option of UDP multicast. Avahi can do that. I'd be willing to do it manually or with something else. But yes, for your WAN case its useless.

--CJD