threads / discuss / 1485

[OT?] git tools at SourceForge ?

Subject: [OT?] git tools at SourceForge ?

## tl;dr

12 messages between Aug 12, 2005 and Aug 13, 2005.

replies: 11people: 7as markdown or json

Wolfgang Denk· Aug 12, 2005, 19:07 UTC · lore

This is somewhat off topic here, so I apologize, but I didn't know any better place to ask:

Has anybody any information if SourceForge is going to provide git / cogito / ... for the projects they host? I asked SF, and they openend a new Feature Request (item #1252867); the message I received sounded as if I was the first person on the planet to ask...

Am I really alone with this?
Best regards,
Wolfgang Denk
-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
There are three things I always forget. Names, faces -  the  third  I
can't remember.                                         - Italo Svevo
Junio C Hamano· Aug 12, 2005, 20:36 UTC · re: Wolfgang Denk · lore

Re: [OT?] git tools at SourceForge ?

Wolfgang Denk <wd@denx.de> writes:
Show 6 quoted lines
> Has anybody any information if SourceForge is going to provide git  /
> cogito / ... for the projects they host? I asked SF, and they openend
> a new Feature Request (item #1252867); the message I received sounded
> as if I was the first person on the planet to ask...
>
> Am I really alone with this?

Earlier Johannes Schindelin sent in a bug report he found on one architecture in their build farm, so apparently he has access to it.

I'd actually welcome people who have access to such a build network, time and of course willingness to start portability fixes on various platforms for git-core.

Daniel Barkalow· Aug 12, 2005, 20:46 UTC · re: Wolfgang Denk · lore

Re: [OT?] git tools at SourceForge ?

On Fri, 12 Aug 2005, Wolfgang Denk wrote:
Show 9 quoted lines
> This is somewhat off topic here, so I apologize, but  I  didn't  know
> any better place to ask:
> 
> Has anybody any information if SourceForge is going to provide git  /
> cogito / ... for the projects they host? I asked SF, and they openend
> a new Feature Request (item #1252867); the message I received sounded
> as if I was the first person on the planet to ask...
> 
> Am I really alone with this?

The git architecture makes the central server less important, and it's easy to run your own. Also, kernel.org is providing space to a set of people with a large overlap with git users, since git hasn't been particularly publicized and kernel.org is hosting git.

	-Daniel
*This .sig left intentionally blank*
Kirby C. Bohling· Aug 12, 2005, 21:06 UTC · re: Daniel Barkalow · lore

Re: [OT?] git tools at SourceForge ?

On Fri, Aug 12, 2005 at 04:46:34PM -0400, Daniel Barkalow wrote:
Show 17 quoted lines
> On Fri, 12 Aug 2005, Wolfgang Denk wrote:
> 
> > This is somewhat off topic here, so I apologize, but  I  didn't  know
> > any better place to ask:
> > 
> > Has anybody any information if SourceForge is going to provide git  /
> > cogito / ... for the projects they host? I asked SF, and they openend
> > a new Feature Request (item #1252867); the message I received sounded
> > as if I was the first person on the planet to ask...
> > 
> > Am I really alone with this?
> 
> The git architecture makes the central server less important, and it's 
> easy to run your own. Also, kernel.org is providing space to a set of 
> people with a large overlap with git users, since git hasn't been 
> particularly publicized and kernel.org is hosting git.
> 

I don't think he wants sourceforge to host git, I think he'd like sourceforge to provide access to source trees via git, instead of cvs. Read that as, I want to do:

git pull http://www.sourceforge.net/packageName
instead of:
cvs -d pserver:www.sourceforge.net co packageName
	While it might be easy to host your own project, Linus has some
infamous quote I'll paraphrase as "Real men don't make backups, they
upload to public FTP and let other's make backups...".
	I know if I was working on OSS project, I wouldn't be too
heartbroken to let someone else run the security, backup, and
general SA duties for me.  More time to write code that way...
    Kirby
Wolfgang Denk· Aug 12, 2005, 22:01 UTC · re: Kirby C. Bohling · lore

Re: [OT?] git tools at SourceForge ?

In message <20050812210611.GF13550@birddog.com> you wrote:
>
> > The git architecture makes the central server less important, and it's 
> > easy to run your own. Also, kernel.org is providing space to a set of 

Yes, cou can - but for a popular project like U-Boot in our case you don't really want to ;-)

Show 6 quoted lines
> > people with a large overlap with git users, since git hasn't been 
> > particularly publicized and kernel.org is hosting git.
> 
> I don't think he wants sourceforge to host git, I think he'd like
> sourceforge to provide access to source trees via git, instead of
> cvs.  Read that as, I want to do:

Correct, that's what I am looking for. My hope is that if enough people ask SF might actually provide such a service.

Best regards,
Wolfgang Denk
-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
Disc space - the final frontier!
Martin Langhoff· Aug 12, 2005, 23:01 UTC · re: Wolfgang Denk · lore

Re: [OT?] git tools at SourceForge ?

Show 6 quoted lines
> > I don't think he wants sourceforge to host git, I think he'd like
> > sourceforge to provide access to source trees via git, instead of
> > cvs.  Read that as, I want to do:
> 
> Correct, that's what I am looking for. My  hope  is  that  if  enough
> people ask SF might actually provide such a service.

Ubuntu's 'launchpad' project is trying to do that (a SCM 'proxy' of sorts) with Arch/Bazaar/BazaarNG/Whatever. It takes massive ammounts of diskpace and computing power to be tracking external cvs/svn repos in your SCM format of choice. The talked abundantly about this at the last Debconf in nightless helsinki.

I know I will be running GIT public repos that mirror CVS repos of a few large-ish projects I work on a lot, and are starting to strain CVS's ability to coordinate work. I am keen on starting a Wiki on 'git/cogito' techniques and usage strategies, and my first entry is going to be about how to track an external project this way.

cheers,
martin
David Lang· Aug 12, 2005, 23:24 UTC · re: Martin Langhoff · lore

Re: [OT?] git tools at SourceForge ?

On Sat, 13 Aug 2005, Martin Langhoff wrote:
Show 12 quoted lines
>>> I don't think he wants sourceforge to host git, I think he'd like
>>> sourceforge to provide access to source trees via git, instead of
>>> cvs.  Read that as, I want to do:
>>
>> Correct, that's what I am looking for. My  hope  is  that  if  enough
>> people ask SF might actually provide such a service.
>
> Ubuntu's 'launchpad' project is trying to do that (a SCM 'proxy' of
> sorts) with Arch/Bazaar/BazaarNG/Whatever. It takes massive ammounts
> of diskpace and computing power to be tracking external cvs/svn repos
> in your SCM format of choice. The talked abundantly about this at the
> last Debconf in nightless helsinki.

Actually, if HPA can (or has) work out the kinks for combining the object stores for different projects I think you will find that a site like sourceforge could actually find that across all their differnt projects git will require significantly less space then first thought. I believe that there is a significant number of files that are going to be the same in different project (for example COPYING, or if a project is forked, most of the files in a project) and therfor will just refer to the same object blob.

now when you start makeing packs this sort of thing can make the optimal logic for deciding what's in each pack 'interesting', but as long as each pack is self contained the rest of the git tools will handle things just fine.

David Lang
-- 
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
  -- C.A.R. Hoare
Linus Torvalds· Aug 12, 2005, 22:27 UTC · re: Daniel Barkalow · lore

Re: [OT?] git tools at SourceForge ?

On Fri, 12 Aug 2005, Daniel Barkalow wrote:
> 
> The git architecture makes the central server less important, and it's 
> easy to run your own.
On the other hand:
 - the git architecture is admirably suited to an _untrusted_ central
   server, ie exactly the SourceForge kind of setup. I realize that the 
   people at SourceForge probably think they are trustworthy, but in the 
   big picture, even SF probably would prefer people to see them as a
   _distribution_ point, not the final authority.
   IOW, with git (unlike, for example CVS), you can have a useful 
   distribution point that is _not_ one that the developers have to 
   control or even necessarily want to control. Which is exactly the
   kind of setup that would match what SF does.
   So with git, developers don't have to trust SF, and if SF is down or 
   something bad happens (disk crash, bad backups, whatever), you didn't 
   "lose" anything - the real development wasn't being done at SF anyway, 
   it was a way to _connect_ the people who do real development.
 - Every developer wants to have their own history and complete source
   control, but very few developers actually have good distribution 
   resources. "kernel.org" works for a few projects, and might be fine to
   expand a bit past what it does now, but kernel.org doesn't eevn try to
   do (nor _want_ to do, I bet) the kinds of things that SF does.
   Yes, developers can just merge with each other directly, and git allows 
   that, but it's actually not very convenient - not because of git
   itself, but because of just doing the maintenance. For example, I don't 
   allow incoming traffic to my machines, and I feel _much_ better that
   way. No MIS, no maintenance, and much fewer security issues.
   This is _exactly_ where something like SF really ends up being helpful. 
   It's a _hosting_ service, and git is eminently suitable to being 
   hosted, exactly because of its distributed approach. It needs very few 
   hosting services: you could make do with a very very limited shell 
   access, and in fact I tried to design the "git push" protocol so that 
   you could give people ssh "shell" access, where the "shell" was not a 
   real shell at all, but something that literally just implemented four
   or five commands ("git-receive-pack" and some admin commands to do 
   things like creation/removal of whole archives etc).
> Also, kernel.org is providing space to a set of people with a large
> overlap with git users, since git hasn't been particularly publicized
> and kernel.org is hosting git.

kernel.org certainly works well enough for the few projects that use it, but I don't think it's going to expand all that much.

And it's possible that git usage won't expand all that much either. But quite frankly, I think git is a lot better than CVS (or even SVN) by now, and I wouldn't be surprised if it started getting some use outside of the git-only and kernel projects once people start getting more used to it. And so I'd be thrilled to have some site like SF support it.

bkbits.net used to do that for BK projects, and there were a _lot_ of projects that used it.

		Linus
Daniel Barkalow· Aug 12, 2005, 23:14 UTC · re: Linus Torvalds · lore

Re: [OT?] git tools at SourceForge ?

On Fri, 12 Aug 2005, Linus Torvalds wrote:
Show 5 quoted lines
> And it's possible that git usage won't expand all that much either. But
> quite frankly, I think git is a lot better than CVS (or even SVN) by now,
> and I wouldn't be surprised if it started getting some use outside of the
> git-only and kernel projects once people start getting more used to it. 
> And so I'd be thrilled to have some site like SF support it.

I certainly think it's going to happen; it's just not surprising that it hasn't happened yet. Once there's a stable release and some publicity, I'd expect SF to see it as worthwhile. But a hosting site with git-only shell access needs to know what the necessary programs are going to be, which we haven't committed to quite yet.

	-Daniel
*This .sig left intentionally blank*
Martin Langhoff· Aug 12, 2005, 23:17 UTC · re: Linus Torvalds · lore

Re: [OT?] git tools at SourceForge ?

>  - the git architecture is admirably suited to an _untrusted_ central
>    server, ie exactly the SourceForge kind of setup. I realize that the

Definitely. And beyond that too. Using SF for CVS means that when SF's CVS service is down (often enough) you can't commit (or even fscking diff) until they are back up. Every single damn operation does a roundtrip. This also means a huge load on their servers.

I'm sure SF will be glad to see CVS fall our of favour.
>    Yes, developers can just merge with each other directly

I take it that you mean an exchange of patches that does not depend on having public repos. What are the mechanisms available on that front, other than patchbombs?

>    This is _exactly_ where something like SF really ends up being helpful.
>    It's a _hosting_ service, and git is eminently suitable to being

Not sure whether SF is offering rsync, but they do support hosting of arbitrarty data -- and a project using GIT can use that to host several developer trees . It'd be nice if SF offered gitweb and similar niceties. As my usage of GIT increases, I may add support for it on Eduforge.org

If I had more (hw/time) resources I'd do the git proxying of CVS projects, but that's huge.

> And so I'd be thrilled to have some site like SF support it.

Eduforge's charter is to host education-related projects, so that's not a free-for-all-comers, but I'm considering git support, as our usage of git is growing.

cheers,
martin
Linus Torvalds· Aug 12, 2005, 23:46 UTC · re: Martin Langhoff · lore

Re: [OT?] git tools at SourceForge ?

On Sat, 13 Aug 2005, Martin Langhoff wrote:
Show 6 quoted lines
> 
> >    Yes, developers can just merge with each other directly
> 
> I take it that you mean an exchange of patches that does not depend on
> having public repos. What are the mechanisms available on that front,
> other than patchbombs?
Just have a shared trusted machine.

A lot of "core" developers end up having machines that they may not control, and that they may not be able to use as major distribution points, but that they _can_ share with others.

For example, "master.kernel.org" ends up being that for the kernel: you don't have to have an account on master, but most of the core developers do, so they can use it as a short-cut that is independent of the actual "public" site.

Similarly, some people are perfectly willing to give other trusted developers a ssh login on their machine - and that's a perfectly fine way to sync repositories directly if you have even a slow DSL link. You'd never want to _distribute_ the result over DSL, though.

The point being that you can certainly sync up to others without going through a public site.

[ We _could_ also just send pack-files as email attachments. There's
  nothing fundamentally wrong with doing the object discovery that
  "git-send-pack" does on its own manually over email.
  In other words: you could easily do something like "Hey, I've got your
  commit as of yesterday, ID <sha1>, can you send me your current
  top-of-tree SHA1 name and the pack-file between the two?" and have 
  direct git-to-git synchronization even over email.
  NOTE NOTE NOTE! BK did this, with a "bk send" and "bk receive". I hated 
  it, which is why I'd never do scripts like that. But I think it's a 
  valid thing to do when you're cursing the fact that the central
  repository is down, and has been down for five hours, and you don't know
  how long it will take to get back up, and you don't have _any_ other
  choices ]
Show 6 quoted lines
> >    This is _exactly_ where something like SF really ends up being helpful.
> >    It's a _hosting_ service, and git is eminently suitable to being
> 
> Not sure whether SF is offering rsync, but they do support hosting of
> arbitrarty data -- and a project using GIT can use that to host
> several developer trees.

The problem with the arbitrary data approach (and rsync) is that the git repositories can get out of sync.

We haven't seen it very often on kernel.org, but we _do_ see it. I think I've got something like three bug reports from people saying "your tree is corrupted" because it so happened that the mirroring was on-going at the same time I did a push, and the mirroring caught an updated HEAD without actually having caught all of the objects that HEAD referenced.

Now, all the git tools do write things in the right order, and mirror scripts etc _tend_ to mirror in alphabetical order (and "objects" come before "refs" ;), so you really have to hit the right window where a git tool updates the git repository at the same time as a mirroring sweep is going on, but it definitely _does_ happen.

It just happens seldom enough that most people haven't noticed. But if you've seen the gitweb page go blank for one of the projects, you now know why it can happen..

And this is inevitable when you have a non-git-aware server. You really need to update the objects in the right order, and to get the right order you do have to be git-aware.

> It'd be nice if SF offered gitweb and
> similar niceties. As my usage of GIT increases, I may add support for
> it on Eduforge.org

I think we'll find that it's a learning process, to just find out what drives site managers mad (we certainly found the problem with lots of small files on kernel.org out ;). Having a few sites that do it and tell the others what gotchas there are involved with it (and what scripts they use for maintaining stuff like auto-packing etc) is all a learning experience.

			Linus
Daniel Barkalow· Aug 13, 2005, 01:16 UTC · re: Martin Langhoff · lore

Re: [OT?] git tools at SourceForge ?

On Sat, 13 Aug 2005, Martin Langhoff wrote:
Show 5 quoted lines
> >    Yes, developers can just merge with each other directly
> 
> I take it that you mean an exchange of patches that does not depend on
> having public repos. What are the mechanisms available on that front,
> other than patchbombs?

If each developer has a trivial web server, they can put their output there, and everyone else can pull from it, because it only needs to serve static files out of a directory structure that the programs create regularly. Of course, this is only strictly different from a public repo in that you don't advertize it beyond the other developers. But it's a within-system equivalent to posting a link to a web-hosted patch set, which people sometimes do to pass things around.

Show 5 quoted lines
> > And so I'd be thrilled to have some site like SF support it.
> 
> Eduforge's charter is to host education-related projects, so that's
> not a free-for-all-comers, but I'm considering git support, as our
> usage of git is growing.

If you contribe the git support to gforge, presumably similar hosting sites will pick it up before too long.

	-Daniel
*This .sig left intentionally blank*

← back to recent threads