On shipping more of our technical docs as manpages

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Sep 26, 2018, 20:44 UTC
Message-ID: <878t3oj8em.fsf@evledraar.gmail.com>
In-Reply-To: <20180926185812.GD30680@sigill.intra.peff.net>

[Changing subject because this stopped being about Stefan's patches a while ago]

On Wed, Sep 26 2018, Jeff King wrote:

Show 6 quoted lines

> On Wed, Sep 26, 2018 at 08:43:14PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> While we're on that subject, I'd much prefer if these technical docs and
>> other asciidoc-ish stuff we would be manpages build as part of our
>> normal "make doc" target. So e.g. this one would be readable via
>> something like "man 3 gitapi-oid-array".

[responding out of order]

Show 13 quoted lines

> Now I'll admit it seems like make-work to me because I would not plan to
> ever look at the formatted output myself. But I guess I don't understand
> the audience for this formatted output. These are APIs internal to Git
> itself. We would not generally want to install gitapi-oid-array into
> /usr/share/man, because only people actually working on Git would be
> able to use it. So it sounds like a convenience for a handful of
> developers (who like to look at this manpage versus the source). It
> doesn't seem like the cost/benefit is there.
>
> And if we were going to generate something external, would it make more
> sense to write in a structured format like doxygen? I am not a big fan
> of it myself, but at least from there you can generate a more richly
> interconnected set of documentation.

It's useful to have a single authoritative source for all documentation that's easy to search through.

I don't really care where that text actually lives, i.e. whether it's all in *.txt to begin with, or in *.c or *.h and pulled together by some "make" step, or much how it's formatted, beyond it being easy to work with. I'm not a big fan of asciidoc, but it's what we've got if we want formatting, inter-linking etc.

My bias here is that I've also contributed actively to the perl project in the past, and with that project you can get an overview of *all* of the docs by typing:

    man perl

That includes stuff like perl585delta(1) which we'd stick in Documentation/RelNotes, and "Internals and C Language Interface". Most of what we'd put in Documentation/technical/api-* & headers is in perlapi(1).

I spent an embarrassingly long time submitting patches to git before discovering that Documentation/technical/api-*.txt even existed, I think something like 1-2 years ago.

We have very well documented stuff like strbuf.h (mostly thanks to you), but how is such documentation supposed to be discovered? Something like:

    git grep -A20 '^/\*$' -- *.h
    <hold in page down>

In terms of getting an overview it's indistinguishable from comments. I.e. there's nothing like an index of:

    man git-api-strbuf     ==> working with strings
    man git-api-sha1-array ==> list, iterate and binary lookup SHA1s

etc.

Sometimes it's obvious where to look, but as someone who uses most of these APIs very occasionally (because I contribute less) I keep (re-)discovering various internal APIs.

I'm also not in the habit of opening the *.h file for something, I usually just start reading the *.c and only later discover there's some API docs in the relevant *.h (or not).

So in terms of implementation I'm a big fan of the perl.git approach of having these docs as comments before the function implementation in the *.c file.

It means you can't avoid seeing it or updating it when source spelunking, and it leaves header files short, which is useful when you'd like to get a general API overview without all the docs. Our documented headers are quite fat, e.g. strbuf.h is 60% of the size of strbuf.c, but less than 20% when you strip the docs.

Show 7 quoted lines

> I'm mildly negative on this, just because it introduces extra work on
> people writing the documentation. Now it has to follow special
> formatting rules, and sometimes the source is uglier (e.g., the horrible
> +-continuation in lists). Are authors now responsible for formatting any
> changes they make to make sure they look good in asciidoc? Or are people
> who care about the formatted output going to come along afterwards and
> submit fixup patches? Either way it seems like make-work.

This part I'm slightly confused by. If you're just saying "let's document stuff in *.h files in free-flowing text", fine. But if we're talking about the difference between Documentation/technical/api-*.txt and having the same stuff in manpages, then the stuff in api-*.txt *is* already asciidoc, and we have a target to format it all up and ship it with git, and distros ship that.

E.g. on Debian you can "apt install git-doc" and browse stuff like file:///usr/share/doc/git-doc/technical/api-argv-array.html which is the HTML formatted version, same for all the other api-*.txt docs.

We just don't have some central index of those, and they're not a default target which means the likes of our own https://git-scm.com don't build them, so they're harder to find.

Every perl installation also ships perlapi and a MB or so of obscure internal docs to everyone, which makes it easier to find via Google et al, which I find useful. So I guess I'm more on the side fence of dropping a hypothetical gitapi-oid-array into /usr/share/man than you are.

ANYWAY

This E-mail is much longer than I intended, sorry about that. I didn't just mean to bitch about how we ship docs, but I was wondering if there was a desire to move to something like what I've outlined above, or whether the status quo was mostly by design and intended.

I just thought I'd write this rather lengthy E-Mails because this is one of the rare areas where you can fully describe an idea in E-Mail without writing any patches.

If the consensus is that something like the exhaustive index "perldoc perl" provides wouldn't be useful for git I can just drop this, but if people are enthusiastic about having that it would be useful to work on it...