threads / discuss / 45441

GSoC Project | Convert interactive rebase to C

Subject: GSoC Project | Convert interactive rebase to C

## tl;dr

9 messages between Mar 20, 2017 and Mar 28, 2017.

replies: 8people: 4as markdown or json

Ivan Tham· Mar 20, 2017, 16:41 UTC · lore
Hi everyone,

I am Ivan Tham. Currently studying in Computer Science in APIIT Malaysia. I am interested particapate in Google Summer of Code 2017 under git organization. I would like to attempt "Add more builtin patterns for userdiff" particularly for shell for my microproject.

I am interested to work on "Convert interactive rebase to C" aiming to port most builtins stuff to C in which we can reduce the size of git. Additionally, I would also like to convert scripts to builtins as an additional milestone.

What do you think of these projects? Would it collide with [Valery Tolstov's Shell to Builtins proposal][0]?

[0]: https://public-inbox.org/git/1489145258.10535.0@smtp.yandex.ru/
Stefan Beller· Mar 20, 2017, 17:41 UTC · re: Ivan Tham · lore

Re: GSoC Project | Convert interactive rebase to C

On Mon, Mar 20, 2017 at 9:41 AM, Ivan Tham <pickfire@riseup.net> wrote:
Show 6 quoted lines
> Hi everyone,
>
> I am Ivan Tham. Currently studying in Computer Science in APIIT Malaysia. I am
> interested particapate in Google Summer of Code 2017 under git organization. I
> would like to attempt "Add more builtin patterns for userdiff" particularly for
> shell for my microproject.

I'd love to see proper shell support! Although there is already some support for shell (by looking at diffs on our test suite) ? So I am not sure what there is left to do? Can you clarify what you're trying there?

> I am interested to work on "Convert interactive rebase to C"
+cc Johannes, who recently worked on rebase and the sequencer.
Show 6 quoted lines
>  aiming to port
> most builtins stuff to C in which we can reduce the size of git. Additionally,
> I would also like to convert scripts to builtins as an additional milestone.
>
> What do you think of these projects? Would it collide with
> [Valery Tolstov's Shell to Builtins proposal][0]?

Curious why all people ask about colliding with Valerys proposal here? I do not think it would collide, as submodules and rebase are very different areas of the code base.

Thanks, Stefan

>
> [0]: https://public-inbox.org/git/1489145258.10535.0@smtp.yandex.ru/
>
Ivan Tham· Mar 21, 2017, 06:05 UTC · re: Stefan Beller · lore

Re: Re: GSoC Project | Convert interactive rebase to C

Stefan Beller <sbeller@google.com> wrote:
Show 13 quoted lines
> On Mon, Mar 20, 2017 at 9:41 AM, Ivan Tham <pickfire@riseup.net> wrote:
> > I am Ivan Tham. Currently studying in Computer Science in APIIT Malaysia. I am
> > interested particapate in Google Summer of Code 2017 under git organization. I
> > would like to attempt "Add more builtin patterns for userdiff" particularly for
> > shell for my microproject.
>
> I'd love to see proper shell support!
> Although there is already some support for shell (by looking at diffs
> on our test
> suite) ? So I am not sure what there is left to do? Can you clarify what you're
> trying there?
>
> > I am interested to work on "Convert interactive rebase to C"

Are you sure about that? From what I had looked into userdiff.c, there is no support for shell. There just a recent patch for [go patterns][0]. Or perhaps I should have rename it as "userdiff.c: patterns for "shell" language"?

Show 12 quoted lines
> +cc Johannes, who recently worked on rebase and the sequencer.
>
> >  aiming to port
> > most builtins stuff to C in which we can reduce the size of git. Additionally,
> > I would also like to convert scripts to builtins as an additional milestone.
> >
> > What do you think of these projects? Would it collide with
> > Valery Tolstov's Shell to Builtins proposal?
>
> Curious why all people ask about colliding with Valerys proposal here?
> I do not think it would collide, as submodules and rebase are very different
> areas of the code base.
[0]: https://public-inbox.org/git/xmqqk27jvg28.fsf@gitster.mtv.corp.google.com/
Johannes Schindelin· Mar 23, 2017, 17:30 UTC · re: Ivan Tham · lore

Re: Re: GSoC Project | Convert interactive rebase to C

Hi Ivan,
On Tue, 21 Mar 2017, Ivan Tham wrote:
Show 16 quoted lines
> Stefan Beller <sbeller@google.com> wrote:
> > On Mon, Mar 20, 2017 at 9:41 AM, Ivan Tham <pickfire@riseup.net> wrote:
> > > I am Ivan Tham. Currently studying in Computer Science in APIIT
> > > Malaysia. I am interested particapate in Google Summer of Code 2017
> > > under git organization. I would like to attempt "Add more builtin
> > > patterns for userdiff" particularly for shell for my microproject.
> >
> > I'd love to see proper shell support!  Although there is already some
> > support for shell (by looking at diffs on our test suite) ? So I am
> > not sure what there is left to do? Can you clarify what you're trying
> > there?
> 
> Are you sure about that? From what I had looked into userdiff.c, there
> is no support for shell. There just a recent patch for [go patterns][0].
> Or perhaps I should have rename it as "userdiff.c: patterns for "shell"
> language"?
I also could not find any shell patterns in the userdiff code...
> > > I am interested to work on "Convert interactive rebase to C"
> >
> > +cc Johannes, who recently worked on rebase and the sequencer.

Glad you are interested! Please note that large parts of the interactive rebase are already in C now, but there is enough work left in that corner.

> > > aiming to port most builtins stuff to C in which we can reduce the
> > > size of git. Additionally, I would also like to convert scripts to
> > > builtins as an additional milestone.

Careful. It is a ton of work to get the rebase -i conversion done, and then a ton of work to get it integrated. That will fill 3 months, very easily.

> > > What do you think of these projects? Would it collide with Valery
> > > Tolstov's Shell to Builtins proposal?

I missed that proposal, and could only find submodule-related mails on the public-inbox server. Care to provide a pointer?

> > Curious why all people ask about colliding with Valerys proposal here?
> > I do not think it would collide, as submodules and rebase are very
> > different areas of the code base.
Indeed ;-)

Ciao, Johannes

Stefan Beller· Mar 23, 2017, 18:19 UTC · re: Johannes Schindelin · lore

Re: Re: GSoC Project | Convert interactive rebase to C

On Thu, Mar 23, 2017 at 10:30 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 6 quoted lines
>> Are you sure about that? From what I had looked into userdiff.c, there
>> is no support for shell. There just a recent patch for [go patterns][0].
>> Or perhaps I should have rename it as "userdiff.c: patterns for "shell"
>> language"?
>
> I also could not find any shell patterns in the userdiff code...

Sorry for the confusion on my part. I am unfamiliar with that part of the diffing lib. I based my assumption on the output of "$ git log -p t/" which contains hunk headers, that make sense most of the time for shell code.

Show 10 quoted lines
>
> Careful. It is a ton of work to get the rebase -i conversion done, and
> then a ton of work to get it integrated. That will fill 3 months, very
> easily.
>
>> > > What do you think of these projects? Would it collide with Valery
>> > > Tolstov's Shell to Builtins proposal?
>
> I missed that proposal, and could only find submodule-related mails on the
> public-inbox server. Care to provide a pointer?

Well I don't think it is a full grown proposal, but a confident attempt to find a good starting point :) e.g. https://public-inbox.org/git/1489145258.10535.0@smtp.yandex.ru/

Show 5 quoted lines
>> > Curious why all people ask about colliding with Valerys proposal here?
>> > I do not think it would collide, as submodules and rebase are very
>> > different areas of the code base.
>
> Indeed ;-)

Although I had some discussion on what the correct behavior of "rebase --recurse-submodules" in a superproject should be. But this sounds like future work after the GSoC is over. ;)

Thanks, Stefan

Inaw Tham· Mar 25, 2017, 02:17 UTC · re: Johannes Schindelin · lore

Re: Re: Re: GSoC Project | Convert interactive rebase to C

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
Show 19 quoted lines
> On Tue, 21 Mar 2017, Ivan Tham wrote:
> > Stefan Beller <sbeller@google.com> wrote:
> > > On Mon, Mar 20, 2017 at 9:41 AM, Ivan Tham <pickfire@riseup.net> wrote:
> > > > I am Ivan Tham. Currently studying in Computer Science in APIIT
> > > > Malaysia. I am interested particapate in Google Summer of Code 2017
> > > > under git organization. I would like to attempt "Add more builtin
> > > > patterns for userdiff" particularly for shell for my microproject.
> > >
> > > I'd love to see proper shell support!  Although there is already some
> > > support for shell (by looking at diffs on our test suite) ? So I am
> > > not sure what there is left to do? Can you clarify what you're trying
> > > there?
> > 
> > Are you sure about that? From what I had looked into userdiff.c, there
> > is no support for shell. There just a recent patch for [go patterns][0].
> > Or perhaps I should have rename it as "userdiff.c: patterns for "shell"
> > language"?
> 
> I also could not find any shell patterns in the userdiff code...
Thanks a lot for replying me, I thought no one was intereted. :D
Show 6 quoted lines
> > > > I am interested to work on "Convert interactive rebase to C"
> > >
> > > +cc Johannes, who recently worked on rebase and the sequencer.
> 
> Glad you are interested! Please note that large parts of the interactive
> rebase are already in C now, but there is enough work left in that corner.
Glad to hear that, I would really like to see interactive rebase in C.
Show 7 quoted lines
> > > > aiming to port most builtins stuff to C in which we can reduce the
> > > > size of git. Additionally, I would also like to convert scripts to
> > > > builtins as an additional milestone.
> 
> Careful. It is a ton of work to get the rebase -i conversion done, and
> then a ton of work to get it integrated. That will fill 3 months, very
> easily.

My main aim is to reduce the extra dependency of perl, but planning to start with rebase, can I make that an optional task where I can help out after I had completed my main task during gsoc?

Show 11 quoted lines
> > > > What do you think of these projects? Would it collide with Valery
> > > > Tolstov's Shell to Builtins proposal?
> 
> I missed that proposal, and could only find submodule-related mails on the
> public-inbox server. Care to provide a pointer?
> 
> > > Curious why all people ask about colliding with Valerys proposal here?
> > > I do not think it would collide, as submodules and rebase are very
> > > different areas of the code base.
> 
> Indeed ;-)

Cheers, Ivan Tham

Johannes Schindelin· Mar 27, 2017, 15:11 UTC · re: Inaw Tham · lore

Re: Re: Re: GSoC Project | Convert interactive rebase to C

Hi Ivan,
On Sat, 25 Mar 2017, Inaw Tham wrote:
Show 14 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > On Tue, 21 Mar 2017, Ivan Tham wrote:
> > > Stefan Beller <sbeller@google.com> wrote:
> > > > On Mon, Mar 20, 2017 at 9:41 AM, Ivan Tham <pickfire@riseup.net> wrote:
> > > >
> > > > > I am interested to work on "Convert interactive rebase to C"
> > > >
> > > > +cc Johannes, who recently worked on rebase and the sequencer.
> > 
> > Glad you are interested! Please note that large parts of the
> > interactive rebase are already in C now, but there is enough work left
> > in that corner.
> 
> Glad to hear that, I would really like to see interactive rebase in C.

Please note that a notable part already made it into C in v2.12.1. There are still a few loose ends to tie, of course; it still makes for a great head start on your project, methinks.

Show 11 quoted lines
> > > > > aiming to port most builtins stuff to C in which we can reduce
> > > > > the size of git. Additionally, I would also like to convert
> > > > > scripts to builtins as an additional milestone.
> > 
> > Careful. It is a ton of work to get the rebase -i conversion done, and
> > then a ton of work to get it integrated. That will fill 3 months, very
> > easily.
> 
> My main aim is to reduce the extra dependency of perl, but planning to
> start with rebase, can I make that an optional task where I can help out
> after I had completed my main task during gsoc?

Sure, you can make it an optional task, and I would be very happy if you followed up on it even after GSoC!

As far as the Perl dependency is concerned, I actually think there is only one serious one left: git add -i.

Granted, there is send-email, but it really does not matter all that much these days *except* if you want to use Git to contribute to projects that still use a mailing list-based patch submission process (the ones that come to mind are: Git, Linux and Cygwin). Most Git users actually do not submit any patches to mailing lists, therefore I tend to ignore this one.

The rest of the Perl scripts interacts with foreign SCMs (archimport, cvsexportcommit, cvsimport, cvsserver, and svn). I *guess* that it would be nice to follow up on the remote-svn work (which has not really gone anywhere so far, AFAICT the main driving contributor pursues different projects these days), but IMHO none of these are really needed to run Git.

Ciao, Johannes

Pickfire· Mar 27, 2017, 16:31 UTC · re: Johannes Schindelin · lore

Re: Re: Re: Re: GSoC Project | Convert interactive rebase to C

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
Show 20 quoted lines
> On Sat, 25 Mar 2017, Ivan Tham wrote:
>
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > > On Tue, 21 Mar 2017, Ivan Tham wrote:
> > > > Stefan Beller <sbeller@google.com> wrote:
> > > > > On Mon, Mar 20, 2017 at 9:41 AM, Ivan Tham <pickfire@riseup.net> wrote:
> > > > >
> > > > > > I am interested to work on "Convert interactive rebase to C"
> > > > >
> > > > > +cc Johannes, who recently worked on rebase and the sequencer.
> > >
> > > Glad you are interested! Please note that large parts of the
> > > interactive rebase are already in C now, but there is enough work left
> > > in that corner.
> >
> > Glad to hear that, I would really like to see interactive rebase in C.
>
> Please note that a notable part already made it into C in v2.12.1. There
> are still a few loose ends to tie, of course; it still makes for a great
> head start on your project, methinks.
Ah, that's great.

And while I was working on the microproject (shell patterns in user diff), I can't produce the output of t/t4034-diff-words.sh manually with:

    mkdir cpp/ && cd cpp/ && git init
    cat > pre <<EOF
    Foo():x(0&&1){}
    cout<<"Hello World!\n"<<endl;
    1 -1e10 0xabcdef 'x'
    [a] a->b a.b
    !a ~a a++ a-- a*b a&b
    a*b a/b a%b
    a+b a-b
    a<<b a>>b
    a<b a<=b a>b a>=b
    a==b a!=b
    a&b
    a^b
    a|b
    a&&b
    a||b
    a?b:z
    a=b a+=b a-=b a*=b a/=b a%=b a<<=b a>>=b a&=b a^=b a|=b
    a,y
    a::b
    EOF
    cat > post <<EOF
    Foo() : x(0&42) { bar(x); }
    cout<<"Hello World?\n"<<endl;
    (1) (-1e10) (0xabcdef) 'y'
    [x] x->y x.y
    !x ~x x++ x-- x*y x&y
    x*y x/y x%y
    x+y x-y
    x<<y x>>y
    x<y x<=y x>y x>=y
    x==y x!=y
    x&y
    x^y
    x|y
    x&&y
    x||y
    x?y:z
    x=y x+=y x-=y x*=y x/=y x%=y x<<=y x>>=y x&=y x^=y x|=y
    x,y
    x::y
    EOF
    echo '* diff="cpp"' > .gitmodules
    git diff --no-index --color-words pre post > output
Surprisingly, it shows (which is very different from the expected output):

diff --git a/pre b/post index 23d5c8a..7e8c026 100644 --- a/pre +++ b/post @@ -1,19 +1,19 @@ Foo():x(0&&1){}Foo() : x(0&42) { bar(x); } cout<<"Hello World!\n"<<endl; 1 -1e10 0xabcdef 'x' [a] a->b a.b !a ~a a++ a-- a*b a&b a*b a/b a%b a+b a-b a<<b a>>b a<b a<=b a>b a>=b a==b a!=b a&b a^b a|b a&&b a||b a?b:z a=b a+=b a-=b a*=b a/=b a%=b a<<=b a>>=b a&=b a^=b a|=b a,y a::bWorld?\n"<<endl; (1) (-1e10) (0xabcdef) 'y' [x] x->y x.y !x ~x x++ x-- x*y x&y x*y x/y x%y x+y x-y x<<y x>>y x<y x<=y x>y x>=y x==y x!=y x&y x^y x|y x&&y x||y x?y:z x=y x+=y x-=y x*=y x/=y x%=y x<<=y x>>=y x&=y x^=y x|=y x,y x::y

Instead of:

<BOLD>diff --git a/pre b/post<RESET> <BOLD>index 23d5c8a..7e8c026 100644<RESET> <BOLD>--- a/pre<RESET> <BOLD>+++ b/post<RESET> <CYAN>@@ -1,19 +1,19 @@<RESET> Foo() : x(0<RED>&&1<RESET><GREEN>&42<RESET>) { <GREEN>bar(x);<RESET> } cout<<"Hello World<RED>!<RESET><GREEN>?<RESET>\n"<<endl; <GREEN>(<RESET>1<GREEN>) (<RESET>-1e10<GREEN>) (<RESET>0xabcdef<GREEN>)<RESET> '<RED>x<RESET><GREEN>y<RESET>' [<RED>a<RESET><GREEN>x<RESET>] <RED>a<RESET><GREEN>x<RESET>-><RED>b a<RESET><GREEN>y x<RESET>.<RED>b<RESET><GREEN>y<RESET> !<RED>a<RESET><GREEN>x<RESET> ~<RED>a a<RESET><GREEN>x x<RESET>++ <RED>a<RESET><GREEN>x<RESET>-- <RED>a<RESET><GREEN>x<RESET>*<RED>b a<RESET><GREEN>y x<RESET>&<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>*<RED>b a<RESET><GREEN>y x<RESET>/<RED>b a<RESET><GREEN>y x<RESET>%<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>+<RED>b a<RESET><GREEN>y x<RESET>-<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET><<<RED>b a<RESET><GREEN>y x<RESET>>><RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET><<RED>b a<RESET><GREEN>y x<RESET><=<RED>b a<RESET><GREEN>y x<RESET>><RED>b a<RESET><GREEN>y x<RESET>>=<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>==<RED>b a<RESET><GREEN>y x<RESET>!=<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>&<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>^<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>|<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>&&<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>||<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>?<RED>b<RESET><GREEN>y<RESET>:z <RED>a<RESET><GREEN>x<RESET>=<RED>b a<RESET><GREEN>y x<RESET>+=<RED>b a<RESET><GREEN>y x<RESET>-=<RED>b a<RESET><GREEN>y x<RESET>*=<RED>b a<RESET><GREEN>y x<RESET>/=<RED>b a<RESET><GREEN>y x<RESET>%=<RED>b a<RESET><GREEN>y x<RESET><<=<RED>b a<RESET><GREEN>y x<RESET>>>=<RED>b a<RESET><GREEN>y x<RESET>&=<RED>b a<RESET><GREEN>y x<RESET>^=<RED>b a<RESET><GREEN>y x<RESET>|=<RED>b<RESET> <RED>a<RESET><GREEN>y<RESET> <GREEN>x<RESET>,y <RED>a<RESET><GREEN>x<RESET>::<RED>b<RESET><GREEN>y<RESET>

That's does not just happens to cpp builtins, it happens to bibtex as well. Is it that I had missed some configuration since I have tested this on a few machines?

Show 14 quoted lines
> > > > > > aiming to port most builtins stuff to C in which we can reduce
> > > > > > the size of git. Additionally, I would also like to convert
> > > > > > scripts to builtins as an additional milestone.
> > >
> > > Careful. It is a ton of work to get the rebase -i conversion done, and
> > > then a ton of work to get it integrated. That will fill 3 months, very
> > > easily.
> >
> > My main aim is to reduce the extra dependency of perl, but planning to
> > start with rebase, can I make that an optional task where I can help out
> > after I had completed my main task during gsoc?
>
> Sure, you can make it an optional task, and I would be very happy if you
> followed up on it even after GSoC!
Yes, I can do that as well. I will be happy to have git be smaller. ^^
> As far as the Perl dependency is concerned, I actually think there is only
> one serious one left: git add -i.
Yes, that as well. But basically the core parts first.
Show 5 quoted lines
> Granted, there is send-email, but it really does not matter all that much
> these days *except* if you want to use Git to contribute to projects that
> still use a mailing list-based patch submission process (the ones that
> come to mind are: Git, Linux and Cygwin). Most Git users actually do not
> submit any patches to mailing lists, therefore I tend to ignore this one.

I won't ignore that but I will do the others first since some package manager pack it with git but instead let it be a subpackage.

Show 5 quoted lines
> The rest of the Perl scripts interacts with foreign SCMs (archimport,
> cvsexportcommit, cvsimport, cvsserver, and svn). I *guess* that it would
> be nice to follow up on the remote-svn work (which has not really gone
> anywhere so far, AFAICT the main driving contributor pursues different
> projects these days), but IMHO none of these are really needed to run Git.

As far as I have concern, the conversion stuff are rarely runned, I don't think it is worth converting to C.

Good luck and have a nice day!  - Ivan
Christian Couder· Mar 28, 2017, 06:27 UTC · re: Pickfire · lore

Re: Re: Re: Re: GSoC Project | Convert interactive rebase to C

On Mon, Mar 27, 2017 at 6:31 PM, Pickfire <pickfire@riseup.net> wrote:
Show 27 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>
>> On Sat, 25 Mar 2017, Ivan Tham wrote:
>>
>> > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>> > > On Tue, 21 Mar 2017, Ivan Tham wrote:
>> > > > Stefan Beller <sbeller@google.com> wrote:
>> > > > > On Mon, Mar 20, 2017 at 9:41 AM, Ivan Tham <pickfire@riseup.net> wrote:
>> > > > >
>> > > > > > I am interested to work on "Convert interactive rebase to C"
>> > > > >
>> > > > > +cc Johannes, who recently worked on rebase and the sequencer.
>> > >
>> > > Glad you are interested! Please note that large parts of the
>> > > interactive rebase are already in C now, but there is enough work left
>> > > in that corner.
>> >
>> > Glad to hear that, I would really like to see interactive rebase in C.
>>
>> Please note that a notable part already made it into C in v2.12.1. There
>> are still a few loose ends to tie, of course; it still makes for a great
>> head start on your project, methinks.
>
> Ah, that's great.
>
> And while I was working on the microproject (shell patterns in user diff),
> I can't produce the output of t/t4034-diff-words.sh manually with:

I don't think it's a good idea to discuss a microproject in the same thread where a project is discussed. I would suggest to move it in another thread where you describe in more details what you want to do and why, what you expect and what happened, and so on.

[...]
> That's does not just happens to cpp builtins, it happens to bibtex as well.
> Is it that I had missed some configuration since I have tested this on a
> few machines?

From a very quick look the problem seems related to how test_decode_color() is used or not.

← back to recent threads