Re: [PATCH] fuzz: reorganise the path for existing oss-fuzz fuzzers
On 16/9/2022 10:55 pm, Junio C Hamano wrote:
Show 11 quoted lines
> "Arthur Chan via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Arthur Chan <arthur.chan@adalogics.com>
>>
>> This patch is aimed to provide a better organisation for oss-fuzz
>> fuzzers, allowing more fuzzers for the git project to be added
>> in a later development.
>>
>> A new folder oss-fuzz has been created and existing fuzzers are
>> moved into the new folders. Makefile has been fixed accordingly.
> "folder" -> "directory" everywhere.
Thanks for the suggestion. I will change it in v2.
Show 34 quoted lines
>> Makefile | 6 +++---
>> fuzz-commit-graph.c => oss-fuzz/fuzz-commit-graph.c | 0
>> fuzz-pack-headers.c => oss-fuzz/fuzz-pack-headers.c | 0
>> fuzz-pack-idx.c => oss-fuzz/fuzz-pack-idx.c | 0
>> 4 files changed, 3 insertions(+), 3 deletions(-)
>> rename fuzz-commit-graph.c => oss-fuzz/fuzz-commit-graph.c (100%)
>> rename fuzz-pack-headers.c => oss-fuzz/fuzz-pack-headers.c (100%)
>> rename fuzz-pack-idx.c => oss-fuzz/fuzz-pack-idx.c (100%)
> It is curious that we do not have any changes to .gitignore
> patterns.
>
> $ git grep fuzz .gitignore Makefile
> .gitignore:/fuzz-commit-graph
> .gitignore:/fuzz_corpora
> .gitignore:/fuzz-pack-headers
> .gitignore:/fuzz-pack-idx
> Makefile:FUZZ_OBJS += fuzz-commit-graph.o
> Makefile:FUZZ_OBJS += fuzz-pack-headers.o
> Makefile:FUZZ_OBJS += fuzz-pack-idx.o
> Makefile:.PHONY: fuzz-objs
> Makefile:fuzz-objs: $(FUZZ_OBJS)
> Makefile:# Always build fuzz objects even if not testing, to prevent bit-rot.
> Makefile:# Building fuzz targets generally requires a special set of compiler flags that
> Makefile:# CFLAGS="-fsanitize=fuzzer-no-link,address" \
> Makefile:# LIB_FUZZING_ENGINE="-fsanitize=fuzzer" \
> Makefile:# fuzz-all
> Makefile:.PHONY: fuzz-all
> Makefile:fuzz-all: $(FUZZ_PROGRAMS)
>
> I do not know what "fuzz_corpora" is, which step in build creates
> it, and why we do not have to bother removing it in "make clean",
> the last of which is not the fault of this patch, but I suspect that
> at least other three existing entries that name $(FUZZ_PROGRAMS)
> need to be updated, because ...
I also have no idea what fuzz_corpora is, I will ask the person who wrote the three fuzzers to see if he got any idea.
And yes, indeed, I miss the change in .gitignore, I will modify it and push it to v2.
Show 25 quoted lines
>> diff --git a/Makefile b/Makefile
>> index d9247ead45b..2d56aae7a1d 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -686,9 +686,9 @@ SCRIPTS = $(SCRIPT_SH_GEN) \
>>
>> ETAGS_TARGET = TAGS
>>
>> -FUZZ_OBJS += fuzz-commit-graph.o
>> -FUZZ_OBJS += fuzz-pack-headers.o
>> -FUZZ_OBJS += fuzz-pack-idx.o
>> +FUZZ_OBJS += oss-fuzz/fuzz-commit-graph.o
>> +FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
>> +FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o
> ... FUZZ_OBJS now live in the oss-fuzz/ directory, and Makefile has
>
> FUZZ_PROGRAMS += $(patsubst %.o,%,$(FUZZ_OBJS))
>
> $(FUZZ_PROGRAMS): all
> $(QUIET_LINK)$(CXX) $(FUZZ_CXXFLAGS) $(LIB_OBJS) $(BUILTIN_OBJS) \
> $(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@
>
> neither of which has been touched by the patch, so presumably the
> executables are now created in the oss-fuzz/ directory as well, and
> they are what .gitignore should be listing, right?
Yes, indeed. sorry for missing out the changes in .gitignore. I will modify it and push it to v2
Show 8 quoted lines
>
> Also, compiling the exectuable files would not be the end of the
> story, right? Do folks (like test script, makefile targets and CI
> recipes) who used to run ./fuzz-commit-graph need to be told that
> they now need to run oss-fuzz/fuzz-commit-graph instead? They may
> not be inside my tree, but what's the best way to inform them? Add
> entries to release notes (not asking you to add one immediately ---
> asking you to help formulating the plans).
In general, for the oss-fuzz project, there will be a dockerfile and build script prepared for each of the target project. The Dockerfile will pull out the target version of the the target project, setting them up for the build script. Then the build script will compile the target project together with the fuzzers. After that it will move the compiled fuzzer into correct location for the oss-fuzz library to grab them and start the fuzzing process. The fuzzing and execution of those fuzzers are all on the oss-fuzz side. We are just trying to push fuzzers to the git upstream in order to allow it to compile and sync with the git repository which the fuzzers depending on. This is the existing execution plan for those fuzzers as far as I know. We are actually in offline discussion with the person who create that three fuzzers and we agree to go on this route. For your reference, you could find the dockerfile and build script of the oss-fuzz for the git repository in https://github.com/google/oss-fuzz/tree/master/projects/git. We will of course need to update the build script on the oss-fuzz tree in order to allow oss-fuzz to retrieve the fuzzers after it has been reloacted. But this won't affect the git tree.
Thanks again for pointing out my careless mistake on .gitignore. I will fix that together with the comment and push a v2. Thanks very much for your time. Cheers
> Thanks.
> ADA Logics Ltd is registered in England. No: 11624074.
> Registered office: 266 Banbury Road, Post Box 292,
> OX2 7DL, Oxford, Oxfordshire , United Kingdom
ADA Logics Ltd is registered in England. No: 11624074. Registered office: 266 Banbury Road, Post Box 292, OX2 7DL, Oxford, Oxfordshire , United Kingdom
Arthur Chan via GitGitGadget
