threads / rfc / 59680

RFC patch, 3 partstest: new testing framework for libification

Subject: [RFC PATCH 0/3] test: new testing framework for libification

## tl;dr

4 messages between May 2, 2023 and May 2, 2023. Diffs are folded; open one to read it.

replies: 3people: 1as markdown or json

Felipe Contreras· May 2, 2023, 04:11 UTC · lore

Calvin Wan proposed a new testing framework for unit tests written in C, and while I agree unit tests can be useful, there's no need to write everything in C.

I'm of the belief that one should pick the language that best fit the task, not write everything in the language that you happen to be already using.

A modern testing framework written in Ruby is much simpler, and more powerful than whatever can be written in C. This patch series shows an example of what can be achieved with just 68 lines of Ruby code, plus a git library with Ruby bindings.

[1] https://lore.kernel.org/git/20230427175007.902278-1-calvinwan@google.com/
Felipe Contreras (3):
  test: add simple shared library
  test: add basic unit testing framework
  test: add basic unit test
 .gitignore       |  1 +
 t/Makefile       | 22 ++++++++++++++++
 t/lib/git.c      |  6 +++++
 t/lib/git.h      |  6 +++++
 t/ruby/git.c     | 13 +++++++++
 t/ruby/testox.rb | 68 ++++++++++++++++++++++++++++++++++++++++++++++++
 t/unit-test.t    | 14 ++++++++++
 7 files changed, 130 insertions(+)
 create mode 100644 t/lib/git.c
 create mode 100644 t/lib/git.h
 create mode 100644 t/ruby/git.c
 create mode 100644 t/ruby/testox.rb
 create mode 100644 t/unit-test.t
-- 
2.40.0+fc1
Felipe Contreras· May 2, 2023, 04:11 UTC · re: Felipe Contreras · lore

[RFC PATCH 1/3] test: add simple shared library

It doesn't contain anything, it's just preparation for further commits.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 .gitignore  | 1 +
 t/Makefile  | 9 +++++++++
 t/lib/git.c | 1 +
 t/lib/git.h | 4 ++++
 4 files changed, 15 insertions(+)
 create mode 100644 t/lib/git.c
 create mode 100644 t/lib/git.h
Show changes to 4 files +15 −0

.gitignore, t/Makefile, t/lib/git.c, t/lib/git.h

diff --git a/.gitignore b/.gitignore
index e875c59054..c249487b9d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -237,6 +237,7 @@
 *.iobj
 *.ipdb
 *.dll
+*.so
 .vs/
 Debug/
 Release/
diff --git a/t/Makefile b/t/Makefile
index 3e00cdd801..d85e3e661d 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -65,6 +65,15 @@ prove: pre-clean check-chainlint $(TEST_LINT)
 $(T):
 	@echo "*** $@ ***"; '$(TEST_SHELL_PATH_SQ)' $@ $(GIT_TEST_OPTS)
 
+lib/git.o: lib/git.c
+	$(QUIET_CC)$(COMPILE.c) -o $@ $<
+
+lib/libgit.so: CFLAGS := -fPIC -Ilib/ $(CFLAGS)
+lib/libgit.so: lib/git.o
+
+%.so::
+	$(QUIET_LINK)$(CC) -shared -o $@ $^ $(LIBS)
+
 pre-clean:
 	$(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)'
 
diff --git a/t/lib/git.c b/t/lib/git.c
new file mode 100644
index 0000000000..40d96ee354
--- /dev/null
+++ b/t/lib/git.c
@@ -0,0 +1 @@
+#include <git.h>
diff --git a/t/lib/git.h b/t/lib/git.h
new file mode 100644
index 0000000000..58a28085f5
--- /dev/null
+++ b/t/lib/git.h
@@ -0,0 +1,4 @@
+#ifndef GIT_H
+#define GIT_H
+
+#endif /* GIT_H */
-- 
2.40.0+fc1
Felipe Contreras· May 2, 2023, 04:11 UTC · re: Felipe Contreras · lore

[RFC PATCH 2/3] test: add basic unit testing framework

Testing full git commands is helpful, but so it is to test specific functions, which is not easy to do from the shell, which is how the current testing framework is implemented.

A much more modern way to test specific functions is to write a binding for a modern language--like Ruby--and then use that binding on a modern testing framework using the same language.

This makes the testing framework much simpler, and also has the advantage that writing the Ruby bindings spots libification issues.

Moreover no forks are needed, and crashes are handled nicely due to Ruby's exception handling.

This is the best of all worlds.

This step doesn't add any actual unit tests, simply prepares the groundwork for writing them.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 t/Makefile       | 13 +++++++++
 t/ruby/git.c     |  7 +++++
 t/ruby/testox.rb | 68 ++++++++++++++++++++++++++++++++++++++++++++++++
 t/unit-test.t    | 10 +++++++
 4 files changed, 98 insertions(+)
 create mode 100644 t/ruby/git.c
 create mode 100644 t/ruby/testox.rb
 create mode 100644 t/unit-test.t
Show changes to 4 files +98 −0

t/Makefile, t/ruby/git.c, t/ruby/testox.rb, t/unit-test.t

diff --git a/t/Makefile b/t/Makefile
index d85e3e661d..4fdad529ab 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -47,6 +47,9 @@ CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl
 # scripts not to run the external "chainlint.pl" script themselves
 CHAINLINTSUPPRESS = GIT_TEST_EXT_CHAIN_LINT=0 && export GIT_TEST_EXT_CHAIN_LINT &&
 
+RUBY_LIBS = $(shell pkg-config --libs ruby-3.0)
+RUBY_CFLAGS = $(shell pkg-config --cflags ruby-3.0)
+
 all: $(DEFAULT_TEST_TARGET)
 
 test: pre-clean check-chainlint $(TEST_LINT)
@@ -65,6 +68,13 @@ prove: pre-clean check-chainlint $(TEST_LINT)
 $(T):
 	@echo "*** $@ ***"; '$(TEST_SHELL_PATH_SQ)' $@ $(GIT_TEST_OPTS)
 
+ruby/git.o: ruby/git.c
+	$(QUIET_CC)$(COMPILE.c) -o $@ $<
+
+ruby/git.so: CFLAGS := -fPIC $(RUBY_CFLAGS) -Ilib/ $(CFLAGS)
+ruby/git.so: LIBS := $(RUBY_LIBS) -Llib/ -lgit $(LIBS)
+ruby/git.so: ruby/git.o | lib/libgit.so
+
 lib/git.o: lib/git.c
 	$(QUIET_CC)$(COMPILE.c) -o $@ $<
 
@@ -74,6 +84,9 @@ lib/libgit.so: lib/git.o
 %.so::
 	$(QUIET_LINK)$(CC) -shared -o $@ $^ $(LIBS)
 
+test-unit: ruby/git.so
+	@LD_LIBRARY_PATH=lib/ ruby -Iruby unit-test.t
+
 pre-clean:
 	$(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)'
 
diff --git a/t/ruby/git.c b/t/ruby/git.c
new file mode 100644
index 0000000000..e75692c582
--- /dev/null
+++ b/t/ruby/git.c
@@ -0,0 +1,7 @@
+#include <ruby.h>
+#include <git.h>
+
+void Init_git(void)
+{
+	VALUE mod = rb_define_module("Git");
+}
diff --git a/t/ruby/testox.rb b/t/ruby/testox.rb
new file mode 100644
index 0000000000..1cc58a3ab3
--- /dev/null
+++ b/t/ruby/testox.rb
@@ -0,0 +1,68 @@
+# Copyright (c) 2023 Felipe Contreras
+
+require 'singleton'
+
+class TestOxException < StandardError
+  attr_reader :actual, :expected
+
+  def initialize(actual, expected)
+    @actual, @expected = actual, expected
+  end
+end
+
+class TestOx
+  include Singleton
+
+  @tests = []
+
+  class << self
+
+    def add(desc, &block)
+      @tests << [ desc, block ]
+    end
+
+    def run
+      success = true
+
+      puts '1..%d' % @tests.length
+
+      @tests.each_with_index do |(desc, block),i|
+        begin
+          instance.instance_exec(&block)
+          puts 'ok %d - %s' % [i + 1, desc]
+        rescue => e
+          success = false
+          puts 'not ok %d - %s' % [i + 1, desc]
+          if e.is_a?(TestOxException)
+            puts '# -%s' % e.expected
+            puts '# +%s' % e.actual
+          else
+            puts '# exception: %s' % e
+          end
+        end
+      end
+
+      return success
+    end
+
+  end
+
+  def assert(string)
+    raise string
+  end
+
+  def ok(bool)
+    raise TestOxException.new(bool.inspect, true) unless bool
+  end
+
+  def is(actual, expected)
+    raise TestOxException.new(actual, expected) unless actual == expected
+  end
+
+end
+
+def test(*args, &block)
+  TestOx.add(*args, &block)
+end
+
+at_exit { exit TestOx.run }
diff --git a/t/unit-test.t b/t/unit-test.t
new file mode 100644
index 0000000000..97d8a14ec3
--- /dev/null
+++ b/t/unit-test.t
@@ -0,0 +1,10 @@
+#!/bin/env ruby
+
+require 'testox'
+require 'git'
+
+test 'basic' do
+  ok(true)
+end
+
+# vim: ft=ruby
-- 
2.40.0+fc1
Felipe Contreras· May 2, 2023, 04:11 UTC · re: Felipe Contreras · lore

[RFC PATCH 3/3] test: add basic unit test

We simply add that function, and test that function.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 t/lib/git.c   | 5 +++++
 t/lib/git.h   | 2 ++
 t/ruby/git.c  | 6 ++++++
 t/unit-test.t | 4 ++++
 4 files changed, 17 insertions(+)
Show changes to 4 files +17 −0

t/lib/git.c, t/lib/git.h, t/ruby/git.c, t/unit-test.t

diff --git a/t/lib/git.c b/t/lib/git.c
index 40d96ee354..54206171cb 100644
--- a/t/lib/git.c
+++ b/t/lib/git.c
@@ -1 +1,6 @@
 #include <git.h>
+
+int ok(void)
+{
+	return 1;
+}
diff --git a/t/lib/git.h b/t/lib/git.h
index 58a28085f5..dbbac9e406 100644
--- a/t/lib/git.h
+++ b/t/lib/git.h
@@ -1,4 +1,6 @@
 #ifndef GIT_H
 #define GIT_H
 
+int ok(void);
+
 #endif /* GIT_H */
diff --git a/t/ruby/git.c b/t/ruby/git.c
index e75692c582..e173dfe03d 100644
--- a/t/ruby/git.c
+++ b/t/ruby/git.c
@@ -1,7 +1,13 @@
 #include <ruby.h>
 #include <git.h>
 
+VALUE rb_ok(VALUE self)
+{
+	return RTEST(ok());
+}
+
 void Init_git(void)
 {
 	VALUE mod = rb_define_module("Git");
+	rb_define_singleton_method(mod, "ok", rb_ok, 0);
 }
diff --git a/t/unit-test.t b/t/unit-test.t
index 97d8a14ec3..27303f9c49 100644
--- a/t/unit-test.t
+++ b/t/unit-test.t
@@ -7,4 +7,8 @@ test 'basic' do
   ok(true)
 end
 
+test 'ok' do
+  ok(Git::ok())
+end
+
 # vim: ft=ruby
-- 
2.40.0+fc1

← back to recent threads