Implement basic testing infrastructure (#19)

* Add configuration for CMake build system alongside autoconf

- Add a modern CMake build system (`CMakeLists.txt`, `CMakePresets.json`) that coexists with the legacy
autoconf/Makefile build
- Shared source files (`midifile.c`, `parseabc.c`, `music_utils.c`, `parser2.c`) are compiled once via OBJECT
libraries and linked into the 8 binaries
- Three presets: `default` (Release), `debug`, `sanitize` (ASan + UBSan)
- Generates `compile_commands.json` for clangd/LSP editor support
- Install rules match the legacy Makefile (binaries, doc files, man pages)
- Pinned to `-std=gnu89` because the codebase mixes K&R `()` and ANSI typed prototypes — in C23/gnu23 (GCC 15+
default), `()` means `(void)`, making these a hard error. Note: **the existing autoconf build is also broken with
GCC 15** for the same reason

```sh
cmake --preset debug
cmake --build --preset debug
cmake --install build/debug --prefix /usr/local

Documentation

- README.md: added Building section with both autoconf and CMake instructions
- doc/readme.txt: added build instructions in the existing preamble
- doc/CHANGES: added changelog entry

Test plan

- All 3 presets configure and build with GCC 15
- Smoke test: abc2midi samples/coleraine.abc produces valid MIDI through mftext
- Sanitizer build (--preset sanitize) runs clean on sample files
- Install layout verified: 8 binaries, 10 doc files, 8 man pages in correct paths
- Build on macOS (untested, should work with AppleClang)

* Implement basic testing infrastructure

The CMake build includes a test suite covering all 8 programs:

- **Smoke tests** verify each binary runs cleanly with `-ver`.
- **Golden-file tests** run each program on a sample input and compare the
  (normalized) output to a checked-in reference. Binary MIDI outputs are
  piped through `mftext` to produce diffable text. Volatile lines (version
  banners, dates, temporary paths) are stripped before comparison.

```sh
ctest --preset debug

ctest --preset debug -L golden
ctest --preset debug -L smoke
```

To regenerate the golden files after an intentional behavioural change,
review the diff, then commit:

```sh
cmake --build build/debug --target update-golden
git diff tests/golden/
```

* Factorize more the test CMake code
This commit is contained in:
Ronan Keryell
2026-04-22 04:42:29 -07:00
committed by GitHub
parent c7f4015945
commit 1d766a85d2
18 changed files with 8421 additions and 0 deletions

83
tests/CMakeLists.txt Normal file
View File

@@ -0,0 +1,83 @@
# abcmidi test suite
#
# Two layers of tests:
#
# 1. Smoke tests - every binary runs with -ver and exits cleanly.
# Catches link errors, missing libraries, trivial crashes.
#
# 2. Golden tests - each program is run on a sample input and its output
# is compared with a checked-in reference file. Binary
# MIDI outputs are first piped through mftext to produce
# diffable text. Volatile lines (version banners, dates,
# tmpdir paths) are stripped before comparison.
#
# To regenerate golden files after an intentional change:
#
# ABCMIDI_UPDATE_GOLDEN=1 ctest --preset debug
#
# Then review the diff and commit the updated tests/golden/*.txt files.
set(SAMPLES_DIR "${PROJECT_SOURCE_DIR}/samples")
set(GOLDEN_DIR "${CMAKE_CURRENT_SOURCE_DIR}/golden")
set(TEST_TMPDIR "${CMAKE_CURRENT_BINARY_DIR}/tmp")
# --- Smoke tests: -ver on every binary ---
foreach(bin IN LISTS ABCMIDI_BINARIES)
add_test(NAME "smoke_${bin}_ver" COMMAND $<TARGET_FILE:${bin}> -ver)
set_tests_properties("smoke_${bin}_ver" PROPERTIES LABELS "smoke")
endforeach()
# --- Golden tests ---
# Build a list of "-D<BIN>=$<TARGET_FILE:bin>" args, one per binary, so that
# run_test.cmake receives all binary paths uniformly.
set(BINARY_DEFS)
foreach(bin IN LISTS ABCMIDI_BINARIES)
string(TOUPPER "${bin}" BIN_UPPER)
list(APPEND BINARY_DEFS "-D${BIN_UPPER}=$<TARGET_FILE:${bin}>")
endforeach()
# Helper: register a golden test for one program against one sample
function(add_golden_test)
cmake_parse_arguments(T "" "TYPE;SAMPLE" "" ${ARGN})
get_filename_component(stem "${T_SAMPLE}" NAME_WE)
set(test_name "${T_TYPE}_${stem}")
add_test(
NAME "${test_name}"
COMMAND "${CMAKE_COMMAND}"
-DTYPE=${T_TYPE}
-DSAMPLE=${SAMPLES_DIR}/${T_SAMPLE}
-DGOLDEN=${GOLDEN_DIR}/${test_name}.txt
-DTMPDIR=${TEST_TMPDIR}
${BINARY_DEFS}
-P "${CMAKE_CURRENT_SOURCE_DIR}/run_test.cmake"
)
set_tests_properties("${test_name}" PROPERTIES LABELS "golden")
endfunction()
# One golden test per program against coleraine.abc
foreach(type IN ITEMS abc2midi abc2abc midi2abc midistats mftext yaps midicopy abcmatch)
add_golden_test(TYPE ${type} SAMPLE coleraine.abc)
endforeach()
# A few extra coverage points on different samples
add_golden_test(TYPE abc2midi SAMPLE demo.abc)
add_golden_test(TYPE abc2abc SAMPLE demo.abc)
# --- Regenerate-golden convenience target ---
#
# Usage: cmake --build build/debug --target update-golden
#
# Equivalent to setting ABCMIDI_UPDATE_GOLDEN=1 and running ctest, but easier
# to discover via cmake --build --target.
add_custom_target(update-golden
COMMAND ${CMAKE_COMMAND} -E env ABCMIDI_UPDATE_GOLDEN=1
${CMAKE_CTEST_COMMAND} -L golden --output-on-failure
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
COMMENT "Regenerating golden test references"
USES_TERMINAL
)