Keep CMake reader/writer test artifacts in the build tree

[vzer200]

Out-of-source CMake tests still write .actual, .rewrite, .actual-rewrite, and .process-output beside the source fixtures. With a read-only source tree, both reader/writer CTests fail while the unit test passes.

Add an optional runjsontests.py --output-dir and use separate build-tree directories for CTest (test-output/ctest) and jsoncpp_check (test-output/jsoncpp_check). The runner copies writer-test inputs there on each run, keeps expected results in the original source directory, and leaves parse-only fixtures in place. Manual calls without the option retain their existing behavior. Existing same-file inputs are recognized by file identity, including case aliases and hardlinks. Directory creation tolerates another caller creating that directory concurrently. The two reader/writer CTests retain their resource lock, and a concurrent custom-target run uses separate files. Multiple independently launched instances of the same entry point still require caller coordination or distinct output directories.

Fixes #1515.

Validation on Windows with LLVM-MinGW Clang 23, CMake 4.4.3, Ninja, and Python 3.12:

• Built the unchanged baseline and final version, including all 132 C++ unit tests.
• Reproduced the original failure with source fixture Write access denied: two reader/writer CTests failed, one unit CTest passed.
• Independently reran all three final CTests with -j 3 and jsoncpp_check while writes to source fixtures were denied: all passed. Verified the denial was effective and all 401 fixture files retained their content and modification times; restored the original permissions afterward.
• Independently exercised 17 real CLI scenarios: default manual behavior, omitted/single-file input, relative and space-containing output directories, parse-only cases, source expected-result authority, refreshed copied inputs, same-file aliases, and concurrent runs with separate output directories.
• After the directory-creation review feedback, independently checked 20 deterministic two-caller creation races: the initial revision fails once per pair, while the final revision passes both calls. Existing output files still cause an error without modification. Reran the 17 CLI scenarios and all three parallel CTests on that final revision.
• After the shared-output review feedback, reproduced the copied-input truncation window with a real runner, then independently ran three overlapping jsoncpp_check/CTest pairs with source writes denied. All six commands passed; both output directories had matching artifact contents, all 401 source fixtures were unchanged, and source permissions were restored.
• git diff --check passed. No C++ library or ABI changes.

Prepared with Codex and reviewed by a separate automated agent. The review found two same-file alias failures in the initial implementation; both were corrected and the final revision was rechecked before submission. No human review is claimed. The read-only reproduction used Windows ACLs, not a Unix read-only mount.

[greptile-apps]

Greptile Summary

The PR adds an optional build-tree output location for the Python reader/writer test runner while preserving source-tree expected results and existing manual behavior.

• Copies non-parse-only inputs into the selected output directory before execution.
• Places CTest and jsoncpp_check artifacts in separate build-tree subdirectories.
• Retains the CTest resource lock for tests that share the CTest artifact directory.
• Documents the new --output-dir option and CMake behavior.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the directory-creation race is addressed and the previously conflicting custom-target and CTest artifacts now use separate directories.

Important Files Changed

Filename	Overview
test/runjsontests.py	Adds optional output-directory creation, input copying with same-file protection, and redirects generated artifacts while retaining source expected-result comparisons.
src/jsontestrunner/CMakeLists.txt	Configures separate build-tree artifact directories for the custom target and CTests while retaining serialization between the two CTests.
CONTRIBUTING.md	Documents default manual artifact placement, the new output-directory option, and CMake’s build-tree behavior.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    S[Source fixtures and expected results] --> R[runjsontests.py]
    R -->|jsoncpp_check| J[Build tree: test-output/jsoncpp_check]
    R -->|CTest reader/writer tests| C[Build tree: test-output/ctest]
    C --> L[CTest resource lock serializes shared artifacts]
    J --> V[Compare generated results with source expectations]
    C --> V

Loading

_{Reviews (3): Last reviewed commit: "Separate CTest and custom target output ..." | Re-trigger Greptile}