Reference Testing (Reftest)

YamDB.jl uses a multi-level reference testing strategy to verify correctness against yamdb (Python) v0.3.0.

Levels

Running Reftests

Level 0 — Equation Functions

Compares individual equation functions (e.g., Sobolev2011, IAEA2008) with known coefficients.

# Generate Python reference data (requires yamdb installed)
python3 reftest/generate_level0.py

# Run Julia comparison
julia --project=. reftest/compare_level0.jl

# Single module
julia --project=. reftest/compare_level0.jl density

Level 2 — Full Database

Compares all substances across all four databases.

# Generate Python reference data
python3 reftest/generate_level2.py

# Run Julia comparison
julia --project=. reftest/compare_level2.jl

# Single database
julia --project=. reftest/compare_level2.jl metals
julia --project=. reftest/compare_level2.jl Janz1992

ULP Tolerance (Level 0)

Level 0 uses ULP (Unit in the Last Place) comparison. The maximum allowed ULP difference is 32.

This tolerance is needed because Python's np.power(x, n) uses the C library pow() function, while Julia's x^n with integer n uses repeated multiplication. These produce slightly different floating-point results.

The worst case is Wangetal2021 in vapour_pressure at T=800 K (31 ULP, reldiff < 5×10⁻¹⁵).

Regenerating Reference Data

Reference data files (reftest/data/level0_*.json, reftest/data/level2_*.json) are generated by Python scripts. To regenerate:

1. Install Python yamdb: pip install yamdb
2. Run the generator scripts from the YamDB.jl root directory
3. Reference data files are not committed to the repository (listed in .gitignore)