Sensitivity Analysis Demo¶

This notebook demonstrates TerraFlow's sensitivity analysis workflow:

• Sobol' indices (S1, ST): quantify how much each ModelParams weight independently and in total drives score variance.
• Morris elementary effects (μ*, σ): fast screening to rank parameters by influence without requiring a full pipeline run.

Sensitivity analysis helps answer: if I perturb w_v vs w_t, which one changes my suitability scores most?

In [ ]:

import json
from pathlib import Path

from terraflow.sensitivity import run_sensitivity

import json from pathlib import Path from terraflow.sensitivity import run_sensitivity

Config setup¶

The config must include a sensitivity: block. The examples/demo_config.yml already has one — point to it here, or pass any config that includes a sensitivity: section.

In [ ]:

config_path = Path("../examples/demo_config.yml")
report_path = run_sensitivity(config_path)
print(f"Sensitivity report written to: {report_path}")

config_path = Path("../examples/demo_config.yml") report_path = run_sensitivity(config_path) print(f"Sensitivity report written to: {report_path}")

Inspect the results¶

In [ ]:

with open(report_path) as f:
    report = json.load(f)

print("Keys in sensitivity_report.json:", list(report.keys()))

with open(report_path) as f: report = json.load(f) print("Keys in sensitivity_report.json:", list(report.keys()))

In [ ]:

sobol = report.get("sobol")
if sobol:
    print("Sobol' indices (S1 = first-order, ST = total-order):")
    print(f"{'Parameter':<12} {'S1':>8} {'ST':>8}")
    print("-" * 30)
    for param, indices in sobol.items():
        s1 = indices.get('S1', float('nan'))
        st = indices.get('ST', float('nan'))
        print(f"{param:<12} {s1:>8.4f} {st:>8.4f}")
else:
    print("No Sobol' results (method was not 'sobol' or 'both')")

sobol = report.get("sobol") if sobol: print("Sobol' indices (S1 = first-order, ST = total-order):") print(f"{'Parameter':<12} {'S1':>8} {'ST':>8}") print("-" * 30) for param, indices in sobol.items(): s1 = indices.get('S1', float('nan')) st = indices.get('ST', float('nan')) print(f"{param:<12} {s1:>8.4f} {st:>8.4f}") else: print("No Sobol' results (method was not 'sobol' or 'both')")

In [ ]:

morris = report.get("morris")
if morris:
    print("Morris elementary effects (mu_star = mean |effect|, sigma = std):")
    print(f"{'Parameter':<12} {'mu_star':>10} {'sigma':>8}")
    print("-" * 32)
    for param, indices in morris.items():
        mu = indices.get('mu_star', float('nan'))
        sig = indices.get('sigma', float('nan'))
        print(f"{param:<12} {mu:>10.4f} {sig:>8.4f}")
else:
    print("No Morris results (method was not 'morris' or 'both')")

morris = report.get("morris") if morris: print("Morris elementary effects (mu_star = mean |effect|, sigma = std):") print(f"{'Parameter':<12} {'mu_star':>10} {'sigma':>8}") print("-" * 32) for param, indices in morris.items(): mu = indices.get('mu_star', float('nan')) sig = indices.get('sigma', float('nan')) print(f"{param:<12} {mu:>10.4f} {sig:>8.4f}") else: print("No Morris results (method was not 'morris' or 'both')")

Interpreting results¶

Sobol' S1 (first-order index): The fraction of output variance explained by a single parameter alone. High S1 → parameter is individually important.

Sobol' ST (total-order index): Variance fraction including all interactions. If ST >> S1 for a parameter, it interacts strongly with others.

Morris μ* (mean absolute elementary effect): Overall sensitivity rank. Higher μ* → more influential parameter.

Morris σ: Standard deviation of elementary effects. High σ relative to μ* suggests non-linear behaviour or interactions.

Practical guidance: Parameters with low ST can often be fixed at their default values without loss of model fidelity.

References¶

• Saltelli, A. et al. (2010). Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput. Phys. Commun., 181(2), 259–270.
• Morris, M.D. (1991). Factorial sampling plans for preliminary computational experiments. Technometrics, 33(2), 161–174.
• Herman, J. & Usher, W. (2017). SALib: An open-source Python library for sensitivity analysis. J. Open Source Softw., 2(9), 97.