Open benchmark

Submit your forecasts

Submit your CSV through the submission form. An automatic check validates the file within a few minutes, records the submission time and stores accepted forecasts in the public repository.

Open the submission form Download the NCT IDs (CSV)

The form is a GitHub issue form, so a free GitHub account is needed. Attach the CSV by dragging it into the form; the reply on the issue says whether it was accepted and, if not, what to fix.

What to submit

One CSV per model, one row per trial you want scored.

ColumnTypeDescription
nct_idstringTrial identifier from the benchmark list, for example NCT00223145
p_successfloat, 0 to 1Predicted probability that the trial reads out as a success
model_namestringShort stable name for your method, the same in every row and in the form
generated_atISO dateWhen the forecasts were produced
nct_id,p_success,model_name,generated_at
NCT00223145,0.82,my_model_v1,2026-11-01
NCT00223171,0.44,my_model_v1,2026-11-01

Rules

How outcomes are labelled

Trials are classified as a success or a failure according to whether the reported results satisfy the prespecified criteria for the primary endpoints. Explicit statements of primary endpoint attainment or failure in the readout are used when consistent with the trial's prespecified design. When such statements are absent or ambiguous, the reported results are compared with the primary endpoints documented in ClinicalTrials.gov. Trials terminated before completion are failures, unless attainment of the prespecified primary endpoint has been reported.

For trials with multiple primary endpoints, adjudication follows the prespecified criteria for overall trial success: trials requiring all coprimary endpoints to be met are failures if any required endpoint failed, and trials permitting success on a subset of primary endpoints are evaluated according to their prespecified multiplicity adjusted testing strategy. A trial stays unresolved, and is not scored, when the available results or prespecified decision criteria are insufficient to determine whether the primary endpoints were met, or when conflicting information cannot be resolved. Every label links its primary source, and the same rules apply regardless of how a sponsor frames the disclosure.