Use the MBAR paper to examine your own calculation

Identify what produced the plotted points

The frozen reading is Shirts and Chodera, $\text{arXiv:0801.1426v3}$, corresponding to J. Chem. Phys. 129, 124105. Section V, Figure 1 compares a single constant-force record with a multitrajectory DNA-hairpin potential of mean force. The extension includes the hairpin, both double-stranded DNA handles and their attachments. Horizontal position is this connected system’s extension; the vertical zero is arbitrary. Error bars reflect the specified statistical treatment.

The experiment used sixteen five-second traces at $296.15\,\mathrm K$, sampled every $0.1\,\mathrm{ms}$. Applied forces ranged from $12.35$ to $14.41\,\mathrm{pN}$, with a target of $14.19\,\mathrm{pN}$. Fifty unequal extension bins require the width correction in Eq. (14). The paper reports approximately ten-percent relative force measurement error.

Locate the sparsely sampled extension region and ask which other force records supply it. Smaller multistate error bars show improved equilibrium estimation for those data. Force calibration contributes a separate systematic uncertainty; a reaction rate requires a dynamical analysis.

Source: Michael R. Shirts and John D. Chodera

Define the reproduced part

We recalculate the self-consistency equations in Eq. (11), target weights and the density conversion in Eq. (14). Inputs are seeded trajectories from our two-coordinate potential. We have not downloaded or reanalyzed the DNA-hairpin records, and the synthetic curves are not labelled as Figure 1 data.

The arXiv copy carries its non-exclusive distribution license. That does not establish permission to redistribute the original experimental records. We copy neither its figure nor its traces. Our diagrams and teaching data are generated here, with parameters, seeds, versions and generation scripts retained.

This limited calculation still permits a strong implementation check: integrate the model independently and submit identical samples to another solver. An experimental reproduction would additionally need complete records, calibration details and data-use permission.

Source: Michael R. Shirts and John D. Chodera

First compare independent solvers

The fixed comparison uses seven windows with two replicas each and one thousand frames per replica, totaling fourteen thousand configurations. The browser default instead saves five hundred per replica, totaling seven thousand; this comparison uses separately retained benchmark records. The browser-model implementation and PyMBAR 4.0.3 receive the same input. Subtract each solver’s first-window free energy before comparing, so both share a zero. PyMBAR documents an independent-sample requirement. Here we compare equation solutions on identical records; blocks and independent seeds address correlated-data uncertainty separately.

The maximum difference between the seven reduced free energies is about $1.26\times10^{-7}$. This checks that both solve the same estimating equations for these data. It adds no missing configurations and does not make the estimate exact.

Refining an independent two-dimensional Simpson integral changes projected-bin probabilities by about $2.09\times10^{-12}$. The refined 480-cell midpoint grid differs from that reference by at most about $4.78\times10^{-6}$ per bin. The quoted errors describe this grid and parameter set; each changed setting needs its own refinement check.

Source: PyMBAR 4.0.3

Then challenge the sampling with independent seeds

Symmetry gives right-half-plane probability $0.5$. Eight independent short runs give mean $0.481$ and between-seed standard deviation $0.129$. Another batch at half the step gives mean $0.436$ and standard deviation $0.200$. Sampling scatter is far larger than solver disagreement.

A smaller step does not automatically make a short-run average closer to truth. These runs have substantial random variation, and eight seeds cannot reliably separate a small discretization bias from that variation. More independent exchange and repeats are needed.

In the one-thousand-frame-per-replica benchmark, a deliberately trapped hidden coordinate retains connected projected windows yet gives an absolute right-population error about $0.457$. A no-overlap preset instead refuses a global reconstruction. One exposes incomplete conditional sampling; the other retains genuinely missing relative normalization.

Turn the checks into a research question

The published method supplies equations and assumptions. Our numerical evidence tests their implementation on an explicit low-dimensional model. Automated coordinate choice, adequate high-dimensional molecular sampling and reliable kinetics from accelerated paths require further validation.

Challenge a learned coordinate first on this benchmark. Does a fixed projected position have a narrow committor distribution? Do different hidden-valley starts converge to one free energy? Does an improvement survive seeds excluded from training? Low-dimensional failures are easier to inspect and explain.

Fix the model, observable and failure criteria before comparing methods. Keep every seed, including failures, and compare error distributions at equal computational effort. The full repeated-error distribution is the comparison to report.