Skip to content

Against the Original Implementations

Twelve of Zeit's newer methods were checked against published implementations: emerging hot spots, recovery metrics, EWMACD, spectral unmixing, Bayesian smoothing of class probabilities, the area of applicability, BEAST, the AR(1) trends with their regional test (PARTS), the range of a map's autocorrelation, kNNDM's spatial folds and the error profile against their R originals, and S-CCD against pyxccd, its authors' Python package. This page times each port against that original on the same inputs, and records how the results were checked to agree. Every number comes from benchmarks/results/vs-originals.json, written by the scripts in benchmarks/vs_originals/, one per method.


At a glance

Time of the original and of Zeit, on the largest workload of each benchmark

Log scale. The originals are timed inside R (pyxccd inside Python), without loading R and its packages. Zeit is the best of three runs. Hover over (or tab to) a row for the exact numbers and the agreement.

Original Zeit, 1 thread Zeit, 19 threads (8 for the AR(1) trends, PARTS and the spatial cross-validation)
Method · workload Time of the call (log scale) Faster, 1 thread / many 10 ms 100 ms 1 s 10 s 100 s Emerging hot spots 100 × 100 cells × 12 dates 6.0 ms 8.7 min 30,481× / 86,056× Recovery metrics 1000 × 1000 px × 21 years 61 ms 52.4 s 535× / 861× EWMACD 10,000 px × 71 dates 6.6 ms 2.8 s 50× / 417× S-CCD 1000 px × 1598 dates, synthetic 142 ms 22.0 s 19× / 155× Spectral unmixing 1000 × 1000 px × 6 bands 132 ms 21.8 s 17× / 165× Bayesian smoothing 1000 × 1000 px × 4 classes 147 ms 10.7 s 9× / 73× Area of applicability 2000 × 100,000 points, 8 predictors 41 ms 1.2 s 3× / 30× Area of applicability 2000 × 100,000 points, 64 predictors 319 ms 5.8 s 6× / 18× BEAST 10,000 px × 144 months 18.1 s 5.5 min 2× / 2× AR(1) trend 200 × 200 px × 30 years 23 ms 1.1 min 931× / 2,888× PARTS 40,000 px in 26 partitions of 1500 3.5 s 4.4 min 20× / 75× Autocorrelation range 300 × 300 px, 2 bands 50 ms 2.7 s 10× / 55× kNNDM folds 5000 points, k = 5 529 ms 40.1 s 71× / 76× Error profile 2000 cross-validated points 10 ms 450 ms 20× / 44× AOA, Mahalanobis 2000 × 100,000 points, 8 predictors 68 ms 15.1 min 2,082× / 13,294×

On one thread, Zeit is 3× (the area of applicability with 8 predictors) to 30,000× (emerging hot spots) faster than the original, and the outputs compared agree to floating-point rounding (for the unmixing, to the tolerance at which the original's solver stops). Compiled loops against R's per-pixel ones account for most of the gain on one thread; the threads multiply it where there is enough work to share. BEAST is the exception: its original is already C, multithreaded and vectorized, and Zeit, a port of its generic kernels, is 1.5× to 1.9× faster on a cube and slower on one series.


Results

Times of the call alone (see Protocol); "1 thread" and "19 threads" (8 for the AR(1) trends, PARTS and the spatial cross-validation) are Zeit's, and the speedups are the original's time over Zeit's. The agreement is checked by the script on every run, which fails if it does not hold.

Emerging hot spots vs sfdep

zeit.emerging_hotspots against the R package sfdep 0.2.5's emerging_hotspot_analysis (queen neighbours, a time lag of 1), on a grid of cells with 12 dates of values. A square of cells gets a trend added over the dates.

Workload sfdep Zeit, 1 thread Zeit, 19 threads Speedup (1 / 19) Agreement
20 × 20 cells, 199 permutations 24.3 s 1.2 ms 1.8 ms 20,883× / 13,308× Gi* max diff 3.6e-15
50 × 50 cells, 199 permutations 804 s (13.4 min) 7.1 ms 4.5 ms 112,995× / 179,860× Gi* max diff 1.8e-14
20 × 20 cells, 9 permutations 1.39 s 1.3 ms 1.6 ms 1,088× / 876× Gi* max diff 3.6e-15
50 × 50 cells, 9 permutations 28.4 s 4.3 ms 2.4 ms 6,594× / 11,888× Gi* max diff 1.8e-14
100 × 100 cells, 9 permutations 520 s (8.7 min) 17.1 ms 6.0 ms 30,481× / 86,056× Gi* max diff 2.0e-14

How they were checked. The Getis-Ord Gi* of every cell and date, which sfdep returns with include_gi = TRUE, is compared with Zeit's (zeit._core.hotspots.gi_star, the same as emerging_hotspots computes); the script fails if they differ by more than 1e-10.

Not the same work

sfdep takes the significance of each Gi* from permutations (199 by default), and that is most of its time; Zeit takes it from the normal distribution of the Gi*, with no permutations. The two therefore classify some cells differently (sfdep also has rules of its own for the patterns; see emerging_hotspots), and the speedups compare two ways of answering the question, not one algorithm in two languages. With 9 permutations, sfdep's time is mostly its own. On the smallest grid, 19 threads are slower than one: there is too little work to share.

Recovery metrics vs RecoverMetricsR

zeit.recovery_metrics against the R package RecoverMetricsR 0.1.0 (GPL-3; run as a black box): compute_recovery_metrics (RRI, R80P, YrYr; pre-window 2, post-window 5) and compute_recovery_time (years to 80% of the pre-disturbance mean), on an annual series of 21 years (2000–2020), every pixel disturbed in 2010 and regrowing steadily.

Workload RecoverMetricsR Zeit, 1 thread Zeit, 19 threads Speedup (1 / 19) Agreement
200 × 200 px × 21 years 1.81 s 7.5 ms 6.1 ms 243× / 298× max rel. diff 5.9e-8
1000 × 1000 px × 21 years 52.4 s 98 ms 61 ms 535× / 861× max rel. diff 6.0e-8

How they were checked. The four layers (RRI, R80P, YrYr, years to recovery) of every pixel, with the same pixels missing in both, to a relative difference of 1e-6. RecoverMetricsR takes the best of the 5 years after the trough where Zeit (Frazier et al. 2018) takes years 4 and 5, and counts the years to recovery from the year after the trough: on steady regrowths these give the same values, so the series are built that way. The remaining difference is float32's precision, Zeit's output type. The time of RecoverMetricsR is of both calls together.

EWMACD vs Brooks' script

Zeit's EWMACD against Brooks' EWMACD v1.3.0 script (Brooks et al. 2014; EWMACD v1.3.0.r, published without a licence: the benchmark runs it from a path you give and never copies it), its per-pixel function EWMACD.pixel.for.rasterEngine applied to every pixel, as its rasterEngine does. Irregular dates over 2005–2012, seasonal pixels with noise, clouds and, in a third of them, a disturbance; 2 harmonics, lambda 0.3, persistence 1, training 2005–2008.

Workload Brooks' script Zeit, 1 thread Zeit, 19 threads Speedup (1 / 19) Agreement
1,000 px × 75 dates 0.30 s 4.8 ms 0.84 ms 62× / 356× the same flags
10,000 px × 71 dates 2.75 s 55 ms 6.6 ms 50× / 417× the same flags

How they were checked. The flags of every pixel and date are identical (the script fails on any difference). The Zeit side is the Rust core that zeit.ewmacd runs (_core.ewmacd.ewmacd_batch), timed without the wrapper's handling of the cube. With a persistence above 1 the two differ on a few dates after the first disturbance (see ewmacd), so the benchmark uses 1. The two workloads have 75 and 71 dates because each draws its own irregular dates.

S-CCD vs pyxccd

zeit.sccd against pyxccd 1.1.0's sccd_detect, the implementation of S-CCD's authors (Ye et al. 2021; Apache-2.0, which GPL-2.0-only Zeit cannot take code from: Zeit's S-CCD is written from the paper, and pyxccd is run as a black box in an environment of its own). pyxccd is called pixel by pixel, as its API does. Two workloads: 4096 pixels of a Landsat Collection 2 stack of Rondonia (2020–2026, the 61 dates of the path these pixels are on), and 1000 synthetic pixels of 35 years every 8 days, a third of the dates cloudy and a third of the pixels with a change: the long series S-CCD was made for. The same pixels also go through Zeit's own ccdc, to time S-CCD's claim, a Kalman update per observation instead of a lasso refit.

Workload pyxccd Zeit, 1 thread Zeit, 19 threads Speedup (1 / 19) Agreement
4096 px × 61 dates (Rondonia) 10.1 s 1.10 s 0.196 s 9× / 52× same number of breaks 85%; pyxccd's breaks within 1 obs. 76%
1000 px × 1598 dates (synthetic) 22.0 s 1.16 s 0.142 s 19× / 155× same number of breaks 100%; breaks within 1 obs. 100%
Workload Zeit's CCDC, 1 thread Zeit's S-CCD, 1 thread CCDC, 19 threads S-CCD, 19 threads S-CCD's speedup (1 / 19)
4096 px × 61 dates (Rondonia) 1.50 s 1.10 s 0.208 s 0.196 s 1.4× / 1.1×
1000 px × 1598 dates (synthetic) 12.1 s 1.16 s 1.43 s 0.142 s 10× / 10×

How they were checked. The script compares the break dates of every pixel: the share of pixels with as many breaks in both, and the share of pyxccd's breaks with one of Zeit's within one clear observation. On the synthetic changes the two agree on every break; on Rondonia, where many statistics are near the threshold, they agree on most, for reasons given in Fidelity. The Zeit side is the Rust core that zeit.sccd runs (_core.sccd.sccd_batch). On the short Rondonia series most of the work is the initialisation that S-CCD shares with CCDC (the lasso fits of the stability test), so S-CCD is only a little faster; on 35 years of observations the Kalman updates replace CCDC's refits and S-CCD is ten times faster, more than the paper's 3–5×. These timings were taken while other builds ran on the machine: over three runs, pyxccd took 5.5–10.1 s on Rondonia and 15.8–22.0 s on the synthetic series, and Zeit varied by up to 2× too, so the speedups are good to a factor of about 2. The Rondonia stack interleaves two Landsat paths; the dates of the other path, with no data in these pixels, are left out, because CCDC refuses a pixel with data on less than half of the dates (S-CCD and pyxccd do not).

Spectral unmixing vs RStoolbox

Zeit's unmixing against the R package RStoolbox 1.0.2.2's mesma with its NNLS solver (method = "NNLS", sum_to_one = FALSE), 4 endmembers (vegetation, soil, water, non-photosynthetic vegetation) and 6 bands, on pixels mixed from them with Dirichlet fractions and noise.

Workload RStoolbox Zeit, 1 thread Zeit, 19 threads Speedup (1 / 19) Agreement
200 × 200 px 0.84 s 47 ms 5.4 ms 18× / 157× fractions max diff 3.7e-4; RMSE never larger
1000 × 1000 px 21.8 s 1.25 s 132 ms 17× / 165× fractions max diff 4.0e-4; RMSE never larger

How they were checked. Both solve the same problem: non-negative fractions, free to sum to anything (mesma's sum_to_one only divides afterwards). mesma's NNLS stops at a tolerance, while Zeit's active set solves each pixel exactly, so the fractions agree to 1e-3 (a median of 7e-5) and Zeit's residual is never larger than mesma's (smaller by up to 5e-7); the script checks both. The Zeit side is the Rust core of zeit.unmix (_core.sma.unmix_batch) with the same constraints, not zeit.unmix's default fully constrained unmixing.

Bayesian smoothing vs sits

zeit.smooth_probs against the R package sits 1.5.4: the Bayesian smoothing that sits_smooth applies to each block (.smooth_fn_bayes: the logits, bayes_smoother_fraction, back to probabilities), on probabilities of 4 classes, a 7 × 7 window, neigh_fraction 0.5 and smoothness 20.

Workload sits Zeit, 1 thread Zeit, 19 threads Speedup (1 / 19) Agreement
200 × 200 px × 4 classes 0.40 s 46 ms 5.4 ms 9× / 74× max diff 6.7e-16
1000 × 1000 px × 4 classes 10.7 s 1.13 s 147 ms 9× / 73× max diff 7.8e-16

How they were checked. Every smoothed probability, to 1e-12 (the largest difference was below 1e-15). sits is timed on its smoothing of the block only, without reading and writing its files, so the comparison is of the computation. sits gives a missing pixel its neighbours' mean where Zeit keeps it missing; the inputs have none.

Area of applicability vs CAST

zeit.train_di and zeit.aoa against the R package CAST 1.0.4's trainDI and then aoa with it (with and without the local point density), on training points in 6 clusters (8 predictors, or 64, as the series of a deep learning model) and new points spread wider.

Workload CAST Zeit, 1 thread Zeit, 19 threads Speedup (1 / 19) Agreement
500 training × 10,000 new 0.09 s 12.5 ms 1.8 ms 7× / 50× DI max diff 2.4e-15
500 × 10,000, with LPD 0.84 s 15.4 ms 3.1 ms 54× / 269× DI max diff 2.4e-15
500 × 100,000 0.31 s 105 ms 10.8 ms 3× / 29× DI max diff 4.0e-15
2000 × 100,000 1.21 s 447 ms 41 ms 3× / 30× DI max diff 1.6e-15
2000 × 100,000, 64 predictors 5.77 s 990 ms 319 ms 6× / 18× DI max diff 1.8e-15

How they were checked. The DI of every new point, to 1e-9 (the largest difference was below 1e-14). Both sides time the training's dissimilarity and the new points' together. CAST finds the nearest training points with FNN's k-d tree; Zeit uses a k-d tree up to 9 predictors and a matrix product beyond, the distances exact either way. With 8 predictors both search a tree, and the gain on one thread is smallest (3×); most of Zeit's advantage there is its threads.

BEAST vs Rbeast

zeit.beast against the R package Rbeast 1.0.2 (GPL ≥ 2), whose C code it ports: beast() on one series, and beast123(), its batch with threads of its own, on a cube. The series are monthly, with a season of period 12, noise, 5% missing values, and changes of level and of slope. Both use beast()'s defaults, from seed 42: on one series, 8,000 samples and 3 chains; on the cube, 2,000 samples and 2 chains. Rbeast runs its AVX2 kernels, the fastest on this processor; Zeit runs the port of its generic kernels.

Workload Rbeast Zeit, 1 thread Zeit, 19 threads Speedup (1 / 19) Agreement
1 series × 360 months 0.13 s 0.19 s 0.7× the same chain as Rbeast's generic kernels
10,000 px × 144 months 328 s on 1 thread, 34.5 s on 19 217 s 18.1 s 1.5× / 1.9× the same chain as Rbeast's generic kernels

The cube's speedups compare each Zeit time with Rbeast on as many threads: Rbeast's beast123() is multithreaded too. On one series Zeit is slower than Rbeast's AVX2 kernels and as fast as its generic ones (0.19–0.21 s): it runs the same scalar arithmetic. On a cube it is faster, because beast123() costs Rbeast more per pixel than beast() does (33 ms against 21 ms here, on one thread).

How they were checked. With Rbeast's generic kernels (cputype = "sse"), Rbeast and Zeit run the same chain. The script checks that the trend and the number of changepoints of the series and of the cube's first pixel are identical to the last bit, and the tests compare every output on six series (see Fidelity). The timed AVX2 runs sum in another order, so their chains drift apart, and the outputs agree only in distribution:

  • the trends differ by a median of 2e-3 standard deviations of the series;
  • Rbeast's mean number of trend changepoints is 0.31 higher on the cube.

Most of that gap is Rbeast's own. Zeit fits each pixel as beast() fits one series, from the seed, and its mean equals beast()'s exactly. beast123() carries its random stream from one pixel to the next and finds more changepoints: 0.06 more on average over 200 of these pixels, with the same generic kernels. Rbeast's 19-thread run also left one pixel unfitted (NaN).

Windows' power throttling

Windows 11 slows down the processes it considers to be in the background (power throttling, or EcoQoS): after a few seconds of work, it moves them to the efficiency cores of a hybrid processor and lowers their clock. A benchmark run from a terminal is such a process. Zeit's unpinned threads ran 2.4 times slower for it. Rbeast's threads, which it pins to processors, were slowed less. The script takes both processes out of power throttling, so the times are of the computation. Zeit's one-thread runs are pinned to the first processor, as Rbeast pins its own.

zeit.ar_trend and zeit.parts against the R package remotePARTS 1.0.4: fitAR_map, the trend of every pixel's series with AR(1) errors, and fitGLS_partition, the test of the region's mean trend by generalized least squares on partitions of the pixels, with an exponential spatial correlation (range 8 pixels), its nugget estimated in each partition, and 6 pairs of partitions crossed. The series are annual, 30 years, with a slope that changes across the map and AR(1) errors whose innovations are spatially correlated. Zeit ran these on 1 thread and on 8 of the 20 logical processors, not on 19.

Workload fitAR_map Zeit, 1 thread Zeit, 8 threads Speedup (1 / 8) Agreement
50 × 50 px × 30 years 4.09 s 7.2 ms 4.1 ms 568× / 1,007× slope max diff 5.3e-8 of the largest, ρ 1.0e-6
100 × 100 px × 30 years 16.6 s 20.2 ms 8.5 ms 821× / 1,960× slope 6.9e-8, ρ 3.4e-6
200 × 200 px × 30 years 66.8 s 71.8 ms 23.1 ms 931× / 2,888× slope 1.1e-7, ρ 7.7e-6
Workload fitGLS_partition Zeit, 1 thread Zeit, 8 threads Speedup (1 / 8) Agreement
2,500 px in 5 partitions of 500 2.47 s 0.120 s 0.030 s 21× / 84× partitions max rel. diff 2.7e-11; overall t 0.85% apart
10,000 px in 6 partitions of 1500 72.5 s 3.61 s 1.15 s 20× / 63× partitions 1.6e-10; overall t 0.97% apart
40,000 px in 26 partitions of 1500 262 s (4.4 min) 12.8 s 3.51 s 20× / 75× partitions 4.7e-15; overall t 0.028% apart

Each partition is a GLS on a dense 1500 × 1500 correlation matrix, so the cost is the same linear algebra on both sides; Zeit's gain on one thread (20×) is smaller than on the per-pixel AR fits, and its threads share the partitions.

How they were checked. The AR(1) fits agree to the tolerance of the two optimizers (the script requires 1e-3 of the largest slope and 1e-2 on ρ; the largest differences were 1.1e-7 and 7.7e-6). The GLS is checked on remotePARTS' own slopes and partitions, read back from R (Zeit's timed runs draw their own partitions of the same size): every partition's estimate, standard error and nugget, and the overall estimate, must agree to 1e-6 (the largest difference was 1.6e-10). The overall t statistic is 0.03% to 1% apart because remotePARTS 1.0.4 computes the correlation between pairs of partitions after the first without their nugget, and Zeit with it (see Fidelity).

Spatial cross-validation vs blockCV and CAST

zeit.autocorrelation_range against the R package blockCV 4.0.1's cv_spatial_autocor (5000 pixels, which fits its variograms with automap); zeit.spatial_folds with method="knndm" against CAST 1.0.4's knndm (k = 5, its example's clustered points in a square of 100, 1000 regular prediction points); zeit.error_profile against CAST's errorProfiles (with scam 1.2.23) of a linear model cross-validated in 5 folds by caret; and the Mahalanobis distance of train_di and aoa against CAST's trainDI and aoa with method = "MD". The maps are white noise of 2 bands averaged over windows of 3, 7 and 15 pixels of 100 m, so correlated up to 300, 700 and 1500 m. Zeit ran on 1 thread and on 8.

Workload Original Zeit, 1 thread Zeit, 8 threads Speedup (1 / 8) Agreement
autocorrelation_range, 300 × 300 px, range 300 m 2.74 s 287 ms 50 ms 10× / 55× range 0.89–0.99 × blockCV's
autocorrelation_range, 300 × 300 px, range 700 m 1.87 s 282 ms 49 ms 7× / 38× range 1.00–1.02 × blockCV's
autocorrelation_range, 300 × 300 px, range 1500 m 1.63 s 278 ms 50 ms 6× / 33× range 0.98–1.01 × blockCV's
kNNDM, 1000 clustered points 1.81 s 80 ms 73 ms 23× / 25× W 0.572 (CAST's folds: 0.685)
kNNDM, 5000 clustered points 40.2 s 565 ms 529 ms 71× / 76× W 0.267 (CAST's folds: 0.269)
error_profile, 2000 points, 5 folds 0.45 s 22.7 ms 10.2 ms 20× / 44× windows max diff 5.3e-15; curve within 1.2%
Mahalanobis DI, 500 training × 10,000 new 20.8 s 15.3 ms 3.0 ms 1,359× / 6,867× DI max diff 3.0e-14
Mahalanobis DI, 2000 × 100,000 909 s (15.1 min) 437 ms 68 ms 2,082× / 13,294× DI max diff 3.3e-13

How they were checked.

  • The range. blockCV and Zeit each draw their own 5000 pixels, so the ranges agree only to the sample: within 2% at 700 and 1500 m, and 1–11% shorter at 300 m, three pixels. The bands are automap's on both sides (2% to 100% of 0.35 of the map's diagonal), and the model is the one that fits best of the spherical, exponential and Gaussian (automap also tries Stein's Matérn). blockCV chose the spherical on every map, Zeit the Gaussian at 300 m. With an exponential or a Gaussian, blockCV reports gstat's range parameter, Zeit's scale, where Zeit's range is the practical range.
  • kNNDM. The two draw different folds (each side clusters its own candidates), so the folds are not compared one by one. The script measures CAST's own folds with Zeit's W, which must equal CAST's to 1e-9, and records the W of each side's folds: smaller is better, closer to the distances from the area to predict. 8 threads gain little: they share only the core's distances, not the clustering of the candidates.
  • The error profile. The windows of the DI and their RMSE, from CAST's predictions and variable weights, must equal CAST's to 1e-9 (the largest difference was 5.3e-15). The monotone curve is Zeit's own P-spline, not scam's: within 1.2% of the range of the metric, on 200 DI values.
  • The Mahalanobis distance. The DI of every new point, to 1e-9. CAST takes 750 times as long with the Mahalanobis distance as with the Euclidean on the same 2000 × 100,000 points (909 s against 1.2 s); Zeit whitens the predictors once and searches the same k-d tree as for the Euclidean distance.

The kNNDM rows are the best of two runs, not three.

Areas of transitions vs survey

zeit.transitions against R's survey 4.5 (Lumley), the reference for design-based estimators, with terra for the mapped areas. The landscape is made up: 2048 × 2048 pixels of 30 m with Forest, Pasture, Crop and Water at two dates; patches are cleared, cropped and regrown between the dates. Two maps of the dates have errors: 6% of the classes are wrong, 80% of the changes are found and 1.5% of the pixels are false change. The strata are not the transitions: they come from an older alert map (its alerts, a buffer of 2 pixels and the rest), with 250 + 200 + 150 points whose reference is the truth.

The R script counts the transitions inside the strata with terra's freq. It then builds svydesign(ids = ~1, strata = ~h, weights = ~A_h / n_h) and estimates each transition's area with:

  • svytotal (direct);
  • the mapped area plus svytotal of y − x (difference);
  • svyratio(~y, ~x) times the mapped area (ratio).

The user's and producer's accuracies come from svyratio and the overall accuracy from svymean. Its timing covers reading the three rasters, so both sides read them from files.

Workload survey + terra Zeit Speedup Agreement
direct, 2048 × 2048 px, 600 points, 16 transitions 0.48 s 0.18 s 2.7× max diff 5.7e-15
difference 0.40 s 0.21 s 1.9× max diff 7.6e-15
ratio 0.41 s 0.19 s 2.2× max diff 5.7e-15

How they were checked. For every transition, the script compares the mapped area, the estimate and its standard error, and the user's and producer's accuracies with their standard errors. It also compares the overall accuracy. Each must agree with survey's to 1e-9 of the value (or of 1 for values below 1). A value survey leaves undefined (NA) must be NaN in Zeit, for example the ratio estimate of a transition no point is mapped as. The fixtures of tests/test_accuracy_survey.py (tests/data/make_survey_fixtures.R) hold the same estimators to 1e-10 on a smaller landscape.

Most of the time on both sides is reading the rasters and counting the pairs, not the estimates. A map of integer classes of up to 2^26 pixels is read once into memory, and Zeit counts its classes, the pairs and the pixels of each stratum per row with a few bincounts; larger maps stay lazy and are counted through dask. Before this, Zeit read each raster several times through dask and took 0.73–0.85 s, 0.6× survey's speed. The Janus benchmark of the same estimators (its tools/bench_validation_vs_survey.py) has the count in Rust.


Protocol

  • The same inputs. Each script draws its inputs with a fixed seed, writes them to a temporary folder (CSV or raw float64), runs the original on the files from Rscript, and runs Zeit on the same arrays in memory.
  • The original's own timing. Each R script (benchmarks/vs_originals/<method>.R) times only the call it benchmarks with system.time (elapsed), after reading the inputs and loading the packages, and prints it; starting R, loading the packages and reading and writing files are not counted. Each original is timed once, with its defaults otherwise, in one R process.
  • Zeit's timing. A warm-up call, then the best of three (best_of in benchmarks/vs_originals/common.py, time.perf_counter), on 1 thread and on all the processors but one (n_jobs=-1: 19 here; the AR(1) trends, PARTS and the spatial cross-validation, timed later, on 8).
  • The check. After timing, each script compares the outputs (the tolerances above) and stops with an error if they disagree, so no timing is recorded for a result that differs.
  • The record. record in common.py writes one entry per method, original, workload and thread count to benchmarks/results/vs-originals.json, with the date and the machine, replacing an older entry of the same key.

Machine

Platform Windows 11 (Windows-11-10.0.26300-SP0)
Processor Intel64 Family 6 Model 183 Stepping 1, GenuineIntel (the Intel Core i5-13600K of the other benchmarks), 20 logical processors
R 4.4.2, with sfdep 0.2.5, RecoverMetricsR 0.1.0, RStoolbox 1.0.2.2, sits 1.5.4, CAST 1.0.4, Rbeast 1.0.2, remotePARTS 1.0.4, blockCV 4.0.1 (with automap 1.1.20), scam 1.2.23 and survey 4.5
Python pyxccd 1.1.0, in a virtual environment of its own
Date 2026-10-24 (the AR(1) trends and PARTS: 2026-10-28; the spatial cross-validation: 2026-10-29; the transitions: 2026-11-03)

Rerunning the benchmarks

From the repository root, with Zeit installed and R with the packages above:

python benchmarks/vs_originals/hotspots.py            # sides 20, 50, 100 (or give your own: ... hotspots.py 20 50)
python benchmarks/vs_originals/recovery.py
python benchmarks/vs_originals/unmix.py
python benchmarks/vs_originals/smooth_probs.py
python benchmarks/vs_originals/aoa.py
python benchmarks/vs_originals/beast.py
python benchmarks/vs_originals/parts.py               # sides 50, 100, 200; ZEIT_BENCH_THREADS=8 by default
python benchmarks/vs_originals/folds.py               # autocor knndm profile mahalanobis (CAST's MD takes 15 min)
python benchmarks/vs_originals/accuracy_survey.py     # zeit.transitions against survey (and terra)
python benchmarks/vs_originals/ewmacd.py "path/to/EWMACD v1.3.0.r"   # Brooks' script, not distributed
PYXCCD_PYTHON=path/to/pyxccd-env/python python benchmarks/vs_originals/sccd.py CCDC_Stack_Rondonia.tif CCDC_Stack_Rondonia_Dates.csv

The scripts look for Rscript at C:\Program Files\R\R-4.4.2\bin\Rscript.exe on Windows and on the PATH elsewhere; set the RSCRIPT environment variable to use another. Brooks' script is in EWMACD v1.3.0.7z at VTechWorks. hotspots.py runs sfdep with 199 permutations only up to 20 × 20 cells (the 50 × 50 run above took sfdep 13 minutes), and with 9 on every grid. Each script prints its rows as it records them; the chart above is redrawn from the JSON by python docs/scripts/make_vs_originals_chart.py.

What is not here

These are the benchmarks of the Zeit library. Janus, the desktop application built on it, runs the same core; the comparison of its tools with the originals is kept with Janus, not on this page. The earlier ports (LandTrendr, CCDC, BFAST, TWDTW, SOM, the deep learning models...) are compared in Performance and Fidelity.