Against the Original Implementations¶
Twelve of Zeit's newer methods were checked against published implementations: emerging hot spots, recovery metrics, EWMACD, spectral unmixing, Bayesian smoothing of class probabilities, the area of applicability, BEAST, the AR(1) trends with their regional test (PARTS), the range of a map's autocorrelation, kNNDM's spatial folds and the error profile against their R originals, and S-CCD against pyxccd, its authors' Python package. This page times each port against that original on the same inputs, and records how the results were checked to agree. Every number comes from benchmarks/results/vs-originals.json, written by the scripts in benchmarks/vs_originals/, one per method.
At a glance¶
Time of the original and of Zeit, on the largest workload of each benchmark
Log scale. The originals are timed inside R (pyxccd inside Python), without loading R and its packages. Zeit is the best of three runs. Hover over (or tab to) a row for the exact numbers and the agreement.
On one thread, Zeit is 3× (the area of applicability with 8 predictors) to 30,000× (emerging hot spots) faster than the original, and the outputs compared agree to floating-point rounding (for the unmixing, to the tolerance at which the original's solver stops). Compiled loops against R's per-pixel ones account for most of the gain on one thread; the threads multiply it where there is enough work to share. BEAST is the exception: its original is already C, multithreaded and vectorized, and Zeit, a port of its generic kernels, is 1.5× to 1.9× faster on a cube and slower on one series.
Results¶
Times of the call alone (see Protocol); "1 thread" and "19 threads" (8 for the AR(1) trends, PARTS and the spatial cross-validation) are Zeit's, and the speedups are the original's time over Zeit's. The agreement is checked by the script on every run, which fails if it does not hold.
Emerging hot spots vs sfdep¶
zeit.emerging_hotspots against the R package sfdep 0.2.5's emerging_hotspot_analysis (queen neighbours, a time lag of 1), on a grid of cells with 12 dates of values. A square of cells gets a trend added over the dates.
| Workload | sfdep | Zeit, 1 thread | Zeit, 19 threads | Speedup (1 / 19) | Agreement |
|---|---|---|---|---|---|
| 20 × 20 cells, 199 permutations | 24.3 s | 1.2 ms | 1.8 ms | 20,883× / 13,308× | Gi* max diff 3.6e-15 |
| 50 × 50 cells, 199 permutations | 804 s (13.4 min) | 7.1 ms | 4.5 ms | 112,995× / 179,860× | Gi* max diff 1.8e-14 |
| 20 × 20 cells, 9 permutations | 1.39 s | 1.3 ms | 1.6 ms | 1,088× / 876× | Gi* max diff 3.6e-15 |
| 50 × 50 cells, 9 permutations | 28.4 s | 4.3 ms | 2.4 ms | 6,594× / 11,888× | Gi* max diff 1.8e-14 |
| 100 × 100 cells, 9 permutations | 520 s (8.7 min) | 17.1 ms | 6.0 ms | 30,481× / 86,056× | Gi* max diff 2.0e-14 |
How they were checked. The Getis-Ord Gi* of every cell and date, which sfdep returns with include_gi = TRUE, is compared with Zeit's (zeit._core.hotspots.gi_star, the same as emerging_hotspots computes); the script fails if they differ by more than 1e-10.
Not the same work
sfdep takes the significance of each Gi* from permutations (199 by default), and that is most of its time; Zeit takes it from the normal distribution of the Gi*, with no permutations. The two therefore classify some cells differently (sfdep also has rules of its own for the patterns; see emerging_hotspots), and the speedups compare two ways of answering the question, not one algorithm in two languages. With 9 permutations, sfdep's time is mostly its own. On the smallest grid, 19 threads are slower than one: there is too little work to share.
Recovery metrics vs RecoverMetricsR¶
zeit.recovery_metrics against the R package RecoverMetricsR 0.1.0 (GPL-3; run as a black box): compute_recovery_metrics (RRI, R80P, YrYr; pre-window 2, post-window 5) and compute_recovery_time (years to 80% of the pre-disturbance mean), on an annual series of 21 years (2000–2020), every pixel disturbed in 2010 and regrowing steadily.
| Workload | RecoverMetricsR | Zeit, 1 thread | Zeit, 19 threads | Speedup (1 / 19) | Agreement |
|---|---|---|---|---|---|
| 200 × 200 px × 21 years | 1.81 s | 7.5 ms | 6.1 ms | 243× / 298× | max rel. diff 5.9e-8 |
| 1000 × 1000 px × 21 years | 52.4 s | 98 ms | 61 ms | 535× / 861× | max rel. diff 6.0e-8 |
How they were checked. The four layers (RRI, R80P, YrYr, years to recovery) of every pixel, with the same pixels missing in both, to a relative difference of 1e-6. RecoverMetricsR takes the best of the 5 years after the trough where Zeit (Frazier et al. 2018) takes years 4 and 5, and counts the years to recovery from the year after the trough: on steady regrowths these give the same values, so the series are built that way. The remaining difference is float32's precision, Zeit's output type. The time of RecoverMetricsR is of both calls together.
EWMACD vs Brooks' script¶
Zeit's EWMACD against Brooks' EWMACD v1.3.0 script (Brooks et al. 2014; EWMACD v1.3.0.r, published without a licence: the benchmark runs it from a path you give and never copies it), its per-pixel function EWMACD.pixel.for.rasterEngine applied to every pixel, as its rasterEngine does. Irregular dates over 2005–2012, seasonal pixels with noise, clouds and, in a third of them, a disturbance; 2 harmonics, lambda 0.3, persistence 1, training 2005–2008.
| Workload | Brooks' script | Zeit, 1 thread | Zeit, 19 threads | Speedup (1 / 19) | Agreement |
|---|---|---|---|---|---|
| 1,000 px × 75 dates | 0.30 s | 4.8 ms | 0.84 ms | 62× / 356× | the same flags |
| 10,000 px × 71 dates | 2.75 s | 55 ms | 6.6 ms | 50× / 417× | the same flags |
How they were checked. The flags of every pixel and date are identical (the script fails on any difference). The Zeit side is the Rust core that zeit.ewmacd runs (_core.ewmacd.ewmacd_batch), timed without the wrapper's handling of the cube. With a persistence above 1 the two differ on a few dates after the first disturbance (see ewmacd), so the benchmark uses 1. The two workloads have 75 and 71 dates because each draws its own irregular dates.
S-CCD vs pyxccd¶
zeit.sccd against pyxccd 1.1.0's sccd_detect, the implementation of S-CCD's authors (Ye et al. 2021; Apache-2.0, which GPL-2.0-only Zeit cannot take code from: Zeit's S-CCD is written from the paper, and pyxccd is run as a black box in an environment of its own). pyxccd is called pixel by pixel, as its API does. Two workloads: 4096 pixels of a Landsat Collection 2 stack of Rondonia (2020–2026, the 61 dates of the path these pixels are on), and 1000 synthetic pixels of 35 years every 8 days, a third of the dates cloudy and a third of the pixels with a change: the long series S-CCD was made for. The same pixels also go through Zeit's own ccdc, to time S-CCD's claim, a Kalman update per observation instead of a lasso refit.
| Workload | pyxccd | Zeit, 1 thread | Zeit, 19 threads | Speedup (1 / 19) | Agreement |
|---|---|---|---|---|---|
| 4096 px × 61 dates (Rondonia) | 10.1 s | 1.10 s | 0.196 s | 9× / 52× | same number of breaks 85%; pyxccd's breaks within 1 obs. 76% |
| 1000 px × 1598 dates (synthetic) | 22.0 s | 1.16 s | 0.142 s | 19× / 155× | same number of breaks 100%; breaks within 1 obs. 100% |
| Workload | Zeit's CCDC, 1 thread | Zeit's S-CCD, 1 thread | CCDC, 19 threads | S-CCD, 19 threads | S-CCD's speedup (1 / 19) |
|---|---|---|---|---|---|
| 4096 px × 61 dates (Rondonia) | 1.50 s | 1.10 s | 0.208 s | 0.196 s | 1.4× / 1.1× |
| 1000 px × 1598 dates (synthetic) | 12.1 s | 1.16 s | 1.43 s | 0.142 s | 10× / 10× |
How they were checked. The script compares the break dates of every pixel: the share of pixels with as many breaks in both, and the share of pyxccd's breaks with one of Zeit's within one clear observation. On the synthetic changes the two agree on every break; on Rondonia, where many statistics are near the threshold, they agree on most, for reasons given in Fidelity. The Zeit side is the Rust core that zeit.sccd runs (_core.sccd.sccd_batch). On the short Rondonia series most of the work is the initialisation that S-CCD shares with CCDC (the lasso fits of the stability test), so S-CCD is only a little faster; on 35 years of observations the Kalman updates replace CCDC's refits and S-CCD is ten times faster, more than the paper's 3–5×. These timings were taken while other builds ran on the machine: over three runs, pyxccd took 5.5–10.1 s on Rondonia and 15.8–22.0 s on the synthetic series, and Zeit varied by up to 2× too, so the speedups are good to a factor of about 2. The Rondonia stack interleaves two Landsat paths; the dates of the other path, with no data in these pixels, are left out, because CCDC refuses a pixel with data on less than half of the dates (S-CCD and pyxccd do not).
Spectral unmixing vs RStoolbox¶
Zeit's unmixing against the R package RStoolbox 1.0.2.2's mesma with its NNLS solver (method = "NNLS", sum_to_one = FALSE), 4 endmembers (vegetation, soil, water, non-photosynthetic vegetation) and 6 bands, on pixels mixed from them with Dirichlet fractions and noise.
| Workload | RStoolbox | Zeit, 1 thread | Zeit, 19 threads | Speedup (1 / 19) | Agreement |
|---|---|---|---|---|---|
| 200 × 200 px | 0.84 s | 47 ms | 5.4 ms | 18× / 157× | fractions max diff 3.7e-4; RMSE never larger |
| 1000 × 1000 px | 21.8 s | 1.25 s | 132 ms | 17× / 165× | fractions max diff 4.0e-4; RMSE never larger |
How they were checked. Both solve the same problem: non-negative fractions, free to sum to anything (mesma's sum_to_one only divides afterwards). mesma's NNLS stops at a tolerance, while Zeit's active set solves each pixel exactly, so the fractions agree to 1e-3 (a median of 7e-5) and Zeit's residual is never larger than mesma's (smaller by up to 5e-7); the script checks both. The Zeit side is the Rust core of zeit.unmix (_core.sma.unmix_batch) with the same constraints, not zeit.unmix's default fully constrained unmixing.
Bayesian smoothing vs sits¶
zeit.smooth_probs against the R package sits 1.5.4: the Bayesian smoothing that sits_smooth applies to each block (.smooth_fn_bayes: the logits, bayes_smoother_fraction, back to probabilities), on probabilities of 4 classes, a 7 × 7 window, neigh_fraction 0.5 and smoothness 20.
| Workload | sits | Zeit, 1 thread | Zeit, 19 threads | Speedup (1 / 19) | Agreement |
|---|---|---|---|---|---|
| 200 × 200 px × 4 classes | 0.40 s | 46 ms | 5.4 ms | 9× / 74× | max diff 6.7e-16 |
| 1000 × 1000 px × 4 classes | 10.7 s | 1.13 s | 147 ms | 9× / 73× | max diff 7.8e-16 |
How they were checked. Every smoothed probability, to 1e-12 (the largest difference was below 1e-15). sits is timed on its smoothing of the block only, without reading and writing its files, so the comparison is of the computation. sits gives a missing pixel its neighbours' mean where Zeit keeps it missing; the inputs have none.
Area of applicability vs CAST¶
zeit.train_di and zeit.aoa against the R package CAST 1.0.4's trainDI and then aoa with it (with and without the local point density), on training points in 6 clusters (8 predictors, or 64, as the series of a deep learning model) and new points spread wider.
| Workload | CAST | Zeit, 1 thread | Zeit, 19 threads | Speedup (1 / 19) | Agreement |
|---|---|---|---|---|---|
| 500 training × 10,000 new | 0.09 s | 12.5 ms | 1.8 ms | 7× / 50× | DI max diff 2.4e-15 |
| 500 × 10,000, with LPD | 0.84 s | 15.4 ms | 3.1 ms | 54× / 269× | DI max diff 2.4e-15 |
| 500 × 100,000 | 0.31 s | 105 ms | 10.8 ms | 3× / 29× | DI max diff 4.0e-15 |
| 2000 × 100,000 | 1.21 s | 447 ms | 41 ms | 3× / 30× | DI max diff 1.6e-15 |
| 2000 × 100,000, 64 predictors | 5.77 s | 990 ms | 319 ms | 6× / 18× | DI max diff 1.8e-15 |
How they were checked. The DI of every new point, to 1e-9 (the largest difference was below 1e-14). Both sides time the training's dissimilarity and the new points' together. CAST finds the nearest training points with FNN's k-d tree; Zeit uses a k-d tree up to 9 predictors and a matrix product beyond, the distances exact either way. With 8 predictors both search a tree, and the gain on one thread is smallest (3×); most of Zeit's advantage there is its threads.
BEAST vs Rbeast¶
zeit.beast against the R package Rbeast 1.0.2 (GPL ≥ 2), whose C code it ports: beast() on one series, and beast123(), its batch with threads of its own, on a cube. The series are monthly, with a season of period 12, noise, 5% missing values, and changes of level and of slope. Both use beast()'s defaults, from seed 42: on one series, 8,000 samples and 3 chains; on the cube, 2,000 samples and 2 chains. Rbeast runs its AVX2 kernels, the fastest on this processor; Zeit runs the port of its generic kernels.
| Workload | Rbeast | Zeit, 1 thread | Zeit, 19 threads | Speedup (1 / 19) | Agreement |
|---|---|---|---|---|---|
| 1 series × 360 months | 0.13 s | 0.19 s | 0.7× | the same chain as Rbeast's generic kernels | |
| 10,000 px × 144 months | 328 s on 1 thread, 34.5 s on 19 | 217 s | 18.1 s | 1.5× / 1.9× | the same chain as Rbeast's generic kernels |
The cube's speedups compare each Zeit time with Rbeast on as many threads: Rbeast's beast123() is multithreaded too. On one series Zeit is slower than Rbeast's AVX2 kernels and as fast as its generic ones (0.19–0.21 s): it runs the same scalar arithmetic. On a cube it is faster, because beast123() costs Rbeast more per pixel than beast() does (33 ms against 21 ms here, on one thread).
How they were checked. With Rbeast's generic kernels (cputype = "sse"), Rbeast and Zeit run the same chain. The script checks that the trend and the number of changepoints of the series and of the cube's first pixel are identical to the last bit, and the tests compare every output on six series (see Fidelity). The timed AVX2 runs sum in another order, so their chains drift apart, and the outputs agree only in distribution:
- the trends differ by a median of 2e-3 standard deviations of the series;
- Rbeast's mean number of trend changepoints is 0.31 higher on the cube.
Most of that gap is Rbeast's own. Zeit fits each pixel as beast() fits one series, from the seed, and its mean equals beast()'s exactly. beast123() carries its random stream from one pixel to the next and finds more changepoints: 0.06 more on average over 200 of these pixels, with the same generic kernels. Rbeast's 19-thread run also left one pixel unfitted (NaN).
Windows' power throttling
Windows 11 slows down the processes it considers to be in the background (power throttling, or EcoQoS): after a few seconds of work, it moves them to the efficiency cores of a hybrid processor and lowers their clock. A benchmark run from a terminal is such a process. Zeit's unpinned threads ran 2.4 times slower for it. Rbeast's threads, which it pins to processors, were slowed less. The script takes both processes out of power throttling, so the times are of the computation. Zeit's one-thread runs are pinned to the first processor, as Rbeast pins its own.
AR(1) trends and PARTS vs remotePARTS¶
zeit.ar_trend and zeit.parts against the R package remotePARTS 1.0.4: fitAR_map, the trend of every pixel's series with AR(1) errors, and fitGLS_partition, the test of the region's mean trend by generalized least squares on partitions of the pixels, with an exponential spatial correlation (range 8 pixels), its nugget estimated in each partition, and 6 pairs of partitions crossed. The series are annual, 30 years, with a slope that changes across the map and AR(1) errors whose innovations are spatially correlated. Zeit ran these on 1 thread and on 8 of the 20 logical processors, not on 19.
| Workload | fitAR_map | Zeit, 1 thread | Zeit, 8 threads | Speedup (1 / 8) | Agreement |
|---|---|---|---|---|---|
| 50 × 50 px × 30 years | 4.09 s | 7.2 ms | 4.1 ms | 568× / 1,007× | slope max diff 5.3e-8 of the largest, ρ 1.0e-6 |
| 100 × 100 px × 30 years | 16.6 s | 20.2 ms | 8.5 ms | 821× / 1,960× | slope 6.9e-8, ρ 3.4e-6 |
| 200 × 200 px × 30 years | 66.8 s | 71.8 ms | 23.1 ms | 931× / 2,888× | slope 1.1e-7, ρ 7.7e-6 |
| Workload | fitGLS_partition | Zeit, 1 thread | Zeit, 8 threads | Speedup (1 / 8) | Agreement |
|---|---|---|---|---|---|
| 2,500 px in 5 partitions of 500 | 2.47 s | 0.120 s | 0.030 s | 21× / 84× | partitions max rel. diff 2.7e-11; overall t 0.85% apart |
| 10,000 px in 6 partitions of 1500 | 72.5 s | 3.61 s | 1.15 s | 20× / 63× | partitions 1.6e-10; overall t 0.97% apart |
| 40,000 px in 26 partitions of 1500 | 262 s (4.4 min) | 12.8 s | 3.51 s | 20× / 75× | partitions 4.7e-15; overall t 0.028% apart |
Each partition is a GLS on a dense 1500 × 1500 correlation matrix, so the cost is the same linear algebra on both sides; Zeit's gain on one thread (20×) is smaller than on the per-pixel AR fits, and its threads share the partitions.
How they were checked. The AR(1) fits agree to the tolerance of the two optimizers (the script requires 1e-3 of the largest slope and 1e-2 on ρ; the largest differences were 1.1e-7 and 7.7e-6). The GLS is checked on remotePARTS' own slopes and partitions, read back from R (Zeit's timed runs draw their own partitions of the same size): every partition's estimate, standard error and nugget, and the overall estimate, must agree to 1e-6 (the largest difference was 1.6e-10). The overall t statistic is 0.03% to 1% apart because remotePARTS 1.0.4 computes the correlation between pairs of partitions after the first without their nugget, and Zeit with it (see Fidelity).
Spatial cross-validation vs blockCV and CAST¶
zeit.autocorrelation_range against the R package blockCV 4.0.1's cv_spatial_autocor (5000 pixels, which fits its variograms with automap); zeit.spatial_folds with method="knndm" against CAST 1.0.4's knndm (k = 5, its example's clustered points in a square of 100, 1000 regular prediction points); zeit.error_profile against CAST's errorProfiles (with scam 1.2.23) of a linear model cross-validated in 5 folds by caret; and the Mahalanobis distance of train_di and aoa against CAST's trainDI and aoa with method = "MD". The maps are white noise of 2 bands averaged over windows of 3, 7 and 15 pixels of 100 m, so correlated up to 300, 700 and 1500 m. Zeit ran on 1 thread and on 8.
| Workload | Original | Zeit, 1 thread | Zeit, 8 threads | Speedup (1 / 8) | Agreement |
|---|---|---|---|---|---|
autocorrelation_range, 300 × 300 px, range 300 m |
2.74 s | 287 ms | 50 ms | 10× / 55× | range 0.89–0.99 × blockCV's |
autocorrelation_range, 300 × 300 px, range 700 m |
1.87 s | 282 ms | 49 ms | 7× / 38× | range 1.00–1.02 × blockCV's |
autocorrelation_range, 300 × 300 px, range 1500 m |
1.63 s | 278 ms | 50 ms | 6× / 33× | range 0.98–1.01 × blockCV's |
| kNNDM, 1000 clustered points | 1.81 s | 80 ms | 73 ms | 23× / 25× | W 0.572 (CAST's folds: 0.685) |
| kNNDM, 5000 clustered points | 40.2 s | 565 ms | 529 ms | 71× / 76× | W 0.267 (CAST's folds: 0.269) |
error_profile, 2000 points, 5 folds |
0.45 s | 22.7 ms | 10.2 ms | 20× / 44× | windows max diff 5.3e-15; curve within 1.2% |
| Mahalanobis DI, 500 training × 10,000 new | 20.8 s | 15.3 ms | 3.0 ms | 1,359× / 6,867× | DI max diff 3.0e-14 |
| Mahalanobis DI, 2000 × 100,000 | 909 s (15.1 min) | 437 ms | 68 ms | 2,082× / 13,294× | DI max diff 3.3e-13 |
How they were checked.
- The range. blockCV and Zeit each draw their own 5000 pixels, so the ranges agree only to the sample: within 2% at 700 and 1500 m, and 1–11% shorter at 300 m, three pixels. The bands are automap's on both sides (2% to 100% of 0.35 of the map's diagonal), and the model is the one that fits best of the spherical, exponential and Gaussian (automap also tries Stein's Matérn). blockCV chose the spherical on every map, Zeit the Gaussian at 300 m. With an exponential or a Gaussian, blockCV reports gstat's range parameter, Zeit's
scale, where Zeit'srangeis the practical range. - kNNDM. The two draw different folds (each side clusters its own candidates), so the folds are not compared one by one. The script measures CAST's own folds with Zeit's W, which must equal CAST's to 1e-9, and records the W of each side's folds: smaller is better, closer to the distances from the area to predict. 8 threads gain little: they share only the core's distances, not the clustering of the candidates.
- The error profile. The windows of the DI and their RMSE, from CAST's predictions and variable weights, must equal CAST's to 1e-9 (the largest difference was 5.3e-15). The monotone curve is Zeit's own P-spline, not scam's: within 1.2% of the range of the metric, on 200 DI values.
- The Mahalanobis distance. The DI of every new point, to 1e-9. CAST takes 750 times as long with the Mahalanobis distance as with the Euclidean on the same 2000 × 100,000 points (909 s against 1.2 s); Zeit whitens the predictors once and searches the same k-d tree as for the Euclidean distance.
The kNNDM rows are the best of two runs, not three.
Areas of transitions vs survey¶
zeit.transitions against R's survey 4.5 (Lumley), the reference for design-based estimators, with terra for the mapped areas. The landscape is made up: 2048 × 2048 pixels of 30 m with Forest, Pasture, Crop and Water at two dates; patches are cleared, cropped and regrown between the dates. Two maps of the dates have errors: 6% of the classes are wrong, 80% of the changes are found and 1.5% of the pixels are false change. The strata are not the transitions: they come from an older alert map (its alerts, a buffer of 2 pixels and the rest), with 250 + 200 + 150 points whose reference is the truth.
The R script counts the transitions inside the strata with terra's freq. It then builds svydesign(ids = ~1, strata = ~h, weights = ~A_h / n_h) and estimates each transition's area with:
svytotal(direct);- the mapped area plus
svytotalof y − x (difference); svyratio(~y, ~x)times the mapped area (ratio).
The user's and producer's accuracies come from svyratio and the overall accuracy from svymean. Its timing covers reading the three rasters, so both sides read them from files.
| Workload | survey + terra | Zeit | Speedup | Agreement |
|---|---|---|---|---|
| direct, 2048 × 2048 px, 600 points, 16 transitions | 0.48 s | 0.18 s | 2.7× | max diff 5.7e-15 |
| difference | 0.40 s | 0.21 s | 1.9× | max diff 7.6e-15 |
| ratio | 0.41 s | 0.19 s | 2.2× | max diff 5.7e-15 |
How they were checked. For every transition, the script compares the mapped area, the estimate and its standard error, and the user's and producer's accuracies with their standard errors. It also compares the overall accuracy. Each must agree with survey's to 1e-9 of the value (or of 1 for values below 1). A value survey leaves undefined (NA) must be NaN in Zeit, for example the ratio estimate of a transition no point is mapped as. The fixtures of tests/test_accuracy_survey.py (tests/data/make_survey_fixtures.R) hold the same estimators to 1e-10 on a smaller landscape.
Most of the time on both sides is reading the rasters and counting the pairs, not the estimates. A map of integer classes of up to 2^26 pixels is read once into memory, and Zeit counts its classes, the pairs and the pixels of each stratum per row with a few bincounts; larger maps stay lazy and are counted through dask. Before this, Zeit read each raster several times through dask and took 0.73–0.85 s, 0.6× survey's speed. The Janus benchmark of the same estimators (its tools/bench_validation_vs_survey.py) has the count in Rust.
Protocol¶
- The same inputs. Each script draws its inputs with a fixed seed, writes them to a temporary folder (CSV or raw float64), runs the original on the files from
Rscript, and runs Zeit on the same arrays in memory. - The original's own timing. Each R script (
benchmarks/vs_originals/<method>.R) times only the call it benchmarks withsystem.time(elapsed), after reading the inputs and loading the packages, and prints it; starting R, loading the packages and reading and writing files are not counted. Each original is timed once, with its defaults otherwise, in one R process. - Zeit's timing. A warm-up call, then the best of three (
best_ofinbenchmarks/vs_originals/common.py,time.perf_counter), on 1 thread and on all the processors but one (n_jobs=-1: 19 here; the AR(1) trends, PARTS and the spatial cross-validation, timed later, on 8). - The check. After timing, each script compares the outputs (the tolerances above) and stops with an error if they disagree, so no timing is recorded for a result that differs.
- The record.
recordincommon.pywrites one entry per method, original, workload and thread count tobenchmarks/results/vs-originals.json, with the date and the machine, replacing an older entry of the same key.
Machine¶
| Platform | Windows 11 (Windows-11-10.0.26300-SP0) |
| Processor | Intel64 Family 6 Model 183 Stepping 1, GenuineIntel (the Intel Core i5-13600K of the other benchmarks), 20 logical processors |
| R | 4.4.2, with sfdep 0.2.5, RecoverMetricsR 0.1.0, RStoolbox 1.0.2.2, sits 1.5.4, CAST 1.0.4, Rbeast 1.0.2, remotePARTS 1.0.4, blockCV 4.0.1 (with automap 1.1.20), scam 1.2.23 and survey 4.5 |
| Python | pyxccd 1.1.0, in a virtual environment of its own |
| Date | 2026-10-24 (the AR(1) trends and PARTS: 2026-10-28; the spatial cross-validation: 2026-10-29; the transitions: 2026-11-03) |
Rerunning the benchmarks¶
From the repository root, with Zeit installed and R with the packages above:
python benchmarks/vs_originals/hotspots.py # sides 20, 50, 100 (or give your own: ... hotspots.py 20 50)
python benchmarks/vs_originals/recovery.py
python benchmarks/vs_originals/unmix.py
python benchmarks/vs_originals/smooth_probs.py
python benchmarks/vs_originals/aoa.py
python benchmarks/vs_originals/beast.py
python benchmarks/vs_originals/parts.py # sides 50, 100, 200; ZEIT_BENCH_THREADS=8 by default
python benchmarks/vs_originals/folds.py # autocor knndm profile mahalanobis (CAST's MD takes 15 min)
python benchmarks/vs_originals/accuracy_survey.py # zeit.transitions against survey (and terra)
python benchmarks/vs_originals/ewmacd.py "path/to/EWMACD v1.3.0.r" # Brooks' script, not distributed
PYXCCD_PYTHON=path/to/pyxccd-env/python python benchmarks/vs_originals/sccd.py CCDC_Stack_Rondonia.tif CCDC_Stack_Rondonia_Dates.csv
The scripts look for Rscript at C:\Program Files\R\R-4.4.2\bin\Rscript.exe on Windows and on the PATH elsewhere; set the RSCRIPT environment variable to use another. Brooks' script is in EWMACD v1.3.0.7z at VTechWorks. hotspots.py runs sfdep with 199 permutations only up to 20 × 20 cells (the 50 × 50 run above took sfdep 13 minutes), and with 9 on every grid. Each script prints its rows as it records them; the chart above is redrawn from the JSON by python docs/scripts/make_vs_originals_chart.py.
What is not here
These are the benchmarks of the Zeit library. Janus, the desktop application built on it, runs the same core; the comparison of its tools with the originals is kept with Janus, not on this page. The earlier ports (LandTrendr, CCDC, BFAST, TWDTW, SOM, the deep learning models...) are compared in Performance and Fidelity.