criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.4089594867934165 | 4.467660791917903 | 4.537083853918982 |
| Standard deviation | 2.9583246582963962e-2 | 7.111512165777246e-2 | 9.8630183799594e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6329831222392386 | 0.7569933988294603 | 0.8259420858327454 |
| Standard deviation | 2.777419483269722e-2 | 0.1194127660725658 | 0.15923017200647174 |
Outlying measurements have moderate (0.4607551465911734%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6046977500081994 | 0.7363662739189749 | 0.7896717513795011 |
| Standard deviation | 1.8035437089565676e-2 | 9.199301165294106e-2 | 0.1203445635574971 |
Outlying measurements have moderate (0.2343570680252269%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.482605307486665 | 1.5131518732220381 | 1.5436984389574113 |
| Standard deviation | 1.859367834307102e-2 | 3.797055459724127e-2 | 4.907414140735034e-2 |
Outlying measurements have moderate (0.18749999999999994%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.020212501248656 | 7.175729386872869 | 7.394256350994207 |
| Standard deviation | 3.5140020568160374e-2 | 0.23328271052251173 | 0.32401084254652773 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2721186046643803 | 0.30613910330498284 | 0.3377002801718966 |
| Standard deviation | 3.094059467928083e-2 | 4.433056597492734e-2 | 5.549324787781382e-2 |
Outlying measurements have moderate (0.3761547948137859%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15622713785809797 | 0.16178386001824915 | 0.16988183050784664 |
| Standard deviation | 6.964985237296566e-3 | 1.0014304700983145e-2 | 1.3611357780965347e-2 |
Outlying measurements have moderate (0.13096788655546182%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.366526853002142 | 4.4553510427710235 | 4.5992616376449105 |
| Standard deviation | 2.789460016841394e-2 | 0.13675791922459113 | 0.172920769662114 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5210760790068889 | 0.6087275012517542 | 0.6552983927491975 |
| Standard deviation | 3.1319097499363124e-2 | 8.346950315047696e-2 | 0.10755086696979893 |
Outlying measurements have moderate (0.2370153502021339%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5207422583680454 | 0.5961428499134248 | 0.6358106515805655 |
| Standard deviation | 3.370332857435049e-2 | 7.195020100822362e-2 | 9.160886388273737e-2 |
Outlying measurements have moderate (0.2332398960553908%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4510481166216778 | 1.6665114597089996 | 1.9010045935028757 |
| Standard deviation | 9.389364886297885e-2 | 0.29880316619424835 | 0.403697627664869 |
Outlying measurements have moderate (0.46962290341298707%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.726871077124088 | 6.753601750019395 | 6.801101213203462 |
| Standard deviation | 1.1478691819314657e-3 | 4.586513962949739e-2 | 5.542159030233366e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.23462880619917997 | 0.26472272683080517 | 0.2992796471531619 |
| Standard deviation | 2.056496176949949e-2 | 3.994391744712119e-2 | 5.384174044184294e-2 |
Outlying measurements have moderate (0.3780390487033654%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.14058047091704398 | 0.14726574921987035 | 0.1558034705946782 |
| Standard deviation | 7.83492630929868e-3 | 1.1451671907433342e-2 | 1.5945269929704603e-2 |
Outlying measurements have moderate (0.13532373267692044%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.279695171877393 | 4.285645210626778 | 4.29157366625077 |
| Standard deviation | 2.5876585035198296e-3 | 6.959104278249623e-3 | 9.144069232049027e-3 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.38509171259405167 | 0.4607402080249206 | 0.5026837636463218 |
| Standard deviation | 2.8086958461547823e-2 | 7.342518838207146e-2 | 9.686449251514404e-2 |
Outlying measurements have moderate (0.4615477167660661%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3749399119988084 | 0.4626199455178721 | 0.5107646130782086 |
| Standard deviation | 3.102153020518017e-2 | 7.049135925859804e-2 | 9.213592713129198e-2 |
Outlying measurements have moderate (0.45793925318065953%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.9822236489853822 | 2.164429403642013 | 2.269525141124177 |
| Standard deviation | 7.37694987738542e-2 | 0.14210759141136997 | 0.19329419728669633 |
Outlying measurements have moderate (0.19336402973134753%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.52069138632578 | 6.67877133506651 | 6.861234636750548 |
| Standard deviation | 3.436214341470825e-2 | 0.19700960139448406 | 0.27360436420950274 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22109238185366847 | 0.24014971593598605 | 0.265842400273832 |
| Standard deviation | 1.5586234327112224e-2 | 2.759599310941488e-2 | 3.5538774440586854e-2 |
Outlying measurements have moderate (0.36213463886479835%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1292375958544047 | 0.13572764367846893 | 0.14287060526237924 |
| Standard deviation | 6.006490061312372e-3 | 1.0013228674533053e-2 | 1.3124276841486234e-2 |
Outlying measurements have moderate (0.22757159412615224%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.1154890795005485 | 4.1304572709153335 | 4.150044672749573 |
| Standard deviation | 7.966649284657576e-3 | 1.9417636601610274e-2 | 2.660090808577532e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3343495753457925 | 0.39609252371216525 | 0.428244516206784 |
| Standard deviation | 5.914460734461877e-3 | 5.87114234359752e-2 | 7.425184608700386e-2 |
Outlying measurements have moderate (0.45555242064323853%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.334229041422077 | 0.3857745702650088 | 0.4152504297696093 |
| Standard deviation | 1.2779779601355588e-2 | 5.009656728702889e-2 | 6.80715816783068e-2 |
Outlying measurements have moderate (0.23552257552723252%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.283663318503386 | 2.3571393552932327 | 2.400364996209343 |
| Standard deviation | 2.99650448720645e-2 | 7.247365285297679e-2 | 9.982534167524168e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.613579980005548 | 6.625306969877177 | 6.641940099121712 |
| Standard deviation | 1.2000352287311854e-3 | 1.5756595666507736e-2 | 1.9806300907961285e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.20523290704935787 | 0.2324731585517293 | 0.2576258148188936 |
| Standard deviation | 2.4288412822305698e-2 | 3.195770393549724e-2 | 3.753016928129646e-2 |
Outlying measurements have moderate (0.37354154388797517%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12738428848723787 | 0.1323447641643941 | 0.13793157640234963 |
| Standard deviation | 6.436865879073717e-3 | 8.356963292955264e-3 | 1.0505154984908595e-2 |
Outlying measurements have moderate (0.11734638513169503%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.8022367043813574 | 3.840531968583188 | 3.8668869719125114 |
| Standard deviation | 2.1247779911694353e-2 | 4.350703174945055e-2 | 6.086427306671414e-2 |
Outlying measurements have moderate (0.18749999999999994%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2714100141874951 | 0.319697000061448 | 0.34952700674815185 |
| Standard deviation | 1.666948524199235e-2 | 4.737236841778331e-2 | 6.439631823242795e-2 |
Outlying measurements have moderate (0.45552387224363233%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.287511982365686 | 0.32383469820342725 | 0.3511851533257868 |
| Standard deviation | 1.631994875788223e-2 | 3.516873589596131e-2 | 4.2879807003242475e-2 |
Outlying measurements have moderate (0.22929985933449742%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.4087751925035263 | 2.4815629354555617 | 2.603835571085559 |
| Standard deviation | 5.159337389322853e-2 | 0.11644145199243729 | 0.1495220607212887 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.555473334125054 | 6.58113285270944 | 6.626684828831155 |
| Standard deviation | 6.09952741797315e-4 | 4.384335535132804e-2 | 5.295104773165239e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19819980315959482 | 0.21935848354357101 | 0.23057512665262847 |
| Standard deviation | 9.507421420269044e-3 | 2.1094204545299294e-2 | 3.0476422288015226e-2 |
Outlying measurements have moderate (0.30204309792342193%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1202728420090309 | 0.12382736684114987 | 0.12911233934376337 |
| Standard deviation | 5.4251048511689244e-3 | 7.2845828100362e-3 | 1.0264544432964214e-2 |
Outlying measurements have moderate (0.11618192128146018%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.4680152005021228 | 3.4903858150840583 | 3.49858620095741 |
| Standard deviation | 7.613389990526812e-4 | 1.5000433239924796e-2 | 1.8221179591545594e-2 |
Outlying measurements have moderate (0.18749999999999994%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22221441803558264 | 0.2466092420897136 | 0.2627530050318455 |
| Standard deviation | 6.892148526396868e-3 | 2.5902677410608154e-2 | 3.7323677468469776e-2 |
Outlying measurements have moderate (0.18866975251749096%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22936853238497862 | 0.2527361083323679 | 0.2654314882760324 |
| Standard deviation | 3.0854452222752148e-3 | 2.4498032862015996e-2 | 3.443725022579456e-2 |
Outlying measurements have moderate (0.18669600187489427%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.4552152580787756 | 2.487265229700521 | 2.531500656365097 |
| Standard deviation | 1.0735183577267118e-3 | 4.192234957603607e-2 | 5.161543016388468e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.625805258421072 | 6.6898140432943665 | 6.730808914755471 |
| Standard deviation | 1.805380467643381e-2 | 6.070710963757098e-2 | 7.792387117700128e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.18310947377499864 | 0.20474283092884837 | 0.22049679423297575 |
| Standard deviation | 1.5247129133261565e-2 | 2.556547832238922e-2 | 3.3581013142834557e-2 |
Outlying measurements have moderate (0.314775134279163%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.11577011803284612 | 0.11830613111983222 | 0.12088490777976058 |
| Standard deviation | 2.811930713955977e-3 | 4.224575699725086e-3 | 5.867698824508354e-3 |
Outlying measurements have moderate (0.10937499999999999%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.824950600243028 | 2.9006341508702462 | 2.9763177014974644 |
| Standard deviation | 4.301813450001646e-2 | 9.453398664554237e-2 | 0.11795702072729497 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19299886188754398 | 0.20206526151337634 | 0.2066327804285619 |
| Standard deviation | 2.434925916528644e-3 | 9.504624849953278e-3 | 1.3258839700334e-2 |
Outlying measurements have moderate (0.1388888888888888%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19093008389366636 | 0.20144313893753052 | 0.2056264339627685 |
| Standard deviation | 8.27673303936587e-4 | 8.825497643646657e-3 | 1.2467940038628885e-2 |
Outlying measurements have moderate (0.13888888888888876%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.159711546790883 | 2.2000790439166544 | 2.2329859297497023 |
| Standard deviation | 2.481652519047776e-2 | 4.3591008979682655e-2 | 5.9020633259390096e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.633215477249905 | 6.6611921554582905 | 6.707667921832278 |
| Standard deviation | 3.564731165003665e-3 | 4.4300373062349685e-2 | 5.544795803613343e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16201825560064106 | 0.18319370174997152 | 0.1927233307530211 |
| Standard deviation | 7.207481771629304e-3 | 1.7273010665611935e-2 | 2.323567963808827e-2 |
Outlying measurements have moderate (0.30078631109422793%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.10778706105900956 | 0.11166472937480837 | 0.11757913316315453 |
| Standard deviation | 3.378021131828806e-3 | 8.116214323669876e-3 | 1.2591395218207102e-2 |
Outlying measurements have moderate (0.2059937546980137%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.636124442489745 | 1.5834283707851378 | 2.071539221454562 |
| Standard deviation | 5.820421274708831e-2 | 0.9389197445980316 | 1.127243724113477 |
Outlying measurements have severe (0.7437508282376862%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.18809461002901015 | 0.192822744631202 | 0.20018542844093948 |
| Standard deviation | 3.5983830298265025e-3 | 8.524500864772017e-3 | 1.2342372465487507e-2 |
Outlying measurements have moderate (0.13888888888888873%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.18829347306535862 | 0.1924071985634833 | 0.19812093007688722 |
| Standard deviation | 3.4162243557108013e-3 | 7.039762941191542e-3 | 9.80878420562754e-3 |
Outlying measurements have moderate (0.1388888888888887%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5829632931587792 | 1.1164939953529636 | 1.4286739374808046 |
| Standard deviation | 9.95653538746804e-2 | 0.5244302984280069 | 0.7209486944924433 |
Outlying measurements have severe (0.7400409068241802%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.125801140871772 | 6.146503694269995 | 6.158820534496044 |
| Standard deviation | 7.763671260968122e-3 | 2.167016146056316e-2 | 2.7787085185237644e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1364445761582643 | 0.14944349564576273 | 0.15611194033398637 |
| Standard deviation | 5.301532078967105e-3 | 1.3197331496082754e-2 | 1.9799425644208696e-2 |
Outlying measurements have moderate (0.26234918371251414%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.10480517265305522 | 0.10761669837469129 | 0.10987043962838039 |
| Standard deviation | 3.175278364126128e-3 | 4.032994666405029e-3 | 5.202238189950753e-3 |
Outlying measurements have slight (9.876543209876543e-2%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.