criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.3976617967528 | 4.580245869835077 | 4.690652634834502 |
| Standard deviation | 2.0794757969139682e-2 | 0.18317074653147275 | 0.2549735019710949 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6675696230086032 | 0.776883628917858 | 0.8192565366662166 |
| Standard deviation | 1.2881721668236423e-2 | 7.521519682382125e-2 | 9.785603877944797e-2 |
Outlying measurements have moderate (0.2239540142307619%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5728419570077676 | 0.7681413189372203 | 0.8344206724335284 |
| Standard deviation | 2.1716336195822805e-3 | 0.1302543827949456 | 0.15306561033123142 |
Outlying measurements have moderate (0.4660376599356107%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.5801129161227436 | 1.638494110979688 | 1.6968753058366324 |
| Standard deviation | 2.8122884172868567e-2 | 7.123443686916883e-2 | 8.47887115194883e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.458594955917458 | 7.591744953232289 | 7.7047894956303935 |
| Standard deviation | 6.56492755650396e-2 | 0.14237842752863175 | 0.1865730113671512 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.24346171224169666 | 0.29694214805992186 | 0.3604057934371667 |
| Standard deviation | 2.5375611757529754e-2 | 7.254570467017031e-2 | 9.42065435060008e-2 |
Outlying measurements have moderate (0.48363860506153616%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.17461550717772398 | 0.18944588508966262 | 0.20849634994576463 |
| Standard deviation | 1.569522586955224e-2 | 2.3784425270449112e-2 | 3.171105578271425e-2 |
Outlying measurements have moderate (0.3149759280494951%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.306459503251972 | 4.3445040013702965 | 4.409809621232853 |
| Standard deviation | 2.0168036399387225e-2 | 6.461284298768485e-2 | 8.687994273235794e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4936425999912899 | 0.6571480872874721 | 0.7369708960407783 |
| Standard deviation | 5.042403080248448e-3 | 0.1159703503501392 | 0.1448525063892411 |
Outlying measurements have moderate (0.4686431375207356%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5406904855008179 | 0.6450949619999543 | 0.7018816854979377 |
| Standard deviation | 7.058276779382297e-3 | 9.861198970812411e-2 | 0.12408504361244792 |
Outlying measurements have moderate (0.4582085056767785%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6022417258294201 | 1.6841156938256365 | 1.8446282469958533 |
| Standard deviation | 3.660607339424908e-2 | 0.11311933452720711 | 0.15247375645470423 |
Outlying measurements have moderate (0.19588592803632218%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.58493769074812 | 6.620376440955321 | 6.644235484871388 |
| Standard deviation | 1.6041421000409173e-2 | 3.979022014167728e-2 | 5.510010171019219e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2515953716627943 | 0.2982813578437587 | 0.35697508683127427 |
| Standard deviation | 3.8095810632345184e-2 | 6.603029093540469e-2 | 8.76416070375693e-2 |
Outlying measurements have severe (0.5770428314038193%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1561293585609833 | 0.164561625159972 | 0.18259900182893032 |
| Standard deviation | 7.5624099598799e-3 | 1.6811815628891995e-2 | 2.5304033092287445e-2 |
Outlying measurements have moderate (0.26825550960960437%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3199730796150106 | 1.3580774740979298 | 1.3881405784547192 |
| Standard deviation | 1.6082580252259504e-2 | 4.207999164388563e-2 | 5.8021274394519734e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.43143411619530525 | 0.5400259461869913 | 0.6118596779948955 |
| Standard deviation | 1.559817753459991e-2 | 0.10297986384419905 | 0.13319013332228796 |
Outlying measurements have moderate (0.4731450301070336%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4558591820241418 | 0.5480388855697432 | 0.5840787139398648 |
| Standard deviation | 9.259897824809987e-3 | 6.457262765774384e-2 | 8.323041844415034e-2 |
Outlying measurements have moderate (0.23241404805800214%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0703579871860711 | 1.143414839269705 | 1.2008335111653046 |
| Standard deviation | 3.611483016478993e-2 | 7.104937861019968e-2 | 8.438943748280135e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.76655899699108 | 6.942534358871853 | 7.055942546625374 |
| Standard deviation | 0.10505871722562435 | 0.1873840556269955 | 0.26190499091121167 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22487636908772402 | 0.26446672986174236 | 0.29957753844733814 |
| Standard deviation | 3.267070189492419e-2 | 4.594319802938497e-2 | 5.703480954979128e-2 |
Outlying measurements have moderate (0.38343201663351567%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1514763199549238 | 0.15884351324760054 | 0.16654534626365672 |
| Standard deviation | 8.464667550474552e-3 | 1.1199514037103366e-2 | 1.5665814882426743e-2 |
Outlying measurements have moderate (0.13369353454947525%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.47908937224929105 | 0.5098715282286624 | 0.5288219204358029 |
| Standard deviation | 2.664349095497276e-2 | 3.299578811030444e-2 | 3.82841334439582e-2 |
Outlying measurements have moderate (0.1917032206707002%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.41898490768289776 | 0.5002750328543092 | 0.5605045527481707 |
| Standard deviation | 4.832683137647109e-2 | 9.0424375114357e-2 | 0.12754287963493235 |
Outlying measurements have moderate (0.4701085715723889%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3824560736793501 | 0.4800620583106744 | 0.533098082687502 |
| Standard deviation | 5.644042748220557e-3 | 9.508181747163272e-2 | 0.11596427260283006 |
Outlying measurements have moderate (0.47510570338117025%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.7043470200642332 | 0.7724986868979613 | 0.8067258952305565 |
| Standard deviation | 4.5371589622969125e-2 | 6.816495751259058e-2 | 7.904435177773292e-2 |
Outlying measurements have moderate (0.218644537211809%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.84601562483537 | 7.018774695578638 | 7.260606707232 |
| Standard deviation | 0.127825441662329 | 0.29171230520736524 | 0.3683523994691031 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.20281581312379177 | 0.26140599917057744 | 0.3047984937487248 |
| Standard deviation | 5.3990731199718596e-2 | 6.712310453531783e-2 | 8.349185156915945e-2 |
Outlying measurements have severe (0.5829376222406907%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.134120285696628 | 0.14804233573388023 | 0.1666703801995621 |
| Standard deviation | 1.5481203347788318e-2 | 2.15466052123966e-2 | 2.98291476537666e-2 |
Outlying measurements have moderate (0.40921690150107853%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2791910446365364 | 0.3069174625382099 | 0.328888392780694 |
| Standard deviation | 2.5786294650042833e-2 | 3.502158537570358e-2 | 4.1360404895719474e-2 |
Outlying measurements have moderate (0.19039977068621936%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3190263290016446 | 0.38826705074946705 | 0.4142190001233151 |
| Standard deviation | 1.0611899779178202e-4 | 4.8885978867708235e-2 | 5.993345982794682e-2 |
Outlying measurements have moderate (0.2345995825072503%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3020056669774931 | 0.38405403109754843 | 0.415287168476425 |
| Standard deviation | 2.436142301882549e-3 | 5.792579478503943e-2 | 7.212984109332378e-2 |
Outlying measurements have moderate (0.45707199593340925%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5905063544972412 | 0.6256520915024643 | 0.6604676910064882 |
| Standard deviation | 3.397855861744806e-2 | 4.041726728790781e-2 | 4.350657422155787e-2 |
Outlying measurements have moderate (0.19149783201131637%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.843591555915434 | 6.86288367979311 | 6.89665556679635 |
| Standard deviation | 5.699967330050182e-3 | 3.219166618237627e-2 | 4.084869462183792e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22988654680326967 | 0.25507452439002615 | 0.2762690341905302 |
| Standard deviation | 2.4752983320209466e-2 | 3.1412721941065466e-2 | 3.6406536448544205e-2 |
Outlying measurements have moderate (0.36703196706800073%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1381402349299086 | 0.15266698784131333 | 0.16765510060673589 |
| Standard deviation | 1.7382125659150472e-2 | 2.2184118794515838e-2 | 3.1625465691599394e-2 |
Outlying measurements have moderate (0.4091547775564313%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21303411480039358 | 0.23883424907689915 | 0.26103563524375206 |
| Standard deviation | 2.4920963726871272e-2 | 3.169717269768169e-2 | 3.638843438305501e-2 |
Outlying measurements have moderate (0.3716128258141996%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.30458693068248976 | 0.3375794067681757 | 0.3594115271650177 |
| Standard deviation | 1.930762490675075e-2 | 3.470937631096206e-2 | 4.242134249943774e-2 |
Outlying measurements have moderate (0.22690604456327582%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3250991966233414 | 0.354114176064589 | 0.400536966690197 |
| Standard deviation | 7.481308128869074e-3 | 4.812041790006728e-2 | 5.836855423955686e-2 |
Outlying measurements have moderate (0.23677889064478963%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4115544283316315 | 0.4705768343950088 | 0.49504906031324936 |
| Standard deviation | 9.730492754897568e-3 | 4.231385135110113e-2 | 5.651618488626205e-2 |
Outlying measurements have moderate (0.219804930908093%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.575342137669698 | 6.617263510040478 | 6.65165318561958 |
| Standard deviation | 2.8077322793072325e-2 | 4.205032941380765e-2 | 4.922914652471623e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.199427626656446 | 0.23377083130947235 | 0.25616228707891425 |
| Standard deviation | 1.1706302883579277e-2 | 3.304098400824048e-2 | 4.75531363420749e-2 |
Outlying measurements have moderate (0.3749709306703501%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13955819933539806 | 0.1568303849271516 | 0.17056141833572405 |
| Standard deviation | 1.381979840496758e-2 | 2.284403355305776e-2 | 3.3961503682415475e-2 |
Outlying measurements have moderate (0.4092480414987286%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1868071110325723 | 0.20633040906064304 | 0.22113863070133244 |
| Standard deviation | 1.2762517782517555e-2 | 2.2720607826333394e-2 | 3.348521122893585e-2 |
Outlying measurements have moderate (0.30947100299801766%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.26617002405808304 | 0.29145292044762755 | 0.33031734490359665 |
| Standard deviation | 1.9958127576324806e-2 | 3.924625722381013e-2 | 5.918806895027377e-2 |
Outlying measurements have moderate (0.3724253292616143%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2652689892041963 | 0.29718845843162855 | 0.32977896120864897 |
| Standard deviation | 2.3471398756361072e-2 | 4.2383832821485175e-2 | 6.0902493142580796e-2 |
Outlying measurements have moderate (0.3754170823161287%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3295937055857697 | 0.3678611670426714 | 0.42527587874792516 |
| Standard deviation | 1.1787882745880433e-2 | 5.476910087343018e-2 | 7.219073988988625e-2 |
Outlying measurements have moderate (0.4559448474554639%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.139644610817413 | 5.234719579651937 | 5.294328857504297 |
| Standard deviation | 4.8203543410636485e-3 | 8.976535854493259e-2 | 0.11256615057601796 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16839435716732018 | 0.2158338725246722 | 0.23244976546411633 |
| Standard deviation | 3.328244606336718e-3 | 3.8654444324706924e-2 | 5.393450151247229e-2 |
Outlying measurements have moderate (0.47970219287915533%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.11999421220222886 | 0.14520718303060026 | 0.15459212327910965 |
| Standard deviation | 3.8864319574950235e-3 | 2.0699431439038584e-2 | 3.1695931590625656e-2 |
Outlying measurements have moderate (0.4083957670742975%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1403916743988197 | 0.17136823296952772 | 0.18863756059905024 |
| Standard deviation | 1.3673975307139367e-2 | 3.1586424962755884e-2 | 4.504655752452047e-2 |
Outlying measurements have moderate (0.4808367805323243%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.17377450811472953 | 0.19506293680840625 | 0.21758354855248604 |
| Standard deviation | 1.919282979404465e-2 | 2.7922071551528774e-2 | 3.795408654309169e-2 |
Outlying measurements have moderate (0.31921184368690875%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1805165980945781 | 0.19853089655710696 | 0.2232588670142124 |
| Standard deviation | 1.717650913300741e-2 | 2.8403233415887703e-2 | 4.682180969247138e-2 |
Outlying measurements have moderate (0.3191966702308556%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.23005153305159184 | 0.2521265631825857 | 0.2730250270656931 |
| Standard deviation | 1.631748975897598e-2 | 2.916033432278672e-2 | 3.886211044359969e-2 |
Outlying measurements have moderate (0.3626214702909041%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.4528209939962835 | 2.5642849521658113 | 2.730279202500242 |
| Standard deviation | 3.795577733641653e-2 | 0.16188014543647905 | 0.21358742849150886 |
Outlying measurements have moderate (0.18873880729835915%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13842618489757622 | 0.1689195827959338 | 0.19040606437613153 |
| Standard deviation | 1.7140791167249116e-2 | 3.478795440907375e-2 | 5.0850985074277495e-2 |
Outlying measurements have moderate (0.4846499073882457%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.11859058712721568 | 0.1443840351048184 | 0.1558193418661213 |
| Standard deviation | 2.943007282376491e-3 | 2.339231135891558e-2 | 3.607477193880235e-2 |
Outlying measurements have moderate (0.4129521671658088%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.