criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.417237195662892 | 4.473773733954658 | 4.627552646998083 |
| Standard deviation | 1.4215026640158612e-3 | 0.10279708310128426 | 0.1222463873888335 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6169145668682177 | 0.7420969488290817 | 0.8120493234552365 |
| Standard deviation | 2.944473458289093e-2 | 0.12053703122170306 | 0.16154808928345604 |
Outlying measurements have moderate (0.46298477839701163%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.575763442000607 | 0.7203239119153295 | 0.7771242940398224 |
| Standard deviation | 2.7299531087919604e-2 | 0.1018737937691928 | 0.13117486363859834 |
Outlying measurements have moderate (0.23779407332397093%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.507222936495964 | 1.5557039521039162 | 1.6272076737326036 |
| Standard deviation | 5.936863376716853e-3 | 6.75644506077525e-2 | 8.553381754912383e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.8856739305047086 | 6.971129978690442 | 7.119725316439144 |
| Standard deviation | 4.066979150229599e-2 | 0.14652823125016026 | 0.19506383157080964 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.26027220647083593 | 0.2977141591691179 | 0.33197358949983025 |
| Standard deviation | 3.7362901540579446e-2 | 4.762826794920077e-2 | 6.038629666381357e-2 |
Outlying measurements have moderate (0.3804638308038593%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.14929447962922682 | 0.15585073832866775 | 0.16524199652093538 |
| Standard deviation | 9.920365564025055e-3 | 1.2051047122317811e-2 | 1.3907024286648724e-2 |
Outlying measurements have moderate (0.13523821143031278%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.358151966247533 | 4.381643011034612 | 4.41666465673552 |
| Standard deviation | 6.461914750380583e-3 | 3.629301422052535e-2 | 4.778832937508156e-2 |
Outlying measurements have moderate (0.18749999999999994%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5116367998816713 | 0.6003142984845908 | 0.6466062389008584 |
| Standard deviation | 3.1823346216697246e-3 | 8.680765176496755e-2 | 0.10690665704256888 |
Outlying measurements have moderate (0.23832434077654152%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5041186109883711 | 0.5998825854567258 | 0.6354458979603805 |
| Standard deviation | 3.0171076057620844e-3 | 6.607784991274267e-2 | 8.21299521890394e-2 |
Outlying measurements have moderate (0.22987848138033629%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3845535034361092 | 1.508562632476848 | 1.580948537350802 |
| Standard deviation | 4.1525361414339046e-2 | 0.12313671038723419 | 0.16716808665804825 |
Outlying measurements have moderate (0.2133569471043632%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.42569538357687 | 6.532183417955214 | 6.5926918316642205 |
| Standard deviation | 3.5768224240460417e-3 | 0.10548200690988624 | 0.14331501819657438 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.24280950853620503 | 0.2741657199226514 | 0.31163312401913573 |
| Standard deviation | 2.4835250414834975e-2 | 4.473446569378466e-2 | 6.041586013824574e-2 |
Outlying measurements have moderate (0.38121929894811735%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13791991215255459 | 0.14598973426027628 | 0.15950432665045983 |
| Standard deviation | 5.317608919012111e-3 | 1.5125133532781668e-2 | 2.2014714503786108e-2 |
Outlying measurements have moderate (0.26873841346461314%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.221512511993448 | 4.275460056688341 | 4.294187849440277 |
| Standard deviation | 9.580987461959012e-4 | 3.6087123708985605e-2 | 4.324998474663219e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3589094239869155 | 0.4552881901636283 | 0.49572397654083034 |
| Standard deviation | 2.3727998919639504e-2 | 7.110637715505351e-2 | 9.338244859908716e-2 |
Outlying measurements have moderate (0.45996353638917675%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.38261823033584125 | 0.4504519203358844 | 0.4878629465850584 |
| Standard deviation | 2.4757890832309698e-2 | 6.5603528786539e-2 | 8.639706430248312e-2 |
Outlying measurements have moderate (0.23848980883983123%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.0206158498331206 | 2.111657575394929 | 2.1649728989541472 |
| Standard deviation | 4.036089470901061e-2 | 8.758417595972988e-2 | 0.11112610822592635 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.436230917865032 | 6.92317690130767 | 7.410122884750308 |
| Standard deviation | 0.3822187850953972 | 0.5863879722155639 | 0.7008234453668711 |
Outlying measurements have moderate (0.21596852267199776%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21921888820361346 | 0.2451507596115698 | 0.2604973407133366 |
| Standard deviation | 1.846857412218963e-2 | 2.7390595311396246e-2 | 3.532244633087414e-2 |
Outlying measurements have moderate (0.18998678464241436%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12947232774315906 | 0.1359691647315187 | 0.14324298176000283 |
| Standard deviation | 5.510099500175802e-3 | 9.961252255933545e-3 | 1.4498581760628433e-2 |
Outlying measurements have moderate (0.2272561993916807%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.0178996339964215 | 4.053568780457378 | 4.067780734708019 |
| Standard deviation | 4.5531374780694023e-4 | 2.6863841534549317e-2 | 3.256034932529407e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3218664806081506 | 0.3759393617013605 | 0.40583469345559325 |
| Standard deviation | 1.1435564830511197e-2 | 5.200428053802305e-2 | 6.90403113432778e-2 |
Outlying measurements have moderate (0.23724156320191792%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3340437003401651 | 0.3849179739621225 | 0.41161452733589005 |
| Standard deviation | 5.037666251155315e-3 | 4.8142525166344256e-2 | 5.8888377542497636e-2 |
Outlying measurements have moderate (0.2343930261673901%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.222715272582718 | 2.3102532023967797 | 2.38525667241629 |
| Standard deviation | 5.9331837325280405e-2 | 0.1033017795621453 | 0.1428283520445235 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.419974430507864 | 6.508028266065594 | 6.597396892379038 |
| Standard deviation | 5.630399499204941e-2 | 9.661428881784859e-2 | 0.13657987482881134 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21030354715844926 | 0.23957934530995165 | 0.26352034156327137 |
| Standard deviation | 2.6333544965851472e-2 | 3.421074697413362e-2 | 4.224755317872165e-2 |
Outlying measurements have moderate (0.3754786916704574%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12137866715068715 | 0.12549832985073192 | 0.12935387413764537 |
| Standard deviation | 5.0531967637695765e-3 | 6.732664865205295e-3 | 9.539153382707423e-3 |
Outlying measurements have moderate (0.11439643352013%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.7969634939872776 | 3.837575772371565 | 3.8764701011223224 |
| Standard deviation | 2.3275969749877588e-2 | 5.201781687144164e-2 | 7.179053927121462e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.25702833401737735 | 0.32590073125356867 | 0.35079570899870305 |
| Standard deviation | 4.242934996000258e-3 | 4.6370402421345484e-2 | 5.7492488410492315e-2 |
Outlying measurements have moderate (0.23794050860121152%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2815533491302631 | 0.3203277907305164 | 0.343505840144644 |
| Standard deviation | 1.0406796587631106e-2 | 3.989603994465537e-2 | 5.352741227414807e-2 |
Outlying measurements have moderate (0.23426126913692727%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.413077852810602 | 2.4364237050995143 | 2.4533988149050856 |
| Standard deviation | 1.3319402241904754e-2 | 2.392134400107499e-2 | 3.0092520490396117e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.576663971325615 | 6.593133020260211 | 6.5991672041727725 |
| Standard deviation | 7.050054337098249e-4 | 1.1211993541530315e-2 | 1.3935350825027846e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.20969559181281106 | 0.2288191708251058 | 0.24030270760736636 |
| Standard deviation | 6.7216881937703e-3 | 2.1005908664087473e-2 | 3.135737824129309e-2 |
Outlying measurements have moderate (0.15808309289001182%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12097570497802995 | 0.12443156556537793 | 0.1352688350846057 |
| Standard deviation | 4.382000518383064e-3 | 9.147317776836996e-3 | 1.3579958248342329e-2 |
Outlying measurements have moderate (0.22741169445544548%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.449462037810008 | 3.466336404562753 | 3.481539401502232 |
| Standard deviation | 1.3716971775359165e-2 | 1.78913967950617e-2 | 2.0981166856305366e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21099585944757565 | 0.24862578056704177 | 0.2624675477667673 |
| Standard deviation | 4.318109762051003e-3 | 2.8735104329056325e-2 | 3.790724065512394e-2 |
Outlying measurements have moderate (0.36256853999322586%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21283278543152848 | 0.2478975603020323 | 0.26095573037096376 |
| Standard deviation | 7.365694367681003e-4 | 2.6141403111223867e-2 | 3.424201968813722e-2 |
Outlying measurements have moderate (0.18875921270221618%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.3344951075026374 | 2.3670514004995007 | 2.4152115714956985 |
| Standard deviation | 2.0291912242336077e-2 | 5.014635351898138e-2 | 6.889953370001668e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.593965920008486 | 6.650854688565232 | 6.6715615885641455 |
| Standard deviation | 3.785574375797296e-3 | 3.831814130050046e-2 | 4.782053715382099e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19167833690104696 | 0.20961325104055706 | 0.22293327430112794 |
| Standard deviation | 1.3164283509769288e-2 | 2.1635367594681335e-2 | 3.252014556578059e-2 |
Outlying measurements have moderate (0.30617299491913275%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.11430533746133734 | 0.11684180535691363 | 0.11942080780948648 |
| Standard deviation | 3.000279678489111e-3 | 4.055096133949978e-3 | 5.088939628285041e-3 |
Outlying measurements have moderate (0.10937500000000001%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.8913167237551534 | 2.9239002078969256 | 2.9424863657477545 |
| Standard deviation | 4.120255952785441e-3 | 2.7617012256182672e-2 | 3.880263700072245e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.17851838614458557 | 0.19831227556205702 | 0.2173815042613165 |
| Standard deviation | 1.7995543070315333e-2 | 2.5534486210139803e-2 | 3.723133775666951e-2 |
Outlying measurements have moderate (0.31588028808045615%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.17618388999867926 | 0.1879127318176971 | 0.2067457875518206 |
| Standard deviation | 8.663903393201826e-3 | 2.150078304765593e-2 | 3.159989372757655e-2 |
Outlying measurements have moderate (0.3112314071205418%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.0876941853821336 | 2.1395407276322658 | 2.1765266790025635 |
| Standard deviation | 3.0503045020822926e-2 | 5.361602196637199e-2 | 7.506313841045915e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.580719829994147 | 6.642128283624213 | 6.676195978627827 |
| Standard deviation | 1.3453872754325857e-2 | 5.9038163279797794e-2 | 7.86759715240266e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.17959252908622148 | 0.193417431794094 | 0.20170376093673337 |
| Standard deviation | 3.995045989687154e-3 | 1.5241224218759349e-2 | 2.2758054409926492e-2 |
Outlying measurements have moderate (0.1550168847539006%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.10736497804937244 | 0.11225214026699178 | 0.11855222404437882 |
| Standard deviation | 4.67163462229344e-3 | 9.467269305373342e-3 | 1.1937235130143333e-2 |
Outlying measurements have moderate (0.23283871299507147%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.2342202130657824 | 1.626254413349064 | 2.071334671401322 |
| Standard deviation | 0.2357918795575603 | 0.48037933704870084 | 0.6695723247028087 |
Outlying measurements have severe (0.7248179604972951%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13707553874283326 | 0.13969709527535606 | 0.1443892940921311 |
| Standard deviation | 3.628364610292504e-3 | 5.695400750778175e-3 | 7.5660315502268805e-3 |
Outlying measurements have moderate (0.109375%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13360133750954145 | 0.1387732372883095 | 0.14332558321990896 |
| Standard deviation | 4.48636503533015e-3 | 7.180554258190091e-3 | 1.0068008032896522e-2 |
Outlying measurements have moderate (0.11360155605461678%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.29335176424865494 | 0.36811016081446724 | 0.5056774289350869 |
| Standard deviation | 9.198748642389107e-3 | 0.13383927326267714 | 0.16344837288343697 |
Outlying measurements have severe (0.7333784686713424%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.153640405369515 | 6.182349156681691 | 6.211057907993867 |
| Standard deviation | 1.6119880338020268e-2 | 3.703076866098885e-2 | 4.9648829660954304e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15562501683958718 | 0.16271576385205208 | 0.16712404326541702 |
| Standard deviation | 5.630867999800869e-3 | 8.63530394581152e-3 | 1.3671382954834253e-2 |
Outlying measurements have moderate (0.12668267848740805%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.10683377320857727 | 0.10977092947944807 | 0.11309737789575412 |
| Standard deviation | 3.5641542338860363e-3 | 5.050781431463473e-3 | 7.351101419916234e-3 |
Outlying measurements have moderate (0.1009871602037519%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.