criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.280369428377526 | 4.338460629749534 | 4.382279481746082 |
| Standard deviation | 2.854469875092036e-2 | 5.8889087641728836e-2 | 8.057983713837938e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6312855430005584 | 0.763930933183777 | 0.8123995044309898 |
| Standard deviation | 3.2197141287421682e-3 | 8.928357094136928e-2 | 0.11193337062725951 |
Outlying measurements have moderate (0.23212668373163847%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6207467440108303 | 0.7529765660431925 | 0.8059381436663292 |
| Standard deviation | 2.5319903808131886e-2 | 9.198979327699568e-2 | 0.12230952438970091 |
Outlying measurements have moderate (0.23364224545829207%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6480348433087784 | 1.6762678209009512 | 1.7052260345080867 |
| Standard deviation | 1.853198279301671e-2 | 3.231518185366341e-2 | 4.547723917698996e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.023508335623774 | 7.153054514453591 | 7.460839627980022 |
| Standard deviation | 2.7912144451351678e-2 | 0.2120915165735695 | 0.27126194042817803 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2730977859973791 | 0.3142283449160459 | 0.37031459041827475 |
| Standard deviation | 2.6408665333292447e-2 | 5.3473821998077535e-2 | 6.503277877797117e-2 |
Outlying measurements have moderate (0.46627838955197193%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.18545527921071173 | 0.2023283542566383 | 0.22461291438570088 |
| Standard deviation | 1.305516605836093e-2 | 2.6555880152602594e-2 | 3.928405763243285e-2 |
Outlying measurements have moderate (0.316536849968417%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.2315118597507535 | 4.343437148185937 | 4.405265262873097 |
| Standard deviation | 4.450843107792858e-2 | 0.10803005186598379 | 0.14278591463247448 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4796022609807551 | 0.6314373297846032 | 0.6857580755386152 |
| Standard deviation | 1.4576226079952903e-2 | 0.10310039865190442 | 0.1316474971065316 |
Outlying measurements have moderate (0.4633697518494028%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5357074390049092 | 0.6313161663165374 | 0.6647225038796023 |
| Standard deviation | 5.965129575655245e-3 | 6.414370832729355e-2 | 7.896072961666249e-2 |
Outlying measurements have moderate (0.22635029406457555%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.478124304994708 | 1.5973827060812862 | 1.6458089139153649 |
| Standard deviation | 2.582958386938966e-2 | 8.321138985590798e-2 | 0.11183553651535227 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.334952970624727 | 6.36144633023044 | 6.387939689836154 |
| Standard deviation | 2.154674173529446e-2 | 3.15821678346784e-2 | 3.723365872490935e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.255650328243064 | 0.30267531614117615 | 0.36971650501982367 |
| Standard deviation | 2.118422269377064e-3 | 6.35576585647126e-2 | 7.847170802644791e-2 |
Outlying measurements have moderate (0.4778528282535466%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.17484244300673404 | 0.18362585132530918 | 0.1964430743828416 |
| Standard deviation | 8.030353066157151e-3 | 1.5042601341045334e-2 | 2.295374720873925e-2 |
Outlying measurements have moderate (0.1558874295002258%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.155846958505208 | 4.175098982501974 | 4.186887808624912 |
| Standard deviation | 8.65125649822085e-3 | 1.9480217777133664e-2 | 2.7225127742032226e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4324579008134606 | 0.5230892938949788 | 0.5726166733935921 |
| Standard deviation | 1.2494448195200819e-2 | 8.724019472746813e-2 | 0.11437858354312457 |
Outlying measurements have moderate (0.4648910005015597%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4430339668518476 | 0.5174681852161788 | 0.5600688664611274 |
| Standard deviation | 2.153428825113224e-2 | 7.307103506035252e-2 | 9.838205913773017e-2 |
Outlying measurements have moderate (0.2377561826799053%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.9401996599917766 | 2.0697739308307064 | 2.1174688590836013 |
| Standard deviation | 4.0226205019280314e-3 | 8.86809020282906e-2 | 0.11014671866315502 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.390580872357532 | 6.455668848535424 | 6.554941392125329 |
| Standard deviation | 3.2654566679533883e-2 | 9.515574475943471e-2 | 0.12652478710944595 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.24771050120471044 | 0.28016738142119724 | 0.3080905094381888 |
| Standard deviation | 2.6904382899818943e-2 | 3.735932362134471e-2 | 4.9847900278835726e-2 |
Outlying measurements have moderate (0.3718805296206312%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15421362983657413 | 0.16357143383987327 | 0.17660948868475054 |
| Standard deviation | 9.277402279341786e-3 | 1.7231216644477895e-2 | 2.334123256083962e-2 |
Outlying measurements have moderate (0.2692944452539295%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.8924605699976382 | 3.913291654099036 | 3.9300506543058873 |
| Standard deviation | 1.6874815539874133e-2 | 2.477776714154696e-2 | 2.8621969464872e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.35451388710256043 | 0.4408629250932184 | 0.488679232089756 |
| Standard deviation | 2.1439500416151794e-2 | 8.324485222311809e-2 | 0.11042703086441909 |
Outlying measurements have moderate (0.47260997518794273%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.37487756391540944 | 0.4493872827927892 | 0.48456595892033266 |
| Standard deviation | 9.5894799486193e-3 | 7.09449378986151e-2 | 9.083115247650676e-2 |
Outlying measurements have moderate (0.46081690168145295%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.0190470672660012 | 2.118630192712468 | 2.1793145830821836 |
| Standard deviation | 4.924421083045476e-2 | 0.10040951064805483 | 0.14014459646225408 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.42001591080043 | 6.478287123889459 | 6.511676462225296 |
| Standard deviation | 1.7014927165291738e-2 | 5.6691038173356655e-2 | 7.710937390503647e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2400085756427143 | 0.28529421863728205 | 0.3394891257712152 |
| Standard deviation | 3.726235993984218e-2 | 5.993638193684167e-2 | 8.200717741845233e-2 |
Outlying measurements have severe (0.5745107288076575%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15517771889495807 | 0.16181759945813648 | 0.17351667048705766 |
| Standard deviation | 5.437339177135165e-3 | 1.2468129800339638e-2 | 1.9713460358732816e-2 |
Outlying measurements have moderate (0.13518397348564318%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.912630040636941 | 2.142044889920726 | 2.273956892582646 |
| Standard deviation | 6.273146891180659e-2 | 0.22622114619969738 | 0.3011234804430734 |
Outlying measurements have moderate (0.2281107545647872%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3098262923231232 | 0.367397810233039 | 0.39679919614476 |
| Standard deviation | 2.462507827052235e-3 | 5.6532050052783944e-2 | 6.789959228272067e-2 |
Outlying measurements have moderate (0.45875382303792434%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.31165353049982514 | 0.3643237164196762 | 0.3911463795866439 |
| Standard deviation | 2.2519903222770338e-3 | 5.108103722059325e-2 | 6.194428719932581e-2 |
Outlying measurements have moderate (0.23758074556627323%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.2143631295039086 | 1.2657208347518463 | 1.3377134563779691 |
| Standard deviation | 2.3650413898943567e-2 | 6.993190803984788e-2 | 9.495512590048959e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.515803379865247 | 6.546468082703844 | 6.564252402039225 |
| Standard deviation | 9.807871664330836e-3 | 3.0352692399680036e-2 | 4.107112622254657e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2322597348509589 | 0.261316105236571 | 0.2832148216397036 |
| Standard deviation | 9.42864105448184e-3 | 3.502846936024088e-2 | 5.141470310815642e-2 |
Outlying measurements have moderate (0.3721734069361042%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.14571020906572515 | 0.1520765449465383 | 0.15895047387831646 |
| Standard deviation | 7.641865656043411e-3 | 1.0025641223513005e-2 | 1.282637684821415e-2 |
Outlying measurements have moderate (0.13237557953423837%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.39654927773881354 | 0.4272843074128711 | 0.4714537720028602 |
| Standard deviation | 1.1533729838750645e-2 | 4.2714868502204545e-2 | 5.766150656914682e-2 |
Outlying measurements have moderate (0.22557043180238867%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.243863342605376 | 0.3022102073084291 | 0.3329805293935351 |
| Standard deviation | 1.2706322228967323e-2 | 5.201189537022177e-2 | 7.147403818622852e-2 |
Outlying measurements have moderate (0.3831195934730498%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.24105417173705063 | 0.30106537803444855 | 0.3264314792856264 |
| Standard deviation | 1.1558592880089952e-2 | 4.649769006556871e-2 | 6.663952671649807e-2 |
Outlying measurements have moderate (0.3790381852641917%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5175649688317208 | 0.5425002978536213 | 0.5648637989379495 |
| Standard deviation | 2.2565201135808805e-2 | 2.911072686069829e-2 | 3.2382519518725304e-2 |
Outlying measurements have moderate (0.18749999999999994%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.6715638475495025 | 6.712417813042218 | 6.763767352244031 |
| Standard deviation | 1.651282574312063e-2 | 5.535142524576e-2 | 7.528082312298742e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21838158699392807 | 0.23967238949165526 | 0.25814145161324026 |
| Standard deviation | 1.4709647623579092e-2 | 2.477829239263161e-2 | 3.7444825781546065e-2 |
Outlying measurements have moderate (0.18830491782922212%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13830385335095838 | 0.1479854911498268 | 0.15622823263738672 |
| Standard deviation | 9.726256866562845e-3 | 1.3350028564817795e-2 | 1.787558321051848e-2 |
Outlying measurements have moderate (0.2633239923523447%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19780868989570688 | 0.21456278135835116 | 0.22952954179587928 |
| Standard deviation | 1.3598482880098642e-2 | 2.129342101461458e-2 | 3.3549624209833304e-2 |
Outlying measurements have moderate (0.3039538492866118%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2382156732666772 | 0.2574766331817955 | 0.28468327011796646 |
| Standard deviation | 1.1887629448529842e-2 | 2.9004108504056297e-2 | 4.0443235792046436e-2 |
Outlying measurements have moderate (0.19014931086745734%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22486527784058125 | 0.24060706195334208 | 0.2635971571929986 |
| Standard deviation | 1.1042036137034104e-2 | 2.4475789262240755e-2 | 3.582777607554566e-2 |
Outlying measurements have moderate (0.18792037805324724%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.32144363262523257 | 0.34073528074501763 | 0.3506915762427525 |
| Standard deviation | 3.6871799602522515e-4 | 1.901176415893425e-2 | 2.2993079542994746e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.614335546757502 | 6.649624944504467 | 6.679471805877256 |
| Standard deviation | 3.342414369871071e-2 | 4.179525119823307e-2 | 4.8956424730968714e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1784738306081272 | 0.2271860053073033 | 0.2746645362162963 |
| Standard deviation | 3.530075557222255e-2 | 6.298408214051478e-2 | 8.740814988152441e-2 |
Outlying measurements have severe (0.5853629329144289%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.14019111010252314 | 0.15219921831240152 | 0.16290411646302663 |
| Standard deviation | 1.0311942042319605e-2 | 1.6615865879784905e-2 | 2.356113540544031e-2 |
Outlying measurements have moderate (0.27042578896791913%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.14498402559963455 | 0.16266290454852644 | 0.1731781207924894 |
| Standard deviation | 8.861393612385417e-3 | 1.9559591564038664e-2 | 3.012044086964837e-2 |
Outlying measurements have moderate (0.2731121724226566%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15989347465195655 | 0.17940614693628354 | 0.18773708433970268 |
| Standard deviation | 5.133987270788639e-3 | 1.726221762836454e-2 | 2.6633333066715347e-2 |
Outlying measurements have moderate (0.3020791806668342%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16698874492382554 | 0.17864545023169032 | 0.19128787896139934 |
| Standard deviation | 9.79239628605946e-3 | 1.6175359334940125e-2 | 2.3240562540495602e-2 |
Outlying measurements have moderate (0.15784313760775565%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22963491442367537 | 0.23902886796810585 | 0.2500755136077512 |
| Standard deviation | 2.019658112828268e-3 | 1.3831974170467214e-2 | 1.6439798342097654e-2 |
Outlying measurements have moderate (0.14506443368430905%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.188258248937927 | 4.231940299601168 | 4.25941536818209 |
| Standard deviation | 2.253617299720645e-2 | 4.511020507305239e-2 | 6.345095296479492e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16588224392439263 | 0.1992389755609717 | 0.22779600828232813 |
| Standard deviation | 2.5333539447097e-2 | 4.3375298233556334e-2 | 6.503690022913547e-2 |
Outlying measurements have severe (0.6422464276786337%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12598601132686738 | 0.14569845392190944 | 0.1545045333062924 |
| Standard deviation | 4.425788945089916e-3 | 1.7625802847712025e-2 | 2.6275857246613247e-2 |
Outlying measurements have moderate (0.2732634653370019%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.