criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.369124842372912 | 4.638961153042449 | 5.146080236132548 |
| Standard deviation | 1.573786849589831e-2 | 0.49980664316852336 | 0.5981005409806097 |
Outlying measurements have moderate (0.22896815003442555%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5587913040071726 | 0.7420234097717184 | 0.8086871834566409 |
| Standard deviation | 2.4061675168923102e-2 | 0.12472938439339583 | 0.15395339072874517 |
Outlying measurements have moderate (0.46543809283404175%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6140028643712867 | 0.7185463796231488 | 0.7790510872473533 |
| Standard deviation | 1.8297856430277912e-2 | 9.944249305901431e-2 | 0.12986538156052363 |
Outlying measurements have moderate (0.23725306838964738%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.5275844989373581 | 1.5724799598538084 | 1.5955858039415034 |
| Standard deviation | 8.774848392931744e-4 | 4.269506826701045e-2 | 5.234742899918548e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.959635912626254 | 7.031855810311754 | 7.104075707997254 |
| Standard deviation | 4.028779699729057e-2 | 9.123502020745931e-2 | 0.12026436859539663 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2564969554950949 | 0.30321668695311627 | 0.35057035037061723 |
| Standard deviation | 3.11437046823762e-2 | 5.947612639654167e-2 | 7.449837322312718e-2 |
Outlying measurements have moderate (0.4746182893848961%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1624752493243986 | 0.1708185349448824 | 0.1879807194033804 |
| Standard deviation | 1.0322969779259797e-2 | 1.6953416930714813e-2 | 2.2379522341200206e-2 |
Outlying measurements have moderate (0.26721557769871296%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.254441934876013 | 4.362370807537445 | 4.501048719743267 |
| Standard deviation | 4.309205274330452e-2 | 0.14689718348127084 | 0.19949263823508218 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.48335216401028447 | 0.6210517391712832 | 0.6680862998352193 |
| Standard deviation | 1.2629974953597412e-3 | 9.186159064389711e-2 | 0.10862166504215705 |
Outlying measurements have moderate (0.45536375162830145%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5520250723711797 | 0.6203955147278369 | 0.6551816809163332 |
| Standard deviation | 2.7756461071077737e-3 | 6.682232933095536e-2 | 8.033520965201355e-2 |
Outlying measurements have moderate (0.22895571717852964%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.552656362997368 | 1.6537442871212988 | 1.7025565769945388 |
| Standard deviation | 3.82963420350656e-2 | 7.676382720669346e-2 | 0.10768735488693948 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.274561732501752 | 6.350017434854938 | 6.403305528833394 |
| Standard deviation | 3.300253168605486e-2 | 7.992703815712712e-2 | 0.11010786452010946 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.26704105746466666 | 0.3017703408823581 | 0.346930611882708 |
| Standard deviation | 2.6299629888238647e-2 | 4.79122124514132e-2 | 6.33346477670539e-2 |
Outlying measurements have moderate (0.3801650705725021%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1478877778641791 | 0.15575636441837784 | 0.16275718643098988 |
| Standard deviation | 8.161609312271643e-3 | 1.078507902806844e-2 | 1.3398916534165806e-2 |
Outlying measurements have moderate (0.1333561094404944%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.165042326501862 | 4.182979613729306 | 4.2026822769439605 |
| Standard deviation | 1.1841485221809011e-2 | 2.3505248340129275e-2 | 3.0974618083620155e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4027335383834725 | 0.49353354146296624 | 0.5583574837546621 |
| Standard deviation | 4.6529748920875136e-2 | 8.977356431758218e-2 | 0.11371706888287073 |
Outlying measurements have moderate (0.47048543047774893%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.41147190874107764 | 0.48746248520304414 | 0.5417266541189747 |
| Standard deviation | 3.2332211507309694e-2 | 7.450715543311753e-2 | 9.416684302559583e-2 |
Outlying measurements have moderate (0.4581990190162519%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.9443216100044083 | 2.1089558494798744 | 2.182875022602578 |
| Standard deviation | 4.9631424371909816e-2 | 0.12286820113694286 | 0.17070901800267035 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.330952830507158 | 6.4307751646053175 | 6.489026645769021 |
| Standard deviation | 3.338265668005545e-2 | 9.93606399916593e-2 | 0.13452603332223476 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2424602057633456 | 0.2759425048613533 | 0.30821338152241273 |
| Standard deviation | 2.992207345375166e-2 | 4.1099483694903974e-2 | 5.050106529482282e-2 |
Outlying measurements have moderate (0.3774610086669811%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13115285256034917 | 0.13655505595581455 | 0.14455940936878958 |
| Standard deviation | 6.679648704941332e-3 | 9.971321045029203e-3 | 1.4452508865022706e-2 |
Outlying measurements have moderate (0.1343136378897644%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.015636787375115 | 4.037298361206922 | 4.055290771997534 |
| Standard deviation | 1.2509553249401506e-2 | 2.182449194162794e-2 | 2.7106904981352268e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.31335294802556746 | 0.411357461342656 | 0.44614626225302345 |
| Standard deviation | 6.244447111556348e-3 | 6.639413583188368e-2 | 8.438248135326669e-2 |
Outlying measurements have moderate (0.4625131147915627%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.35109579115669476 | 0.4113159011649259 | 0.4458508520037867 |
| Standard deviation | 2.0328382670413703e-2 | 5.726444936535675e-2 | 7.395430518396265e-2 |
Outlying measurements have moderate (0.23740434163675359%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.200751578364361 | 2.3138582449143237 | 2.3833835485850914 |
| Standard deviation | 5.1887881583146056e-2 | 0.11475089395602699 | 0.1605618111592331 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.354304842010606 | 6.4371806835236685 | 6.4760645060208235 |
| Standard deviation | 3.2096951123094186e-2 | 6.382928141215866e-2 | 8.979834154341639e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2518973452279655 | 0.27869912484883874 | 0.31078907060291383 |
| Standard deviation | 3.011407357614609e-2 | 3.741601209879408e-2 | 4.611122819363334e-2 |
Outlying measurements have moderate (0.3722587376426991%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12610154753366215 | 0.13233065917118092 | 0.13786915931563098 |
| Standard deviation | 6.933788139382645e-3 | 9.152589501394564e-3 | 1.1807045309943613e-2 |
Outlying measurements have moderate (0.2244822111719924%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.725245875875771 | 3.752429299378491 | 3.7664202672549436 |
| Standard deviation | 1.8440850503821243e-3 | 2.6036808009774166e-2 | 3.141578704952059e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2567326440184843 | 0.33530473981772957 | 0.36257499043676944 |
| Standard deviation | 1.529670736999833e-2 | 5.6043362707750524e-2 | 7.316170659134644e-2 |
Outlying measurements have moderate (0.4650432493418126%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.28413246274431003 | 0.33294333749897004 | 0.3719149915026113 |
| Standard deviation | 2.272270699686726e-2 | 4.801134717913627e-2 | 5.966918266113162e-2 |
Outlying measurements have moderate (0.23825932882292092%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.305591552871192 | 2.3720170869364665 | 2.4524925165023888 |
| Standard deviation | 3.4185708350391586e-2 | 8.36293345566968e-2 | 0.11001787602126774 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.4502174019835365 | 6.50911512932483 | 6.5418682569907105 |
| Standard deviation | 4.40568532697716e-3 | 5.5790665796293264e-2 | 7.055286107980166e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2231910102913389 | 0.2571706626146139 | 0.2758049207538716 |
| Standard deviation | 8.221042866043977e-3 | 3.2242636404713344e-2 | 4.350064421894951e-2 |
Outlying measurements have moderate (0.3681908722450957%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12135496185627354 | 0.1270150817259881 | 0.13733902186413616 |
| Standard deviation | 5.014578776543065e-3 | 1.2161069244232554e-2 | 1.8349108391938464e-2 |
Outlying measurements have moderate (0.23668389786030947%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.151647701000911 | 3.240446755588588 | 3.2752934788392545 |
| Standard deviation | 1.745403111520067e-2 | 6.226667063290438e-2 | 8.047506019659455e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22562127715355018 | 0.2687914049315926 | 0.2874205739911607 |
| Standard deviation | 7.605454896545596e-3 | 3.595335739068494e-2 | 4.94531745218986e-2 |
Outlying measurements have moderate (0.372053858399504%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.23418948620092123 | 0.2641255775281267 | 0.2801083454018226 |
| Standard deviation | 6.864567959576445e-3 | 3.1238923978399096e-2 | 4.415215827323875e-2 |
Outlying measurements have moderate (0.36425635885962127%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.9925971326641352 | 2.0027611478350686 | 2.017734364759235 |
| Standard deviation | 1.5129453337673349e-3 | 1.5701090308299005e-2 | 1.99329832195307e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.612564756747815 | 6.645427852165085 | 6.695492992874279 |
| Standard deviation | 1.044070008356357e-2 | 4.82956772843151e-2 | 6.383818593471248e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19125636450044112 | 0.22426158398709956 | 0.23922979067370762 |
| Standard deviation | 1.041283541337027e-2 | 2.6728159574803123e-2 | 4.0054959722172705e-2 |
Outlying measurements have moderate (0.36480008101352057%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.11634001195220013 | 0.12204855632569542 | 0.13071962004441956 |
| Standard deviation | 7.270889731167545e-3 | 1.2394918287266504e-2 | 1.8718918474001055e-2 |
Outlying measurements have moderate (0.2381644617433537%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5233253319966025 | 0.5605355465401468 | 0.5875599151258939 |
| Standard deviation | 1.6972994784450877e-2 | 3.680183618316713e-2 | 4.6241759185729446e-2 |
Outlying measurements have moderate (0.1933620672012617%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.20848874914623192 | 0.21866776674489505 | 0.24058600249635573 |
| Standard deviation | 6.948805487344438e-3 | 2.307685739301923e-2 | 3.30827500526785e-2 |
Outlying measurements have moderate (0.3073531343710475%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.20051997052683468 | 0.21393117135998585 | 0.23782204505470064 |
| Standard deviation | 6.008180712351235e-3 | 2.6123594896148197e-2 | 3.707438325091374e-2 |
Outlying measurements have moderate (0.3139285622609669%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.38771287674899213 | 0.3965336446229533 | 0.4070491906222742 |
| Standard deviation | 4.86745074691722e-3 | 1.1212687158276677e-2 | 1.5503240079045624e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.560284413499176 | 6.615748115335616 | 6.637856057463675 |
| Standard deviation | 9.468326419058481e-3 | 3.913839058493744e-2 | 5.105610535545464e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1862284852444039 | 0.2172879125161368 | 0.2359063918265747 |
| Standard deviation | 1.207824010420689e-2 | 3.068349314857652e-2 | 4.4786932449428225e-2 |
Outlying measurements have moderate (0.3188226521171174%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1126113735803301 | 0.11486173362336558 | 0.118652962548731 |
| Standard deviation | 2.5724073834855112e-3 | 5.097881282368707e-3 | 7.507573833065984e-3 |
Outlying measurements have moderate (0.10950750503619831%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1188451766573387 | 0.13730748065523202 | 0.14437639034231634 |
| Standard deviation | 5.036853707938802e-3 | 1.5436580829242142e-2 | 2.297440657083622e-2 |
Outlying measurements have moderate (0.2712973795401646%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1432600364451127 | 0.16168430263632325 | 0.17723609378555835 |
| Standard deviation | 1.4937552918125312e-2 | 2.3375651743296883e-2 | 3.8979108563443284e-2 |
Outlying measurements have moderate (0.40895695420695366%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13711826361915364 | 0.15951287500203118 | 0.1699474615644985 |
| Standard deviation | 9.08686836943808e-3 | 2.00384380445556e-2 | 3.164413662068193e-2 |
Outlying measurements have moderate (0.4025918460164006%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1748929045308614 | 0.18095142488091367 | 0.18476884773408528 |
| Standard deviation | 3.791972092421764e-3 | 6.248885312105818e-3 | 1.0111425162499394e-2 |
Outlying measurements have moderate (0.13888888888888867%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.189897961936367 | 6.234930069983723 | 6.30584326789176 |
| Standard deviation | 6.856788371806033e-3 | 7.152068782645063e-2 | 8.925473813949254e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1568501253204886 | 0.18880970294824995 | 0.20497689230493657 |
| Standard deviation | 1.1171134609583262e-2 | 2.936186956745371e-2 | 4.3523741260811294e-2 |
Outlying measurements have moderate (0.4730790741358454%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.10083760507985148 | 0.10784559870767009 | 0.11356558099699517 |
| Standard deviation | 7.012766357684745e-3 | 9.933947278802991e-3 | 1.3782549470906465e-2 |
Outlying measurements have moderate (0.23561296810640991%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.