criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.315349496249837 | 4.376574911797434 | 4.43780032734503 |
| Standard deviation | 3.7548476186369306e-2 | 7.556108126714561e-2 | 9.126907535170943e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5477212689875159 | 0.7102593190193147 | 0.7685783597517002 |
| Standard deviation | 2.4693915369425667e-2 | 0.11083445696863863 | 0.1346487472110438 |
Outlying measurements have moderate (0.45989631050971447%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.61892455286943 | 0.7072422190819149 | 0.7562147947088912 |
| Standard deviation | 2.5561698487301637e-2 | 8.513705445689117e-2 | 0.11309731888457633 |
Outlying measurements have moderate (0.2331523563796313%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.548568060999969 | 1.5530784056190896 | 1.5596688601090136 |
| Standard deviation | 4.410251651461401e-4 | 6.225738128240142e-3 | 7.830513930252284e-3 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.858920099171883 | 6.952030498921886 | 7.077075327255443 |
| Standard deviation | 4.108494346923988e-2 | 0.12578489299952308 | 0.16996244320108136 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2633208477462176 | 0.2997814068797743 | 0.336241966013331 |
| Standard deviation | 2.872184974441668e-2 | 4.572861640176954e-2 | 5.145301854075819e-2 |
Outlying measurements have moderate (0.37851164012625776%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.14774249794379787 | 0.15587275505788958 | 0.1660247224774671 |
| Standard deviation | 3.2520933206544016e-3 | 1.3226619967978427e-2 | 1.6977055458865756e-2 |
Outlying measurements have moderate (0.2604079905175249%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.248491985457804 | 4.278409871265467 | 4.30042620754466 |
| Standard deviation | 1.2459419922379311e-2 | 2.9310711180089707e-2 | 3.647667289769897e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.46491146300104447 | 0.5938900372275384 | 0.6376219411649799 |
| Standard deviation | 3.7083429560880177e-3 | 8.655370054497101e-2 | 0.10527653164971662 |
Outlying measurements have moderate (0.23850574439494132%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5119216678831435 | 0.5891424890445099 | 0.6312450799584137 |
| Standard deviation | 1.612969004354886e-2 | 7.37945167310218e-2 | 9.72317687908921e-2 |
Outlying measurements have moderate (0.23443917722065621%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4118478491054702 | 1.5874725291184102 | 1.690953897625756 |
| Standard deviation | 6.267611400350381e-2 | 0.17351510469758163 | 0.23897998385530808 |
Outlying measurements have moderate (0.22956484594506904%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.417428916989593 | 6.449962956391876 | 6.506485157566203 |
| Standard deviation | 7.98821913485881e-3 | 5.4282587708247766e-2 | 6.591014469122418e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.250589883220382 | 0.28384937720460585 | 0.31474730556859865 |
| Standard deviation | 2.4722980831506265e-2 | 4.362992619094933e-2 | 5.6754468807583794e-2 |
Outlying measurements have moderate (0.3788370142695779%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13840897801861982 | 0.14484033779551783 | 0.15737585755164868 |
| Standard deviation | 6.79905117260615e-3 | 1.3585739820023117e-2 | 1.94140659223889e-2 |
Outlying measurements have moderate (0.2650034144599894%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.174895733000691 | 4.198193376435786 | 4.2299676154307235 |
| Standard deviation | 1.4607388304423714e-2 | 3.0300896462409153e-2 | 3.7879327618081354e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.37867128036123177 | 0.4675902779957445 | 0.5151447901256082 |
| Standard deviation | 1.4295254153434758e-2 | 8.464792732574175e-2 | 0.10982244151743258 |
Outlying measurements have moderate (0.4702012304178545%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3798199875818682 | 0.4475560665869125 | 0.49631142571524833 |
| Standard deviation | 3.287548143270888e-2 | 6.775867313736919e-2 | 8.967515088612488e-2 |
Outlying measurements have moderate (0.45739455025697756%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.9641051721725185 | 2.0944946289188615 | 2.1663716299178 |
| Standard deviation | 3.965822116394224e-2 | 0.1252199953533884 | 0.16599282350112046 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.389509303626255 | 6.431554704751761 | 6.453519683876948 |
| Standard deviation | 1.478699674771633e-2 | 3.984870343993674e-2 | 5.072594644268444e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.23596151560001694 | 0.25859203779031914 | 0.28122255998062123 |
| Standard deviation | 1.9890302889738528e-2 | 2.9440901025708534e-2 | 3.9095450908027365e-2 |
Outlying measurements have moderate (0.19035642381931098%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12723911077491287 | 0.1322408801493068 | 0.13773727460723292 |
| Standard deviation | 5.011420137885087e-3 | 7.916532915867107e-3 | 1.0886347528482285e-2 |
Outlying measurements have moderate (0.11648447391701333%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.030944908413707 | 4.081915884337529 | 4.132886860261351 |
| Standard deviation | 3.187803423861624e-2 | 6.441262537277809e-2 | 7.919229307026746e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.33069430840744946 | 0.39594865903867077 | 0.4368448371224076 |
| Standard deviation | 1.1010557542200233e-3 | 6.1674610826186596e-2 | 7.589976168256972e-2 |
Outlying measurements have moderate (0.4597500856201415%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3208103680226486 | 0.3905354325652297 | 0.4149671739942278 |
| Standard deviation | 1.0482592369953636e-2 | 4.863395241853262e-2 | 6.440447963743542e-2 |
Outlying measurements have moderate (0.234257206223954%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.1836265042384184 | 2.2833887956615704 | 2.339364780253163 |
| Standard deviation | 9.3836183147874e-3 | 9.444915048275083e-2 | 0.11988738077365493 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.365625541429836 | 6.428751335643635 | 6.465165456270067 |
| Standard deviation | 2.1640016537771874e-2 | 6.190476916133605e-2 | 8.409480938389637e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22008316660649144 | 0.25631204892890913 | 0.28649746059760217 |
| Standard deviation | 2.2770538895657135e-2 | 4.0756082595126426e-2 | 5.293008335246335e-2 |
Outlying measurements have moderate (0.38022469540294%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.11936740723980446 | 0.12614776247343928 | 0.1333056154433247 |
| Standard deviation | 7.96439331985152e-3 | 1.1007603787706524e-2 | 1.5323304417741728e-2 |
Outlying measurements have moderate (0.23396815217894837%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.7789290994351177 | 3.8207640330165304 | 3.8453822592255165 |
| Standard deviation | 1.4803037673118524e-2 | 4.132084740803081e-2 | 5.685335811491671e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2534595910110511 | 0.32093891035886674 | 0.3468121320244487 |
| Standard deviation | 6.860186120320577e-3 | 4.6330719301858975e-2 | 5.975164597370648e-2 |
Outlying measurements have moderate (0.2382848807445319%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.25333311199210584 | 0.310853411705466 | 0.3336835915397387 |
| Standard deviation | 5.2224962404463435e-3 | 4.108902510898793e-2 | 5.272404189185966e-2 |
Outlying measurements have moderate (0.2360266784138583%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.215652029000921 | 2.332584743771804 | 2.377497056084394 |
| Standard deviation | 5.4905605866224505e-3 | 7.986783477431103e-2 | 0.10372054241440969 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.497285571997054 | 6.538251239163098 | 6.554805289163293 |
| Standard deviation | 8.578985444090421e-3 | 2.856694404673671e-2 | 3.8229940895165064e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22815283410600387 | 0.25149975417763926 | 0.2623347467370331 |
| Standard deviation | 1.0121524362821592e-2 | 2.0294943143992394e-2 | 2.9672849175903262e-2 |
Outlying measurements have moderate (0.18080405641818428%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12107014296285949 | 0.1250901724623613 | 0.13098072441668288 |
| Standard deviation | 5.1269693557692815e-3 | 7.400799536262943e-3 | 1.0076427888657107e-2 |
Outlying measurements have moderate (0.1162815523104748%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.3807876320837145 | 3.4402008313948804 | 3.47365185576867 |
| Standard deviation | 5.814130349171685e-3 | 6.1503575114978434e-2 | 7.725183170750452e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21872963439673185 | 0.253688692523477 | 0.2764449696677427 |
| Standard deviation | 1.977267989620757e-2 | 3.623174448611888e-2 | 5.451304564181571e-2 |
Outlying measurements have moderate (0.37548715411460987%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21898751020344207 | 0.25485258884601836 | 0.2738898023676787 |
| Standard deviation | 1.296254810690273e-2 | 3.282610271537838e-2 | 4.6139064302121384e-2 |
Outlying measurements have moderate (0.3698623451155906%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.3004832683315426 | 2.323407221604915 | 2.3468141402481706 |
| Standard deviation | 4.647505306922806e-3 | 2.9211140568911855e-2 | 3.504535889784847e-2 |
Outlying measurements have moderate (0.18749999999999994%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.566537903010612 | 6.598931214148858 | 6.611082286229552 |
| Standard deviation | 1.0321317791143425e-2 | 2.419847560348658e-2 | 3.0603780938806828e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2060056886991434 | 0.21949255864278205 | 0.2340002401427935 |
| Standard deviation | 1.4322445294144474e-2 | 1.868368903819566e-2 | 2.4756025194775107e-2 |
Outlying measurements have moderate (0.15668321738072133%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.10860832112818268 | 0.11417952821366703 | 0.11787296469399389 |
| Standard deviation | 5.490723215374912e-3 | 7.087777979179855e-3 | 8.542775439900958e-3 |
Outlying measurements have moderate (0.11708034353648254%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.740949511121774 | 2.8314735809332583 | 2.923933050369669 |
| Standard deviation | 6.512892670343354e-2 | 0.11217472063259543 | 0.1354600048234863 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19483089570680426 | 0.20520636240455334 | 0.2249383141800839 |
| Standard deviation | 7.987428416403142e-3 | 2.0086763415126073e-2 | 2.8664553847290535e-2 |
Outlying measurements have moderate (0.30313465186970384%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19019354132809288 | 0.19760559852091117 | 0.21189996178792272 |
| Standard deviation | 2.3585919531700777e-3 | 1.584886360114441e-2 | 2.318793822109513e-2 |
Outlying measurements have moderate (0.15542143236099085%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.7682327972397616 | 1.8739459915389791 | 1.9530659555845582 |
| Standard deviation | 5.3186460507276934e-2 | 0.10819623092333501 | 0.1492142097610797 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.542962006002199 | 6.586401920813539 | 6.602107728314271 |
| Standard deviation | 1.868774543327678e-4 | 2.9833001095302597e-2 | 3.6457886207885856e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1841200889070428 | 0.19863499815158625 | 0.21023552410770208 |
| Standard deviation | 7.891448032552654e-3 | 1.9047916584045258e-2 | 2.6913632244957977e-2 |
Outlying measurements have moderate (0.30186723855076014%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.10393179578322574 | 0.10986795488427911 | 0.11893845683655292 |
| Standard deviation | 7.163984974339078e-3 | 1.1748405247004146e-2 | 1.8632374726522533e-2 |
Outlying measurements have moderate (0.3164631714123882%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.170617286079505 | 0.18419720613852 | 0.19676948297298322 |
| Standard deviation | 1.1113507510367061e-2 | 1.7226234795191343e-2 | 2.4923137251180203e-2 |
Outlying measurements have moderate (0.15839578842773158%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.11776056600118474 | 0.12280739282880729 | 0.13044170552472376 |
| Standard deviation | 4.960578274520352e-3 | 9.710592438170004e-3 | 1.4565643770138402e-2 |
Outlying measurements have moderate (0.2304760527425652%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12043986181084793 | 0.12445944199070731 | 0.1294036795751405 |
| Standard deviation | 3.957103333119288e-3 | 6.513472637781698e-3 | 9.714317478456379e-3 |
Outlying measurements have moderate (0.11385755297664628%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1779371422603112 | 0.18601998007967877 | 0.19441395544466408 |
| Standard deviation | 6.291289717817138e-3 | 1.1233378845999998e-2 | 1.687573226289986e-2 |
Outlying measurements have moderate (0.14683018475993756%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.096813270374696 | 6.1447096161809895 | 6.208196742984001 |
| Standard deviation | 3.020018099778099e-2 | 6.0981637976921174e-2 | 7.573970732056336e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1582927251912552 | 0.17063549220892418 | 0.17828923218403855 |
| Standard deviation | 8.739666232890292e-3 | 1.4663017696812965e-2 | 2.2641772904829435e-2 |
Outlying measurements have moderate (0.26103810631586305%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.10304768353493163 | 0.10874682244369244 | 0.11293829568225242 |
| Standard deviation | 4.531096492314733e-3 | 7.362103792031119e-3 | 1.0863975393163425e-2 |
Outlying measurements have moderate (0.2035161860900709%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.