criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.41605664999466774 | 0.47237367668397684 | 0.5715288150604465 |
| Standard deviation | 5.943404175519765e-3 | 9.560377705868621e-2 | 0.11746619375413857 |
Outlying measurements have moderate (0.47615917285257986%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.702986082081528 | 2.051920873829052 | 2.2273326579518957 |
| Standard deviation | 3.7775529963255394e-3 | 0.33395565373589897 | 0.4050961631294249 |
Outlying measurements have moderate (0.46313250230501884%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6981185048377179 | 2.0180466303942617 | 2.2145068171194 |
| Standard deviation | 8.51854992358313e-2 | 0.30942200371369694 | 0.4120139284407216 |
Outlying measurements have moderate (0.458460616252327%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.7471555029442243 | 1.9318164711500383 | 2.0655496924203667 |
| Standard deviation | 5.274070493953309e-2 | 0.1855773717610191 | 0.2561029402605463 |
Outlying measurements have moderate (0.22354414864024524%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6984049726694743 | 1.8327500604621794 | 2.048201183133642 |
| Standard deviation | 3.7929388660510654e-2 | 0.206314645347402 | 0.270680070096496 |
Outlying measurements have moderate (0.23073422776266997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.2374473837359499 | 1.4089364678681402 | 1.5133061062466975 |
| Standard deviation | 8.001320950764546e-2 | 0.16652146865627926 | 0.21235614359984853 |
Outlying measurements have moderate (0.23252237538161954%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5429625920078252 | 0.5745504572587379 | 0.6264797420044488 |
| Standard deviation | 1.8986879477865537e-3 | 5.333624450466044e-2 | 6.49244280259302e-2 |
Outlying measurements have moderate (0.2216696763998307%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.41970469150692225 | 0.4390883301863748 | 0.4745303369351556 |
| Standard deviation | 6.11262389021143e-4 | 3.423385497979766e-2 | 4.123053546856888e-2 |
Outlying measurements have moderate (0.20983643741947064%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4630989314864564 | 1.7281394101228216 | 1.8969216792538646 |
| Standard deviation | 0.1361958092529676 | 0.27396688217471393 | 0.38467507050840516 |
Outlying measurements have moderate (0.4611436987617587%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4211696474376367 | 1.6812358300206445 | 1.8296816325212906 |
| Standard deviation | 1.5014235419358868e-2 | 0.25513620936747383 | 0.34282089613527217 |
Outlying measurements have moderate (0.457595385465836%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4791418389359023 | 1.724884068814087 | 1.8670183231903745 |
| Standard deviation | 2.160891726346398e-3 | 0.24364113852961813 | 0.3282450001008655 |
Outlying measurements have moderate (0.23776346001946147%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3570441699994262 | 1.7168856127294323 | 1.8596034614796129 |
| Standard deviation | 1.7042836904479053e-2 | 0.2486816049639778 | 0.32959275357632417 |
Outlying measurements have moderate (0.23836314959248298%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1832224466731227 | 1.3036294118367853 | 1.4240363770004478 |
| Standard deviation | 0.10035234451466034 | 0.14590434309623707 | 0.18322906181502355 |
Outlying measurements have moderate (0.23050999667180955%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5653448381684332 | 0.5822256860216536 | 0.6040815315160822 |
| Standard deviation | 1.1090021628963886e-3 | 2.4193950905370635e-2 | 3.0246375573894948e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3727794915030245 | 0.4190675614596936 | 0.5095094558764686 |
| Standard deviation | 1.9987874475191347e-4 | 8.980860685318301e-2 | 0.10666691850482904 |
Outlying measurements have moderate (0.4787366242271205%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.2037471238387902 | 1.3433009801695637 | 1.4828548365003371 |
| Standard deviation | 8.921531816425465e-2 | 0.17251360696219836 | 0.21926885016014844 |
Outlying measurements have moderate (0.23519733181717503%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1612897701609959 | 1.3494104495154413 | 1.6009006516778754 |
| Standard deviation | 9.451548395615339e-2 | 0.24562620554161438 | 0.32530901925235167 |
Outlying measurements have moderate (0.47052602012403205%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1997154124013225 | 1.3755103633496522 | 1.5261284008738585 |
| Standard deviation | 6.479743112868164e-2 | 0.18637918591664301 | 0.24040133880933093 |
Outlying measurements have moderate (0.2367024091933766%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.2419225904923223 | 1.4310006231225998 | 1.6200786557528772 |
| Standard deviation | 0.14931194647987794 | 0.22930047682360163 | 0.285291060086459 |
Outlying measurements have moderate (0.46196619518302%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1400647753374265 | 1.2951478283545537 | 1.436053911684212 |
| Standard deviation | 6.194509894609487e-2 | 0.18580159491042228 | 0.22641718881612904 |
Outlying measurements have moderate (0.23813740072015513%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4318261748067016 | 0.5180517906028399 | 0.6166359273775015 |
| Standard deviation | 5.0253956052175745e-2 | 0.10141526396404045 | 0.14226661386082595 |
Outlying measurements have moderate (0.4745176338967552%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.31007299774864805 | 0.35874524712501926 | 0.4396324042463675 |
| Standard deviation | 3.0900494442724982e-2 | 8.090205401989972e-2 | 0.11099340339750582 |
Outlying measurements have moderate (0.480797699611164%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0260606183243606 | 1.2204960241421456 | 1.3308668945198103 |
| Standard deviation | 5.2612669875088613e-2 | 0.1891085685286319 | 0.254890606892952 |
Outlying measurements have moderate (0.4593228640501929%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0194874718193507 | 1.1603645644512046 | 1.239907322077973 |
| Standard deviation | 3.641684524336597e-2 | 0.13618736170278625 | 0.18369613011159966 |
Outlying measurements have moderate (0.23227626065878687%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.9636431219987571 | 1.2044462632923871 | 1.3027927011692857 |
| Standard deviation | 4.344034839003355e-2 | 0.1706533483803712 | 0.22712136952827278 |
Outlying measurements have moderate (0.23783855216513666%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0339169040889828 | 1.1970929157087085 | 1.286572079456164 |
| Standard deviation | 5.2638236860673926e-2 | 0.15609219710895667 | 0.20664327710582403 |
Outlying measurements have moderate (0.23564069613004257%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0596487293041719 | 1.163413863642442 | 1.2323718706660052 |
| Standard deviation | 2.8330704776152058e-2 | 9.908383661653825e-2 | 0.13092099089518494 |
Outlying measurements have moderate (0.2163408695369091%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.39923392511627753 | 0.4400451711001855 | 0.5005639622468152 |
| Standard deviation | 4.032307587792337e-2 | 6.606837452203132e-2 | 9.343120115825028e-2 |
Outlying measurements have moderate (0.45667811766345334%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2837947630597045 | 0.3199648875235774 | 0.3611890522981412 |
| Standard deviation | 3.6198763176756125e-2 | 4.940734235180818e-2 | 6.0124884491748465e-2 |
Outlying measurements have moderate (0.3790303268444778%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.8440213697322179 | 0.9527166089337697 | 1.033579341506993 |
| Standard deviation | 6.540965563997579e-2 | 0.120015984806719 | 0.1683886377686584 |
Outlying measurements have moderate (0.23461528913645255%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.800021189812469 | 0.9306580776046758 | 1.0063192272300512 |
| Standard deviation | 4.777091149555411e-2 | 0.12969800647557836 | 0.17473194041362816 |
Outlying measurements have moderate (0.23742946525740924%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.8064227179856971 | 0.967735300685187 | 1.0314744941891454 |
| Standard deviation | 4.875876020874137e-3 | 0.11882362364034033 | 0.14719936210975998 |
Outlying measurements have moderate (0.233806242411841%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.8193228533327783 | 0.9535360296134361 | 1.0511476555863433 |
| Standard deviation | 4.3479111204457355e-2 | 0.13075190718701293 | 0.17521904100994198 |
Outlying measurements have moderate (0.23701571012398961%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.9678050077466953 | 1.0309546849378723 | 1.0773524213764176 |
| Standard deviation | 2.2912914873207575e-2 | 6.115192395617421e-2 | 8.069342735040283e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.38272101025359007 | 0.4049353907715461 | 0.44289098181070585 |
| Standard deviation | 2.4338792303834414e-3 | 3.635512860991849e-2 | 4.386111876020303e-2 |
Outlying measurements have moderate (0.2197113336196875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.241127295861952 | 0.2666364661490661 | 0.28365645731100814 |
| Standard deviation | 1.4071148918829568e-2 | 2.7098605411979098e-2 | 3.665678343336511e-2 |
Outlying measurements have moderate (0.18789805373381543%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6099876050138846 | 0.7377368846676594 | 0.7919378964979842 |
| Standard deviation | 1.858083199840621e-2 | 8.805337163502956e-2 | 0.1157766981235228 |
Outlying measurements have moderate (0.23286235798470367%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6262352499979897 | 0.7308612606848328 | 0.7878332644340844 |
| Standard deviation | 1.328845068578513e-2 | 0.10079298900833952 | 0.13157120680361112 |
Outlying measurements have moderate (0.23716342127204784%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6476484294156156 | 0.7366660797900598 | 0.7866304082902692 |
| Standard deviation | 2.519789045054479e-2 | 8.632720846053382e-2 | 0.1153876740378991 |
Outlying measurements have moderate (0.2322218730519055%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6351833756634733 | 0.7266872990421689 | 0.7835854117547569 |
| Standard deviation | 3.219462977677373e-2 | 8.960063105354533e-2 | 0.12175694447758048 |
Outlying measurements have moderate (0.23394120128171264%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.8915105959143451 | 0.9417220677290364 | 0.9736493186283042 |
| Standard deviation | 1.958857201329095e-2 | 4.9515422015970506e-2 | 6.800777089815967e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.30244974800007185 | 0.3231815491872112 | 0.34391335037435056 |
| Standard deviation | 1.987921001040999e-2 | 2.4476644195484353e-2 | 2.7998833974312846e-2 |
Outlying measurements have moderate (0.20743727556279043%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.20209800223108484 | 0.2245677759224135 | 0.2346609773081986 |
| Standard deviation | 6.167861187866629e-3 | 2.0705184002438995e-2 | 2.8755249089188004e-2 |
Outlying measurements have moderate (0.15815718784253596%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5119646833312193 | 0.574482694605346 | 0.6808905969410262 |
| Standard deviation | 1.30560477714909e-2 | 0.10242622220137391 | 0.1313231147817775 |
Outlying measurements have moderate (0.4692792626591792%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5066717109123904 | 0.5596540773318944 | 0.6376625183766009 |
| Standard deviation | 2.219425308673334e-2 | 7.417225135981803e-2 | 9.672981807023978e-2 |
Outlying measurements have moderate (0.23610058655266142%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.550210283126944 | 0.6232865531274001 | 0.6778043616322975 |
| Standard deviation | 4.567956069919166e-3 | 8.137262546532101e-2 | 0.11471509071072544 |
Outlying measurements have moderate (0.2356761781890072%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5286243580000398 | 0.5883142676242036 | 0.7274237899982836 |
| Standard deviation | 1.3137555723964411e-2 | 9.508835203475606e-2 | 0.121345683440913 |
Outlying measurements have moderate (0.46261785965484803%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.7499394811675302 | 0.7792966671040631 | 0.7972669080409105 |
| Standard deviation | 1.3993706786299652e-2 | 2.9714261093628867e-2 | 4.150049376916513e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.26580209072174815 | 0.29442586886861444 | 0.31497950810628633 |
| Standard deviation | 2.0872217571295947e-2 | 3.0067358730421028e-2 | 3.86490695223816e-2 |
Outlying measurements have moderate (0.18801407970285464%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16469232000332948 | 0.1894236382502034 | 0.1998930482487291 |
| Standard deviation | 7.575045776014033e-3 | 2.1910729074338994e-2 | 3.281505823356187e-2 |
Outlying measurements have moderate (0.3117070964289774%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3009289869951317 | 0.3455188186702192 | 0.3668750760061812 |
| Standard deviation | 8.82068232264524e-3 | 3.213066573349636e-2 | 4.236027585683256e-2 |
Outlying measurements have moderate (0.22176784587032936%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.32219932594489364 | 0.34847465327948157 | 0.36199988798580307 |
| Standard deviation | 1.790113949701199e-3 | 2.4821843190357e-2 | 3.1235191594190345e-2 |
Outlying measurements have moderate (0.2018812445685988%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.31857737528480357 | 0.34057954122727097 | 0.360168302193415 |
| Standard deviation | 1.0624734716804595e-2 | 2.5242073175799747e-2 | 3.461357095077338e-2 |
Outlying measurements have moderate (0.20555456905507902%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.31189051900582854 | 0.3460643345442804 | 0.35802437754500716 |
| Standard deviation | 8.112316718325019e-4 | 2.288599482666916e-2 | 2.7620536183956303e-2 |
Outlying measurements have moderate (0.19417687466320993%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.49068387001170777 | 0.578169198149529 | 0.6097353550242891 |
| Standard deviation | 4.916979092134455e-3 | 5.883829322414212e-2 | 7.28989166756987e-2 |
Outlying measurements have moderate (0.2264261876868599%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.24350412012815165 | 0.26142219677810014 | 0.28710771184269107 |
| Standard deviation | 1.4769864569422525e-2 | 2.7721738785194072e-2 | 4.0687074493704345e-2 |
Outlying measurements have moderate (0.1888838602331882%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.