criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.9694503089285718 | 4.011468790102299 | 4.042350370335043 |
| Standard deviation | 7.5311609468319e-4 | 4.0032417143351616e-2 | 4.8895320880522834e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.9645169180002995 | 1.0304232258101063 | 1.0571735779346152 |
| Standard deviation | 1.701892262038507e-2 | 4.8026153164808356e-2 | 6.177729200001005e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.8447406929917634 | 0.9553924105778909 | 0.9939524587037644 |
| Standard deviation | 2.9722664136594967e-3 | 7.395982185005161e-2 | 8.90506166617518e-2 |
Outlying measurements have moderate (0.20926085470299718%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6549669652486045 | 1.687678682124897 | 1.715919540503819 |
| Standard deviation | 1.1152851527003182e-2 | 3.9271335752607224e-2 | 4.851116556579734e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.426535761493142 | 7.48933609258044 | 7.546971030496934 |
| Standard deviation | 5.0173075227949476e-2 | 7.901910165676325e-2 | 0.1038405260522091 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4064377427639556 | 0.5205187537576421 | 0.6246649170061573 |
| Standard deviation | 9.278842533897005e-2 | 0.12199305353708911 | 0.1405951860216079 |
Outlying measurements have moderate (0.4822211935513638%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19581845680174106 | 0.21843726528540072 | 0.24971649850259484 |
| Standard deviation | 1.0647447066306426e-2 | 3.5456678616979674e-2 | 4.401783146368433e-2 |
Outlying measurements have moderate (0.4752903755934375%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6951518600835698 | 0.7347666654768545 | 0.7738415504300065 |
| Standard deviation | 3.25176299151182e-2 | 4.727640943021586e-2 | 5.4788018822340995e-2 |
Outlying measurements have moderate (0.19102744118153878%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.728242853685515 | 0.8110253787296339 | 0.864342294375092 |
| Standard deviation | 1.0260084931496966e-3 | 7.810365832401363e-2 | 9.610203048341281e-2 |
Outlying measurements have moderate (0.22367513285702337%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.7083292058650841 | 0.7987418538292937 | 0.8449561945017194 |
| Standard deviation | 1.1661902028329424e-2 | 8.60054717034529e-2 | 0.11023048429473138 |
Outlying measurements have moderate (0.22894279274647444%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1285597115432513 | 1.199609381233434 | 1.2437014603595646 |
| Standard deviation | 3.957040633496995e-2 | 7.280601591190995e-2 | 0.10182629501040703 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.367450912177446 | 6.514348173608596 | 6.5886557058329345 |
| Standard deviation | 2.606705212383531e-2 | 0.1422677996287848 | 0.1846721664216625 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3513855912606232 | 0.42903414460730954 | 0.49673232271137147 |
| Standard deviation | 4.250825574049693e-2 | 8.685294844692032e-2 | 0.11383398966761256 |
Outlying measurements have moderate (0.47617051676136735%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.195657754658896 | 0.21894151976885043 | 0.2468910402870582 |
| Standard deviation | 3.1138761713002016e-2 | 3.713993700579285e-2 | 4.450207398368811e-2 |
Outlying measurements have moderate (0.4773753059247024%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3388594620046206 | 0.3719136521691932 | 0.4378907576707813 |
| Standard deviation | 5.607612638130491e-3 | 4.685197483411087e-2 | 5.9979551530120544e-2 |
Outlying measurements have moderate (0.23461602764744108%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5707519316256366 | 0.6796522111684075 | 0.7471061345452957 |
| Standard deviation | 5.20151344220115e-2 | 0.11134962612503814 | 0.15577816327817134 |
Outlying measurements have moderate (0.46361721120072624%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5719253168754221 | 0.6562154867557789 | 0.7081795766340898 |
| Standard deviation | 4.342480858425276e-2 | 8.570351379539619e-2 | 0.12000592511774685 |
Outlying measurements have moderate (0.23568683272108476%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.8512260081624845 | 0.8801248567651783 | 0.9244221185726929 |
| Standard deviation | 2.0884817058686167e-4 | 4.65374746774869e-2 | 5.642677776362853e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.501278588334874 | 5.600104559330551 | 5.643833951497072 |
| Standard deviation | 3.225422306284355e-2 | 7.260824868761279e-2 | 0.10098870535535882 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.277672852636897 | 0.37271577746165957 | 0.45389926812276826 |
| Standard deviation | 4.4882704331636575e-2 | 0.10994405684661815 | 0.13565915789212082 |
Outlying measurements have severe (0.7247480686497085%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.17497765534547702 | 0.19479194171306313 | 0.2210837998450087 |
| Standard deviation | 2.2221892626478043e-2 | 3.1765619887444224e-2 | 4.260449820299906e-2 |
Outlying measurements have moderate (0.4755186091112002%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.28764996144222094 | 0.3187159619421194 | 0.34842433720089805 |
| Standard deviation | 3.140358128796358e-2 | 4.0116884862542154e-2 | 4.9539851751907164e-2 |
Outlying measurements have moderate (0.36844100031883015%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.48668547723355005 | 0.5941240244549894 | 0.6759710184220846 |
| Standard deviation | 5.118310650868807e-2 | 0.10341734114832536 | 0.12695465719770815 |
Outlying measurements have moderate (0.4677694634081367%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4385712589719333 | 0.5429047949952898 | 0.5866211157511618 |
| Standard deviation | 2.6557104894778534e-2 | 7.402448661267763e-2 | 0.1009585182255311 |
Outlying measurements have moderate (0.23686782689985275%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.7444706389966693 | 0.7726278518069497 | 0.8131364957989717 |
| Standard deviation | 6.783234040043523e-3 | 3.856075616018218e-2 | 5.0166970046623065e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.509005762247398 | 4.825163544373936 | 4.987308237748948 |
| Standard deviation | 3.319803300109925e-2 | 0.30517058467416236 | 0.37445712947093035 |
Outlying measurements have moderate (0.18896502255831818%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2733001066662837 | 0.3793350056002964 | 0.47516433386772405 |
| Standard deviation | 9.091567453174772e-2 | 0.11308149427309033 | 0.1339566785419821 |
Outlying measurements have severe (0.7252744860384711%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1686533437454879 | 0.19747251844366676 | 0.22811514061080138 |
| Standard deviation | 3.526015320487203e-2 | 4.370080752649378e-2 | 4.7645492829355915e-2 |
Outlying measurements have severe (0.6430335682761069%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2554327023390215 | 0.2847146626042862 | 0.31300636517213815 |
| Standard deviation | 2.2301038631692267e-2 | 3.7266635356763805e-2 | 4.8680677898192595e-2 |
Outlying measurements have moderate (0.3708156984819239%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.31922596800723113 | 0.5022994117904697 | 0.5755699835359943 |
| Standard deviation | 3.61985415297614e-2 | 0.12694496208970743 | 0.16921113728382578 |
Outlying measurements have moderate (0.4847104592041176%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4114530283404747 | 0.5274272307142382 | 0.5961141989646421 |
| Standard deviation | 4.2799468254088424e-2 | 0.11488999181882505 | 0.15862575843675983 |
Outlying measurements have moderate (0.4794192715204146%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6095162594162199 | 0.6396948436073823 | 0.6612810097030888 |
| Standard deviation | 4.600453612511046e-4 | 2.8656848576206262e-2 | 3.511283472621652e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.468354516000545 | 3.5216434870817466 | 3.576090028247563 |
| Standard deviation | 4.1910935871415325e-2 | 7.985929922143577e-2 | 0.11196909899242206 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2877980138291605 | 0.36556332265172387 | 0.40840277989627793 |
| Standard deviation | 1.5827211333089508e-2 | 7.47724944271823e-2 | 9.893348868832301e-2 |
Outlying measurements have moderate (0.47665781335770824%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15931274842377105 | 0.18467072382611352 | 0.21558085376713684 |
| Standard deviation | 2.3926469882284234e-2 | 3.783523455890284e-2 | 5.451609851811372e-2 |
Outlying measurements have moderate (0.4844899918600573%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2002104920404963 | 0.2421790299070805 | 0.2841475677736647 |
| Standard deviation | 2.2461141212942806e-2 | 5.7223907622715536e-2 | 7.480443532443372e-2 |
Outlying measurements have severe (0.5798502078346356%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.41122091887518764 | 0.45971839762448025 | 0.4991143569980826 |
| Standard deviation | 1.8203038751380518e-2 | 5.449635066195007e-2 | 7.443322790262569e-2 |
Outlying measurements have moderate (0.23262643357790053%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.40712036125114537 | 0.4732524806665121 | 0.5411337141655773 |
| Standard deviation | 4.3772666751465295e-2 | 7.541586284077675e-2 | 0.10622067187022419 |
Outlying measurements have moderate (0.4615443255018577%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.48781136200220016 | 0.532834805687647 | 0.5694769325600646 |
| Standard deviation | 1.9355822072663585e-2 | 5.5534547975979995e-2 | 7.15140465037853e-2 |
Outlying measurements have moderate (0.22752504962953646%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.466599118660573 | 2.513036000790938 | 2.5471359582516016 |
| Standard deviation | 2.8490907116097156e-2 | 4.893508983498946e-2 | 6.84255510856306e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.27984406049654353 | 0.32735648231149145 | 0.3606938511293265 |
| Standard deviation | 2.192720542287936e-2 | 5.028443501169996e-2 | 6.951378316009478e-2 |
Outlying measurements have moderate (0.4586119672382881%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.151227572538725 | 0.18604796711564023 | 0.22060964815116477 |
| Standard deviation | 3.2698742233391795e-2 | 4.982521200362336e-2 | 6.733564213436424e-2 |
Outlying measurements have severe (0.6505291090836911%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1765362733397827 | 0.20315800359821878 | 0.2200087743678725 |
| Standard deviation | 1.082969176130218e-2 | 2.8771370985516135e-2 | 4.096495113567642e-2 |
Outlying measurements have moderate (0.31890643321240586%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.29318371143745026 | 0.36060139268617303 | 0.40103123849742894 |
| Standard deviation | 9.74528124467803e-3 | 6.471791046829749e-2 | 8.36234482050166e-2 |
Outlying measurements have moderate (0.46968162858345963%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2926723187119933 | 0.35480467052002496 | 0.38891135145604494 |
| Standard deviation | 2.2954741332796402e-2 | 5.9995062093245406e-2 | 7.876600567830326e-2 |
Outlying measurements have moderate (0.4658455194000571%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3178311014926294 | 0.3780250735599111 | 0.4005468068753544 |
| Standard deviation | 1.1527290374942822e-2 | 4.226599951821496e-2 | 5.6810722770700005e-2 |
Outlying measurements have moderate (0.23047014685197037%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6659241891866259 | 1.7165705049750006 | 1.7946848073734145 |
| Standard deviation | 2.613936465544009e-2 | 7.386893690456454e-2 | 9.361773629414037e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.24531204540107865 | 0.28629556113407795 | 0.3330817570007639 |
| Standard deviation | 2.526521164457905e-2 | 5.5070636011370244e-2 | 8.012241818145076e-2 |
Outlying measurements have severe (0.5695952408718873%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.14729271736044514 | 0.17498426174651185 | 0.19385795691422472 |
| Standard deviation | 1.7667629806699746e-2 | 3.138030750242392e-2 | 4.514575817953299e-2 |
Outlying measurements have moderate (0.4797561843655489%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.13230587178674927 | 0.16118103940950407 | 0.17438480371700799 |
| Standard deviation | 1.1051224128001143e-2 | 2.818967669204836e-2 | 4.447431894259679e-2 |
Outlying measurements have severe (0.5476002416030165%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19196693706099824 | 0.20233805681103453 | 0.21408747782940432 |
| Standard deviation | 9.537130669578857e-3 | 1.571429686661423e-2 | 2.3266307733318378e-2 |
Outlying measurements have moderate (0.15467356492811468%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.20135879592368533 | 0.2139495056897027 | 0.23257382927679118 |
| Standard deviation | 1.2761752706175562e-2 | 2.0776629872286262e-2 | 2.8163421915405554e-2 |
Outlying measurements have moderate (0.30265027576344544%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21580453702112817 | 0.23462297382133937 | 0.2574265605214168 |
| Standard deviation | 1.8101295314818112e-2 | 2.5490393925637976e-2 | 3.772578756684511e-2 |
Outlying measurements have moderate (0.18940995052314794%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.9948989876284031 | 1.0266836050850543 | 1.0432852279172948 |
| Standard deviation | 2.0315072000812506e-2 | 3.2053872614253236e-2 | 3.833980564471484e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19014060026577984 | 0.20999469825425573 | 0.22788173046638904 |
| Standard deviation | 1.8229516775502063e-2 | 2.6154456016165676e-2 | 3.671306287554518e-2 |
Outlying measurements have moderate (0.31468021078845143%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.12693271892411367 | 0.14881678363765657 | 0.16009950845701887 |
| Standard deviation | 8.602272375239445e-3 | 2.2335973632758976e-2 | 3.323034464239344e-2 |
Outlying measurements have moderate (0.4103718024298177%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.