criterion performance measurements
overview
want to understand this report?
0% Lookups; 50% Insertions; 50% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.866470473632944 | 0.9389358675416588 | 1.049675715412377 |
| Standard deviation | 6.972102534818211e-3 | 0.11069304038514871 | 0.14335549505601422 |
Outlying measurements have moderate (0.23243406471325923%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0794114189920947 | 1.3628904696003399 | 1.4607797144611445 |
| Standard deviation | 8.085852677307548e-4 | 0.18918318548353652 | 0.22525694221688808 |
Outlying measurements have moderate (0.2373294083828091%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.2595807180793297 | 1.39671247164794 | 1.4683378279011474 |
| Standard deviation | 1.2237917875609128e-2 | 0.1328337704961206 | 0.1654116748543686 |
Outlying measurements have moderate (0.22300787517520362%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6136057474413974 | 1.8662193440231931 | 2.0549655791692203 |
| Standard deviation | 0.1329326398408739 | 0.2648742201193246 | 0.36844175137113466 |
Outlying measurements have moderate (0.23788046259294088%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.53634760374795 | 7.790410230501341 | 8.202929631133884 |
| Standard deviation | 4.3661372748223126e-2 | 0.3927986327612656 | 0.5015442793116582 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6538466151687317 | 0.8100004872303543 | 0.9341794761239726 |
| Standard deviation | 7.384282961741682e-2 | 0.162823131339189 | 0.22046334889762123 |
Outlying measurements have moderate (0.4758321287684066%) effect on estimated standard deviation.
0% Lookups; 50% Insertions; 50% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.34458618204735103 | 0.3625531655470695 | 0.3932062406256591 |
| Standard deviation | 4.216010253003333e-3 | 3.144813529464358e-2 | 3.903373217001275e-2 |
Outlying measurements have moderate (0.21755157239838027%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3983350429128526 | 0.4235888414556636 | 0.49634087750018807 |
| Standard deviation | 1.2788416284970003e-3 | 4.868343749840934e-2 | 5.83211495470101e-2 |
Outlying measurements have moderate (0.231517249438235%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0321954622607639 | 1.2096912124428247 | 1.3700366059401858 |
| Standard deviation | 8.793772224938345e-2 | 0.1919582859917108 | 0.2687600891708633 |
Outlying measurements have moderate (0.4612175386704678%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.000966731862718 | 1.1939147622858097 | 1.3047065647903462 |
| Standard deviation | 5.727114917196252e-2 | 0.18789941388867118 | 0.25586270799999195 |
Outlying measurements have moderate (0.4605728090004991%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3419846032969265 | 1.5010190246642499 | 1.7583897076165158 |
| Standard deviation | 8.191999299378949e-2 | 0.24572700258949187 | 0.3216909284026253 |
Outlying measurements have moderate (0.4635609814930294%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.2530051778364095 | 2.421122268296434 | 2.6771855065071577 |
| Standard deviation | 4.432698716239725e-2 | 0.2639814050369827 | 0.3435956345710508 |
Outlying measurements have moderate (0.22946348133498246%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6335927496499305 | 0.6921397367617222 | 0.7729353363932508 |
| Standard deviation | 2.9864156527293226e-2 | 9.541813507885816e-2 | 0.12830972420053016 |
Outlying measurements have moderate (0.23715406350718568%) effect on estimated standard deviation.
20% Lookups; 40% Insertions; 40% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2990307133319826 | 0.32391769660474284 | 0.3705964763157681 |
| Standard deviation | 6.790782310975366e-4 | 4.625618798754143e-2 | 5.679494773628576e-2 |
Outlying measurements have moderate (0.23802791685416222%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.32799047124899516 | 0.36948724416712747 | 0.4516996250022203 |
| Standard deviation | 2.8599449445931846e-4 | 8.188511596042697e-2 | 9.58326920591378e-2 |
Outlying measurements have moderate (0.4801166752311239%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.8272285594302957 | 0.9959909191414529 | 1.1032095925177903 |
| Standard deviation | 9.47784788768331e-2 | 0.17661756513752122 | 0.24761091974393182 |
Outlying measurements have moderate (0.4689441790292767%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.7488294583672541 | 0.9714016865364101 | 1.10142119928787 |
| Standard deviation | 7.493359466752736e-2 | 0.21860569142169237 | 0.30026720274783725 |
Outlying measurements have moderate (0.4807169902721145%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.8036354200157803 | 1.1028279629020592 | 1.226203147150348 |
| Standard deviation | 3.735680895745952e-2 | 0.21257412642514564 | 0.284922783348277 |
Outlying measurements have moderate (0.4737157992154847%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4127674095822538 | 1.6797327950832066 | 1.8417712675824685 |
| Standard deviation | 0.1142231189951417 | 0.2684387935211774 | 0.3742118228660984 |
Outlying measurements have moderate (0.46176246479988875%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5913877885177499 | 0.6698214346912815 | 0.7423598723701919 |
| Standard deviation | 4.037763737748418e-2 | 9.05281444810108e-2 | 0.11387941327275634 |
Outlying measurements have moderate (0.23663432500464207%) effect on estimated standard deviation.
40% Lookups; 30% Insertions; 30% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2628216947385226 | 0.3009080628118439 | 0.3749454199988395 |
| Standard deviation | 1.8387535169095803e-2 | 7.044935110741173e-2 | 9.969856020583534e-2 |
Outlying measurements have severe (0.5794758028620033%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.29710308850189904 | 0.3508787018345174 | 0.4046543151671358 |
| Standard deviation | 4.3166714250269356e-2 | 6.453298567468273e-2 | 7.498359446363279e-2 |
Outlying measurements have moderate (0.47112975022218967%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5816681089927442 | 0.826782305309583 | 0.9297117233102957 |
| Standard deviation | 4.5937517872516764e-2 | 0.176480719182898 | 0.2144882531765525 |
Outlying measurements have moderate (0.4785667583618153%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6465959080984854 | 0.8319448593398798 | 0.922624407915767 |
| Standard deviation | 3.424544875774338e-2 | 0.17982120891273318 | 0.2436608958788724 |
Outlying measurements have moderate (0.47909706344570124%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.9967814729970996 | 1.0269685117691552 | 1.0569088539359655 |
| Standard deviation | 1.764542290695068e-2 | 3.805816461776719e-2 | 5.1498835531599035e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4426675169064158 | 1.4677034527897679 | 1.4879591017524945 |
| Standard deviation | 1.3456426750053652e-2 | 2.6994599742121043e-2 | 3.667808024980901e-2 |
Outlying measurements have moderate (0.18749999999999997%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5686582199947832 | 0.6536624134344796 | 0.7598800019386545 |
| Standard deviation | 4.242679012895678e-2 | 0.10463076753827875 | 0.13017585918200442 |
Outlying measurements have moderate (0.46188567374187567%) effect on estimated standard deviation.
50% Lookups; 25% Insertions; 25% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2563380556172342 | 0.2927949214490945 | 0.3744125064331456 |
| Standard deviation | 2.0414699063474172e-2 | 6.803556087127507e-2 | 9.080603613579856e-2 |
Outlying measurements have severe (0.5791643207647044%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2685717450908851 | 0.3036115032696398 | 0.33993308965291363 |
| Standard deviation | 3.0880931609550744e-2 | 4.671853783099592e-2 | 5.886443374131296e-2 |
Outlying measurements have moderate (0.3788832069105542%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5375340676218912 | 0.6744695544584829 | 0.7655601736296376 |
| Standard deviation | 4.8718896963665465e-2 | 0.1319455998835988 | 0.17553457824777938 |
Outlying measurements have moderate (0.4744826882069745%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.540284790988153 | 0.6711510503707055 | 0.7596438077501564 |
| Standard deviation | 4.4594860747262544e-2 | 0.1286667336191499 | 0.17685819502161818 |
Outlying measurements have moderate (0.47342899140403916%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6189785603764903 | 0.7712571386709897 | 0.8563510491294437 |
| Standard deviation | 1.949236459429224e-2 | 0.14511110850165998 | 0.18558233863743148 |
Outlying measurements have moderate (0.4724134673368348%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0317161387483793 | 1.2228115634576777 | 1.3250422434939537 |
| Standard deviation | 6.0801429169397264e-2 | 0.1814430178593834 | 0.2357805819301405 |
Outlying measurements have moderate (0.4556454072663122%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.5322756116202072 | 0.5916061465389552 | 0.6257051657485135 |
| Standard deviation | 2.254491716788226e-2 | 5.920338418521745e-2 | 8.152531460874185e-2 |
Outlying measurements have moderate (0.22562119902631916%) effect on estimated standard deviation.
60% Lookups; 20% Insertions; 20% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2382470755983377 | 0.26267032625328285 | 0.287093576908228 |
| Standard deviation | 2.4190394364700385e-2 | 3.21617143242976e-2 | 3.685100901291874e-2 |
Outlying measurements have moderate (0.3666486320833808%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2352304885881798 | 0.25967794099783836 | 0.28390348140072696 |
| Standard deviation | 2.6257447721041414e-2 | 3.260116038494053e-2 | 3.4403814387954826e-2 |
Outlying measurements have moderate (0.36827701794482526%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3825640350114554 | 0.5367076513775828 | 0.5930445851245167 |
| Standard deviation | 1.7296689489768313e-3 | 0.1071016633786107 | 0.13183424440541852 |
Outlying measurements have moderate (0.4754764586558433%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.42544219731098565 | 0.533866167602658 | 0.5892332974757059 |
| Standard deviation | 5.33538495366892e-3 | 0.10229328091091544 | 0.1278649093458445 |
Outlying measurements have moderate (0.4734006743905936%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.570169337457628 | 0.6559116838325281 | 0.7216957312484737 |
| Standard deviation | 3.99165979179088e-2 | 8.527313248006117e-2 | 0.11382613641333546 |
Outlying measurements have moderate (0.23555537231062282%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.9891519665015949 | 1.0443659333365456 | 1.0947988106684838 |
| Standard deviation | 4.335349267316851e-2 | 6.022873554592162e-2 | 7.311624854123903e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.47295107899117284 | 0.5415055478976379 | 0.5693411088383679 |
| Standard deviation | 2.3798124583360447e-2 | 5.162419204442124e-2 | 6.428347440869861e-2 |
Outlying measurements have moderate (0.22313793859264303%) effect on estimated standard deviation.
70% Lookups; 15% Insertions; 15% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.20460144067299552 | 0.25141798494267276 | 0.32784482479910365 |
| Standard deviation | 2.9958215397792996e-2 | 6.78706411877036e-2 | 8.887805812624672e-2 |
Outlying measurements have severe (0.584562346074393%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.18869292634481097 | 0.21225006392633075 | 0.22445602979633905 |
| Standard deviation | 9.409368017169864e-3 | 2.3271080615783226e-2 | 3.486401858272563e-2 |
Outlying measurements have moderate (0.30926262756075135%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3586892571195979 | 0.3933799489980932 | 0.42239766725106165 |
| Standard deviation | 1.2977111522935854e-2 | 3.595451322481452e-2 | 4.6545916350315145e-2 |
Outlying measurements have moderate (0.22077478837418332%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3577461156698215 | 0.4000980827125507 | 0.46441242338187294 |
| Standard deviation | 4.377080951223772e-4 | 6.064599650054138e-2 | 7.448265784393415e-2 |
Outlying measurements have moderate (0.457496106887165%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.41771441416373517 | 0.47359002947511425 | 0.510423285435536 |
| Standard deviation | 2.32080117954562e-2 | 5.434554496302535e-2 | 7.077474890404858e-2 |
Outlying measurements have moderate (0.23145967866281844%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.6691811920609325 | 0.7452537570619219 | 0.8329046452539236 |
| Standard deviation | 3.756270824912161e-2 | 9.543915412434553e-2 | 0.1323083840868225 |
Outlying measurements have moderate (0.23511340966291447%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.36872791600762866 | 0.4365433533312171 | 0.4596691740771348 |
| Standard deviation | 1.4729257498402148e-2 | 5.008468147545006e-2 | 6.680733487618234e-2 |
Outlying measurements have moderate (0.2314525170666296%) effect on estimated standard deviation.
80% Lookups; 10% Insertions; 10% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.19375323911258924 | 0.21959620243101705 | 0.2335084423493956 |
| Standard deviation | 6.600353272287911e-3 | 2.6307495402937066e-2 | 3.76265618769282e-2 |
Outlying measurements have moderate (0.31317210860013367%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomicModifyIORef HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15565147368922932 | 0.17315599282964186 | 0.18748273315189481 |
| Standard deviation | 1.2926808545425444e-2 | 2.1690827442577946e-2 | 3.542845869404315e-2 |
Outlying measurements have moderate (0.3148938358661476%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops IntMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.22892703060177158 | 0.2452991108352823 | 0.25507907280465586 |
| Standard deviation | 1.7756100062567871e-3 | 1.7285218763953537e-2 | 2.3627766108345556e-2 |
Outlying measurements have moderate (0.17482600160631773%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/IORef atomic-primops HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.23591974120160253 | 0.24868344301726514 | 0.2597051223146264 |
| Standard deviation | 1.2037216194303365e-2 | 1.6341110246275253e-2 | 1.976199390878015e-2 |
Outlying measurements have moderate (0.1710505287393976%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/TVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.24691481593763456 | 0.27630556623567826 | 0.2887664053030312 |
| Standard deviation | 6.049060078464005e-3 | 2.3011742665307654e-2 | 3.1698302018454604e-2 |
Outlying measurements have moderate (0.18197852791809502%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/MVar HashMap
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.4287205864384305 | 0.4469712751864184 | 0.4645125424431171 |
| Standard deviation | 1.1889092854377755e-2 | 1.9508025036828188e-2 | 2.755266057883879e-2 |
Outlying measurements have moderate (0.1875%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/StmContainers.Map
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.3148337755083048 | 0.34183710667033057 | 0.35966977033482783 |
| Standard deviation | 9.701672332691175e-3 | 2.6665018309812695e-2 | 3.678202416369316e-2 |
Outlying measurements have moderate (0.20987688085501488%) effect on estimated standard deviation.
90% Lookups; 5% Insertions; 5% Deletions/Concurrent HashTable
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16907791731419922 | 0.18847161336533544 | 0.19902905429221895 |
| Standard deviation | 2.903912806543049e-3 | 2.2695745146749373e-2 | 3.232775314974696e-2 |
Outlying measurements have moderate (0.3133793653732204%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.