# Added Pont to Plane Distance, Region growing is slower now
# This can be disabled with setting z_thresh=0.0 and getting the previous speed results

(ppb) ➜  polylidar git:(major_changes_refactor) ./cmake-build/bin/run-bench --benchmark_repetitions=3 --benchmark_display_aggregates_only    
2020-06-07 10:03:27
Running ./cmake-build/bin/run-bench
Run on (24 X 3800 MHz CPU s)
CPU Caches:
  L1 Data 32K (x12)
  L1 Instruction 32K (x12)
  L2 Unified 512K (x12)
  L3 Unified 16384K (x4)
Load Average: 1.01, 0.54, 0.47
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
-----------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                               Time             CPU   Iterations
-----------------------------------------------------------------------------------------------------------------------------------------
ImagesAndSparseMesh/BM_Create_PointCloud/real_time_mean                                               106 us          106 us            3
ImagesAndSparseMesh/BM_Create_PointCloud/real_time_median                                             106 us          106 us            3
ImagesAndSparseMesh/BM_Create_PointCloud/real_time_stddev                                            1.23 us         1.23 us            3
ImagesAndSparseMesh/BM_Create_TriMesh/real_time_mean                                                  364 us          364 us            3
ImagesAndSparseMesh/BM_Create_TriMesh/real_time_median                                                363 us          363 us            3
ImagesAndSparseMesh/BM_Create_TriMesh/real_time_stddev                                               1.66 us         1.66 us            3
ImagesAndSparseMesh/BM_ComputeTriangleNormals/real_time_mean                                          304 us          304 us            3
ImagesAndSparseMesh/BM_ComputeTriangleNormals/real_time_median                                        304 us          303 us            3
ImagesAndSparseMesh/BM_ComputeTriangleNormals/real_time_stddev                                       2.03 us         2.03 us            3
ImagesAndSparseMesh/BM_CopyTriMesh/real_time_mean                                                    4750 us         4748 us            3
ImagesAndSparseMesh/BM_CopyTriMesh/real_time_median                                                  4698 us         4697 us            3
ImagesAndSparseMesh/BM_CopyTriMesh/real_time_stddev                                                   129 us          129 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygons/real_time_mean                                        295 us          295 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygons/real_time_median                                      295 us          295 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygons/real_time_stddev                                     1.26 us         1.25 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsOptimized/real_time_mean                               392 us          253 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsOptimized/real_time_median                             393 us          253 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsOptimized/real_time_stddev                            1.23 us        0.234 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/1/real_time_mean                   290 us          290 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/1/real_time_median                 289 us          289 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/1/real_time_stddev                1.26 us         1.27 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/2/real_time_mean                   334 us          334 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/2/real_time_median                 334 us          334 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/2/real_time_stddev               0.390 us        0.383 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/3/real_time_mean                   561 us          561 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/3/real_time_median                 561 us          561 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/3/real_time_stddev                1.54 us         1.54 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/4/real_time_mean                   873 us          873 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/4/real_time_median                 873 us          873 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/4/real_time_stddev                4.40 us         4.36 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/1/real_time_mean          396 us          255 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/1/real_time_median        395 us          255 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/1/real_time_stddev       2.52 us         1.08 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/2/real_time_mean          368 us          246 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/2/real_time_median        365 us          244 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/2/real_time_stddev       6.72 us         4.15 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/3/real_time_mean          518 us          333 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/3/real_time_median        517 us          334 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/3/real_time_stddev       5.76 us         2.43 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/4/real_time_mean          642 us          382 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/4/real_time_median        645 us          379 us            3
ImagesAndSparseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/4/real_time_stddev       5.87 us         5.65 us            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/1/real_time_mean                            89.8 ms         89.5 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/1/real_time_median                          90.4 ms         89.8 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/1/real_time_stddev                          1.51 ms         1.43 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/2/real_time_mean                             118 ms          117 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/2/real_time_median                           116 ms          116 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/2/real_time_stddev                          2.90 ms         1.93 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/3/real_time_mean                             143 ms          143 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/3/real_time_median                           143 ms          143 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/3/real_time_stddev                         0.381 ms        0.380 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/4/real_time_mean                             173 ms          173 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/4/real_time_median                           174 ms          174 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormals/4/real_time_stddev                          1.30 ms         1.30 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/1/real_time_mean                   61.3 ms         20.8 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/1/real_time_median                 61.3 ms         21.2 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/1/real_time_stddev                 1.06 ms        0.573 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/2/real_time_mean                   58.7 ms         21.3 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/2/real_time_median                 58.5 ms         21.5 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/2/real_time_stddev                 2.07 ms         1.16 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/3/real_time_mean                   59.1 ms         22.6 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/3/real_time_median                 60.3 ms         23.3 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/3/real_time_stddev                 2.50 ms         1.75 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/4/real_time_mean                   68.0 ms         24.1 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/4/real_time_median                 67.3 ms         24.1 ms            3
DenseMesh/BM_ExtractPlanesAndPolygonsFromMultipleNormalsOptimized/4/real_time_stddev                 2.49 ms         1.11 ms            3



(ppb) ➜  polylidar git:(major_changes_refactor) ✗ pytest python_tests/benchmark_test.py
===================================================================================================== test session starts =====================================================================================================
platform linux -- Python 3.7.6, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/jeremy/Documents/UMICH/Research/polylidar
plugins: benchmark-3.2.3
collected 20 items                                                                                                                                                                                                            

python_tests/benchmark_test.py ....................                                                                                                                                                                     [100%]


------------------------------------------------------------------------------------------------ benchmark: 20 tests -------------------------------------------------------------------------------------------------
Name (time in us)                   Min                    Max                   Mean                StdDev                 Median                   IQR            Outliers         OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_clusters[2-1000]          753.8510 (1.0)       1,459.3720 (1.07)        789.6324 (1.0)         71.4556 (5.23)        775.8610 (1.0)         15.5570 (1.07)        18;31  1,266.4121 (1.0)         611           1
test_clusters[3-1000]        1,150.2410 (1.53)      1,359.8010 (1.0)       1,168.0174 (1.48)        13.6576 (1.0)       1,165.2610 (1.50)        14.5075 (1.0)         103;7    856.1517 (0.68)        745           1
test_clusters[4-1000]        1,556.2910 (2.06)      1,624.7820 (1.19)      1,584.3352 (2.01)        14.5405 (1.06)      1,582.0510 (2.04)        23.7220 (1.64)        219;0    631.1795 (0.50)        611           1
test_clusters[5-1000]        2,002.2620 (2.66)      2,083.8230 (1.53)      2,037.4585 (2.58)        19.6444 (1.44)      2,037.1470 (2.63)        35.2390 (2.43)        182;0    490.8076 (0.39)        470           1
test_clusters[6-1000]        2,461.4620 (3.27)      2,656.9930 (1.95)      2,507.2811 (3.18)        32.5430 (2.38)      2,504.1075 (3.23)        57.5110 (3.96)        126;2    398.8384 (0.31)        350           1
test_clusters[7-1000]        2,911.2820 (3.86)      3,051.8920 (2.24)      2,966.9153 (3.76)        38.8100 (2.84)      2,957.7125 (3.81)        58.6200 (4.04)        108;0    337.0504 (0.27)        326           1
test_clusters[8-1000]        3,330.5530 (4.42)      3,495.5820 (2.57)      3,390.7177 (4.29)        46.8285 (3.43)      3,376.1685 (4.35)        75.7890 (5.22)         97;0    294.9228 (0.23)        276           1
test_clusters[9-1000]        3,750.9040 (4.98)      5,118.9650 (3.76)      3,838.9576 (4.86)       103.9315 (7.61)      3,814.2790 (4.92)        94.0595 (6.48)          5;2    260.4874 (0.21)        232           1
test_clusters[10-1000]       4,200.5340 (5.57)      5,710.5740 (4.20)      4,310.1171 (5.46)       151.3377 (11.08)     4,271.8440 (5.51)        97.1597 (6.70)          7;6    232.0123 (0.18)        227           1
test_clusters[2-10000]       9,367.9080 (12.43)     9,627.9780 (7.08)      9,495.8285 (12.03)       67.7889 (4.96)      9,491.3680 (12.23)      120.0492 (8.27)         41;0    105.3094 (0.08)        103           1
test_clusters[3-10000]      14,280.6820 (18.94)    16,790.0150 (12.35)    14,535.9505 (18.41)      289.5276 (21.20)    14,520.2575 (18.72)      138.9490 (9.58)          1;1     68.7950 (0.05)         70           1
test_clusters[4-10000]      19,541.7270 (25.92)    20,187.5580 (14.85)    19,772.7743 (25.04)       98.9859 (7.25)     19,768.0380 (25.48)      116.8997 (8.06)         11;1     50.5746 (0.04)         51           1
test_clusters[5-10000]      25,644.3040 (34.02)    26,658.3140 (19.60)    25,849.0565 (32.74)      172.4456 (12.63)    25,820.7375 (33.28)      105.9745 (7.30)          6;3     38.6861 (0.03)         40           1
test_clusters[6-10000]      30,810.5290 (40.87)    32,371.1590 (23.81)    31,268.8051 (39.60)      486.9431 (35.65)    31,025.4320 (39.99)      648.3460 (44.69)         6;0     31.9808 (0.03)         32           1
test_clusters[7-10000]      36,594.3830 (48.54)    37,350.9830 (27.47)    36,861.4549 (46.68)      156.1049 (11.43)    36,829.3520 (47.47)       99.1085 (6.83)          4;3     27.1286 (0.02)         29           1
test_clusters[8-10000]      42,186.2690 (55.96)    44,215.9800 (32.52)    42,737.6805 (54.12)      421.7678 (30.88)    42,666.7080 (54.99)      282.4660 (19.47)         4;2     23.3986 (0.02)         24           1
test_clusters[9-10000]      48,151.1820 (63.87)    51,781.9860 (38.08)    48,981.1293 (62.03)      936.5056 (68.57)    48,591.6940 (62.63)      353.5177 (24.37)         3;3     20.4160 (0.02)         21           1
test_100k_array_lmax        51,596.9150 (68.44)    60,640.6040 (44.60)    54,036.7646 (68.43)    2,149.6361 (157.39)   53,261.7480 (68.65)    2,262.5162 (155.95)        3;1     18.5059 (0.01)         17           1
test_clusters[10-10000]     53,908.7580 (71.51)    55,310.8390 (40.68)    54,412.5224 (68.91)      456.0746 (33.39)    54,223.1785 (69.89)      686.8790 (47.35)         5;0     18.3781 (0.01)         18           1
test_100k_array_3d_lmax     59,455.9030 (78.87)    61,508.5260 (45.23)    60,163.7223 (76.19)      633.1570 (46.36)    60,000.5635 (77.33)      282.4710 (19.47)         4;4     16.6213 (0.01)         18           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
===================================================================================================== 20 passed in 21.87s =====================================================================================================
