0x00 IO performance performance: 1000(131832 B): read-write: 13.414876s 2000(270908 B): read-write: 25.270484s 5000(664204 B): read-write: 64.528306s About 20MB/s 0x01 IO & Computing time benchmark (with embed_rate = 0.2) (1)file sum = 100 serial: A. (format)time elapsed: 0.583396s (embed)time elapsed: 30.847788s (extract)time elapsed: 57.884814s real 89.99s user 89.58s sys 0.37s CPU 99% B. (format)time elapsed: 0.595019s (embed)time elapsed: 30.959033s (extract)time elapsed: 57.931772s real 90.16s user 89.68s sys 0.46s CPU 99% C. (format)time elapsed: 0.690753s (embed)time elapsed: 31.222347s (extract)time elapsed: 57.818952s real 90.43s user 89.80s sys 0.47s CPU 99% spark: A. (format table)time elapsed: 6.776471s (read table)time elapsed: 0.060646s (process)time elapsed: 39.127508 s real 59.50s user 11.81s sys 1.09s CPU 21% B. (format table)time elapsed: 6.470915s (read table)time elapsed: 1.542012s (process)time elapsed: 35.019916 s real 58.32s user 11.82s sys 1.11s CPU 22% C. (format table)time elapsed: 5.734575s (read table)time elapsed: 1.485326s (process)time elapsed: 35.217565 s real 57.13s user 12.15s sys 1.06s CPU 23% (2)file sum = 200 serial: A. (format)time elapsed: 1.147411s (embed)time elapsed: 62.815709s (extract)time elapsed: 118.217859s real 182.86s user 181.89s sys 0.88s CPU 99% B. (format)time elapsed: 1.201338s (embed)time elapsed: 62.710831s (extract)time elapsed: 117.897544s real 182.49s user 181.71s sys 0.72s CPU 99% C. (format)time elapsed: 1.131681s (embed)time elapsed: 62.880462s (extract)time elapsed: 118.124955s real 182.82s user 181.95s sys 0.77s CPU 99% spark: A. (format table)time elapsed: 10.535134s (read table)time elapsed: 2.932193s (process)time elapsed: 47.566111 s real 84.84s user 13.98s sys 1.30s CPU 18% B. (format table)time elapsed: 29.667743s (read table)time elapsed: 0.136938s (process)time elapsed: 46.093956 s real 135.58s user 14.52s sys 1.62s CPU 11% C. (format table)time elapsed: 13.773469s (read table)time elapsed: 3.047374s (process)time elapsed: 45.717876 s real 90.10s user 13.60s sys 1.43s CPU 16% (3)file sum = 500 serial: A. (format)time elapsed: 2.763806s (embed)time elapsed: 162.806317s (extract)time elapsed: 299.778606s real 466.03s user 462.83s sys 2.85s CPU 99% B. (format)time elapsed: 2.773738s (embed)time elapsed: 160.579258s (extract)time elapsed: 296.961467s real 461.01s user 458.86s sys 1.80s CPU 99% C. (format)time elapsed: 2.991173s (embed)time elapsed: 161.909154s (extract)time elapsed: 296.754535s real 462.34s user 460.02s sys 2.00s CPU 99% spark: A. (format table)time elapsed: 18.399418s (read table)time elapsed: 0.298133s (process)time elapsed: 84.794703 s real 143.67s user 18.91s sys 1.93s CPU 14% B. (format table)time elapsed: 31.979529s (read table)time elapsed: 7.316972s (process)time elapsed: 82.664176 s real 196.24s user 19.40s sys 2.49s CPU 11% C. (format table)time elapsed: 27.639130s (read table)time elapsed: 0.228689s (process)time elapsed: 85.939402 s real 187.12s user 19.50s sys 2.71s CPU 11% (4)file sum = 1000 serial: A. (format)time elapsed: 6.372794s (embed)time elapsed: 329.023151s (extract)time elapsed: 600.438977s real 936.53s user 931.93s sys 3.11s CPU 99% B. (format)time elapsed: 5.329308s (embed)time elapsed: 327.279783s (extract)time elapsed: 596.171131s real 929.46s user 925.83s sys 3.09s CPU 99% C. (format)time elapsed: 5.180158s (embed)time elapsed: 318.844047s (extract)time elapsed: 585.748167s real 910.47s user 907.03s sys 2.98s CPU 99% spark: A. (format table)time elapsed: 70.172458s (read table)time elapsed: 17.022568s (process)time elapsed: 159.624198 s real 431.68s user 31.02s sys 4.54s CPU 8% B. (format table)time elapsed: 87.720660s (read table)time elapsed: 15.644418s (process)time elapsed: 176.987813 s real 428.69s user 29.12s sys 3.91s CPU 7% C (format table)time elapsed: 120.458491s (read table)time elapsed: 0.586775s (process)time elapsed: 163.598833 s real 486.40s user 28.65s sys 5.33s CPU 6% (5)file sum = 2000 serial: A. (format)time elapsed: 14.960961s (embed)time elapsed: 679.357936s (extract)time elapsed: 1256.341536s real 1951.35s user 1942.26s sys 3.25s CPU 99% B. (format)time elapsed: 34.474327s (embed)time elapsed: 660.667673s (extract)time elapsed: 1232.833102s real 1928.65s user 1900.41s sys 3.30s CPU 98% C. (format)time elapsed: 10.912531s (embed)time elapsed: 661.449268s (extract)time elapsed: 1233.958305s real 1907.00s user 1901.39s sys 3.19s CPU 99% spark: A. (format table)time elapsed: 176.126157s (read table)time elapsed: 1.319059s (process)time elapsed: 343.191293 s real 956.85s user 65.04s sys 11.55s CPU 8% B. (format table)time elapsed: 262.840257s (read table)time elapsed: 76.712754s (process)time elapsed: 368.793729 s real 931.69s user 51.49s sys 6.66s CPU 6% C. (format table)time elapsed: 210.607617s (read table)time elapsed: 1.118159s (process)time elapsed: 354.279433 s real 995.37s user 56.69s sys 11.84s CPU 6% (6)file sum = 5000 serial: A. (format)time elapsed: 39.880657s (embed)time elapsed: 1652.537536s (extract)time elapsed: 3067.980390s real 4761.09s user 4727.25s sys 7.79s CPU 99% B. (format)time elapsed: 99.882375s (embed)time elapsed: 1600.979427s (extract)time elapsed: 2970.367912s real 4671.91s user 4578.77s sys 7.62s CPU 98% C. (format)time elapsed: 30.885324s (embed)time elapsed: 1612.269242s (extract)time elapsed: 2973.932865s real 4617.78s user 4596.83s sys 7.40s CPU 99% spark: A. (format table)time elapsed: 958.231221s (read table)time elapsed: 88.844213s (process)time elapsed: 852.649969 s real 3162.58s user 101.27s sys 30.68s CPU 4% B. (format table)time elapsed: 486.504693s (read table)time elapsed: 3.625127s (process)time elapsed: 811.479755 s real 2361.70s user 126.35s sys 36.82s CPU 6% C. (format table)time elapsed: 715.228832s (read table)time elapsed: 42.838806s (process)time elapsed: 821.420032 s real 2282.66s user 138.61s sys 30.21s CPU 7% 0x02 model performance benchmark (with file sum = 5000) (1)embed_rate = 0.2 serial: spark: