012 python cProfile (part 1)

Using tool cProfile to take a look: how to measure and judge performance.

More detail reference: https://docs.python.org/2/library/profile.html
Example: https://github.com/vash-hsu/python/blob/master/python_test/brute_force.py

to guess 5-digits integer

• rainbowtable takes 0.758 seconds because he needs to build table for 1~99999 in advance.
• brute_force takes 0.287 seconds because he only needs to try 1~43851 before answering.

python -m cProfile -s cumtime ./brute_force.py -l 1 -s 5
----- Run 1
* Random Gen User Digit Passcode 43851 as SHA1 dbb97b9a933f6a1993640fbe6c558cbfe08e2a39
* Try to lookup from rainbow table --> 43851
* Try to lookup from brute force   --> 43851
         287789 function calls in 1.053 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.001    0.001    1.053    1.053 brute_force.py:3(<module>)
        1    0.000    0.000    1.047    1.047 brute_force.py:126(main)
        1    0.000    0.000    1.047    1.047 brute_force.py:99(example_for_password_guessing)
        1    0.027    0.027    0.758    0.758 brute_force.py:75(guess_by_rainbowtable)
        1    0.378    0.378    0.732    0.732 brute_force.py:60(rainbow_table_fill)
        1    0.140    0.140    0.289    0.289 brute_force.py:86(guess_by_realtime_brute_force)
   143851    0.253    0.000    0.253    0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
   143852    0.245    0.000    0.245    0.000 {_hashlib.openssl_sha1}
        1    0.004    0.004    0.004    0.004 hashlib.py:72(<module>)
        5    0.004    0.001    0.004    0.001 {range}

.

to guess 6-digits integer

• rainbowtable takes 7.580 seconds because he needs to build table for 1~999999 in advance.
• brute_force takes 1.554 seconds because he only needs to try 1~235113 before answering.

python -m cProfile -s cumtime ./brute_force.py -l 1 -s 6
----- Run 1
* Random Gen User Digit Passcode 235113 as SHA1 8eb0632e437d719d4a30c3bbfd0862703b726482
* Try to lookup from rainbow table --> 235113
* Try to lookup from brute force   --> 235113
         2470313 function calls in 9.140 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.001    0.001    9.140    9.140 brute_force.py:3(<module>)
        1    0.000    0.000    9.134    9.134 brute_force.py:126(main)
        1    0.000    0.000    9.134    9.134 brute_force.py:99(example_for_password_guessing)
        1    0.307    0.307    7.580    7.580 brute_force.py:75(guess_by_rainbowtable)
        1    3.798    3.798    7.273    7.273 brute_force.py:60(rainbow_table_fill)
  1235113    2.150    0.000    2.150    0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
  1235114    2.074    0.000    2.074    0.000 {_hashlib.openssl_sha1}
        1    0.754    0.754    1.554    1.554 brute_force.py:86(guess_by_realtime_brute_force)
        5    0.051    0.010    0.051    0.010 {range}

.

Want more friendly report? Try cProfile + cachegrind!

Loop only once, 6-digits

python -m cProfile -o cpf_1_6.out ./brute_force.py -l 1 -s 6
pyprof2calltree -i cpf_1_6.out -o cpf_1_6.calltree
kcachegrind cpf_1_6.calltree

Loop twice, 6-digits

python -m cProfile -o cpf_2_6.out ./brute_force.py -l 2 -s 6
pyprof2calltree -i cpf_2_6.out -o cpf_2_6.calltree
kcachegrind cpf_2_6.calltree

Loop twice, 6-digits, reusing Rainbow Table

python -m cProfile -o cpf_2_6_r.out ./brute_force.py -l 2 -s 6 -r
pyprof2calltree -i cpf_2_6_r.out -o cpf_2_6_r.calltree
kcachegrind cpf_2_6_r.calltree

Hint:

yum install graphviz

wget https://pypi.python.org/packages/source/p/pyprof2calltree/pyprof2calltree-1.3.2.tar.gz

tar -zxvf pyprof2calltree-1.3.2.tar.gz

cd pyprof2calltree-1.3.2

ython setup.py install -v

yum install kcachegrind

  At this moment, we got some feeling about what will happen if we test scenario "loop triple, 6-digit, reusing rainbow table". And, those performance test results do not only show which method is more economy but also help us to dig which code to improve.

continue ...