# Performance

Obviously, the performance depends on the machine on which Triceps runs. One of the Perl unit tests,

    t/Perf.t

allows you to measure the Triceps performace on your own machine. By default it runs only one thousand iterations, to be fast and not delay the run of the full tests suite. But the number can be increased by setting an environment variable, such as:

    $ TRICEPS_PERF_COUNT=100000 perl t/Perf.t
    Performance test, 100000 iterations, real time.
    Empty Perl loop 0.004426 s, 22592534.34 per second.
    Empty Perl function of 5 args 0.017969 s, 5565025.41 per second.
    Empty Perl function of 10 args 0.021972 s, 4551308.65 per second.
    Row creation from array and destruction 0.169338 s, 590536.61 per second.
    Row creation from hash and destruction 0.192646 s, 519086.75 per second.
    Rowop creation and destruction 0.074600 s, 1340491.48 per second.
    Calling a dummy label 0.029692 s, 3367945.01 per second.
    Calling a chained dummy label 0.031990 s, 3125995.16 per second.
      Pure chained call 0.002298 s, 43513891.48 per second.
    Calling a Perl label 0.168692 s, 592796.78 per second.
    Row handle creation and destruction 0.077752 s, 1286142.62 per second.
    Repeated table insert (single hashed idx, direct) 0.114952 s, 869927.66 per second.
    Repeated table insert (single hashed idx, direct & Perl construct) 0.214694 s, 465778.63 per second.
      RowHandle creation overhead in Perl 0.099742 s, 1002584.92 per second.
    Repeated table insert (single ordered int idx, direct) 0.119679 s, 835568.66 per second.
    Repeated table insert (single perl sorted idx, direct) 0.619665 s, 161377.42 per second.
    Repeated table insert (single hashed idx, call) 0.122374 s, 817165.07 per second.
    Table insert makeRowArray (single hashed idx, direct) 0.410910 s, 243362.22 per second.
      Excluding makeRowArray 0.241573 s, 413954.20 per second.
    Table insert makeRowArray (double hashed idx, direct) 0.484066 s, 206583.19 per second.
      Excluding makeRowArray 0.314729 s, 317733.69 per second.
      Overhead of second index 0.073156 s, 1366935.21 per second.
    Table insert makeRowArray (single ordered int idx, direct) 0.460580 s, 217117.61 per second.
      Excluding makeRowArray 0.291242 s, 343356.65 per second.
    Table insert makeRowArray (single ordered string idx, direct) 0.907807 s, 110155.59 per second.
      Excluding makeRowArray 0.738469 s, 135415.23 per second.
    Table insert makeRowArray (single perl sorted idx, direct) 12.140260 s, 8237.06 per second.
      Excluding makeRowArray 11.970923 s, 8353.57 per second.
    Table lookup (single hashed idx) 0.088578 s, 1128951.72 per second.
    Table lookup (single ordered int idx) 0.127015 s, 787309.36 per second.
    Table lookup (single ordered string idx) 0.258478 s, 386879.52 per second.
    Table lookup (single perl sorted idx) 3.727754 s, 26825.81 per second.
    Lookup join (single hashed idx) 1.635421 s, 61146.32 per second.
    Nexus pass (1 row/flush) 0.194848 s, 513219.77 per second.
    Nexus pass (10 rows/flush) 0.796733 s, 1255125.02 per row per second.
      Overhead of each row 0.066876 s, 1495302.06 per second.
      Overhead of flush 0.127972 s, 781419.84 per second.

An important caveat, the test is of the Perl interface, so it includes all the overhead of constructing the Perl objects. I've tried to structure it so that some of the underlying performance can be deduced, but it's still approximate. I haven't done the performance testing of just the underlying &Cpp; implementation yet, it will be better.

The computations are done with the real elapsed time, so if the machine is not idle, the time of the other processes will get still counted against the tests, and the results will show slower than they really are.

The numbers above are from my old-ish laptop (dual-CPU Intel Core2 T9900 3GHz). The time in seconds printed for each value is for the whole test loop of 100K iterations. The per second number shows the opposite, how many loop iterations were done per second.

I've had an opportunity to run the performance tests on a few more laptops. All of the same Core2 generation, but with the different CPU frequencies. The 2GHz version showed expectedly an about 30% lower performance proportionally. A 2.2GHz CPU also showed an about proportional change, except for the inter-thread communication through the nexus, it went down more than proportional. I'm not sure, what was up with the 2.2GHz CPU, maybe the timing worked out just wrong to add more overhead.

The speed is improved a lot by the high optimization levels, the numbers above were produced with GCC -O3. The newer versions of both the GCC and Perl produce a marked improvement of performance on the old ones.

Here is the row-by-row description of the measurements:

    Performance test, 100000 iterations, real time.

The first thing it prints is the iteration count, to set the expectations for the run length and precision. The shorter runs tend to include more randomness.

    Empty Perl loop 0.004426 s, 22592534.34 per second.
    Empty Perl function of 5 args 0.017969 s, 5565025.41 per second.
    Empty Perl function of 10 args 0.021972 s, 4551308.65 per second.

A calibration to see, how much overhead is added by the execution of the loop itself. As it turns out, not much. The Perl function calls add more but still don't dominate the numbers.

    Row creation from array and destruction 0.169338 s, 590536.61 per second.

The

    makeRowArray()

for a row of 5 fields. Each created row gets destroyed before the next one gets created.

    Row creation from hash and destruction 0.192646 s, 519086.75 per second.

The

    makeRowHash()

for a row of 5 fields.

    Rowop creation and destruction 0.074600 s, 1340491.48 per second.

The

    makeRowop()

from an existing row. Same thing, each rowop gets destroyed before constructing the next one.

    Calling a dummy label 0.029692 s, 3367945.01 per second.

Repeated calls of a dummy label with the same rowop object. This can be thought of as the basic overhead of a &Cpp; label call, with a limited amount (25-30% or so) of the Perl overhead mixed in.

    Calling a chained dummy label 0.031990 s, 3125995.16 per second.
      Pure chained call 0.002298 s, 43513891.48 per second.

Repeated calls of a dummy label that has another dummy label chained to it. The pure part is the difference from the previous case that gets added by appending another chained dummy label.

    Calling a Perl label 0.168692 s, 592796.78 per second.

Repeated calls of a Perl label with the same rowop object. The Perl label has an empty

    sub

but that empty

    sub

still gets executed, along with all the support functionality.

    Row handle creation and destruction 0.077752 s, 1286142.62 per second.

The creation of a table's row handle from a single row, including the creation of the Perl wrapper for the row handle object.

    Repeated table insert (single hashed idx, direct) 0.114952 s, 869927.66 per second.

Insert of the same row into a table. Since the row is the same, it keeps replacing the previous one, and the table size stays at 1 row. The code is

    $tSingleHashed->insert($row1)

. Even though the row is the same, a new row handle gets constructed for it every time by the table. Single hashed idx means that the table has a single Hashed index, on an

    int32

field. Direct means the direct

    insert()

call, as opposed to using the table's input label.

    Repeated table insert (single hashed idx, direct & Perl construct) 0.214694 s, 465778.63 per second.
      RowHandle creation overhead in Perl 0.099742 s, 1002584.92 per second.

The same, only the row handles are constructed in Perl before inserting them:

    $tSingleHashed->insert($tSingleHashed->makeRowHandle($row1))

. And the second line shows that the overhead of wrapping the row handles for Perl is pretty noticeable (it's the difference from the previous test case).

    Repeated table insert (single ordered int idx, direct) 0.119679 s, 835568.66 per second.

The same thing, only for a table that uses an Ordered index that executes a comparison on the same

    int32

field. The performance is very close to the hashed index.

    Repeated table insert (single perl sorted idx, direct) 0.619665 s, 161377.42 per second.

The same thing, only for a table that uses a PerlSorted index that executes a Perl comparison on the same

    int32

field. As you can see, it gets about 5 times slower.

    Repeated table insert (single hashed idx, call) 0.122374 s, 817165.07 per second.

The same thing, again the table with a single Hashed index, but this time by sending the rowops to its input label.

    Table insert makeRowArray (single hashed idx, direct) 0.410910 s, 243362.22 per second.
      Excluding makeRowArray 0.241573 s, 413954.20 per second.

Now the different rows get inserted into the table, each row having a different key. At the end of this test the table contains 100K rows (or however many were requested by the environment variable). Naturally, this is slower than the repeated insertions of the same row, since the tree of the table's index becomes deeper and requires more comparisons and rebalancing. This performance will be lower in the tests with more rows, since the index will become deeper and will create more overhead. Since the rows are all different, they are created on the fly, so this row creation overhead needs to be excluded to get the actual Table's performance.

    Table insert makeRowArray (double hashed idx, direct) 0.484066 s, 206583.19 per second.
      Excluding makeRowArray 0.314729 s, 317733.69 per second.
      Overhead of second index 0.073156 s, 1366935.21 per second.

Similar to previous one but on a table that has two Hashed indexes (both on the same

    int32

field). The details here also compute the overhead contributed by the second index.

    Table insert makeRowArray (single ordered int idx, direct) 0.460580 s, 217117.61 per second.
      Excluding makeRowArray 0.291242 s, 343356.65 per second.
    Table insert makeRowArray (single ordered string idx, direct) 0.907807 s, 110155.59 per second.
      Excluding makeRowArray 0.738469 s, 135415.23 per second.

Similar but for a table with an Ordered index. This version on an int field is just slightly slower. Using a string key instead of an integer makes the index about 3 times slower.

    Table insert makeRowArray (single perl sorted idx, direct) 12.140260 s, 8237.06 per second.
      Excluding makeRowArray 11.970923 s, 8353.57 per second.

Similar but for a table with a PerlSorted index that executes the comparison in a Perl expression. As you can see, with the makeRowArray overhead excluded it's about 50 times slower than the ordering done in &Cpp; (and it gets even worse for the larger row sets).

    Table lookup (single hashed idx) 0.088578 s, 1128951.72 per second.

Finding a row in a table of 100K (or however many iterations requested) rows by a hashed index.

    Table lookup (single ordered int idx) 0.127015 s, 787309.36 per second.
    Table lookup (single ordered string idx) 0.258478 s, 386879.52 per second.

Finding a row in a table of 100K (or however many iterations requested) rows by an ordered index, with an integer and string field used as a key.

    Table lookup (single perl sorted idx) 3.727754 s, 26825.81 per second.

Finding a row in a table of 100K (or however many iterations requested) rows by a Perl sorted index. Just like row insertion, it's massively slower than the ordering in &Cpp; but not quite that much, only by a factor of 30 rather than 50.

    Lookup join (single hashed idx) 1.635421 s, 61146.32 per second.

Joins in the LookupJoin template (and the performance of the JoinTwo template will be the same, since it consists of two LookupJoins). It performs essentially the same table lookup, only with the added overhead of constructing the new combined rows, and the general overhead of the extra Perl label calls.

    Nexus pass (1 row/flush) 0.194848 s, 513219.77 per second.
    Nexus pass (10 rows/flush) 0.796733 s, 1255125.02 per row per second.
      Overhead of each row 0.066876 s, 1495302.06 per second.
      Overhead of flush 0.127972 s, 781419.84 per second.

The passing of the rows between threads through a Nexus. The time also includes the draining and stopping of the app, so it's a little pessimistic, and gets more pessimistic for the short runs. Both cases pass 100K of transactions, so the second one passes 10 times more rows. The difference in performance of different transaction sizes allows to compute the overhead of the transaction passing (i.e. flushes) versus the overhead of adding extra rows to transactions.

performance