Introduction

I have been professionally writing C/C++ code for about 6 years; overall, I have about 10 years of experience. Most of the code I’ve worked with has used return-based error handling. That being said, I occasionally run into code bases that are more exception oriented. I’ve often wondered how the two approaches compare to each other, so I’ve finally decided to do a few experiments to find out. The experiments in this post are purely focused on program performance. I do not consider the coding style and code complexity aspects of the error handling approaches.

All of my experiments will take the same form. I’ll develop two functions for each experiment; one function is return-based, the other exception-based. The functions will be provided with an integer argument that they’ll have to perform “error checking” logic on. The error checking logic will be identical between the two, and only differ based on error reporting. Each experiment will call the functions in a loop and I’ll be using cpptqdm to measure the loop’s performance.

My code can be found here: github.com/hankedan000/cpp-experiments

A note on CPU affinities

I ran all experiments with the CPU affinity set on the process; otherwise, the results were too inconsistent.

Experiment 1 - Binary Error Conditions

This experiment focuses on functions that can only report a binary error condition; either it fails or it doesn’t. The return-based function returns a bool, and the exception-based function will either throw or not.

bool ret_fun(int n) {
    if (n % 7 == 0) {
        return false;
    }
    return true;
}

void exc_fun(int n) {
    if (n % 7 == 0) {
        throw std::exception();
    }
}

The experiment’s calling code would call each function 10000000 times in three different circumstances.

1. mixed errors – caller passes in n = 0 to n = 9999999
2. all errors – caller always passes n = 0
3. no errors – caller always passes n = 1

Results

The takeaway from these results is that exception-based error handling is extremely costly if the exceptions are frequent.

Boolean returns are performant and consistent across all scenarios. I think the consistency part makes perfect sense because the caller is always checking for the error condition, regardless if it occured or not. I believe the performance delta between the 3 bool return cases is actually due to the error counter logic that the caller code is doing.

===========================================
EXPERIMENT 1
===========================================
== bool return - mixed errors
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 98.7 MHz | 0s<0s]   
errors = 1428572
== bool return - no errors
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 102.2 MHz | 0s<0s]  
errors = 0
== bool return - all errors
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 93.9 MHz | 0s<0s]  
errors = 10000000
== exception - mixed errors
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 2.9 MHz | 3s<0s]  
errors = 1428572
== exception - no errors (try-catch inside loop)
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 90.8 MHz | 0s<0s]  
errors = 0
== exception - no errors (try-catch outside loop)
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 93.6 MHz | 0s<0s]  
errors = 0
== exception - all errors
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 344.4 kHz | 29s<0s]  
errors = 10000000

Experiment 2 - Multiple Error Conditions

This experiment focuses on functions that can report multiple error conditions. The return-based function now returns an int instead of a bool, and the exception-based function is throwing 1 of a few custom exception types.

// possible return codes
#define OTHER   0
#define MULT7   1
#define MULT42  2
#define MULT100 3

int ret_fun2(int n) {
    if (n % 100 == 0) {
        return MULT100;
    } else if (n % 42 == 0) {
        return MULT42;
    } else if (n % 7 == 0) {
        return MULT7;
    }
    return OTHER;
}

// possible exceptions
class Mult7Exception : public std::exception {};
class Mult42Exception : public std::exception {};
class Mult100Exception : public std::exception {};

void exc_fun2(int n) {
    if (n % 100 == 0) {
        throw Mult100Exception();
    } else if (n % 42 == 0) {
        throw Mult42Exception();
    } else if (n % 7 == 0) {
        throw Mult7Exception();
    }
}

Results

The results from this experiment aren’t too enlightening, especially after seeing the results from experiment 1.

I think the only interesting result is that the switch-case calling code consistently under performed the if-else caller by a small margin. I’m not sure why this is, but it’s probably something compiler or use case dependent.

===========================================
EXPERIMENT 2
===========================================
== return code - switch case
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 59.5 MHz | 0s<0s]  
mult7 = 1180952; mult42 = 233334; mult100 = 100000
== return code - if else
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 62.5 MHz | 0s<0s]  
mult7 = 1180952; mult42 = 233334; mult100 = 100000
== exception
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 2.7 MHz | 4s<0s]  
mult7 = 1180952; mult42 = 233334; mult100 = 100000

Experiment 3 - Infrequent Error Conditions

This experiment focuses on functions that report extremely infrequent errors. The functions are similar to the previous experiment, but instead of checking if the input is a multiple of something, they’re now checking for equality. Each loop should only report 3 error conditions over all 10000000 iterations; that’s a 0.000030% error rate.

// possible return codes
#define OTHER    0
#define EQUAL7   1
#define EQUAL42  2
#define EQUAL100 3

int ret_fun3(int n) {
    if (n == 7) {
        return EQUAL7;
    } else if (n == 42) {
        return EQUAL42;
    } else if (n == 100) {
        return EQUAL100;
    }
    return OTHER;
}

// possible exceptions
class Equal7Exception : public std::exception {};
class Equal42Exception : public std::exception {};
class Equal100Exception : public std::exception {};

void exc_fun3(int n) {
    if (n == 7) {
        throw Equal7Exception();
    } else if (n == 42) {
        throw Equal42Exception();
    } else if (n == 100) {
        throw Equal100Exception();
    }
}

Results

I’ll be honest, I wrote this experiment hoping to show that exceptions can outperform return-based code. I guess this goes to show that exceptions only make sense in truly exceptional circumstances (when only considering performance).

===========================================
EXPERIMENT 3
===========================================
== return code - switch case
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 85.2 MHz | 0s<0s]  
equal7 = 1; equal42 = 1; equal100 = 1
== return code - if else
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 84.1 MHz | 0s<0s]  
equal7 = 1; equal42 = 1; equal100 = 1
== exception
  ████████████████████████████████████████▏ 100.0% [10000000/10000000 | 94.7 MHz | 0s<0s]  
equal7 = 1; equal42 = 1; equal100 = 1