JavaScript Performance Measurement & Variability

So, I have a couple of questions about performance in JavaScript. I do high performance C/C++/CUDA professionally, so I’m trying to see what of that knowledge is applicable to JavaScript.

In another thread I played with some performance optimization for the Sum Primes challenge. I mostly used C style optimizations, but I think that my performance is reasonable.

repl link

https://repl.it/repls/ShamefulKnownScans

code
// ********************************************************************************
// Sieve of Eratoshenes sumPrimes demo and performance study
// ********************************************************************************

// --------------------------------------------------------------------------------
// Performance logging
// --------------------------------------------------------------------------------
const performance = require('perf_hooks').performance;

// --------------------------------------------------------------------------------
// Debug via console logging flag
// --------------------------------------------------------------------------------
const debug = false;

// --------------------------------------------------------------------------------
// Brief: Sum primes from 2 to num (inclusive)
//
// Inputs:
//  num - upper bound to sum primes (inclusive)
//
// Outputs:
//  sum - sum of primes from 2 to num (inclusive)
// --------------------------------------------------------------------------------
function sumPrimes(num) {
  if (debug) console.log("WARNING - DEBUGGING LOGGING WILL DECREASE PREFORMANCE");

  // Bounds checking
  if (num <= 1)
    return 0;

  // Make boolean array of odd numbers
  const upper = Math.floor((num - 1)/2);
  const isPrime = Array(upper).fill(true); // 'Guess' all are prime
  if (debug) console.log(" -- Upper: " + upper);

  // Initalize sum
  let sum = 2;

  // Mark multiples of each prime as false (not prime)
  const sqrtUpper = Math.floor((Math.sqrt(num) - 1)/2); // Done when i*i marked
  for (let i = 0; i <= sqrtUpper; ++i)
    // Check if number is multiple of any smaller odd number
    if (isPrime[i]) {
      // Add to sum
      const prime = 2*i+3; // Note that number = index*2+3
      sum += prime;

      // Mark all multiples of this number as false (not prime)
      const primeSqaredIndex = 2*i*i + 6*i + 3; // Everything below prime*2 marked
      for (let j = primeSqaredIndex; j < upper; j+=prime)
        isPrime[j] = false;
    }
  if (debug) console.log(" -- isPrime: " + isPrime);

  // Count remaining primes in sum
  for (let i = sqrtUpper + 1; i < upper; ++i)
    if (isPrime[i])
      sum += 2*i+3;
  if (debug) console.log(" -- Sum: " + sum);

  // Return
  if (debug) console.log("END DEBUGGING OUTPUT");
  return sum;
}

// --------------------------------------------------------------------------------
// Performance testing
// --------------------------------------------------------------------------------
// Test cases
const maxPower = 8;
const numRuns = 25;
console.log("--------------------------------------------------");
console.log("Summary")
console.log(" - Number of Test Cases : " + maxPower);
console.log(" - Runs Per Test Case   : " + numRuns);
console.log("--------------------------------------------------\n\n");

for (let i = 0; i < maxPower; i++) {
  /// Log test case
  const num = 5*(10**i);
  console.log("--------------------------------------------------");
  console.log("Test Case " + i);
  console.log("--------------------------------------------------");

  // Multiple runs
  let sum = 0;
  let times = [];
  for (let j = 0; j < numRuns; j++) {
    // Time execution
    const t0 = performance.now();
    sum = sumPrimes(num);
    const t1 = performance.now();

    // Log time elapsed
    times.push(t1 - t0);
  }

  // Compute stats
  const minTime = Math.min(...times);
  const maxTime = Math.max(...times);
  const avgTime = times.reduce((sum, item) => sum + item, 0) / numRuns;
  const variance = times.reduce((sumSqDiff, item) => sumSqDiff + (avgTime - item)**2, 0) / numRuns;
  const stdDev = Math.sqrt(variance);

  // Output
  console.log(" - Problem Values");
  console.log("     Num                : " + num);
  console.log("     Result             : " + sum);
  console.log(" - Statistics");
  console.log("     Average Time       : " + avgTime);
  console.log("     Minimum Time       : " + minTime);
  console.log("     Maximum Time       : " + maxTime);
  console.log("     Standard Deviation : " + stdDev);
  console.log("     Variance           : " + variance);
  console.log("--------------------------------------------------\n\n");

}

// ********************************************************************************

I noticed two things:

  1. It seems that the first call to console.time("foo") was longer than the others by a few tenths of a second. I added a dummy timing run at the start to throw away that data. Is that normal? I couldn’t find anything about it Googling, so I wasn’t sure if it was normal, or something I did, or something repl did.

  2. It seems like there is a lot of variation in run times. That is in part because of how small my test cases are in some cases, but I was wondering how ‘swingy’ in general the performance of JavaScript can be.

Thanks : )

It depends on environment I guess, but V8 is hugely optimized to the point that if you run same function more than once in the same scope it won’t give you absolute result, as it uses a lot of memoization optimizations internally. To have “unoptimized” performance you need to change scope for every run, the way it’s done in JSPerf or Benchmark.js.

Interesting. I suppose with so much between the code and the metal you really need those libraries to help isolate the performance of the function calls. I haven’t done much with JavaScript libraries so I’ll have to explore a bit. Thanks!

Do you intend to use JavaScript for some processing demanding task?
Whatever runs JS in the browsers (V8,SpiderMonkey etc) or Node.js, is not having that as a priority (single thread I/O … etc etc), but even so, in my professional life as a Dev, I have never faced any performance issue on a given task (For example I am running an intensive task with TestCafe/Puppeteer - End to End for 25 websites, and does really well - I think that supports to a certain degree some parallelism)

They claiming that Swift 5.x is crazy insanely fast. You may want to use that if that is the case.

Blockquote
Fast. Swift was built with performance in mind. Not only does its simple syntax and hand-holding help you develop faster, it also lives up to its name: as stated on apple.com, Swift is 2.6x faster than Objective-C and 8.4x faster than Python.

I don’t have a particular use case for fast JavaScript code - I work in C 90% of the time and my go-to for fast scripting languages is Python + Numpy/SciPy. I am just learning about JavaScript as something to do while my code compiles : ) My interest in making JavaScript fast is purely academic - I am curious about how much of my C optimization knowledge is valid for JavaScript.

With your knowledge of C, would you be able to create a run time environment? Or how do you intend to fiddle with performance optimizations?

This looks pretty cool https://github.com/v8/v8. Right?

While I would love to have enough time to tinker with how to make run time environments, I am aiming much smaller.

Often students on FCC that are working on the Project Euler type problems run into performance issues that trigger the infinite loop protection bounds, so I’d like to have go-to recommendations for writing relatively fast code.

As part of that, I think it helps to be able to say “here is function a, here is function b, run them and see the performance difference.”

The infinite loop protection thing is more a matter of memory allocation rather than performance, please correct me if I am wrong.
To hit this set limits you really have to write crazy stuff which you better have the warning to let you know that you really have to do better than that.

The second part with

Blockquote
“here is function a, here is function b, run them and see the performance difference.”

That is interesting.

I was under the impression that it was a question of timing out when the loop protection was triggered (or maybe both), though if you are filling up the allocation, you are going to be slow no matter what, I’d imagine.

1 Like

I just run in Chrome console window.performance.memory.jsHeapSizeLimit.
What do you think of this?

That is about 2GB of memory available.

Also @JeremyLT. This could be more accurate than what you have used previously to test?

Performance.measure()

Creates a named timestamp in the browser’s performance entry buffer between two specified marks (known as the start mark and end mark , respectively).

JavaScript is much more functional language comparing to purely procedural C, and optimizations are not always straightforward. My favorite example:

function evenHalvesWithFilterMap(arr) {
    return arr.filter((n) => n % 2 === 0).map((n) => n / 2);
}

/* VS */

function evenHalvesWithReduce(arr) {
    return arr.reduce((a, n) => n % 2 === 0 ? a.concat(n / 2) : a, []);
}

First iterates through the array twice, the second iterates once but creates new instance of accumulator on each step. What do you think will run faster if you have let’s say million entries?

This is the sort of thing that fascinates me. I would assume, based on what I know from C, that the second where new objects are being created in memory is slower. At least in that realm, my general rules of thumb are

  1. FLOPs are cheap, but don’t do FLOPs you don’t need.
  2. Memory is expensive

Yeah, I just got performance.now() to work in my script, which helps me make more meaningful statistics. I’m tweaking my script with that currently. I am hoping for something I can easily show students in repl (though they should get comfortable with dev tools eventually, as I understand it).

…I’ve just realized that for someone who used to C, this will be very straightforward as they know the price of duplicating an array :slight_smile:

Yes the second one is quadratic and will be tremendously slower, but a lot of people still think that less calls in chain - the better

2 Likes

And to demonstrate the price of accidentally quadratic .reduce():

Where does that benchmark function comes from? Is that some npm module?

No, that’s a snippet I wrote for myself and saved in console (hint: you can save snippets in browser’s console :wink: ):

function benchmark(fn, runs = 100) {
    let i = runs;
    const a = [];
    while (i > 0) {
        const x = performance.now();
        fn();
        a.push(performance.now() - x);
        i -= 1;
    }
    
    return {
        runs,
        min: Math.min(...a),
        max: Math.max(...a),
        avg: a.reduce((a, b) => a + b, 0) / runs,
    };
}
1 Like

Nice one. Thanks for sharing.

No problem! As I wrote it for myself there are few naming issues… so sorry for that :wink:

I am wondering if

performance.measure() against performance.now()

does perform differently

Edit: just found out that performance.now is used inside of measure