NodeJS Client Library Poor Perforamance

For background context on the type of data I am dealing with, I have a time series data for edge computing devices.

I tried to benchmark between the use of sorted sets and Redis Time Series, which is done by timing the simple operations of writing/reading about 100,000 requests. The results I obtained was the following:

  • Sorted sets: 2.016176666 seconds for 100,000 writes and 2.169808666 seconds for 100,000 reads
  • RTS: 5.895935991 seconds for 100,000 writes and 7.637590636 seconds for 100,000 reads

As you can see, the results for time series is extremely slow compared to sorted sets, which I don’t think is what I should be expecting.

Is there important things I should be looking for that might have caused such low performance? I am leaning towards that the client library is a bottleneck but I could definitely be wrong…

I used the following client library for NodeJS Redis
NodeRedis
redistimeseries-js

Any insight would be really appreciated. Thank you!

Hi, here’s a neat trick to take the client out of the equation: compare bulk loads!
Here’s how to do it: https://redis.io/topics/mass-insert

If you prepare two files, one containing ZADDs, and one containing TS.ADDs, you can then time how long it takes to redis-cli to bulk load everything. It’s not precisely a scientific benchmark, but it’s a starting point.

If the results don’t match your times (2s vs 5s), then it’s definitely time to investigate the clients.

1 Like

Hi there @Kido,
Regarding the performance of the two different ways of modelling your data ( using sorted sets or RedisTimeSeries ), and whether you should use sorted sets or RedisTimeSeries, it will always be bound to the complexity of your case, to how long you plan to store your data, and how you plan to query it. So to answer to wether Sorted Sets are faster or slower than RedisTimeseries, is it depends.
It’s will always be a trade off between ingestion (writes), query performance(reads) and flexibility (what you can do).
The RedisTimeSeries GA Blog post covers in depth both options ( and adds a few more ).

If you’re comparing RedisTimeSeries out of the box vs Sorted Sets, there are things you should take into account:

  • WRITES: On the example above, you’re comparing uncompressed data ( sorted sets ) vs compressed one ( timeseries ). This is extremely important if you plan to keep large datasets. You should see a tremendous difference in the memory usage right away. Furthermore, insertion on sorted sets is O(log(N)), so it takes longer to insert, the larger the set becomes. Insertion on RedisTimeSeries is constant indepent on the number of datapoints you have on a series ( we’ve benchmark 800 Million datapoints with constant ingestion in following blog of RedisTimeseries 1.2 )
  • READS: I believe that on the example above you’re querying in the simples manner possible via TS.GET correct? ( which is completly valid depending on the use case ). Notwistanding, if you plan to do aggregations over time (TS.RANGE), a common requirement on TSBDs use cases, you will need to either switch to a solution with the proper toolset for that ( example RedisTimeseries ) or do client side work. On reads is where you will notice a improvement on performance if you switch to RedisTimeSeries ( but again it depends on the use case ).

So, my educated gess is that there is no poor performance on the nodejs client, or either solutions. The answer to chose either sorted sets of redistimeseries will be strictly bound to your usecase, but on either ways you should be able to ingest in the order of millions of datapoints per second.

IMHO performance per-se should not be the ultimate decision factor but the proper toolset for the problem depending on how complex it is ( we should be able to get you to hit your performance requirements as soon as you’ve weighted other factors =) ).

1 Like