Hi, I have been going over the docs for RS2 and have the following questions:
What is the syntax for PREFIX? Can I specify some pattern or is a wildcard always implied?
A common redis key pattern is movies:123, but also movies:123:trailer movies:123:comments etc… where additional data is stored under the same namespace as the “parent” object.
Is it possible to only index movies:\d+ or movies:[^:] - or is it just “movies:*”?
If not, can one use FILTER to achieve a similar result (by restricting the keyspace indexed based on some data in the hash) - and if yes, what is the performance penalty of that?
RS 2 migration
What happens to existing indexes if one migrates to the RS 2 code? Docs say that if PREFIX is omitted, RS2 will index all hashes. Will that happen to our existing indexes if we migrate - i.e. suddenly RS will add every hash saved in the system into all existing RS indexes?
The syntax for prefix is, well, prefix. Behind the scene, we use trie to make it very efficient. This is why we split it to PREFIX and FILTER and not put it all under FILTER.
You can do more complicated rules with FILTER and check for specific values inside the hash (you can also check multiple values and decide the interaction between them like OR/AND) but it will come with a performance penalty as we need to calculate the filter for each hash change and we need to read the hash from the Redis (even if we are not going to index it).
Hope it answers your question, let me know if anything else is not clear.
Thank you, yes it does. I guess you are suggesting to try the filter if I have keys that I want to exclude, but be prepared for an unknown performance hit.
What about my second question - what happens with existing redis indexes in the rdb file from RS1.
Regarding first question, I would also suggest to create separate (logically) indexes based filter criteria. that will further simplify the queries and overall performance. and it depends use cases to use cases.
on 2nd question. I dont that impact your existing implementation with prefix queries. using RS for secondary search means RS follows the data written in hashes and synchronously indexes it. so Hashes keys and RS index will co-exists.
@michaelmasouras sorry missed the second question, as the docs say you must find a way to tell RS2 which hashes to index in which index, it can either be done using PREFIX or FILTER (or combination of both) but if you do not specify either PREFIX or FILTER then yes all the hashes will be indexed.
If you really do not have a way to tell RediSearch which hashes to index in which index then, before upgrading, you can extract for each index all the documents that indexed by the index and send a simple Lua that will add a field with the index name. Then using a filter you can tell RS2 which documents to index in for each index.
Sorry for being a bit slow. I am reading these docs but I can’t find any information on migration from RS1:
I have an existing index made with RS1. I shut down the server, build and install RS2 and start the server:
a. If I don’t do anything, will RS now index every single hash set (hset etc…) in the existing index.
b. What do I have to do to prevent it from doing so for the existing index. Do I use the FT.CREATE command using the same index name?