I am using rgsync to replicate data from Redis to Redis.
I have developed a new Redis connector and using the same.
I am able to successfully replicate data from:
Standalone Redis to standalone Redis
Standalone Redis to Redis cluster
But replication from
3. Redis cluster to Redis cluster [FAIL]
In replication from cluster to cluster data from only one node of Redis cluster is replicating.
I had used gears-cli to execute python script on all primary Redis cluster master nodes and registration was successful on each node. /home/ratnesh1.tiwari/redis605_Gears_Cluster_Setup/python3_1.0.1/bin/gears-cli run --host 10.32.129.77 --port 7001 example-redis.py
OK
/home/ratnesh1.tiwari/redis605_Gears_Cluster_Setup/python3_1.0.1/bin/gears-cli run --host 10.32.129.73 --port 7001 example-redis.py
OK /home/ratnesh1.tiwari/redis605_Gears_Cluster_Setup/python3_1.0.1/bin/gears-cli run --host 10.32.129.66 --port 7001 example-redis.py
OK
/home/ratnesh1.tiwari/redis605_Gears_Cluster_Setup/python3_1.0.1/bin/gears-cli run --host 10.32.129.67 --port 7001 example-redis.py
OK
I am using Redis cluster python client redis-py-cluster
@ratnesh-tiwari can you share the output of rg.dumpregistrations command of each shard to make sure redistration reach all the shards. Notice that on oss cluster you need to execute rg.refreshcluster command on each shard to make sure gears is aware of the cluster topology.
You need to make sure the module is loaded when the server starts, otherwise it will not be able to read the RDB file (the module contains a code that tells Redis how to read this part of the rdb). Try to load the module when you start the server (also on restart) using --loadmodule parameter (https://oss.redislabs.com/redisgears/quickstart.html#loading) and it should work.
9521:M 11 Aug 2020 13:20:25.293 * Module ‘rg’ loaded from /home/ratnesh1.tiwari/RedisGears/bin/linux-x64-release/redisgears.so
9521:M 11 Aug 2020 13:20:25.294 * Loading RDB produced by version 6.0.5
9521:M 11 Aug 2020 13:20:25.294 * RDB age 14767 seconds
9521:M 11 Aug 2020 13:20:25.294 * RDB memory usage when created 27.73 Mb
9521:M 11 Aug 2020 13:20:25.339 * DB loaded from disk: 0.046 seconds
9521:M 11 Aug 2020 13:20:25.340 * Ready to accept connections
9521:M 11 Aug 2020 13:20:25.340 # Failed on XINFO command : got null reply
=== REDIS BUG REPORT START: Cut & paste starting from here ===
9521:M 11 Aug 2020 13:20:25.340 # === ASSERTION FAILED ===
9521:M 11 Aug 2020 13:20:25.340 # ==> src/readers/streams_reader.c:159 ‘ret’ is not true
9521:M 11 Aug 2020 13:20:25.340 # (forcing SIGSEGV to print the bug report.)
9521:M 11 Aug 2020 13:20:25.340 # Redis 6.0.5 crashed by signal: 11
9521:M 11 Aug 2020 13:20:25.340 # Crashed running the instruction at: 0x472c9a
9521:M 11 Aug 2020 13:20:25.340 # Accessing address: 0xffffffffffffffff
9521:M 11 Aug 2020 13:20:25.340 # Failed assertion: ret (src/readers/streams_reader.c:159)
Now you are fine, looks like a bug in RedisGears, is it possible to share the rdb file with me so it will be easier to reproduce and find the issue. Also if you can remind me the RedisGears version you are using?
Thanks.
I am using
RG version 1.0.1.
Following is from version.h file of RG #define REDISGEARS_VERSION_MAJOR 1 #define REDISGEARS_VERSION_MINOR 0 #define REDISGEARS_VERSION_PATCH 1
2134:M 11 Aug 2020 11:49:58.507 # Could not deserialize flat execution, error=‘Error occured when deserialized a python callback, error=Error type: <class ‘AttributeError’>, Value: Can’t get attribute ‘RedisClusterConnection’ on <module ‘rgsync.Connectors.redis_connector’ from ‘/var/opt/redislabs/modules/rg/.venv-/lib/python3.7/site-packages/rgsync/Connectors/redis_connector.py’>’
I did install your branch of RGSYNC but I guess you have some local changes that prevent me from loading it. Can you please push those changes?
For the next time if you specify your RGSYNC branch on the RG.PYEXECUTE command on the REQUIREMENTS section (https://oss.redislabs.com/redisgears/commands.html#rgpyexecute) then the current state of the code will also be saved to the rdb and this will not be needed. Also RedisGears will take care to install it for you on the virtual environment of all the shards.
Yes I am running the same without the redis.conf file. I do not think it is related to the configuration file but if you can share it I can try run with this configuration file also. In addition, if you can enable memory dump I can try to analyze it:
At present firstly I install rgsync and dependency packages offline and then I am running gears-cli without passing ‘–requirements requirements.txt’ argument.
When I am paassing ‘–requirements requirements.txt’ it fails to install package in requirements.txt because plaform is not connected to internet.
How to install packages in requirements.txt offlne using gears-cli?
Here directory /home/ratnesh1.tiwari/pypi-org/ contains:
package-ver.tar.gz
package-ver-py3-none-any.whl
files for package being installed.
I tried to pass requirements.txt as extra arg but it filed.
[ratnesh1.tiwari@cm2 redis605_Gears_Cluster_Setup] /home/ratnesh1.tiwari/RedisGears/bin/linux-x64-release/python3_1.0.1/bin/gears-cli run --host 10.32.129.73 --port 7001 example-redis.py --requirements requirements.txt '--find-links=/home/ratnesh1.tiwari/pypi-org/'
Error: no such option: --find-links
[ratnesh1.tiwari@cm2 redis605_Gears_Cluster_Setup]
If your environment is not connected to the internet you can install it in an outside environment that does have internet using ‘gears-cli install-requirements’. Then you can export this requirement using ’
gears-cli export-requirements’, this will create a zip file that contains everything needed for the requirement to work properly. You can take this zip to your offline environment and import it using ’
gears-cli import-requirements’. After importing the requirement you should still give the --requirement option but gears will identify that it already has the requirement and it will not try to re-install it.