Hi all,
Love your work on RTS. We love the benchmarking result of RTS with 1TB of raw financial tick data.
We encounter following error after RTS server restart. I wonder if RTS supports RDB or AOF persistent? Thanks.
776:M 20 Feb 2020 15:46:09.092 # Error loading data from RDB (short read or EOF). Read performed by module 'timeseries' about type 'TSDB-TYPE' after reading '0' bytes of a value.
Andy
It seems like an issue with the RDB deserialization, which version of RTS are you using?
Can you attach the RDB?
redis_version:5.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:79857a93f73253c0
redis_mode:standalone
os:Linux 4.14.146-120.181.amzn2.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:7.3.1
process_id:1282
run_id:a4f4e66633cdd443d1190aabcb591fab44bd6ec2
tcp_port:6379
uptime_in_seconds:6602
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:5166438
executable:/usr/local/bin/redis-server
config_file:/etc/redis/redis.conf
connected_clients:32
client_recent_max_input_buffer:2
client_recent_max_output_buffer:0
blocked_clients:0
used_memory:2092608328
used_memory_human:1.95G
used_memory_rss:2169384960
used_memory_rss_human:2.02G
used_memory_peak:7891646504
used_memory_peak_human:7.35G
used_memory_peak_perc:26.52%
used_memory_overhead:1431602
used_memory_startup:791416
used_memory_dataset:2091176726
used_memory_dataset_perc:99.97%
allocator_allocated:2092676304
allocator_active:2093699072
allocator_resident:2179067904
total_system_memory:33069961216
total_system_memory_human:30.80G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.00
allocator_frag_bytes:1022768
allocator_rss_ratio:1.04
allocator_rss_bytes:85368832
rss_overhead_ratio:1.00
rss_overhead_bytes:-9682944
mem_fragmentation_ratio:1.04
mem_fragmentation_bytes:76839520
mem_not_counted_for_evict:2798
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:574276
mem_aof_buffer:2798
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0
loading:0
rdb_changes_since_last_save:6048
rdb_bgsave_in_progress:0
rdb_last_save_time:1582224653
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:171
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:1273856
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:175
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:270184448
aof_current_size:15038791875
aof_base_size:14387324147
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:20
total_connections_received:847
total_commands_processed:2260126
instantaneous_ops_per_sec:0
total_net_input_bytes:36730287027
total_net_output_bytes:9320493313
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
evicted_keys:0
keyspace_hits:115969
keyspace_misses:1343
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:29872
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
role:master
connected_slaves:0
master_replid:76a0ba0dfe36f57995859870243d52bb8731dc8b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
used_cpu_sys:99.035822
used_cpu_user:717.799139
used_cpu_sys_children:166.188894
used_cpu_user_children:2618.338202
cluster_enabled:0
db0:keys=1373,expires=0,avg_ttl=0
Hey Guy,
Please see version below. Redis failed with original error message when we try to attach the RDB file.
redis_version:5.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:79857a93f73253c0
redis_mode:standalone
os:Linux 4.14.146-120.181.amzn2.x86_64 x86_64
Hey I’ve attached the redis.log and here’s the appendonly.aof:
Let me know if you can’t access it.
Thanks!
redis.log (32.5 KB)