We are faced with the task of optimizing the calculations for detecting people on the GPU and we want to apply the BATCSIZE and MINBATCHSIZE parameters for models in RedisAI. But when adding these parameters, we do not get the expected system behavior.
We use a bundle of Redis - RedisGears - Redis AI
Redis is used to store stream and key-value structures.
One client sends the image from the camera in key-value, and sends the key and another metadata to stream ‘detection: process’ as a task for detection.
gb = GearsBuilder('StreamReader') gb.map(process) gb.register('detection:process')
From the message to stream, we get the key and the image from the key-value, and then start the model prediction.
Uploading a frozen graph model to the Tensorflow backend in RedisAI.
cat model.pb | AI.MODELSET detector:model TF GPU BATCHSIZE 32 MINBATCHSIZE 8 INPUTS image_arrays OUTPUTS detections BLOB
Running model prediction inside RedisGears.
import redisAI def run_model(image_tensor): model_runner = redisAI.createModelRunner('detector:model') redisAI.modelRunnerAddInput(model_runner, 'image_arrays', image_tensor) [redisAI.modelRunnerAddOutput(model_runner, output) for output in ['detections']] output = redisAI.modelRunnerRun(model_runner) return output
When adding the BATCHSIZE and MINBATCHSIZE parameters, we expect that steam tasks will stop at the execution of MODELRUN, batch up to MINBATCHSIZE and perform calculations on a GPU with several images, but the system does not wait for the accumulation of tasks, but continues to recognize one image at a time.
How can we test this case with our problem and how should the system work correctly with the BATCHSIZE and MINBATCHSIZE parameters?
Do you have examples using these parameters?