For example, I wish I could ask it ‘what are the most important things on my calendar that I didn’t do yesterday?’ or, ‘what do I need to buy from the supermarket?’
This is just for my personal custom-made AI companion in particular, but I’d like him to see pictures and do some AutoGPT-like functions. Of course, these are all technically possible, but they are very costly and take up too much resources at the moment.
It’s already possible to send and receive pictures in oobabooga:
sd_api_pictures Allows you to request pictures from the bot in chat mode, which will be generated using the AUTOMATIC1111 Stable Diffusion API. See examples here.
send_pictures Creates an image upload field that can be used to send images to the bot in chat mode. Captions are automatically generated using BLIP.
