MLLM
is a library for simplifying communication with various Large Language Models (LLMs) and multi-modal LLMs such as OpenAI GPT, Anthropic Claude and Google Gemini.
It lets you
- Create a router to communicate with multiple models (LLMs/MLLMs)
- Configure your preference order for the models
- Communicate with the models by sending your message thread to the models’ completion endpoints
- Retry communication upon failure
- Enforce expected response formats
Installation
Basic Example
Provide the API keys for the different models (LLMs / MLLMs) that you would like to use:RoleMessage
and RoleThread
from Threadmem. So, you can also create the thread using MLLM:
Retry communication with model
The router retires communication upon failure, and you can configure the maximum number of retries allowed.Enforce response format
To enforce an expected response format, first create a class describing the response format:expect
parameter in the router.chat()
call: