Overview of Step-by-Step Guide
-
Define Your Tool:
- Create a class that inherits from Tool.
- Use the
@action
decorator to define actions that the tool can perform.
-
Initialize the Tool and Thread:
- Instantiate your tool and a RoleThread for message management.
-
Obtain the Tool Schema:
- Retrieve the JSON schema of the tool to inform the LLM about available actions.
-
Craft the System Prompt:
- Create a system prompt that includes:
- The role of the assistant.
- The tool schema.
- Instructions on how to format responses.
- Create a system prompt that includes:
-
Handle User Inputs:
- Define a method to process user questions.
-
Implement Request Processing with Retries:
- Use the
@retry
decorator to handle potential exceptions and retry the request. - Post the user’s question to the thread.
- Get the LLM’s response.
- Parse the response into the expected format.
- Execute the action using the tool.
- Use the
-
Parse and Execute LLM Responses:
- Convert the LLM’s output into a
V1ActionSelection
object. - Find the corresponding action in the tool.
- Execute the action with the provided parameters.
- Convert the LLM’s output into a
Prerequisites
- Python installed on your system.
- Necessary Python packages installed:
- toolfuse
- rich
- threadmem
- litellm
- skillpacks
- tenacity
- dotenv
Setup
- Import required modules: Essential libraries for tool definition, threading, LLM completion, and logging.
- Logging setup: Configures logging to display information-level logs.
- Environment variables: Loads environment variables from a .env file for configurations.
Defining the Tool
This is a straightforward implementation of theTool
class, designed to perform one primary operation: adding two numbers.
Calculator Tool
- Inherits from
Tool
class - Defines an
add
action using the@action
decorator
add
Method
- Accepts two integers as parameters
- Returns their sum
Integrating with the LLM
The interesting part is not really the ability to perform mathematical calculations, but to use this tool to give an LLM a new skill. All we have to do is get the schema of the tool actions and observations and give them to our LLM.Thinker Class
- Manages the interaction between the LLM and the tool
- Initializes the Calculator tool and a RoleThread for message threading
- Retrieves the JSON schema of the tool, which outlines its available actions and parameters.
- Crafts a system prompt that instructs the LLM on how to utilize the tool and format responses
ask
Method
ask
method will cover handling user queries. It performs the following functions:
- Accepts a user question
- Copies the current thread state
- Processes the request
Processing Requests with Retries
- @retry Decorator: Retries the request up to 5 times if exceptions occur (e.g., incorrect response formats).
- Posting the Question: Adds the user’s question to the thread.
- LLM Completion: Calls the LLM to get a response using the current thread messages.
- Parsing the Response: Expects the LLM to return a JSON-formatted action selection, which is parsed into a V1ActionSelection object.
- Executing the Action: Finds the corresponding action in the tool and executes it with the provided parameters.
- Result: Returns the updated thread and the result of the action.
Running the Script
- Instantiate the
Thinker
class. - Call the
ask
method with questions. The tool can handle both numerical and word-based numbers. - Print or handle the results obtained from the LLM-assisted tool execution.
- Extend LLM capabilities with custom tools
- Perform specific actions beyond text generation
- Utilize external functionalities seamlessly
- Enhance overall AI system performance
What’s next?
- Play around with this example at Jupyter notebook.
- Look at more realistic example of a SeleniumBrowser tool implementation.
- Dive into an elaborated implementation of the Desktop tool from the AgentDesk project, which adds the
Tool
sause on top of the Agentd-powered VMs.