ToolFuse is a library for building tools for LLM-based agents. It provides a simple interface for defining tools and their actions and observations.

​
Motivation

AI agent developers know that one of the biggest challenges in building agents is defining the tools they can use. Tools do specific tasks and interact with their environment. They can fetch data, navigate a UI, summarize text, review an image, type, click buttons, talk to an API, write to files, etc.

The challenge breaks down into multiple smaller problems:

  • How to let the model know what tools it can use?
  • How does the LLM know what the tool can do?
  • How do we enforce the rules around the actions?
  • How to wield that tool based on the model’s reasoning and response?

ToolFuse focuses on all of these problems and makes it easier that ever to use tools.

​
Installation

pip install toolfuse

​
ToolFuse in action

Let’s take a look at a super simple example. Let’s say we have a class Desktop that lets your agent interact with that desktop and get information from the environment like the mouse postion, as well as take actions like moving the mouse. The class gets defined as follows:

class Desktop(Tool):
    @observation
    def get_mouse_position(self):
        """Get mouse position on a screen"""
        # Implement the logic to get the current mouse position
        return x, y

    @action
    def move_mouse(self, x: int, y: int):
        """Move mouse to the given coordinates"""
        # Implement the logic to move the mouse to the given coordinates
        pass
    # Implement other actions and observations as needed

When we create an instance of the Desktop class, we can use it with an agent, we can request a schema of the tool actions and observations:

tool = Desktop()
schema = tool.json_schema()

[{'name': 'get_mouse_position',
  'parameters': {'type': 'object', 
   'properties': {}, 
   'required': []},
  'description': 'Get mouse position on a screen.'},
 {'name': 'move_mouse',
  'parameters': {'type': 'object',
   'properties': {'x': {'type': 'integer'}, 'y': {'type': 'integer'}},
   'required': ['x', 'y']},
  'description': 'Move mouse to the given coordinates.'}]

Now, if we get a JSON that corresponds to this schema, we can feed it to a tool and ask it to perform an action.

jdict = {"tool": "move_mouse", "parameters": {"x": 100, "y": 100}}
action = tool.find_action(jdict['tool'])
print(f"Found action: {action.name}")
result = tool.use(action, **jdict['parameters'])

​
What’s next?

  • Look at the detailed example of using a Tool with an LLM at Weather Agent tutorial.