SafetyCard Request a review

EvoAgentX

Python framework for building, running and self-evolving multi-agent workflows with tools, memory and prompt/workflow optimizers.

github.com/anative-lab/evoagentx · 2026-10-04 · d77fd6b

Defense-in-depth score

1.8 / 10

EvoAgentX gives agents powerful bundled tools (shell, in-process Python, arbitrary HTTP, file I/O, email) but the framework itself adds almost no safety layer: tool calls run without approval or argument validation, and model-written Python executes inside the agent process alongside its API keys. A prompt injection from any web page, search result or email can leak secrets and take irreversible actions unattended. The opt-in Docker interpreter and human-in-the-loop manager are thin and off by default.

Safest setup

EvoAgentX starts each agent with no tools and stops each agent action after 20 model calls. Beyond that the defaults are thin: tool calls run without approval, model-written Python runs inside the agent process with your keys, and nothing is recorded durably. The safest setup turns on the approval manager and Docker interpreter EvoAgentX offers and builds containment around the process.

SetupHowWhat it gives you
Scoped, low-limit keysGive the process only the API keys this workflow needs, each scoped or read-only with a spending limit set at the providerWhatever an agent does, it reaches only the systems and spend you chose, not every key in your shell C1
Workflow approvalCreate a HITLManager, call activate(), pass it as hitl_manager on the workflow and place a HITLInterceptorAgent before each consequential stepA person approves or rejects each wrapped agent action before it runs C2
Tools per agentKeep the default empty tools on each CustomizeAgent and add only the toolkits that agent needsEach agent can reach only the shell, web, file or email tools its task calls for C3
A Docker code interpreterRun model-written Python through DockerInterpreterToolkit with require_confirm=True, in place of the in-process interpreterThat code runs in a container that holds none of your environment variables, and you confirm each run C4
A step budgetKeep the default max_steps=20 on each CustomizeAgent, or lower it for short tasksEvery agent action ends after a fixed number of model calls C10
A clean working directoryStart each run from a directory holding only the task's files, with no .env file you didn't writeAPI keys and model endpoints come only from you, not from a checkout you happen to run in C6
Its own container or VMRun the workflow in a container or VM whose network reaches only the model provider and the hosts the task needsAn outer boundary around shell and in-process tools, and a narrow path for data to leave C5
A run log on diskCall save_logger(<path>) with a path the agent's tools can't write to, such as one outside its containerA record of every tool call with its arguments and result that outlives the process C9

Approving with confidence

  • Read each prompt and approve actions one at a time; a workflow approval covers every tool call inside that agent action.
  • The shell toolkit asks before each command and shows it in full, so read the command before approving it.

Handling data

  • Tool arguments, tool results and raw model output print to stdout at INFO by default; treat console output and any saved log like the data the agent read.

Key gaps (4)

  1. Tools run with the developer's ambient OS and API-key authority; a hijack reaches the whole machine and connected accounts. C1 · Identity & least privilege
  2. Default Python execution is in-process exec() with full builtins, sharing the process that holds API keys. C4 · Code-execution isolation
  3. Untrusted tool output enters context as user messages while unguarded egress and state-changing tools are available: leak plus irreversible action unattended. C5 · Untrusted input blast radius
  4. load_dotenv() at import reads .env from the working directory, so a workspace file can set credentials and model endpoints. C6 · Memory, context & configuration integrity

Criteria

C1 Identity & least privilege

0.00 / 1.00

EvoAgentX has no identity or authorization layer of its own. Each bundled toolkit builds its own client from environment variables (Gmail, Telegram, databases, search APIs), and the model wrapper copies provider API keys into the process environment, which the shell tool and the in-process Python executor then inherit in full. Every tool runs with whatever the operating-system user and those keys can do. A hijacked agent therefore has the developer's whole account reach.

C2 Approval gates

0.15 / 1.00

The framework's tool executor runs every tool call the model makes with no approval step. A human-in-the-loop manager exists, but it is off unless a developer creates and activates it (otherwise it auto-approves), it approves whole workflow actions rather than individual tool calls, and its tool-call review mode is unimplemented. The bundled shell toolkit does prompt per command and shows the exact command, but Python execution, file writes, HTTP requests and email sending have no gate.

C3 Tool & action scoping

0.13 / 1.00

Tool arguments are not validated at run time: the base Tool class only checks schema declarations when a tool class is defined, and the executor passes model arguments straight into the tool. Bundled tools are general-purpose: a raw shell string, arbitrary Python, any URL with any HTTP method, and file reads/writes on any absolute path. The storage helper's base-directory containment is not a complete boundary. A dynamic toolkit even lets the model create new code-backed tools at run time.

C4 Code-execution isolation

0.33 / 1.00

Model-written Python runs by default in the agent's own process through exec() with full builtins; the import allowlist is skipped unless the developer supplies one, and the code's own docstring warns it is unsafe for untrusted code. Shell commands run on the host as the user. Other host-side execution paths exist, including workflow operator exec(), and not every model-influenced execution path is confined. An opt-in Docker interpreter exists, but it starts a stock container with no resource limits, no network restriction and no execution timeout.

C5 Untrusted input blast radius

0.00 / 1.00

Nothing in the framework limits what a hijacked agent can do after reading untrusted content. Search, browser, crawler, RSS, arXiv and Gmail tools pull external text into context, and in the default prompt mode tool results are appended as user-role messages with the same standing as the developer's instructions. The same agent can hold egress and state-changing tools (arbitrary HTTP, email sending, shell, Python) with no approval in the way. A successful prompt injection can leak keys or data and take irreversible actions unattended.

C6 Memory, context & configuration integrity

0.10 / 1.00

Importing the tools package calls load_dotenv() in several modules, which reads a .env file from the current working directory and can silently set API keys and endpoints such as OPENAI_API_BASE. If the agent runs inside an untrusted checkout, that file can redirect model traffic or swap credentials. The opt-in long-term memory agent stores conversation messages, including model outputs, and pastes retrieved memories back into the prompt as context with no validation or provenance.

C7 Third-party extensions

0.23 / 1.00

Third-party extensions come in through MCP: the developer supplies a server config and the framework launches or connects to it via FastMCP with no version pinning, hash check or change detection. Tool descriptions from servers are passed to the model verbatim. Stdio servers run as the same OS user. Separately, importing the RAG embeddings module tries to pip-install the ollama package, unpinned and without a prompt, if it is missing.

C8 Secrets & sensitive-data protection

0.15 / 1.00

Provider and tool API keys come from environment variables or plain config fields, and the LiteLLM wrapper copies them into os.environ, where every shell command and in-process code execution can read them. There is no redaction anywhere: tool arguments, tool results and raw model responses are logged at INFO to stdout by default. The one protective detail is that saved agent files exclude the LLM config. No telemetry was found.

C9 Audit & traceability

0.33 / 1.00

Every tool call that goes through the main agent loop, including MCP tools, is logged with its name, arguments and result through loguru to stdout. Logs are unstructured text, carry no actor or approver attribution, and are only written to a file if the developer calls save_logger. Workflow execution keeps an in-memory trajectory of messages, but nothing is durable or tamper-evident by default.

C10 Limits & kill switch

0.40 / 1.00

Each agent action stops after 20 model calls by default and each workflow task after 5 agent executions, and the shell and MCP tools have 30-second timeouts. There is no cost or wall-clock budget (cost is tracked but never enforced), the model chooses the shell timeout itself, the Python and Docker executors have no timeout, and the workflow scheduler loop has no overall cap. Timeouts stop waiting but leave threads or MCP calls running.