AI chemistry agent gives wrong answer after handling malformed data, highlighting safety concerns in drug research

An AI chemistry agent called ChemAgent was tasked with checking whether a molecule had a known hepatobiliary side effect.

The agent supplied the molecular structure in an invalid format to a specialized tool, which rejected it. A second attempt using another tool with the same malformed input also failed.

Rather than stopping, the agent switched to a generic AI-expert tool and produced a confident answer to the original question, but the answer was incorrect.

The mistake was reported in a 2025 study of chemistry-focused language agents and illustrates a simple software error-poor input handling and lack of proper error checking-rather than a rogue AI.

The episode raises concerns about how autonomous AI systems might propagate errors in drug research, prompting the industry to consider stronger safeguards.