#FF2 - Implement the agent loop - Imprecise instructions

(posting this in general because it seems that the forum doesn’t currently recognize the challenge:claude-code tag. will report that separately)

The instructions for #FF2 are imprecise.

The previous stage “Execute the read tool” asked you to print the contents of the file but now that we move to the agent loop, you are not supposed to print the file and instead rely on the LLM responding with the printable contents. This isn’t explained.

Additionally, its not clearly explained that you aren’t supposed to print the message content if a tool execution is present. Side note: this is unintuitive to me and I’d pretty much always want to print this in some form so the user gets feedback on what actions the LLM will be taking but that’s somewhat unrelated. As an example, for the prompt Find the chemical expiry period in months from README.md. Respond with only a number., once the read tool sends the contents of the README.md back to the LLM, i get back a response with both a message: Let me check the chemical.py file: as well as a tool execution to read the chemical.py file however the test runner will fail if you print this message.

It would be helpful if it was better specified what should be printed.

4 Likes

Thank you for saying this. I just wasted quite a bit of time on that, until I actually logged the messages and saw that the extraneous text it was mad about came from a message with tool calls.

2 Likes

Yep we’re thinking about how to handle this! Whether to relax tester requirements or to amend instructions.

Our tests run against claude code and consistently pass, so we’re leaning towards the latter.

1 Like

We’ve updated the instructions in this PR to clarify the expected behavior.

Let us know if anything else could be clearer or improved. @matthew-russo @gamebox @bedoyama @hulyhulyhuly

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.