# Confusing test config on #WZ8

**URL:** <https://forum.codecrafters.io/t/confusing-test-config-on-wz8/2519>\
**Category:** Challenges\
**Tags:** challenge:interpreter\
**Created:** [October 22, 2024, 12:39pm UTC](https://forum.codecrafters.io/t/confusing-test-config-on-wz8/2519 "2024-10-22T12:39:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Convl](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/convl/32/2509_2.png) [@Convl](https://forum.codecrafters.io/u/Convl)\
**Post date:** [October 22, 2024, 12:39pm UTC](https://forum.codecrafters.io/t/confusing-test-config-on-wz8/2519/1 "2024-10-22T12:39:06Z")

</div>

The tester at this stage does not check whether you correctly report syntax errors, just whether you exit with code 65. That, in itself, would be fine, if a little bit confusing (since you will already have implemented error reporting by the time you get to this stage if you are following the instructions in the book). The problem, though, is that your programm will actually fail the tests if it does report errors, even if it exits with code 65 afterwards. More confusingly still, it seems to me like the tester will, in fact, tolerate your app reporting errors before exiting, but only if they are in exactly the format that the tester expects. If your error reporting deviates from that even slightly, you will fail the tests, and the message you will get about this won’t be very helpful, either.

For instance, the first test is this:

```auto
remote: [tester::#WZ8] [test-1] [test.lox] "hello
remote: [tester::#WZ8] [test-1] $ ./your_program.sh parse test.lox

```

To which my program responded with this (and then exited with code 65):

```auto
remote: [your_program] [line 1] Error: Unterminated string.
remote: [your_program] [line 1] Error at '': Expect expression.

```

If I had only reported the first line and exited with 65, that, too, would have been fine with the tester. Alternatively, if I do not report anything and just exit with 65, that is also fine. But reporting something that does not line up to the testers expectations of what an error report should look like yields this rather confusing message:

```auto
remote: [tester::#WZ8] [test-1] 𐄂 [line 1] Error at '': Expect expression.
remote: [tester::#WZ8] [test-1] Expected line #1 on stdout to be "", got "[line 1] Error at '': Expect expression."

```

---

<div class="post-metadata">

**Author:** ![rohitpaulk](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/rohitpaulk/32/6_2.png) [@rohitpaulk](https://forum.codecrafters.io/u/rohitpaulk)\
**Post date:** [October 23, 2024, 3:07pm UTC](https://forum.codecrafters.io/t/confusing-test-config-on-wz8/2519/2 "2024-10-23T15:07:25Z")

</div>

@Convl you should be able to print errors to stderr instead of stdout (I think this is what the book does too?). The tester only validates stdout.

---

<div class="post-metadata">

**Author:** ![Convl](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/convl/32/2509_2.png) [@Convl](https://forum.codecrafters.io/u/Convl)\
**Post date:** [October 24, 2024, 5:21pm UTC](https://forum.codecrafters.io/t/confusing-test-config-on-wz8/2519/3 "2024-10-24T17:21:46Z")

</div>

I can’t really go back and check now because I have added tons of code since this issue came up, but now that you say it, I think that very well may have been the reason for the problem. Thanks @rohitpaulk !

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/3X/7/0/700657133935c15703e22c7c8870394f6e6dc27d.svg) [@system](https://forum.codecrafters.io/u/system)\
**Post date:** [October 29, 2024, 5:22pm UTC](https://forum.codecrafters.io/t/confusing-test-config-on-wz8/2519/4 "2024-10-29T17:22:21Z")

</div>

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.