Confusing test config on #WZ8

The tester at this stage does not check whether you correctly report syntax errors, just whether you exit with code 65. That, in itself, would be fine, if a little bit confusing (since you will already have implemented error reporting by the time you get to this stage if you are following the instructions in the book). The problem, though, is that your programm will actually fail the tests if it does report errors, even if it exits with code 65 afterwards. More confusingly still, it seems to me like the tester will, in fact, tolerate your app reporting errors before exiting, but only if they are in exactly the format that the tester expects. If your error reporting deviates from that even slightly, you will fail the tests, and the message you will get about this won’t be very helpful, either.

For instance, the first test is this:

remote: [tester::#WZ8] [test-1] [test.lox] "hello
remote: [tester::#WZ8] [test-1] $ ./your_program.sh parse test.lox

To which my program responded with this (and then exited with code 65):

remote: [your_program] [line 1] Error: Unterminated string.
remote: [your_program] [line 1] Error at '': Expect expression.

If I had only reported the first line and exited with 65, that, too, would have been fine with the tester. Alternatively, if I do not report anything and just exit with 65, that is also fine. But reporting something that does not line up to the testers expectations of what an error report should look like yields this rather confusing message:

remote: [tester::#WZ8] [test-1] 𐄂 [line 1] Error at '': Expect expression.
remote: [tester::#WZ8] [test-1] Expected line #1 on stdout to be "", got "[line 1] Error at '': Expect expression."

@Convl you should be able to print errors to stderr instead of stdout (I think this is what the book does too?). The tester only validates stdout.

I can’t really go back and check now because I have added tons of code since this issue came up, but now that you say it, I think that very well may have been the reason for the problem. Thanks @rohitpaulk !

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.