I have now seen it has something to do whether $PATH is search backwards or forwards. There are two places (type and the CMD branch) where the path is searched, and any choice (forwards/backwards) produces a different error, so 4 different tests fail; this is definitely bizzare:
a) both forward fails
remote: [tester::#MG5] Running tests for Stage #MG5 (The type builtin: executable files)
remote: [tester::#MG5] [setup] export PATH=/tmp/qux:$PATH
remote: [tester::#MG5] [setup] export PATH=/tmp/quz:$PATH
remote: [tester::#MG5] [setup] export PATH=/tmp/foo:$PATH
remote: [tester::#MG5] [setup] PATH is now: /tmp/foo:/tmp/quz:/tmp/qux:/vcpkg:...
remote: [tester::#MG5] [setup] Available executables:
remote: [tester::#MG5] [setup] - my_exe
remote: [tester::#MG5] Running ./your_program.sh
remote: [your-program] $ type cat
remote: [your-program] cat is /usr/bin/cat
remote: [tester::#MG5] ✓ Received expected response
remote: [your-program] $ type cp
remote: [your-program] cp is /usr/bin/cp
remote: [tester::#MG5] ✓ Received expected response
remote: [your-program] $ type mkdir
remote: [your-program] mkdir is /usr/bin/mkdir
remote: [tester::#MG5] ✓ Received expected response
remote: [your-program] $ type my_exe
remote: [your-program] my_exe is /tmp/foo/my_exe
remote: [tester::#MG5] ^ Line does not match expected value.
remote: [tester::#MG5] Expected: "my_exe is /tmp/quz/my_exe"
remote: [tester::#MG5] Received: "my_exe is /tmp/foo/my_exe"
remote: [your-program] $
remote: [tester::#MG5] Assertion failed.
remote: [tester::#MG5] Test failed
b) both backwards fails (this was my original case - and i think the wanted behavior: search the last-added folder to PATH first):
remote: [tester::#UN3] Running tests for Stage #UN3 (Redirection - Append stderr)
remote: [tester::#UN3] [setup] export PATH=/tmp/strawberry/strawberry/pineapple:$PATH
remote: [tester::#UN3] Running ./your_program.sh
remote: [your-program] $ ls -1 nonexistent >> /tmp/foo/bar.md
remote: [your-program] ls: cannot access 'nonexistent': No such file or directory
remote: [tester::#UN3] ^ Line does not match expected value.
remote: [tester::#UN3] Expected: "ls: nonexistent: No such file or directory"
remote: [tester::#UN3] Received: "ls: cannot access 'nonexistent': No such file or directory"
remote: [your-program] $
remote: [tester::#UN3] Assertion failed.
remote: [tester::#UN3] Test failed
c) type forward, cmd backwards:
remote: [tester::#UN3] Running tests for Stage #UN3 (Redirection - Append stderr)
remote: [tester::#UN3] [setup] export PATH=/tmp/strawberry/strawberry/pineapple:$PATH
remote: [tester::#UN3] Running ./your_program.sh
remote: [your-program] $ ls -1 nonexistent >> /tmp/foo/bar.md
remote: [your-program] ls: cannot access 'nonexistent': No such file or directory
remote: [tester::#UN3] ^ Line does not match expected value.
remote: [tester::#UN3] Expected: "ls: nonexistent: No such file or directory"
remote: [tester::#UN3] Received: "ls: cannot access 'nonexistent': No such file or directory"
remote: [your-program] $
remote: [tester::#UN3] Assertion failed.
remote: [tester::#UN3] Test failed
d) cmd forward, type backwards:
remote: [tester::#MG5] Running tests for Stage #MG5 (The type builtin: executable files)
remote: [tester::#MG5] [setup] export PATH=/tmp/qux:$PATH
remote: [tester::#MG5] [setup] export PATH=/tmp/quz:$PATH
remote: [tester::#MG5] [setup] export PATH=/tmp/foo:$PATH
remote: [tester::#MG5] [setup] PATH is now: /tmp/foo:/tmp/quz:/tmp/qux:/vcpkg:...
remote: [tester::#MG5] [setup] Available executables:
remote: [tester::#MG5] [setup] - my_exe
remote: [tester::#MG5] Running ./your_program.sh
remote: [your-program] $ type cat
remote: [your-program] cat is /bin/cat
remote: [tester::#MG5] ^ Line does not match expected value.
remote: [tester::#MG5] Expected: "cat is /usr/bin/cat"
remote: [tester::#MG5] Received: "cat is /bin/cat"
remote: [your-program] $
remote: [tester::#MG5] Assertion failed.
remote: [tester::#MG5] Test failed