I don’t know why this is, but in much later stages of this challenge the very basic cd ~ test is flaky and failing randomly. It happened multiple times for me; once it was fixed by changing the access test from if(access(red_path, F_OK) != -1) to if(access(red_path, R_OK) != -1) (or the other way around), and the last time it was fixed by setting debug: true! This clearly must be some kind of bug in the test suite.
Also in case it matters, here is the entire code for the stage in which changing the debug setting made it pass the tests: main.c · GitHub (This was in #qm8 )
I’m not that familiar with Linux. The test could be flaky. Or it could be some kind of effective vs real UID permission issue?
I don’t bother checking the permission, and had no issues with that test. I just try to change directory and react to the result. My cd simply expands the path and do return chdir(path) == 0 ? SYSTEM_OK : SYSTEM_NOT_FOUND;.
Wouldn’t X_OK be the more proper flag to check?
Yeah, chdir might be the better way to go. Idk if switching to a directory counts as “executing” it. But I do think that if a directory exists should be based on visibility, not permissions, so really there should be two checks: access with F_OK (visibility) for “not found” and chdir for “permission denied”
But in any way, this is not really the point. The home directory should have all of those permissions anyway, so the test failing here points more to something else being wrong, esp. when no change in code, and just in the debug settings of codecrafters.yml, can make it work again.
I’ve gotten the same issue again, this time it was not reverting so easy, the error message was:
remote: [tester::#GP4] Running tests for Stage #GP4 (Navigation - The cd builtin: Home directory)
remote: [tester::#GP4] Running ./your_program.sh
remote: [your-program] $ cd /tmp/pear/pineapple/blueberry
remote: [your-program] $ pwd
remote: [your-program] /tmp/pear/pineapple/blueberry
remote: [tester::#GP4] Received current working directory response
remote: [your-program] $ cd ~
remote: [your-program] cd: /tmp/pear/orange/strawberr: No such file or directory
remote: [tester::#GP4] ^ Expected prompt ("$ ") but received "cd: /tmp/pear/orange/strawberr: No such file or directory"
remote: [your-program] $
remote: [tester::#GP4] Assertion failed.
remote: [tester::#GP4] Test failed (try setting 'debug: true' in your codecrafters.yml to see more details)
remote:
This seems to indicate that for some reason the folder names are cut off after a point (see strawberr).
That particular error does not look like the test runner’s fault. Even if the test runner had flaw in directory creation and setup the path itself comes rather innocently and can be traced via the test case calling getRandomDirectory which calls RandomWord (the words themselves are in the top that file), and the path itself is not modified after construction.
Basically 3 random words are joined together with / onto /tmp, which get assigned to HOME for the test. That would make me suspect there is at least a bug somewhere in getting the HOME path into the red_path, so I would check if the cutoff is caused by my_getenv or reduce_path.
This is the opposite problem, but could cause other sporadic issues. In reduce_path you have an indeterminate byte:
// For cd ~ case we have:
// home_ends_in_slash=0
// old_path_len = 1
// Suppose home_len=10
buf_len = home_len - home_ends_in_slash + old_path_len;
// Then buf_len=11
buf = malloc(buf_len+1);
// buf is allocated with 12 then
// Copy home_len=10 bytes, sets buf[0]...buf[9]
memmove(buf, home, home_len - home_ends_in_slash);
// Omitted: IF not taken here because old_path_len<=1
// Set buf[11]='\0'.
buf[buf_len] = '\0';
// PROBLEM: buf[10] is indeterminate.
1 Like
Yes, this solved one of the issues, I have to replace old_path_len with old_path_len-1. Thanks for the feedback and we will look into other errors first before we open posts then for failing tests.
Small follow-up: We got hit with this error now:
remote: [tester::#UN3] Running tests for Stage #UN3 (Redirection - Append stderr)
remote: [tester::#UN3] [setup] export PATH=/tmp/orange/pear/orange:$PATH
remote: [tester::#UN3] Running ./your_program.sh
remote: [your-program] $ ls -1 nonexistent >> /tmp/baz/bar.md
remote: [your-program] ls: cannot access 'nonexistent': No such file or directory
remote: [tester::#UN3] ^ Line does not match expected value.
remote: [tester::#UN3] Expected: "ls: nonexistent: No such file or directory"
remote: [tester::#UN3] Received: "ls: cannot access 'nonexistent': No such file or directory"
remote: [your-program] $
remote: [tester::#UN3] Assertion failed.
remote: [tester::#UN3] Test failed
This might be a weird unix vs windows edgecase, but it’s on a different test anyway and we’ll see if we can’t close it ourselves in some other way.
(The strange thing about this bug is that this is the exact result bash gives - so something must be afoot, we just don’t know what)
For ls:
- On Git Bash in Windows: I get the “cannot access” version.
- In WSL in Wnidows: I get the “ls: xyz: No such file or directory” version.
- On an older VPS running Ubuntu Server: I get the “cannot access” version.
It seems they have a directory of the binaries used, so the online testers should run the correct version for the tests. This is definitely strange.
1 Like
I have now seen it has something to do whether $PATH is search backwards or forwards. There are two places (type and the CMD branch) where the path is searched, and any choice (forwards/backwards) produces a different error, so 4 different tests fail; this is definitely bizzare:
a) both forward fails
remote: [tester::#MG5] Running tests for Stage #MG5 (The type builtin: executable files)
remote: [tester::#MG5] [setup] export PATH=/tmp/qux:$PATH
remote: [tester::#MG5] [setup] export PATH=/tmp/quz:$PATH
remote: [tester::#MG5] [setup] export PATH=/tmp/foo:$PATH
remote: [tester::#MG5] [setup] PATH is now: /tmp/foo:/tmp/quz:/tmp/qux:/vcpkg:...
remote: [tester::#MG5] [setup] Available executables:
remote: [tester::#MG5] [setup] - my_exe
remote: [tester::#MG5] Running ./your_program.sh
remote: [your-program] $ type cat
remote: [your-program] cat is /usr/bin/cat
remote: [tester::#MG5] ✓ Received expected response
remote: [your-program] $ type cp
remote: [your-program] cp is /usr/bin/cp
remote: [tester::#MG5] ✓ Received expected response
remote: [your-program] $ type mkdir
remote: [your-program] mkdir is /usr/bin/mkdir
remote: [tester::#MG5] ✓ Received expected response
remote: [your-program] $ type my_exe
remote: [your-program] my_exe is /tmp/foo/my_exe
remote: [tester::#MG5] ^ Line does not match expected value.
remote: [tester::#MG5] Expected: "my_exe is /tmp/quz/my_exe"
remote: [tester::#MG5] Received: "my_exe is /tmp/foo/my_exe"
remote: [your-program] $
remote: [tester::#MG5] Assertion failed.
remote: [tester::#MG5] Test failed
b) both backwards fails (this was my original case - and i think the wanted behavior: search the last-added folder to PATH first):
remote: [tester::#UN3] Running tests for Stage #UN3 (Redirection - Append stderr)
remote: [tester::#UN3] [setup] export PATH=/tmp/strawberry/strawberry/pineapple:$PATH
remote: [tester::#UN3] Running ./your_program.sh
remote: [your-program] $ ls -1 nonexistent >> /tmp/foo/bar.md
remote: [your-program] ls: cannot access 'nonexistent': No such file or directory
remote: [tester::#UN3] ^ Line does not match expected value.
remote: [tester::#UN3] Expected: "ls: nonexistent: No such file or directory"
remote: [tester::#UN3] Received: "ls: cannot access 'nonexistent': No such file or directory"
remote: [your-program] $
remote: [tester::#UN3] Assertion failed.
remote: [tester::#UN3] Test failed
c) type forward, cmd backwards:
remote: [tester::#UN3] Running tests for Stage #UN3 (Redirection - Append stderr)
remote: [tester::#UN3] [setup] export PATH=/tmp/strawberry/strawberry/pineapple:$PATH
remote: [tester::#UN3] Running ./your_program.sh
remote: [your-program] $ ls -1 nonexistent >> /tmp/foo/bar.md
remote: [your-program] ls: cannot access 'nonexistent': No such file or directory
remote: [tester::#UN3] ^ Line does not match expected value.
remote: [tester::#UN3] Expected: "ls: nonexistent: No such file or directory"
remote: [tester::#UN3] Received: "ls: cannot access 'nonexistent': No such file or directory"
remote: [your-program] $
remote: [tester::#UN3] Assertion failed.
remote: [tester::#UN3] Test failed
d) cmd forward, type backwards:
remote: [tester::#MG5] Running tests for Stage #MG5 (The type builtin: executable files)
remote: [tester::#MG5] [setup] export PATH=/tmp/qux:$PATH
remote: [tester::#MG5] [setup] export PATH=/tmp/quz:$PATH
remote: [tester::#MG5] [setup] export PATH=/tmp/foo:$PATH
remote: [tester::#MG5] [setup] PATH is now: /tmp/foo:/tmp/quz:/tmp/qux:/vcpkg:...
remote: [tester::#MG5] [setup] Available executables:
remote: [tester::#MG5] [setup] - my_exe
remote: [tester::#MG5] Running ./your_program.sh
remote: [your-program] $ type cat
remote: [your-program] cat is /bin/cat
remote: [tester::#MG5] ^ Line does not match expected value.
remote: [tester::#MG5] Expected: "cat is /usr/bin/cat"
remote: [tester::#MG5] Received: "cat is /bin/cat"
remote: [your-program] $
remote: [tester::#MG5] Assertion failed.
remote: [tester::#MG5] Test failed
I can’t exactly tell what is going on, but desired behavior is: if absolute path then try that, otherwise go through each PATH folder left-to-right (forward) and select the first name matching executable.
I notice you have path_from_filename → file_exists which does a fopen(path,"rb"); check. This is not the right check, since a file that is not executable could be selected instead. I tested file for eligibility using access(path, X_OK).
I think the test runner might even add “red-herrings” that are not executable to test one’s shell behavior, based on this post: [Shell][Python] mg5 multiple my exe? - #5 by andy1li
This actually solved it, Thank you! I am a bit confused why it didnt fail me in previous stages, but good that it works now.
@hypatia-of-sva Sorry for the confusion! We recently fixed the faulty test cases in #MG5, which previously weren’t catching cases where the result wasn’t executable. Glad to hear it’s working now!