Ls-tree: Test randomness and lack of debug

[JSchneidler]

Looking for help, but also providing feedback.

In the tests for ls-tree, it seems 3 randomly named files are generated to test the sorting/ordering of the output. In this case, it is “blueberry”, “pear”, and “mango”. When running tests locally (I would use the same tree object if I knew what it was, see feedback below), codecrafters test reports:
Expected “blueberry\nmango\npear\n” as stdout, got: “blueberry\npear\nmango\n”
Yet when I do:
git ls-tree --name-only b96a33bbd2d5ba747aa429e9089de785adb6e2e2 && ~/code/codecrafters/git/your_program.sh ls-tree --name-only b96a33bbd2d5ba747aa429e9089de785adb6e2e2
I get output identical to the official git implementation (directories alphabetically, then files alphabetically) with every failing test case codecrafters gives me:

blueberry
mango
pear
blueberry
mango
pear

I’m not sure where the issue is, how the test suite is calling my program differently, or how to effectively debug this. The test suite is pretty much a black box and my output matches the official implementation.

My feedback:

1. The debug info for this test is rather useless. All it adds is “Writing a tree to git storage..” with no information about the tree itself and it seems to get deleted right after the tests run or I can’t find it. So reproducing locally outside of the test suite is rather difficult. I’m not sure how/if I can decode the SHA1 it gives me to see what the tree object it created looks like.
2. Tests should be deterministic. Rather than generating a random/pseudorandom set of files, the tests should use a predefined set of files/words in multiple test cases to ensure sorting, ordering, etc.. is done correctly. I didn’t notice this issue until I had moved onto the write tree challenge as the tests passed the first time and still do intermittently.

Repo published: GitHub - JSchneidler/codecrafters-git-go

[andy1li]

Hey @JSchneidler, looks like you’ve got past this stage. Do you happen to remember what was wrong? Would love to see if we can improve the tester / instructions.

Regarding the randomness, it’s there to prevent hardcoded solutions.

[JSchneidler]

To be honest, I just kept running it until the tests passed so I could continue. It is frustrating, but I want to keep my momentum.

I would be open to spending more time investigating, but at a minimum would like to see more debug information about the tree object it is using for tests, maybe even an option to not delete it, etc.. so I could analyze a little more. But considering a SHA1 can’t just be reversed, it is not practical for me to reproduce locally. I’ve tried creating files with the same names and get the expected output as shown in OP. But whether it’s the directory structure, file contents, or whatever that’s different, I don’t get the same SHA1.

[andy1li]

The entries are expected to be sorted lexicographically. Could you give that a try?

[JSchneidler]

I found the issue. I read somewhere that directories come before files, but this is only the case when the basename matches. I mistakenly read this as all directories come before files. Removing that logic from my code gives the standard lexicographic sorting and tests pass. Thanks for your help, I really appreciate it!