Multiple completions #WH6 - Zsh like behavior

[h3r0cybersec]

I’m stuck on Stage Multiple completions #WH6.

As suggested, I’m trying to use an external library to implement the command autocompletion feature. The library in question is the following: GitHub - chzyer/readline: Readline is a pure go(golang) implementation for GNU-Readline kind library.
After several management issues, probably due to the fact that this library works more like zsh than bash, I reached the point where, following the list of similar commands shown on the next line, in the case of this library, a selector is provided directly on this line, rather than replicating the behavior of bash, which for similar commands displays the possible “completers” on the next line and then displays the existing command on the next line, waiting for the user to add the correct command.

Here are my logs:

[tester::#WH6] Typed "xyz_"
[your-program] $ xyz_
[tester::#WH6] ✓ Prompt line matches "$ xyz_"
[tester::#WH6] Pressed "<TAB>" (expecting bell to ring)
[tester::#WH6] Pressed "<TAB>" (expecting autocomplete to "xyz_baz  xyz_foo  xyz_quz")
[tester::#WH6] ✓ Received bell
[your-program] xyz_baz  xyz_foo  xyz_quz
[tester::#WH6] ✓ Prompt line matches "xyz_baz  xyz_foo  xyz_quz"
[tester::#WH6] Didn't find expected line.
[tester::#WH6] Expected: "$ xyz_"
[tester::#WH6] Received: "" (no line received)
[tester::#WH6] Assertion failed.
[tester::#WH6] Test failed

So expected bash behavior:

$ xyz_
xyz_baz  xyz_foo  xyz_quz
$ xyz_

My shell behavior based on current readline library:

So far, I’ve already made changes to the original library code to ring the bell when autocompletion returns no results, which wasn’t supported in the original code: Fixed a small bug that prevented the terminal from playing Bell sound when a command's autocompletion was not found by h3r0cybersec · Pull Request #256 · chzyer/readline · GitHub , and modified the results to be returned in alphabetical order, which is also not handled directly by the library.

What should I do now? Am I doing something wrong? Because I don’t want to make any more changes to the original library and focus on the shell implementation.

[andy1li]

Hey @h3r0cybersec, I tried running your code and noticed the following behavior:

1. Type go
2. Press Tab
   • Unexpected: Both the “ring” and autocompletion are triggered simultaneously
   • Expected: Only “ring” should trigger
3. Press Tab again
   • Expected: Now autocompletion should trigger

Could you try fixing this part first? Once that’s resolved, we can move forward with the next bit.

---

Hint: You shouldn’t need to modify the readline library. You can just do something like fmt.Print("\a") to ring the bell.

[h3r0cybersec]

Yes, I can try.
The problem, however, stems from how the library is implemented. Before I added the bell sound when no autocompletion was found, the library’s default behavior was to display autocompletion options similar to the command entered. This is different from what you correctly described and how the bash shell works. So, the first “TAB” rings the bell and the second shows similar options.
I’ll write here again as soon as I’ve fixed this behavior.

[h3r0cybersec]

Hey andy1li,
I updated the code with the fix discussed and it now works according to the logic described. The only difference is the context menu that opens after the second “TAB,” due to the implementation of the library in use. The test expects the list of commands found for autocompletion to be printed on a new line, and then displays the prompt pre-populated with the user’s input on the next line.

I’m waiting for the next steps

[andy1li]

@h3r0cybersec Nicely done!

One thing you could try is suppressing the default output from readline and manually printing a custom list when multiple matches are found.

Let me know if you’d like assistance with that!

[h3r0cybersec]

Hey @ andy1li, yes, I can do that, but it still involves modifying the library code exactly like this last modification. This is because it’s designed to work as a zsh shell, not a bash one.

At this point, couldn’t we relax the test by implementing this different autocompletion management instead of just one type? At least extend it with the features of the most popular shells.

If I had known before using it that it would cost me to modify that part of the library code, I would have implemented all the autocompletion logic from scratch.

Regardless, I’ll listen to what you think is the best solution, so I look forward to hearing your thoughts on what’s been said in this comment.

[andy1li]

but it still involves modifying the library code exactly like this last modification.

@h3r0cybersec There is probably no need to modify the library code.

You can try something like this instead:

[h3r0cybersec]

ok i will check this

[h3r0cybersec]

Hey @andy1li, I checked your solution, but it’s not feasible. It’s probably sufficient to pass the test, but it’s not the correct way to proceed. This solution actually affects the shell’s behavior.

This is an example of an invalid command, so no autocompletion and the above is the output:

This is an example of a builtin command that correctly returns the non-nil completion option:

This is an example of external command:

As you can see, the prompt at the bottom is not returned correctly but you have to press a key for it to be visible again.

So the point here is that we’re trying to force behavior that isn’t expected from this library. There are three solutions now:

• abandon the library and rewrite this custom part
• modify the library so it can handle the autocompletion result in this way (I keep repeating this because it is the only way to properly achieve the desired result)
• use another library like (The Go Programming Language, GitHub - eiannone/keyboard: Simple library to listen for keystrokes from the keyboard) that implements basic functionalities for terminal handling to low level with the only necessary code to handle the terminal’s function.

Let me know what you think. Thanks for the support!

[andy1li]

Because I don’t want to make any more changes to the original library and focus on the shell implementation.

That makes sense, but I don’t think modifying readline is necessary. The completer function is quite flexible and should allow for the behavior we need.

This is an example of an invalid command, so no autocompletion and the above is the output:

Hmm, looks like this should’ve been handled in an earlier stage (#QM8 Missing completions):

Here’s some pseudocode:

if len(completions) == 0 {
  // ring bell first
  return nil
}

This is an example of external command:

Could you try pushing your latest code without using the customized readline? That’ll help us move forward with debugging.

Here’s the output I got when running your code without the customized readline:

[h3r0cybersec]

@andy1li So I’ve reverted the code back to using the original library without my changes. You should now have access to the updated code. However, the autocompletion result with this library is as follows:

with the selection option on next “TAB”:

In any case, I found a solution to avoid touching code added to that library. I created a PoC that I’m releasing below using the library: golang.org/x/term.

This is the basis on which the library I’m currently using was built. Below is the PoC with the code that implements what the tests require:

package main

import (
   "os"
   "strings"

   "golang.org/x/term"
)

var prompt string = "$ "
var showMatchesOnNextTAB bool = false

func autocompleter(terminal *term.Terminal, line string, pos int, key rune) (newLine string, newPos int, ok bool) {
   commands := []string{"echo", "export", "pwd", "exit", "cd", "go", "gobuster"}
   // current text
   prefix := line[:pos]

   // handle TAB key
   if key == '\t' {
   	// store all matches
   	var matches []string
   	for _, cmd := range commands {
   		if strings.HasPrefix(cmd, prefix) {
   			matches = append(matches, cmd)
   		}
   	}
   	switch len(matches) {
   	case 0:
   		terminal.Write([]byte("\a"))
   		return line, pos, true
   	case 1:
   		terminal.Write([]byte("\a"))
   		match := matches[0]
   		return match, len(match), true
   	default:
   		if showMatchesOnNextTAB {
   			terminal.Write([]byte(prompt + line))
   			terminal.Write([]byte("\r\n"))
   			terminal.Write([]byte(strings.Join(matches, "   ") + "\r\n"))
   			showMatchesOnNextTAB = false
   			return line, pos, false
   		}
   		terminal.Write([]byte("\a"))
   		showMatchesOnNextTAB = true
   		return line, pos, true
   	}
   }

   // return original content
   return line, pos, false
}

func main() {
   oldState, err := term.MakeRaw(int(os.Stdin.Fd()))
   if err != nil {
   	// Handle error
   }
   defer term.Restore(int(os.Stdin.Fd()), oldState)
   // Create a new terminal with a prompt
   terminal := term.NewTerminal(os.Stdin, prompt)
   terminal.AutoCompleteCallback = func(line string, pos int, key rune) (newLine string, newPos int, ok bool) {
   	return autocompleter(terminal, line, pos, key)
   }

   // Read lines of input
   for {
   	_, err := terminal.ReadLine()
   	if err != nil {
   		break
   	}
   }
}

So do you want to try to investigate that problem or do I just continue with this?

[andy1li]

So I’ve reverted the code back to using the original library without my changes.

@h3r0cybersec Nice! Let’s go back and fix the earlier stages one at a time. The first one is #QM8 Missing completions:

Note that you can re-run previous tests using our CLI:

codecrafters test --previous

Hints:

1. The bell can be triggered by printing "\a"
2. The default autocompletion result can be suppressed by return nil instead of return completions within the completer function.

Let me know if you’d like more hints along the way!

---

In any case, I found a solution to avoid touching code added to that library. I created a PoC that I’m releasing below using the library: golang.org/x/term.

Nice work! I’d suggest focusing on one library at a time for now. Once we’ve sorted things out with readline, we can definitely explore x/term if you’d like.

[h3r0cybersec]

Hey @andy1li , I finally managed to solve the problem by implementing the AutoCompleter interface to customize a specific function in my code, as shown below:

...
type CustomComplete struct {
	terminal *readline.Terminal
	commands []*readline.PrefixCompleter
}

func (cc *CustomComplete) Do(line []rune, offset int) ([][]rune, int) {
	prefix := string(line[:offset])
	var matches [][]rune

	for _, cmd := range cc.commands {
		if strings.HasPrefix(string(cmd.Name), prefix) {
			matches = append(matches, cmd.Name)
		}
	}

	switch len(matches) {
	case 0:
		cc.terminal.Bell()
		return [][]rune{}, offset
	case 1:
		match := matches[0]
		return [][]rune{match[offset:]}, offset
	default:
		orderedMatches := CandidateOrdered(matches)
		sort.Sort(orderedMatches)
		if showMatchesOnNextTAB {
			// go on next line
			suggestions := make([]string, len(orderedMatches))
			for _, match := range orderedMatches {
				suggestions = append(suggestions, string(match))
			}
			showMatchesOnNextTAB = false
			cc.terminal.Write([]byte("\r\n" + strings.TrimSpace(strings.Join(suggestions, " ")) + "\r\n"))
			// on next line print prompt with current line
			return [][]rune{line[offset:]}, offset
		}
		cc.terminal.Write([]byte("\a"))
		showMatchesOnNextTAB = true
		return [][]rune{}, offset
	}
}
...

this is the place where the core logic is implemented.
Thank you so much for your support!