# core/lexers/shell.lexer.ts

> 45 lines of code and 11 definitions.

Tree: GovLab Docs
Language: typescript
Layer: processing
Canonical: https://banes-lab.com/anatomy/docs#file-docs-core-lexers-shell-lexer-ts
Source text: https://banes-lab.com/source/docs/core/lexers/shell.lexer.ts.txt

Listed in [core/lexers](https://banes-lab.com/api/source/docs/core/lexers.md), after [core/lexers/code.go.lexer.ts](https://banes-lab.com/source/docs/core/lexers/code.go.lexer.ts.md).

## Definitions

- `tokenizeCommand` (lexical_declaration, line 49, exported)
- `QUOTES` (lexical_declaration, line 1)
- `WHITESPACE` (lexical_declaration, line 2)
- `Tokenizer` (class_declaration, line 4)
- `words` (public_field_definition, line 5)
- `current` (public_field_definition, line 6)
- `quote` (public_field_definition, line 7)
- `tokenize` (method_definition, line 9)
- `feed` (method_definition, line 17)
- `inQuote` (method_definition, line 33)
- `flush` (method_definition, line 41)

## Contained in

- [core/lexers](https://banes-lab.com/anatomy/docs/folder-docs-core-lexers.md)

## Used by

- [core/validators/package.validator.ts](https://banes-lab.com/source/docs/core/validators/package.validator.ts.md)

## Linked from

- [core/validators](https://banes-lab.com/anatomy/docs/folder-docs-core-validators.md)

## Source

```typescript
const QUOTES: ReadonlySet<string> = new Set(['"', "'"]);
const WHITESPACE: ReadonlySet<string> = new Set([" ", "\t", "\n", "\r"]);

class Tokenizer {
    private readonly words: string[] = [];
    private current = "";
    private quote = "";

    public tokenize(command: string): string[] {
        for (const char of command) {
            this.feed(char);
        }
        this.flush();
        return this.words;
    }

    private feed(char: string): void {
        if (this.quote !== "") {
            this.inQuote(char);
            return;
        }
        if (QUOTES.has(char)) {
            this.quote = char;
            return;
        }
        if (WHITESPACE.has(char)) {
            this.flush();
            return;
        }
        this.current += char;
    }

    private inQuote(char: string): void {
        if (char === this.quote) {
            this.quote = "";
            return;
        }
        this.current += char;
    }

    private flush(): void {
        if (this.current !== "") {
            this.words.push(this.current);
            this.current = "";
        }
    }
}

export const tokenizeCommand = function tokenizeCommand(command: string): string[] {
    return new Tokenizer().tokenize(command);
};
```