# gizmodel/sql/lex

SQL tokens. One lexer for every dialect: identifiers bare, "quoted", [bracketed] (SQL Server) or `backquoted` (MySQL); strings '...' with '' inside, E'...' escapes and $tag$...$tag$ (PostgreSQL) and N'...' (SQL Server, Oracle); numbers; parameters $1, ?, ?1, :name and @name; operators; and comments -- and /* */.

```gizlang
import * as Lex from "gizmodel/sql/lex";
```

## Types

### `Token`

```gizlang
type Token = { kind: i64; text: string; at: i64; }
```

A token: its kind, its text, and its byte offset in the source.

- `kind`: `i64`
- `text`: `string`
- `at`: `i64`

### `Literals`

```gizlang
type Literals = { text: string; values: Vec<string>; types: Vec<i64>; }
```

A text with its compared literals as parameters (comparedLiterals): `text` has $1, $2, ... where they were, `values`
are their texts and `types` their PostgreSQL types: 0 (unknown) for a string, as a quoted literal is, int4 or int8 for
an integer.

- `text`: `string`
- `values`: `Vec<string>`
- `types`: `Vec<i64>`

## Functions

### `keyword`

```gizlang
function keyword(token: Token, word: string): bool
```

An unquoted ASCII keyword, compared without allocating a lowercase token.

Effects: fail.

### `skipParens`

```gizlang
function skipParens(toks: Vec<Token>, at: i64): i64
```

The index after the parenthesized group that starts at token `at`, or the END token if it is not closed.

Effects: fail.

### `userOperator`

```gizlang
function userOperator(t: string): bool
```

Whether an operator token is none of the built-in ones the lexer knows: a user-defined operator (parse.giz).

Effects: fail, alloc.

### `lex`

```gizlang
function lex(sql: string): Vec<Token> throws string
```

The tokens of `sql`, ending with one END token, or the position and reason of the first
malformed one.

Effects: fail, alloc.

### `valueAs`

```gizlang
function valueAs(check: string, replacement: string): string
```

A domain check on VALUE with each VALUE as `replacement` (a quoted column, or a parenthesized cast).

Effects: fail, alloc.

### `domainChecks`

```gizlang
function domainChecks(domain: string, text: string): Vec<string>
```

A domain's CHECK constraints as "name\u0003check" (the domain object's fifth part, "\u0004" between; before, a fourth
part held one unnamed check, DOMAIN_check).

Effects: fail, alloc.

### `comparedLiterals`

```gizlang
function comparedLiterals(sql: string): Option<Literals>
```

The literals that a comparison takes in SQL (`k = 42`, `name = 'x'`) as parameters, so texts that differ only in them
share one parsed statement and one shape (psql's and psycopg2's queries carry their values in the text). Only a plain
'...' string and an integer, after = < > <= >= <> or !=, and before the end, a word, ")", "," or ";". None when the
text has none, or has parameters of its own.

Effects: fail, alloc.

### `positionalBinds`

```gizlang
function positionalBinds(sql: string, count: i64): string
```

SQL with each :name bind written as $N. When COUNT is the number of distinct names, a name
repeated keeps one number; else each occurrence takes the next number. Quotes and comments are kept.

Effects: fail, alloc.

### `engineNameOf`

```gizlang
function engineNameOf(text: string, dialect: string): string
```

The engine's name for the quoted identifier `text`: Oracle keeps a name with a lowercase letter, PostgreSQL one with an
uppercase letter, in their encoded form; the rest fold to the unquoted spelling and keep their text; so do the engine's own names.

Effects: fail, alloc.

### `foldQuoted`

```gizlang
function foldQuoted(sql: string, dialect: string): string
```

`sql` with each double-quoted identifier that `dialect` folds differently from its text spelled as encodedName. Other
dialects, a text without quotes and an unlexable text come back unchanged; an encoded name is left alone.

Effects: fail, alloc.

## Constants

### `IDENT`

```gizlang
const IDENT: i64 = 1
```

Token kind: an identifier or a keyword.

### `QUOTED`

```gizlang
const QUOTED: i64 = 2
```

A quoted identifier: never a keyword, case kept.

### `STRING`

```gizlang
const STRING: i64 = 3
```

Token kind: a string literal.

### `NUMBER`

```gizlang
const NUMBER: i64 = 4
```

Token kind: a number.

### `PARAM`

```gizlang
const PARAM: i64 = 5
```

Token kind: a parameter.

### `OP`

```gizlang
const OP: i64 = 6
```

Token kind: an operator or punctuation.

### `END`

```gizlang
const END: i64 = 7
```

Token kind: the end of the text.

### `BLOB`

```gizlang
const BLOB: i64 = 8
```

X'...' blob literal, its text the hex digits.

