gizmodel/sql/lex
SQL tokens. One lexer for every dialect: identifiers bare, "quoted", [bracketed] (SQL Server) or backquoted (MySQL); strings '...' with '' inside, E'...' escapes and $tag$...$tag$ (PostgreSQL) and N'...' (SQL Server, Oracle); numbers; parameters $1, ?, ?1, :name and @name; operators; and comments -- and /* */.
import * as Lex from "gizmodel/sql/lex";Types
Token
type Token = { kind: i64; text: string; at: i64; }A token: its kind, its text, and its byte offset in the source.
kind:i64text:stringat:i64
Literals
type Literals = { text: string; values: Vec<string>; types: Vec<i64>; }A text with its compared literals as parameters (comparedLiterals): text has $1, $2, ... where they were, values
are their texts and types their PostgreSQL types: 0 (unknown) for a string, as a quoted literal is, int4 or int8 for
an integer.
text:stringvalues:Vec<string>types:Vec<i64>
Functions
keyword
function keyword(token: Token, word: string): boolAn unquoted ASCII keyword, compared without allocating a lowercase token.
Effects: fail.
skipParens
function skipParens(toks: Vec<Token>, at: i64): i64The index after the parenthesized group that starts at token at, or the END token if it is not closed.
Effects: fail.
userOperator
function userOperator(t: string): boolWhether an operator token is none of the built-in ones the lexer knows: a user-defined operator (parse.giz).
Effects: fail, alloc.
lex
function lex(sql: string): Vec<Token> throws stringThe tokens of sql, ending with one END token, or the position and reason of the first
malformed one.
Effects: fail, alloc.
valueAs
function valueAs(check: string, replacement: string): stringA domain check on VALUE with each VALUE as replacement (a quoted column, or a parenthesized cast).
Effects: fail, alloc.
domainChecks
function domainChecks(domain: string, text: string): Vec<string>A domain's CHECK constraints as "name\u0003check" (the domain object's fifth part, "\u0004" between; before, a fourth part held one unnamed check, DOMAIN_check).
Effects: fail, alloc.
comparedLiterals
function comparedLiterals(sql: string): Option<Literals>The literals that a comparison takes in SQL (k = 42, name = 'x') as parameters, so texts that differ only in them
share one parsed statement and one shape (psql's and psycopg2's queries carry their values in the text). Only a plain
'...' string and an integer, after = < > <= >= <> or !=, and before the end, a word, ")", "," or ";". None when the
text has none, or has parameters of its own.
Effects: fail, alloc.
positionalBinds
function positionalBinds(sql: string, count: i64): stringSQL with each :name bind written as $N. When COUNT is the number of distinct names, a name repeated keeps one number; else each occurrence takes the next number. Quotes and comments are kept.
Effects: fail, alloc.
engineNameOf
function engineNameOf(text: string, dialect: string): stringThe engine's name for the quoted identifier text: Oracle keeps a name with a lowercase letter, PostgreSQL one with an
uppercase letter, in their encoded form; the rest fold to the unquoted spelling and keep their text; so do the engine's own names.
Effects: fail, alloc.
foldQuoted
function foldQuoted(sql: string, dialect: string): stringsql with each double-quoted identifier that dialect folds differently from its text spelled as encodedName. Other
dialects, a text without quotes and an unlexable text come back unchanged; an encoded name is left alone.
Effects: fail, alloc.
Constants
IDENT
const IDENT: i64 = 1Token kind: an identifier or a keyword.
QUOTED
const QUOTED: i64 = 2A quoted identifier: never a keyword, case kept.
STRING
const STRING: i64 = 3Token kind: a string literal.
NUMBER
const NUMBER: i64 = 4Token kind: a number.
PARAM
const PARAM: i64 = 5Token kind: a parameter.
OP
const OP: i64 = 6Token kind: an operator or punctuation.
END
const END: i64 = 7Token kind: the end of the text.
BLOB
const BLOB: i64 = 8X'...' blob literal, its text the hex digits.