Free-form

Free-form style is the default one. It can also be enabled with configuration re2c:api:style = free-form. In this style interface primitives should be defined as free-form pieces of code with interpolated variables of the form @@{var} or optionally just @@ if there is a single variable. The set of variables is specific to each primitive. Here's how free-form generic API can be defined in terms of integer variables cursor, limit, marker, ctxmarker and a string (or a byte slice) data:

/*!re2c


  re2c:YYPEEK       = "data[cursor]";


  re2c:YYSKIP       = "cursor++";


  re2c:YYBACKUP     = "marker = cursor";


  re2c:YYRESTORE    = "cursor = marker";


  re2c:YYBACKUPCTX  = "ctxmarker = cursor";


  re2c:YYRESTORECTX = "cursor = ctxmarker";


  re2c:YYRESTORETAG = "cursor = ${tag}";


  re2c:YYLESSTHAN   = "limit - cursor < @@{len}";


  re2c:YYSTAGP      = "@@{tag} = cursor";


  re2c:YYSTAGN      = "@@{tag} = -1";


  re2c:YYSHIFT      = "cursor += @@{shift}";


  re2c:YYSHIFTSTAG  = "@@{tag} += @@{shift}";
*/

Function-like

Function-like style is enabled with configuration re2c:api:style = functions. In this style primitives should be defined as closures accepting the necessary arguments. Here's how function-like generic API can be defined in terms of integer variables cursor, limit, marker, ctxmarker and a string (or a byte slice) data:

YYPEEK := func() byte { return data[cursor] }
YYSKIP := func() { cursor++ }
YYBACKUP := func() { marker = cursor }
YYRESTORE := func() { cursor = marker }
YYBACKUPCTX := func() { ctxmarker = cursor }
YYRESTORECTX := func() { cursor = ctxmarker }
YYLESSTHAN := func(n int) bool { return limit-cursor < n }
YYSHIFT := func(n int) { cursor += n }

code-template ::


      name '=' code-exprs ';'


    | CODE_TEMPLATE ';'


    | '<undefined>' ';'
code-exprs ::


      <EMPTY>


    | code-exprs code-expr
code-expr ::


      STRING


    | VARIABLE


    | optional


    | list
optional ::


      '(' CONDITIONAL '?' code-exprs ')'


    | '(' CONDITIONAL '?' code-exprs ':' code-exprs ')'
list ::


      '[' VARIABLE ':' code-exprs ']'


    | '[' VARIABLE '{' NUMBER '}' ':' code-exprs ']'


    | '[' VARIABLE '{' NUMBER ',' NUMBER '}' ':' code-exprs ']'

A code template is a sequence of string literals, variables, optional elements and lists, or a reference to another code template, or a special value <undefined>. Variables are placeholders that are substituted during code generation phase. List variables are special: when expanding list templates, re2go repeats expressions the right hand side of the column a few times, each time replacing occurrences of the list variable with a value specific to this repetition. Lists have optional bounds (negative values are counted from the end, e.g. -1 means the last element). Conditional names start with a dot. Both conditionals and variables may be either local (specific to the given code template) or global (allowed in all code templates). When re2go reads syntax file, it checks that each code template uses only the variables and conditionals that are allowed in it.

For example, the following code template defines if-then-else construct for a C-like language:

code:if_then_else =


    [branch{0}: topindent "if " cond " {" nl


        indent [stmt: stmt] dedent]


    [branch{1:-1}: topindent "} else" (.cond ? " if " cond) " {" nl


        indent [stmt: stmt] dedent]


    topindent "}" nl;

Here branch is a list variable: branch{0} expands to the first branch (which is special, as there is no else part), branch{1:-1} expands to all remaining branches (if any). stmt is also a list variable: [stmt: stmt] is a nested list that expands to a list of statements in the body of the current branch. topindent, indent, dedent and nl are global variables, and .cond is a local conditional (their meaning is described below). This code template could produce the following code:

if x {


    // do something
} else if y {


    // do something else
} else {


    // don't do anything
}

Here's a list of all code templates supported by re2go with their local variables and conditionals. Note that a particular definition may, but does not have to use local variables and conditionals. Any unused code templates should be set to <undefined>.

Here's a list of all global variables that are allowed in syntax files:

Here's a list of all global conditionals that are allowed in syntax files:

//go:generate re2go $INPUT -o $OUTPUT --api simple
package main
// Expect a null-terminated string.
func lex(yyinput string) int {


    yycursor := 0


    count := 0


    for { /*!re2c


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        *      { return -1 }


        [\x00] { return count }


        [a-z]+ { count += 1; continue }


        [ ]+   { continue }


    */


    }
}
func main() {


    assert_eq := func(x, y int) { if x != y { panic("error") } }


    assert_eq(lex("\000"), 0)


    assert_eq(lex("one two three\000"), 3)


    assert_eq(lex("f0ur\000"), -1)
}

//go:generate re2go $INPUT -o $OUTPUT --api simple
package main
// Expects a null-terminated string.
func lex(yyinput string) int {


    var yycursor, yymarker int


    yylimit := len(yyinput) - 1 // lim points at the terminating null


    count := 0


    for { /*!re2c


        re2c:YYCTYPE = byte;


        re2c:yyfill:enable = 0;


        re2c:eof = 0;


        str = ['] ([^'\\] | [\\][^])* ['];


        *    { return -1 }


        $    { return count }


        str  { count += 1; continue }


        [ ]+ { continue }


    */


    }
}
func main() {


    assert_eq := func(x, y int) { if x != y { panic("error") } }


    assert_eq(lex("\000"), 0)


    assert_eq(lex("'qu\000tes' 'are' 'fine: \\'' \000"), 3)


    assert_eq(lex("'unterminated\\'\000"), -1)
}

//go:generate re2go $INPUT -o $OUTPUT --api simple
package main
import "strings"
/*!max:re2c*/
// Expects YYMAXFILL-padded string.
func lex(str string) int {


    // Pad string with YYMAXFILL zeroes at the end.


    yyinput := str + strings.Repeat("\000", int(YYMAXFILL))


    yycursor := 0


    yylimit := len(yyinput)


    count := 0


    for { /*!re2c


        re2c:YYCTYPE = byte;


        re2c:YYFILL = "return -1";


        str = ['] ([^'\\] | [\\][^])* ['];


        [\x00] {


            // Check that it is the sentinel, not some unexpected null.


            if yycursor - 1 == len(str) { return count } else { return -1 }


        }


        str  { count += 1; continue }


        [ ]+ { continue }


        *    { return -1 }


    */


    }
}
func main() {


    assert_eq := func(x, y int) { if x != y { panic("error") } }


    assert_eq(lex(""), 0)


    assert_eq(lex("'qu\000tes' 'are' 'fine: \\'' "), 3)


    assert_eq(lex("'unterminated\\'"), -1)


    assert_eq(lex("'unexpected \000 null\\'"), -1)
}

//go:generate re2go $INPUT -o $OUTPUT
package main
// Returns "fake" terminating null if cursor has reached limit.
func peek(str string, cur int) byte {


    if cur >= len(str) {


        return 0 // fake null


    } else {


        return str[cur]


    }
}
// Expects a string without terminating null.
func lex(str string) int {


    var cur int


    count := 0


    for { /*!re2c


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        re2c:YYPEEK = "peek(str, cur)";


        re2c:YYSKIP = "cur += 1";


        *      { return -1 }


        [\x00] { return count }


        [a-z]+ { count += 1; continue }


        [ ]+   { continue }


    */


    }
}
func main() {


    assert_eq := func(x, y int) { if x != y { panic("error") } }


    assert_eq(lex(""), 0)


    assert_eq(lex("one two three"), 3)


    assert_eq(lex("f0ur"), -1)
}

               <-- shift -->


             >-A------------B---------C-------------D#-----------E->


             buffer       token    marker         limit,


                                                  cursor
>-A------------B---------C-------------D------------E#->


             buffer,  marker        cursor        limit


             token


               <-- shift -->


             >-A------------B---------C-------------D#--E (EOF)


             buffer       token    marker         limit,


                                                  cursor
>-A------------B---------C-------------D---E#........


             buffer,  marker       cursor limit


             token

//go:generate re2go $INPUT -o $OUTPUT
package main
import (


    "os"


    "strings"
)
const BUFSIZE uint = 4096
type Input struct {


    file     *os.File


    yyinput  []byte


    yycursor uint


    yymarker uint


    yylimit  uint


    token    uint


    eof      bool
}
func fill(in *Input) int {


    if in.eof { return -1 } // unexpected EOF


    // Error: lexeme too long. In real life can reallocate a larger buffer.


    if in.token < 1 { return -2 }


    // Shift buffer contents (discard everything up to the current token).


    copy(in.yyinput[0:], in.yyinput[in.token:in.yylimit])


    in.yycursor -= in.token


    in.yymarker -= in.token


    in.yylimit -= in.token


    in.token = 0


    // Fill free space at the end of buffer with new data from file.


    n, _ := in.file.Read(in.yyinput[in.yylimit:BUFSIZE])


    in.yylimit += uint(n)


    in.yyinput[in.yylimit] = 0


    // If read less than expected, this is the end of input.


    in.eof = in.yylimit < BUFSIZE


    return 0
}
func lex(yyrecord *Input) int {


    count := 0


    for {


        yyrecord.token = yyrecord.yycursor


    /*!re2c


        re2c:api = record;


        re2c:eof = 0;


        re2c:YYCTYPE = byte;


        re2c:YYFILL = "fill(yyrecord) == 0";


        str = ['] ([^'\\] | [\\][^])* ['];


        *    { return -1 }


        $    { return count }


        str  { count += 1; continue }


        [ ]+ { continue }


    */


    }
}
func main() () {


    fname := "input"


    content := "'qu\000tes' 'are' 'fine: \\'' ";


    // Prepare input file: a few times the size of the buffer, containing


    // strings with zeroes and escaped quotes.


    f, _ := os.Create(fname)


    f.WriteString(strings.Repeat(content, int(BUFSIZE)))


    f.Seek(0, 0)


    count := 3 * int(BUFSIZE) // number of quoted strings written to file


    // Prepare lexer state: all offsets are at the end of buffer.


    in := &Input{


        file:     f,


        // Sentinel at `yylimit` offset is set to zero, which triggers YYFILL.


        yyinput:  make([]byte, BUFSIZE+1),


        yycursor: BUFSIZE,


        yymarker: BUFSIZE,


        yylimit:  BUFSIZE,


        token:    BUFSIZE,


        eof:      false,


    }


    // Run the lexer.


    if lex(in) != count { panic("error"); }


    // Cleanup: remove input file.


    f.Close();


    os.Remove(fname);
}

               <-- shift -->                 <-- need -->


             >-A------------B---------C-----D-------E---F--------G->


             buffer       token    marker cursor  limit
>-A------------B---------C-----D-------E---F--------G->


             buffer,  marker cursor               limit


             token


               <-- shift -->                 <-- need -->


             >-A------------B---------C-----D-------E-F        (EOF)


             buffer       token    marker cursor  limit
>-A------------B---------C-----D-------E-F###############


             buffer,  marker cursor                   limit


             token                        <- YYMAXFILL ->

//go:generate re2go $INPUT -o $OUTPUT
package main
import (


    "os"


    "strings"
)
/*!max:re2c*/
const BUFSIZE uint = 4096
type Input struct {


    file     *os.File


    yyinput  []byte


    yycursor uint


    yylimit  uint


    token    uint


    eof      bool
}
func fill(in *Input, need uint) int {


    if in.eof { return -1 } // unexpected EOF


    // Error: lexeme too long. In real life can reallocate a larger buffer.


    if in.token < need { return -2 }


    // Shift buffer contents (discard everything up to the current token).


    copy(in.yyinput[0:], in.yyinput[in.token:in.yylimit])


    in.yycursor -= in.token


    in.yylimit -= in.token


    in.token = 0


    // Fill free space at the end of buffer with new data from file.


    n, _ := in.file.Read(in.yyinput[in.yylimit:BUFSIZE])


    in.yylimit += uint(n)


    // If read less than expected, this is end of input => add zero padding


    // so that the lexer can access characters at the end of buffer.


    if in.yylimit < BUFSIZE {


        in.eof = true


        for i := uint(0); i < YYMAXFILL; i += 1 { in.yyinput[in.yylimit+i] = 0 }


        in.yylimit += YYMAXFILL


    }


    return 0
}
func lex(yyrecord *Input) int {


    count := 0


    for {


        yyrecord.token = yyrecord.yycursor


    /*!re2c


        re2c:api = record;


        re2c:YYCTYPE = byte;


        re2c:YYFILL = "if r := fill(yyrecord, @@); r != 0 { return r }";


        str = ['] ([^'\\] | [\\][^])* ['];


        [\x00] {


            // Check that it is the sentinel, not some unexpected null.


            if yyrecord.token == yyrecord.yylimit - YYMAXFILL { return count } else { return -1 }


        }


        str  { count += 1; continue }


        [ ]+ { continue }


        *    { return -1 }


    */


    }
}
func main() () {


    fname := "input"


    content := "'qu\000tes' 'are' 'fine: \\'' ";


    // Prepare input file: a few times the size of the buffer, containing


    // strings with zeroes and escaped quotes.


    f, _ := os.Create(fname)


    f.WriteString(strings.Repeat(content, int(BUFSIZE)))


    f.Seek(0, 0)


    count := 3 * int(BUFSIZE) // number of quoted strings written to file


    // Prepare lexer state: all offsets are at the end of buffer.


    // This immediately triggers YYFILL, as the YYLESSTHAN condition is true.


    in := &Input{


        file:     f,


        yyinput:  make([]byte, BUFSIZE+YYMAXFILL),


        yycursor: BUFSIZE,


        yylimit:  BUFSIZE,


        token:    BUFSIZE,


        eof:      false,


    }


    // Run the lexer.


    if lex(in) != count { panic("error"); }


    // Cleanup: remove input file.


    f.Close();


    os.Remove(fname);
}

//go:generate re2go $INPUT -o $OUTPUT -i --api simple
package main
import "errors"
const u32Limit uint64 = 1<<32
var (


    eSyntax   = errors.New("syntax error")


    eOverflow = errors.New("overflow error")
)
func parse_u32(yyinput string) (uint32, error) {


    var yycursor, yymarker int


    result := uint64(0)


    add := func(base uint64, offset byte) {


        result = result * base + uint64(yyinput[yycursor-1] - offset)


        if result >= u32Limit {


            result = u32Limit


        }


    }


    /*!re2c


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        end = "\x00";


        '0b' / [01]        { goto bin }


        "0"                { goto oct }


        ""   / [1-9]       { goto dec }


        '0x' / [0-9a-fA-F] { goto hex }


        *                  { goto err }


    */
bin:


    /*!re2c


        end   { goto end }


        [01]  { add(2, '0'); goto bin }


        *     { goto err }


    */
oct:


    /*!re2c


        end   { goto end }


        [0-7] { add(8, '0'); goto oct }


        *     { goto err }


    */
dec:


    /*!re2c


        end   { goto end }


        [0-9] { add(10, '0'); goto dec }


        *     { goto err }


    */
hex:


    /*!re2c


        end   { goto end }


        [0-9] { add(16, '0');    goto hex }


        [a-f] { add(16, 'a'-10); goto hex }


        [A-F] { add(16, 'A'-10); goto hex }


        *     { goto err }


    */
end:


    if result < u32Limit {


        return uint32(result), nil


    } else {


        return 0, eOverflow


    }
err:


    return 0, eSyntax
}
func main() {


    test := func(num uint32, str string, err error) {


        if n, e := parse_u32(str); !(n == num && e == err) {


            panic("error")


        }


    }


    test(1234567890, "1234567890\000", nil)


    test(13, "0b1101\000", nil)


    test(0x7fe, "0x007Fe\000", nil)


    test(0644, "0644\000", nil)


    test(0, "9999999999\000", eOverflow)


    test(0, "123??\000", eSyntax)
}

//go:generate re2go -c $INPUT -o $OUTPUT -i --api simple
package main
import "errors"
var (


    eSyntax   = errors.New("syntax error")


    eOverflow = errors.New("overflow error")
)
/*!conditions:re2c*/
const u32Limit uint64 = 1<<32
func parse_u32(yyinput string) (uint32, error) {


    var yycursor, yymarker int


    result := uint64(0)


    yycond := yycinit


    add := func(base uint64, offset byte) {


        result = result * base + uint64(yyinput[yycursor-1] - offset)


        if result >= u32Limit {


            result = u32Limit


        }


    }


    /*!re2c


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        re2c:YYGETCOND = "cond";


        re2c:YYSETCOND = "cond = @@";


        <*> * { return 0, eSyntax }


        <init> '0b' / [01]        :=> bin


        <init> "0"                :=> oct


        <init> ""   / [1-9]       :=> dec


        <init> '0x' / [0-9a-fA-F] :=> hex


        <bin, oct, dec, hex> "\x00" {


            if result < u32Limit {


                return uint32(result), nil


            } else {


                return 0, eOverflow


            }


        }


        <bin> [01]  { add(2, '0');     goto yyc_bin }


        <oct> [0-7] { add(8, '0');     goto yyc_oct }


        <dec> [0-9] { add(10, '0');    goto yyc_dec }


        <hex> [0-9] { add(16, '0');    goto yyc_hex }


        <hex> [a-f] { add(16, 'a'-10); goto yyc_hex }


        <hex> [A-F] { add(16, 'A'-10); goto yyc_hex }


    */
}
func main() {


    test := func(num uint32, str string, err error) {


        if n, e := parse_u32(str); !(n == num && e == err) {


            panic("error")


        }


    }


    test(1234567890, "1234567890\000", nil)


    test(13, "0b1101\000", nil)


    test(0x7fe, "0x007Fe\000", nil)


    test(0644, "0644\000", nil)


    test(0, "9999999999\000", eOverflow)


    test(0, "123??\000", eSyntax)
}

//go:generate re2go -f $INPUT -o $OUTPUT
package main
import (


    "fmt"


    "os"
)
// Use a small buffer to cover the case when a lexeme doesn't fit.
// In real world use a larger buffer.
const BUFSIZE int = 10
type State struct {


    file     *os.File


    yyinput  []byte


    yycursor int


    yymarker int


    yylimit  int


    token    int


    yystate  int
}
const (


    lexEnd = iota


    lexReady


    lexWaitingForInput


    lexPacketBroken


    lexPacketTooBig
)
func fill(st *State) int {


    shift := st.token


    used := st.yylimit - st.token


    free := BUFSIZE - used


    // Error: no space. In real life can reallocate a larger buffer.


    if free < 1 { return lexPacketTooBig }


    // Shift buffer contents (discard already processed data).


    copy(st.yyinput[0:], st.yyinput[shift:shift+used])


    st.yycursor -= shift


    st.yymarker -= shift


    st.yylimit -= shift


    st.token -= shift


    // Fill free space at the end of buffer with new data.


    n, _ := st.file.Read(st.yyinput[st.yylimit:BUFSIZE])


    st.yylimit += n


    st.yyinput[st.yylimit] = 0 // append sentinel symbol


    return lexReady
}
func lex(yyrecord *State, recv *int) int {


    var yych byte


    /*!getstate:re2c*/
loop:


    yyrecord.token = yyrecord.yycursor


    /*!re2c


        re2c:api = record;


        re2c:eof = 0;


        re2c:YYFILL = "return lexWaitingForInput";


        packet = [a-z]+[;];


        *      { return lexPacketBroken }


        $      { return lexEnd }


        packet { *recv = *recv + 1; goto loop }


    */
}
func test(expect int, packets []string) {


    // Create a pipe (open the same file for reading and writing).


    fname := "pipe"


    fw, _ := os.Create(fname)


    fr, _ := os.Open(fname)


    // Initialize lexer state: `state` value is -1, all offsets are at the end


    // of buffer.


    st := &State{


        file:     fr,


        // Sentinel at `yylimit` offset is set to zero, which triggers YYFILL.


        yyinput:  make([]byte, BUFSIZE+1),


        yycursor: BUFSIZE,


        yymarker: BUFSIZE,


        yylimit:  BUFSIZE,


        token:    BUFSIZE,


        yystate:  -1,


    }


    // Main loop. The buffer contains incomplete data which appears packet by


    // packet. When the lexer needs more input it saves its internal state and


    // returns to the caller which should provide more input and resume lexing.


    var status int


    send := 0


    recv := 0


    for {


        status = lex(st, &recv)


        if status == lexEnd {


            break


        } else if status == lexWaitingForInput {


            if send < len(packets) {


                fw.WriteString(packets[send])


                send += 1


            }


            status = fill(st)


            if status != lexReady {


                break


            }


        } else if status == lexPacketBroken {


            break


        }


    }


    // Check results.


    if status != expect || (status == lexEnd && recv != send) {


        panic(fmt.Sprintf("got %d, want %d", status, expect))


    }


    // Cleanup: remove input file.


    fr.Close()


    fw.Close()


    os.Remove(fname)
}
func main() {


    test(lexEnd, []string{})


    test(lexEnd, []string{"zero;", "one;", "two;", "three;", "four;"})


    test(lexPacketBroken, []string{"??;"})


    test(lexPacketTooBig, []string{"looooooooooooong;"})
}

//go:generate re2go $INPUT -o $OUTPUT --api simple
package main
// This example shows how to combine reusable re2c blocks: two blocks
// ('colors' and 'fish') are merged into one. The 'salmon' rule occurs
// in both blocks; the 'fish' block takes priority because it is used
// earlier. Default rule * occurs in all three blocks; the local (not
// inherited) definition takes priority.
const (


    Color = iota


    Fish


    Dunno
)
/*!rules:re2c:colors


    *                            { panic("eh!") }


    "red" | "salmon" | "magenta" { return Color }
*/
/*!rules:re2c:fish


    *                            { panic("oh!") }


    "haddock" | "salmon" | "eel" { return Fish }
*/
func lex(yyinput string) int {


    var yycursor, yymarker int


    /*!re2c


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        !use:fish;


        !use:colors;


        * { return Dunno }  // overrides inherited '*' rules


    */
}
func main() {


    assert_eq := func(x, y int) { if x != y { panic("error") } }


    assert_eq(lex("salmon"), Fish);


    assert_eq(lex("what?"), Dunno);
}

//go:generate re2go $INPUT -o $OUTPUT --input-encoding utf8 --api simple
package main
// This example supports multiple input encodings: UTF-8 and UTF-32.
// Both lexers are generated from the same rules block, and the use
// blocks add only encoding-specific configurations.
/*!rules:re2c


    re2c:yyfill:enable = 0;


    "∀x ∃y" { return 0; }


    *       { return 1; }
*/
func lexUTF8(yyinput []uint8) int {


    var yycursor, yymarker int


    /*!use:re2c


        re2c:encoding:utf8 = 1;


        re2c:YYCTYPE = uint8;


    */
}
func lexUTF32(yyinput []uint32) int {


    var yycursor, yymarker int


    /*!use:re2c


        re2c:encoding:utf32 = 1;


        re2c:YYCTYPE = uint32;


    */
}
func main() {


    assert_eq := func(x, y int) { if x != y { panic("error") } }


    assert_eq(lexUTF8([]uint8{0xe2, 0x88, 0x80, 0x78, 0x20, 0xe2, 0x88, 0x83, 0x79}), 0)


    assert_eq(lexUTF32([]uint32{0x2200, 0x78, 0x20, 0x2203, 0x79}), 0)
}

//go:generate re2go $INPUT -o $OUTPUT --api simple
package main
import "reflect"
type SemVer struct { major, minor, patch int }
func s2n(s string) int { // convert pre-parsed string to a number


    n := 0


    for _, c := range s { n = n*10 + int(c-'0') }


    return n
}
func parse(yyinput string) *SemVer {


    var yycursor, yymarker int


    // Final tag variables available in semantic action.


    /*!svars:re2c format = 'var @@ int;'; */


    // Intermediate tag variables used by the lexer (must be autogenerated).


    /*!stags:re2c format = 'var @@ int;'; */


    /*!re2c


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        re2c:tags = 1;


        num = [0-9]+;


        @t1 num @t2 "." @t3 num @t4 ("." @t5 num)? [\x00] {


            major := s2n(yyinput[t1:t2])


            minor := s2n(yyinput[t3:t4])


            patch := 0


            if t5 != -1 { patch = s2n(yyinput[t5:yycursor-1]) }


            return &SemVer{major, minor, patch}


        }


        * { return nil }


    */
}
func main() {


    assert_eq := func(x, y *SemVer) {


        if !reflect.DeepEqual(x, y) { panic("error") }


    }


    assert_eq(parse("23.34\000"), &SemVer{23, 34, 0})


    assert_eq(parse("1.2.9999\000"), &SemVer{1, 2, 9999})


    assert_eq(parse("1.a\000"), nil)
}

//go:generate re2go $INPUT -o $OUTPUT --tags
package main
import (


    "os"


    "reflect"


    "strings"
)
const BUFSIZE int = 4095
type Input struct {


    file     *os.File


    yyinput  []byte


    yycursor int


    yymarker int


    yylimit  int


    token    int


    // Intermediate tag variables must be part of the lexer state passed to YYFILL.


    // They don't correspond to tags and should be autogenerated by re2c.


    /*!stags:re2c format = "\t@@ int\n"; */


    eof      bool
}
type SemVer struct { major, minor, patch int }
func s2n(s []byte) int { // convert pre-parsed string to a number


    n := 0


    for _, c := range s { n = n*10 + int(c-'0') }


    return n
}
func fill(in *Input) int {


    if in.eof { return -1 } // unexpected EOF


    // Error: lexeme too long. In real life can reallocate a larger buffer.


    if in.token < 1 { return -2 }


    // Shift buffer contents (discard everything up to the current token).


    copy(in.yyinput[0:], in.yyinput[in.token:in.yylimit])


    in.yycursor -= in.token


    in.yymarker -= in.token


    in.yylimit -= in.token


    // Tag variables need to be shifted like other input positions. The check


    // for -1 is only needed if some tags are nested inside of alternative or


    // repetition, so that they can have -1 value.


    /*!stags:re2c format = "\tif in.@@ != -1 { in.@@ -= in.token }\n"; */


    in.token = 0


    // Fill free space at the end of buffer with new data from file.


    n, _ := in.file.Read(in.yyinput[in.yylimit:BUFSIZE])


    in.yylimit += n


    in.yyinput[in.yylimit] = 0


    // If read less than expected, this is the end of input.


    in.eof = in.yylimit < BUFSIZE


    return 0
}
func parse(in *Input) []SemVer {


    // Final tag variables available in semantic action.


    /*!svars:re2c format = "var @@ int;"; */


    vers := make([]SemVer, 0)


    for {


        in.token = in.yycursor


    /*!re2c


        re2c:api = record;


        re2c:eof = 0;


        re2c:yyrecord = in;


        re2c:YYCTYPE = byte;


        re2c:YYFILL = "fill(in) == 0";


        num = [0-9]+;


        num @t1 "." @t2 num @t3 ("." @t4 num)? [\n] {


            major := s2n(in.yyinput[in.token:t1])


            minor := s2n(in.yyinput[t2:t3])


            patch := 0


            if t4 != -1 { patch = s2n(in.yyinput[t4:in.yycursor-1]) }


            vers = append(vers, SemVer{major, minor, patch})


            continue


        }


        $ { return vers }


        * { return nil }


    */


    }
}
func main() () {


    fname := "input"


    content := "1.22.333\n";


    expect := make([]SemVer, 0, BUFSIZE)


    for i := 0; i < BUFSIZE; i += 1 { expect = append(expect, SemVer{1, 22, 333}) }


    // Prepare input file (make sure it exceeds buffer size).


    f, _ := os.Create(fname)


    f.WriteString(strings.Repeat(content, BUFSIZE))


    f.Seek(0, 0)


    // Initialize lexer state: all offsets are at the end of buffer.


    in := &Input{


        file:     f,


        // Sentinel at `yylimit` offset is set to zero, which triggers YYFILL.


        yyinput:  make([]byte, BUFSIZE+1),


        yycursor: BUFSIZE,


        yymarker: BUFSIZE,


        yylimit:  BUFSIZE,


        token:    BUFSIZE,


        eof:      false,


    }


    // Run the lexer and check results.


    if !reflect.DeepEqual(parse(in), expect) { panic("error"); }


    // Cleanup: remove input file.


    f.Close();


    os.Remove(fname);
}

//go:generate re2go $INPUT -o $OUTPUT --api simple
package main
import "reflect"
type SemVer struct { major, minor, patch int }
func s2n(s string) int { // convert pre-parsed string to a number


    n := 0


    for _, c := range s { n = n*10 + int(c-'0') }


    return n
}
func parse(yyinput string) *SemVer {


    var yycursor, yymarker int


    // Final tag variables used in semantic action.


    /*!svars:re2c format = 'var @@ int;'; */


    // Intermediate tag variables used by the lexer (must be autogenerated).


    /*!stags:re2c format = 'var @@ int;'; */


    /*!re2c


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        re2c:captvars = 1;


        num = [0-9]+;


        (num) "." (num) ("." num)? [\x00] {


            _ = yytl0; _ = yytr0; // some variables are unused


            major := s2n(yyinput[yytl1:yytr1])


            minor := s2n(yyinput[yytl2:yytr2])


            patch := 0


            if yytl3 != -1 { patch = s2n(yyinput[yytl3+1:yytr3]) }


            return &SemVer{major, minor, patch}


        }


        * { return nil }


    */
}
func main() {


    assert_eq := func(x, y *SemVer) {


        if !reflect.DeepEqual(x, y) { panic("error") }


    }


    assert_eq(parse("23.34\000"), &SemVer{23, 34, 0})


    assert_eq(parse("1.2.9999\000"), &SemVer{1, 2, 9999})


    assert_eq(parse("1.a\000"), nil)
}

//go:generate re2go $INPUT -o $OUTPUT --api simple
package main
import "reflect"
const (


    mtagRoot int = -1


    tagNone int = -1
)
// An m-tag tree is a way to store histories with an O(1) copy operation.
// Histories naturally form a tree, as they have common start and fork at some
// point. The tree is stored as an array of pairs (tag value, link to parent).
// An m-tag is represented with a single link in the tree (array index).
type mtagElem struct {


    elem int


    pred int
}
type mtagTrie = []mtagElem
type Ver = []int // unbounded number of version components
func s2n(s string) int { // convert pre-parsed string to a number


    n := 0


    for _, c := range s { n = n*10 + int(c-'0') }


    return n
}
// Append a single value to an m-tag history.
func add_mtag(trie *mtagTrie, mtag int, value int) int {


    *trie = append(*trie, mtagElem{value, mtag})


    return len(*trie) - 1
}
// Recursively unwind tag histories and collect version components.
func unwind(trie mtagTrie, x int, y int, str string) Ver {


    // Reached the root of the m-tag tree, stop recursion.


    if x == mtagRoot && y == mtagRoot {


        return []int{}


    }


    // Unwind history further.


    ver := unwind(trie, trie[x].pred, trie[y].pred, str)


    // Get tag values. Tag histories must have equal length.


    if x == mtagRoot || y == mtagRoot {


        panic("tag histories have different length")


    }


    ex := trie[x].elem


    ey := trie[y].elem


    if ex != tagNone && ey != tagNone {


        // Both tags are valid string indices, extract component.


        ver = append(ver, s2n(str[ex:ey]))


    } else if !(ex == tagNone && ey == tagNone) {


        panic("both tags should be tagNone")


    }


    return ver
}
func parse(yyinput string) []int {


    var yycursor, yymarker int


    trie := make([]mtagElem, 0)


    // Final tag variables available in semantic action.


    /*!svars:re2c format = 'var @@ int;'; */


    /*!mvars:re2c format = "var @@ int;"; */


    // Intermediate tag variables used by the lexer (must be autogenerated).


    /*!stags:re2c format = 'var @@ int'; separator = "\n\t"; */


    /*!mtags:re2c format = "\t@@ := mtagRoot\n"; */


    /*!re2c


        re2c:tags = 1;


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        re2c:YYMTAGP = "@@ = add_mtag(&trie, @@, yycursor)";


        re2c:YYMTAGN = "@@ = add_mtag(&trie, @@, tagNone)";


        num = [0-9]+;


        @t1 num @t2 ("." #t3 num #t4)* [\x00] {


            ver := make([]int, 0)


            ver = append(ver, s2n(yyinput[t1:t2]))


            ver = append(ver, unwind(trie, t3, t4, yyinput)...)


            return ver


        }


        * { return nil }


    */
}
func main() {


    assert_eq := func(x, y []int) {


        if !reflect.DeepEqual(x, y) { panic("error") }


    }


    assert_eq(parse("1\000"), []int{1})


    assert_eq(parse("1.2.3.4.5.6.7\000"), []int{1, 2, 3, 4, 5, 6, 7})


    assert_eq(parse("1.\000"), nil)
}

//go:generate re2go $INPUT -o $OUTPUT -8si --api simple
package main
/*!include:re2c "unicode_categories.re" */
func lex(yyinput string) int {


    var yycursor, yymarker int


    /*!re2c


        re2c:yyfill:enable = 0;


        re2c:YYCTYPE = byte;


        // Simplified "Unicode Identifier and Pattern Syntax"


        // (see https://unicode.org/reports/tr31)


        id_start    = L | Nl | [$_];


        id_continue = id_start | Mn | Mc | Nd | Pc | [\u200D\u05F3];


        identifier  = id_start id_continue*;


        identifier { return 0 }


        *          { return 1 }


    */
}
func main() {


    if lex("_Ыдентификатор\000") != 0 {


        panic("error")


    }
}

const (


    ResultOk = iota


    ResultFail
)
/*!re2c


    number = [1-9][0-9]*;
*/

// floating-point numbers
frac  = [0-9]* "." [0-9]+ | [0-9]+ ".";
exp   = 'e' [+-]? [0-9]+;
float = frac exp? | [0-9]+ exp;
float { return ResultOk }

//go:generate re2go $INPUT -o $OUTPUT -i --api simple
package main
/*!include:re2c "definitions.go" */
func lex(yyinput string) int {


    var yycursor, yymarker int


    /*!re2c


        re2c:YYCTYPE = byte;


        re2c:yyfill:enable = 0;


        *      { return ResultFail }


        number { return ResultOk }


        !include "extra_rules.re.inc";


    */
}
func main() {


    assert_eq := func(x, y int) { if x != y { panic("error") } }


    assert_eq(lex("123\000"), ResultOk)


    assert_eq(lex("123.4567\000"), ResultOk)
}

//go:generate re2go $INPUT -o $OUTPUT -i --header lexer/state.go
package main
import "./lexer" // the package is generated by re2c
/*!header:re2c:on*/
package lexer
type State struct {


    Data string


    Cur /*!stags:re2c format=", @@"; */ int
}
/*!header:re2c:off*/
func lex(yyrecord *lexer.State) int {


    var t int


    /*!re2c


        re2c:header = "lexer/state.go";


        re2c:api = record;


        re2c:YYCTYPE = byte;


        re2c:YYINPUT = "yyrecord.Data";


        re2c:YYCURSOR = "yyrecord.Cur";


        re2c:yyfill:enable = 0;


        re2c:tags = 1;


        re2c:tags:prefix = "Tag";


        [a]* @t [b]* { return t }


    */
}
func main() {


    st := &lexer.State{Data:"ab\x00",}


    if lex(st) != 1 {


        panic("error")


    }
}

// Code generated by re2c, DO NOT EDIT.
package lexer
type State struct {


    Data string


    Cur, Mar, Tag1 int
}