Language

A Parser is a program that reads the tokens and converts it into an abstract syntax tree (AST). It is the second part of an interpreter. What the usage looks like // for an input `let five = 5;` input := `let five = 5;` l := lexer.New(input) p := parser.New(l) // result is a list of statements result := []ast.Statement{ &ast.LetStatement{ Token: token.Token{Type: token.LET, Literal: "let"}, Name: &ast.Identifier{ Token: token.Token{Type: token.IDENT, Literal: "five"}, Value: "five", }, Value: &ast.IntegerLiteral{ Token: token.Token{Type: token.INT, Literal: "5"}, Value: 5, }, }, } input2 := "1 + 2 + 3;" l2 := lexer.New(input2) p2 := parser.New(l2) result2 := []ast.Statement{ &ast.ExpressionStatement{ Token: token.Token{Type: token.INT, Literal: "1"}, Expression: &ast.InfixExpression{ Token: token.Token{Type: token.PLUS, Literal: "+"}, Left: &ast.IntegerLiteral{ Token: token.Token{Type: token.INT, Literal: "1"}, Value: 1, }, Operator: "+", Right: &ast.InfixExpression{ Token: token.Token{Type: token.PLUS, Literal: "+"}, Left: &ast.IntegerLiteral{ Token: token.Token{Type: token.INT, Literal: "2"}, Value: 2, }, Operator: "+", Right: &ast.IntegerLiteral{ Token: token.Token{Type: token.INT, Literal: "3"}, Value: 3, }, }, }, }, } The parser also needs to handle operator precedence and associativity. It is a complex part of an interpreter. ...

A Lexer is a program that reads the code and converts it into tokens. It is the first part of an interpreter. What the usage looks like // for an input `let five = 5;` input := `let five = 5;` l := lexer.New(input) // result is a list of tokens result := []tokens.Token{ {Type: tokens.LET, Literal: "let"}, {Type: tokens.IDENT, Literal: "five"}, {Type: tokens.ASSIGN, Literal: "="}, {Type: tokens.INT, Literal: "5"}, {Type: tokens.SEMICOLON, Literal: ";"}, {Type: tokens.EOF, Literal: ""}, } input2 := "1 + 2 + 3;" l2 := lexer.New(input2) result2 := []tokens.Token{ {Type: tokens.INT, Literal: "1"}, {Type: tokens.PLUS, Literal: "+"}, {Type: tokens.INT, Literal: "2"}, {Type: tokens.PLUS, Literal: "+"}, {Type: tokens.INT, Literal: "3"}, {Type: tokens.SEMICOLON, Literal: ";"}, {Type: tokens.EOF, Literal: ""}, } Let’s see how to implement a sample lexer in Go. ...

An interpreter is a program that reads and executes code. It is a way to execute code without compiling it. It is a great way to execute code on the fly. It is used in many places like scripting languages, command line tools, etc. Parts of an Interpreter ( Simplified ) A minimal interpreter can be represented in 3 parts: lexer: It reads the code and converts it into tokens. parser: It reads the tokens and converts it into an abstract syntax tree (AST). evaluator: It reads the AST and executes the code. Note: There can be more parts like optimizer, IR generator etc. for advanced interpreters. ...

Grammer in computer science is a set of rules that defines the structure of a language. It defines a language in a structured way and parsers are built based on it. BNF Backus-Naur Form, or BNF, is a notation technique used to express context-free grammars. It is used to define syntax of programming languages, datastructures, etc. EBNF Extended Backus-Naur Form, or EBNF, is a notation technique used to express context-free grammars. It is an extension of BNF. ...