Backend development

Gregory Vinčić

Robert Griesemer, Ken Thompson and Rob Pike are the original authors of the Go language which is currently supervised by Russ Cox. Ian wrote the first compiler as a frontend to gcc. Read more on their (FAQ)

The story

  1960       1970          1980          1990        2000      2007-2009

                                                +- Javascript  --+
                                               /                  \
                +----- C -----------------+---+ PHP/Perl        ---+
               /                           \                        \
- Algol ------+                             +-- Java             ----+- Go
               \                                                    /
                \       +--- Modula ---+-- Python               -- +
                 \     /                                          /
                  +---+----------+         o-- Ada 95            /
                  |               \                             /
              Pascal               +---- Oberon             ---+

Download and install Go in the $HOME directory.

https://go.dev/dl

$ cd $HOME
$ wget https://go.dev/dl/go1.22.0.linux-amd64.tar.gz
$ tar xvfz go1.22.0.linux-amd64.tar.gz
$ export PATH="$PATH:$HOME/go/bin"
$ go version
go version go1.22.0 linux/amd64

$ ls -1 $HOME/go/bin
go
gofmt

For more information refer to "Download and install".

$ cd neon; tree/
.
├── go.mod
├── main.go
└── main_test.go

0 directories, 3 files

"A basic Go package has all its code in the project’s root directory."

go.dev/doc/modules/layout

$ cd neon/; tree
.
├── cmd
│   └── neon
│       ├── main.go
│       └── main_test.go
├── booking.go
├── go.mod
├── hotel.go
└── room.go

2 directories, 6 files

$ cd neon/; tree
.
├── cmd
│   └── neon
│       ├── booking.go
│       ├── hotel.go
│       ├── main.go
│       ├── main_test.go
│       └── room.go
├── internal
│   ├── crypt
│   ├── mqtt.go
│   └── postgres.go
└── go.mod

4 directories, 8 files

"Server project"

go.dev/doc/modules/layout#server-project

$ cd neon/; tree
.
├── cmd
│   └── neon
│       ├── main.go
│       └── main_test.go
├── internal
│   ├── crypt
│   ├── mqtt.go
│   └── postgres.go
├── booking.go
├── go.mod
├── hotel.go
├── room.go
└── system.go

4 directories, 9 files

"Don't waste a good name"

Gregory Vinčić

In Go; modules are used to group packages and functions.

"You can collect related packages into modules, then publish the modules for other developers to use."

go.dev/doc/modules/developing

An interface describes behavior.

"Interfaces in Go provide a way to specify the behavior of an object: if something can do this, then it can be used here."

go.dev/doc/effective_go#interfaces_and_types

Modules are initiated using the repository URL.

$ go mod init github.com/gregoryv/uptime

This can then be used

$ go get github.com/gregoryv/uptime[@VERSION]

Find modules on pkg.go.dev

Sharing code with others requires dependency management.

Module


package main

import (
    "fmt"
    "time"

    "github.com/gregoryv/uptime"
)

func main() {
    start := time.Now()
    dur := uptime.Since(start)
    fmt.Println(dur.String())
}

"... package name should be good: short, concise, evocative. ...Use the package structure to help you choose good names. "

go.dev/doc/effective_go#package-names

Package

"Interfaces with only one or two methods are common in Go code, and are usually given a name derived from the method, such as io.Writer for something that implements Write."

go.dev/doc/effective_go#interfaces_and_types


package fmt

type Stringer interface {
    String() string
}

Interface


func main() {
    fmt.Println(painted(color(2)))
}

func painted(val fmt.Stringer) string {
    return fmt.Sprintf("%T.String(): %s", val, val.String())
}

type color int

func (c color) String() string {
    switch c {
    case 1:
        return "red"
    case 2:
        return "green"
    default:
        return "white"
    }
}

Respond to HTTP requests in Go by implementing the http.Handler interface.


package http // import "net/http"

// A Handler responds to an HTTP request. Handler.ServeHTTP should
// write reply headers and data to the ResponseWriter and then return.
// ...
type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

// The HandlerFunc type is an adapter to allow the use of ordinary
// functions as HTTP handlers. If f is a function with the appropriate
// signature, HandlerFunc(f) is a Handler that calls f.
type HandlerFunc func(ResponseWriter, *Request)

func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
    f(w, r)
}


package main

import (
    "fmt"
    "log"
    "net/http"
)

func main() {
    err := http.ListenAndServe(":8080", http.HandlerFunc(sayHello))
    if err != nil {
        log.Fatal(err)
    }
}

func sayHello(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello, world!")
}

Direct design uses named functions as http handlers, implementing the http.HandlerFunc interface

Func

$ go run helloworld.go


package main

import (
    "fmt"
    "log"
    "net/http"
)

func main() {
    err := http.ListenAndServe(":8080", sayHello())
    if err != nil {
        log.Fatal(err)
    }
}

func sayHello() http.HandlerFunc {
    txt := "Hello, world!"
    return func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, txt)
    }
}

Closure design uses named functions returning a http.HandlerFunc.

Closure


func main() {
    handler := &Greeter{
        // complex setup
    }
    err := http.ListenAndServe(":8080", handler)
    if err != nil {
        log.Fatal(err)
    }
}

type Greeter struct {
    // complex relations, eg. database
}

func (g *Greeter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello, world!")
}

Method design declares types that implement http.Handler

Method


package main

import (
    "fmt"
    "log"
    "net/http"
)

func main() {
    ctl := Controller{
        // complex setup
    }

    if err := http.ListenAndServe(":8080", ctl.sayHello()); err != nil {
        log.Fatal(err)
    }
}

type Controller struct{}

func (c *Controller) sayHello() http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, "Hello, world!")
    }
}

Combo design declares types with multiple methods with http.HandlerFunc signature or return one (closure).

Combo

For systems serving many resources


package http

// ServeMux is an HTTP request multiplexer. It matches the URL of each
// incoming request against a list of registered patterns and calls
// the handler for the pattern that most closely matches the URL.
type ServeMux struct {
    // Has unexported fields.
}

multiplexer
      n 1: a device that can interleave two or more activities


func (c *Controller) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    switch {
    case r.URL.Path == "/bye" && r.Method == "GET":
        c.sayGoodbye(w, r)

    case r.URL.Path == "/" && r.Method == "GET":
        c.sayHello(w, r)

    default:
        http.Error(w, "routing failed", http.StatusBadRequest)
    }
}

Manual routing.

Error prone and complex.


func main() {
    var ctl Controller
    http.HandleFunc("GET /bye", ctl.sayGoodbye)
    http.HandleFunc("GET /", ctl.sayHello) // everything else
    if err := http.ListenAndServe(":8080", nil); err != nil {
        log.Fatal(err)
    }
}

type Controller struct{}

func (c *Controller) sayHello(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello, world!")
}

func (c *Controller) sayGoodbye(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Goodbye, world!")
}

Use http singleton ServeMux.


func main() {
    var ctl Controller

    mux := http.NewServeMux()
    mux.HandleFunc("GET /bye", ctl.sayGoodbye)
    mux.HandleFunc("GET /", ctl.sayHello)

    if err := http.ListenAndServe(":8080", mux); err != nil {
        log.Fatal(err)
    }
}

type Controller struct{}

func (c *Controller) sayHello(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello, world!")
}

func (c *Controller) sayGoodbye(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Goodbye, world!")
}

Create your own http.ServeMux.


package main

import (
    "encoding/json"
    "log"
    "net/http"
)

func main() {
    // available people, ie. our data store
    people := []Person{
        {Id: "p1", Name: "John"},
        {Id: "p2", Name: "Jane"},
    }
    mux := http.NewServeMux()
    mux.Handle("GET /person/{id}", servePersonById(people))

    if err := http.ListenAndServe(":8080", mux); err != nil {
        log.Fatal(err)
    }
}


func servePersonById(people []Person) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        id := r.PathValue("id")
        for _, person := range people {
            if person.Id == id {
                json.NewEncoder(w).Encode(person)
                return
            }
        }
        // ... handle not found
    }
}

type Person struct {
    Id   string
    Name string
}

Closure

From the dictionary

encode
  
    <algorithm, hardware> To convert {data} or some physical
    quantity into a given format.  E.g. {uuencode}.

In Go this equals to marshal type X to []byte


package main

import (
    "encoding/json"
    "fmt"
)

func main() {
    user := Contact{
        Firstname: "John",
        Lastname:  "Doe",
        Age:       23,
        Alive:     true,
    }
    data, _ := json.Marshal(user)
    fmt.Print(string(data))
}

type Contact struct {
    Firstname string
    Lastname  string
    Age       int
    Alive     bool
}

$ go run ./examples/encoding.go
{"Firstname":"John","Lastname":"Doe","Age":23,"Alive":true}


package main

import (
    "encoding/json"
    "fmt"
)

func main() {
    user := Contact{
        Firstname: "John",
        Lastname:  "Doe",
        Age:       23,
        Alive:     true,
    }
    data, _ := json.MarshalIndent(user, "", "  ")
    fmt.Print(string(data))
}

type Contact struct {
    Firstname string
    Lastname  string
    Age       int
    Alive     bool
}

$ go run ./examples/nicejson.go
{
  "Firstname": "John",
  "Lastname": "Doe",
  "Age": 23,
  "Alive": true
}


package main

import (
    "encoding/json"
    "fmt"
)

func main() {
    user := Contact{
        Firstname: "John",
        Lastname:  "Doe",
    }
    data, _ := json.MarshalIndent(user, "", "  ")
    fmt.Print(string(data))
}

type Contact struct {
    Firstname string `json:"firstname"` // change name
    Lastname  string `json:"-"`         // ignore this field
}

Control naming of fields, eg. lowercase names.

$ go run ./examples/fieldtags.go
{
  "firstname": "John"
}


type Contact struct {
    Firstname string `json:"firstname"` // change name
    Lastname  string `json:"-"`         // ignore this field
    *Address  `json:"address"`
}

type Address struct {
    Street     string `json:"street"`
    Number     uint   `json:"no"`                    // rename entirely
    PostalCode int    `json:"postal_code,omitempty"` // skip empty value
}

$ go run ./examples/complex_struct.go
{
  "firstname": "John",
  "address": {
    "street": "Lexington road",
    "no": 137
  }
}

main_test.go


package main

import "testing"

func TestSum(t *testing.T) {
    if s := Sum(1, 2); s != 3 {
        t.Errorf("got %v, expect 3", s)
    }
}

main.go


func Sum(a, b int) int {
    return a + b
}

.
├── go.mod
├── main.go
└── main_test.go

0 directories, 3 files

$ go test -v .
=== RUN   TestSum
--- PASS: TestSum (0.00s)
PASS
ok  	github.com/gregoryv/project	0.001s


package main

import (
    "net/http"
    "net/http/httptest"
    "testing"
)

func Test_sayHello(t *testing.T) {
    // prepare response recorder and request
    w := httptest.NewRecorder()
    r := httptest.NewRequest("GET", "/", http.NoBody)

    // call handler
    handler := sayHello() // using the closure design
    handler(w, r)

    // check result
    resp := w.Result()
    if v := resp.StatusCode; v != 200 {
        t.Error(resp.Status)
    }
}

Quick unit test of specific handlers using httptest.Recorder

setup
call handler
check response

$ go test -v -cover .
=== RUN   Test_sayHello
--- PASS: Test_sayHello (0.00s)
PASS
coverage: 60.0% of statements
ok  	goback/examples/httptest	0.002s	coverage: 60.0% of statements

main_test.go


func Test_sayHello(t *testing.T) {
    // prepare response recorder and request
    w := httptest.NewRecorder()
    r := httptest.NewRequest("GET", "/", http.NoBody)

    // call handler
    handler := sayHello() // using the closure design
    handler(w, r)

    // check result
    resp := w.Result()
    if v := resp.StatusCode; v != 200 {
        t.Error(resp.Status)
    }
}

main.go


func main() {
    if err := http.ListenAndServe(":8080", sayHello()); err != nil {
        log.Fatal(err)
    }
}

func sayHello() http.HandlerFunc {
    txt := "Hello, world!"
    return func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, txt)
    }
}

Serverside rendered HTML for several use cases

The obvious, you love the old school
API docs
Server stats
Performance, improve user experience and load times


package main

import (
    "html/template"
    "log"
    "os"
)

func main() {
    tpl, err := template.ParseFiles("examples/index.html")
    if err != nil {
        log.Fatal(err)
    }

    model := map[string]any{
        "Title": "<script>alert('WOW');</script>",
    }
    if err := tpl.Execute(os.Stdout, model); err != nil {
        log.Fatal(err)
    }
}


<html>
<body>

<h1>{{.Title}}</h1>

</body>
</html>

$ go run examples/htmltpl.go
<html>
<body>

<h1>&lt;script&gt;alert(&#39;WOW&#39;);&lt;/script&gt;</h1>

</body>
</html>


package main

import (
    "embed"
    "html/template"
    "log"
    "os"
)

func main() {
    model := map[string]any{
        "Title": "<script>alert('WOW');</script>",
    }
    err := tpl.ExecuteTemplate(os.Stdout, "index.html", model)
    if err != nil {
        log.Fatal(err)
    }
}

var tpl *template.Template = template.Must(
    template.ParseFS(assets, "assets/*.html"),
)

//go:embed assets/*.html
var assets embed.FS

"Package embed provides access to files embedded in the running Go program."

pkg.go.dev/embed

$ go run examples/embed.go
<html>
<body>

<h1>&lt;script&gt;alert(&#39;WOW&#39;);&lt;/script&gt;</h1>

</body>
</html>

main.go


func main() {
    h := &Hotel{
        Name: "Purple glow",
        Rooms: []Room{
            {Number: "1"},
            {Number: "2"},
            {Number: "3"},
        },
    }

    mux := http.NewServeMux()
    mux.Handle("POST /room/{number}", BookRoom(h.Rooms))
    mux.Handle("GET /room", ServeRooms(h.Rooms))
    mux.Handle("GET /", ServeHotel(h))

    if err := http.ListenAndServe(":8080", mux); err != nil {
        log.Fatal(err)
    }
}

type Hotel struct {
    Name  string `json:"name"`
    Rooms []Room `json:"-"` // rooms are rendered separately
}

type Room struct {
    Number string `json:"number"` // using string for simplicty
    Booked bool   `json:"booked"`
}

main.go (continued)


func BookRoom(rooms []Room) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        number := r.PathValue("number")
        // update room state
        for i, room := range rooms {
            if room.Number != number {
                continue
            }
            rooms[i].Booked = true
        }
        json.NewEncoder(w).Encode(rooms)
    }
}

func ServeRooms(rooms []Room) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        json.NewEncoder(w).Encode(rooms)
    }
}

func ServeHotel(h *Hotel) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        json.NewEncoder(w).Encode(h)
    }
}

$ cd neon/; tree
.
├── go.mod
├── main.go
└── main_test.go

0 directories, 3 files

main_test.go


func Test_main(t *testing.T) {
    go main()
    <-time.After(time.Millisecond) // bad test strategy

    cases := []struct {
        exp int // expected status code
        *http.Request
    }{
        {200, newRequest(t, "POST", "http://localhost:8080/room/1")},
        {200, newRequest(t, "GET", "http://localhost:8080/room")},
        {200, newRequest(t, "GET", "http://localhost:8080/")},
    }
    for _, c := range cases {
        resp, err := http.DefaultClient.Do(c.Request)
        if err != nil {
            t.Fatal(err)
        }
        if resp.StatusCode != c.exp {
            t.Error(resp.Status)
        }
    }
}

func newRequest(t *testing.T, method, path string) *http.Request {
    r, err := http.NewRequest(method, path, http.NoBody)
    if err != nil {
        t.Fatal(err)
    }
    return r
}

$ go test -v -cover .
=== RUN   Test_main
--- PASS: Test_main (0.00s)
PASS
coverage: 94.4% of statements
ok      github.com/   0.006s  [0m...

Concepts covered in this presentation

Go language/structure
module	for dependency management
package	grouping related code
interface	behavior abstraction
Package
net/http	handling requests and routing
net/http/httptest	utilities for HTTP testing
testing	automated testing of Go packages
encoding/json	marshaling datatypes to JSON
html/template	rendering HTML
embed	provides access to embedded files

Backend development

Go concepts
Project structure
HTTP routing
Handler design
Test handlers
JSON Encoding

General problems

Test driven development
Package testdata
Documentation
Logging
Error handling
Database IO

Concurrency problems

Concurrency
Benchmark
Testing strategy
... more

Backend development