go-training.slide

Learning go
January 25th, 2024
Tags: go golang xebia

Patrick Akil
Software engineer, Xebia

http://www.xebia.com/

pakil@xebia.com
https://github.com/PatAkil/
https://www.linkedin.com/in/patrick-akil-721b07105/

#----------------------------------------------

* About me
.image examples/images/microphone.JPG  305 456
- Fitness, Podcasting, Content creation

- IoT, Ecommerce, Health Tech
- Effectiveness: organisational, team and personal
- Running golang in production @Wavin, @Duxxie & more!

#----------------------------------------------

* About you
- Who are you?
- What are your expectations and goals?

#----------------------------------------------

* Your experience

- What programming languages?
- Go installed on your laptop?
- Written any go code?
- Completed "A tour of go"?
- Completed go tutorials? Read a book?
- Used go tool-chain? go get, go build, go install?
- json/xml marshalling?
- Built an http-client and http-server?
- Used channels and goroutines?
- Unit testing?
- Own open-source project? Production code?

#----------------------------------------------

* Housekeeping

#----------------------------------------------

* Overall agenda

Past

- Startup
- Basics (A Tour of Go)

Now

- Interfaces
- More on functions
- Testing
- Concurrency
- Generics
- & More!

#----------------------------------------------

* Approach

- Learn by example
- See code, run code
- Associate with something you already know
- Questions
- Feedback

#----------------------------------------------
* Language
#----------------------------------------------

* Why go was created?

- Help Google solve problems: handle web requests
- Simple: limited -> easy to read
- Scale: infrastructure, teams, code bases
- Compile, startup and run fast
- Safe and reliable
- Optimum between C++ and Python

Also works for us

.link https://bradfitz.com/2020/01/30/joining-tailscale

#----------------------------------------------

* Similarities with Java (or Kotlin)

- General purpose
- Curly-braced
- Typestrong
- Compiled
- Garbage collected
- Object oriented
- Performant

#----------------------------------------------

* Differences with Java

Missing in go:

- Constructors (but uses "constructor" functions)
- Inheritance (but has "embedding" to compose)
- Exceptions
- Annotations
- Mutability

Missing in java:

- Rich standard library and toolchain
- Built-in concurrency

#----------------------------------------------

* Sweetspot for go

- Serving the web
- Components that need to be fast and reliable
- High throughput and concurrency
- Serve spiky traffic (fast startup eases scaling up)
- Large scale (people, lifetime and size)

Guaranteed backwards compatability 🙌

#----------------------------------------------

* Summary of the language

- Keywords:

    break case continue default defer else (fallthrough) for (goto) if return switch
    package import func const interface map range struct type chan go select var

- Types:

	error string rune bool byte int int8 int16 int32 int64 float32 float64
	uint uint8 uint16 uint32 uint64 (uintptr) (complex64) (complex128)

- Constants:

	true false iota

- Zero value:

	nil

- Functions:

    append cap close complex copy delete len make new (panic) (recover)

#----------------------------------------------

* Useful sources of information

.link https://go.dev

.link https://go.dev/tour

.link https://go.dev/doc/effective_go

.link https://golangweekly.com/ golangweekly.com/ (newsletter to keep up)

.link https://github.com/avelino/awesome-go

#----------------------------------------------

* Getting our hands dirty

#----------------------------------------------

#----------------------------------------------

#* Installation git and golang

#----------------------------------------------

#- Install git:

#.link https://git-scm.com/download

#    $ brew install git               # on mac
#    $ from download                  # on windows

#- Install go:

#.link https://golang.org/doc/install#download

#    $ brew install go              # on mac
#    $ sudo apt-get install golang  # on linux
#    $ from msi file to c:\Go       # on windows


#----------------------------------------------

* Verify installation

#----------------------------------------------

Verify git:

    $ git --version

Verify go:

    $ go version

At least version go1.18

No output means something's missing!

#----------------------------------------------

#* More setup (macos, unix)

#- Make sure the directory of the "go"-executable is in the PATH env-var

#    $ echo $PATH

#    $ PATH=${PATH}:/usr/local/bin         # in ~/.bash_profile

#- Make the directory of your self-made executables is in the PATH env-var

#    $ export PATH=${PATH}:${HOME}/go/bin   # in ~/.bash_profile

#----------------------------------------------

* Get the training material on your laptop

    $ cd ${HOME}             # on unix/mac
    $ cd /d %USERPROFILE%    # on windows

    $ git clone https://github.com/xebia/go-training.git

Everything will end up in go-training:

    go-training
    ├── go-training.slide
    ├── examples/
    └── solutions/

Switch to this dir:

    $ cd go-training

And build and install the training project

    $ go install ./...    # Results end up in ${HOME}/go/...


#----------------------------------------------

* Dev tools

- IntelliJ IDEA or Goland
- unix terminal or windows command
- git

PRO TIP:

- Format-on-save: Use "filewatcher"-plugin

# Use "filewatcher"-plugin with "goimports"

#    $ go get -u golang.org/x/tools/cmd/goimports


#----------------------------------------------

* Run the interactive presentation locally

As described in the `goPresent.md`

    $ go get golang.org/x/tools/cmd/present
    $ go run golang.org/x/tools/cmd/present -http=:3999 -use_playground=true

The presentation should be available in your browser at:

.link http://127.0.0.1:3999


#----------------------------------------------

* First program

Note that many examples in the presentation are editable and executable (Run!)

.play -edit examples/first/first.go

Start using the go toolchain:

    $ go fmt             # standard formatter (goimports is even better)

    $ go run first.go    # compiles and runs right away
    $ go build           # creates executable "./first" or ./first.exe in .

    $ go install         # creates executable "first" in ${HOME}/go/bin
    $ first              # or ${HOME}/go/bin/first
    Hi everybody!


#----------------------------------------------

* Exercise 1: Write, build and run your first program

Prepare dedicated directory for exercises:

    $ mkdir -p exercises            # base directory for all exercises
    $ cd exercises

    $ mkdir -p first                # dir for this exercise
    $ cd first

Tasks:

- Create a file first.go
- Write
- Compile
- Format
- Install
- Run

    Peek: examples/first/first.go

#----------------------------------------------

* Where do the executables end up?

- "Workspace"


    ${HOME}/go           # on unix
    %HOMEPATH%/go        # on windows
    $env:GOPATH          # on powershell

- Contains all go stuff

Convention:

    ${HOME}/go
        ├── pkg/   # libraries
        └── bin/   # executables

- The toolchain uses these conventions

#NB: We are no longer using `GOPATH`

#----------------------------------------------

* Language basics

#----------------------------------------------

* Creating packages

- Group related stuff
- Each package in dedicated directory
- More coarse-grained than Java: can contain  multiple files

- Package name first line of source file

     package main // package that results in executable with same name as package

or

     package news // package that results in library that is accessible via 'news'

NB:

- Minimize exported surface
- Make it short and sweet: fmt, http, log, ..

#----------------------------------------------

* Using other packages <packageusage>

- Import
- Use package-name as prefix

.play -edit examples/packageusage/main.go

#----------------------------------------------

* Comments

    /* a comment */  and // another one

Document your packages:

- Package level comment
- Every exported (capitalized) name in a program should have a comment

Verify documentation: should be a good summary

    go doc -all

Enforce rules:

    golint

- helps you minimize your public exports


#----------------------------------------------
* Commenting example <patientstore>

.play -edit examples/patientstore/patientstore.go


#----------------------------------------------

* Variables (vars)

- Name and type swapped (from Java perspective)
- Have reasonable defaults (not nil)

.play -edit examples/vars/vars.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Loops <for,while,iterate>
- for
.play -edit examples/for/for.go /START OMIT/,/END OMIT/
- while-like
.play -edit examples/while/while.go /START OMIT/,/END OMIT/
- iterate
.play -edit examples/iterate/iterate.go /START OMIT/,/END OMIT/

#----------------------------------------------

* If, else <if>
.play -edit examples/if/if.go

#----------------------------------------------

* Switch <switching>
- On any type
- No fallthrough unless explicitly stated
- Multiple cases as comma-separated list
.code -edit examples/switching/switch.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Exercise 2: Play around with control flow

Use dedicated directory for this exercise:

    $ exercises/flow

Tasks:

- Sum of all values from 1 up to 100
- Sum incremental values untill their sum exceeds 1000, return number of loops

    Peek: solutions/flow

#----------------------------------------------

* Functions
- Core building block
- Scope: based on case
- Java: static methods

 func ConvertIt( arg int ) string { // public
     return convertInternal( arg )
 }

 func convertInternal( arg int ) string { // internal, private function
     return fmt.Sprintf( "My integer value as string: %d", arg)
 }

- Can return multiple values

 func swap(x, y string) (string, string) { 
    return y, x 
 }

(More on functions later)


#----------------------------------------------

* Exercise 3: Move function to dedicated library

Continue with previous exercise:

    $ exercises/flow

Tasks:

- Put calculation logic in separate library

    Peek: solutions/flow


#----------------------------------------------

* Error handling

- Multiple return values
- if error is nil, the call worked

  resp, err := doSomethingThatCanFail(arg1, arg2)
  if err != nil {
      return fmt.Errorf("Error doing something that can fail: %s", err) // early return to minimize indentation
  }
  // continue with success path

  // use _ (=blank) if you don't care
  resp, _ := doit(arg1, arg2)

- Function signature tells that things can go wrong
- Keep indentation low

!!! All your own API's should use this pattern !!!
Do not use panic and recover

#----------------------------------------------

* Exercise 4: Idiomatic error-handling

Use dedicated directory for this exercise:

    $ exercises/errorhandling

Tasks:

- Read a file
- Capitalize the content of the file
- Write this capitalized content to a different new file
- Use Proper error-handling
- Advanced: Read as json and write as xml

TIP:

- File access using the "io/ioutil"-package ("os" nowadays)
- Capitalize using "strings" or "bytes"-package

    Peek: solutions/errorhandling

#----------------------------------------------

* Data

#----------------------------------------------

* Struct <struct>

- Equivalent to Java class
- Case of variable determines accessibility (private, public)

.play -edit examples/struct/struct.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Initializing structs

    student := Student{
            FullName:    "Freek Grol",
            AddressLine: "...., De Bilt, Nederland",
            Study: Study{
                Name: "Geography",
            },
            ProgressYear: 3,
    }

or

    student := Student{} // Default values for all fields

or

    student := Student{}
    student.FullName = "Freek Grol"
    student.AddressLine = "..."



#----------------------------------------------

* Pointers

- Pointer to memory where object resides

    var intPointer *int = new(int)
    *intPointer = 9

    var patient *Patient = &Patient{}
    patient2 := &Patient{}
    var patient *Patient    // nil (default value)

As parameters

    func adjust( a *int ) {
        *a = 9
    }

    func dosomethingWithSideEffects( p *Patient ) {
        p.Age = 42
    }

    p := Patient{}
    dosomethingWithSideEffects(&p)



#----------------------------------------------

* Why pointers?

- For methods that mutate data

  func MarkDeceasedWrong(p Patient) { // patient NOT adjusted after function returns
    p.Deceased = true
  }
  func MarkDeceased(p *Patient) { // patient adjusted after ..
    p.Deceased = true
  }

- For passing around huge structs

  func ProcessRontgenImage ( img *Image ) { ... }

- Indicate Optional (poor mans)

  type Person struct {
    Name string
    Car  *Car // optional
  }

#----------------------------------------------

* Initializing pointers

    student := &Student{            // NOTE: starts with '&'
            FullName:    "...   ",
            AddressLine: "...., De Bilt, Nederland",
    }

or

    func NewStudent(fullName, adddressLine string,
                    birthdate time.Time,
                    studyName string, progressYear int) *Student { // NOTE: returns *
        return &Student{
            FullName:    fullName,
            AddressLine: adddressLine,
        }
    }

or

    student := new(Student)
    student.FullName = "Freek Grol"
    student.AddressLine = "..."

#----------------------------------------------

* Struct methods <methods>

.play -edit examples/methods/methods.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Enumerations <enum>

.play -edit examples/enum/enum.go /START OMIT/,/END OMIT/

#----------------------------------------------


#----------------------------------------------

* Slices (more on it later)

- Built-in (Generics!)
- Equivalent of an array/list

.play -edit solutions/model/model.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Exercise 5: Model your business domain

Use dedicated directory for this exercise:

    $ exercises/model

.image examples/images/uml.png

Tasks:

- Use structs, enums, pointers
- Use slices

    Peek: solutions/model

#----------------------------------------------

* Containers

- array and slice
- map

#----------------------------------------------

* Slices

- Can contain everything: primitives, structs, slices, maps etc
- Like Java ArrayList
- Sortable
- Supported operations:

    append, replace, get-on-idx, get-on-range, iterate

Fixed length (not extendable in size)

    numbers := []int{10, 20, 30, 40}
    s := [...]string{"Voetbal", "Hockey"} // idiomatic: let compiler count

Dynamic size (start small, auto extend)

    var slice0 []string = []string{}   // empty
    slice1 := []string{}               // empty
    slice2 := []string{"a", "b", "c"}  // initialize with data
    slice3 := make( []string, 0, 5 )   // optimization: empty with reserved capacity

#----------------------------------------------

* Slices in action <slice>

.play -edit examples/slice/slice.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Always keep result of append

    // wrong
    append( letters, "e" )

    // right
    letters = append(letters, "e")

Why?

- Realloc when no longer fits
- Returns new pointer

#----------------------------------------------
* Anonymous slice

- For one-time use only

.play -edit examples/anonymous/main.go /START OMIT/,/END OMIT/

- Useful for table driven tests

#----------------------------------------------

* Map
- Store key-value pairs (like Java HashMap)
- Typically key is primitive, value can be everything: primitives, structs, slices, maps etc
- Supported operations:

    get-on-key, put-on-key, replace-on-key, delete-on-key, iterate

initialization:

    var m1 map[string]int = make(map[string]int)
    m2 := make(map[string]int)
    m3 := map[string]int{}
    m4 := map[string]int{
         "route": 66,
    }

- Random iteration order

#----------------------------------------------
* Maps in action <maps>

.play -edit examples/maps/maps.go /START OMIT/,/END OMIT/


#----------------------------------------------

* Exercise: slices and maps

Use dedicated directory for this exercise:

    $ exercises/slicesmaps

Task:

- Use maps and slices to group the following people on hobby:

    "Julia":  "voetbal", "tekenen"
    "Sophie": "hockey"
    "Mila":   "tekenen"
    "Emma":   "volleybal", "turnen"
    "Tess":   "hardlopen"
    "Zoë":    "kunst", "Voetbal"
    "Noor":   "voetbal"
    "Elin":   "Hockey"
    "Sara":   "voetbal", "turnen"
    "Yara":   "tekenen"

TIP: Copy and paste and use editor in column-mode

    Peek: solutions/groupby

#----------------------------------------------
* Embedding
#----------------------------------------------

* Embedding

- Inheritance or composition?

.play -edit examples/sync/sync.go /START OMIT/,/END OMIT/

#----------------------------------------------
* Interfaces
#----------------------------------------------

* Interfaces
- Duck-typing: no explicit "implements"
- Very useful to improve testability (dependency injection)

example from stdlib

   package fmt;

   // Accepts anything that implements the "Writer"-interface:
   // Examples of Writers: file, buffer, stdout, network, http-response, zip-file etc
   func Fprintf(w io.Writer, format string, a ...interface{}) (n int, err error) { ... }

io.Writer-interface:

    type Writer interface {
        Write(p []byte) (n int, err error)
    }
#other example

#  // The business logic layer can return this EnrichedError as regular 'error' .
#   // The http layer convert this error into an appropriate http-response (200, 400, 403, 500 etc)
#   type EnrichedError struct {
#        Kind ErrorKind // invalid-input, not-authorized, internal-error etc
#        Message string
#   }
#   func (e HttpError)Error() string { // implement Error-interface
#      return e.Message
#   }
#   func GetHttpStatus(err error) int { ... }

# Refactor using interfaces when initially done

#----------------------------------------------

* Interface <interfaces>
- Naming convention: ends with "er"
- Keep them small. Why?
- Composable

Typestrong variant

    type Datastorer interface {
        Put(patientUid string, patient Patient) error
        Get(patientUid string) (Patient, bool, error)
        Remove(patientUid string) error
    }

Composable variant

    package db

    type Service interface {
        UserService
        DeviceService
    }

#----------------------------------------------
* Example usage of interface <interfaces>

- For dependency injection
- The business logic of PatientService is testable without a "real"-datastore

.code -edit examples/interfaces/main.go  /START OMIT/,/END OMIT/

#----------------------------------------------
* Any type

    any (since 1.18)

Empty interface in the past:

    interface{}

- Accepts anything
- NOT typestrong
- Like Java Object or void* in C

.play -edit  examples/emptyinterface/main.go /START OMIT/,/END OMIT/

- Used by standard json and xml packages to (un) serialize

#----------------------------------------------

* Exercise: interfaces

Use dedicated directory for this exercise:

    $ exercises/interfaces

Tasks:

- Implement a simple in-memory database that implements the following interface:

    type Datastorer interface {
    	Put(key string, value interface{}) error
    	Get(key string) (interface{}, bool, error)
    	Remove(key string) error
    }

TIP: Create a constructor function ans let your IDE do the magic
TIP: Use a map as in-memory datastore

    Peek: examples/interfaces


#----------------------------------------------
* More on functions
#----------------------------------------------

* Package initialisation
- Executed only once at startup of package

  func init() {
      rand.Seed(time.Now().UnixNano())
  }

- For "const"-like struct initialisations

  var knownMembers []Member

  func init() {
      knownMembers = []Member{
        { ... },
        { ... },
        { ... },
      }
  }

- Can be multiple, however the order of execution is random. So only use once


#----------------------------------------------

* Defer <defer>

- Guaranteed to be called when function returns
- Can be multiple, last defer is executed first
- Cleanup of file-handles, mutexes, channels and connections
- Debugging: log "enter" and "leave" of function
- Unit-testing: "setup" and "teardown"

.code -edit examples/defer/unlock.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Variadic functions <variadic>
.play -edit examples/variadic/variadic.go

#----------------------------------------------

* Closures <functionsAsParams>

- Lambda's in Java
- Functions as variables, input and output-parameters

.code -edit examples/functionsAsParams/main.go /START OMIT/,/END OMIT/


#----------------------------------------------
* Function as input parameter: delegate

.play -edit examples/transactional/main.go /START OMIT/,/END OMIT/

#----------------------------------------------
* Function as output parameter: interceptor, proxy

.code -edit examples/proxy/proxy.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Function as output parameter: factory

- Pass data into handlers without global data

    func main() {
      db := NewDatabase("localhost:5432")

      http.HandleFunc("/hello", hello(db))
      http.ListenAndServe(":3000", nil)
    }

    func hello(db Database) func(http.ResponseWriter, *http.Request) {
      return func(w http.ResponseWriter, r *http.Request) {
        // has access to db without using globals vars
        ...
      }
    }

#----------------------------------------------

* More examples usages of closures <fireforget>

Combine with go routines to execute work in background

.play -edit examples/fireforget/main.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Multi-line defers

     func doit() {
        defer func() {
            // code to
            //    cleanup
            //       resource
            // with error-handling an logging
        }()
        ....
     }

#----------------------------------------------

* Cleanup

    func NewGrpcClient(addressPort string) (NotificationClient, func(), error) {
        // Prepare connection to the server.
        conn, err := grpc.Dial(addressPort, grpc.WithInsecure(), grpc.WithBlock())
        if err != nil {
            return nil, nil , fmt.Errorf("Error creating notifapi-grpc-client: %v", err)
        }
        cleanup := func() {
            if conn != nil {
                conn.Close()
            }
        }
        return NewNotificationClient(conn), cleanup, nil
    }

    func doit() error {
        client, cleanup, err := NewGrpcClient(":16123")
        if err != nil {
            return err
        }
        defer cleanup()
    .....

#----------------------------------------------

#* Overridable functions <emailapi>
#- Interfaces more suitable for overriding however you might encounter this

#.code -edit examples/emailapi/api.go

#- Can be useful as alternative for dependency injection

#----------------------------------------------

* Exercise: Practice with advanced functions

Use dedicated directory for this exercise:

    $ exercises/moreonfunctions

Tasks:

- Sort on age ascending

.code examples/sorting/main.go  /START OMIT/,/END OMIT/

    Peek: examples/sorting/main.go (use package "sort")

#----------------------------------------------
* Testing
#----------------------------------------------

* Testing

Essential for software with a long predicted lifetime

Why:

- Need “Safety-net" so you dare to keep improving and extending your software

How:

- Do from beginning
- Tests should be easy and fast to run
- Test against the API: not against the internals
- Prefer HTTP (as the ultimate API) to trigger your business logic, so you have freedom to change the internals and still keep your safety net intact


#----------------------------------------------

* Unit testing <testit>
- Part of toolchain
- In same package, dedicated file
- Filename convention: `<lalala>_test.go`

.code examples/testit/reverse_test.go /START OMIT/,/END OMIT/

    $ go test
    --- FAIL: TestReverseAscii (0.00s)
        reverse_test.go:13: ERROR : Expecting[marc] Received[cram]
    FAIL
    exit status 1

#----------------------------------------------
* testify package

Widely adopted package which includes common assertions

To start using the package:

    go get github.com/stretchr/testify/assert/...


Import package:

    import "github.com/stretchr/testify/assert"

Use in test:

    ....
    sut := NewAppointmentStore(NewBasicUuider())
    appointment, found, err := sut.GetAppointmentOnUid(c, "a")
    assert.NoError(t, err)
    assert.True(t, found)
    assert.Equal(t, "a", appointment.AppointmentUID)
    ....


#----------------------------------------------
* Exercise unit test

Create a new directory in the exercises directory

    exercises/directorysplitter

Develop a package that provides a directorysplitter that converts "a/b/c" into "a", "b" and "c". Tasks:

- Create unit tests that verifies correct behavior of splitter

   Consider using package "github.com/stretchr/testify/assert"

- Run unit test and observe failure

   go test

- Implement a string splitter (TIP: use package strings.Split)

- Run unit test and observe  success

  Peek: solutions/directorysplitter/directorysplitter_test.go

#----------------------------------------------
* Table driven test (when in Rome, do as ....)

- Very readable and easy to extend

.code solutions/tests/validateemail_test.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Example usage

Test service and verify if different input are validated correctly

- input: request-body (as struct or as json)
- expected output: http-response-code (and response message+code)

#----------------------------------------------
* Exercise table driven test

Continue with previous example

    exercises/directorysplitter

Tasks:

- Implement a string-splitter with the following signature

    Split( input string, seperator string ) []string

- Create a table driven test to verify the following situations

    input       seperator       expected-output
    --------    ---------       ------------------------
    "a/b/c"      "/"            []string{"a", "b", "c"}
    "a/b/c"      ","            []string{"a/b/c"}
    "abc"        "/"            []string{"abc"}


- TIP: your IDE can create the boilerplate for a table driven test

   Peek: solutions/tests/validateemail_test.go

#----------------------------------------------

* Design for testability

#----------------------------------------------

* Example 1: Send sms with pin-code

Situation:

- To finalize the registration process a patient is challenged with pin-code in an sms to verify its phone-number.

Goal:

- Verify that the sms is sent and that it has the right phone-number and correct content

Problem:

- Correct sms sending in unit-tests would be hard to prove and expensive

Solution:

- Use fake/mock sms-sender in unit-tests and the real sms-sender in production

#----------------------------------------------

* Example 2: Interact with datastore

Situation:

- The registration service relies on a third-party datastore to store and fetch its patients

Goal:

- Verify correct interaction with datastore in unit tests

Problem:

- Setting a dedicated database for each unit-test of each developer is slow and expensive

Solution:

- Use fake datastore in unit-tests and the real one in production

#----------------------------------------------
* How to fake/mock an sms-sender or datastore?

- Dependency injection (like in java)
- (Overridable function with default implementation)
- ...

#----------------------------------------------
* Dependency injection

Distinguish 3 interfaces    :

.code solutions/registrationService/externalApis.go  /START OMIT/,/END OMIT/

Inject these dependencies as parameters into the constructor of the registration     service:

Consequences:

- Registration service only depends on interface
- Can use "mock" version during test and real ones for production

#----------------------------------------------
* Example implementation

.code solutions/registrationService/service.go  /START OMIT/,/END OMIT/

#----------------------------------------------
* Mocks and stubs

- Create mocks by hand, or use testing framework


    package dbtestdata

    import (
        "context"

        "github.com/patakil/myproject/internal/db"
    )

    // enforces the type to implement the interface methods
    var _ db.UserService = (*MockUserService)(nil)

    type MockUserService struct {
        CreateUserFunc func(ctx context.Context, user db.User) (db.User, error)
        GetUserFunc    func(ctx context.Context, userID string) (db.User, error)
        DeleteUserFunc func(ctx context.Context, userID string) error

        UpsertUsersFunc       func(ctx context.Context, users ...db.User) error
    }

#----------------------------------------------
* Testing Tips
- Mocks and stubs could conventionally be placed in a `testdata` directory
- The `testdata` gets ignored by the go tool, see `go help packages` on the terminal
- Service suites can make your testing life easier

    import (
    	"context"
    	"testing"

    	dbtestdata "github.com/patakil/myproject/internal/db/testdata" // alias
    	"github.com/stretchr/testify/suite"
    )

    type serviceSuite struct {
    	suite.Suite
    	db dbtestdata.MockUserService
    }

    func TestService(t *testing.T) {
    	suite.Run(t, new(serviceSuite))
    }
    func (s *serviceSuite) SetupSuite() {
    // set up your mock implementations here


#----------------------------------------------
* Exercise dependency injection

Create a new directory in the exercises directory

    exercises/maketestable

Tasks:

- Write unit tests for the package

     examples/nontestable

- TIP: Requires  some serious refactoring

    Peek: solutions/testable

#----------------------------------------------

* Benchmarking <benchmark>
- Premature optimization is ...
- Never optimize before measuring first

.code examples/benchmark/mycalclib_test.go /START OMIT/,/END OMIT/

- On profiling: https://www.youtube.com/watch?v=O-bJ4s8OdcA


#----------------------------------------------
* Exercise benchmarking

Create a new directory in the exercises directory

    exercises/benchmarking

Tasks:

- Determine as what size of a struct pass-by-reference becomes faster than pass-by-value

.code examples/benchmark/mycalclib.go  /START OMIT/,/END OMIT/

    Peek: examples/benchmark


#----------------------------------------------

* More on testing

- Standard library offers utils for testing http-clients and http-servers
- Code coverage: IntellijIDEA
- Race condition detection
- Continuous integration

#----------------------------------------------
* Concurrency
#----------------------------------------------

# Whatever mix of packages you use: Consumers just have to understand a single error-handling strategy

* Concurrency


Mimic the real world a little better:

- Processing multiple requests at the same time
- Divide and conquer
- Speedup: Concurrent fetching before merge
- Cache eviction: Scheduled background "thread" removes old entries at regular intervals
- Synchronzed state-machine: select loop is the heart
- Fan out: Fire and forget
- Return earlier by moving non-criticals (like notification) off the main thread

#----------------------------------------------

* First class citizens for concurrency:

- Goroutines
- Channels
- Select-loop

#----------------------------------------------

* Goroutines

- Think very, very lightweight threads
- More that 100.000 can be created in single process

    // start go-routine
    go functionName( parameter1, parameter2 )

- Go-routine terminates when function returns

#----------------------------------------------

* Channels

- Think pipe or queue to communicate to/from goroutine(s)

   // create blocking channel: writer returns when other side has read
   blockingChannel := make(chan Response)

   // create buffered channel: write returns immediately until channel is full (10)
   resultChannel := make(chan Response, 10)

   // write to channel
   resultChannel <- value

    // signal 'done' to reading side
    defer close(resultChannel)

   // read from channel
   value, ok := <- resultChannel

   // read until remote writer has closed channel
   for val := range resultChannel {}

- Reader can close when it is sure all writers are done
- Writer can close when it is the only writer of that channel

#----------------------------------------------

* Blocking versus buffered channels

.image examples/images/channels.png

#----------------------------------------------

* Select-loop

- Wait for events from multiple channels

    for {
        select { // blocks untill msg received on one of its channels
            case msg := <-msgChannel:
                fmt.Printf("msg: %s\n", msg) // stay in loop
            case <-tick:
                fmt.Println("tick.") // stay in loop
            case <-boom:
                fmt.Println("BOOM!")
                return // abort loop
        }
    }

- Should NOT do any blocking actions within loop

#----------------------------------------------

* Problems

Multiple go-routines need the share data:

Alternative approaches:

- Communicate by sharing -> protect shared resource with mutex
- Share by communicating -> send copy over channel

#----------------------------------------------

* Divide the work <channels>

.play -edit examples/channels/channels.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Select example <select>
- Wait for events on multiple channels

.play -edit examples/select/select.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Some remarks:

- Don't overdo it
- Most of your code is synchronous
- Hide within packages (not part of external interface)
- Having too many go-routines can blow up your system, so still consider using bounded pools and prepare to give back-pressure


#----------------------------------------------

* Exercise: concurrency

Use dedicated directory for this exercise:

    $ exercises/concurrency

We have an operation that takes approx. 1 second to complete that we have to call 100 times to solve our task

   Peek: examples/slowapi

Tasks:

- Can you speed it up and make it run within 2 seconds?

- Same but wait no more than 1 second (ignore the rest). Also report the number of results received.

    // 2 solution examples

    Peek:solutions/concurrency
    Peek:solutions/concurrency2


#----------------------------------------------
* Generics
#----------------------------------------------
* Why generics

Functions, methods and data structures in a generic form, with standard types factored out

How:

-   Custom types, and custom type parameters, often labeled "T"

When to use:

-   When you notice yourself writing the same code, however the only difference is the type

Caution:

-   Reduces code, but might add complexity

#----------------------------------------------
* Function signature

    func genericFunction[T int | string](value T) T {
       ...
    }

Let's break it down:

    [T int | string]

"T" is the custom type parameter of type int, or of type string

    (value T)

T is then used as the type for the "value" parameter
T is also used as return type

    func genericFunction[T any](value T) T {

The "any" type is a catch-all with regards to types
"any" is the same as an "interface{}" type

#----------------------------------------------
* Custom types
.play -edit examples/generics/genericPrint/main.go /START OMIT/,/END OMIT/

#----------------------------------------------
* Max value of a certain type
.play -edit examples/generics/genericMax/main.go /START OMIT/,/END OMIT/

#----------------------------------------------
* Can be used in testing as well

If you ever use GRPC...

    func toPtr[T any](val T) *T {
        return &val
    }


#----------------------------------------------
* And in interfaces too!

    var _ Iterator[any] = (*SpannerIterator[any])(nil)

    type Iterator[T any] interface {
    	Do(f func(t T) error) error
    	ToResults() ([]T, error)
    	ToSingleResult() (T, error)
    }

    type SpannerIterator[T any] struct {
    	it         *spanner.RowIterator
    }

    func NewSpannerIterator[T any](it *spanner.RowIterator) Iterator[T] {
    	return &SpannerIterator[T]{
    		it: it,
    	}
    }


	user, err := NewSpannerIterator(User).ToSingleResult() // uses generic spanner iterator




#----------------------------------------------

* Exercise: Create a generic function
    Use examples/generics/genericFunction/main.go

Tasks:

- Create a generic function that returns the keys of any map
- Compare your generic function to the existing functions, make sure the output is the same

    Peek: solutions/generics/main.go

#----------------------------------------------
* Dependency management
#----------------------------------------------
* In other ecosystems

- maven in Java
- npm for javascript

#----------------------------------------------
* The old way

- Use "go get" to fetch latest and greatest versions of your dependencies
- Rely on well behaving citizens: backwards compatility
- Repair work unpredictable: Upgrade your own code or offer pr on dependency
- Okay when you test, build and deploy every day

#----------------------------------------------
* New approach

- Based on semantic versioning:

    major.minor.release
    breaking.feature.fix

- Have reproducable builds
- Repair work more predictable: you decide when and what to upgrade
- If build broken, downgrade to lower version of dependency or upgrade your own code

#----------------------------------------------
* Go modules

- A module is a tree of go packages, versioned as a whole
- "go.mod"-file in its root stores module-name and dependencies

    module github.com/xebia/my-example

    go 1.18

    require (
    	github.com/google/uuid v1.2.0
    	github.com/gorilla/mux v1.8.0
    	github.com/stretchr/testify v1.7.0
    )

Real-life example:

    github.com/xebia/go-training/go.mod

#----------------------------------------------
* Major version upgrade

- Backwards incompatible changes
- Would break code of consumers
- Create new package with major version-number  in the import path

    import (
        "rsc.io/quote"
        quoteV2 "rsc.io/quote/v2"
    )

- Old and new version can live side-by-side

Note:

- One does not lighty introduce a new major version, because it requires you to maintain multiple trees

#----------------------------------------------
* Example commands

Create a new package outside go-training

    // assume your code builds and tests
    go mod init github.com/my/project    # creates a new go.mod and go.sum files
    go install ./... && go test ./...    # adds regular and test dependencies
    go list -m all
    go mod tidy                          # removes unused dependencies
    cat go.mod                           # you should commit this file
    cat go.sum                           # you should commit this file

Upgrade a module

    go list -m -versions github.com/my/dependency1 # list versions of a module
    go get github.com/my/dependency1               # latest
    go get github.com/other/dependency2@v1.0.3     # specfic version of githash

#----------------------------------------------

* Interesting videos

- On how to use go modules and upgrade versions

.link https://www.youtube.com/watch?v=aeF3l-zmPsY
.link https://www.youtube.com/watch?v=H_4eRD8aegk
#----------------------------------------------

* Exercise: go-module + Generics

Use dedicated directory for this exercise:

    $ exercises/extraGenerics

Take the following list of integers:

    []int{1, 2, 2, 1, 2, 3, 1, 2, 4, 6, 7, 8, 9, 5, 2, 1234}

Tasks:

    Write 2 functions:
    1 to check if the number 9 is in the array // expected output: true
    2 to return the list of integers that occur multiple times in the original list
    // expected output []int{1, 2}

- Install the package:

    github.com/samber/lo

- And simplify your solutions with the Contains and FindDuplicates functions from the library

#----------------------------------------------

* Exercise: Create a new go-module

Create a new directory outside go-training

     ${HOME}/myuuid

Tasks:

- Use the following source-code

.code examples/myuuid/myuuid.go /START OMIT/,/END OMIT/

- Create a go module

# go mod init github.com/xebia/myuuid
# go mod tidy

#----------------------------------------------

* Exercise: Upgrade an existing go-module

Continue with same directory outside go-training

     ${HOME}/myuuid

Tasks:

- Improve the source-code to use the package

    github.com/google/uuid

- Upgrade module dependencies

# go get github.com/google/uuid
# go install
# go mod tidy
# cat go.mod


#----------------------------------------------

* Debugging

- Use intelliJ IDEA or Goland

#----------------------------------------------

* Exercise: fix using debugger

    Use examples/debugging

Tasks:

- Fix the code by making use of the debugger


#----------------------------------------------

* The standard library

#----------------------------------------------

* Rich standard libraries
Just a few examples:

- logging, flags,
- file I/O, os
- sync, sort
- networking, http, (client and server)
- encoding (json, xml, mime)
- compression, crypto
- templates
- sql
- integrating C-code (.link https://dave.cheney.net/tag/cgo )
- ...


#----------------------------------------------

* Serialisation

- Based on annotations
- xml, json etc in stdlib

example:

   type Person struct {
      Name      string   `json:"name"      xml:"PersonName"`
      Interests []string `json:"interests" xml:"PersonInterests"`
      Children  []Child  `json:"children"  xml:"Person_Children"`
   }

   type Child struct {
      Name string `json:"name"          xml:"name"`
      Age  int    `json:"age,omitempty" xml:"age,omitempty"`
   }

- Why are attributes global?

#----------------------------------------------

* Exercise: Parse json and write xml

Use dedicated directory for this exercise:

    $ exercises/serialisation

Use the file examples/patientjson/patient.json as input

Tasks:

- Copy the content of the file into a raw string variable (backticks) and unserialize the json

     TIP: Use package encoding/json

- Convert the populated structure into xml on stdout

     TIP: Use package encoding/xml


    Peek: examples/encoding/encoding.go

#----------------------------------------------

* Override standard json serialisation

If needed, you can override the standard json and xml serialisation

    type UnixTime struct {
        UnderlyingTime time.Time
    }

    // for json
    func (t UnixTime) MarshalJSON() ([]byte, error) { ... }
    func (t *UnixTime) UnmarshalJSON(data []byte) error { ... }


    // for xml
    func (t MyDate) MarshalText() ([]byte, error) { ... }
    func (t *MyDate) UnmarshalText(data []byte) error { ... }

#----------------------------------------------

* Flags <flags>

- Read command-line arguments
.play -edit examples/flags/flags.go /START OMIT/,/END OMIT/

#----------------------------------------------
* HTTP
#----------------------------------------------

* HTTP client

Request and response:

- Request.Method: POST, PUT, DELETE and GET
- Request.Url: REST-ful?
- *.Headers: Content-Type, Accept, Authorization (Basic or Bearer)
- *.Payload: Json, XML
- Timeout
- Response.StatusCode: 200, 201, 202, 301, 307, 400, 401, 403, 404, 500, 501, 503

Based on Swagger/OpenApi-spec?

Standard library provides API

#----------------------------------------------

* Example HTTP GET <myhttpclient>

.code examples/myhttpclient/main.go /START OMIT/,/END OMIT/

Tips:

- https://mholt.github.io/curl-to-go/

#----------------------------------------------

* HTTP server

- Methods: POST, PUT, DELETE, GET AND HEAD
- Url: REST-ful?
- Payload: Json, XML
- Middleware for non-functionals (CORS, Auth, Monitoring)
- Response status-code and error-message

Based on Swagger/OpenApi-spec?

Standard library provides API:

- HTTP/2 capable
- Each request runs in its own goroutine
- Fileserver and reverse proxy included
- Basic routing capabilities

#----------------------------------------------

* Example server with standard library

.code examples/webserver/main.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Example web-part contains no business logic

.code examples/webserver/web.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Example business logic-part
- Pure business logic
- Contains nothing http related
- Typically has "injected" dependencies

    type patientWebService struct {
    }

    func (s *patientWebService)getPatientOnUID(patientUID string) (Patient, error) {
        ...
    }

    func (s *patientWebService)createPatient(patient Patient) (Patient, error) {
        ...
    }

    func (s *patientWebService)modifyPatientOnUid(patient Patient) (Patient, error) {
        ...
    }

    func (s *patientWebService)deletePatientOnUid(patient Patient) (error) {
        ...
    }


#----------------------------------------------
* 3rd party routers

- Advanced routing capabilities such as grouping
- Better middleware support
- Examples:
    - gorilla/mux
    - go-chi/chi


#----------------------------------------------

* Middleware <middleware>

- Like Java "servlet filters"

Example with "Alice"

.code examples/middleware/main.go /START OMIT/,/END OMIT/

- Later more on middleware functions

#----------------------------------------------

* Exercise: http server

- Create a HTTP server that supports POST on /api/patient to create a patient
- Test it with curl or your previously created HTTP-client

    # Example curl request to submit a new patient
    curl -vvv \
        -X POST \
        --data '{"uid":"1", "fullName":"Marc","addressLine":"a","allergies":["peanut"]}' \
        -H 'Content-Type: application/json' \
        -H 'Accept: application/json' \
        http://localhost:8080/api/patient

- Create a unit test in which you simulate client behavior (using httptest-package)

#----------------------------------------------
* Tooling
#----------------------------------------------
* Static analysis <staticanalysis>

.code examples/staticanalysis/staticanalysis.go

- go vet, golint, gocyclo, varcheck, errcheck, test-coverage, ...
https://bettercodehub.com/docs/faq
=> goreporter

#----------------------------------------------

* How are these tools created?

- By parsing go
- Stdlib provides apis to parse: go/ast, go/parser

Examples:

- Extract documentation from code
- Create boiler-plate code

#----------------------------------------------

* Exercise: detect problems with static analysis tools


#----------------------------------------------

* Triggering a generator

    go generate ./...  # recursively triggers <command> in package
                       # where a go file contains //go:generate <command>

Example:

- Trigger creation of source-code from grpc spec
- Create mocks from interface

    _________spec.go ___________________________________

    //go:generate protoc -I/usr/local/include -I  ../.. -I . --go_out=plugins=grpc:. --grpc-gateway_out=logtostderr=true:. --swagger_out=logtostderr=true:. ./notifications.proto

    //go:generate mockgen -source=notifications.pb.go -destination=notificationClientMock.go -package=notificationapi NotificationClient

    func init() {
    }


#----------------------------------------------


* Extras

#----------------------------------------------

* Contract based interfaces

Why:

- Describe exchange of info for both sides on the line
- Not just for communication with external parties
- Prevent "breaking the contract". If so, use semantic versioning (major version number update) to make the breaking change explicit.

Be backwards compatible:

- Never remove mandatory fields, can remove optionals.
- Never change naming and ... of existing fields
- Only add new optional fields

How:

- IDL (=interface definition language)
- Technology independent


#----------------------------------------------

* IDL examples

- (WSDL/XSD)
- Swagger/OpenApi spec
- GRPC (Protocol buffers)


Target technologies:

- Java
- Golang
- Python
- Javascript
- ...

#----------------------------------------------
* When to use what?

Outside:

- Simple and widely understood
- Easy to pass through firewalls
-> REST?

Inside:

- Easy to use
- Something more powerful
- Something you can base your "conventions" on.
-> GRPC?

#----------------------------------------------
* GRPC request/response

Example fragment

    ...
    service PatientInfo {
        rpc GetPatientOnUid (GetPatientOnUidRequest) returns (GetPatientOnUidReply) {
            option (google.api.http) = {
                post: "/api/patient/{patientUid}",
            };
        }
    }

    message GetPatientOnUidRequest {
        string patientUid = 1;
    }

    message GetPatientOnUidReply {
        Error error = 1;
        Patient patient = 2;
    }
    ...

#----------------------------------------------
* GRPC streaming

- From client
- To client
- Bi-directional

Example fragment

    ...
    service FlightInfoAsync {
        rpc GetHistory (HistoryRequest) returns (stream Flight) {} // one-way
        rpc KeepSynchronizing (stream Acknowledgement) returns (stream FlightPdu) {} // two-ways
    }
    ...


#----------------------------------------------
* More GRPC

Convert contract to go code to base your server and client on:

    //go:generate protoc -I  ../.. -I . \
            --go_out=plugins=grpc:. \
            --grpc-gateway_out=logtostderr=true:. \
            ./patients.proto

Create mocks, so consumers can easily unit test their own software without access to a real service.

    //go:generate mockgen -source=patients.pb.go -destination=patientinfoClientMock.go -package=patientinfoapi PatientInfoClient

The resulting golang-api (for consumers of the contract) is a good candidate for "semantic versioning" using go modules

#----------------------------------------------
* HTTP -> GRPC gateway

.image examples/images/grpc_gateway.png  400 1000

#----------------------------------------------


* Templating <template>

- web-server: generate html response using template
- code-generator: generate source-code from dsl
- Minimize logic in your template (move common concerns to template.FuncMap)

.play -edit examples/template/template.go  /START OMIT/,/END OMIT/

#----------------------------------------------

* Synchronisation <sync>
- Alternative to channels
- Use for small syncs or lazy initialisation
.play -edit examples/sync/sync.go /START OMIT/,/END OMIT/

#----------------------------------------------

* Code review findings


#----------------------------------------------

* Populating a slices of pointers

.play -edit examples/codereviewfindings/slice_pointers/main.go /START OMIT/,/END OMIT/


    Peek: codereviewfindings/slice_pointers

#----------------------------------------------

* Populating a slice from go routine

.play -edit examples/codereviewfindings/populate_slice_in_parallel/main.go /START OMIT/,/END OMIT/


    Peek: codereviewfindings/populate_slice_in_parallel

#----------------------------------------------

* Populating a map from go routine

.play -edit examples/codereviewfindings/populate_map_in_parallel/main.go /START OMIT/,/END OMIT/

    Peek: codereviewfindings/populate_map_in_parallel

#----------------------------------------------

* Inline go routines

.play -edit examples/codereviewfindings/inline_goroutine/main.go /START OMIT/,/END OMIT/

    Peek: codereviewfindings/inline_goroutine

#----------------------------------------------

* Final assigment

We are building a new platform for patients to gain insights in their medical history.
To do so, we have to allow patients to register with our platform.

When registering, we will validate that the email address provided belongs to the patient in question.

We will do so by sending the patient a randomly generated pincode which they'll have to enter to verify their email.


Tasks:

- Split up into three groups.
- Create a plan of approach within the group.
- Draw the architecture

We'll reconvene in 15 minutes

#----------------------------------------------
* Final assigment

Implement the plan!

We've prepared a separate repository here:
.link https://github.com/xebia/go-exercise

Checkout the project on your laptop

    $ git clone https://github.com/xebia/go-exercise.git
    $ cd patientregistration
    $ go install ./...

Don't forget to read the readme 👀
Some building blocks come out of the box 🙌


#----------------------------------------------

* Conclusions

- Understandability and simplicity first
- Write tests

#----------------------------------------------

* Understandability is king

.image examples/images/sweet.png

#----------------------------------------------

* Just start

.image examples/images/minimum.png

#----------------------------------------------

* Expectations and goals

- Back to expectations at beginning of training

#----------------------------------------------

* Improvements

- Retro

#----------------------------------------------

* What's next?

- Zo snel mogelijk aan de bak. Niet laten verstoffen.

#----------------------------------------------

* Done!

#----------------------------------------------

* Complete tool-chain
  $ go list ./...                  # list sub-packages
  $ go list -f {{.Deps}}           # list external deps
  
  # no flame-wars, easy to diff, allow mechanical manipulation
  $ go fmt ./...                   # format recursively or
  $ goimports -w .                # even better: adds/removes imports
  
  # fetch external packages
  $ go get golang.org/x/tools/cmd/goimports # Be effective within seconds

  # testing: tooling supports coverage, benchmark, race-condition-detection
  $ go test ./...                  # test recursively 

  # compile
  $ go build or go run             # add hoc
  $ go install                     # to ${GOPATH}/bin

  $ godoc -http=:8000              # documentation of sources in ${GOROOT} and ${GOPATH}
  $ go vet                         # lint like static analysis

- All CLI -> Easy to automate



#----------------------------------------------

* Simple build system
- Based on filesystem and conventions
- No Makefiles, autotools, build.xml, pom.xml etc
- The tool knows how to build
- Easy to integrate 3rd packages: only integrate source-code

#----------------------------------------------

* The goods
- Simple -> Easy to learn
- Source-code is easy to read and understand
- Easy deployment: self contained binary (no separate runtime)
- Supports cross compilation
- Well suited to host in the cloud: Very fast startup -> fast automatic scale-up while handling traffic spikes

#----------------------------------------------

* The not so good
- No immutability by default

#----------------------------------------------

* So why go?
- Simple, readable language
- Rich standard libraries
- Complete tool-chain
- Good at concurrency
- Fast
- Easy to build and deploy
- Used by many startups
- Boring language -> is good

- Language to get things done

.link https://golang.org/

#----------------------------------------------

* Go do it
- Start experimenting
.image examples/images/gophereartrumpet.jpg
.caption Go is cool:  Tell it around
- Let's build something


#----------------------------------------------
* Extras
#----------------------------------------------

* Wrapping errors
TODO



- Used in large scale project

#----------------------------------------------


* "Kind of" generics <generate>
- go get github.com/clipperhouse/gen
.code examples/generate/generate.go
  $ go generate ./...
  generates cyclist_slice.go with "SortBy"- and "Where"-methods on CyclistSlice
- Just commit and distribute with your package

.link https://github.com/xebia/go-training/blob/master/examples/generate/cyclist_slice.go cyclist_slice.go
#----------------------------------------------


* Dependency analysis

- Golang does not allow cyclic dependencies :)
- Standard tooling

    go list -f '{{ .Name }}: {{ .Doc }}' # current package
    go list -f '{{ join .Imports  "\n" }}' # Nice input for a script

- Try to minimize your dependencies
- Sometimes a little duplication is better than introducing a huge dependency
- Duck-typing allows add-hoc interface creation

#----------------------------------------------

* Exercise: dependency analysis


#----------------------------------------------


* Overridable functions <emailapi>
- Interfaces more suitable for overriding however you might encounter this

.code -edit examples/emailapi/api.go

- Can be useful in combination with build-tags

#----------------------------------------------