prometheus筆記

in monitor

Histograms vs Summaries

Histogram與Summaries都適合拿來observe request duration or response size.
使用histogram and summary時, 除了有_count與_sum的matrics以外, 還會有buckets or quantile
可以拿來計算quantiles

Comment and share

go test tool筆記

in golang

run all test except vendor

1
go test $(go list ./... | grep -v /vendor/)

go tool cover

1
2
3
go test -coverprofile coverage.txt <package name>
go tool cover -html=coverage.txt -o coverage.html
chrome coverage.html

example script to run all package test and generate coverage file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
#!/usr/bin/env bash
set -e
out="coverage.txt"
tmp="profile.coverage.out"
except="example|testproto"
: > $out
i=0
for d in $(go list ./... | grep -v vendor); do
    # except some package
    if [[ $d =~ $except ]]; then
        continue
    fi
    
    echo -e "TESTS FOR: for \033[0;35m${d}\033[0m"
    go test -v -coverprofile=$tmp $d
    if [ -f $tmp ]; then
        # remove mode:xxx if not first package for go tool cover
        if [ "$i" != 0 ]; then
            sed -i '' -e '/^mode:/d' ./$tmp
        fi
        cat $tmp >> $out
        rm $tmp
    fi
    echo ""
    ((i++))
done

Comment and share

golang test 筆記

in golang

Table Driven Test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
func TestAdd(t * testing.T) {
  tcs := []struct{ 
    Name string 
    A, B, Expected int
  }{
    {"foo", 1, 1, 2},
    {"bar", 1, -1, 0},
  }
  for _, tc := range tcs {
    t.Run(tc.Name, func(t *testing.T) {
      actual := tc.A + tc.B
      if actual != expected {
        t.Errorf("%d + %d = %d, exected %d", tc.A, tc.B, actual, tc.Expected)
      }
    })
  }
}

Test fixtures

go test set pwd as package directory
use relative path “test-fixtures” directory as a place to store test data(loading config, model data, binary data …etc)

1
2
3
4
func TestAdd(t *testing.T) {
  data := filepath.Join("test-fixtures", "add_data.json")
  // Do something with data
}

Golden files(Test flag)

expected output放在.golden file中
當update flag為true時, 代表需要更新expected output file
Very useful to test complex structures

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
var update = flag.Bool("update", false, "update golden files")
func TestAdd(t *testing.T) {
  // ... table (probably!)
  for _, tc := range tcs {
    actual := doSomething(tc)
    golden := filepath.Join("test-fixtures", tc.Name+".golden")
    if *update {
      ioUtil.WriteFile(golden, actual, 0644)
    }
    expected, _ := ioutil.ReadFile(golden)
    if !bytes.Equal(actual, expected) {
      //FAIL!
    }
  }
}

1
2
go test
go test -update

Global State

Do not use global state.
If you have to do, Use default, and let test can easily mock it.

1
2
3
4
5
const defaultPort = 1000
type ServerOpts {
  Port int // default it to defaultPort somewhere
}

Test Helpers

TestTempFile

Test helper接受testing.T參數, helper中的錯誤直接在helper中處理掉
helper return a function to clean

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
func testTempFile(t * testing.T) (string, func()) {
  t.Helper()
  tf, err := iotuil.TempFile("", "test")
  if err != nil {
    t.Fatalf("err: %s", err)
  }
  tf.Close()
  return tf.Name(), func() { os.Remove(tf.Name()) }
}
func TestThing(t *testing T) {
  tf, tfclose := testTempFile(t)
  defer tfclose()
  // doSomething with tf
}

TestChdir

切換dir並在function結束前切換回來
注意testChdir會回傳一function, 該function會將dir切回原本old
由於defer會在defer當下處理完參數所以在defer當下即會切換到dir
然後在TestThing結束前呼叫os.chdir(old)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
func testChdir(t *testing.T, dir string) func() {
  t.Helper()
  old, err := os.Getwd()
  if err != nil {
    t.Fatalf("err: %s", err)
  }
  if err := os.Chdir(dir); err != nil {
    t.Fatalf("err: %s", err)
  }
  return func() { os.Chdir(old) }
}
func TestThing(t *testing.T) {
  defer testChdir(t, "/other")()
}

Networking

不需要mock net.Conn

1
2
3
4
5
6
7
8
9
10
11
12
13
14
func TestConn(t *testing.T) (client, server net.Conn) {
  t.Helper()
  ln, err := net.Listen("tcp", "127.0.0.1:0")
  var server net.Conn
  go func() {
    defer ln.Close()
    server, err = ln.Acccept()
  }()
  client, err := net.Dial("tcp", ln.Addr().String())
  return client, server
}

Test Subprocessing

Check if git is installed. If not, skip test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
var testHasGit bool
func init() {
  if _, err := exec.LookPath("git"); err == nil {
    testHasGit = true
  }
}
func TestGitGetter(t *testing.T) {
  if !testHasGit {
    t.Log("git not found, skipping")
    t.Skip()
  }
}

Mock Subprocess

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
func helperProcess(s ...string) *exec.Cmd {
  cs := []string{"-test.run=TestHelperProcess", "--"}
  cs = append(cs, s...)
  env := []string{
    "GO_WANT_HELPER_PROCESS=1"
  }
  cmd := exec.Command(os.Args[0], cs...)
  cmd.Env = append(env, os.Environ()...)
  return cmd
}
func TestHelperProcess(*testing.T) {
  if os.Getenv("GO_WANT_HELPER_PROCESS") != "1" {
    return
  }
  defer os.Exit(0)
  args := os.Args
  for len(args) > 0 {
    if args[0] == "--" {
      args = args[1:]
      break
    }
    args = args[1:]
  }
  ...
  cmd, args := args[0], args[1:]
  switch cmd {
    case "foo":
      // ...
  }
}

Testing as a public API

提供使用你的package的user一個測試你的package的public API(Maybe a mock or a helper)
test.go will be ignored by go, so use testing.go for Test for API
Example:

  • API Server
    • TestServer(t) (net.Addr, io.Closer) => Returns a fully started in-memory server and a closer to close it.
  • interface for downloading files
    • TestDownloader(t, Downloader) => Tests all the properties a downloader should have.
    • struct DownloaderMock[] => Implements Downloader as a mock

Timing-Dependent Test

timeMultiplier is configurable.
maybe lower at laptop and higher at server

1
2
3
4
5
6
7
8
9
func TestThing(t * testing.T) {
  timeout := 3 * time.Minute * timeMultiplier
  select {
  case <- thingHappened:
  case <- time.After(timeout):
    t.Fatal("timeout")
  }
}

Complex Structs Compare(maybe graph)

How to compare complex structs in test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
type ComplexThing struct {/* ... */}
func (c *ComplexThing) testString() string {
  // produce human-friendly output for test comparison
}
//----------------------------------------
func TestComplexThing(t *testing.T) {
  c1, c2 := createComplexThings()
  if c1.testString() != c2.testString() {
    t.Fatalf("no match:\n\n%s\n\n%s", c1.testString(), c2.testString())
  }
}

Comment and share

glide筆記

in golang

https://github.com/Masterminds/glide

安裝

1
brew install glide

Usage

source code需要放在gopath下

1
2
3
4
5
6
7
$ glide create                            # Start a new workspace
$ open glide.yaml                         # and edit away!
$ glide get github.com/Masterminds/cookoo # Get a package and add to glide.yaml
$ glide install                           # Install packages and dependencies
# work, work, work
$ go build                                # Go tools work normally
$ glide up                                # Update to newest versions of the package

get 3rd party library and all their dependencies

1
2
glide get --all-dependencies <path>
glide up --all-dependencies

Comment and share

gvm筆記與vscode整合

in golang

https://github.com/moovweb/gvm

安裝

1
2
3
4
5
6
7
8
xcode-select --install
brew update
brew install mercurial
bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
gvm install go1.4 -B
gvm use go1.4
export GOROOT_BOOTSTRAP=$GOROOT
gvm install 1.8

指令

1
2
3
4
5
6
gvm install [version]
gvm uninstall [version]
gvm listall
gvm list
gvm use [version]
gvm implode    // uninstall gvm

使用gvm來管理workspace

1
2
3
4
5
6
7
go use [version]
mkdir -p ~/golang/  
cd ~/golang/  
gvm pkgset create --local  
gvm pkgset use --local  
mkdir src pkg bin
go env

利用pkgset將GOPATH設為自定的workspace path

vscode 整合

設定gopath與goroot為go env的值

1
2
go use [version]
go env

1
2
3
4
{
  "go.gopath":"/Users/meep007/cwz/develop/practice/golang:/Users/meep007/cwz/develop/practice/golang/.gvm_local/pkgsets/go1.8/local:/Users/meep007/.gvm/pkgsets/go1.8/global",
  "go.goroot": "/Users/meep007/.gvm/gos/go1.8"
}

Comment and share

  • page 1 of 1

Cwza

Hello everyone.
I’m cwza.
Welcome to my blog.


Software Engineer


Taiwan/Taipei