文档章节

go race detector

chuqq
 chuqq
发布于 2018/09/25 14:44
字数 1177
阅读 443
收藏 0
Go

「深度学习福利」大神带你进阶工程师,立即查看>>>

https://golang.org/doc/articles/race_detector.html

 

Data Race Detector

Introduction

Usage

Report Format

Options

Excluding Tests

How To Use

Typical Data Races

Race on loop counter

Accidentally shared variable

Unprotected global variable

Primitive unprotected variable

Supported Systems

Runtime Overhead

 

Introduction

Data races are among the most common and hardest to debug types of bugs in concurrent systems. A data race occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write. See the The Go Memory Model for details.

Here is an example of a data race that can lead to crashes and memory corruption:

func main() {
	c := make(chan bool)
	m := make(map[string]string)
	go func() {
		m["1"] = "a" // First conflicting access.
		c <- true
	}()
	m["2"] = "b" // Second conflicting access.
	<-c
	for k, v := range m {
		fmt.Println(k, v)
	}
}

Usage

To help diagnose such bugs, Go includes a built-in data race detector. To use it, add the -race flag to the go command:

$ go test -race mypkg    // to test the package
$ go run -race mysrc.go  // to run the source file
$ go build -race mycmd   // to build the command
$ go install -race mypkg // to install the package

Report Format

When the race detector finds a data race in the program, it prints a report. The report contains stack traces for conflicting accesses, as well as stacks where the involved goroutines were created. Here is an example:

WARNING: DATA RACE
Read by goroutine 185:
  net.(*pollServer).AddFD()
      src/net/fd_unix.go:89 +0x398
  net.(*pollServer).WaitWrite()
      src/net/fd_unix.go:247 +0x45
  net.(*netFD).Write()
      src/net/fd_unix.go:540 +0x4d4
  net.(*conn).Write()
      src/net/net.go:129 +0x101
  net.func·060()
      src/net/timeout_test.go:603 +0xaf

Previous write by goroutine 184:
  net.setWriteDeadline()
      src/net/sockopt_posix.go:135 +0xdf
  net.setDeadline()
      src/net/sockopt_posix.go:144 +0x9c
  net.(*conn).SetDeadline()
      src/net/net.go:161 +0xe3
  net.func·061()
      src/net/timeout_test.go:616 +0x3ed

Goroutine 185 (running) created at:
  net.func·061()
      src/net/timeout_test.go:609 +0x288

Goroutine 184 (running) created at:
  net.TestProlongTimeout()
      src/net/timeout_test.go:618 +0x298
  testing.tRunner()
      src/testing/testing.go:301 +0xe8

Options

The GORACE environment variable sets race detector options. The format is:

GORACE="option1=val1 option2=val2"

The options are:

  • log_path (default stderr): The race detector writes its report to a file named log_path.pid. The special names stdoutand stderr cause reports to be written to standard output and standard error, respectively.
  • exitcode (default 66): The exit status to use when exiting after a detected race.
  • strip_path_prefix (default ""): Strip this prefix from all reported file paths, to make reports more concise.
  • history_size (default 1): The per-goroutine memory access history is 32K * 2**history_size elements. Increasing this value can avoid a "failed to restore the stack" error in reports, at the cost of increased memory usage.
  • halt_on_error (default 0): Controls whether the program exits after reporting first data race.

Example:

$ GORACE="log_path=/tmp/race/report strip_path_prefix=/my/go/sources/" go test -race

Excluding Tests

When you build with -race flag, the go command defines additional build tag race. You can use the tag to exclude some code and tests when running the race detector. Some examples:

// +build !race

package foo

// The test contains a data race. See issue 123.
func TestFoo(t *testing.T) {
	// ...
}

// The test fails under the race detector due to timeouts.
func TestBar(t *testing.T) {
	// ...
}

// The test takes too long under the race detector.
func TestBaz(t *testing.T) {
	// ...
}

How To Use

To start, run your tests using the race detector (go test -race). The race detector only finds races that happen at runtime, so it can't find races in code paths that are not executed. If your tests have incomplete coverage, you may find more races by running a binary built with -race under a realistic workload.

Typical Data Races

Here are some typical data races. All of them can be detected with the race detector.

Race on loop counter

func main() {
	var wg sync.WaitGroup
	wg.Add(5)
	for i := 0; i < 5; i++ {
		go func() {
			fmt.Println(i) // Not the 'i' you are looking for.
			wg.Done()
		}()
	}
	wg.Wait()
}

The variable i in the function literal is the same variable used by the loop, so the read in the goroutine races with the loop increment. (This program typically prints 55555, not 01234.) The program can be fixed by making a copy of the variable:

func main() {
	var wg sync.WaitGroup
	wg.Add(5)
	for i := 0; i < 5; i++ {
		go func(j int) {
			fmt.Println(j) // Good. Read local copy of the loop counter.
			wg.Done()
		}(i)
	}
	wg.Wait()
}

Accidentally shared variable

// ParallelWrite writes data to file1 and file2, returns the errors.
func ParallelWrite(data []byte) chan error {
	res := make(chan error, 2)
	f1, err := os.Create("file1")
	if err != nil {
		res <- err
	} else {
		go func() {
			// This err is shared with the main goroutine,
			// so the write races with the write below.
			_, err = f1.Write(data)
			res <- err
			f1.Close()
		}()
	}
	f2, err := os.Create("file2") // The second conflicting write to err.
	if err != nil {
		res <- err
	} else {
		go func() {
			_, err = f2.Write(data)
			res <- err
			f2.Close()
		}()
	}
	return res
}

The fix is to introduce new variables in the goroutines (note the use of :=):

...
			_, err := f1.Write(data)
			...
			_, err := f2.Write(data)
			...

Unprotected global variable

If the following code is called from several goroutines, it leads to races on the service map. Concurrent reads and writes of the same map are not safe:

var service map[string]net.Addr

func RegisterService(name string, addr net.Addr) {
	service[name] = addr
}

func LookupService(name string) net.Addr {
	return service[name]
}

To make the code safe, protect the accesses with a mutex:

var (
	service   map[string]net.Addr
	serviceMu sync.Mutex
)

func RegisterService(name string, addr net.Addr) {
	serviceMu.Lock()
	defer serviceMu.Unlock()
	service[name] = addr
}

func LookupService(name string) net.Addr {
	serviceMu.Lock()
	defer serviceMu.Unlock()
	return service[name]
}

Primitive unprotected variable

Data races can happen on variables of primitive types as well (boolintint64, etc.), as in this example:

type Watchdog struct{ last int64 }

func (w *Watchdog) KeepAlive() {
	w.last = time.Now().UnixNano() // First conflicting access.
}

func (w *Watchdog) Start() {
	go func() {
		for {
			time.Sleep(time.Second)
			// Second conflicting access.
			if w.last < time.Now().Add(-10*time.Second).UnixNano() {
				fmt.Println("No keepalives for 10 seconds. Dying.")
				os.Exit(1)
			}
		}
	}()
}

Even such "innocent" data races can lead to hard-to-debug problems caused by non-atomicity of the memory accesses, interference with compiler optimizations, or reordering issues accessing processor memory .

A typical fix for this race is to use a channel or a mutex. To preserve the lock-free behavior, one can also use thesync/atomic package.

type Watchdog struct{ last int64 }

func (w *Watchdog) KeepAlive() {
	atomic.StoreInt64(&w.last, time.Now().UnixNano())
}

func (w *Watchdog) Start() {
	go func() {
		for {
			time.Sleep(time.Second)
			if atomic.LoadInt64(&w.last) < time.Now().Add(-10*time.Second).UnixNano() {
				fmt.Println("No keepalives for 10 seconds. Dying.")
				os.Exit(1)
			}
		}
	}()
}

Supported Systems

The race detector runs on darwin/amd64freebsd/amd64linux/amd64, and windows/amd64.

Runtime Overhead

The cost of race detection varies by program, but for a typical program, memory usage may increase by 5-10x and execution time by 2-20x.

下一篇: rust
chuqq

chuqq

粉丝 3
博文 51
码字总数 12806
作品 1
南京
程序员
私信 提问
加载中
请先登录后再评论。
AirPush-Detector

这是一个检测android程序中是否含有AirPush广告框架或者其他广告框架的简单软件。

开源好!
2012/12/12
1.7K
0
OSSIM架构与组成综述

OSSIM架构与组成综述 OSSIM布道师 李晨光 一、背景 如果运维工程师手里没有高效的管理工具支持,就很难快速处理故障。市面上有很多运维监控工具,例如商业版的 Solarwinds、ManageEngine以及...

OSSIM
2016/01/28
6.5K
1
DSFacialGestureDetector

DSFacialGestureDetector 是视频流 CIDetector。 pod 'DSFacialGestureDetector' 或者导入 Detector 目录的所有文件到你的项目中...

叶秀兰
2014/11/17
357
0
IMSI-Catcher 探测器--Android-IMSI-Catcher-Detector

Android-IMSI-Catcher-Detector 是一个基于安卓的项目,它能在 GSM/UMTS 网络中侦测并避开假冒基站(IMSI 捕捉器)。

孔小菜
2015/03/12
1.7K
0
HDFS的reportWrittenBlock函数解析

可先看下网友的解析:http://fire-balrog.iteye.com/blog/812281 以下为我的解析: 当一个Block经由所有的DataNode写完后,就需要告诉namenode可以执行reportWrittenBlock函数了。 下面就来解...

强子哥哥
2014/12/19
95
0

没有更多内容

加载失败,请刷新页面

加载更多

信息系统项目管理师(15)

软件需求分析的目的是对各种需求信息进行分析并抽象描述,为目标系统建立一个概念模型。通过需求分析,可以检测和解决需求之间的冲突;发现系统的边界;并详细描述出系统的需求。 访问控制授...

LitStone
15分钟前
5
0
按属性值对对象数组进行排序 - Sorting an array of objects by property values

问题: I've got the following objects using AJAX and stored them in an array: 我使用AJAX获得了以下对象并将它们存储在数组中: var homes = [ { "h_id": "3", "ci......

法国红酒甜
21分钟前
3
0
spring-boot集成redis

集成的客户端 1)lettuce方式集成 <dependency><groupId>org.apache.commons</groupId><artifactId>commons-pool2</artifactId></dependency><dependency><groupId>io.lettuce</gro......

简到珍
55分钟前
12
0
Java 8中Lambda表达式默认方法的模板方法模式,你够了解么?

为了以更简单的术语描述模板方法,考虑这个场景:假设在一个工作流系统中,为了完成任务,有4个任务必须以给定的执行顺序执行。在这4个任务中,不同工作流系统的实现可以根据自身情况自定义任...

北柠Java
56分钟前
3
0
阅文集团上半年总收入32.6亿元 同比增长9.7%

  腾讯科技讯,8 月 11 日消息,阅文集团(0772.HK)公布 2020 年中期业绩。报告显示,阅文集团 2020 上半年实现总收入 32.6 亿元,同比增长 9.7%;毛利润为 17.3 亿元,同比增长 6.8%。 ...

gfhtw
59分钟前
14
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部