go - can someone explain to me this golang data race behaviour?
问题描述
I don't know what to look for to find if this is an already asked question, so sorry in advance if it is.
I'm new to golang and I was playing a bit with goroutines and I found out that when compiling the following code:
package main
import (
"fmt"
"math/big"
"sync"
)
const (
jobCount = 5
threadCount = 1
)
type workerResult struct {
job int64
processId int
result *big.Int
}
func main() {
var hashMap sync.Map
jobs := make(chan int64, jobCount)
results := make(chan workerResult, jobCount)
var wg sync.WaitGroup
for i := 0; i < threadCount; i++ {
wg.Add(1)
go worker(jobs, results, i, &hashMap, &wg)
}
go func(){
for i := int64(0); i < jobCount; i++ {
jobs <- i
}
close(jobs)
}()
go func(){
wg.Wait()
close(results)
}()
for result := range results {
fmt.Println(result.job, result.processId, result.result.String())
}
}
func worker(jobs <-chan int64, results chan<- workerResult, id int, hashMap *sync.Map, wg *sync.WaitGroup) {
for i := range jobs {
results <- workerResult{i, id, fib(i, hashMap)}
}
(*wg).Done()
}
func fib(num int64, hashMap *sync.Map) *big.Int {
if num < 2 {
return big.NewInt(num)
} else {
hashedResult, ok := (*hashMap).Load(num)
if ok{
return hashedResult.(*big.Int)
} else {
result := fib(num-1, hashMap)
result = result.Add(result, fib(num-2, hashMap))
(*hashMap).Store(num, result)
return result
}
}
}
somehow the compiler would just remove the processing of the fibonacci algorithm and straight-out put the last output value for the else clause of the if num < 2.
the output looks like this:
0 0 0
1 0 1
2 0 4
3 0 4
4 0 4
if I change jobCount to something else, lets say 10, the output will be:
0 0 0
1 0 1
2 0 128
3 0 128
4 0 128
5 0 128
6 0 128
7 0 128
8 0 128
9 0 128
when I attached the debugger and put a breakpoint at any place I saw different results, still not good but different:
0 0 0
1 0 1
2 0 1
3 0 2
4 0 4
5 0 8
6 0 32
7 0 64
8 0 64
9 0 128
so then I realized, maybe it's the compiler optimizing the algorithm thinking that can be optimized, so I changed the struct workerResult and swapped the type of result(field) from
*big.Int to string, leaving the code like this:
package main
import (
"fmt"
"math/big"
"sync"
)
const (
jobCount = 10
threadCount = 1
)
type workerResult struct {
job int64
processId int
result string
}
func main() {
var hashMap sync.Map
jobs := make(chan int64, jobCount)
results := make(chan workerResult, jobCount)
var wg sync.WaitGroup
for i := 0; i < threadCount; i++ {
wg.Add(1)
go worker(jobs, results, i, &hashMap, &wg)
}
go func(){
for i := int64(0); i < jobCount; i++ {
jobs <- i
}
close(jobs)
}()
go func(){
wg.Wait()
close(results)
}()
for result := range results {
fmt.Println(result.job, result.processId, result.result)
}
}
func worker(jobs <-chan int64, results chan<- workerResult, id int, hashMap *sync.Map, wg *sync.WaitGroup) {
for i := range jobs {
results <- workerResult{i, id, fib(i, hashMap).String()}
}
(*wg).Done()
}
func fib(num int64, hashMap *sync.Map) *big.Int {
if num < 2 {
return big.NewInt(num)
} else {
hashedResult, ok := (*hashMap).Load(num)
if ok{
return hashedResult.(*big.Int)
} else {
result := fib(num-1, hashMap)
result = result.Add(result, fib(num-2, hashMap))
(*hashMap).Store(num, result)
return result
}
}
}
And I now have good results:
jobCount = 10
0 0 0
1 0 1
2 0 1
3 0 2
4 0 4
5 0 8
6 0 16
7 0 32
8 0 64
9 0 128
jobCount = 20
0 0 0
1 0 1
2 0 1
3 0 2
4 0 4
5 0 8
6 0 16
7 0 32
8 0 64
9 0 128
10 0 256
11 0 512
12 0 1024
13 0 2048
14 0 4096
15 0 8192
16 0 16384
17 0 32768
18 0 65536
19 0 131072
can someone explain to me the why of this behavior, how can I expect this to happen and how can I force the compiler into not doing so?
EDIT: it's not a compiler issue, it's a data race, pointed out in the comments, but I still don't understand how it's happening and why the second version of the code doesn't have such condition
running the code with the -race argument gave me this for the first version of the code:
0 0 0
1 0 1
==================
WARNING: DATA RACE
Read at 0x00c000100048 by main goroutine:
math/big.(*Int).Text()
/usr/local/go/src/math/big/intconv.go:25 +0x3bd
math/big.(*Int).String()
/usr/local/go/src/math/big/intconv.go:40 +0x39f
main.main()
/home/illic/go/src/awesomeProject/fib.go:44 +0x221
Previous write at 0x00c000100048 by goroutine 7:
math/big.(*Int).Add()
/usr/local/go/src/math/big/int.go:121 +0x1e3
main.fib()
/home/illic/go/src/awesomeProject/fib.go:64 +0xfc
main.worker()
/home/illic/go/src/awesomeProject/fib.go:50 +0x56
Goroutine 7 (finished) created at:
main.main()
/home/illic/go/src/awesomeProject/fib.go:28 +0x195
==================
==================
WARNING: DATA RACE
Read at 0x00c000100040 by main goroutine:
math/big.(*Int).Text()
/usr/local/go/src/math/big/intconv.go:25 +0x3ec
math/big.(*Int).String()
/usr/local/go/src/math/big/intconv.go:40 +0x39f
main.main()
/home/illic/go/src/awesomeProject/fib.go:44 +0x221
Previous write at 0x00c000100040 by goroutine 7:
math/big.(*Int).Add()
/usr/local/go/src/math/big/int.go:132 +0x252
main.fib()
/home/illic/go/src/awesomeProject/fib.go:64 +0xfc
main.worker()
/home/illic/go/src/awesomeProject/fib.go:50 +0x56
Goroutine 7 (finished) created at:
main.main()
/home/illic/go/src/awesomeProject/fib.go:28 +0x195
==================
2 0 4
3 0 4
4 0 4
Found 2 data race(s)
for the second version of the code this was the output:
0 0 0
1 0 1
2 0 1
3 0 2
4 0 4
EDIT2:
After the advice on the comments and the one only answer I realized that the problem was in how I handled additions, I was using the pointers stored in the hashmap, so I was basically destroying the values stored from previous calculations on each new execution, the solution to this is to actually allocate a new space in memory for that calculation, this version of the code should be free of race conditions:
package main
import (
"fmt"
"math/big"
"sync"
)
const (
jobCount = 10
threadCount = 1
)
type workerResult struct {
job int64
processId int
result *big.Int
}
func main() {
var hashMap sync.Map
jobs := make(chan int64, jobCount)
results := make(chan workerResult, jobCount)
var wg sync.WaitGroup
for i := 0; i < threadCount; i++ {
wg.Add(1)
go worker(jobs, results, i, &hashMap, &wg)
}
go func(){
for i := int64(0); i < jobCount; i++ {
jobs <- i
}
close(jobs)
}()
go func(){
wg.Wait()
close(results)
}()
for r := range results {
fmt.Println(r.job, r.processId, r.result.String())
}
}
func worker(jobs <-chan int64, results chan<- workerResult, id int, hashMap *sync.Map, wg *sync.WaitGroup) {
for i := range jobs {
results <- workerResult{i, id, fib(i, hashMap)}
}
(*wg).Done()
}
func fib(num int64, hashMap *sync.Map) *big.Int {
if num < 2 {
return big.NewInt(num)
} else {
hashedResult, ok := (*hashMap).Load(num)
if ok{
return hashedResult.(*big.Int)
} else {
result := big.Int{}
result.Set(fib(num-1, hashMap))
result = *result.Add(&result, fib(num-2, hashMap))
(*hashMap).Store(num, &result)
return &result
}
}
}
解决方案
Documentation for big.Int says:
An Int represents a signed multi-precision integer. The zero value for an Int represents the value 0.
Operations always take pointer arguments (*Int) rather than Int values, and each unique Int value requires its own unique *Int pointer. To "copy" an Int value, an existing (or newly allocated) Int must be set to a new value using the Int.Set method; shallow copies of Ints are not supported and may lead to errors.
So, we'll be using Int.Set in our program to remove race conditions.
package main
import (
"fmt"
"math/big"
"sync"
)
const (
jobCount = 5
threadCount = 1
)
type workerResult struct {
job int64
processID int
result *big.Int
}
func main() {
var hashMap sync.Map
jobs := make(chan int64, jobCount)
results := make(chan workerResult, jobCount)
var wg sync.WaitGroup
for i := 0; i < threadCount; i++ {
wg.Add(1)
go worker(jobs, results, i, &hashMap, &wg)
}
go func() {
for i := int64(0); i < jobCount; i++ {
jobs <- i
}
close(jobs)
}()
go func() {
wg.Wait()
close(results)
}()
for r := range results {
fmt.Println(r.job, r.processID, r.result.String())
}
}
func worker(jobs <-chan int64, results chan<- workerResult, id int, hashMap *sync.Map, wg *sync.WaitGroup) {
for i := range jobs {
results <- workerResult{i, id, fib(i, hashMap)}
}
wg.Done()
}
func fib(num int64, hashMap *sync.Map) *big.Int {
if num < 2 {
z := big.Int{}
return z.Set(big.NewInt(num))
}
hashedResult, ok := hashMap.Load(num)
if ok {
z := big.Int{}
return z.Set(hashedResult.(*big.Int))
}
result := fib(num-1, hashMap)
result = result.Add(result, fib(num-2, hashMap))
hashMap.Store(num, result)
z := big.Int{}
return z.Set(result)
}
See, if this program helps you!
推荐阅读
- mysql - 搜索外部 wordpress 页面的数据库 - 没有多站点设置
- javascript - 视差 WordPress 主题标签仅在 Google Chrome 中不起作用
- fonts - 如何强制Win10在C:\Windows\Fonts中安装字体？
- spring - springfox api-doc中的别名主机名
- ionic4 - 为内联模板和样式生成 Ionic 4 CLI 配置
- sql - SQL 选择文本搜索
- django - 在有消息等待时，在 Amazon Elastic Beanstalk 上使用 SQS 的 Celery 经常暂停很长时间
- react-select - 如何确定哪些 react-select 样式键映射到哪些组件
- php - 在 PHP 中获取当前月份的第一个和最后一个日期
- angular - 找不到名称“环境”。在角度 7