I think these are other versions of #15658 and #16136. There is something wrong on FreeBSD but I don't understand what it is and I have not been able to recreate it myself.

If anybody has a FreeBSD machine I can ssh into that would help.

Our FreeBSD builders run on GCE. I can create you an instance there.

Our FreeBSD builders run on GCE. I can create you an instance there.

That would help if you have time. Thanks.

@ianlancetaylor, done. Emailed you details.

I think I was incorrect in thinking that this was related to the other FreeBSD problems. The other FreeBSD problems are less frequent, and I have not been able to recreate them at all.

I now think this is the same as issue #13926: a problem that arises when there are many simultaneous callbacks from C to Go, which is exactly what the CgoCallbackGC test case does. The problem is that on FreeBSD you can surprisingly large delays during the callback. My theory is that when enough of those delays occur, the test times out. That is consistent with the failures reported on the builder, which are in all cases the program crashing from SIGQUIT; SIGQUIT is sent by the runtime test harness when the test program runs for more than 1 minute (runTestProg in runtime/crash_test.go).

Working on a patch.

CL https://golang.org/cl/25047 mentions this issue.

TestCgoCallbackGC is still failing on FreeBSD on the builders. The change did not fix it, at least not completely.

We may just have to punt on this for Go 1.7, which should be okay since IIUC this is not a regression in Go 1.7. (Or is it?)

To the best of my knowledge this is not a regression in 1.7.

@adg, @broady, should we at least document this as a known issue in the release notes for Go 1.7? And then punt this to Go 1.8 for actually fixing.

It'd be a pity to ship something with known crashes for a "first class port". I'd rather warn FreeBSD users about this bug so they don't deploy something with cgo callbacks in production and see crashes.

Agree, sounds like the right move.

On 27 July 2016 at 15:46, Brad Fitzpatrick [email protected] wrote:

@adg https://github.com/adg, @broady https://github.com/broady,
should we at least document this as a known issue in the release notes for
Go 1.7? And then punt this to Go 1.8 for actually fixing.

It'd be a pity to ship something with known crashes for a "first class
port". I'd rather warn FreeBSD users about this bug so they don't deploy
something with cgo callbacks in production and see crashes.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16396 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AIDilbbWTN44lyUFsJ__eMiYKdLcEpZgks5qZvCsgaJpZM4JOHyK
.

Maybe, but if it's not a regression then the release notes isn't really the place for this.

We could say something like "Reduced runtime crashes on FreeBSD, but the issue is still present. See Issue 16396."

I don't think it matters whether it's a regression. Had I known about this in Go 1.6 I would have also argued we need a dedicated "Known Issues" section in our release notes, especially for first class ports.

Or: we demote FreeBSD to no longer be a first-class port.

I suppose it wasn't known at the time, so we couldn't document as a known issue. SGTM.

Perhaps we should also amend the 1.6 release notes (with a date for said addendum).

CL https://golang.org/cl/25283 mentions this issue.

CL https://golang.org/cl/27313 mentions this issue.

Does anyone have any tips on reproducing this?
I'm currently running with the following and haven't seen the test take over 7 seconds.

setenv GOMAXPROCS 2
runtime> sh -c 'while true; do go test -run=TestCgoCallbackGC -cpu=1,2,4; sleep 1; done'

Test machine is dual E5-2640 @ 2.50GHz running FreeBSD 11.0-RELEASE-p3 go version go1.7.3 freebsd/amd64

Also testing this on 10.1 (metal) and 10.2 (gce VM) currently nothing longer than 8 seconds, tests are still running.

Test has been running in a loop on three different machines for over 28 hours not a single failure so far.

@stevenh Which version of FreeBSD? It's possible that changes to the kernel scheduler have eliminated the problem. As it says in https://golang.org/cl/25047 the problem was much worse on FreeBSD than on GNU/Linux for some reason.

Or of course it's possible that something we've changed in the runtime has fixed the problem, though I don't know what that could be.

I tested on 10.1 (metal), 10.2 (GCE) and 11 (metal) all under golang 1.7.3.

With all machines clearing well over 24 hours (coming up to 32 now) on tight loop running this test I think it may be good to remove the 604efe1 for 1.8 to see if we can confirm it is indeed no longer an issue.

All three machines have now been running the test in loop for over 3 days no errors.

Thanks for trying it so thoroughly. Sounds like this issue can be closed.

@ianlancetaylor, did you mean to close this? It sounds like we need to at least remove the t.Skip for this test on FreeBSD.

CL https://golang.org/cl/33903 mentions this issue.

CL https://golang.org/cl/33926 mentions this issue.

Hello,
I've starting running builds with CGO_ENABLED=1 on freebsd-arm-paulzhol and it fails consistently:

ok  	regexp/syntax	4.107s
--- FAIL: TestCgoCallbackGC (180.11s)
	crash_test.go:105: testprogcgo CgoCallbackGC exit status: exit status 2
	crash_cgo_test.go:61: expected "OK\n", but got:
		SIGQUIT: quit
		PC=0x20263cd0 m=0
		signal arrived during cgo execution
		
		goroutine 166 [syscall, locked to thread]:
		runtime.cgocall(0xc0c78, 0x30a7d7c4, 0x0)
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/cgocall.go:131 +0x130 fp=0x30a7d7ac sp=0x30a7d790
		main._Cfunc_foo()
			_/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/_obj/_cgo_gotypes.go:176 +0x38 fp=0x30a7d7c0 sp=0x30a7d7ac
		main.CgoCallbackGC.func2(0x30a5a040)
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/callback.go:81 +0x14 fp=0x30a7d7d4 sp=0x30a7d7c0
		runtime.goexit()
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/asm_arm.s:998 +0x4 fp=0x30a7d7d4 sp=0x30a7d7d4
		created by main.CgoCallbackGC
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/callback.go:83 +0xd8
		
		goroutine 1 [chan receive]:
		main.CgoCallbackGC()
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/callback.go:86 +0x114
		main.main()
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/testdata/testprogcgo/main.go:34 +0x1ac
		
		goroutine 17 [runnable, locked to thread]:
		runtime.goexit()
			/tmp/gobuilder/freebsd-arm-paulzhol-6ceba5d5e85b/go/src/runtime/asm_arm.s:998 +0x4

System is an ALLWINNER A20 running FreeBSD 11.0-RELEASE-p2. I've noticed CGO_ENABLED was off by default because the bootstrap 1.4.3 compilers need to be built with CGO_ENABLED=0.

@paulzhol This issue is closed and the problem you are reporting has different symptoms. Please open a new issue with the details. Thanks.

@ianlancetaylor I've opened #18811 but I do have the following in my log which look like the "Most are:" example in the original issue:

ok      regexp/syntax   4.205s
--- FAIL: TestCgoCallbackGC (204.17s)
        crash_test.go:106: testprogcgo CgoCallbackGC exit status: signal: quit
        crash_cgo_test.go:66: expected "OK\n", but got:
FAIL
FAIL    runtime 427.836s
ok      runtime/debug   0.088s

But the builder failed to push that:

2017/01/27 02:25:42 Failed: exit status 1
Build complete, duration 41m53.837957541s. Result: error: exit status 1
 RunTime:41m53.837957541s Builder:freebsd-arm-paulzhol PackagePath:]
2017/01/27 02:25:45 Post https://build.golang.org/result?builder=freebsd-arm-paulzhol&key=XXXXXXXXXXXX&version=1: dial tcp: lookup build.golang.org on 192.168.XXX.XXX:53: server misbehaving