From: Steven D. Vormwald (sdvormwa_at_mtu_dot_edu)
Date: Fri Jan 11 2008 - 09:44:03 PST
Paul H. Hargrove wrote:
> Let us see what GASNet is seeing when it probes the hardware. Please
> follow the following steps, sending the output of the final command:
>
> $ cd [YOUR_BERKELY_UPC_BUILD_DIR]
> $ cd dbg/gasnet/vapi-conduit
> $ make testgasnet-seq
> [...output omitted...]
> $ env GASNET_TRACEMASK=C GASNET_TRACEFILE=stdout
> ./contrib/gasnetrun_vapi -n1 ./testgasnet | grep HCA
>
> When things are working correctly, you should expect output roughly
> like the following:
>
> GASNet reporting enabled - tracing and statistical output directed to
> stdout
> 0 0.001157s> (C) Probing HCAs for active ports
> 0 0.001887s> (C) Probe found HCA 'mthca0'
> 0 0.001976s> (C) Probe found HCA 'mthca0', port 1
> 0 0.001985s> (C) Probe found 1 active port(s) on 1 HCA(s)
> 0 0.001997s> (C) vapi-conduit HCA properties (1 of 1) = {
> 0 0.002004s> (C) HCA id = 'mthca0'
> 0 0.002006s> (C) HCA vendor id = 0x2c9
> 0 0.002008s> (C) HCA vendor part id = 0x6274
> 0 0.002010s> (C) HCA hardware version = 0xa0
> 0 0.002012s> (C) HCA firmware version =
>
> -Paul
I should note that these cards (to the best of my knowledge) do not
support the Mellanox VAPI, and thus I didn't enable support for it when
building the compiler:
[sdvormwa@gilbert vapi-conduit]$ make testgasnet-seq
../other/Makefile-conduit.mak:245: warning: overriding commands for
target `Makefile'
Makefile:512: warning: ignoring old commands for target `Makefile'
make[1]: Entering directory
`/usr/local/build/berkeley_upc-2.6.0-dbg/dbg/gasnet/vapi-conduit'
../other/Makefile-conduit.mak:245: warning: overriding commands for
target `Makefile'
Makefile:512: warning: ignoring old commands for target `Makefile'
make[2]: Entering directory
`/usr/local/build/berkeley_upc-2.6.0-dbg/dbg/gasnet/vapi-conduit'
../other/Makefile-conduit.mak:245: warning: overriding commands for
target `Makefile'
Makefile:512: warning: ignoring old commands for target `Makefile'
ERROR: vapi-conduit support was not detected at configure time
try re-running configure with --enable-vapi
make[2]: *** [do-error] Error 1
make[2]: Leaving directory
`/usr/local/build/berkeley_upc-2.6.0-dbg/dbg/gasnet/vapi-conduit'
make[1]: *** [testgasnet] Error 2
make[1]: Leaving directory
`/usr/local/build/berkeley_upc-2.6.0-dbg/dbg/gasnet/vapi-conduit'
make: *** [testgasnet-seq] Error 2
[sdvormwa@gilbert vapi-conduit]$
Running the same series of commands in ibv-conduit produced the following:
[sdvormwa@gilbert ibv-conduit]$ env GASNET_SSH_NODEFILE=~/.mpihosts
GASNET_TRACEMASK=C GASNET_TRACEFILE=stdout ./contrib/gasnetrun_ibv -n1
./testgasnet | grep HCA
GASNet reporting enabled - tracing and statistical output directed to stdout
libibverbs: Warning: no userspace device-specific driver found for
/sys/class/infiniband_verbs/uverbs0
GASNet gasnetc_init returning an error code: GASNET_ERR_RESOURCE
(Problem with requested resource)
at /usr/local/src/berkeley_upc-2.6.0/gasnet/vapi-conduit/gasnet_core.c:986
reason: unable to open any HCA ports
GASNet gasnet_init_GASNET_SEQFASTdebugtracestatssrclines returning an
error code: GASNET_ERR_RESOURCE (Problem with requested resource)
at
/usr/local/src/berkeley_upc-2.6.0/gasnet/vapi-conduit/gasnet_core.c:1546
ERROR calling: gasnet_init(&argc, &argv)
at: /usr/local/src/berkeley_upc-2.6.0/gasnet/tests/testgasnet.c:185
error: GASNET_ERR_RESOURCE (Problem with requested resource)
0 0.000897s> (C) Probing HCAs for active ports
0 0.001658s> (C) Probe failed to locate any HCAs
gasnet_exit(): ERROR: signal 11 received during exit... goodbye.
[initiating collective exit]
Cleaning up orphaned processes...
[sdvormwa@gilbert ibv-conduit]$
Steven Vormwald