From: Steven D. Vormwald (sdvormwa_at_mtu_dot_edu)
Date: Mon Jan 07 2008 - 09:57:02 PST
Hello, I'm having trouble using the ibv-conduit in the 2.6.0 release: [sdvormwa@gilbert UPC]$ GASNET_BACKTRACE=1 /usr/local/berkeley_upc-2.6.0/bin/upcrun -np 1 ./a.out libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0 GASNet gasnetc_init returning an error code: GASNET_ERR_RESOURCE (Problem with requested resource) at /usr/local/src/berkeley_upc-2.6.0/gasnet/vapi-conduit/gasnet_core.c:986 reason: unable to open any HCA ports GASNet gasnet_init_GASNET_SEQFASTnodebugnotracenostatsnosrclines returning an error code: GASNET_ERR_RESOURCE (Problem with requested resource) at /usr/local/src/berkeley_upc-2.6.0/gasnet/vapi-conduit/gasnet_core.c:1546 UPC Runtime: GASNet error GASNET_ERR_RESOURCE(Problem with requested resource): gasnet_init failed! [0] 0: /home/sdvormwa/UPC/./a.out [0x440913] gasneti_bt_execinfo ??:0 [0] 1: /home/sdvormwa/UPC/./a.out [0x442db3] gasneti_print_backtrace ??:0 [0] 2: /home/sdvormwa/UPC/./a.out [0x404d14] upcri_gaserr ??:0 [0] 3: /home/sdvormwa/UPC/./a.out [0x4288bc] bupc_init_reentrant ??:0 [0] 4: /home/sdvormwa/UPC/./a.out [0x403591] main ??:0 [0] 5: /lib64/tls/libc.so.6(__libc_start_main+0xdb) [0x3ca971c3fb] ?? ??:0 [0] 6: /home/sdvormwa/UPC/./a.out [0x40318a] _start ??:0 [sdvormwa@gilbert UPC]$ openib was installed using prebuilt packages, configured and tested as documented in the user manual that came with the infiniband cards (see download.qlogic.com/manual/62612/IB6054601-00E.pdf). The libibverbs documentation for the warning says to ensure that /usr/local/lib/infiniband/mthca.so exists and is executable, which I have verified on all the nodes. I've run through the rest of the included troubleshooting documentation, and have yet to find a solution. Any help would be much appreciated. Thanks, Steven Vormwald This is upcc (the Berkeley Unified Parallel C compiler), v. 2.6.0 (getting remote translator settings...) ----------------------+--------------------------------------------------------- UPC Runtime | v. 2.6.0, built on Dec 3 2007 at 13:21:31 ----------------------+--------------------------------------------------------- UPC-to-C translator | v. 2.6.0, built on Oct 15 2007 at 15:50:19 ----------------------+--------------------------------------------------------- Translator location | http://upc-translator.lbl.gov/upcc-2.6.0.cgi ----------------------+--------------------------------------------------------- networks supported | udp smp ibv ----------------------+--------------------------------------------------------- default network | ibv ----------------------+--------------------------------------------------------- pthreads support | available (if used, default is 2 pthreads per process) ----------------------+--------------------------------------------------------- Configured with | '--with-translator=http://upc-translator.lbl.gov/upcc-2 | .6.0.cgi' '--enable-ibv' '--verbose' | 'IBV_LIBS=-libverbs -ldl' 'LDFLAGS=-ldl' | '--without-mpi-cc' '--disable-mpi' | '--prefix=/usr/local/berkeley_upc-2.6.0//opt' | '--with-multiconf-magic=opt' ----------------------+--------------------------------------------------------- Configure features | berkeleyupc,upcr,gasnet,upc_collective,upc_io, | upc_memcpy_async,upc_ptradd,upc_thread_distance, | upc_tick,upc_sem,upc_dump_shared,upc_trace_printf, | upc_trace_mask,upc_local_to_shared,upc_atomics,pupc, | upc_memcpy_vis,nodebug,notrace,nostats,nogasp, | segment_fast,os_linux,cpu_x86_64,cpu_64,cc_gnu, | packedsptr ----------------------+--------------------------------------------------------- Configure id | gilbert.cse.mtu.edu Mon Dec 3 13:16:56 EST 2007 root ----------------------+--------------------------------------------------------- Binary interface | 64-bit x86_64-unknown-linux-gnu ----------------------+--------------------------------------------------------- Runtime interface # | Runtime supports 3.0 -> 3.9: Translator uses 3.6 ----------------------+--------------------------------------------------------- | --- BACKEND SETTINGS (for ibv network) --- ----------------------+--------------------------------------------------------- C compiler | /usr/bin/gcc | GNU/3.4.6/3.4.6 20060404 (Red Hat 3.4.6-3) | gcc version 3.4.6 20060404 (Red Hat 3.4.6-3) | Reading specs from | /usr/lib/gcc/x86_64-redhat-linux/3.4.6/specs | Configured with: ../configure --prefix=/usr | --mandir=/usr/share/man --infodir=/usr/share/info | --enable-shared --enable-threads=posix | --disable-checking --with-system-zlib | --enable-__cxa_atexit --disable-libunwind-exceptions | --enable-java-awt=gtk --host=x86_64-redhat-linux ----------------------+--------------------------------------------------------- C compiler flags | -O3 --param max-inline-insns-single=35000 --param | inline-unit-growth=10000 --param | large-function-growth=200000 -Winline ----------------------+--------------------------------------------------------- linker | /usr/bin/gcc | GNU/3.4.6/3.4.6 20060404 (Red Hat 3.4.6-3) | gcc version 3.4.6 20060404 (Red Hat 3.4.6-3) | Reading specs from | /usr/lib/gcc/x86_64-redhat-linux/3.4.6/specs | Configured with: ../configure --prefix=/usr | --mandir=/usr/share/man --infodir=/usr/share/info | --enable-shared --enable-threads=posix | --disable-checking --with-system-zlib | --enable-__cxa_atexit --disable-libunwind-exceptions | --enable-java-awt=gtk --host=x86_64-redhat-linux ----------------------+--------------------------------------------------------- linker flags | -O3 --param max-inline-insns-single=35000 --param | inline-unit-growth=10000 --param | large-function-growth=200000 -Winline -ldl | -L/usr/local/berkeley_upc-2.6.0//opt/lib -lupcr-ibv-seq | -lumalloc -L/usr/local/berkeley_upc-2.6.0//opt/lib | -L/usr/lib64 -lgasnet-ibv-seq -libverbs -ldl -lpthread | -L/usr/lib/gcc/x86_64-redhat-linux/3.4.6 -lgcc -lm ----------------------+---------------------------------------------------------