Re: UDP Conduit Environment Issue

From: James Dinan (
Date: Tue Apr 28 2009 - 08:35:35 PDT

  • Next message: Nikita Andreev: "Overlapping put/get events"
    Hi Paul,
    Thank you for your response, I'll go ahead and file a feature request.
    A little more info: I'm doing some hybrid MPI+UPC investigation and I've 
    run into this problem when launching mixed BUPC-UDP and MPICH2 with the 
    Hydra process manager.  The problem is that, at the call to MPI_Init(), 
    MPICH2 needs to get some information from the environment to contact the 
    process manager (specifically PMI_ID and PMI_PORT).
    Unfortunately if I tell upcc --uses-mpi it will initialize MPI before 
    the user-defined main runs.  So there is no chance to do the 
    sevenv(getenv()) workaround.
    Thanks again,
    Paul H. Hargrove wrote:
    > Jim,
    >  I am glad to hear that you are (mostly) pleased with the Berkeley UPC 
    > project.
    >  The short answer is that UPC_ENVPREFIX should be working if you are 
    > using bupc_getenv(), but has no promise to set the variable in the 
    > "true" environment.
    >  The propagation of the environment in a UPC application does not have a 
    > well-defined guaranteed behavior.  In fact, neither does MPI.  So, while 
    > using mpi-conduit is working for you it is not certain to do so with a 
    > different MPI implementation.  To carry that to the next step: while you 
    > may have a way to work around the lack of a working getenv() with 
    > Berkeley UPC, there is no guarantee that you'll be able to do so with 
    > another UPC compiler.  So, in the long term I'd suggest that your best 
    > course of action, IMHO, is to find a way to avoid use of getenv() or 
    > bupc_getenv() on compute nodes.  If I understand correctly, you are 
    > trying to set variables that control third-party libraries, so I doubt 
    > you can do much better than what you describe (manually "republishing" 
    > the environment variables you need).
    >  Having said my piece above, I acknowledge that you do have a valid 
    > concern especially given that a user may not know at compilation time 
    > which environment variables are read by third-party libraries.  The main 
    > reason we don't automatically propagate all the environment variables 
    > from the execution environment to the environment of the UPC threads is 
    > that there are cases where things break if we were to propagate certain 
    > variables (and of course we can't always know what vars are bad).  So, I 
    > think it would be beneficial (and safe) if UPC_ENVPREFIX or another 
    > mechanism were available to ensure some set of user-specified 
    > environment variables reached the "true" environment (not just the 
    > private one).
    >  If you feel strongly that this feature is needed for your work. I would 
    > encourage you to visit our Bugzilla ( and enter 
    > a feature request (a bug with Severity=enhancement).
    > -Paul
    > James Dinan wrote:
    >> Hi,
    >> I've run into a slight problem with how the environment is propagated
    >> when using the UDP conduit.  I'm using BUPC 2.8.0.
    >> My code is calling a non-upc library function which queries an
    >> environment variable.  The environment variable can be read fine in main
    >> but not in the non-upc library I am calling.  Looking through the
    >> translated code I realized this is because the upc compiler is
    >> substituting bupc_getenv for calls to getenv.  BUPC is then reading the
    >> correct value from its private copy of the environment.
    >> Unfortunately I can't do this substitution in the library code.  One
    >> workaround is to call bupc_getenv() followed by setenv in my UPC code to
    >> manually copy the variable from one environment to the other.  But this
    >> is not ideal.
    >> Is there a better way to ensure that BUPC propagates values from its
    >> environment into the main environment?  I tried setting UPC_ENVPREFIX
    >> but that didn't help.  Also, I noticed this problem does not occur when
    >> using the mpi conduit so it may be restricted to environment handling
    >> for the udp conduit.
    >> I've attached a test case that exercises my problem.  Here is what the
    >> output looks like on my system:
    >> ---snip---
    >> $ make
    >> gcc -Wall   -c -o library.o library.c
    >> upcc -network=udp -o env-test library.o env-test.upc
    >> $ export MYVAR=success
    >> $ upcrun -n 1 ./env-test
    >> UPCR: UPC thread 0 of 1 on bblogin (process 0 of 1, pid=23587)
    >> Environment lookup test.  Variable name is MYVAR.
    >> main: success
    >> env_test: (null)
    >> Manually propagating variable from upc main
    >> main: success
    >> env_test: success
    >> ---snip---
    >> Also, a big thanks to all who have contributed to Berkeley UPC.  This is
    >> a wonderful project!
    >> Best wishes,
    >>  ~Jim.

  • Next message: Nikita Andreev: "Overlapping put/get events"