Re: upc_threadof() inconsistency?

From: Paul H. Hargrove (PHHargrove_at_lbl_dot_gov)
Date: Sun Apr 11 2010 - 14:40:51 PDT

  • Next message: Reinhold Bader: "Re: upc_threadof() inconsistency?"
    Reinhold,
    
    I am normally able to provide quick and reasonably accurate answers to 
    folks on this list.  However, in this case I will admit that 
    multi-dimensional arrays in C have always been a weak point of mine - 
    and adding UPC shared distribution into the mix doesn't help.  Since I 
    was not initially sure what to expect for correct output, I ran your 
    code through GCCUPC and Cray's compiler
    
    With Cray's compiler I received the result I imagine you were expecting:
    
    a[0][2] is on thread 0 with value 0
    a[1][2] is on thread 1 with value 1
    a[2][2] is on thread 2 with value 2
    a[3][2] is on thread 3 with value 3
    a[4][2] is on thread 4 with value 4
    a[5][2] is on thread 5 with value 5
    a[6][2] is on thread 6 with value 6
    a[7][2] is on thread 7 with value 7
    
    When I tried GCCUPC it produces a result different from that of Berkeley 
    UPC and Cray UPC:
    
    a[0][2] is on thread 2 with value 0
    a[1][2] is on thread 5 with value 1
    a[2][2] is on thread 0 with value 2
    a[3][2] is on thread 3 with value 3
    a[4][2] is on thread 6 with value 4
    a[5][2] is on thread 1 with value 5
    a[6][2] is on thread 4 with value 6
    a[7][2] is on thread 7 with value 7
    
    So, what I have to go on is that three compilers produce three different 
    results.
    This leaves me not certain what to say.
    
    However, if I print ALL of a[][] then the result from Berkeley UPC does 
    appear to be incorrect.  The values printed appear to vary from run to 
    run, indicating a race condition, but the following entries are consistent:
    
    a[0][0] is on thread 0 with value 0
    a[1][0] is on thread 1 with value 0
    a[2][0] is on thread 2 with value 0
    a[3][0] is on thread 3 with value 1
    ...
    a[0][1] is on thread 1 with value 0
    a[1][1] is on thread 2 with value 0
    a[2][1] is on thread 3 with value 1
    ...
    a[0][2] is on thread 2 with value 0
    a[1][2] is on thread 3 with value 1
    
    This pattern suggests to me that the address computation on the write 
    and the read might not be the same, with a shifting taking place in one 
    or the other.
    I will enter a bug report for this observed wrong behavior.
    
    So, I am certain that Berkeley UPC is producing an incorrect result.
    However, I am afraid I am not able to know what the correct result 
    should be.
    
    -Paul
    
    
    Reinhold Bader wrote:
    >  Hello,
    >
    >   the following program, built with the Berkeley UPC (2.8 or 2.10)
    >   installations on the sgi Altix (IA64) as well sgi ICE (Nehalem)
    >   systems ...
    >
    >
    > #include <upc.h>
    > #include <stdlib.h>
    > #include <stdio.h>
    >
    >
    > shared [*] int a[THREADS][3];
    >
    > int main(void) {
    >   int i, j, q;
    >   for (i=0; i<3; i++) {
    >     a[MYTHREAD][i] = MYTHREAD;
    >   }
    >   upc_barrier;
    >   if (MYTHREAD == 0) {
    >     for (q=0; q<THREADS; q++) {
    >        j = upc_threadof(&a[q][2]);
    >        printf("a[%d][2] is on thread %d with value %d\n",q,
    >                j,a[q][2]);
    >     }
    >
    >   }
    >   return 0;
    > }
    >
    >
    >
    >   ... produces the following result.
    >
    > upcrun -n 8 ./symmetric_data.exe
    > UPCR: UPC thread 1 of 8 on r1i2n2 (process 1 of 8, pid=5833)
    > UPCR: UPC thread 2 of 8 on r1i2n2 (process 2 of 8, pid=5834)
    > UPCR: UPC thread 3 of 8 on r1i2n2 (process 3 of 8, pid=5835)
    > UPCR: UPC thread 4 of 8 on r1i2n2 (process 4 of 8, pid=5836)
    > UPCR: UPC thread 5 of 8 on r1i2n2 (process 5 of 8, pid=5837)
    > UPCR: UPC thread 6 of 8 on r1i2n2 (process 6 of 8, pid=5838)
    > UPCR: UPC thread 7 of 8 on r1i2n2 (process 7 of 8, pid=5839)
    > UPCR: UPC thread 0 of 8 on r1i2n2 (process 0 of 8, pid=5832)
    > a[0][2] is on thread 2 with value 0
    > a[1][2] is on thread 3 with value 1
    > a[2][2] is on thread 4 with value 2
    > a[3][2] is on thread 5 with value 3
    > a[4][2] is on thread 6 with value 4
    > a[5][2] is on thread 7 with value 5
    > a[6][2] is on thread 0 with value 6
    > a[7][2] is on thread 1 with value 7
    >
    >
    >    The value of upc_threadof() appears to be inconsistent with the
    >    MYTHREADS value. On the other hand, the available documentation
    >    repeatedly mentions constructions like
    >    if (upc_threadof(&data) == MYTHREADS) {
    >
    >       ...
    >
    >     }
    >    to ensure locality of reference. Any explanations or hints?
    >
    > Best Regards
    >
    >   
    
    
    -- 
    Paul H. Hargrove                          PHHargrove_at_lbl_dot_gov
    Future Technologies Group                 Tel: +1-510-495-2352
    HPC Research Department                   Fax: +1-510-486-6900
    Lawrence Berkeley National Laboratory     
    

  • Next message: Reinhold Bader: "Re: upc_threadof() inconsistency?"