From: Paul H. Hargrove (PHHargrove_at_lbl_dot_gov)
Date: Sun Apr 11 2010 - 14:40:51 PDT
Reinhold,
I am normally able to provide quick and reasonably accurate answers to
folks on this list. However, in this case I will admit that
multi-dimensional arrays in C have always been a weak point of mine -
and adding UPC shared distribution into the mix doesn't help. Since I
was not initially sure what to expect for correct output, I ran your
code through GCCUPC and Cray's compiler
With Cray's compiler I received the result I imagine you were expecting:
a[0][2] is on thread 0 with value 0
a[1][2] is on thread 1 with value 1
a[2][2] is on thread 2 with value 2
a[3][2] is on thread 3 with value 3
a[4][2] is on thread 4 with value 4
a[5][2] is on thread 5 with value 5
a[6][2] is on thread 6 with value 6
a[7][2] is on thread 7 with value 7
When I tried GCCUPC it produces a result different from that of Berkeley
UPC and Cray UPC:
a[0][2] is on thread 2 with value 0
a[1][2] is on thread 5 with value 1
a[2][2] is on thread 0 with value 2
a[3][2] is on thread 3 with value 3
a[4][2] is on thread 6 with value 4
a[5][2] is on thread 1 with value 5
a[6][2] is on thread 4 with value 6
a[7][2] is on thread 7 with value 7
So, what I have to go on is that three compilers produce three different
results.
This leaves me not certain what to say.
However, if I print ALL of a[][] then the result from Berkeley UPC does
appear to be incorrect. The values printed appear to vary from run to
run, indicating a race condition, but the following entries are consistent:
a[0][0] is on thread 0 with value 0
a[1][0] is on thread 1 with value 0
a[2][0] is on thread 2 with value 0
a[3][0] is on thread 3 with value 1
...
a[0][1] is on thread 1 with value 0
a[1][1] is on thread 2 with value 0
a[2][1] is on thread 3 with value 1
...
a[0][2] is on thread 2 with value 0
a[1][2] is on thread 3 with value 1
This pattern suggests to me that the address computation on the write
and the read might not be the same, with a shifting taking place in one
or the other.
I will enter a bug report for this observed wrong behavior.
So, I am certain that Berkeley UPC is producing an incorrect result.
However, I am afraid I am not able to know what the correct result
should be.
-Paul
Reinhold Bader wrote:
> Hello,
>
> the following program, built with the Berkeley UPC (2.8 or 2.10)
> installations on the sgi Altix (IA64) as well sgi ICE (Nehalem)
> systems ...
>
>
> #include <upc.h>
> #include <stdlib.h>
> #include <stdio.h>
>
>
> shared [*] int a[THREADS][3];
>
> int main(void) {
> int i, j, q;
> for (i=0; i<3; i++) {
> a[MYTHREAD][i] = MYTHREAD;
> }
> upc_barrier;
> if (MYTHREAD == 0) {
> for (q=0; q<THREADS; q++) {
> j = upc_threadof(&a[q][2]);
> printf("a[%d][2] is on thread %d with value %d\n",q,
> j,a[q][2]);
> }
>
> }
> return 0;
> }
>
>
>
> ... produces the following result.
>
> upcrun -n 8 ./symmetric_data.exe
> UPCR: UPC thread 1 of 8 on r1i2n2 (process 1 of 8, pid=5833)
> UPCR: UPC thread 2 of 8 on r1i2n2 (process 2 of 8, pid=5834)
> UPCR: UPC thread 3 of 8 on r1i2n2 (process 3 of 8, pid=5835)
> UPCR: UPC thread 4 of 8 on r1i2n2 (process 4 of 8, pid=5836)
> UPCR: UPC thread 5 of 8 on r1i2n2 (process 5 of 8, pid=5837)
> UPCR: UPC thread 6 of 8 on r1i2n2 (process 6 of 8, pid=5838)
> UPCR: UPC thread 7 of 8 on r1i2n2 (process 7 of 8, pid=5839)
> UPCR: UPC thread 0 of 8 on r1i2n2 (process 0 of 8, pid=5832)
> a[0][2] is on thread 2 with value 0
> a[1][2] is on thread 3 with value 1
> a[2][2] is on thread 4 with value 2
> a[3][2] is on thread 5 with value 3
> a[4][2] is on thread 6 with value 4
> a[5][2] is on thread 7 with value 5
> a[6][2] is on thread 0 with value 6
> a[7][2] is on thread 1 with value 7
>
>
> The value of upc_threadof() appears to be inconsistent with the
> MYTHREADS value. On the other hand, the available documentation
> repeatedly mentions constructions like
> if (upc_threadof(&data) == MYTHREADS) {
>
> ...
>
> }
> to ensure locality of reference. Any explanations or hints?
>
> Best Regards
>
>
--
Paul H. Hargrove PHHargrove_at_lbl_dot_gov
Future Technologies Group Tel: +1-510-495-2352
HPC Research Department Fax: +1-510-486-6900
Lawrence Berkeley National Laboratory