UPC: Distributed Shared Memory Programming

Preface vii

1. Introductory Tutorial 1

1.1 Getting Started 1

1.2 Private and Shared Data 3

1.3 Shared Arrays and Affinity of Shared Data 6

1.4 Synchronization and Memory Consistency 8

1.5 Work Sharing 10

1.6 UPC Pointers 11

1.7 Summary 14

Exercises 14

2. Programming View and UPC Data Types 17

2.1 Programming Models 17

2.2 UPC Programming Model 20

2.3 Shared and Private Variables 21

2.4 Shared and Private Arrays 23

2.5 Blocked Shared Arrays 25

2.6 Compiling Environments and Shared Arrays 30

2.7 Summary 30

Exercises 31

3. Pointers and Arrays 33

3.1 UPC Pointers 33

3.2 Pointer Arithmetic 35

3.3 Pointer Casting and Usage Practices 38

3.4 Pointer Information and Manipulation Functions 40

3.5 More Pointer Examples 43

3.6 Summary 47

Exercises 47

4. Work Sharing and Domain Decomposition 49

4.1 Basic Work Distribution 50

4.2 Parallel Iterations 51

4.3 Multidimensional Data 54

4.4 Distributing Trees 62

4.5 Summary 71

Exercises 71

5. Dynamic Shared Memory Allocation 73

5.1 Allocating a Global Shared Memory Space Collectively 73

5.2 Allocating Multiple Global Spaces 78

5.3 Allocating Local Shared Spaces 82

5.4 Freeing Allocated Spaces 89

5.5 Summary 90

Exercises 90

6. Synchronization and Memory Consistency 91

6.1 Barriers 92

6.2 Split-Phase Barriers 94

6.3 Locks 99

6.4 Memory Consistency 108

6.5 Summary 113

Exercises 114

7. Performance Tuning and Optimization 115

7.1 Parallel System Architectures 116

7.2 Performance Issues in Parallel Programming 120

7.3 Role of Compilers and Run-Time Systems 122

7.4 UPC Hand Optimization 123

7.5 Case Studies 128

7.6 Summary 135

Exercises 135

8. UPC Libraries 137

8.1 UPC Collective Library 137

8.2 UPC-IO Library 141

8.3 Summary 146

References 147

Appendix A: UPC Language Specifications, v1.1.1 149

Appendix B: UPC Collective Operations Specifications, v1.0 183

Appendix C: UPC-IO Specifications, v1.0 203

Appendix D: How to Compile and Run UPC Programs 243

Appendix E: Quick UPC Reference 245

Index 251