Structured Parallel Programming: How Informatics Can Help Overcome the Software Dilemma

Scientific Programming, Sep 2018

The state-of-the-art programming of parallel computers is far from being successful. The main challenge today is, therefore, the development of techniques and tools that improve programmers

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

http://downloads.hindawi.com/journals/sp/1996/570310.pdf

Structured Parallel Programming: How Informatics Can Help Overcome the Software Dilemma

Journal of Structured Parallel Programming: H o w Informatics Can Help O v e r c o m e t h e Software D i l e m m a HELMAR BURKHART burkhart@ifi. 0 ROBERT FRANK frank@ifi. 0 GUIDO HACHLER 0 0 lnstitutfiir lnformatik , Universitiit Basel, Mittlere Strasse 142, CH-4056 Basel , Switzerland The state-of-the-art programming of parallel computers is far from being successful. The main challenge today is, therefore, the development of techniques and tools that improve programmers' productivity. Programmability, portability, and reusability are key issues to be solved. In this article we shall report about our ongoing efforts in this direction. After a short discussion of the software dilemma found today, we shall present the Basel approach. We shall summarize our algorithm description methodology and discuss the basic concepts of the proposed skeleton language. An algorithmic example and comments on implementation aspects will explain our work in more detail. We shall summarize the current state of the implementation and conclude with a discussion of related work. © 1996 by John Wiley & Sons, Inc. 1 SOFTWARE IS THE PROBLEM Computer power is a precondition to tackle scien­ tific problems. The term Grand Challenges has been coined for the variety of applications that need to be solved, and the discussion has some­ times been reduced to the question of who will have the first teraflop computer. However, this ap­ proach has already been critized [ 2 ]. We believe that the real challenge is to develop concepts that enable programmers to use parallel systems in a more productive way. Today, software develop­ ment for parallel systems is still in its infancy. Pro­ grams are written by using low-level concepts, resuiting in software that is often neither portable, reusable, nor maintainable [ 3 ]. To overcome this software dilemma, sound soft­ ware engineering techniques need to be applied. Structured programming was introduced by in­ formaticians two decades ago to overcome the soft­ ware crisis of sequential systems. Other techniques have since followed. It is high time to transfer suc­ cessful software production techniques to paral­ lel processing! What we need is progress toward the solution of the "big P" challenges: programmability portability - performance. We like to have pro­ grammability at a high level of abstraction to get correct and maintainable software. However, this influences the performance because high-level constructs are always in danger of performance loss. We also prefer to have portable software to minimize software changes if the computer envi­ ronment changes. But a very high level of portabil­ ity is again a source of inefficient solutions. In a word: These three attributes can only be optimized together. It is one of the main challenges today to develop high-level abstractions that still preserve the performance that users expect of parallel svstems. 2 THE BASEL APPROACH What basic elements are necessary for further progress in programming parallel systems? Below, we shall list the problem areas we consider to be priority topics. 2.1 Developing a Taxonomy Describing the Essential Elements of Parallel Algorithms Many research projects emphasize the need for more "programming tools." There is no doubt that tools are needed to develop parallel processor soft­ ware (we. too, are building tools within our project as you will see). However. tools can easily become obsolete as soon as new generations of parallel systems are announced. Programming methodol­ ogy. on the other hand, has to remain stable over a much longer period. This is the reason why the first step will have to be the investigation of meth­ odological aspects, such as the development of a common terminology, algorithmic descriptions and classifications. as well as concepts on how to address software problems. As parallel processing projects are usually interdisciplinary, a common conceptual base used by people with different backgrounds is essential. 2.2 Supporting High-Level Abstractions for Parallel Programming Programming parallel systems at a mainly system­ oriented level is a major weakness of today's envi­ ronments. Having developed a methodology of the type mentioned above, the next step will have to consist of transferring these concepts into lan­ guages and programming tools. Instead of integrat­ ing these-concepts within a single language (e.g., High-Performance Fortran) we propose a separa­ tion into two language levels: 1. The core of a parallel application, we call it a skeleton, should be written in a language that as much as possible enforces program­ mability and correctness. This statement certainly applies to programming in general. For parallel systems it is, however, even more important because parallel programs areal­ ways more complicated than sequential pro­ grams. Carriero and Gelernter [ 7 ] already introduced the notion of coordination lan­ guages for this level. The coordination of processes is certainly one aspect that is im­ portant. However, there are other aspects that are equally important: management of data, support of compositional program­ ming using basic software building blocks, etc. This is the reason why we call it the layer of a skeleton programming language. 2. On the basic building block level traditional languages should be used for programming. Today, Fortran and C dominate science and engineering applications. The investment in existing software is tremendous, and in­ formaticians should not ignore these eco­ nomical aspects. 2.3 Make Parallel Software Reusable One challenge of software development is the soft­ ware factory. i.e., the availability of building blocks that may he reused. \Ve cannot expect complete application programs to he reusable because there are always slight differences. even in the same problem domain, e.g.. different calculation se­ quences and different input/ output formats. We can. though, expect to get reusable component,; on two levels of granularity. 1. Reuse-in-the-large targets for reusable pro­ gram parts that are building blocks for appli­ cation programs. This is possible for regular problems using only a small number of coor­ dination schemes. On massively parallel sys­ tems, this trend is emphasized because hun­ dreds or thousands of different processes cannot possibly be managed individually. Libraries of algorithmic skeletons will play a central role. 2. If such a librarv offers no solution there is still a possibility to apply the reuse-in-the­ small concept. Reusable process topology and data distribution patterns will always be necessary for writing parallel programs. Such a support can be given either on the language or library level. 2.4 Make Parallel Software Portable The software dilemma mentioned above has been tackled from the system side. High-level interfaces, FIGUHE 1 The Basel approach. :-;o-called parallel virtual machines, have been de­ fined to hide the existence of the different operating svstems and architectures. PAR\1ACS, EX­ PRESS, PlCL, PYM . MPL and p4 are a subset of the models mentioned in the respective literature ([ 22 ] presents an up-to-date list). ~lost of these models are very similar because they address the same type of parallel computers. so-called mes­ sage-passing systems. In the near future, stan­ dardization (e.g., ~IPI) will provide a technical ba­ sis. It is crucial that software researchers benefit from these developments and offer support in the form of libraries and tools. Our project provides answers to these four topics (Fig. 1 presents the core items): 1. BACS (Basel Algorithm Classification Scheme), the vocabulary and methodology for describing parallel algorithms and pro­ grams. 2. ALWAN, the language used for writing algo­ rithmic skeletons offering reuse-in -the­ small constructs. 3. BALI, a library of reuse-in-the-large con­ structs: algorithmic skeletons that contain ALW AI'\ modules to be modified and ex­ tended by programmers. 4. TIANA, the program generator that pro­ duces portable source code sksletons for dif­ ferent target systems and programming lan­ guages. This integrated approach provides additional benefits. Different user groups may interact and profit from synergies by using common sy:-;tem ele­ ments: programmers (how to program?), bench­ markers (how to evaluate?). and students (how to learn?) who share common systern components. Both programmers and benchmarkers benefit from the skeleton approach because skeletons can be completed Pither into application programs or syn­ thetic benchmark suites by adding artificial loads. Finally. teaching and resParch nicely interact be­ cause the library can offer both courseware and production software. In this article. we shall summarize BACS and concentrate on AL\VAl\" and TIAl\"A. 3 BACS: A FRAMEWORK FOR PARALLELISM BACS [ 4 ] is a framework for the de:,;cription and classification of parallel algorithms. As will be shown in Section 4, it also provides a terminologi­ cal platform on which structured programming of parallel systems can be built. 3.1 BACS Summary Parallel algorithms can be classified regarding pro­ cess properties (administration and binding), in­ teraction properties (coordination of processes). and data properties (partitioning and placement). In BACS, we are refining this list and end up with a generic description tuple that fully charac­ terizes a parallel algorithm (Fig. 2). Process Properties The process topology defines the geometric struc­ ture and connectivity of the process set. \Ve concentrate on regular topologies such as grids, hypercubes. trees, farms, etc. And we distinguish between static and dynamic process structures. A process structure is called static if a process topol­ ogy remains unchanged during the execution of the actual algorithm. As of today, static algorithms are dominating the field of numerical applications. The execution structure defines the compositing order of calculation and interaction building blocks. This can be expressed using control struc­ tures such as sequences, conditions, and itera­ tions. We created a formula -like notation to pro­ vide the algorithm with a kind of signature. process properties data properties structure I topology I exec. structure II interaction II partitioning I placement Interactions define the coordination of the pro­ cesses at run-time. Coordination operators are used for data exchange, the signaling of events, and for consistency purposes. BACS identifies di­ rect interactions that link exactly two processes and global interactions involving more than two processes. Global interactions are called total if all processes of a topology are interacting. They are called partial if only a subset of the processes is in­ volved. Data Properties Parallel algorithms normally work with distributed data. The data distribution consists of the data partitioning and the data placement, i.e., mapping the partitioned data to processes. Arrays are typi­ cally partitioned in a blockwise or cyclic manner. Each dimension of an array may be partitioned separately, e.g., a two-dimensional array may be split in rows, columns, or subblocks. 3.2 Example: Systolic Matrix Multiplication Systolic algorithms are well-known candidates for massively parallel execution. Our sample multipli­ cation algorithm makes use of blockwise distribu­ tions of two input matrices A and B, and an output matrix C. The process topology used is a static torus. All processes operate on their matrix blocks and compute an intermediate C block. The regular execution pattern, where interactions (here in two dimensions) are followed by local computations, is typical for systolic design. Thus, the execution structure consists of several parts: a prerotation of the A matrix, a prerotation of the B matrix, and the systolic part, a fixed loop consisting of the cal­ culation and the rotation of one position both for A and B. The BACS tuple is a compact description of these algorithmic properties (Fig. 3). We shall elaborate this example in Section 4.6. 3.3 TINA: The Skeleton Generator Prototype The tuple information provides a first, coarse­ grained view of a parallel algorithm. However, it is too informal to be usable as input for a program generator tool. In his dissertation, Stephan Gutz­ willer [ 6, 16 ] elaborated the BACS terminology into a script-like, C-hased language that can be used to specify all parallel aspects of a program. The script contains entries for the specification of process topologies, data partitions and distribu­ tions, and the overall execution structure. Within the script, the programmer also specifies the paral­ lel virtual machine and the programming language for which the code is used. The program generator, called TINA, reads the input script and produces a source code output. TINA is a kind of text merger. Predefined templates (stored in supporting libraries) are filled with the relevant parameter information. The TINA proto­ type supports PV"'1 and EXPRESS in a C language environment. Tll\A was a rapid prototype used for first porta­ bility studies. The script language also bridged the gap between our environment and the more prob­ lem-oriented descriptions of an SPP partnership project [ 10 ]. Although the prototype was quite use­ ful, we decided to redesign the skeleton language and to put even more emphasis on enhanced pro­ grammability. While knowledge collected in the support libraries was reused, the language itself changed completely. The next section will intro­ duce the basic language aspects and an example. 4 ALWAN: A PROGRAMMING LANGUAGE FOR SKELETON PARALLELISM Below, part of a language called ALWAN is de­ scribed, with which it is possible to implement an algorithm, starting with a BACS tuple, in a most platform-independent way [ 5 ]. ALWAN is based on MODULA-2, a structured high-levellanguage. As ALWAN is a skeleton proProcess properties Data properties partitioning A,B,C [Block,Block} execution structure /Rot, fRol, Fi:cr(C', fRot,fRol} interactions {Rot, Rol} gramming language, only a subset of MODULA-2 is used to which a few new concepts had to be introduced reflecting the necessities of parallel programming. Furthermore. AL\VAl'\ allows calling external procedures for the actual calculation parts (which may already be present) or input and output. 4.1 Topologies Topologies are the core of programming with AL­ W AI'\ on a parallel machine. A topology specifies the geometric structure of a nurnber of processes as well as the neighborhoods and possible commu­ nication paths. A topology is declared very much like a procedure and. on the parallel machine . it can in fact be viewed as the procedure running on each parallel proceso;. All statements within a topology are executed in paralleL all other state­ ments are executed only by one process (the con­ troller). It would prevent reusability if the programmer needed to specify all topology properties each time a slightly different topology is declared. Thus AL­ W AI'\ introduces a mechanism (INHERIT) allow­ ing it to inherit properties from a given topology. w·e provide a library of frequently used topologies. so the programmer simply needs to inherit one of these topologies and expand the inherited defini­ tions with the data definitions necessarv for the given algorithm. TOPOLOGY SystolicMult(p:CARDINAL); INHERIT Torus(p,p); VAR a: BEGIN END SystolicMult; such as col_id, row_id, west, and north. These properties will be used throughout the fol­ lowing examples. 4.2 Communication Communication is essential on any parallel ma­ chine. Shared memory systems define implicit communication while message-passing systems define an explicit one. ALWAI'\ also requires ex­ plicit communication, but while message-passing mechanisms quite often require a well-paired send and receive function. AL WAI'\ requires only a spe­ cial assignment construct that includes both the send and the receive function. Communication in ALWAI'\ is initiated by a simple assignment with the location of the communication data being specified by the variable@ location construct. Depending on whether the location specifier is on the left side of the assignment or on the right side, the data will be sent or fetched from the view of the initiating process. The directions in which com­ munication takes place (e.g., west) depend on to­ pology and may either be inherited from a prede­ fined topology or specified by the DIRECTION construct not explained here. The following exam­ ple describes a communication where processes store the contents of their local variables ' a' from their western neighbors in their local variables ' a' . Occasionally, not all processes have to participate in an interaction. The set of processes that initiate this communication may be specified by the AC­ TIVE ... DO construct which acts as a selector. ACTIVE row_id <= i DO a: =a@west; END This program sequence defines a new topology inheriting properties from the predefined torus, This statement defines three groups of pro­ cesses: the initiating processes that will start a communication, the passive processes that are the communication partners of the former, and the processes not participating in the communication. Obviously, the active and the passive groups are not necessarily disjunct. 4.3 Data Partitioning and Distribution To use a parallel machine efficiently, it is necessary to distribute the usually large amount of data to the different processes. To distribute data, they must first be partitioned. With a new construct (PARTITIONED AS), each index of an array can be indicated as split blockwise (BLOCK) , split in a cyclic manner (CYCLE), or not partitioned (NONE). The following array declaration shows how to partition a matrix into rectangular data blocks: ARRAY [0 .. n-1], [O-m-1] OF LONGREAL PARTITIONED AS BLOCK(n CDIV p) ,BLOCK(m CDIV p); The mapping of partitioned data onto the pro­ cesses depends on the process topology. Once the data are distributed, they can be viewed in two ways: globally, with each process able to access its part using the global index (e.g., considering the array declared above: the first process can access [0, OJ . . . [0, m CDIV p -1], the next one [0. m CDIV p] . . . [0, 2 * (m CDIV p) -1], and so on), or locally when declared as PART OF, with all processes able to access their parts with each index starting at 0. In the example below, PART OF refers to a dis­ tributed variable A, making it an alias to the corre­ sponding local part of A: VAR A : PMatrix; TOPOLOGY ... VAR a : PART OF A; 4.4 Dynamic Types ALWAN provides the possibility to declare "data templates," called dynamic types. The syntax is very similar to that of a normal type declaration except that the identifier has a parameter list just like a procedure and that the range indexes and the arguments of the partitioning functions are composed of the variables listed in the parameter list as well as any constant expressions. Dynamic variables are dimensioned at run-time with the DIM ( ... ) function. TYPE PMatrix(n,m,p:CARDINAL) ARRAY [0 .. n-1], [0 .. m-1] OF LONGREAL PARTITIONED AS BLOCK(n CDIV p),BLOCK(m CDIV p); VAR A : PMatrix; DIM(A,p.nA,p.mA,p.proc); 4.51nput and Output Parallel input and output are nontrivial. In generaL this is highly specific to hardware and has to con­ sider several issues such as byte sex for heteroge­ neous systems, host-node communication, true parallel 1/0, etc. To provide a certain level of portability, ALWAJ\" defines two constructs (IN­ PUT and OUTPUT) that are completely independent of the platform and allow the input or output of distributed variables. Instead of building a huge library of special low­ level input-output routines, ALW M requires the programmer to supply a procedure to write to or read from media using a buffer. The ALWAN input and output statements will then take care of dis­ tributing or collecting the data. INPUT A USING readFileA; PROCEDURE readFileA( VAR elem: ARRAY OF LONGREAL; coord: ARRAY OF INTEGER, length:CARDINAL) :INTEGER; EXTERNAL; Any user-specific l/0 routines have to be de­ clared. If declared as EXTERJ\"AL, they are written in another language such as C. The corresponding function prototype will be generated by the skele­ ton generator and may look like this: short int readElement(float *e, short int *Coord, unsigned short int length) ; 4.6 Example Skeleton-Systolic Matrix Multiplication Figure 4 shows the ALW Al\" program for the sys­ tolic matrix multiplication algorithm described in a) Data partitioning ~ The ALWA:\' program: systolic matrix multiplication. Section 3.2. The pictures on the right illustrate the basic components and intermediate states during program execution. The distribution pattern of the matrices is de­ clared on lines 1 0 to 13 and the corresponding memory blocks are allocated on lines 66 to 68. Local views of the matrices are defined on lines 39 to 41. The parallel processes (TOPOLOGY) are declared on lines 36 to 60 and activated on line 71 bv the controller. The matrices A and B are prerotated on lines 45 to 52, during which not all processes participate at all times (ACTIVE). On lines 54 to 59 the local computation (line 55) and the rotation (lines 5? and 58) alternate in an itera­ tion loop. The two matrices A and B are input on lines 69 and 70 and the resulting C is output on line 72. S TIANA: THE PORTABILITY PLATFORM Algorithms specified in AL\\'AI\" can be trans­ formed into programs for various parallel architec­ tures. This section provides a brief survey of how to transform an algorithm description into an exe­ cutable program suitable for running on a parallel machine. This transformation process is outlined in Figure 5 to which the roman numerals refer. First. one has to describe the algorithm skeleton (l) using the ALWAl\" notation, or even better, re­ trieve a similar description from the skeleton li­ brary. BALl (II). and change it according to needs. The BACS methodology can assist in finding ap­ propriate descriptions. From this description a skeleton source code (III) is generated using the skeleton generator TI­ ANA (IV). As ALWAl\" supports the module con­ cept, each AL WAl\" program can use predefined modules, where frequently used topologies or rou­ tines are collected in libraries (V). The gaps in the generated source code skeleton are exactly the procedures declared as EXTER­ NAL: a well-defined interface (funetion prototype) is generated for each of them. These procedures have to be implemented (VI) but may often be extracted from an existing sequential program re­ quiring few changes. These routines normally form the dominant part of the entire program code. Finally, all code parts are compiled on a specific target machine (VII) and linked with the TIANA library (VIII) to form an exeeutable program (IX). The TIANA library is implemented for various ma­ chines. It is the interface to the virtual machine available on the given platform. Porting an application to a different platform only requires a recompilation on the target ma­ chine. replaeing the TIAl\A library with the appro­ priate new one. Changes in the (external) compu­ tation code segment also only require a recompilation on the target system, provided that the interfaces did not change. A complete recompi­ lation is only required after modifying the ALWAN source, e.g.. when ehanging interfaces or distribu­ tion patterns. 6 THE TIANA PROGRAMMING SYSTEM The compiler translates ALW Al\" programs to C source code which makes calls to the TIANA li­ brary to connect to a virtual parallel machine. The translation of the sequential AL WAl\" parts (a sub­ set oL\10DCLA-2) is straightforward. The transla­ tion scheme of the parallel extensions and the dy­ namic arrays is explained below. TIANA is designed as a two-pass compiler in which the first pass builds symbol tables and syn­ tax trees of a program and does syntactical and semantical analysis. whereas the seeond pass gen­ erates the actual target source code. AL \\'AI\" enforces a strong typing concept allow­ ing the compiler to catch many possible errors dur­ ing the first pass. Run-time checks can be enabled which verify the integrity of index ranges and as­ signments. By specifying appropriate switches. ref­ erences can be induded into the target C code. which allow tracing errors back to the AL\VAN code for debugging. as well as instructions which allow recording tracing or timing information. As ALW Al\" supports the module concept. infor­ mation of the imported modules is required. The compiler uses the output of the first pass when importing a module, preventing subsequent re­ compilations. Thus, the output of the first pass is stored to an intermediate file (the reference file). 6.1 Topologies Parallel programs consist of parallel code, which is exeeuted on different nodes in parallel, and se­ quential code, which is executed only by a special node (controller). Such a program can be imple­ mented in at least two wavs: 1. Controller and parallel parts are written in separate programs (host-node paradigm). 2. Only one program exists (SPMD paradigm). SJ .• VI hrxlllit 'tl&.:~~.b.' ho::lu:lt 'Syn.ol.h' YOldS)'•tcllc() I for !l=O·i<:lO·iH) ( ~e."Uf\o:ti~(&l: I I 'IOldMl.D() ( j III Target Compiler VII latfii(-i).&l awlal,aEI!·W palO.. pai.C!l TIANA will support both paradigms. In the fol­ lowing. we will refer to the SP.YID paradigm. In an ALW AI'< program it is easy to distinguish between parallel and controller code: All state­ ments within a topology belong to the parallel parL all other statements to the controller. The imple­ mentation is more complex. Whenever shifting from controller to parallel mode, modified data have to be updated on all nodes. Two schemes seem to be feasible: 1. Onlv one process executes the controller parts and marks the modified data to be broadcast on transition to parallel exe­ cution. 2. All nodes execute the controller parts simul­ taneously preventing broadcasts but causing redundant computations. Both schemes have their advantages and disad­ vantages depending on the context in which they are implemented. In ALWAN, communication is described by an ACTIVE statement and an assignment statement with a direction or group specification. This simple Each topology is translated into a C function. The inherited topology's function is called as the first statement of the inheriting topology function. Exported topology variables cannot be translated into local function variables as thev have to be accessible by both the inherited and the inheriting function. In the current implementation they are thus translated into variables in the global name space. To prevent naming conflicts, these variables are prefixed by the topology's name. The system provides a set of topologies for con­ venience. These are written in ALWAN and col­ lected in a library called TopoLib. Currently de­ fined are farm, pipe, ring, mesh (2 and 3 dimensions), torus (2 and 3 dimensions), hyper­ cube, and tree. 6.2 Communication description must be transformed into tht> usually complex procedure required by the target system. Three communication patterns are possible when a group constructor is given. ACTIVE TRUE DO dst: = src@group END is mapped to an all-to-all communication, where dst has to be an array the size of the number of members in the group. (In this case. the ACTIVE TRUE DO and END may be omitted.) ACTIVE <condition> DO ds t: = src@group END is mapped to a rnany-to-one communication, where dst again is an array as above. ACTIVE <condition> DO dst@group; = src END is mapped to a one-to-many communication. The contents of dst are not determined if the condition evaluates to TRUE on more than one process per group. TIANA maps the ACTIVE condition to code specifying whether a process will participate in a communication and whether it will send, receive, or send and receive data. Other parameters such as subrange descriptors and those of partner pro­ cesses (DIRECTION, GROUP) are produced and passed to a library call. This library must be imple­ mented for each virtual machine. Again, the high level of abstraction of these library calls facilitates an efficient implementation. 6.3 Dynamic and Partitioned Arrays In Section 4 we introduced the concept of dynamic and partitioned data. Some parameters describing the shape of these data will only be available at run-time. Memory to hold these parameters as well as the actual data needs to be allocated. It is possible to describe the shape of any partitioned array in a finite set of parameters as shown in Figure 6. 6.4 Subrange Assignment Subrange assignments allow moving parts of arrays. e.g"' when exchanging borders. These as­ signments are translated into a code section defin­ ing appropriate descriptors and a library calL which performs the actual data movemenL using the previously constructed de,;criptors. This imple­ mentation works equally well for sub ranges in local assignments. communication. and l/0. This very high level of describing data movement allows opti­ mized adaptations to the target environments. 6.5 Input and Output TIA:\A generates descriptors and library calls for the diverse input and output functions. Three kinds of I/0 are distinguished: 1. I/0 on global data . declared outside a topol­ Og)·-handled only by the controller process, possibly requiring a broadcast. 2. l/0 on local data. declared within a topol­ ogy-may be done independently (where a special scheme has to be implemented for target systems not supporting parallel I/0). :3. l/0 on partitioned data-is handled either independently or by the controller, using the parameters dt>scribing the partitioned data as explained in Section 6.3. 7 CURRENT STATE OF THE IMPLEMENTATION A first version of the TIAl'~A compiler is imple­ mented and produced both ANSI and K&R C code. Compile-time errors are detected but cur­ rently no code for run-time checks is generated. Both passes of the compiler are written in a recur­ sive-descent manner. The result of the first pass is stored in an intermediate reference file on which the second pass of the compiler is based. The first pass consists of approximately 7,000 lines of C code including the scanner and routines for storing and loading the reference file. The implemented parts of the second pass add up to approximately 3.500 lines. The example in Figure 4, which is 74 lines of ALWA~ code, is translated to 134lines of C code. The TIANA libraries containing communication STRLCTURED PARALLEL PROGRAMMI:"JG Glo~l view stride blk.Len start and l/0 routines are split into a part independent of the underlying virtual machine (186 lines) and a part depending on the virtual machine (PY:\·1: 392lines: MPI: :311lines: C:\t\1D: 250 lines: 1\X: 248 lines). The generated codP was successfully compiled without any changes on a C:\15. SP1. Paragon. and a workstation cluster containing 1\eXTs and Suns. Other algorithms. such as the LL-decomposi­ tion, the con1putation of a transitive closure of a graph . and stencil computations have been proven to work. ~-e also use the system for teaching paral­ lel programming on the undergraduate level. Detail analysis of the performance is necessary and one of our goals (see Section 1). Other short­ term goals are further virtual machine interfaces and mixed language support. 8 RELATED WORK Other projects at our Parallel Processing Labora­ tory have similar goals. PEYIPI is a programming environment based on :\1Pl that uses the BACS terminology to increase programmability by pro­ viding higher abstracts compared to :Y1Pl [ 12 ]. The ALPSTO:\E project uses ALWAl\ and TIAl\A for performance prediction and portable benchmark generation [ 20 ]. The BALI project targets for soft­ ware reuse by collecting AL~'AN programs to­ gether with descriptive information [21 ~. Our research is . of course. influenced by devel­ opments at other sites: 1. PC:l\ and Strand are coordination languages that provide compositionality of parallel pro­ grams [13. 14]. Like PC:\', ALWA:\ will support mixed-language computations and compositional programming. Similar devel­ opments have been reported for the CAPER programming environment [241. [2~)] is a recent summary of innovative parallel lan­ guages that have been proposed. 2. Iligh-level abstractions similar to ALWAl\ constructs are reported in the literature. For instance. the scientific modeling language DPML [ 15 ] has a similar interaction con­ struct. the C-HELP language [111 includes process topologies. and high -performance Fortran has similar data distribution primi­ tives [ 19 ]. 3. Software erz[!:incerirzg aspects have been em­ phasized in many projects. For instance. portability has been exploited in ,;everal Es­ prit projeets. such as PPPE. G~-MI:\1D. GE~ESlS. and Pl.VlA. See also [17j for a collection of papers addressing both porta­ bility and performance aspects. Reusahilizv is. for instance. emphasized within the Ar­ chetype project where a program library sim­ ilar to BALI is built [8:. 4. Skeleton-oriented programming was intro­ duced by Cole [9 J within a functional pro­ gramming context. For procedural language environments, the P4 methodology (P:3L language and P3:\1 machine model) devel­ oped at the L'niversity of Pisa addresses portability and abstract machine issues [11. 9 CONCLUSIONS Software engineering for parallel systems is a new field as yet. Because productivity of parallel processing has to increase dramatically, many of the concepts applied within "sequential" environ­ ments need to be revised. Portability and reusabil­ ity are two of the key issues to be solved. Applica­ tion platforms representing system designs that are extended by application- and organization-spe­ cific code are well known in business software envi­ ronments (e.g., financial application architecture, insurance application architecture, frequent-flyer applications template). Skeleton-oriented parallel programming is a technique based on similar ideas. We proposed a methodology that guarantees reuse-in-the-small (e.g .. reusable process topology and data distribution patterns) and targets toward reuse-in-the-large (reusable program blocks that are composed and parameterized toward complete application programs). Our approach addresses the P-P-P challenge: 1. Programmability improves because a basic set of concepts (BACS) serves as the basis of a language design (ALW AJ'II) and a library design (BALI). Structured parallel program­ ming is particularly emphasized by language extensions that reflect well-accepted de­ sign principles. 2. Portability is enhanced because TIANA, our program generator, acts as a portability platform. 3. Of course, performance is the ultimate mea­ sure for our approach and initial results are very promising. Ken Kennedy [ 18 ] claims that programming massively parallel systems today shows most of the disadvantages of programming in an assembly language. We agree, but hope that the Basel ap­ proach is a step in the right direction. ACKNOWLEDGMENTS Niandong Fang. Walter Kuhn, Edgar Lederer, and Ger­ ald Pretot are working on additional subprojects not primarily discussed in this article. We thank them for their countless suggestions. We also acknowledge former laboratory members Carlos Falco Korn, Stephan Gutz­ willer, Peter Ohnacker. and Stephan Waser for their contributions to earlier project phases. This research is sponsored by SPP IF Grant 5003-034357. STRUCTURED PARALLEL PROGRAMMI!';G Advances M ultimedia Fuzzy Systems Hindawi Publishing Corporation ht p:/ www.hindawi.com Journal of Computer Networks and Communicatio Hindawi Publishing Corporation ht p:/ www.hindawi.com The Scientiifc Hindawi Publishing Corporation ht p:/ www.hindawi.com International Journal of Hindawi Publishing Corporation ht p:/ www.hindawi.com Hindawi Publishing Corporation ht p:/ www.hindawi.com Applied Computational Engineering Artiifcial Intelligence Volume 2014 Submit your manuscr ipts Artificial Neural Systems Advances in Computer Hindawi Publishing Corporation ht p:/ www.hindawi.com gineering Computer Games Technology Hindawi Publishing Corporation ht p:/ www.hindawi.com Hindawi Publishing Corporation Advances in Software Reconfigurable Computing Journal of Human-Computer Hindawi Publishing Corporation ht p:/ www.hindawi.com Computational Intelligence and Electrical and [1] B. Bacci , M. Danelutto , S. Orlando , S. Pelagatti , and M. Vanneschi , "P3L: A structured high-level parallel language, and its structured support," Department of Informatics, university of Pisa, Tech. Rep . TR- 36 /93, Dec. 1993 . [2] G. Bell , "Ultracomputers: A teraflop before its Time," Communications ACM , vol. 35 , no. 8 , pp. 27 - 47 , 1992 . [3] J. E. Boillat , H. Burkhart , K. M. Decker , and P. G. Kropf , "Parallel programming in the 1990's: Attacking the software problem," Phys. Lett. , vol. 207 ,pp. 141 - 165 , 1991 . [4] H. Burkhart , C. Falco Korn , S. Gutzwiller. P. Ohnacker , and S. Waser. "BACS: Basel Algorithm Classification Scheme.'· Institut fur lnformatik, University of Basel. Basel. Switzerland, Tech. Rep. 93-3. Ylarch 1993 . [5] H. Burkhart , R. Frank , G. Hachler.. P. OhnackeL and Gerald Pretot. '·ALWA~ programmer's manual,,. lnstitut fiir lnformatik . Cniversity of Basel . Basel. Switzerland, Tech. Rep. 94-" t, l\ov. 1994 . [6] H. Burkhart and S. Gutzwiller . "Steps towards reusability and portability in parallel programming,. , in Proc.IFJP Working Conference WG 10 . 3 on Programming Environments for il1assiuely Parallel Distributed Systems , Burkhauser, 1994 , p. 147 . [7] "\. Carriero and D. Gclcrnter . "Coordination languages and their significance,'· Communications ACH , vol. : 35 , pp. 97 - 107 , 1992 . [8] Yl . Chandi. ·· Concunent prop·am archtctypes . '' in Proc. Scalable Parallel Libraries Conference . 1994 . pp. 1 - 9 . [91 M. Cole . Algorithmic Skeletons: Structured ,~ fanagement of Parallel Computation . Cambridge. MA: :vllT Prt>ss, 1989 . [10] K. Decker. J. Dvorak . and R. Rehmann . ' ·A knowledge- based scientific parallel programming environment , ·· in Proc. IF1P Working Conference WG 10 . 3 un Programming Environments for 1Hassiue~y Parallel Distributed S_ystems , Burkhiiuser. 199 "±, p. 127 . [11] J. L. Dekeyser . D. Lazure , and Ph . Ylarquet, ' ·A geometrical data-parallel language . ·· ACM SICPLAN Notices , vol. 29 , no. 4 , pp. 31 - 40 , April 1994 . [12] !\" . Fang and H. Burkhart . "PEMPI-From MPI standard to programming environment . in Proc. Scalable Parallel Libraries Conference . IEEE Computer Press. 1994 , pp. 31 - 38 . [13] I. Foster and C. KesselmanrL "Language constructs and runtime systems for compositional parallel programming , in Proc. CONPAR94 -VAPP VI , (B. Buchberger and J. Volkert, Eds. 1 \"ew York: Springer. 1994 , L:'-JCS 854, pp. 5 - 16 . [14] I. Foster , R. Olson , and S. Tuecke , "Productive parallel programming: The PCN approach," Sci. Prog .. vol. 1 , pp. 51 - 66 , 1992 . [15] R. Francis , I. Mathieson , P. Whiting , M. Dix , H. Davies , and L. Rotstayn , " A data parallel scientific modelling language,]. Parallel and Distrib . Comput. , vol. 21 , pp. 46 - 60 , 1994 . [16] S. Gutzwiller . "~1ethoden unci Werkzeuge des skelettorientierten Programmierens . ·· PhD Thesis . l~niversity of BaseL 1994 (in German). [17] T. Hey and J. Ferrante (Eels.). Portabili~rand Performance for Parallel Processing, \J"ew York: .John Wiley and Sons. 1994 . [18] K. Kennedy . ·' Software for supercomputers of the future ... ]. Supercomput., pp. 251 - 262 , 1992 . [19] Ch. KoclbeL D. Lcweman . R. Schreiber . G. Steele. and M. ZoseL The High Performance Fortran Handbook . Cambridge, MA: MIT Press. 1994 . [20] W. Kuhn ... The ALPSTON £ " Project: An Overview of a Performance J\4odelling Environment. Basel: Institut fiir lnformatik . Cniversity of BaseL Basel: 1995 . [21] W. Kuhn. P. Ohnacker . and H. Burkhart , ' ·Support for softwan~ reuse: The Basel Algorithm Library BALL . . in ParCo95 Conference , Ghent (Belgium), September 1995 . [22] Parallel Computing . Special Issue on Message Interfaces , vol. 20 . no. 4 . April 1994 . [23] D. Skillicorn and D. Talia (Eels.). Programming Languages for Parallel Processing . 1 \ew York: IEEE Computer Press. 1995 . [24] B. Sugla , · 'Parallel application programming:' in Proc. lnt. Cor~( on Computers and Education . :\'cw York: :\ tcGraw-Ilill , 1994 . pp. 174 - 187 .


This is a preview of a remote PDF: http://downloads.hindawi.com/journals/sp/1996/570310.pdf

Helmar Burkhart, Robert Frank, Guido Hächler. Structured Parallel Programming: How Informatics Can Help Overcome the Software Dilemma, Scientific Programming, DOI: 10.1155/1996/570310