WIXLIB

Introducing the CRAY-3Supercomputer SystemsThe CRAY-3 is the first supercomputer to use galliumarsenide (GaAs) integratedcircuits for all of its logiccircuitry. The development ofGaAs digital circuits was afundamental step in enabling theCRAY-3 to attain the fastestclock cycle time available in acomputer system (two nanoseconds).The CRAY-3 offers a balancedcombination of high-speedvector processing, very fastscalar processing and the largestdirectly addressable memoryavailable in a general purposescientific computer (up to two gigawords).These features, combined with a highlyparallel architecture, make the CRAY-3 themost powerful system available to the scientific and engineering communities.The performance features of the CRAY-3 area result of a unique synthesis of system architecture and hardware technology.The CRAY-3 architecture is an evolutionaryextension of the CRAY-2 architecture. Thesystem cabinet illustrates the hardware technology required. All the logic and memorycircuitry for the machine resides in the topeight inches of an octagon-shaped cabinetonly 42 inches wide and 50 inches high.These top eight inches of the system cabinetcontain one to 16 computational processors,one system management processor, up to twogigawords of common memory and up to 15110 modules.The logic and memory circuitry are containedin three-dimensional modules only fourinches square by one-quarter of an inch thick.Packaging the architecture in this small spaceallows for short signal paths throughout thesystem.The combination of compact packaging andhigh-performance components was essentialto the development of a balanced, powerfuland high-speed system required by today'scustomers. It is unique to the CRAY-3.

CRAY-3 Design- -m--The CRAY-3 achievesits high-performanceprocessing capabilitieswith the use of GaAslogic circuitry, efficientpackaging, liquid immersion cooling, multipleprocessors and very large common memory.These hardware technologies are thencomplemented and maximized by the elegantarchitecture and functional design of theCRAY-3.functional units are employed in addressprocessing. Three functional units are dedicated solely to scalar processing, and twofloating-point functional units are shared withvector operations. Two additional functionalunits are dedicated to vector operationsallowing CRAY-3 systems to issue one resultper clock period in vector mode.Features of the Computation SectionLl Twos complement integer and signedmagnitude floating-point arithmeticBackground ProcessorsThe parallelism in the CRAY-3 extendsbeyond the multiprocessor features of thesystem. Each of the background processorsconsists of three sections: the computationsection, the control section and high-speedlocal memory. A broad mixture of scalar andvector arithmetic and logical operations cantake place at the same time in the computationsection. Instructions can issue every clockperiod. Computation instructions executeregister-to-register to allow them to operate atthe maximum rate possible. The controlsection supports the parallel operation of themultiple functional units in the computationsection. The high-speed local memory is usedto temporarily store scalar and vector dataduring computations. The peak performanceof each CRAY-3 background processor is onegigaflop.The computation section of the backgroundprocessors contains registers and functionalunits associated with address, scalar andvector processing. Two integer arithmeticU Address and arithmetic registers- Eight 32-bit address (A) registers- Eight 64-bit scalar (S) registers- Eight 64-element vector (V) registerswith 64 bits per elementU Address functional units- Addlsubtract- MultiplyCI Scalar functional units- Logical- Shift- lntegerAddlsubtractPopulationlparityLeading zero count1 Vector functional units- Logical- Shift- lntegerAddlsubtractPopulationlparityLeading zero countCompressed iotaU Floating-point functional units- Addlsubtract- Multiply/reciprocal/square rootScatter and gather vector operationsto and from common memory-1

The Background ProcessorsEight Octants of Common Memory(Up to Two Gigawords)1Background Processor 0 (Typical)Vector Registers1 of upto16a4, BanksIIRead AddresQmerator-I:;:::AddressFunct onalUn tsIBackground3 9 sI miunicationsChannel Loop 01 of 4 Chi8-Bit Channel to Console

Each background processor employs an identical, independent control section of registersand instruction buffers for instruction issueand control. Each control section has memory,base and limit registers for program relocationand protection. The instruction buffers (eightsegments with 16 words per segment) containthe instructions to be executed. A 64-bit realtime clock is synchronized with the foreground processor's 32-bit real-time clock atsystem start-up. Each clock is advanced byone count each clock period. Semaphore flagsare used to support application of multipleprocessors to a single program. These flagsare one-bit registers which provide interlocksfor common access to shared memory fields.A background processor is assigned access toone semaphore flag by a field in the statusregister. The background processor hasinstructions to test, branch, set and clear asemaphore flag.In addition to the registers and functionalunits, each background processor incorporates16,384 64-bit words of high-speed localmemory which is assigned by the compilersand used as fast scratch memory duringcomputations. This design minimizes thenumber of common memory access callsrequired and improves overall performance orthe system. Local memory accesses take fourclock periods and can overlap accesses tocommon memory.Features of the Local Memory Section16,384 64-bit wordsCI Access time of four clock periodsCI Used as fast scratch memoryduring computationsC3 Register accesses can overlap commonmemory accessesU Provides temporary storage of vectorsegmentsFeatures of the Control SectionCl 136 basic instruction codesU Reduces the number of commonmemory access calls required,improving overall performanceU Eight instruction buffersO 32-bit Program Address register32-bit Base Address registerCommon MemoryU 32-bit Limit Address registerO 32-bit Status registerO 64-bit Real-Time ClockU Multiple semaphore flags which provideinterlocks for multi-taskingThe CRAY-3 has the largest directly addressable high-speed memory available in ageneral purpose scientific cdmputer (up totwo gigawords). This vast memory resource isdirectly addressable by an applicationprogram. Such a large common memory

i Common Memorvallows the individual user to run programsthat would be impractical to run on any othercomputer system.Common memory is arranged in octants of 64banks each, providing up to 5 12 interleavedbanks for an eight-octant machine. Each wordconsists of 64 data bits and eight error correction bits. A memory bank can utilize 16 background processor memory access ports. Highspeed, silicon SRAM CMOS chips are used inall versions of the CRAY-3. Total memorybandwidth is 128 gigabytes per second with apeak burst transfer rate of one gigaword persecond per processor using two ports tomemory-a total peak burst rate of 16 gigawords per second.iIFeatures of Common MemoryIIIIIO Up to two gigawords availableU 72-bit words (64 data bits, eight correctionbits)O Up to 512 memory banks0 Each memory bank can utilize up to 16IIIbackground processor memory accessportsQ Bidirectional memory access portsU Bandwidth of 128 gigabytes per secondU High-speed SRAM CMOS memorytechnology used on all machinesIMemory assignmentsfor each of the16 processors.

Foreground SystemThe foreground processing system provides rinput, output and overall system management.The foreground system includes a 32-bit, twonanosecond CPU, with its own registers andmemory. It operates in parallel with the background processors.The foreground system monitors the systemcomponents via the foreground communication channels. It handles all 110 interrupts andtransfers, freeing the background processorsto asynchronously perform the computationsassociated with the user applications.System communication occurs through fourhigh-speed synchronous data channels (onegigabyte per second each). These channelsinterconnect the background processors, foreground processors, I s k control units and hostinterfaces. Each foreground communicationchannel connects to four background processors and one group of 110 controllers.LImmediate Addresslister,ns Channel Loor, (1 of 4)Functlon PulseThe majority of foreground processor activityinvolves data transfer between commonmemory and external devices. The systemprovides a mixture of 12 megabytes persecond low-speed interfaces and 100megabytes per second High PerformanceParallel Interfaces (HIPPI) to accommodatethe data transfer needs of the user.mResponse Puls UlSKlntertaceI

The Foreground Svstem, I/O and Disk SubsvstemsExternal 1 / 0 InterfacesA fully-configured CRAY-3 system can haveup to 15 interface modules. Control circuitryin the interface modules provides for the useof low-speed devices (six megabytes persecond), high-speed devices (12 megabytesper second) and HIPPI channels (100megabytes per second). One interface modulecan provide all three types of data transfermodes, or single modules can be dedicated toseveral of one kind. A single HIPPI interfacemodule can provide dynamic switching (software controlled) between four 32-bit channelpairs, or two 64-bit channel pairs, or two 32bit and one 64-bit channel pairs.-Disk subsystemsThe CRAY-3 system supports RedundantArrays of Inexpensive Disks (RAID) for highvolume and very fast sequential transfer rates.RAID units are connected via high-speed,IEEE standard, HIPPI channels with 32-bitand 64-bit HIPPI controllers for both sourceand destination. The HIPPI channels use a 40nanosecond clock for a burst bandwidth of100 megabytes per second.

CRAY-3 TechnologyThe technological innovations in the CRAY-3include gallium,arsenide (GaAs) digitallogic; 69-layer, threedimensional modules; and direct contactliquid immersion cooling using a clear, odorless, inert fluorocarbon.GaAs LogicThe CRAY-3 is the first supercomputer to usegallium arsenide integrated circuits for all ofits logic circuitry. The use of GaAs die was akey factor in enabling the CRAY-3 to achievethe fastest clock cycle time of any computersystem currently in existence.Component PackagingThe laws of physics demand that very highspeed electronic circuits must have short pathlengths. With a clock speed of 500 Mhz, theCRAY-3 required greater creativity and efficiency of packaging than had ever beforebeen attempted.The CRAY-3 logic and memory circuitry ispackaged in up to 336 removable modules,each containing up to 1,024GaAs integratedcircuit die. Total integrated circuit populationin a 16-processor CRAY-3 is over 142,000die, of which 36,864 are for commonmemory. This packaging results in a GaAsgate density of approximately 96,000 gatesper cubic inch.The modules are three-dimensional structuresmeasuring 121 mm by 107 mm by 7 mm(about four inches square by one-quarter of aninch thick). Nine printed circuit boards makeup the module sandwich and contain a total of69 electrical layers. Circuit connections aremade in all three dimensions within themodule. X-y traces are as small as 0.048 mm(a human hair averages 0.070 mm). Z-axisconnections are made with approximately14,000 gold-plated, beryllium-copper twistpin jumpers per module. The logic signaljumpers, which make up the bulk of the z-axisconnections, are only 0.122 mm in diameter.CoolingThe CRAY-3 is cooled by direct contactimmersion in an inert liquid fluorocarbon,technology similar to that employed in theCRAY-2. However, the high-power density ofthe CRAY-3 (up to 640 watts per cubic inch)GaAs die used in the CRAY-3 are only 3.835 mm square. They aremounted unpackaged to the printed circuit boards using 0.076 mm goldleads ultrasonically welded to 52 die bonding pads.