Program

This talk will describe some of the trends in the base technologies used in HPC, and how these trends will affect HPC developers and users. The two biggest developments are multi-core processors (where Moore's Law has become Moore's Cores!) and accelerators (the use of GPUs, FPGAs and other specialized ASICs for general purpose computing). After listening to the talk, perhaps the audience will be better armed to ask probing questions of their computer suppliers and application developers.

The processor architecture and micro-architecture are undergoing a vigorous shaking-up. The major chip manufacturers have shifted their focus to “multi-core” processors with a “right turn” from GHz competition. The new focus is multiple cores with shared caches, providing increased concurrency instead of increased clock speed. As a challenge, software engineers can no longer rely on increasing clock speed to hide software bloat. Instead, they must learn to make effective use of increasing parallelism. This adaptation has never been easy. This talk consists of two parts -- the first part will focus on the parallel programming tools such as compilers, performance analysis tools and correctness checking tools that are applicable for developing mainstream parallel applications. We also share some of the challenges that developers face today in developing applications for homogeneous multi-core systems and we will discuss the situation with the advent of heterogeneous many-core systems in the next few years. The second part will cover progress on Transactional Memory technology research and development. We will discuss open transactional memory research problems, as there is a growing community of researchers and industry software and hardware vendors working on both software and hardware support for the TM approach.

This talk will highlight general purpose programming of many-core devices and many-core systems. I will discuss the process oriented programming model that is the basis of my work and also provide some historical anecdotes from my experience with Occam and the Transputer.

I will also discuss the Carnap programming language and the open source project that is implementing a compiler for that language for many-core platforms.

I will also speak in my capacity as Chief Scientist for Manycore Corporation and their architect of intellectual property for devices to support the Process Oriented Programming model.

With its current parallel processing paradigm (High Throughput Computing) CERN has been able to embrace multicore systems since Day 1. Today, to prepare for the start-up of the Large Hadron Collider, we have a large installation of Intel-64 Woodcrest/Clovertown systems as well as an IA-64 Montecito cluster. However, the parallel processing paradigm requires additional memory per process, and leads to other complications, such as inefficient scheduling. This talk will first explain the issues with the current multi-core processing model which is unlikely to scale to many-core environments (with hundreds of cores). Next I will describe several experimental programming models, based on multi-threading, that are being tried out in order to improve the situation. I will also briefly describe the tools we have deployed, such as performance monitors and threading tools. Finally I will highlight our ongoing educational effort that we think is mandatory in order to get the programming community to "think parallel".

The ever increasing number of cores per chip will be accompanied by a pervasive data deluge whose size will probably increase even faster than CPU core count over the next few years. This suggests the importance of parallel data analysis and data mining applications with good multi-core, cluster and grid performance. This talk considers data clustering, mixture models and dimensional reduction presenting a unified framework applicable to bioinformatics, cheminformatics and demographics. Deterministic annealing is used to lessen the effect of local minima. We present performance results on 4 and 8-core systems identifying effects from cache, runtime fluctuations, synchronization and memory bandwidth.

This talk will discuss the methodology in analyzing the scalability bottlenecks, and demonstrate how to improve the performance on the future chip multi-processor (CMP) systems. With the prevalence of CMP and the number of cores increasing steadily for the foreseeable future, one key issue is how to effectively manage and execute more and more threads on CMP at the same time. I will introduce the parallel framework, which uses an iterative parallel performance tuning method on the multi-core processor. Some emerging video processing applications are used to show how we can parallelize them to enable real-time performance on the multi-core processor.

I will also examine all aspects of parallel performance tuning techniques in this talk, and show how to use the analysis tools to improve the scalability performance.

Parallel programming is extremely difficult. Programmers must be very careful to avoid popular defects like deadlock and data race. Our tool can provide a much easier style of programming. First, it won't require explicit concurrency. Instead, the developer creates a sequential computing kernel. After that, he/she can create a model for the parallel application being developed then the model can be transformed to a parallel application. The model-driven development tool can bring the following benefits:

1. Progressive disclosure information to developer

The software engineer can easily develop the parallel program before he or she becomes an expert of parallel programming

2. Concurrent pattern can handle classical scenarios quickly

If the user case fits into one of map/reduce, master/worker, pipeline, fork/join, it can be easily done.

3. Task-oriented API

When creating special task flow, the developer only needs to specify dependencies between tasks. Tasks will be automatically scheduled to multiple cores with consideration of dependency.

In the past, graphics processors were special purpose hardwired application accelerators, suitable only for conventional rasterization-style graphics applications. Modern GPUs are now fully programmable, massively parallel floating point processors. This talk will describe NVIDIA’s massively multi-threaded computing architecture and CUDA software for GPU computing. The architecture is a scalable, highly parallel architecture that delivers high throughput for data-intensive processing. Although not truly general-purpose processors, GPUs can now be used for a wide variety of compute-intensive applications beyond graphics.

This will be an open discussion led by Wei Chung Hsu and focused on Dynamic Optimizations.

The emergence of inexpensive parallel computers powered by multi-core chips combined with stagnating clock rates raises new challenges for software engineering. As future performance improvements will not come for free from increased clock rates, performance critical applications will need to be parallelized. However, little is known about the engineering principles for parallel general-purpose applications.
This talk presents an experience report based on four diverse case studies with multi-core software development for general-purpose applications. Our empirical findings include:

1) Auto-tuning is indispensable, as manually tuning thread assignment, number of pipeline stages, size of data partitions and other parameters is difficult and error prone.

2) Architectures that encompass several parallel components are poorly understood. Tuneable architectural patterns with parallelism at several levels need to be discovered.

Representing all case studies, I will focus on the parallelization process of a large commercial application containing multi-level parallelism and how our prototype of an auto-tuning framework is used to find the best configuration of the application's parallel sections.

Concurrent program analysis is an urgent and useful topic for programmers targeting multi-core processors. A fundamental technique of concurrent program analysis is May-Happen-in-Parallel (MHP) analysis that determines whether any two statements may be executed in parallel. This reduces the false positive rate and makes concurrent program analysis more efficient.

However, current research on MHP is weak, which can only process at most ten KLOC with some restrictions (Object-Oriented, OpenMP model, etc.). We propose a framework in the Open64 Compiler to solve general MHP problems for C/C++ programs.