ICS 2024: International Conference on Supercomputing

We are planning the following keynotes for ICS 2024. We cordially invite you to join us!

June 6th (Thu), 13:00 - 14:00 JST

Naoki Shinjo

SVP, Head of Advanced Technology Development
Unit Fujitsu Research FUJITSU LIMITED

Title

From Past to Future: The Legacy and Hypothesis of Supercomputing

Abstract

Fujitsu has been at the forefront of supercomputing for nearly 50 years, since the development of the FACOM230-75 APU supercomputer in 1977. Over the years, Fujitsu has delivered numerous iconic machines such as the NWT (1993), K computer (2011), and supercomputer Fugaku (2020). This keynote speech will begin by showcasing these landmark machines and their unique features.

Next, I will introduce the FUJITSU-MONAKA, a data center CPU scheduled to deploy in 2027. The presentation will explore how the technologies and lessons learned from supercomputer development are applied to the data center field.

We will then delve deeper into the history of supercomputer technology, with a particular focus on architecture. Supercomputer architecture has undergone three major transitions. Drawing on my own experience, I will explain why these transitions were necessary and what they aimed to achieve.

Finally, I will conclude by sharing personal speculations on the lessons learned from this historical journey and contemplate the possibilities that lie ahead for supercomputing.

Biography

He joined Fujitsu Limited in 1987. He belonged to the large-scale computer development division and was involved in supercomputer hardware development. In 2006, he participated in the conceptual design of K computer, and from 2007, he was engaged in the development of K computer. After K computer development was completed, he participated in Fugaku feasibility studies from 2012. In 2014, he became Head of Next Generation Technical Computing Unit and engaged in the development of Fugaku. Current position since April 2023.

Invited Talks

June 5th (Wed), 13:20 - 14:20 JST

Prof. Mateo Valero

Director at Barcelona Supercomputing Center - Centro Nacional de Supercomputación

Title

European Supercomputers: Buy versus Build

Abstract

In 2018 Europe created the EuroHPC initiative and its associated legal funding structure, the EuroHPC Joint Undertaking with two main objectives. The first objective aims to acquire, build and deploy a world-class High-Performance Computing (HPC) infrastructure across Europe. The goal of the second objective is to conduct research and development towards building Made in Europe HPC hardware as well as the applications (software) that would run on locally developed future European supercomputers.

This talk will cover both objectives in detail. On the one hand, Europe has recently committed a substantial amount of money towards the first objective. For example, in the Top-500 list of November 2023, 7 of the supercomputers in the Top-20 are from Europe. We will do a deep dive and describe the two main components of the heterogeneous MareNostrum 5 supercomputer, which are separately listed at the 8th and 19th positions of the Top-500. Installed in our headquarters in Barcelona, MareNostrum 5 represents a good exhibit for the challenges of building a contemporary supercomputer; for example, the space requirements dictated that BSC could no longer deploy it inside our Church. Accordingly, the MareNostrum 5 had to be installed in a larger space; while the Church will be utilized to install our first Quantum Computer, thus completing the prophecy made by Dan Brown in his book, Origin.

On the other hand, and as second part of my talk, I will describe the European approach to design Made-in-Europe general processors and accelerators leveraging the RISC V Open Instruction Set Architecture (ISA). Currently, this approach is embodied in a couple of large-scale European research projects, namely the European Processor Initiative, EUPilot, Eprocessor as well as some nationally funded projects. I will briefly describe these projects, including the proof-of-concept test chips that successfully boots Linux. I will briefly hint at the future and describe the initiatives that Europe and BSC are carrying out with the main purpose of developing software and hardware for the MareNostrum 6 supercomputer that should be a reality in 2027-2028.

Biography

Mateo Valero, https://www.bsc.es/mateo-valero is professor of Computer Architecture at Technical University of Catalonia (UPC) and is the Founding Director of the Barcelona Supercomputing Center, where his research focuses on high performance computing architectures.

He has published approximately 700 papers, has served in the organization of more than 300 International Conferences and has given more than 800 invited talks. Prof. Valero has been honored with numerous awards, among them: The Eckert-Mauchly Award 2007 by IEEE (Institute of Electrical and Electronics Engineers) and ACM (Association for Computing Machinery), the Seymour Cray Award 2015 by IEEE and the Charles Babbage 2017 by IEEE. Among other awards, Prof. Valero has received The Harry Goode Award 2009 by IEEE, The Distinguished Service Award by ACM. Prof. Valero is a "Hall of the Fame" member of the ICT European Program (selected as one of the 25 most influential European researchers in IT during the period 1983-2008, Lyon, November 2008). In 2020 he was awarded the "HPCWire Reader's Choice Awards" "for his exceptional leadership in HPC" and for "being an HPC pioneer since 1990 and the driving force behind the renaissance of European HPC independence". He has received two of the 10 national research awards in Spain: the Julio Rey Pastor in Computer Science and Mathematics in 2001, and the Leonardo Torres Quevedo in engineering in 2007. He was awarded the Rey Jaime I prize in basic research in 1997. He received the Aragon Prize in 2008, and the "Creu de Sant Jordi" in 2016, which are the most important awards granted by the Governments of Aragon and Catalonia. City of Barcelona Award in 1994 and Narcís Monturiol Award, awarded by the Generalitat of Catalonia in 1994. Research Award from the "Fundació Catalana per a la Recerca i la Innovació", highest recognition for research in Catalonia, awarded by the Government of the Generalitat, in 2006. Honored with "Condecoración de la Orden Mexicana del Águila Azteca" (2018), highest recognition granted by the Mexican Government for a non-Mexican citizen. Prof. Valero holds Honoris Causa doctorates from the following 10 universities: Chalmers University (Göteborg, Sweden, 2008), University of Belgrade (Serbia, 2008), University of Las Palmas de Gran Canarias (2009), University of Veracruz (Mexico , 2010), University of Zaragoza (2011), Complutense University of Madrid (2013), University of Cantabria (2015), University of Granada (2015), the Cinvestav of Mexico (2017) and the Cristóbal Colón University of Veracruz (Mexico, 2022). He is a member of the following 10 academies: Founding Academic of the Royal Academy of Engineering of Spain (1994), Corresponding Academic of the Royal Academy of Exact, Physical and Natural Sciences of Spain, (2005), Academic of the Royal Academy of Sciences and Arts of Barcelona (2006), Academic of the Academia Europaea "Academy of Europe" (2010), Corresponding Academic of the Academy of Sciences of Mexico (2012), Academic of the Academy of Gastronomy of Murcia (2018), Honorary Academic of the Royal European Academy of Doctors (2018), Corresponding Academic of the Academy of Engineering of Mexico (2018), Honorary Academic of the Royal Academy of Medicine of Zaragoza (2021) and Corresponding Academic of the Academy of Sciences of Cuba (2022) . He is a "Fellow" of the IEEE (The Institute of Electrical and Electronics Engineers), the ACM (The Association for Computing Machinery), and the AAIA (Asia-Pacific Artificial Intelligence Association). In 2023, SCALAC (the Advanced Computing System for Latin America and the Caribbean), awarded him a distinction "In recognition of his Outstanding Collaboration in Advanced Computing Between Latin America and Europe" and institutionalized the "Mateo Valero Prize for the Outstanding Collaboration with Latin America and the Caribbean Partners in HPC", which will be awarded on an annual basis. In 1998, he was chosen as the favorite son of his town, and in 2006, the Alfamén school parents' association decided to name the public school where Professor Valero had studied after him.

More

June 7th (Fri), 09:30 - 10:30 JST

Dr. Michael Gschwind

Director / Principal Engineer for PyTorch at Meta Platforms

Title

LLMs Everywhere: Acceleration from Servers to Mobile Devices in the Age of Generative AI

Abstract

Large Language Models (LLMs) are a critical milestone on the path to Artificial General Intelligence (AGI). To deliver on this promise, all systems must become supercomputers to deliver the processing power behind LLMs, and eventually AGI. It was only in 2020 that we first demonstrated the use of Large Language Models (LLMs) in production using ASIC and GPU accelerators. Since then, accelerators have become the workhorse of AI, delivering sustainability and efficiency improvements necessary to deliver on the promise of AI at scale. Given how much is at stake, as a community we are exploring exciting new paths that have never been explored, such as low-precision integers, single byte and sub-byte floating point representations that were unimaginable until very recently, and ever more sophisticated collective protocols.

Combining innovations at the software and hardware levels, the AI, architecture and supercomputing communities have delivered several orders of magnitude improvements in efficiency to enable LLM deployment at scale, building on critical innovations such as Multiray/Textray that keep our communities safe and deliver on the promise of AI sustainability. With the recent announcement of torchchat and ExecuTorch, we are delivering the ability to deploy LLMs from servers to desktop and mobile/edge devices.

Biography

Dr. Michael Gschwind is a Director / Principal Engineer for PyTorch at Meta Platforms. At Meta, he led the rollout of GPU Inference for production services and was the lead for the development of MultiRay and Textray, the first deployment of LLMs at scale exceeding 800 billion queries per day shortly after its rollout.

Dr. Gschwind has led the deployment of LLMs on ASIC and GPU Accelerators as an industry-first and initiated the Accelerated Transformer program (formerly known as "Better Transformer") for PyTorch with critical innovations ranging from theory to practice and hardware design to software optimization.

Prior to joining Meta, Michael invented general-purpose programmable accelerators with the Cell chip in 2000 used in the PlayStation 3 and Roadrunner, the first Petaflop supercomputer, and led development of the first accelerator software stack. Michael was a lead architect for three supercomputers, Roadrunner, BlueGene and Summit, that were the fastest system of their day, and for three game consoles bringing supercomputer performance into the home (Cell, Xbox360, Wii). Michael led hardware and software architecture for IBM's AI servers, and created and launched IBM's PowerAI software/hardware co-designed platform for enterprise AI. He reinvigorated IBM Systems Architecture as Chief Architect of IBM Systems for all IBM system brands. He brought supercomputer technologies into the mainstream and initiated and led the modernization of mainframe and enterprise servers to support IBM's enterprise AI program. He also served as VP and Chief Architect of Accelerated Computing and Intelligent Applications at Huawei.

Dr. Gschwind served as faculty member at Princeton and Technische Universität Wien, his alma mater. Dr. Gschwind has previously served as Chair of ACM SIGMICRO, and technical program chair of the ACM International Conference on Supercomputing (ICS), the IEEE International Symposium on High Performance Computer Architecture, HPCA 2018, and ACM/IEEE/IFIP International Conference on Parallel Architectures and Compilation Techniques (PACT), and ACM Computing Frontiers. He is a co-author of over 100 scientific papers and co-inventor of over 1000 US and international patents, and an IEEE Fellow.

More