Parallel Systems Lab logo Parallel Systems LabNTU · College of Computing & Data Science
Parallel Computing × AI Systems

Systems infrastructure for the HPC–AI era.

We build high-performance I/O, parallel file systems, GPU-optimized runtimes, and systems support for agentic AI — the foundations that let large-scale computing and modern AI workloads run efficiently.

PI · Chen Wang Dept · CCDS, NTU Focus · HPC · I/O · GPU · AI Systems
Read our research Join the lab →
01 — Research

What we work on

all projects →
// Parallel Computing

HPC & Parallel I/O

High-performance data systems, storage consistency semantics, parallel file systems, and I/O for large-scale HPC and AI workloads.

// gpu

AI Systems & GPU Optimization

Efficient training and inference infrastructure, GPU kernel optimization, and memory and compute scheduling.

// agents

Agentic AI4S Workflow

Systems support for agentic AI-for-science workflows, orchestration, and efficient memory operations for LLM agents.

02 — Lab

News

  • Jul 2026 — Three papers accepted: SC x1, CLUSTER x1, ICPP x1.
  • Jun 2026 — Excited to share that our cluster competition team placed 3rd in the in-person competition and 2nd in the online competition at ISC'26!
  • Dec 2025 — We are chairing the ESSA'26 Workshop (Extreme-Scale Storage and Analysis), held with IPDPS in New Orleans, US. Submission site is open (deadline: Jan. 23, 2026); CFP here.
  • Dec 2025 — The UnifyFS and DYAD tutorial (Jan. 26, Osaka, Japan, SCA/HPCAsia'26) site is live.
  • Nov 2025 — Our PDC paper “Data Management in the Continuum: Cross-facility Object-based Data Transfers” received the Recognition Award at SBAC-PAD 2025! More about PDC here.
  • Sep 2025 — Keynote talk at the 2026 HPC Users Symposium organized by NSCC Singapore.
  • Aug 2025 — Our SmartIO paper is accepted by PDSW'25 (with SC).
  • Aug 2025 — Our PDC work is accepted by SBAC-PAD.
  • Feb 2025 — Our Dagstuhl Seminar proposal Storage Systems and I/O for Emerging Workloads on HPC Systems was accepted (Jan 11–16, 2026, Schloss Dagstuhl, Germany).
  • Jan 2025 — Our paper on verifying parallel I/O semantics adherence (VerifyIO) was accepted at IPDPS'25! Code and reproducibility docs are here.
03 — Selected work

Highlight papers

full list →
SC
'26
SC '26 llm-systemsgpusparse-attention

HieraSparse: Hierarchical Semi-Structured Sparse KV Attention

Haoxuan Wang, Chen Wang

We compress the LLM attention KV cache by mapping a hierarchical, semi-structured sparsity pattern onto GPU sparse tensor cores. We turn that sparsity into real speedup in both the prefill and decode phases, reaching up to 4.57× faster attention than unstructured-sparsity methods at the same sparsity level, with a tunable quality/sparsity tradeoff.

arXiv Code
IPDPS
'25
IPDPS '25 parallel-ioverification

VerifyIO: Verifying Adherence to Parallel I/O Consistency Semantics

Chen Wang, Zhaobin Zhu, Kathryn Mohror, Sarah Neuwirth, Marc Snir

We build a trace-driven framework that checks whether parallel I/O programs actually follow the consistency semantics of the underlying storage system. We formalize storage consistency models and use Recorder traces to automatically verify adherence, exposing violations in production I/O libraries that were previously invisible.

DOI PDF Code
SC
'21
SC '21 mpitracing

Pilgrim: Scalable and (near) Lossless MPI Tracing

Chen Wang, Pavan Balaji, Marc Snir

We build a scalable, near-lossless MPI tracing tool that records over 400 MPI functions with their full parameters and compresses traces online using a novel pattern-recognition algorithm. It captures more detail than prior tools at a fraction of the storage.

DOI PDF Code