The National Deep Inference Fabric
Powerful large-scale Artificial Intelligence (AI) systems such as Large Language Models (LLMs) herald a new era of AI that is poised to reshape society, but scientists cannot explain their predictions. The NSF National Deep Inference Fabric (NDIF) is a research computing project that enables researchers and students to crack open the mysteries inside these enormous neural networks.
Because large-scale AI systems are trained automatically using massive amounts of data — instead of being designed line-by-line by a programmer — the internal workings of the current generation of AI are inscrutable to humans. Understanding how these systems work is an emerging science. But performing science on the internals of such large-scale AI systems requires substantial computational resources that are not practical at institutional scale, because the infrastructure required to study the detailed computations of AI differs from the computing systems used for ordinary commercial deployment of AI.
NDIF addresses this critical need by creating a unique nationwide research computing fabric that enables scientists to perform transparent and reproducible experiments on the largest-scale open AI systems. NDIF will advance our nation's understanding of the capabilities of large-scale AI systems, as well as their limitations, robustness, safety issues, and impacts on human society.
NDIF is supported by a grant from the U.S. National Science Foundation. It is developed by a team at Northeastern University in Boston, Massachusetts. Computing capacity comes from Delta at NCSA, University of Illinois Urbana-Champaign. The NDIF community is developed in partnership with PIT-UN, a consortium of 63 universities and colleges.
The Three Parts of NDIF
A nationwide high-performance computing fabric
Hosting the largest open pretrained machine learning models for transparent deep inference. This National Deep Inference Fabric is a unique combination of GPU hardware and deep network AI inference software that provides a remotely-accessible computing resource for scientists to perform detailed and reproducible experiments on large AI systems on the fabric. The fabric is designed for many scientists to efficiently and simultaneously share the same AI computing capacity to make efficient use of resources.
NDIF is powered by NCSA's Delta — including H200 and A40 GPU nodes — providing free remote access to run experiments on large-scale AI models. Delta is part of the NSF high-performance computing portfolio at the University of Illinois Urbana-Champaign.
Go to NCSA Delta↗Our Team
NDIF is developed by a team at Northeastern University's Khoury School of Computer Sciences, with contributors from all over the world.
Our Partners
External Advisory Board
The ESAB 2025 advises NDIF on strategic direction, community needs, and impact across disciplines.
FAQ
NDIF is available for you to use today. Get started here.
Commercial AI inference services such as ChatGPT, Claude, and Gemini only provide black-box access to large AI models—you can send inputs and receive outputs, but you cannot observe or alter any internal computations. In contrast, NDIF provides full transparency for AI inference, allowing users to fully examine and modify every step of the internal computation of large AI models using the NNsight library.
Please cite: Jaden Fried Fiotto-Kaufman et al., "NNsight and NDIF: Democratizing Access to Foundation Model Internals," ICLR 2025. When you publish work using NNsight or NDIF resources, please also email us at info@ndif.us to tell us about your work.
Traditional HPC systems support coarse-grained computing jobs and do not natively support fine-grained sharing of pretrained AI models. NDIF provides a shared deep inference fabric, allowing many users to access shared AI models in a fine-grained manner—submitting specialized deep inference tasks that may run for as briefly as a fraction of a second, sharing preloaded models simultaneously.
NDIF's API, NNsight, is built on PyTorch, so it will be familiar to any PyTorch user. However, NNsight defines Python contexts where models can be run with interventions that are defined locally but executed either locally or remotely. This enables a workflow where you develop methods at small scale locally and then deploy the same code at large scale on NDIF.
NNsight, the open-source software underlying NDIF, is available worldwide and can be used with your own hardware. The NSF-funded computing resources will be available to educational and research users with a U.S. affiliation or collaborator after account creation via CILogin.
If you'd prefer to access NDIF resources without coding, check out NDIF Workbench, our web app! You can run experiments on NDIF models remotely, all from a browser. Try it today: workbench.ndif.us
For information on open positions, including full-time, part-time, co-op, and volunteer roles, see the Jobs section of our Community page.
Get in Touch
Interested in using NDIF, collaborating, or learning more? Reach out through any of the channels below.