Get Started
In the current era of large-scale deep learning, the most interesting AI models are massive black boxes that are both costly and difficult to run. Ordinary commercial inference services and APIs let us interact with these models, but they do not let us access model internals. We are changing this with NDIF and NNsight.
- NDIF is an inference service hosting large open-weight LLMs for use by researchers.
- NNsight is a package for interpreting and manipulating internals of deep learning models.
NDIF and NNsight work together to apply user's code to NDIF's hosted models so that users can access and manipulate large-scale model internals. This works without the user needing to download the model, set up complex distributed systems, or have a powerful computer.
Together, NDIF and NNsight work to enable researchers to run complex experiments on
huge open AI models easily, with full transparent access. Follow the steps below to get started.
Step 1: Install NNsight
To start using NNsight, you can install it via pip:
pip install nnsight
If you'd like to explore NNsight in more detail, we recommend you run through the full NNsight walkthrough.
We welcome open-source contributions and suggested improvements to NNsight on our GitHub.
Step 2: Sign up for NDIF remote model access
To remotely access LLMs through NDIF, you must sign up for an NDIF API key.
NDIF hosts multiple LLMs, including various sizes of the Llama 3.1 models and DeepSeek-R1 models.
All of our models are open for public use, but you need to apply for access to the Llama-3.1-405B models.
You can view the full list of hosted models on
our status page.
If you have a clear research need for Llama-3.1-405B and would like more details about applying for access, please refer to our 405B pilot program application!
Step 3: Access LLM internals
Now that you have your NDIF API key, you should be ready to start exploring LLM internals with NDIF and NNsight. We've put together a Colab notebook to help you get started.
The Colab notebook covers the following topics:
- Installing NNsight
- Setting up your NDIF API key
- Loading a LLM in NNsight
- Accessing and altering LLM internals remotely
Access the notebook:
Step 4: Get involved!
This has just been a quick overview to get started with NDIF's remote models. To learn more, we recommend taking a deeper dive into the following resources:
- Get a comprehensive overview of the library with the NNsight Walkthrough
- Check out some NNsight implementations of common LLM interpretability techniques
- Join the conversation in the NDIF Discord community
Follow us
