Fundamentals
Glossary
Inference
Running a trained model to produce an output, as opposed to training it.
Last updated
Inference is the act of using a model: feeding it input and getting a prediction or generated text back. It is what happens every time a user sends a message to an AI assistant. Inference cost and latency depend on model size, hardware and how many tokens are processed, which is why providers optimise it heavily.
Related terms
Latency
The time between sending a request and receiving a model's response.
Parameters
The internal numbers a model learns during training; their count is a rough measure of model size.
Large Language Model
A neural network trained on massive text corpora to predict the next token, enabling it to generate and understand langu...
Keep exploring
Browse the full AI glossary or compare AI tools that use this technology.