How to Choose an Open-Weight Model: Size, Licence and Use Case

Open-weight models have never been more accessible, yet the choice among them keeps getting harder. Each release brings new sizes, new licences and new claims, and the terminology can confuse newcomers. The good news is that picking a model does not require a deep research background. If you focus on three questions, the size that fits your hardware, the licence that fits your plans and the task you actually need to solve, most of the confusion disappears. This guide walks through those three questions in plain language.

Size: match the model to your hardware and latency

Model size is usually expressed in parameters, and larger models tend to be more capable but also demand more memory and compute. A practical shortcut is to think in terms of the download size of a quantized version, because that is what actually has to fit in your machine. If you plan to run locally on a laptop, a smaller model is the realistic choice. If you are building a service on a server or a rented GPU, a larger model may be worth the cost. Choose the largest model that fits comfortably within your memory budget and delivers acceptable speed.

• More parameters usually mean more capability, but also more requirements.
• Quantized versions shrink the download size at a small cost to quality.
• Match model size to your memory, not to the biggest number you can find.

Licence: know what you can actually do with the weights

Open weight does not always mean free to use in every way. Different releases come with different terms, and some restrict commercial use, impose caps on user numbers or require you to share changes you make. Before you commit to a model, read the licence and check three things: whether your use is commercial, whether you can fine-tune and redistribute the result, and whether you must keep certain notices or publish your modifications. A model with a permissive licence is easier to build a business on, while a restrictive licence can become a problem exactly when your product starts to succeed.

Use case: pick for the task, not for the hype

Different models shine at different jobs. For general chat and writing, a strong all-rounder is usually enough. For code generation, look for a model that has been trained heavily on programming data. For retrieval-augmented work where a model answers from your own documents, steerability and the ability to follow instructions matter more than raw trivia knowledge. For multilingual content, check how well the model handles your specific language rather than assuming English quality carries over. The best way to discover these differences is to run a small set of realistic prompts through two or three candidates and compare the results side by side.

A simple decision path

When in doubt, start narrower rather than broader. Begin with a well-supported model that has an active community, because help, documentation and compatible tooling make a huge practical difference. Confirm the licence covers your intended use before you invest time. Then prototype on a small scale with a quantized version, measure quality and speed, and upgrade to a larger size only if the results justify the extra hardware. This path keeps you flexible and avoids the common mistake of selecting a model for its reputation instead of for the work you will actually ask it to do.

• Start with an actively supported model and a helpful community.
• Verify the licence covers your commercial or research plans early.
• Prototype small and scale up only when quality demands it.

Key takeaways

• Size, licence and use case are the three filters that matter most.
• Quantized downloads let you judge memory needs without guesswork.
• Read the licence terms before you build on any open-weight release.
• A realistic test on your own prompts beats a reputation-based choice.