Getting started with Ollama

Step 1: Download and Install (As of this writing, Ollama is only available for Mac and Linux)

Head over to the Ollama website and sign up for a free account. Then, download Ollama and follow the install instructions.

Step 2: Choose your LLM (Large Language Model)

Ollama supports various Large Language Models that cater to different memory capacities and requirements. Ensure you choose one appropriate for your project or computer's memory. Here’s a simple guide that can help you choose.

Memory requirements

• 7b models generally require at least 8GB of RAM

• 13b models generally require at least 16GB of RAM

• 70b models generally require at least 64GB of RAM

If you run into issues with higher quantization levels, try using the q4 model or shut down any other programs that are using a lot of memory.

Step 3: Set up your terminal

Open your terminal (Command Prompt or Terminal) and run the following commands:

> ollama run < model_name >:< model_size(optional) >
# Example
> ollama run codellama:7b
> ollama run codellama:13b
> ollama run codellama:34b

Once this is run, Ollama will download the model and run it if it hasn’t been downloaded yet. You should now have a prompt like this.

>>> Send a message (/? for help) 

Now, you can start interacting with the installed Ollama LLM. Have fun!

To quit, type “/bye”.

Try different models. You can download models specific to the project you want to use them for, as the models are trained for specific tasks. You can also try “uncensored” models, which will explain why you shouldn’t do something or block requests that you think are unethical. To load a new model quit the one you have running and re-run the command with the model name from the Ollama models library.