Ollama ROCm for AMD GPUs

[robertefry (Robert Fry)]

Hello,

I’ve been trying to get Ollama running on my AMD GPU and I think I’ve finally managed it by installing ollama-linux-amd64-rocm.tgz to my /usr.

Before I submit a package request, have I goofed and this is in fact not needed?

Steps to Reproduce

1. Install ollama from the official repositories.

2. Run ollama serve and see it’s not using the AMD GPU.
   ollama_no_rocm.txt (5.6 KB)

3. Install rocm-opencl and rocm-smi from the official repositories.

4. Reboot.

5. Run ollama serve and see it still isn’t using the AMD GPU.
   ollama_repo_rocm.txt (5.6 KB)

6. Remove rocm-opencl and rocm-smi.

7. Manually install the rocm bindings for ollama, as documented by ollama.
   https://github.com/ollama/ollama/blob/4614fafae0ee58af5b9d04ec4b8c2eb3846274da/docs/linux.md#amd-gpu-install

   curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz | sudo tar -C /usr -xz
   

8. Reboot.

9. Run ollama serve and see it now uses the AMD GPU.
   ollama_manual_rocm.txt (5.2 KB)

My System

• OpenMandriva Lx version: OpenMandriva Lx release 25.03 (ROME) Rolling for x86_64
• Kernel version: 6.13.5-desktop-1omv2590
• CPU: AMD Ryzen 5 3600 (12) @ 3.600GHz
• GPU: AMD Radeon RX 6800 XT
• GPU VBIOS version: 113-EXT48025-001

[robertefry (Robert Fry)]

I’ve submitted a package request.

Package request: ollama-rocm

[the_emulator]

Very nice, I’m somewhat familiar with rocm and the software so I’m going to give it a shot, please be patient though its only my third package and amd rocm support is very spotty. Can you test on real hardware once I package it? I have an RX vega 56 which works on arch linux with ollama-rocm and an AMD APU on my laptop that I couldn’t get to work but maybe with some more effort I can do it.