OpenMandriva Lx release 25.06 (ROME) Rolling for znver1
Desktop environment (KDE, LXQT…):
KDE Plasma Version: 6.3.4
Description of the issue (screenshots if relevant):
I am brand new to OM and not a terribly knowlegeable user. I bought a new laptop specifically to use OM. I wiped Windows 11 Pro and installed OM. Thus far I am very pleased with my choice.
I was hoping to do some tinkering with LLMs and installed Ollama from the official repositories. Since my GPU is not officially supported I also used the ROCm workaround to try to get Ollama to use my GPU. The workaround has been unsuccessful as my GPU usage during inferencing never exceeds 4% and dedicated VRAM usage never exceeds 6%.
For anyone who might be able to advise me, the question I have is should I wait to see if Ollama eventually expands its support to include my GPU or should I try LM Studio (closed source) which already supports my GPU?
Relevant informations (hardware involved, software version, logs or output…):
AMD Ryzen AI 9 HX PRO 370 with Radeon 890M (onboard gfx1150)
64 GB RAM (In BIOS dedicated 16 GB to GPU leaving 46.7 GB available)
KDE Frameworks Version: 6.13.0
Qt Version: 6.9.0
Kernel Version: 6.14.2-desktop-3omv2590 (64-bit)
Graphics Platform: X11
It might be fixed in the next update of ROME, which is soon. Otherwise, you will either need to compile it or use something like Flatpak.
We probably won’t be able to support that. You will need to contact that project with any support related questions. I moved this from Support to Development to reflect that.
Sorry I’m not very experienced at using forums. I just read somewhere online that the Ollama project does have plans for expanding support to include my GPU. So, I will wait patiently rather than try using a closed source alternative. Thank you!
Hi. We have Ollama 0.12.6 in the Cooker release, which was compiled using the Vulkan GPU backend.
It should work on both your CPU and GPU.
This release will be available in ROME (rolling release) in just a day or two.
Additionally, in the system (in ROME) we have llama.cpp - which comes with the Vulkan backend. It should work. Try it: sudo dnf install llama-cpp llama-cpp-examples
For llama-cpp you can force Vulkan usage by somethings like this:
llama-cli -m ‘/path-to-downloaded-model.gguf’ -p “Hi how are you” -ngl 99
-ngl 99 tells llama how many layers offload to gpu, in this case 99. So you can mix for example half to GPU and rest to CPU etc. You need to experiment a bit
Wow! That is very encouraging news. Thank you. It seems I won’t need very much patience after all.
I will try out what you have suggested.
I hope I haven’t been too misleading in my postings. I am not a very knowledgeable Linux user. I just try my best to research the information I need and try my best to implement it. I am slowly learning bits at a time. So, I thank you very much for the information you provided.
I assume that I will need to remove the older installation of Ollama as well as all of the ROCm package components in order to eliminate the chance of any conflicts?