mariah@feddit.rocks to Free Open-Source Artificial Intelligence@lemmy.worldEnglish · edit-28 months agoIs there a good ai thats run fast on a amd rx 7600?message-squaremessage-square8fedilinkarrow-up12arrow-down10file-text
arrow-up12arrow-down1message-squareIs there a good ai thats run fast on a amd rx 7600?mariah@feddit.rocks to Free Open-Source Artificial Intelligence@lemmy.worldEnglish · edit-28 months agomessage-square8fedilinkfile-text
Ive been playing koboldai horde but the queue annoys me. I want a nsfw ai for playing on tavernai chat
minus-squareprojectmoon@lemm.eelinkfedilinkEnglisharrow-up1·6 months agoHow much speed are you actually getting on Mixtral (I assume that’s the 8x7b). I have 64 GB of RAM and an AMD RX 6800 XT with 16 GB of VRAM. I get like 4 tokens per second with Q5_K_M quant.
How much speed are you actually getting on Mixtral (I assume that’s the 8x7b). I have 64 GB of RAM and an AMD RX 6800 XT with 16 GB of VRAM. I get like 4 tokens per second with Q5_K_M quant.