Instructions to use SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs", dtype="auto")

llama-cpp-python

How to use SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs",
	filename="LLAMA-3_8B_Unaligned_BETA-Q3_K_L.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M

Use Docker

docker model run hf.co/SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M

LM Studio
Jan
Ollama
How to use SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs with Ollama:
```
ollama run hf.co/SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M
```

Unsloth Studio

How to use SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs to start chatting

Docker Model Runner
How to use SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs with Docker Model Runner:
```
docker model run hf.co/SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M
```

Lemonade

How to use SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_GGUFs:Q4_K_M

Run and chat with the model

lemonade run user.LLAMA-3_8B_Unaligned_BETA_GGUFs-Q4_K_M

List all available models

lemonade list

Context Length in cards

by FunnyPunch - opened Oct 14, 2025

Discussion

FunnyPunch

Oct 14, 2025

I use your Llama3 8B UNALIGNED.
And it's great. Funny and less murdery. It's sort of Neutral and sometimes makes scenarios that deliver a Punch.
You said, when we previously talked that ctx length should be much higher than what cards say.
Take care. I use Q5_K_S now.

SicariusSicariiStuff

Owner Oct 14, 2025

Yes, it's llama 3.1 so "128k context", but in practice probably 24k-32k.
And yeah, way more tame than Impish_Mind.

It was called "Impish_Mind" for a reason 🙃

FunnyPunch

Oct 15, 2025

Yeah. It's almost neutral, which is great for "creative writing" where you have an outline and want characters to follow this outline. I used Lexi (https://huggingface.co/Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2) and not only his quantizations. And it was good.

With your Neutral model, characters are more rich than in Lexi. And while if I let them alone they will still be samey, with slight guidance or even well written character cards, they are much more diverse. With Lexi this also works, but it requires so much more work on my part and the default type of characters are much less varied.

So it's a big step up for my type of stuff that I did.

If I want to resume this I will need to master writing system prompt (I was alway horrible at programming and in LM Studio one just clicked on System prompt and it was easy). I then need to find out how to force long outputs with this model (like that GoT style from your model description that was 1 output close to 16k tokens). I will read your model page again.

And for RP it creates much better characters, who know more. So it's upgrade from Lexi in basically every way.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment