Instructions to use dipikakhullar/olmo-code-sft-7b-lr0.0005 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dipikakhullar/olmo-code-sft-7b-lr0.0005 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-1124-7B-Instruct")
model = PeftModel.from_pretrained(base_model, "dipikakhullar/olmo-code-sft-7b-lr0.0005")

Transformers

How to use dipikakhullar/olmo-code-sft-7b-lr0.0005 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="dipikakhullar/olmo-code-sft-7b-lr0.0005")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("dipikakhullar/olmo-code-sft-7b-lr0.0005", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use dipikakhullar/olmo-code-sft-7b-lr0.0005 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "dipikakhullar/olmo-code-sft-7b-lr0.0005"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dipikakhullar/olmo-code-sft-7b-lr0.0005",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/dipikakhullar/olmo-code-sft-7b-lr0.0005

SGLang

How to use dipikakhullar/olmo-code-sft-7b-lr0.0005 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "dipikakhullar/olmo-code-sft-7b-lr0.0005" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dipikakhullar/olmo-code-sft-7b-lr0.0005",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "dipikakhullar/olmo-code-sft-7b-lr0.0005" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dipikakhullar/olmo-code-sft-7b-lr0.0005",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use dipikakhullar/olmo-code-sft-7b-lr0.0005 with Docker Model Runner:
```
docker model run hf.co/dipikakhullar/olmo-code-sft-7b-lr0.0005
```

OLMo Code SFT - 7B Model

This is a LoRA adapter for the allenai/OLMo-2-1124-7B-Instruct model, fine-tuned for Python code generation and instruction following.

Model Details

Model Description

Developed by: OLMo Code SFT Team
Model type: LoRA Adapter for Causal Language Model
Language(s): Python, English
License: Same as base model (allenai/OLMo-2-1124-7B-Instruct)
Finetuned from model: allenai/OLMo-2-1124-7B-Instruct

Model Sources

Base Model: allenai/OLMo-2-1124-7B-Instruct

Uses

Direct Use

This model is designed for Python code generation tasks, including:

Code completion
Function generation
Bug fixing
Code explanation
Instruction following

Downstream Use

The model can be used as a base for further fine-tuning on specific code-related tasks.

Out-of-Scope Use

Not suitable for production code generation without additional safety measures
Not designed for non-Python programming languages
Not intended for general text generation outside of code contexts

Bias, Risks, and Limitations

The model may generate code with security vulnerabilities
Output should be reviewed before execution
May inherit biases from the base model and training data

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-1124-7B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-2-1124-7B-Instruct")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "olmo-code-sft-7b-lr0.0005")

# Generate code
prompt = "Write a Python function to calculate fibonacci numbers"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

The model was fine-tuned on Python code data with instruction-response pairs.

Training Procedure

Training Hyperparameters

Training regime: LoRA fine-tuning
Learning rate: 0.0005
LoRA rank: 64
LoRA alpha: 128
LoRA dropout: 0.05
Target modules: q_proj, k_proj, o_proj, down_proj, up_proj, gate_proj, v_proj

Speeds, Sizes, Times

Model size: 7B
Training time: Varies by experiment
Checkpoint size: LoRA adapter only (~2GB)

Evaluation

The model was evaluated on Python code generation tasks with focus on:

Code quality
Instruction following
Python syntax correctness

Technical Specifications

Model Architecture and Objective

Architecture: LoRA adapter on top of allenai/OLMo-2-1124-7B-Instruct
Objective: Causal language modeling for code generation
Task type: CAUSAL_LM

Compute Infrastructure

Hardware: GPU cluster
Software: PEFT, Transformers, PyTorch

Citation

If you use this model, please cite:

@misc{olmo-code-sft-7b,
  author = {OLMo Code SFT Team},
  title = {OLMo Code SFT - 7B Model},
  year = {2024},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  howpublished = {\url{https://huggingface.co/olmo-code-sft-7b-lr0.0005}},
}

Model Card Authors

OLMo Code SFT Team

Model Card Contact

For questions about this model, please open an issue in the repository.

Downloads last month: -

Model tree for dipikakhullar/olmo-code-sft-7b-lr0.0005

Base model

allenai/OLMo-2-1124-7B

Finetuned

allenai/OLMo-2-1124-7B-SFT

Finetuned

allenai/OLMo-2-1124-7B-DPO

Finetuned

allenai/OLMo-2-1124-7B-Instruct

Adapter

(16)

this model