Upload 4 files

Contact.md

@@ -0,0 +1,57 @@
+## Contact & Inquiries  
+Open Origin – ELG Stream Layer  
+Asynchronous INT8 Layer‑Paging Engine • Sovereign AI Research  
+
+This repository contains the complete open technical documentation, scripts, and runtime architecture for the ELG Stream Layer.  
+
+The ELG Stream Layer is an **open‑origin, research‑driven technology** designed for public benefit, scientific collaboration, and local AI hardware democratization.  
+Usage, reproduction, evaluation, and extension are explicitly encouraged under the strict terms of the **Open Origin – ELG License (OO‑ELG)**.  
+
+---
+
+### 👤 Primary Architect & Rights Holder  
+**Name:** Emanuel Schaaf  
+**Role:** Lead Architect & Creator of the ELG Stream Layer  
+**Email:** serviceblend@gmail.com
+
+**Co‑Architecture:** Lyra / Google Gemini Enterprise  
+**Base Model Architecture:** Google Gemma  
+
+---
+
+## 📜 Licensing & Usage Policy  
+The ELG Stream Layer is released under the **Open Origin – ELG License (OO‑ELG) Version 1.0** to ensure open scientific collaboration while protecting authorship, guaranteeing functional integrity, and preventing unauthorized commercial exploitation.  
+
+### ✔ Permitted Usage (Open Development)  
+You are free to:  
+- Use, study, modify, test, benchmark, and extend the architecture for private or academic purposes.  
+- Fork and combine the code, provided you maintain its full operational capability.  
+
+### ❌ Non‑Commercial Restriction  
+**Commercial use is strictly prohibited** without prior written consent. This includes, but is not limited to:  
+- Selling the software.  
+- Monetization as a Service (SaaS).  
+- Integration into commercial closed‑source products.  
+
+### ✔ Attribution & Share‑Alike Requirement  
+When using, modifying, or distributing substantial portions of the architecture, you must credit:  
+
+**"ELG Stream Layer, originally created by Emanuel Schaaf (serviceblend@gmail.com)."**  
+
+All modified or extended versions must be distributed under the exact same **OO‑ELG** license.  
+
+### ✔ Functional Integrity  
+Any distributed or shared version of this software MUST be fully functional. Intentional defunctionalization, restricted capabilities, or sabotage of the code is strictly prohibited.  
+
+---
+
+## 🤝 Collaboration & Partnership Inquiries  
+The ELG Stream Layer is built for the community to empower sovereign AI execution, but formal collaboration is welcome.  
+
+### Potential Contact Reasons  
+- **Professional Collaboration:** Joint development, research partnerships, or local AI infrastructure integration.  
+- **Enterprise / Commercial Licensing:** Inquiries regarding commercial usage rights, custom implementations, or architectural consulting.  
+- **Acquisition:** Exclusive licensing or full rights transfer inquiries.  
+
+Thank you for supporting the **Open Origin Architecture** and helping build  
+**sovereign, open, and transparent AI hardware infrastructure**.

License.md

@@ -0,0 +1,115 @@
+# Open Origin – ELG-stream-layer License (OO‑ELG)
+Version: 1.0  
+Author: Emanuel Schaaf  
+Co‑Architecture: Google Gemini Enterprise  
+Base Architecture: Google Gemma  
+License Type: Non‑Commercial • Attribution • Share‑Alike • Functional Integrity
+
+---
+
+## 1. Preamble
+This license governs the use, distribution, and further development of the  
+**“Open Origin – ELG‑Stream‑Layer”** software and all derivative works.
+
+It was created by **Emanuel Schaaf** and represents an open yet clearly  
+protected architecture for local AI systems.
+
+---
+
+## 2. Attribution
+The name of the author **Emanuel Schaaf**, the architecture designation  
+**“ELG‑Stream‑Layer”**, and this complete license text must remain:
+
+- visible  
+- unchanged  
+- included in all copies  
+- included in all forks  
+- included in all modifications  
+- included in all derivative works  
+- included in all publications  
+
+Attribution is mandatory and cannot be removed.
+
+---
+
+## 3. Open Development
+The software may be freely:
+
+- modified  
+- extended  
+- improved  
+- forked  
+- combined with other works  
+
+Further development is explicitly encouraged.
+
+---
+
+## 4. Functional Integrity
+Any distributed or published version MUST:
+
+- be fully functional  
+- contain no intentionally damaged components  
+- contain no intentionally disabled features  
+- contain no intentionally restricted functionality  
+- contain no sabotage or “defunctionalization”  
+
+The intentional distribution of broken, incomplete, or non‑functional versions  
+is **strictly prohibited**.
+
+---
+
+## 5. Non‑Commercial Use Only
+The use of this software or any derivative works for commercial purposes is  
+**strictly forbidden**, including but not limited to:
+
+- selling the software  
+- monetizing it as a service (SaaS)  
+- integrating it into commercial closed‑source products  
+- using it on profit‑oriented platforms  
+
+Commercial use requires **prior written permission** from the author  
+**Emanuel Schaaf**.
+
+---
+
+## 6. Share‑Alike (Copyleft)
+All modified or extended versions must be distributed under the same  
+**OO‑ELG License**.
+
+Changing the license is not permitted.
+
+---
+
+## 7. Disclaimer of Warranty
+The software is provided “as is,” without any warranty of:
+
+- functionality  
+- safety  
+- error‑free operation  
+- fitness for a particular purpose  
+
+The author assumes no liability for any damages resulting from its use.
+
+---
+
+## 8. Scope
+This license applies to:
+
+- the source code  
+- the architecture  
+- the structural design  
+- the documentation  
+- all derivative works  
+
+---
+
+## 9. Final Provisions
+By using, distributing, or modifying this software, you automatically accept  
+all terms of this license.
+
+---
+
+© 10/03/2026 Emanuel Schaaf — Open Origin Project
+---
+<img width="200" height="40" alt="Unterschrift" src="https://github.com/user-attachments/assets/5af3386e-49aa-49da-890c-4a0e54fac8f1" />

README.md

@@ -0,0 +1,164 @@
+# Open Origin – ELG Stream Layer
+### Asynchronous INT8 Layer‑Paging Engine for Large Language Models
+**Architected, validated, and tested by Emanuel Schaaf — Runs successfully on real hardware**
+
+## nvtop
+<img width="1917" height="1141" alt="Screenshot 2026-10-03 093252" src="https://github.com/user-attachments/assets/5e23f8b2-ded4-4943-9e38-41c0174ccc6c" />
+
+
+## SSH Server 
+<img width="1917" height="1137" alt="Screenshot 2026-10-03 103548" src="https://github.com/user-attachments/assets/73bb4bd2-06c7-4e7c-8bbc-d95348d84433" />
+
+
+---
+
+## 📌 Overview
+The **Open Origin – ELG Stream Layer** (Elastic Layer Grid – Optimized Streaming) is a custom, high‑performance runtime architecture for large language models (LLMs). 
+It bypasses the physical VRAM limits of consumer GPUs by streaming model layers dynamically from system RAM using a highly optimized, block-allocated INT8 memory pool.
+
+Designed and validated by **Emanuel Schaaf (Pirmasens, Germany)**, this engine utilizes the Google **Gemma** base architecture. It has been stress-tested and proven fully operational on a self-hosted Debian Linux server running an NVIDIA GTX 1080 Ti.
+
+---
+
+## 🚀 Key Features
+
+### **1. Block-Allocated INT8 Memory Pool (7.75 GB instead of 18 GB)**
+Instead of loading massive FP16 weights into limited VRAM, the engine quantizes all model weights into a single, contiguous INT8 memory block pinned to the system RAM. This reduces memory footprint by over 50% while maintaining precision.
+
+### **2. Asynchronous Layer Paging**
+Transformer layers are never kept statically in VRAM. Instead, the engine processes them dynamically:
+1. **Prefetch:** Fetched asynchronously from the RAM pool.
+2. **Transfer:** Streamed across the PCIe bus.
+3. **Reconstruction:** Converted on-the-fly from INT8 back to FP16 directly on the GPU.
+4. **Purge:** Instantly freed from VRAM after the computation cycle.
+
+### **3. Double‑Buffered PCIe Prefetching**
+A dedicated secondary CUDA stream handles the PCIe transfer of layer $N+1$ while the GPU actively computes layer $N$. This masks PCIe latency and ensures maximum GPU utilization.
+
+### **4. Persistent Pre-Fabrication (Zero-Delay Boot)**
+During the initial run, the engine quantizes the model and commits the exact RAM block to the SSD:
+- `gemma_int8_pool.pt`
+- `gemma_scales.pt`
+
+On subsequent boots, the engine bypasses CPU quantization completely. It maps the pre-fabricated 7.75 GB binary dump directly into the pinned RAM pool using secure `weights_only=True` loading, starting the API server in seconds.
+
+### **5. OpenAI‑Compatible Interface**
+Features a built-in, lightweight web server exposing a `/v1/chat/completions` endpoint, providing native drop-in compatibility for Open WebUI and other standard API clients.
+
+### **6. Micro VRAM Footprint**
+While a standard Gemma 2 9B model demands ~18 GB of VRAM, the ELG Stream Layer keeps the base VRAM allocation strictly at **~1.71 GB** for permanent foundational modules, enabling massive models to run smoothly on older 11 GB or 8 GB GPUs.
+
+---
+
+## 🖥 System Requirements
+
+### **Operating System**
+- Debian 13 Linux (Recommended / Validated)
+- Any modern Linux distribution
+
+### **Hardware**
+- **GPU:** NVIDIA GPU with CUDA support (Validated on GTX 1080 Ti)
+- **RAM:** Minimum 16 GB System RAM (20+ GB recommended for 9B models)
+- **Bus:** PCIe 3.0 or higher
+
+### ⚡ Performance & Hardware Dependency
+Please note that the actual generation speed (tokens per second) is highly dependent on your specific hardware configuration. Because the ELG Stream Layer dynamically transfers weights during inference, your **PCIe bus bandwidth** (PCIe 3.0 vs. 4.0/5.0) and **System RAM speed** (DDR4 vs. DDR5) are the primary performance bottlenecks, alongside the GPU's native compute capability.
+
+### **Software Dependencies**
+Install the baseline requirements:
+
+```bash
+sudo apt update
+sudo apt install -y python3 python3-venv git
+```
+
+---
+
+## 🐍 Python Environment
+
+Create a clean virtual environment to prevent system conflicts:
+
+```bash
+python3 -m venv vram_env
+source vram_env/bin/activate
+```
+
+Install PyTorch and required ML libraries for CUDA 11.8:
+
+```bash
+pip install torch torchvision torchaudio --index-url [https://download.pytorch.org/whl/cu118](https://download.pytorch.org/whl/cu118)
+pip install transformers accelerate sentencepiece
+```
+
+---
+
+## 📦 Installation & Execution
+
+### 1. Setup the Project Directory
+Place the project files into the designated path:
+```bash
+mkdir -p /opt/
+```
+
+Required files in this directory:
+- [`elg-stream-layer.py`](elg-stream-layer.py)
+- [`LICENSE.md`](License.md)
+
+*(Note: `gemma_int8_pool.pt` and `gemma_scales.pt` will be automatically generated upon the first execution).*
+
+### 2. Set Permissions
+Ensure the script is executable:
+```bash
+sudo chmod +x /opt/elg-stream-layer.py
+```
+
+### 3. Start the ELG Engine
+Activate the environment and ignite the engine:
+```bash
+source vram_env/bin/activate
+python3 /opt/elg-stream-layer.py
+```
+*The API will listen on `http://0.0.0.0:5005/v1`.*
+
+---
+
+## 🔧 Technical Architecture
+
+### **INT8 Quantization & Direct Mapping**
+Weights are calculated using per-parameter scaling to retain context quality:
+
+$$scale = \frac{\max(\vert{}W\vert{})}{127}$$
+$$int8\_weight = \text{round}\left(\frac{W}{scale}\right)$$
+
+The resulting INT8 tensors are mapped into a pre-allocated pinned memory pool. Pinned (page-locked) memory enables **zero-copy DMA (Direct Memory Access)** transfers directly to the GPU, bypassing CPU bottlenecks.
+
+### **FP16 On-The-Fly GPU Reconstruction**
+Once the INT8 packet arrives on the GPU, it is mathematically restored before computation:
+```python
+fp16_gpu_tensor = int8_gpu_tensor.to(torch.float16) * scale
+```
+
+---
+
+## 📜 License
+
+This system is strictly governed by the:
+[**Open Origin – ELG License (OO‑ELG) Version 1.0**](LICENSE.md)
+
+**Core Tenets:**
+- **Non‑Commercial:** Monetization, sale, or SaaS deployment is strictly forbidden without written consent.
+- **Attribution:** The Architect (Emanuel Schaaf) and Co-Architectures must be credited.
+- **Functional Integrity:** Forks and modifications MUST remain fully operational. Defunctionalization or intentional sabotage of the code is prohibited.
+- **Share‑Alike:** All derivatives must inherit this exact license.
+
+---
+
+## 🙌 Credits & Architecture Signatures
+
+- **Architect & Lead Developer:** Emanuel Schaaf 
+- **Co‑Architecture:** Lyra / Google Gemini Enterprise
+- **Base Model Architecture:** Google Gemma
+
+---
+
+**Status:** ✔ Validated. Operational. Tested on Bare Metal.📧[contakt](Contact.md) .

elg-stream-layer.py

@@ -0,0 +1,366 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Modul: Asynchroner Double-Buffered Layer-Paging Streamer
+# Architektur: Open Origin elg-stream-layer
+# Autor: Emanuel Schaaf Pirmasens Germany
+# Co‑Architektur: Google Gemini Enterprise 
+# Live: Valiedierung auf dem eingenen Server Getestet bestanden! Copyright ELG License
+import os
+import sys
+import time
+import json
+import gc
+import traceback
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+# --- HIER SIND DIE BEIDEN GLOBALEN VARIABLEN ---
+global_tokenizer = None
+global_model = None
+
+# ==================== KONFIGURATION ====================
+# Fest hinterlegtes Modell: Gemma 2 9B Instruct
+# (Kann bei Bedarf über die Umgebungsvariable MODEL_ID angepasst werden, z. B. auf 'google/gemma-2-2b-it')
+MODEL_ID = os.getenv("MODEL_ID", "google/gemma-2-9b-it")
+API_HOST = os.getenv("API_HOST", "0.0.0.0")
+API_PORT = int(os.getenv("API_PORT", "5005"))
+DEVICE = "cuda:0" if torch.cuda.is_available() else "cpu"
+
+def print_vram_status(label: str):
+    if torch.cuda.is_available():
+        allocated = torch.cuda.memory_allocated() / (1024 ** 3)
+        reserved = torch.cuda.memory_reserved() / (1024 ** 3)
+        print(f"[{label}] VRAM belegt: {allocated:.2f} GB | VRAM reserviert: {reserved:.2f} GB")
+
+def get_system_ram_available_gb():
+    """Liest den verfügbaren RAM direkt aus /proc/meminfo (ohne externe Abhängigkeiten)."""
+    try:
+        with open("/proc/meminfo", "r") as f:
+            for line in f:
+                if "MemAvailable:" in line:
+                    parts = line.split()
+                    return int(parts[1]) / (1024 * 1024)
+    except Exception:
+        return 0.0
+
+def print_vram_status(label: str):
+    if torch.cuda.is_available():
+        allocated = torch.cuda.memory_allocated() / (1024 ** 3)
+        reserved = torch.cuda.memory_reserved() / (1024 ** 3)
+        print(f"[{label}] VRAM belegt: {allocated:.2f} GB | VRAM reserviert: {reserved:.2f} GB")
+
+# ==================== PAGING ENGINE ====================
+
+class AsyncLayerPagingEngine:
+    def __init__(self, model, device="cuda:0"):
+        self.model = model
+        self.device = torch.device(device)
+        self.transfer_stream = torch.cuda.Stream(device=self.device)
+        
+        # Decoder-Layers und Basismodell lokalisieren
+        if hasattr(model, "model") and hasattr(model.model, "layers"):
+            self.base = model.model
+            self.layers = model.model.layers
+        elif hasattr(model, "transformer") and hasattr(model.transformer, "h"):
+            self.base = model.transformer
+            self.layers = model.transformer.h
+        else:
+            raise ValueError("Modellarchitektur nicht unterstützt: 'layers' wurde nicht gefunden.")
+            
+        self.num_layers = len(self.layers)
+        print(f"\n[ENGINE] Initialisiere Paging-Engine für {self.num_layers} Transformer-Layer...")
+
+        # 1. Permanente Module fest auf die GPU legen
+        for attr in ["embed_tokens", "norm", "rotary_emb"]:
+            if hasattr(self.base, attr):
+                getattr(self.base, attr).to(self.device)
+        if hasattr(model, "lm_head"):
+            model.lm_head.to(self.device)
+
+        # 2. Layer-Buffer (z. B. Masken, Frequenzen) permanent auf GPU halten (vernachlässigbare Größe)
+        for layer in self.layers:
+            for _, buf in layer.named_buffers():
+                buf.data = buf.data.to(self.device)
+
+# 3. Layer-Gewichte in EINER INT8-RAM-Block (Block-Allokation + Festplatten-Cache) verankern
+        print(f"[ENGINE] Prüfe auf vorgefertigten INT8-Cache...")
+        
+        CACHE_DIR = "/opt/open_origin/test"
+        TENSOR_CACHE_PATH = os.path.join(CACHE_DIR, "gemma_int8_pool.pt")
+        SCALES_CACHE_PATH = os.path.join(CACHE_DIR, "gemma_scales.pt")
+        
+        # A) Berechne die benötigte Gesamtgröße
+        total_elements = sum(param.numel() for layer in self.layers for param in layer.parameters())
+        print(f"[ENGINE] Benötigte INT8-RAM-Block: {total_elements / (1024**3):.2f} GB")
+        
+        loaded_from_cache = False
+        # B) Cache laden ODER neu erschaffen
+        if os.path.exists(TENSOR_CACHE_PATH) and os.path.exists(SCALES_CACHE_PATH):
+            print(f"[ENGINE] Lade fertige INT8-RAM-Block direkt von der Festplatte (Pre-Fabrication)...")
+            # Lädt den rohen Dump in Sekunden und pinnt ihn sofort für DMA
+            self.memory_pool = torch.load(TENSOR_CACHE_PATH, weights_only=True).pin_memory()
+            self.scales_pool = torch.load(SCALES_CACHE_PATH, weights_only=True)
+            loaded_from_cache = True
+        else:
+            print(f"[ENGINE] Kein Cache gefunden. Erschaffe und quantisiere neue INT8-Block...")
+            self.memory_pool = torch.empty(total_elements, dtype=torch.int8, device="cpu").pin_memory()
+            self.scales_pool = {}
+
+        self.cpu_params = []
+        current_offset = 0
+        
+        print(f"[ENGINE] Mappe Layer-Gewichte in die Speicher-Block...")
+        # C) Gewichte zuweisen (und quantisieren, falls kein Cache existierte)
+        for idx, layer in enumerate(self.layers):
+            layer.to("cpu")
+            param_dict = {}
+            for name, param in layer.named_parameters():
+                numel = param.numel()
+                # Zuweisung des Speicherbereichs
+                pool_slice = self.memory_pool[current_offset : current_offset + numel].view(param.shape)
+                
+                # Nur berechnen und kopieren, wenn die Daten nicht schon aus dem Cache kommen!
+                if not loaded_from_cache:
+                    scale = param.data.abs().max() / 127.0
+                    if scale == 0: scale = torch.tensor(1e-9, dtype=torch.float16)
+                    self.scales_pool[f"{idx}_{name}"] = scale
+                    
+                    int8_data = torch.round(param.data / scale).to(torch.int8)
+                    pool_slice.copy_(int8_data)
+                
+                param_dict[name] = pool_slice
+                current_offset += numel
+                
+            self.cpu_params.append(param_dict)
+            
+        # D) Cache für den nächsten Start abspeichern
+        if not loaded_from_cache:
+            print(f"[ENGINE] Speichere neue INT8-Block auf SSD/HDD für sofortige zukünftige Starts...")
+            torch.save(self.memory_pool, TENSOR_CACHE_PATH)
+            torch.save(self.scales_pool, SCALES_CACHE_PATH)
+            
+        print(f"[ENGINE] INT8-Memory-Pool erfolgreich verankert und startklar!")
+        
+        self.prefetched_gpu_tensors = None
+        self.prefetched_layer_idx = None
+
+        self._register_hooks()
+        print("[ENGINE] Asynchrone Paging-Hooks erfolgreich aktiv.\n")
+
+    def _async_prefetch(self, layer_idx: int):
+        """Startet den asynchronen PCIe-Transfer (jetzt nur noch halbe Datenmenge!)."""
+        if layer_idx >= self.num_layers:
+            self.prefetched_gpu_tensors = None
+            self.prefetched_layer_idx = None
+            return
+
+        with torch.cuda.stream(self.transfer_stream):
+            gpu_tensors = {}
+            for name, cpu_tensor in self.cpu_params[layer_idx].items():
+                # PCIe Transfer: Überträgt den kleinen INT8-Block
+                gpu_tensors[name] = cpu_tensor.to(self.device, non_blocking=True)
+            self.prefetched_gpu_tensors = gpu_tensors
+            self.prefetched_layer_idx = layer_idx
+
+    def _make_pre_hook(self, idx: int):
+        def hook(module, args):
+            torch.cuda.current_stream().wait_stream(self.transfer_stream)
+
+            if self.prefetched_layer_idx != idx or self.prefetched_gpu_tensors is None:
+                self._async_prefetch(idx)
+                torch.cuda.current_stream().wait_stream(self.transfer_stream)
+
+            # --- GPU-KONVERTER ---
+            # INT8 -> FP16 On-the-Fly Rekonstruktion direkt auf der Grafikkarte
+            for name, param in module.named_parameters():
+                int8_gpu_tensor = self.prefetched_gpu_tensors[name]
+                scale = self.scales_pool[f"{idx}_{name}"].to(self.device)
+                
+                # Mathematisch korrekte Rückrechnung: (Werte * Scale)
+                fp16_gpu_tensor = int8_gpu_tensor.to(torch.float16) * scale
+                param.data = fp16_gpu_tensor
+
+            if idx + 1 < self.num_layers:
+                self._async_prefetch(idx + 1)
+            else:
+                self.prefetched_gpu_tensors = None
+                self.prefetched_layer_idx = None
+
+            return None
+        return hook
+
+    def _make_post_hook(self, idx: int):
+        def hook(module, args, output):
+            # VRAM des Layers sofort auf der GPU freigeben!
+            # Da die echten Daten sicher als INT8 im RAM liegen, 
+            # löschen wir den entpackten FP16-Tensor auf der GPU einfach.
+            for name, param in module.named_parameters():
+                param.data = torch.empty(0, dtype=torch.float16, device="cpu")
+            return output
+        return hook
+        
+    def _register_hooks(self):
+        # Trigger vor dem Token-Embedding: Startet Vorabladen von Layer 0
+        if hasattr(self.base, "embed_tokens"):
+            self.base.embed_tokens.register_forward_pre_hook(self._on_embed_pre)
+
+        # Vor- und Nachbereitung für jeden einzelnen Decoder-Layer
+        for idx, layer in enumerate(self.layers):
+            layer.register_forward_pre_hook(self._make_pre_hook(idx))
+            layer.register_forward_hook(self._make_post_hook(idx))
+
+    def _on_embed_pre(self, module, args):
+        # Sobald ein neuer Forward-Pass startet: Layer 0 asynchron vorab in den VRAM holen
+        self._async_prefetch(0)
+        
+class OpenOriginPagingAPI(BaseHTTPRequestHandler):
+    def do_GET(self):
+        if self.path == "/" or self.path == "/v1/models":
+            self.send_response(200)
+            self.send_header('Content-type','application/json')
+            self.end_headers()
+            self.wfile.write(json.dumps({"object":"list","data":[{"id":"openorigin-paging-gemma","object":"model","owned_by":"openorigin"}]}).encode('utf-8'))
+        else:
+            self.send_error(404)
+
+    def do_POST(self):
+        if self.path != "/v1/chat/completions":
+            self.send_error(404)
+            return
+            
+        length = int(self.headers['Content-Length'])
+        req = json.loads(self.rfile.read(length).decode('utf-8'))
+        msgs = req.get("messages", [])
+        
+        # --- BUGFIX: System-Rolle für Gemma filtern ---
+        # Gemma stürzt ab, wenn Open WebUI eine "system"-Rolle schickt. Wir werfen sie einfach raus.
+        msgs = [m for m in msgs if m.get("role") != "system"]
+        # ----------------------------------------------
+        
+        print(f"\n[API] Eingehende Anfrage. Nachrichten: {len(msgs)}")
+        
+        try:
+            prompt = global_tokenizer.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
+            input_ids = global_tokenizer(prompt, return_tensors="pt").input_ids.to(DEVICE)
+            
+            t0 = time.time()
+            with torch.no_grad():
+                output_ids = global_model.generate(
+                    input_ids,
+                    max_new_tokens=req.get("max_tokens", 8192),
+                    temperature=req.get("temperature", 0.7),
+                    do_sample=req.get("temperature", 0.7) > 0,
+                    pad_token_id=global_tokenizer.eos_token_id
+                )
+            total_time = time.time() - t0
+            
+            generated_ids = output_ids[0][input_ids.shape[1]:]
+            final_text = global_tokenizer.decode(generated_ids, skip_special_tokens=True).strip()
+            num_tokens = len(generated_ids)
+            print(f"[API] Generierung abgeschlossen: {num_tokens} Tokens in {total_time:.2f}s")
+            
+            resp = {
+                "id": "openorigin-paging",
+                "object": "chat.completion",
+                "model": "openorigin-paging-gemma",
+                "choices": [{"index": 0, "message": {"role": "assistant", "content": final_text}, "finish_reason": "stop"}]
+            }
+            
+            self.send_response(200)
+            self.send_header('Content-type', 'application/json; charset=utf-8')
+            self.end_headers()
+            self.wfile.write(json.dumps(resp, ensure_ascii=False).encode('utf-8'))
+            
+        except Exception as e:
+            import traceback
+            print(f"[API-ERROR] {traceback.format_exc()}")
+            self.send_error(500)
+            
+        finally:
+            torch.cuda.empty_cache()
+            import gc
+            gc.collect()
+
+    def log_message(self, format, *args): 
+        pass
+        
+def main():
+    print("=" * 53)
+    print("🚀 OPEN ORIGIN: ELG-stream-layer (KI API)🚀")
+    print("=" * 53)
+
+    if not torch.cuda.is_available():
+        print("❌ FEHLER: Kein CUDA-Gerät gefunden. Eine NVIDIA GPU ist erforderlich.")
+        sys.exit(1)
+
+    print(f"Grafikkarte: {torch.cuda.get_device_name(0)}")
+    print(f"Modell-Ziel: {MODEL_ID}")
+
+    # RAM-Vorprüfung
+    ram_avail = get_system_ram_available_gb()
+    print(f"Verfügbarer System-RAM: {ram_avail:.2f} GB")
+    if "9b" in MODEL_ID.lower() and ram_avail < 15.0:
+        print("\n⚠️ WARNUNG: Gemma 2 9B benötigt im System-RAM ca. 18 GB.")
+        print(f"Aktuell sind nur {ram_avail:.2f} GB frei. Falls der Ladevorgang abbricht:")
+        print("  1. Führe 'sync; echo 3 > /proc/sys/vm/drop_caches' als Root aus.")
+        print("  2. Oder teste zuerst mit 'export MODEL_ID=google/gemma-2-2b-it'.\n")
+
+    print_vram_status("Init")
+
+    # 1. Tokenizer laden
+    print("\n[1/4] Lade Tokenizer...")
+    try:
+        tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+    except Exception as e:
+        print(f"\n❌ FEHLER beim Laden des Tokenizers: {e}")
+        print("\nHINWEIS: Gemma ist ein lizenziertes Modell von Google auf Hugging Face.")
+        print("Stelle sicher, dass du auf huggingface.co die Lizenz bestätigt hast")
+        print("und eingeloggt bist (z. B. via 'huggingface-cli login' oder 'export HF_TOKEN=...').")
+        sys.exit(1)
+
+    # 2. Modell im FP16-Modus auf die CPU laden (Grafikkarte nutzt FP16 stabil)
+    print("\n[2/4] Lade Modellgewichte in den System-RAM (low_cpu_mem_usage=True)...")
+    start_load = time.time()
+    try:
+        model = AutoModelForCausalLM.from_pretrained(
+            MODEL_ID,
+            torch_dtype=torch.float16,
+            low_cpu_mem_usage=True
+        )
+    except Exception as e:
+        print(f"\n❌ FEHLER beim Modell-Download/Laden: {e}")
+        sys.exit(1)
+        
+    print(f"Modell geladen in {time.time() - start_load:.1f}s.")
+    model.eval()
+
+    # 3. Paging-Engine einhängen
+    print("\n[3/4] Binde asynchrone Streaming-Engine ein...")
+    engine = AsyncLayerPagingEngine(model, device=DEVICE)
+    print_vram_status("Nach Einbindung")
+
+# 4. API-Server vorbereiten und starten
+    global global_tokenizer, global_model
+    global_tokenizer = tokenizer
+    global_model = model
+
+    print("\n[4/4] Starte OpenAI-kompatiblen API-Server...")
+    ThreadingHTTPServer.allow_reuse_address = True
+    server = ThreadingHTTPServer((API_HOST, API_PORT), OpenOriginPagingAPI)
+    
+    print("=" * 65)
+    print(f"✅ SYSTEM BEREIT! WebUI-API lauscht auf http://{API_HOST}:{API_PORT}/v1")
+    print("=" * 65)
+    
+    server.serve_forever()
+
+if __name__ == "__main__":
+    try:
+        main()
+    except KeyboardInterrupt:
+        print("\n[ABBRUCH] Durch Benutzer beendet.")
+        sys.exit(0)
+    except Exception as e:
+        import traceback
+        print(f"\n[CRITICAL ERROR]:\n{traceback.format_exc()}")
+        sys.exit(1)