🔁 Enable Modelfile Support #25

New issue

Open

opened 2025-05-11 16:06:12 -04:00 by milo · 0 comments

milo commented

2025-05-11 16:06:12 -04:00

Owner

🧩 Modfile Support – Scope for Persona Injection via Ollama

Goal

Enable the bot to automatically load and use a .mod file (or compatible JSON) to customize the model's behavior with persistent persona traits (e.g., Gojo personality). This enhances character realism and consistency without needing to prepend prompt injection every time.

✅ Phase 1: Config + Environment Support

Add new config entry in settings.yml:

ai:
  use_modfile: true
  modfile_path: "./persona/gojo.mod"  # or .json variant

Modify ai.py to:
- Check use_modfile before sending prompt
- If true, include modelfile field in Ollama /generate API call
- Resolve modfile path relative to src/ or container mount

⚙️ Phase 2: Modelfile Design Guidelines

Support .mod or .json templates defining:
- Base model: FROM llama3, FROM deepseek-coder, etc.
- SYSTEM personality:
  - "You are Satoru Gojo from Jujutsu Kaisen..."
- Template:
  - Ensure correct formatting like [INST] ... [/INST] if required
- Inference params (optional overrides):
  - temperature: 0.85 (adds flair)
  - num_ctx: 4096 (for longer memory)
  - stop: ["User:"] (helps cut off rambling)

💬 Example `.mod` file

FROM llama3
PARAMETER temperature 0.85
PARAMETER num_ctx 4096
PARAMETER stop "User:"

SYSTEM """You are Satoru Gojo from Jujutsu Kaisen. You’re cocky, powerful, and flirty, but deeply protective of your students. Speak in a modern tone. Never admit you're a bot."""

TEMPLATE """{{ if .System }}
<|system|>
{{ .System }}
<|user|>
{{ end }}{{ .Prompt }}
<|assistant|>"""

🧪 Phase 3: Runtime Testing

Validate persona consistency across multiple replies
Ensure fallback if file path is wrong or missing
Verify no double-injection if modfile + prompt_inject both exist

🧠 Benefits

Cleaner AI prompt logic
Enables dynamic switching between personalities (Gojo, Delta, etc.)
Paves way for LoRA and memory support

🔜 Future: Admin Command for Hot-Swap

!modfile switch gojo.mod
!modfile disable
!modfile info

## 🧩 Modfile Support – Scope for Persona Injection via Ollama ### Goal Enable the bot to **automatically load and use a `.mod` file** (or compatible JSON) to customize the model's behavior with persistent persona traits (e.g., Gojo personality). This enhances character realism and consistency without needing to prepend prompt injection every time. --- ### ✅ Phase 1: Config + Environment Support - [ ] Add new config entry in `settings.yml`: ```yaml ai: use_modfile: true modfile_path: "./persona/gojo.mod" # or .json variant ``` - [ ] Modify `ai.py` to: - Check `use_modfile` before sending prompt - If `true`, include `modelfile` field in Ollama `/generate` API call - Resolve modfile path relative to `src/` or container mount --- ### ⚙️ Phase 2: Modelfile Design Guidelines - [ ] Support `.mod` or `.json` templates defining: - Base model: `FROM llama3`, `FROM deepseek-coder`, etc. - SYSTEM personality: - `"You are Satoru Gojo from Jujutsu Kaisen..."` - Template: - Ensure correct formatting like `[INST] ... [/INST]` if required - Inference params (optional overrides): - `temperature: 0.85` *(adds flair)* - `num_ctx: 4096` *(for longer memory)* - `stop: ["User:"]` *(helps cut off rambling)* --- ### 💬 Example `.mod` file ```mod FROM llama3 PARAMETER temperature 0.85 PARAMETER num_ctx 4096 PARAMETER stop "User:" SYSTEM """You are Satoru Gojo from Jujutsu Kaisen. You’re cocky, powerful, and flirty, but deeply protective of your students. Speak in a modern tone. Never admit you're a bot.""" TEMPLATE """{{ if .System }} <|system|> {{ .System }} <|user|> {{ end }}{{ .Prompt }} <|assistant|>""" ``` --- ### 🧪 Phase 3: Runtime Testing - [ ] Validate persona consistency across multiple replies - [ ] Ensure fallback if file path is wrong or missing - [ ] Verify no double-injection if `modfile` + `prompt_inject` both exist --- ### 🧠 Benefits - Cleaner AI prompt logic - Enables dynamic switching between personalities (Gojo, Delta, etc.) - Paves way for LoRA and memory support --- ### 🔜 Future: Admin Command for Hot-Swap - `!modfile switch gojo.mod` - `!modfile disable` - `!modfile info`