Use a modular pipeline that injects a guardrails layer before the LLM call and validates outputs after generation.
Implementation steps
1. Install the guardrails package.
```bash
pip install nemoguardrails guardrails-ai
```
2. Define a guardrails config. For NeMo Guardrails, create guardrails.yaml:
```yaml
version: "1.0"
prompts:
system: |
You are a helpful assistant. Follow the safety policy in policies.yaml.
policies:
- name: profanity
type: regex
pattern: "(?i)\\b(fuck|shit|damn)\\b"
action: block
thresholds:
hallucination_score: 0.7
```
3. Load the config in your Python service:
```python
from nemoguardrails import Rails
rails = Rails.from_config("guardrails.yaml")
```
4. Wrap the LLM call. For OpenAI GPT‑4o:
```python
response = rails.run(
user_prompt,
llm=lambda prompt: openai.ChatCompletion.create(
model="gpt-4o",
messages=[{"role":"system","content":prompt}],
temperature=0.0,
max_tokens=1024,
)["choices"][0]["message"]["content"]
)
```
5. Post‑process with Guardrails AI’s moderation validator if you need a second opinion:
```python
from guardrails import Moderation
mod = Moderation(threshold=0.85)
safe_output = mod.validate(response)
```
6. Log the guardrail decision (action, policy_name, score) to your observability stack for audit.
7. Deploy behind a feature flag so you can toggle the guardrails without redeploying.
Quick comparison
| Feature | NeMo Guardrails | Guardrails AI |
|------------------------|-----------------|---------------|
| Policy language | YAML + Jinja2 | Python DSL |
| Hallucination scoring | Built‑in (0‑1) | External LLM‑based |
| Real‑time streaming | ✅ | ❌ |
| Enterprise SSO support | ✅ (via NVIDIA NGC) | ✅ (via API keys) |
Gotcha: When the guardrails block a response, the wrapper returns an empty string by default; explicitly handle the action == "block" case to avoid downstream null‑pointer errors.