Hibernates/Hibernates-2B-R1-V1

A highly efficient 2B parameter language model optimized for reasoning and dialogue tasks.

Model Overview

Hibernates-2B is a custom transformer architecture designed for advanced language understanding and generation. Built with performance and efficiency in mind, it leverages state-of-the-art techniques for natural language processing.

Key Features

2B Parameters
4096 Token Context Window
Custom Transformer Architecture
Optimized for CPU and GPU Inference
Multi-Turn Dialogue Support

Technical Specifications

Architecture: Custom Transformer
Parameters: 2 Billion
Context Length: 4096 tokens
Model Type: Decoder-only
Tokenizer: Custom WordPiece
Format: SafeTensors

Usage Guide

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_id = "Hibernates-2B-R1-V1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Example conversation
messages = [
    {"role": "system", "content": "You are a helpful AI assistant."},
    {"role": "user", "content": "How can you help me today?"}
]

# Generate response
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
outputs = model.generate(
    inputs["input_ids"],
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.95
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Performance Characteristics

Strengths

Efficient Resource Usage
Strong Reasoning Capabilities
Multi-Turn Dialogue
Context Awareness
Instruction Following

Considerations

Resource Requirements: 8GB+ GPU RAM recommended
Task Specificity: Best suited for dialogue and reasoning tasks
Language Support: Primary focus on English
Model Size: Optimized for balance of performance and efficiency

License and Usage

Research and commercial use permitted
Attribution appreciated but not required
No warranty provided

Citation

If you use this model in your research, please cite:

@software{hibernates2b_2024,
  title={Hibernates-2B: Efficient Language Model for Reasoning},
  year={2024},
  version={R1-V1}
}

Acknowledgments

Built using PyTorch and Hugging Face Transformers. Special thanks to the open-source AI community.

Download Instructions

Due to file size limitations, the model files are hosted externally. Download them from:

Place these files in the root directory of the project before running.

NextHibernates/Hibernates-2B-R1-V1 · Discussions