README.md

hibernatesai's picture

Update README.md

7ecf221 verified 2 days ago

metadata

license: apache-2.0
language:
  - en
metrics:
  - accuracy
base_model:
  - meta-llama/Llama-3.1-8B-Instruct
pipeline_tag: reinforcement-learning

A highly efficient 2B parameter language model optimized for reasoning and dialogue tasks.

Model Overview

Hibernates-2B is a custom transformer architecture designed for advanced language understanding and generation. Built with performance and efficiency in mind, it leverages state-of-the-art techniques for natural language processing.

Key Features

• 2B Parameters

• 4096 Token Context Window

• Custom Transformer Architecture

• Optimized for CPU and GPU Inference

• Multi-Turn Dialogue Support

Technical Specifications

• Architecture: Custom Transformer

• Parameters: 2 Billion

• Context Length: 4096 tokens

• Model Type: Decoder-only

• Tokenizer: Custom WordPiece

• Format: SafeTensors

Usage Guide

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_id = "Hibernates-2B-R1-V1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Example conversation
messages = [
    {"role": "system", "content": "You are a helpful AI assistant."},
    {"role": "user", "content": "How can you help me today?"}
]

# Generate response
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
outputs = model.generate(
    inputs["input_ids"],
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.95
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Performance Characteristics

Strengths

• Efficient Resource Usage

• Strong Reasoning Capabilities

• Multi-Turn Dialogue

• Context Awareness

• Instruction Following

Considerations

• Resource Requirements: 8GB+ GPU RAM recommended

• Task Specificity: Best suited for dialogue and reasoning tasks

• Language Support: Primary focus on English

• Model Size: Optimized for balance of performance and efficiency

License and Usage

• Research and commercial use permitted

• Attribution appreciated but not required

• No warranty provided

Citation

If you use this model in your research, please cite:

@software{hibernates2b_2024,
  title={Hibernates-2B: Efficient Language Model for Reasoning},
  year={2024},
  version={R1-V1}
}

Acknowledgments

Built using PyTorch and Hugging Face Transformers. Special thanks to the open-source AI community.

Download Instructions

Due to file size limitations, the model files are hosted externally. Download them from:

1. model-00001-of-00002.safetensors

2. model-00002-of-00002.safetensors

Place these files in the root directory of the project before running.