ASTRA: Initial SMILES Generator

This model is part of the ASTRA (Advanced Solvation Transformer for Rational Additives) framework. It is a GPT-2 based Causal Language Model (CausalLM) trained on a large-scale dataset of canonical SMILES strings. While this model has learned the fundamental grammar and structural rules necessary to generate chemically valid molecules autonomously, it has not yet been optimized for specific quantum chemical properties.

This model serves as the starting point (pre-generator) for the ASTRA Active Learning loop.

Model Details

  • Architecture: GPT-2 (CausalLM)
  • Stage: Initial (Before Active Learning)
  • Task: Autoregressive SMILES Generation
  • Data: Large-scale unlabelled canonical SMILES (~4M -> ~260K)

Usage

from astra.tokenizer import initial_gpt_tokenizer_with_vocabulary
from astra.model import GPTForCausalLM

# Load the tokenizer and model
vocab_file = "./vocab.txt"
tokenizer = initial_gpt_tokenizer_with_vocabulary(path=vocab_file)

pretrained_path = "<path/to/pretrained/model>" # Replace with your actual paths
model = GPTForCausalLM.from_pretrained(pretrained_gpt_path)

More Information

For more details on data preparation, downstream fine-tuning, and the full active learning loop, please visit our GitHub Repository.

Downloads last month
9
Safetensors
Model size
85.9M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support