FoundationVision
/

unitok_tokenizer

Feature Extraction

Model card Files Files and versions

unitok_tokenizer

3.31 GB

3 contributors

History: 8 commits

machuofan's picture

Upload unitok_tokenizer.pth with huggingface_hub

7f6ea34 verified 11 months ago

.gitattributes

1.52 kB

initial commit 12 months ago
README.md

3.62 kB

Update README.md 12 months ago
config.json

2.79 kB

update readme 12 months ago
unitok_tokenizer.pth
Detected Pickle imports (4)
- "torch._utils._rebuild_tensor_v2",
- "torch.bfloat16",
- "torch.FloatStorage",
- "collections.OrderedDict"
How to fix it?
3.31 GB
xet

Upload unitok_tokenizer.pth with huggingface_hub 11 months ago