i6eal.de
DEEN
Start a project
← Back to the AI Dependency AtlasExact package identity dossier

Hugging Face Tokenizers

pypi:tokenizers
pypi

This dossier links the stable identity pypi:tokenizers to 6 exact observed versions across 9 public repositories.

pypipypi:tokenizersecosystem:name + exact version + evidence path

A package identity or provider interface in published code does not prove configuration, an account, procurement, data transfer or an API call.

ecosystem:name + exact version + evidence path
9repositories
12exact evidence rows
6exact versions
1reported license expression
0OSV records returned
Exact published evidence

Observed exact versions and registry metadata

Publication age and licenses come from deps.dev metadata. They are context—not a maintenance, legal or portability verdict.

Exact version0.21.027 Nov 2024
Repositories
2
Occurrences
2
Reported licenses
non-standard
no verified publish attestation reportedNo OSV record returned for this exact version
Open exact source ↗
Exact version0.21.113 Mar 2025
Repositories
1
Occurrences
1
Reported licenses
non-standard
no verified publish attestation reportedNo OSV record returned for this exact version
Open exact source ↗
Exact version0.21.224 Jun 2025
Repositories
1
Occurrences
1
Reported licenses
non-standard
no verified publish attestation reportedNo OSV record returned for this exact version
Open exact source ↗
Exact version0.22.029 Aug 2025
Repositories
1
Occurrences
1
Reported licenses
non-standard
no verified publish attestation reportedNo OSV record returned for this exact version
Open exact source ↗
Exact version0.22.119 Sept 2025
Repositories
3
Occurrences
6
Reported licenses
non-standard
no verified publish attestation reportedNo OSV record returned for this exact version
Open exact source ↗
Exact version0.22.205 Jan 2026
Repositories
1
Occurrences
1
Reported licenses
non-standard
no verified publish attestation reportedNo OSV record returned for this exact version
Open exact source ↗
Observed relations

Repositories carrying this identity

F13-RAGlandeshauptstadt-muenchen/f-13-rag
0.22.14 Occurrences
→
kiva-llm-gatewaybaden-wuerttemberg/innenministerium/kiva.platform/kiva-llm-gateway
0.21.01 Occurrence
→
Coding Agent Usage Simulationuba-ki-lab/coding-agent-usage-simulation
0.22.11 Occurrence
→
GSA Extractionuba-ki-lab/gsa-extraction
0.21.21 Occurrence
→
LLM Questionnaire Benchmarking Frameworkuba-ki-lab/llm-questionnaire-benchmarking-framework
0.22.01 Occurrence
→
LLM Testframeworkuba-ki-lab/llm-testframework
0.21.11 Occurrence
→
Objection managementuba-ki-lab/objection-management
0.21.01 Occurrence
→
Ressource Efficient Computer Visionuba-ki-lab/ressource-efficient-computer-vision
0.22.21 Occurrence
→
Retrieval Evaluationuba-ki-lab/retrieval-evaluation
0.22.11 Occurrence
→
Exact published evidence

exact evidence rows

Coding Agent Usage Simulationuba-ki-lab/coding-agent-usage-simulation
pypi:tokenizers@0.22.1uv.lock
declaration relationship not reporteddevelopment scope not reported
Open exact source ↗
F13-RAGlandeshauptstadt-muenchen/f-13-rag
pypi:tokenizers@0.22.1requirements-dev.txt
generated or transitive declarationnot marked as development-only
Open exact source ↗
F13-RAGlandeshauptstadt-muenchen/f-13-rag
pypi:tokenizers@0.22.1requirements-gpu.txt
generated or transitive declarationnot marked as development-only
Open exact source ↗
F13-RAGlandeshauptstadt-muenchen/f-13-rag
pypi:tokenizers@0.22.1requirements-testing.txt
generated or transitive declarationnot marked as development-only
Open exact source ↗
F13-RAGlandeshauptstadt-muenchen/f-13-rag
pypi:tokenizers@0.22.1requirements.txt
generated or transitive declarationnot marked as development-only
Open exact source ↗
GSA Extractionuba-ki-lab/gsa-extraction
pypi:tokenizers@0.21.2uv.lock
declaration relationship not reporteddevelopment scope not reported
Open exact source ↗
kiva-llm-gatewaybaden-wuerttemberg/innenministerium/kiva.platform/kiva-llm-gateway
pypi:tokenizers@0.21.0poetry.lock
declaration relationship not reportednot marked as development-only
Open exact source ↗
LLM Questionnaire Benchmarking Frameworkuba-ki-lab/llm-questionnaire-benchmarking-framework
pypi:tokenizers@0.22.0uv.lock
declaration relationship not reporteddevelopment scope not reported
Open exact source ↗
LLM Testframeworkuba-ki-lab/llm-testframework
pypi:tokenizers@0.21.1uv.lock
declaration relationship not reporteddevelopment scope not reported
Open exact source ↗
Objection managementuba-ki-lab/objection-management
pypi:tokenizers@0.21.0uv.lock
declaration relationship not reporteddevelopment scope not reported
Open exact source ↗
Ressource Efficient Computer Visionuba-ki-lab/ressource-efficient-computer-vision
pypi:tokenizers@0.22.2uv.lock
declaration relationship not reporteddevelopment scope not reported
Open exact source ↗
Retrieval Evaluationuba-ki-lab/retrieval-evaluation
pypi:tokenizers@0.22.1uv.lock
declaration relationship not reporteddevelopment scope not reported
Open exact source ↗
Interpretation boundary

Exact identities in, explicit limits out

Only exact npm and PyPI tuples are enriched. Reported SPDX expressions are metadata; no compatibility, obligation or legal conclusion is inferred.

Retrieval, parsing, matching and publishing use no generative AI model.

i6eal (2026): Hugging Face Tokenizers — exact AI dependency evidence dossier, data state 19 Jul 2026. https://i6eal.de/en/tools/ki-abhaengigkeitsatlas/paket/tokenizers-7ba00902/

Reading this dossier

Does package presence prove that a provider is used?
No. Even a direct interface declaration does not establish configuration, credentials, procurement, data transfer or an API call.
Why are exact versions required?
OSV and registry metadata can be linked reproducibly only to an observed package@version tuple. The collector never substitutes a newest release for a range.
Does a missing row mean the dependency is absent?
No. It means not observed within the bounded files and repository checkpoint. Incomplete trees and parser failures remain explicit.

Need a permanent dependency evidence trail for another public code cohort?

We build source-backed data products with stable identities, reproducible joins and boundaries that remain visible.

Discuss a data projectExplore all tools
Related tools

Useful next steps

These tools complement the current result.

View all tools
MonitorGerman public-sector AI Code RadarWhich AI building blocks does German public administration publish as open code? Exact packages, containers, commit SHAs and file paths from openCode, with permanent dossiers and no deployment claim.MonitorGerman public-sector AI registerWhere does German public administration already use AI? Search published system profiles from federal, state and municipal bodies, with lifecycle, governance fields, stable identifiers and original evidence.Provenance checkPublic-sector AI system passportSee which public records can be linked to each published government AI system through stable identifiers or exact source-declared repository URLs—with a source for every edge and visible evidence boundaries.