# -------------------------------------------- # CITATION file created with {cffr} R package # See also: https://docs.ropensci.org/cffr/ # -------------------------------------------- cff-version: 1.2.0 message: 'To cite package "rtiktoken" in publications use:' type: software license: MIT title: 'rtiktoken: A Byte-Pair-Encoding (BPE) Tokenizer for OpenAI''s Large Language Models' version: 0.0.6 doi: 10.32614/CRAN.package.rtiktoken identifiers: - type: url value: https://github.com/DavZim/rtiktoken/ abstract: A thin wrapper around the tiktoken-rs crate, allowing to encode text into Byte-Pair-Encoding (BPE) tokens and decode tokens back to text. This is useful to understand how Large Language Models (LLMs) perceive text. authors: - family-names: Zimmermann-Kollenda given-names: David email: david_j_zimmermann@hotmail.com - name: Roger Zurawicki - name: Authors of the dependent Rust crates repository: https://davzim.r-universe.dev repository-code: https://github.com/DavZim/rtiktoken commit: 3cb11358a32cbbe3f50cf9849b145b20afed93a0 url: https://davzim.github.io/rtiktoken/ contact: - family-names: Zimmermann-Kollenda given-names: David email: david_j_zimmermann@hotmail.com