Search results

Incorrect result of Korean sample with character '고' in ARM64 architecture

Status: Open.

#9 In NLPOptimize/flash-tokenizer;

· vsantosu opened on Dec 11, 2025

“张” value is error

💢 BugWe are exploring different optimization methods and seeking ideas to benchmark these techniques or iWe are exploring different optimization methods and seeking ideas to benchmark these techniques or i

Status: Open.

#7 In NLPOptimize/flash-tokenizer;

· yegeli opened on May 15, 2025

如何在C++11进行调用

🤲🏻 Feature request 🤲🏻New feature or requestNew feature or request

Status: Open.

#6 In NLPOptimize/flash-tokenizer;

· yourbikun opened on Apr 28, 2025

C#能调用吗？

🤲🏻 Feature request 🤲🏻New feature or requestNew feature or request

Status: Open.

#5 In NLPOptimize/flash-tokenizer;

· dfengpo opened on Apr 24, 2025

question: does it work with PreTrainedTokenizerFast

❔QuestionQuestionQuestion

🤲🏻 Feature request 🤲🏻New feature or requestNew feature or request

Status: Open.

#4 In NLPOptimize/flash-tokenizer;

· geraldstanje opened on Apr 9, 2025