InKhmer
Building the digital future
About
Projects
Blog
Tools
Learn
Contact
KM
ខ្មែរ
EN
English
All terms
tokenisation (ការបំបែកជា token)
Cutting text into small units (tokens) a model can process.
Definition
Cutting text into small units (tokens) a model can process.
See also
Token
Natural language processing (NLP)