discourse-mirrors/discourse-ai
0
0
Fork 0
mirror of https://ghfast.top/https://github.com/discourse/discourse-ai.git synced 2026-05-25 00:54:32 +08:00
Code Issues Projects Releases Packages Wiki Activity Actions
discourse-ai/lib/tokenizer
History
Exact
Rafael dos Santos Silva 67a1257b89
FEATURE: Gemini Tokenizer (#1088)
2025-01-23 18:20:35 -03:00
..
all_mpnet_base_v2_tokenizer.rb DEV: port directory structure to Zeitwerk (#319) 2023-11-29 15:17:46 +11:00
anthropic_tokenizer.rb DEV: port directory structure to Zeitwerk (#319) 2023-11-29 15:17:46 +11:00
basic_tokenizer.rb FEATURE: Gemini Tokenizer (#1088) 2025-01-23 18:20:35 -03:00
bert_tokenizer.rb DEV: port directory structure to Zeitwerk (#319) 2023-11-29 15:17:46 +11:00
bge_large_en_tokenizer.rb DEV: port directory structure to Zeitwerk (#319) 2023-11-29 15:17:46 +11:00
bge_m3_tokenizer.rb FEATURE: Add BGE-M3 embeddings support (#569) 2024-04-10 17:24:01 -03:00
gemini_tokenizer.rb FEATURE: Gemini Tokenizer (#1088) 2025-01-23 18:20:35 -03:00
llama3_tokenizer.rb FEATURE: Llama 3 tokenizer (#615) 2024-05-13 12:45:52 -03:00
mixtral_tokenizer.rb Mixtral (#376) 2023-12-26 14:49:55 -03:00
multilingual_e5_large_tokenizer.rb DEV: port directory structure to Zeitwerk (#319) 2023-11-29 15:17:46 +11:00
open_ai_gpt4o_tokenizer.rb FEATURE: GPT4o Tokenizer (#721) 2024-07-22 15:26:14 -03:00
open_ai_tokenizer.rb FIX/REFACTOR: FoldContent revamp (#866) 2024-10-25 11:51:17 -03:00