有没有免费离线的开源文本文字转Embeddings向量的工具?
openai的收费,而且还需要连线,数据安全没办法保证
网友回复
transformers可以实现离线生成embeddings向量,支持python、nodejs,甚至浏览器都能运行:
<script defer type="module"> import { pipeline } from '//repo.bfw.wiki/bfwrepo/js/transformers@2.3.0.js'; const generateEmbeddings = await pipeline( 'feature-extraction', 'Xenova/all-MiniLM-L6-v2' ); const output1 = await generateEmbeddings("hello,my friend", { pooling: 'mean', normalize: true, }); console.log(output1) </script>