Lucene分词报错：”TokenStream contract violation: close() call missing”

全栈程序员站长

发布于 2022-09-15 17:53:58

6480

大家好，又见面了，我是你们的朋友全栈君。

Lucene使用IKAnalyzer分词时报错：”TokenStream contract violation: close() call missing” 解决办法是每次完成后必须调用关闭方法。

如果报错：java.lang.illegalstateexception: tokenstream contract violation: reset()/close() call missing,则要在tokenStream.incrementToken()，原因是lucene从4.6.0开始tokenstream使用方法更改的问题，在使用incrementtoken方法前必须调用reset方法，详见api http://lucene.apache.org/core/4_6_0/core/index.html 。

以下正确示例代码(第10行和22行调用reset()和close()方法)：

public Set<String> slicing(String text){
    Set<String> result = new HashSet<>();
    StringReader reader = null;
    TokenStream tokenStream = null;
    try {
        reader = new StringReader(text);
        tokenStream = analyzer.tokenStream("", reader);  
        CharTermAttribute charTermAttribute  = tokenStream.getAttribute(CharTermAttribute.class); 
        OffsetAttribute offsetAttribute = tokenStream.addAttribute(OffsetAttribute.class);  
        tokenStream.reset();
            while (tokenStream.incrementToken()) {  
                int startOffset = offsetAttribute.startOffset();  
                int endOffset   = offsetAttribute.endOffset();
                if((endOffset - startOffset) > 1){
                    String term = charTermAttribute.toString(); 
                    result.add(term);
                }
            }  
    } catch (IOException e) {
        e.printStackTrace();
    } finally{
        IOs.close(tokenStream, reader);
    }
    return result;
}

http://www.lizi.pw/archives/56

org.wltea.analyzer.lucene.IKAnalyzer

Exception in thread "main" java.lang.IllegalStateException: 词典尚未初始化，请先调用initial方法
at org.wltea.analyzer.dic.Dictionary.getSingleton(Dictionary.java:137)
at org.wltea.analyzer.core.CJKSegmenter.analyze(CJKSegmenter.java:80)
at org.wltea.analyzer.core.IKSegmenter.next(IKSegmenter.java:116)
at org.wltea.analyzer.lucene.IKTokenizer.incrementToken(IKTokenizer.java:88)

转载于:https://www.cnblogs.com/softidea/p/6084043.html

发布者：全栈程序员栈长，转载请注明出处：https://javaforall.cn/163117.html原文链接：https://javaforall.cn

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

如有侵权请联系 cloudcommunity@tencent.com 删除

java