java 使用CRF遇到的问题汇总

1、libCRFPP.so放在idea项目 resources下,打jar包时打在jar中。  

     jar包工具类

/*
 * Class NativeUtils is published under the The MIT License:
 *
 * Copyright (c) 2012 Adam Heinrich <adam@adamh.cz>
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 * 
 * The above copyright notice and this permission notice shall be included in all
 * copies or substantial portions of the Software.
 * 
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 
 * SOFTWARE.
 */
 package cz.adamh.utils;
 import java.io.
 *;
/** 
 * A simple library class which helps with loading dynamic libraries stored in the 
 * JAR archive. These libraries usualy contain implementation of some methods in
 * native code (using JNI - Java Native Interface).
 *  
 * @see http://adamheinrich.com/blog/2012/how-to-load-native-jni-library-from-jar
 * @see https://github.com/adamheinrich/native-utils
 *
 */public class NativeUtils {
      /**     * Private constructor - this class will never be instanced
      */    private NativeUtils() {
          }
      
      /**
      * Loads library from current JAR archive
      *      * The file from JAR is copied into system temporary directory and then loaded.
  The temporary file is deleted after exiting.
      * Method uses String as filename because the pathname is "abstract", not system-dependent.
      *      * @param path The path of file inside JAR as absolute path (beginning with '/'), e.g. /package/File.ext
     * @throws IOException If temporary file creation or read/write operation fails
     * @throws IllegalArgumentException If source file (param path) does not exist
     * @throws IllegalArgumentException If the path is not absolute or if the filename is shorter than three characters (restriction of {@see File#createTempFile(java.lang.String, java.lang.String)}).
     */    public static void loadLibraryFromJar(String path) throws IOException {
         if (!path.startsWith("/")) {
            throw new IllegalArgumentException("The path has to be absolute (start with '/').");
        }
         // Obtain filename from path
                 String[] parts = path.split("/");
        String filename = (parts.length > 1) ? parts[parts.length - 1] : null;
         // Split filename to prexif and suffix (extension)
                 String prefix = "";
        String suffix = null;
        if (filename != null) {            parts = filename.split("\\.", 2);
                    prefix = parts[0];
                                suffix = (parts.length > 1) ? "."+parts[parts.length - 1] : null;
         // Thanks, davs! :-)
                }
                 // Check if the filename is okay
                if (filename == null || prefix.length() < 3) {
                            throw new IllegalArgumentException("The filename has to be at least 3 characters long.");
                        }
                         // Prepare temporary file
                File temp = File.createTempFile(prefix, suffix);
                temp.deleteOnExit();
                 if (!temp.exists()) {
                    throw new FileNotFoundException("File " + temp.getAbsolutePath() + " does not exist.");
                }
                 // Prepare buffer for data copying
                         byte[] buffer = new byte[1024];
                         int readBytes;
                          // Open and check input stream
                         InputStream is = NativeUtils.class.getResourceAsStream(path);
                                 if (is == null) {
                                             throw new FileNotFoundException("File " + path + " was not found inside JAR.");
                }
                 // Open output stream and copy data between source file in JAR and the temporary file
                         OutputStream os = new FileOutputStream(temp);
                         try {            while ((readBytes = is.read(buffer)) != -1) {
                os.write(buffer, 0, readBytes);
            }        } finally {            
// If read/write fails, close streams safely before throwing an exception            
os.close();            is.close();        }         
// Finally, load the library        System.load(temp.getAbsolutePath());    }}

2、需要安装CRF相关信息

  网上找到两种方式:

  出现这种情况的原因是找不到libcrfpp.so.0等库文件,解决方案一为(貌似此方法对root用户不管用):

  1. 修改/etc/ld.so.conf文件
  2. 加入include /usr/local/lib
  3. 执行/sbin/ldconfig -v,刷新LIB库

  解决方案二为建立以下符号链接:

 ln -s /usr/local/lib/libcrfpp.a /usr/lib/libcrfpp.a   ln -s /usr/local/lib/libcrfpp.so /usr/lib/libcrfpp.so   ln -s /usr/local/lib/libcrfpp.so.0 /usr/lib/libcrfpp.so.0 连接 https://zxdcs.github.io/post/16/crf_java/  python 用户连接 http://midday.me/article/94d6bd4973264e1a801f8445904a810d  公司线上环境是docker容器方式不可用,实际用的方式一。 

3、再有是连接库使用训练出来的model文件。路径网上均采用相对路劲,实际容器中不可用,采用绝对路径后解决。

Caused by: java.lang.RuntimeException: feature_index.cpp(193) [mmap_.open(model_filename)] mmap.h(153) [(fd = ::open(filename, flag | O_BINARY)) >= 0] open failed: model at org.chasen.crfpp.CRFPPJNI.new_Tagger(Native Method) at org.chasen.crfpp.Tagger.<init>(Tagger.java:183) at com.jd.app.server.LoadCRFModel.<clinit>(LoadCRFModel.java:89) ... 63 more

  这个错误可以采用3解决。

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏贾志刚-OpenCV学堂

Android平台上OpenCV 深度网络实现对象检测

Android平台上OpenCV 深度网络实现对象检测 自OpenCV3.3发布包含深度神经网络(DNN)模块的SDK以后,OpenCV4Android SDK...

4627
来自专栏Python

利用ForgeryPy生成虚拟数据

 在程序研发过程中,我们往往需要大量的虚拟实验数据。Python中有多个包可以用于生成虚拟数据,其中功能较为完善的是ForgeryPy。

670
来自专栏写代码的海盗

spark在yarn-cluster上面执行报错

在单机模式下执行成功的spark程序,在yarn上面就报错。异常信息如下: 1 14/08/14 02:05:42 INFO DAGScheduler: Co...

2995
来自专栏施炯的IoT开发专栏

转贴-WP7开发资源大收集

文章作者: jason huang 文章标签: Microsoft, Windows Phone 7, WP7 转贴链接: WP7开发资源大收集 这里收集...

1778
来自专栏Hadoop实操

Spark2Streaming读Kerberos环境的Kafka并写数据到HBase

2722
来自专栏雨尘分享

iOS 10.3 Label高度计算问题 (UITableView+FDTemplateLayoutCell)1.前言2.问题发现3.解决问题4.列举下解决问题的方法

3654
来自专栏LinXunFeng的专栏

iOS - Swift 仿微信小红点(无数字)

1123
来自专栏生信技能树

肿瘤全外显子测序数据分析流程大放送

这个一个肿瘤外显子项目的文章发表并且公布的公共数据,我这里给出全套分析流程代码。只需要你肯实践,就可以运行成功。 PS:有些后起之秀自己运营公众号或者博客喜欢批...

6918
来自专栏cmazxiaoma的架构师之路

Android多线程+单线程+断点续传+进度条显示下载

2243
来自专栏数据库新发现

Oracle诊断案例-Spfile案例一则

情况说明: 系统:SUN Solaris8 数据库版本:9203 问题描述:工程人员报告,数据库在重新启动时无法正常启动.检查发现UNDO表空间丢失. 问题诊断...

713

扫码关注云+社区