java 使用CRF遇到的问题汇总

1、libCRFPP.so放在idea项目 resources下,打jar包时打在jar中。  

     jar包工具类

/*
 * Class NativeUtils is published under the The MIT License:
 *
 * Copyright (c) 2012 Adam Heinrich <adam@adamh.cz>
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 * 
 * The above copyright notice and this permission notice shall be included in all
 * copies or substantial portions of the Software.
 * 
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 
 * SOFTWARE.
 */
 package cz.adamh.utils;
 import java.io.
 *;
/** 
 * A simple library class which helps with loading dynamic libraries stored in the 
 * JAR archive. These libraries usualy contain implementation of some methods in
 * native code (using JNI - Java Native Interface).
 *  
 * @see http://adamheinrich.com/blog/2012/how-to-load-native-jni-library-from-jar
 * @see https://github.com/adamheinrich/native-utils
 *
 */public class NativeUtils {
      /**     * Private constructor - this class will never be instanced
      */    private NativeUtils() {
          }
      
      /**
      * Loads library from current JAR archive
      *      * The file from JAR is copied into system temporary directory and then loaded.
  The temporary file is deleted after exiting.
      * Method uses String as filename because the pathname is "abstract", not system-dependent.
      *      * @param path The path of file inside JAR as absolute path (beginning with '/'), e.g. /package/File.ext
     * @throws IOException If temporary file creation or read/write operation fails
     * @throws IllegalArgumentException If source file (param path) does not exist
     * @throws IllegalArgumentException If the path is not absolute or if the filename is shorter than three characters (restriction of {@see File#createTempFile(java.lang.String, java.lang.String)}).
     */    public static void loadLibraryFromJar(String path) throws IOException {
         if (!path.startsWith("/")) {
            throw new IllegalArgumentException("The path has to be absolute (start with '/').");
        }
         // Obtain filename from path
                 String[] parts = path.split("/");
        String filename = (parts.length > 1) ? parts[parts.length - 1] : null;
         // Split filename to prexif and suffix (extension)
                 String prefix = "";
        String suffix = null;
        if (filename != null) {            parts = filename.split("\\.", 2);
                    prefix = parts[0];
                                suffix = (parts.length > 1) ? "."+parts[parts.length - 1] : null;
         // Thanks, davs! :-)
                }
                 // Check if the filename is okay
                if (filename == null || prefix.length() < 3) {
                            throw new IllegalArgumentException("The filename has to be at least 3 characters long.");
                        }
                         // Prepare temporary file
                File temp = File.createTempFile(prefix, suffix);
                temp.deleteOnExit();
                 if (!temp.exists()) {
                    throw new FileNotFoundException("File " + temp.getAbsolutePath() + " does not exist.");
                }
                 // Prepare buffer for data copying
                         byte[] buffer = new byte[1024];
                         int readBytes;
                          // Open and check input stream
                         InputStream is = NativeUtils.class.getResourceAsStream(path);
                                 if (is == null) {
                                             throw new FileNotFoundException("File " + path + " was not found inside JAR.");
                }
                 // Open output stream and copy data between source file in JAR and the temporary file
                         OutputStream os = new FileOutputStream(temp);
                         try {            while ((readBytes = is.read(buffer)) != -1) {
                os.write(buffer, 0, readBytes);
            }        } finally {            
// If read/write fails, close streams safely before throwing an exception            
os.close();            is.close();        }         
// Finally, load the library        System.load(temp.getAbsolutePath());    }}

2、需要安装CRF相关信息

  网上找到两种方式:

  出现这种情况的原因是找不到libcrfpp.so.0等库文件,解决方案一为(貌似此方法对root用户不管用):

  1. 修改/etc/ld.so.conf文件
  2. 加入include /usr/local/lib
  3. 执行/sbin/ldconfig -v,刷新LIB库

  解决方案二为建立以下符号链接:

 ln -s /usr/local/lib/libcrfpp.a /usr/lib/libcrfpp.a   ln -s /usr/local/lib/libcrfpp.so /usr/lib/libcrfpp.so   ln -s /usr/local/lib/libcrfpp.so.0 /usr/lib/libcrfpp.so.0 连接 https://zxdcs.github.io/post/16/crf_java/  python 用户连接 http://midday.me/article/94d6bd4973264e1a801f8445904a810d  公司线上环境是docker容器方式不可用,实际用的方式一。 

3、再有是连接库使用训练出来的model文件。路径网上均采用相对路劲,实际容器中不可用,采用绝对路径后解决。

Caused by: java.lang.RuntimeException: feature_index.cpp(193) [mmap_.open(model_filename)] mmap.h(153) [(fd = ::open(filename, flag | O_BINARY)) >= 0] open failed: model at org.chasen.crfpp.CRFPPJNI.new_Tagger(Native Method) at org.chasen.crfpp.Tagger.<init>(Tagger.java:183) at com.jd.app.server.LoadCRFModel.<clinit>(LoadCRFModel.java:89) ... 63 more

  这个错误可以采用3解决。

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏祝威廉

对Spark的那些【魔改】

这两年做streamingpro时,不可避免的需要对Spark做大量的增强。就如同我之前吐槽的,Spark大量使用了new进行对象的创建,导致里面的实现基本没有...

781
来自专栏JadePeng的技术博客

HtmlAgilityPack 库 StackOverflowException 解决方案

     最近试用HtmlAgilityPack 来解析html,试用过程中程序会抛出StackOverflowException异常,从MSDN上可以看到,从...

2645
来自专栏Spark学习技巧

hadoop系列之MR经典案例分享二

4、MapReduce的join(hive已经实现) http://database.51cto.com/art/201410/454277.htm ? 这三种...

31210
来自专栏叁金大数据

EmguCV学习——视频与图片互转

其实视频转图片在上篇文章中已经有些眉目了,其实就是按帧读取视频,然后把帧保存就ok。然后自己再加个进度条美化一下。。。这代码简单易懂,还是直接上代码吧。

731
来自专栏Java与Android技术栈

当RxJava遇到AOP

公司打算开发一款全新的To C产品,因此我开始做一些搭建框架的事儿以及POC。新的产品能够使用一些比较新的技术,在新产品中我大量使用了Rx。这就导致了原先的AO...

642
来自专栏Spark学习技巧

textFile构建RDD的分区及compute计算策略

1,textFile A),第一点,就是输入格式,key,value类型及并行度的意义。 def textFile( path: String, mi...

1957
来自专栏葬爱家族

Android高德之旅(13)公交搜索

在LBS系统中怎能少了公交这个角色,大多数城市中,主要的公共交通工具还是公交车,高德地图当然会有相应的api来查询公交信息,这篇文章就来记录下公交信息查询。

562
来自专栏菩提树下的杨过

Enterprise Library 4.1学习笔记7----缓存应用程序块之SqlDependency

本文是在Artech“[原创]Enterprise Library深入解析与灵活应用(2): 通过SqlDependency实现Cache和Database的同...

19710
来自专栏草根专栏

设计模式学习(二): 观察者模式 (C#)

《深入浅出设计模式》学习笔记第二章 需求: 开发一套气象监测应用,如图: ? 气象站,目前有三种装置,温度、湿度和气压感应装置。 WeatherData对象追踪...

3145
来自专栏码匠的流水账

spring security reactive获取security context

本文主要研究下reactive模式下的spring security context的获取。

882

扫码关注云+社区