npc_gzip笔记 - plus studio

plus sign

发布于 2024-02-29 08:05:34

1300

发布于 2024-02-29 08:05:34

文章被收录于专栏：个人博客

npc_gzip笔记

论文笔记

npc_gzip 的论文名叫做 "Low-Resource" Text Classification: A Parameter-Free Classification Method with Compressors ,意为不需要参数，使用压缩器的文本分类方法。论文的代码也只有仅仅的十四行，就在部分数据集上取得了超越 bert 的效果。

npc_gzip由一个无损压缩器，一个基于距离的度量函数和K近邻算法组成。

使用压缩器进行分类的直觉是有两方面 1. 压缩器擅长捕捉规律性； 2. 来自同一类别的对象比不同类别的对象具有更多的规律性。

x1 = Japan's Seiko Epson Corp. has developed a 12-gram flying microrobot.

x2 = The latest tiny flying robot has been unveiled in Japan.

x3 = Michael Phelps won the gold medal in the 400 individual medley.

核心代码真的真的就非常简单了

import gzip2 
import numpy as np
for ( x1 , _ ) in test_set :
	Cx1 = len ( gzip . compress ( x1 . encode () ) )
	distance_from_x1 = []
	for ( x2 , _ ) in training_set :
		Cx2 = len ( gzip . compress ( x2 . encode () )
		x1x2 = " " . join ([ x1 , x2 ])
		Cx1x2 = len ( gzip . compress ( x1x2 . encode () )
		ncd = ( Cx1x2 - min ( Cx1 , Cx2 )) / max ( Cx1 , Cx2 )
		distance_from_x1 . append ( ncd )
		sorted_idx = np . argsort ( np . array ( distance_from_x1 ) )
	top_k_class = training_set [ sorted_idx [: k ] , 1]
	predict_class = max ( set ( top_k_class ) , key = top_k_class . count )

这种方法是 DNN 的简单、轻量级和通用的替代方案。很简单，因为它不需要任何预处理或训练。它的轻量级在于它不需要参数或 GPU 资源进行分类。由于压缩器是数据类型不可知的，非参数方法不会带来潜在的假设。

代码实践

作者在GitHub上开源了他的代码 npc_gzip .我们先把代码拉到本地

git clone https://github.com/bazingagin/npc_gzip

接下来安装依赖项，有条件的话创建一个虚拟环境

cd ./npc_gzip
pip install -r requirements.txt

安装完了之后运行main_text.py

python main_text.py

注意，如果你遇到了这个问题

Traceback (most recent call last):
  File "main_text.py", line 2, in <module>
    from data import *
  File "/home/npc_gzip/data.py", line 12, in <module>
    from datasets import load_dataset
  File "/home/.conda/envs/npc_gzip/lib/python3.7/site-packages/datasets/__init__.py", line 43, in <module>
    from .arrow_dataset import Dataset
  File "/home/.conda/envs/npc_gzip/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 59, in <module>
    from huggingface_hub import HfApi, HfFolder
  File "/home/.conda/envs/npc_gzip/lib/python3.7/site-packages/huggingface_hub/__init__.py", line 322, in __getattr__
    submod = importlib.import_module(submod_path)
  File "/home/.conda/envs/npc_gzip/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/.conda/envs/npc_gzip/lib/python3.7/site-packages/huggingface_hub/hf_api.py", line 32, in <module>
    import requests
  File "/home/.conda/envs/npc_gzip/lib/python3.7/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/home/.conda/envs/npc_gzip/lib/python3.7/site-packages/urllib3/__init__.py", line 42, in <module>
    "urllib3 v2.0 only supports OpenSSL 1.1.1+, currently "
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.0.2u  20 Dec 2019'. See: https://github.com/urllib3/urllib3/issues/2168

urllib3 v2.0（您安装的版本）需要 OpenSSL 1.1.1+ 才能正常工作，因为它依赖于 OpenSSL 1.1 的一些新功能.

安装旧版本即可解决