blocks|key|1064218|text|一种简单的方法是对核心图像数据进行散列。对于PNG，您可以通过只计算“关键块”(即以大写字母开头的块)来实现这一点。JPEG具有类似但更简单的文件结构。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1064219|ImageMagick中的可视散列在对图像进行散列时对其进行解压缩。在您的情况下，您可以立即散列压缩的图像数据，因此(如果实现正确)它应该和散列原始文件一样快。|1064220|这是一个很小的Python脚本，用来说明这个想法。它可能对您起作用，也可能不起作用，但至少应该给出我的意思的一个指示:)|1064221|import+struct
import+os
import+hashlib

def+png(fh):
++++hash+=+hashlib.md5()
++++assert+fh.read(8)[1:4]+==+"PNG"
++++while+True:
++++++++try:
++++++++++++length,+=+struct.unpack(">i",fh.read(4))
++++++++except+struct.error:
++++++++++++break
++++++++if+fh.read(4)+==+"IDAT":
++++++++++++hash.update(fh.read(length))
++++++++++++fh.read(4)+#+CRC
++++++++else:
++++++++++++fh.seek(length%2B4,os.SEEK_CUR)
++++print+"Hash:+%25r"+%25+hash.digest()

def+jpeg(fh):
++++hash+=+hashlib.md5()
++++assert+fh.read(2)+==+"\xff\xd8"
++++while+True:
++++++++marker,length+=+struct.unpack(">2H",+fh.read(4))
++++++++assert+marker+&+0xff00+==+0xff00
++++++++if+marker+==+0xFFDA:+#+Start+of+stream
++++++++++++hash.update(fh.read())
++++++++++++break
++++++++else:
++++++++++++fh.seek(length-2,+os.SEEK_CUR)
++++print+"Hash:+%25r"+%25+hash.digest()


if+__name__+==+'__main__':
++++png(file("sample.png"))
++++jpeg(file("sample.jpg"))|code-block|syntax|javascript|1064222|entityMap^0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|N|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|O|8|@]|9|@]|A|$]]|$1|F|3|G|5|H|7|P|8|@]|9|@]|A|$I|J]]|$1|K|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|L|$]]

One simple way to do it is to hash the core image data. For PNG, you could do this by counting only the "critical chunks" (i.e. the ones starting with capital letters). JPEG has a similar but simpler file structure.

The visual hash in ImageMagick decompresses the image as it hashes it. In your case, you could hash the compressed image data right away, so (if implemented correctly) a it should be just as quick as hashing the raw file.

This is a small Python script illustrating the idea. It may or may not work for you, but it should at least give an indication to what I mean :)

<pre><code>import struct
import os
import hashlib

def png(fh):
 hash = hashlib.md5()
 assert fh.read(8)[1:4] == "PNG"
 while True:
 try:
 length, = struct.unpack("&gt;i",fh.read(4))
 except struct.error:
 break
 if fh.read(4) == "IDAT":
 hash.update(fh.read(length))
 fh.read(4) # CRC
 else:
 fh.seek(length+4,os.SEEK_CUR)
 print "Hash: %r" % hash.digest()

def jpeg(fh):
 hash = hashlib.md5()
 assert fh.read(2) == "\xff\xd8"
 while True:
 marker,length = struct.unpack("&gt;2H", fh.read(4))
 assert marker &amp; 0xff00 == 0xff00
 if marker == 0xFFDA: # Start of stream
 hash.update(fh.read())
 break
 else:
 fh.seek(length-2, os.SEEK_CUR)
 print "Hash: %r" % hash.digest()


if __name__ == '__main__':
 png(file("sample.png"))
 jpeg(file("sample.jpg"))
</code></pre>

blocks|key|20856|text|使用Python+Imaging来提取图片数据(iPython中的示例)要容易得多：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|20857|In+[1]:+import+Image

In+[2]:+import+hashlib

In+[3]:+im+=+Image.open('foo.jpg')

In+[4]:+hashlib.md5(im.tobytes()).hexdigest()
Out[4]:+'171e2774b2549bbe0e18ed6dcafd04d5'|code-block|syntax|javascript|20858|这适用于PIL可以处理的任何类型的图像。tobytes方法返回包含像素数据的字符串。|offset|length|style|CODE|20859|顺便说一下，MD5散列现在被认为是相当弱的。最好使用SHA512：|20860|In+[6]:+hashlib.sha512(im.tobytes()).hexdigest()
Out[6]:+'6361f4a2722f221b277f81af508c9c1d0385d293a12958e2c56a57edf03da16f4e5b715582feef3db31200db67146a4b52ec3a8c445decfc2759975a98969c34'|20861|在我的机器上，计算2500x1600JPEG的MD5校验和大约需要0.07秒。使用SHA512，它需要0,10秒。完整示例：|20862|#!/usr/bin/env+python3

from+PIL+import+Image
import+hashlib
import+sys

im+=+Image.open(sys.argv[1])
print(hashlib.sha512(im.tobytes()).hexdigest(),+end="")|20863|对于电影，您可以使用例如ffmpeg从其中提取帧，然后按照上面所示的方式处理它们。|20864|entityMap|0|LINK|mutability|MUTABLE|url|http://ffmpeg.org/^0|0|0|K|7|0|0|0|0|0|C|6|0|0^^$0|@$1|2|3|4|5|6|7|14|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|15|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|16|8|@$I|17|J|18|K|L]]|9|@]|A|$]]|$1|M|3|N|5|6|7|19|8|@]|9|@]|A|$]]|$1|O|3|P|5|D|7|1A|8|@]|9|@]|A|$E|F]]|$1|Q|3|R|5|6|7|1B|8|@]|9|@]|A|$]]|$1|S|3|T|5|D|7|1C|8|@]|9|@]|A|$E|F]]|$1|U|3|V|5|6|7|1D|8|@]|9|@$I|1E|J|1F|1|1G]]|A|$]]|$1|W|3|-4|5|6|7|1H|8|@]|9|@]|A|$]]]|X|$Y|$5|Z|10|11|A|$12|13]]]]

It is much easier to use the Python Imaging Library to extract the picture data (example in iPython):

<pre><code>In [1]: import Image

In [2]: import hashlib

In [3]: im = Image.open('foo.jpg')

In [4]: hashlib.md5(im.tobytes()).hexdigest()
Out[4]: '171e2774b2549bbe0e18ed6dcafd04d5'
</code></pre>

This works on any type of image that PIL can handle. The <code>tobytes</code> method returns the a string containing the pixel data. 

BTW, the MD5 hash is now seen as pretty weak. Better to use SHA512:

<pre><code>In [6]: hashlib.sha512(im.tobytes()).hexdigest()
Out[6]: '6361f4a2722f221b277f81af508c9c1d0385d293a12958e2c56a57edf03da16f4e5b715582feef3db31200db67146a4b52ec3a8c445decfc2759975a98969c34'
</code></pre>

On my machine, calculating the MD5 checksum for a 2500x1600 JPEG takes around 0.07 seconds. Using SHA512, it takes 0,10 seconds. Complete example:

<pre><code>#!/usr/bin/env python3

from PIL import Image
import hashlib
import sys

im = Image.open(sys.argv[1])
print(hashlib.sha512(im.tobytes()).hexdigest(), end="")
</code></pre>

For movies, you can extract frames from them with e.g. <a href="http://ffmpeg.org/" rel="noreferrer">ffmpeg</a>, and then process them as shown above.

blocks|key|2079770|text|我会使用元数据剥离器来预处理您的散列：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2079771|从ImageMagick包中你可以看到...|2079772|mogrify+-strip+blah.jpg|code-block|syntax|javascript|2079773|如果你这样做了|2079774|identify+-list+format+|2079775|显然，它适用于所有引用的格式。|2079776|entityMap^0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|R|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|S|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|T|8|@]|9|@]|A|$]]|$1|K|3|L|5|F|7|U|8|@]|9|@]|A|$G|H]]|$1|M|3|N|5|6|7|V|8|@]|9|@]|A|$]]|$1|O|3|-4|5|6|7|W|8|@]|9|@]|A|$]]]|P|$]]

I would use a metadata stripper to preprocess your hashing : 

From ImageMagick package you have ...

<pre><code>mogrify -strip blah.jpg
</code></pre>

and if you do 

<pre><code>identify -list format 
</code></pre>

it apparently works with all the cited formats.

blocks|key|20959|text|您可以使用ImageMagick套件中的stream：|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|20960|$+stream+-map+rgb+-storage-type+short+image.tif+-+%7C+sha256sum
d39463df1060efd4b5a755b09231dcbc3060e9b10c5ba5760c7dbcd441ddcd64++-|code-block|syntax|javascript|20961|或|20962|$+sha256sum+<(stream+-map+rgb+-storage-type+short+image.tif+-)
d39463df1060efd4b5a755b09231dcbc3060e9b10c5ba5760c7dbcd441ddcd64++/dev/fd/63|20963|此示例适用于RGB格式的TIFF文件，每个样本具有16位(即每像素48位)。所以我使用map+to+rgb和一个short+storage-type+(如果RGB值是8位，你可以在这里使用char+)。|style|CODE|20964|此方法报告与verbose+Imagemagick+identify命令报告的相同signature哈希：|20965|$+identify+-verbose+image.tif+%7C+grep+signature
signature:+d39463df1060efd4b5a755b09231dcbc3060e9b10c5ba5760c7dbcd441ddcd64|20966|(对于ImageMagick+v6.x；identify在版本7上报告的散列与使用stream获得的散列是different的，但后者可以由任何能够提取原始位图数据的工具重现-例如对于某些图像类型的dcraw。)|20967|entityMap|0|LINK|mutability|MUTABLE|url|https://www.imagemagick.org/|1|https://imagemagick.org/script/stream.php|2|https://www.imagemagick.org/script/command-line-options.php#map_stream_|3|https://www.imagemagick.org/script/command-line-options.php#storage-type|4|https://www.imagemagick.org/discourse-server/viewtopic.php?t=23399#p98342^0|5|B|0|K|6|1|0|0|0|0|1E|3|1K|5|2M|4|17|3|2|1Q|C|3|0|Q|8|15|9|0|0|K|8|15|6|2R|5|1H|9|4|0^^$0|@$1|2|3|4|5|6|7|1C|8|@]|9|@$A|1D|B|1E|1|1F]|$A|1G|B|1H|1|1I]]|C|$]]|$1|D|3|E|5|F|7|1J|8|@]|9|@]|C|$G|H]]|$1|I|3|J|5|6|7|1K|8|@]|9|@]|C|$]]|$1|K|3|L|5|F|7|1L|8|@]|9|@]|C|$G|H]]|$1|M|3|N|5|6|7|1M|8|@$A|1N|B|1O|O|P]|$A|1P|B|1Q|O|P]|$A|1R|B|1S|O|P]]|9|@$A|1T|B|1U|1|1V]|$A|1W|B|1X|1|1Y]]|C|$]]|$1|Q|3|R|5|6|7|1Z|8|@$A|20|B|21|O|P]|$A|22|B|23|O|P]]|9|@]|C|$]]|$1|S|3|T|5|F|7|24|8|@]|9|@]|C|$G|H]]|$1|U|3|V|5|6|7|25|8|@$A|26|B|27|O|P]|$A|28|B|29|O|P]|$A|2A|B|2B|O|P]]|9|@$A|2C|B|2D|1|2E]]|C|$]]|$1|W|3|-4|5|6|7|2F|8|@]|9|@]|C|$]]]|X|$Y|$5|Z|10|11|C|$12|13]]|14|$5|Z|10|11|C|$12|15]]|16|$5|Z|10|11|C|$12|17]]|18|$5|Z|10|11|C|$12|19]]|1A|$5|Z|10|11|C|$12|1B]]]]

You can use <a href="https://imagemagick.org/script/stream.php" rel="nofollow noreferrer">stream</a> which is part of the <a href="https://www.imagemagick.org" rel="nofollow noreferrer">ImageMagick</a> suite:
<pre><code>$ stream -map rgb -storage-type short image.tif - | sha256sum
d39463df1060efd4b5a755b09231dcbc3060e9b10c5ba5760c7dbcd441ddcd64 -
</code></pre>
or
<pre><code>$ sha256sum &lt;(stream -map rgb -storage-type short image.tif -)
d39463df1060efd4b5a755b09231dcbc3060e9b10c5ba5760c7dbcd441ddcd64 /dev/fd/63
</code></pre>
This example is for a TIFF file which is RGB with 16 bits per sample (i.e. 48 bits per pixel). So I use <a href="https://www.imagemagick.org/script/command-line-options.php#map_stream_" rel="nofollow noreferrer">map</a> to <code>rgb</code> and a <code>short</code> <a href="https://www.imagemagick.org/script/command-line-options.php#storage-type" rel="nofollow noreferrer">storage-type</a> (you can use <code>char</code> here if the RGB values are 8-bits).
This method reports the same <code>signature</code> hash that the verbose Imagemagick <code>identify</code> command reports:
<pre><code>$ identify -verbose image.tif | grep signature
signature: d39463df1060efd4b5a755b09231dcbc3060e9b10c5ba5760c7dbcd441ddcd64
</code></pre>
(for ImageMagick v6.x; the hash reported by <code>identify</code> on version 7 is <a href="https://www.imagemagick.org/discourse-server/viewtopic.php?t=23399#p98342" rel="nofollow noreferrer">different</a> to that obtained using <code>stream</code>, but the latter may be reproduced by any tool capable of extracting the raw bitmap data - such as <code>dcraw</code> for some image types.)

I'm writing a script to calculate the MD5 sum of an image excluding the EXIF tag.

In order to do this accurately, I need to know where the EXIF tag is located in the file (beginning, middle, end) so that I can exclude it.

How can I determine where in the file the tag is located?

The images that I am scanning are in the format TIFF, JPG, PNG, BMP, DNG, CR2, NEF, and some videos MOV, AVI, and MPG.

Compute hash of only the core image data (excluding metadata) for an image

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我正在编写一个脚本来计算不包括EXIF标记的图像的MD5和。为了准确地做到这一点，我需要知道EXIF标记在文件中的位置(开始、中间、结束)，以便我可以排除它。如何确定标记在文件中的位置？我正在扫描的图像格式是TIFF、JPG、PNG、BMP、DNG、CR2、NEF，以及一些视频MOV、AVI和MPG。

问仅计算镜像的核心镜像数据(不包括元数据)的哈希值
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问仅计算镜像的核心镜像数据(不包括元数据)的哈希值EN