blocks|key|3948369|text|我认为我们在这里得到了一个很好的例子：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3948370|https://colab.research.google.com/github/tensorflow/datasets/blob/master/docs/overview.ipynb#scrollTo=BC4pEXtkp4K-|offset|length|3948371|import+matplotlib.pyplot+as+plt
import+numpy+as+np
import+tensorflow+as+tf
import+tensorflow_datasets+as+tfds

#+where+mnsit+train+is+a+tf+dataset
mnist_train+=+tfds.load(name="mnist",+split=tfds.Split.TRAIN)
assert+isinstance(mnist_train,+tf.data.Dataset)

mnist_example,+=+mnist_train.take(1)
image,+label+=+mnist_example["image"],+mnist_example["label"]

plt.imshow(image.numpy()[:,+:,+0].astype(np.float32),+cmap=plt.get_cmap("gray"))
print("Label:+%25d"+%25+label.numpy())|code-block|syntax|javascript|3948372|因此，可以像访问字典一样访问数据集的每个单独组件。假设不同的数据集有不同的字段名称(波士顿住房不会有image和value，但可能有‘feature’和'target‘或'price'：|3948373|cnn+=+tfds.load(name="cnn_dailymail",+split=tfds.Split.TRAIN)
assert+isinstance(cnn,+tf.data.Dataset)
cnn_ex,+=+cnn.take(1)
print(cnn_ex)|3948374|返回一个dict()，里面有键'article'，'highlight‘，里面有numpy字符串。|3948375|entityMap|0|LINK|mutability|MUTABLE|url^0|0|0|36|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|X|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Y|8|@]|9|@$D|Z|E|10|1|11]]|A|$]]|$1|F|3|G|5|H|7|12|8|@]|9|@]|A|$I|J]]|$1|K|3|L|5|6|7|13|8|@]|9|@]|A|$]]|$1|M|3|N|5|H|7|14|8|@]|9|@]|A|$I|J]]|$1|O|3|P|5|6|7|15|8|@]|9|@]|A|$]]|$1|Q|3|-4|5|6|7|16|8|@]|9|@]|A|$]]]|R|$S|$5|T|U|V|A|$W|C]]]]

I think we get a good example here:

<a href="https://colab.research.google.com/github/tensorflow/datasets/blob/master/docs/overview.ipynb#scrollTo=BC4pEXtkp4K-" rel="noreferrer">https://colab.research.google.com/github/tensorflow/datasets/blob/master/docs/overview.ipynb#scrollTo=BC4pEXtkp4K-</a>

<pre><code>import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
import tensorflow_datasets as tfds

# where mnsit train is a tf dataset
mnist_train = tfds.load(name="mnist", split=tfds.Split.TRAIN)
assert isinstance(mnist_train, tf.data.Dataset)

mnist_example, = mnist_train.take(1)
image, label = mnist_example["image"], mnist_example["label"]

plt.imshow(image.numpy()[:, :, 0].astype(np.float32), cmap=plt.get_cmap("gray"))
print("Label: %d" % label.numpy())
</code></pre>

So each individual component of the dataset can be accessed sort of like a dictionary. Presumably different datasets have different field names (Boston housing won't have image, and value, but might have 'features' and 'target' or 'price':

<pre><code>cnn = tfds.load(name="cnn_dailymail", split=tfds.Split.TRAIN)
assert isinstance(cnn, tf.data.Dataset)
cnn_ex, = cnn.take(1)
print(cnn_ex)
</code></pre>

returns a dict() with keys ['article', 'highlight'] with numpy strings inside.

blocks|key|3948428|text|以下是我自己对这个问题的解决方案：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3948429|def+dataset2numpy(dataset,+steps=1):
++++"Helper+function+to+get+data/labels+back+from+TF+dataset"
++++iterator+=+dataset.make_one_shot_iterator()
++++next_val+=+iterator.get_next()
++++with+tf.Session()+as+sess:
++++++++for+_+in+range(steps):
+++++++++++inputs,+labels+=+sess.run(next_val)
+++++++++++yield+inputs,+labels|code-block|syntax|javascript|3948430|请注意，此函数将生成数据集批次的输入/标签。这些步骤控制将从数据集中取出多少批次。|3948431|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|L|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|M|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|J|$]]

Here is my own solution to the problem:

<pre><code>def dataset2numpy(dataset, steps=1):
 "Helper function to get data/labels back from TF dataset"
 iterator = dataset.make_one_shot_iterator()
 next_val = iterator.get_next()
 with tf.Session() as sess:
 for _ in range(steps):
 inputs, labels = sess.run(next_val)
 yield inputs, labels
</code></pre>

Please note that this function will yield inputs/labels of dataset batch. The steps control how many batches from a dataset will be taken out.

blocks|key|3948444|text|这对我很有效|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3948445|features+=+np.array([list(x[0].numpy())+for+x+in+list(ds_test)])
labels+=+np.array([x[1].numpy()+for+x+in+list(ds_test)])



#+NOTE:+ds_test+was+created
iris,+iris_info+=+tfds.load('iris',+with_info=True)
ds_orig+=+iris['train']
ds_orig+=+ds_orig.shuffle(150,+reshuffle_each_iteration=False)
ds_train+=+ds_orig.take(100)
ds_test+=+ds_orig.skip(100)|code-block|syntax|javascript|3948446|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

This worked for me

<pre><code>features = np.array([list(x[0].numpy()) for x in list(ds_test)])
labels = np.array([x[1].numpy() for x in list(ds_test)])



# NOTE: ds_test was created
iris, iris_info = tfds.load('iris', with_info=True)
ds_orig = iris['train']
ds_orig = ds_orig.shuffle(150, reshuffle_each_iteration=False)
ds_train = ds_orig.take(100)
ds_test = ds_orig.skip(100)
</code></pre>

blocks|key|3948459|text|如果您的tf.data.Dataset是批处理的，下面的代码将检索所有y标签：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|3948460|y+=+np.concatenate([y+for+x,+y+in+ds],+axis=0)|code-block|syntax|javascript|3948461|entityMap|0|LINK|mutability|MUTABLE|url|https://www.tensorflow.org/api_docs/python/tf/data/Dataset^0|4|F|4|F|0|0|0^^$0|@$1|2|3|4|5|6|7|S|8|@$9|T|A|U|B|C]]|D|@$9|V|A|W|1|X]]|E|$]]|$1|F|3|G|5|H|7|Y|8|@]|D|@]|E|$I|J]]|$1|K|3|-4|5|6|7|Z|8|@]|D|@]|E|$]]]|L|$M|$5|N|O|P|E|$Q|R]]]]

In case your <a href="https://www.tensorflow.org/api_docs/python/tf/data/Dataset" rel="noreferrer"><code>tf.data.Dataset</code></a> is batched, the following code will retrieve all the y labels:
<pre><code>y = np.concatenate([y for x, y in ds], axis=0)
</code></pre>

blocks|key|3948475|text|如果您可以将图像和标签保留为tf.Tensor，您可以这样做|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|3948476|images,+labels+=+tuple(zip(*dataset))|code-block|syntax|javascript|3948477|将数据集的效果想象为zip(images,+labels)。当我们想要找回图像和标签时，我们可以简单地unzip它。|3948478|如果需要numpy数组版本，请使用np.array()对其进行转换|3948479|images+=+np.array(images)
labels+=+np.array(labels)|3948480|entityMap|0|LINK|mutability|MUTABLE|url|https://realpython.com/python-zip-function/#unzipping-a-sequence^0|E|9|0|0|A|J|1F|5|0|0|H|A|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@$9|Z|A|10|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|11|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|12|8|@$9|13|A|14|B|C]]|D|@$9|15|A|16|1|17]]|E|$]]|$1|M|3|N|5|6|7|18|8|@$9|19|A|1A|B|C]]|D|@]|E|$]]|$1|O|3|P|5|H|7|1B|8|@]|D|@]|E|$I|J]]|$1|Q|3|-4|5|6|7|1C|8|@]|D|@]|E|$]]]|R|$S|$5|T|U|V|E|$W|X]]]]

If you are OK with keeping the images and labels as <code>tf.Tensor</code>s, you can do
<pre><code>images, labels = tuple(zip(*dataset))
</code></pre>
Think of the effect of the dataset as <code>zip(images, labels)</code>. When we want to get images and labels back, we can simply <a href="https://realpython.com/python-zip-function/#unzipping-a-sequence" rel="nofollow noreferrer">unzip</a> it.
If you need the numpy array version, convert them using <code>np.array()</code>:
<pre><code>images = np.array(images)
labels = np.array(labels)
</code></pre>

blocks|key|3952508|text|import+numpy+as+np
import+tensorflow+as+tf

batched_features+=+tf.constant([[[1,+3],+[2,+3]],
++++++++++++++++++++++++++++++++[[2,+1],+[1,+2]],
++++++++++++++++++++++++++++++++[[3,+3],+[3,+2]]],+shape=(3,+2,+2))
batched_labels+=+tf.constant([[0,+0],
++++++++++++++++++++++++++++++[1,+1],
++++++++++++++++++++++++++++++[0,+1]],+shape=(3,+2,+1))
dataset+=+tf.data.Dataset.from_tensor_slices((batched_features,+batched_labels))
classes+=+np.concatenate([y+for+x,+y+in+dataset],+axis=0)
unique+=+np.unique(classes,+return_counts=True)
labels_dict+=+dict(zip(unique[0],+unique[1]))
print(classes)
print(labels_dict)
#+{0:+3,+1:+3}|type|code-block|depth|inlineStyleRanges|entityRanges|data|syntax|javascript|3952509|unstyled|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|G|8|@]|9|@]|A|$B|C]]|$1|D|3|-4|5|E|7|H|8|@]|9|@]|A|$]]]|F|$]]

<pre class="lang-py prettyprint-override"><code>import numpy as np
import tensorflow as tf

batched_features = tf.constant([[[1, 3], [2, 3]],
 [[2, 1], [1, 2]],
 [[3, 3], [3, 2]]], shape=(3, 2, 2))
batched_labels = tf.constant([[0, 0],
 [1, 1],
 [0, 1]], shape=(3, 2, 1))
dataset = tf.data.Dataset.from_tensor_slices((batched_features, batched_labels))
classes = np.concatenate([y for x, y in dataset], axis=0)
unique = np.unique(classes, return_counts=True)
labels_dict = dict(zip(unique[0], unique[1]))
print(classes)
print(labels_dict)
# {0: 3, 1: 3}
</code></pre>

blocks|key|3952511|text|https://www.tensorflow.org/tutorials/images/classification|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|3952512|import+matplotlib.pyplot+as+plt

plt.figure(figsize=(10,+10))
for+images,+labels+in+train_ds.take(1):
++for+i+in+range(9):
++ax+=+plt.subplot(3,+3,+i+%2B+1)
++plt.imshow(images[i].numpy().astype("uint8"))
++plt.title(class_names[labels[i]])
++plt.axis("off")|code-block|syntax|javascript|3952513|entityMap|0|LINK|mutability|MUTABLE|url^0|0|1M|0|0|0^^$0|@$1|2|3|4|5|6|7|P|8|@]|9|@$A|Q|B|R|1|S]]|C|$]]|$1|D|3|E|5|F|7|T|8|@]|9|@]|C|$G|H]]|$1|I|3|-4|5|6|7|U|8|@]|9|@]|C|$]]]|J|$K|$5|L|M|N|C|$O|4]]]]

<a href="https://www.tensorflow.org/tutorials/images/classification" rel="nofollow noreferrer">https://www.tensorflow.org/tutorials/images/classification</a>
<pre><code>import matplotlib.pyplot as plt

plt.figure(figsize=(10, 10))
for images, labels in train_ds.take(1):
 for i in range(9):
 ax = plt.subplot(3, 3, i + 1)
 plt.imshow(images[i].numpy().astype(&quot;uint8&quot;))
 plt.title(class_names[labels[i]])
 plt.axis(&quot;off&quot;)
</code></pre>

blocks|key|3952534|text|您可以使用TF+dataset方法unbatch()对数据集取消批处理，然后可以轻松地从中检索数据和标签：|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|3952535|ds_labels=[]
for+images,+labels+in+ds.unbatch():
++++ds_labels.append(labels)+#+or+labels.numpy().argmax()+for+int+labels|code-block|syntax|javascript|3952536|entityMap|0|LINK|mutability|MUTABLE|url|https://www.tensorflow.org/api_docs/python/tf/data/Dataset#unbatch^0|H|7|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@$A|R|B|S|1|T]]|C|$]]|$1|D|3|E|5|F|7|U|8|@]|9|@]|C|$G|H]]|$1|I|3|-4|5|6|7|V|8|@]|9|@]|C|$]]]|J|$K|$5|L|M|N|C|$O|P]]]]

You can use TF Dataset method <a href="https://www.tensorflow.org/api_docs/python/tf/data/Dataset#unbatch" rel="nofollow noreferrer">unbatch</a>() to unbatch the dataset, then you can easily retrieve the data and the labels from it:
<pre><code>ds_labels=[]
for images, labels in ds.unbatch():
 ds_labels.append(labels) # or labels.numpy().argmax() for int labels
</code></pre>

there are plenty of examples how to create and use TensorFlow datasets, e.g.

<pre><code>dataset = tf.data.Dataset.from_tensor_slices((images, labels))
</code></pre>

My question is how to get back the data/labels from the TF dataset in numpy form? In other words want would be reverse operation of the line above, i.e. I have a TF dataset and want to get back images and labels from it.

How to extract data/labels back from TensorFlow dataset

有很多如何创建和使用TensorFlow数据集的示例，例如dataset = tf.data.Dataset.from_tensor_slices((images, labels))我的问题是如何以numpy的形式从TF数据集中取回数据/标签？换句话说，want将是上面这行的反向操作，即我有一个TF数据集，并希望从其中...

问如何从TensorFlow数据集中提取数据/标签
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何从TensorFlow数据集中提取数据/标签EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何从TensorFlow数据集中提取数据/标签
EN