前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >tf.compat.v1.ReaderBase

tf.compat.v1.ReaderBase

作者头像
狼啸风云
发布2022-06-05 10:56:39
1580
发布2022-06-05 10:56:39
举报

参考 tf.compat - 云+社区 - 腾讯云

目录

Class ReaderBase

__init__

Properties

reader_ref

supports_serialize

Methods

num_records_produced

num_work_units_completed

read

read_up_to

reset

restore_state

serialize_state


Class ReaderBase

Base class for different Reader types, that produce a record every step.

Conceptually, Readers convert string 'work units' into records (key, value pairs). Typically the 'work units' are filenames and the records are extracted from the contents of those files. We want a single record produced per step, but a work unit can correspond to many records.

Therefore we introduce some decoupling using a queue. The queue contains the work units and the Reader dequeues from the queue when it is asked to produce a record (via Read()) but it has finished the last work unit.

Eager Compatibility

Readers are not compatible with eager execution. Instead, please use tf.data to get data into your model.

__init__

View source

代码语言:javascript
复制
__init__(
    reader_ref,
    supports_serialize=False
)

Creates a new ReaderBase.

Args:

  • reader_ref: The operation that implements the reader.
  • supports_serialize: True if the reader implementation can serialize its state.

Raises:

  • RuntimeError: If eager execution is enabled.

Properties

reader_ref

Op that implements the reader.

supports_serialize

Whether the Reader implementation can serialize its state.

Methods

num_records_produced

View source

代码语言:javascript
复制
num_records_produced(name=None)

Returns the number of records this reader has produced.

This is the same as the number of Read executions that have succeeded.

Args:

  • name: A name for the operation (optional).

Returns:

  • An int64 Tensor.

num_work_units_completed

View source

代码语言:javascript
复制
num_work_units_completed(name=None)

Returns the number of work units this reader has finished processing.

Args:

  • name: A name for the operation (optional).

Returns:

  • An int64 Tensor.

read

View source

代码语言:javascript
复制
read(
    queue,
    name=None
)

Returns the next record (key, value) pair produced by a reader.

Will dequeue a work unit from queue if necessary (e.g. when the Reader needs to start reading from a new file since it has finished with the previous file).

Args:

  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • name: A name for the operation (optional).

Returns:

  • A tuple of Tensors (key, value).
  • key: A string scalar Tensor.
  • value: A string scalar Tensor.

read_up_to

View source

代码语言:javascript
复制
read_up_to(
    queue,
    num_records,
    name=None
)

Returns up to num_records (key, value) pairs produced by a reader.

Will dequeue a work unit from queue if necessary (e.g., when the Reader needs to start reading from a new file since it has finished with the previous file). It may return less than num_records even before the last batch.

Args:

  • queue: A Queue or a mutable string Tensor representing a handle to a Queue, with string work items.
  • num_records: Number of records to read.
  • name: A name for the operation (optional).

Returns:

  • A tuple of Tensors (keys, values).
  • keys: A 1-D string Tensor.
  • values: A 1-D string Tensor.

reset

View source

代码语言:javascript
复制
reset(name=None)

Restore a reader to its initial clean state.

Args:

  • name: A name for the operation (optional).

Returns:

  • The created Operation.

restore_state

View source

代码语言:javascript
复制
restore_state(
    state,
    name=None
)

Restore a reader to a previously saved state.

Not all Readers support being restored, so this can produce an Unimplemented error.

Args:

  • state: A string Tensor. Result of a SerializeState of a Reader with matching type.
  • name: A name for the operation (optional).

Returns:

  • The created Operation.

serialize_state

View source

代码语言:javascript
复制
serialize_state(name=None)

Produce a string tensor that encodes the state of a reader.

Not all Readers support being serialized, so this can produce an Unimplemented error.

Args:

  • name: A name for the operation (optional).

Returns:

  • A string Tensor.
本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
原始发表:2022-06-04,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • Class ReaderBase
    • __init__
    • Properties
      • reader_ref
        • supports_serialize
        • Methods
          • num_records_produced
            • num_work_units_completed
              • read
                • read_up_to
                  • reset
                    • restore_state
                      • serialize_state
                      领券
                      问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档