XIV（3）--Read/Write O

XIV系列：

《XIV（1）—Hardware Overview》

《XIV （2）--Logical system concepts》

  之前的文章曾经说过HOST发过来的data会在XIV上存2份，即Primary Copy和Secondary Copy。只有当这2份都同时存在时，系统才是Full Redundancy状态。那主机在XIV上读写数据分别是怎么进行的呢？请看本篇：

-Each write is written to the cache of two data modules// 每个写操作是先写到2个Data Module中的Cache中的

-Host is acknowledged as soon as two cache copies are available //只有当两份Cache都写完时才会发送一个Acknowledge给Host

-De-staging to the disk drives takes place: //至于什么时候将cache中的数据Flush到Disk上是各个Module独立进行的

–In the background

–Independently on each module

  Write Operation Overview

p_w_picpath

1.Host sends write to interface

2.Interface sends write to primary data module

3.Primary data module sends write to secondary data module

4.Host is acknowledged only after write is completed on both modules

上图只是讲述了Host写数据操作的大致步骤，涉及到XIV内部具体是怎么进行的呢？

Write Operations

1, Host sends a write request to one of the i_nodes

2, i_node consults with the Slice Table, determines primary node ID and disk #

3, i_node forwards request to relevant module’s primary cache node

4, Primary cache node consults with the Slice Table, forwards request to the secondary cache node

5, Both cache nodes consult with their local Partition Table to determine physical location on disks

6, Both cache nodes save the written buffer in their memory cache

7, Secondary cache node send an ack to the primary cache, which then acks the i_node, which then acks the host

这里有两个Table，Slice Table和Local Partition Table。一个负责整套XIV的元数据，一个是负责盘上面的。可以看出所有的涉及到写到哪个Node的具体哪块Disk时，是要查询Slice Table。而最后写到Disk上哪块Block上时，是查询Local Partition Table的。

Slice table

--It’s an index stored all the slices info for the whole system

--It’s existed in every module’s memory

--i_node and cte can query it and know the slices (primary and secondary slices) are stored in which module and which disk

Partition table

--Each cache node holds a Partition Table that keeps one entry for each physical partition that exists on the module

--It keeps translation maps between a (vol ID, logical partition #) pairs to (disk ID,  physical partition #) pairs

看完写操作，再来看读操作。

Read Operations

1, Host sends a read request to one of the i_nodes

2, i_node consults with the Slice Table, determines primary node ID and disk #

–A read request will always be directed to the primary copy of the data

3, i_node forwards request to relevant module’s cache node

4, Cache node consults with its local Partition Table, determines physical location on disk

5, Cache node reads the data from the memory cache, if there, or from the disk

6, Cache node sends data to i_node, which gives it to the host

同样地，读操作也涉及到Slice Table和Local Partition Table。

看到这里，我感觉和我之前接触到的分布式文件系统（Distributed File system）非常类似，例如MooseFS，Google的GFS，Hadoop File System等等。有机会在深入研究之后对比下两者实现方式的异同点。