HDFS ls查看文件-按照时间排序

sparkle123

发布于 2021-01-04 10:34:51

3.6K0

发布于 2021-01-04 10:34:51

Hadoop2.8.0以下版本，内置是不支持按照时间等属性排序的。我们可以结合Shell命令来实现按照文件最近修改的时间对 ls 命令输出结果进行排序。比如，按照最近修改的时间降序排序（也就是越早修改的文件显示在最下面）

hdfs dfs -ls   /spark2-history | sort -r -k6,7

为啥是-k6,7,因为第6，7个字段是时间字段。

-rwxrwx---   2 ps_pro               hadoop    1809416 2020-12-27 10:22 /spark2-history/application_1608084068201_400785_1
-rwxrwx---   2 ap_poc               hadoop     107590 2020-12-27 10:22 /spark2-history/application_1608084068201_400797_1
-rwxrwx---   2 fcv                  hadoop     113705 2020-12-27 10:22 /spark2-history/application_1608084068201_400783_1
-rwxrwx---   2 datamart_pro         hadoop     991453 2020-12-27 10:22 /spark2-history/application_1608084068201_400780_2
-rwxrwx---   2 datamart_pro         hadoop     986122 2020-12-27 10:22 /spark2-history/application_1608084068201_400795_1
-rwxrwx---   2 datamart_pro         hadoop      80683 2020-12-27 10:22 /spark2-history/application_1608084068201_400711_2
-rwxrwx---   2 rdvs                 hadoop     163833 2020-12-27 10:21 /spark2-history/application_1608084068201_400779_2
Found 60153 items

效果等同于2.8.0以上的

hdfs dfs -ls -t  /spark2-history

Usage: hadoop fs -ls [-d] [-h] [-R] [-t] [-S] [-r] [-u] <args> Options: -d: Directories are listed as plain files. -h: Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864). -R: Recursively list subdirectories encountered. -t: Sort output by modification time (most recent first). -S: Sort output by file size. -r: Reverse the sort order. -u: Use access time rather than modification time for display and sorting.

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

如有侵权请联系 cloudcommunity@tencent.com 删除

file