Bug 8 - TestDFSIO 测试 (edit)
Status: CONFIRMED (edit)
Alias: None (edit)
(show other bugs)
1.0
:
()
Depends on: ()
Blocks: ()
 
Reported: 2023-06-13 02:45 UTC by
Modified: 2023-06-13 02:45 UTC (History)

0 users (edit)
(never email me about this bug)
(add)

Current Est.: %Complete: Gain:
0.0 0.0 + 0 0.0
Summarize time (including time for bugs blocking this bug)

Attachments

:

Status:
of
[tag] [reply] [−] Description 2023-06-13 02:45:39 UTC
一、TestDFSIO 测试

	TestDFSIO的用法如下:
	Usage: TestDFSIO [genericOptions] -read [-random | -backward | -skip [-skipSize Size]] | -write | -append | -clean [-compression codecClassName] [-nrFiles N] [-size Size[B|KB|MB|GB|TB]] [-resFile resultFileName] [-bufferSize Bytes] [-rootDir]


	注意:在执行read ,先要执行write创建所需要的数据;每个一个case执行完,都用clean清空测试环境。 

	1.清除命令
	  hadoop jar /usr/hadoop-parafs/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.3-tests.jar TestDFSIO -clean



	2.TestDFSIO write
	测试hadoop写的速度。

		 1> 向HDFS文件系统中写入数据,10个文件,每个文件10MB,文件存放到/benchmarks/TestDFSIO/io_data下面。
			 hadoop jar /usr/hadoop-parafs/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.3-tests.jar TestDFSIO -write -nrFiles 10 -size 2GB
		 
		 2> 查看写入的结果:
			cat TestDFSIO_results.log



	3.TestDFSIO read
	测试hadoop读文件的速度

		 1> 从HDFS文件系统中读入10个文件,每个文件大小为10MB
			 hadoop jar /usr/hadoop-parafs/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.3-tests.jar TestDFSIO -read -nrFiles 10 -size 2GB

		 2> 查看写入的结果:
			cat TestDFSIO_results.log