文档章节

BeeGFS开发环境搭建3-测试工具

LastRitter
 LastRitter
发布于 06/13 19:17
字数 1200
阅读 36
收藏 0

安装好之后,如果我们想对其元数据服务的性能进行评估和测试,可以安装MDTEST工具,具体的步骤如下:

安装和配置MPICH

MDTEST测试工具依赖MPI协议,所以需要首先安装MPICHMPI协议的一种实现)。

  • 安装和配置(三个节点使用相同操作):
# http://www.mpich.org/

$ yum install -y gcc gcc-c++ gcc-gfortran

$ wget http://www.mpich.org/static/downloads/3.3/mpich-3.3.tar.gz
$ tar xvf mpich-3.3.tar.gz
$ cd mpich-3.3/
$ ./configure --prefix=/usr/local/mpich
$ make -j $(nproc) && make install

$ cp -r examples/ /usr/local/mpich

$ vi ~/.bashrc
PATH=$PATH:/usr/local/mpich/bin
MANPATH=$MANPATH:/usr/local/mpich/man
export PATH MANPATH

$ . ~/.bashrc
  • 测试安装是否正常(三个节点间SSH需要配置免密登录):
$ mpirun -np 10 /usr/local/mpich/examples/cpi
Process 2 of 10 is on sacd01
Process 3 of 10 is on sacd01
Process 8 of 10 is on sacd01
Process 1 of 10 is on sacd01
Process 0 of 10 is on sacd01
Process 5 of 10 is on sacd01
Process 9 of 10 is on sacd01
Process 4 of 10 is on sacd01
Process 7 of 10 is on sacd01
Process 6 of 10 is on sacd01
pi is approximately 3.1415926544231256, Error is 0.0000000008333325
wall clock time = 0.002448

$ vi servers
sacd01:2
sacd02:2
sacd03:2

$ mpirun -np 10 -f servers /usr/local/mpich/examples/cpi
Process 6 of 10 is on sacd01
Process 7 of 10 is on sacd01
Process 0 of 10 is on sacd01
Process 5 of 10 is on sacd03
Process 1 of 10 is on sacd01
Process 2 of 10 is on sacd02
Process 8 of 10 is on sacd02
Process 9 of 10 is on sacd02
Process 3 of 10 is on sacd02
Process 4 of 10 is on sacd03
pi is approximately 3.1415926544231256, Error is 0.0000000008333325
wall clock time = 0.001612

安装和配置MDTEST

  • 安装和配置MDTEST
$ wget https://sourceforge.net/projects/mdtest/files/mdtest%20latest/mdtest-1.9.3/mdtest-1.9.3.tgz

$ mkdir mdtest-1.9.3 && tar xvf mdtest-1.9.3.tgz -C mdtest-1.9.3 && cd mdtest-1.9.3
$ MPI_CC=mpicc make

$ cp mdtest /usr/bin/
  • 测试安装是否正常:
$ cd /mnt/beegfs/
$ mdtest -I 10 -z 5 -b 2 -L
-- started at 05/08/2019 17:17:42 --

mdtest-1.9.3 was launched with 1 total task(s) on 1 node(s)
Command line used: mdtest -I 10 -z 5 -b 2 -L
Path: /mnt
FS: 122.3 GiB   Used FS: 11.0%   Inodes: 61.2 Mi   Used Inodes: 1.8%

1 tasks, 630 files/directories

SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   Directory creation:       1305.970       1305.970       1305.970          0.000
   Directory stat    :       2253.825       2253.825       2253.825          0.000
   Directory removal :       1170.048       1170.048       1170.048          0.000
   File creation     :       1045.978       1045.978       1045.978          0.000
   File stat         :       2025.179       2025.179       2025.179          0.000
   File read         :       1334.224       1334.224       1334.224          0.000
   File removal      :       1623.509       1623.509       1623.509          0.000
   Tree creation     :        429.608        429.608        429.608          0.000
   Tree removal      :        799.761        799.761        799.761          0.000

-- finished at 05/08/2019 17:17:45 --

$ vi servers
sacd01:2
sacd02:2
sacd03:2

$ mpirun -np 10 -f servers mdtest -d /mnt/beegfs -I 10 -z 5 -b 2 -L
-- started at 05/08/2019 17:25:13 --

mdtest-1.9.3 was launched with 10 total task(s) on 2 node(s)
Command line used: mdtest -d /mnt/beegfs -I 10 -z 5 -b 2 -L
Path: /mnt
FS: 122.3 GiB   Used FS: 64.1%   Inodes: 61.2 Mi   Used Inodes: 2.5%

10 tasks, 6300 files/directories

SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   Directory creation:       2778.114       2778.114       2778.114          0.000
   Directory stat    :       4538.071       4538.071       4538.071          0.000
   Directory removal :       3858.621       3858.621       3858.621          0.000
   File creation     :       2146.615       2146.615       2146.615          0.000
   File stat         :       4305.599       4305.599       4305.599          0.000
   File read         :       3106.186       3106.186       3106.186          0.000
   File removal      :       3695.281       3695.281       3695.281          0.000
   Tree creation     :        670.204        670.204        670.204          0.000
   Tree removal      :        748.542        748.542        748.542          0.000

-- finished at 05/08/2019 17:25:27 --

$ mpirun -np 10 -f servers mdtest -d /mnt/beegfs -i 2 -I 100 -b 10 -u -L -F
-- started at 05/08/2019 17:26:11 --

mdtest-1.9.3 was launched with 10 total task(s) on 2 node(s)
Command line used: mdtest -d /mnt/beegfs -i 2 -I 100 -b 10 -u -L -F
Path: /mnt
FS: 122.3 GiB   Used FS: 64.1%   Inodes: 61.2 Mi   Used Inodes: 2.5%

10 tasks, 1000 files

SUMMARY: (of 2 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :       2150.731       1990.272       2070.502         80.229
   File stat         :       4334.598       4025.630       4180.114        154.484
   File read         :       2337.454       2256.293       2296.873         40.581
   File removal      :       4273.461       3998.520       4135.991        137.470
   Tree creation     :        243.232         55.898        149.567         93.666
   Tree removal      :        296.417        265.496        280.957         15.461

-- finished at 05/08/2019 17:26:13 --
  • 查看命令帮助:
$ mdtest --help
Usage: mdtest [-b branching_factor] [-B] [-c] [-C] [-d testdir] [-D] [-e number_of_bytes_to_read]
              [-E] [-f first] [-F] [-h] [-i iterations] [-I items_per_dir] [-l last] [-L]
              [-n number_of_items] [-N stride_length] [-p seconds] [-r]
              [-R[seed]] [-s stride] [-S] [-t] [-T] [-u] [-v]
              [-V verbosity_value] [-w number_of_bytes_to_write] [-y] [-z depth]
	-b: branching factor of hierarchical directory structure
	-B: no barriers between phases
	-c: collective creates: task 0 does all creates
	-C: only create files/dirs
	-d: the directory in which the tests will run
	-D: perform test on directories only (no files)
	-e: bytes to read from each file
	-E: only read files/dir
	-f: first number of tasks on which the test will run
	-F: perform test on files only (no directories)
	-h: prints this help message
	-i: number of iterations the test will run
	-I: number of items per directory in tree
	-l: last number of tasks on which the test will run
	-L: files only at leaf level of tree
	-n: every process will creat/stat/read/remove # directories and files
	-N: stride # between neighbor tasks for file/dir operation (local=0)
	-p: pre-iteration delay (in seconds)
	-r: only remove files or directories left behind by previous runs
	-R: randomly stat files (optional argument for random seed)
	-s: stride between the number of tasks for each test
	-S: shared file access (file only, no directories)
	-t: time unique working directory overhead
	-T: only stat files/dirs
	-u: unique working directory for each task
	-v: verbosity (each instance of option increments by one)
	-V: verbosity value
	-w: bytes to write to each file after it is created
	-y: sync file after writing
	-z: depth of hierarchical directory structure

© 著作权归作者所有

LastRitter
粉丝 42
博文 46
码字总数 210462
作品 0
武汉
高级程序员
私信 提问
BeeGFS开发环境搭建2-安装部署

在上一篇介绍完环境的基本配置后,现在正式开始部署,并解决安装中可能存在的两个问题。 安装软件包 我的测试环境使用, 安装的版本。 使用下载好的包进行安装: 使用安装: 初始化配置 初始...

LastRitter
06/13
0
0
BeeGFS开发环境搭建4-源码编译

如果我们需要动态调试和开发,那么就需要自己手动编译了,而且能够用最简单的方式运行起来,下面就讲解自己探索和分析的过程。 参考文档: 初始化环境 安装依赖的软件包: 获取官方源码 获取...

LastRitter
06/13
0
0
并行文件系统 BeeGFS 现已开源

导读 2月23日 ThinkParQ 宣布完整的 BeeGFS 并行文件系统 的源码现已开源。由于 BeeGFS 是专为要求性能的环境开发的,所以它在开发时十分注重安装的简易性以及高度灵活性,包括融合了在存储服...

linuxprobe
2016/04/18
0
0
BeeGFS开发环境搭建1-环境配置

常用于高性能计算中的分布式文件存储,其对巨量小文件的支持相比于其他大多数文件系统而言,要好太多,这里介绍搭建的详细过程。一共使用台服务器,一个磁盘分成两个区(分别用于元数据和数据...

LastRitter
06/13
0
0
并行文件系统 BeeGFS 现已开源

2月23日 ThinkParQ 宣布完整的 BeeGFS 并行文件系统 的源码现已开源。由于 BeeGFS 是专为要求性能的环境开发的,所以它在开发时十分注重安装的简易性以及高度灵活性,包括融合了在存储服务器...

oschina
2016/04/18
4.9K
12

没有更多内容

加载失败,请刷新页面

加载更多

八、Docker Swarm

Docker Swarm有两件事:一个企业级的Docker主机安全集群,另一个是用于协调微服务应用程序的引擎。 在集群方面,它将一个或多个Docker节点组合在一起,并允许你将他们作为一个集群来管理。开...

倪伟伟
昨天
1
0
Fragment懒加载其实很简单

前言 记得去年面试的时候, 面了一家小公司, 那个面试官问我, fragment的懒加载做过吗?我说没做过(确实没做过).后来面试快结束了, 又问我, 懒加载没做过是吗?后来可想而知也没收到offer, (ಥ_...

天王盖地虎626
昨天
2
0
聊聊dubbo的TimeoutFilter

序 本文主要研究一下dubbo的TimeoutFilter ListenableFilter dubbo-2.7.2/dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/ListenableFilter.java public abstract class Liste......

go4it
昨天
6
0
方法与数组

方法 方法就是完成特定功能的代码块;在很多语言里面都有函数的定义,函数在Java中被称为方法 格式: 修饰符 返回值类型 方法名(参数类型 参数名1,参数类型 参数名2…) throw 异常{ 函数体;...

凹凸凸
昨天
4
0
死磕 java同步系列之StampedLock源码解析

问题 (1)StampedLock是什么? (2)StampedLock具有什么特性? (3)StampedLock是否支持可重入? (4)StampedLock与ReentrantReadWriteLock的对比? 简介 StampedLock是java8中新增的类,...

彤哥读源码
昨天
15
1

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部