分布式文件系统之fastDFS部署

srayban • • Linux干货

fastDFS下载地址:
https://github.com/happyfish100/fastdfs
部署参考文档:
http://joelhy.github.io/2015/01/27/FastDFS-v5-06-deploy/

我们接下来来部署fastDFS

部署结构:

192.168.42.150 node1 [Tracker]
192.168.42.151 node2 [group1 Storage]
192.168.42.152 node3 [group1 Storage]
192.168.42.153 node4 [group2 Storage]
192.168.42.154 node5 [group2 Storage]

FastDFS 5.x 取消了对 libevent 的依赖，添加了对 libfastcommon 的依赖。 本部署说明用到的软件版本

libfastcommon
git clone https://github.com/happyfish100/libfastcommon.git

FastDFS
git clone https://github.com/happyfish100/fastdfs.git

fastdfs-nginx-module
git clone https://github.com/happyfish100/fastdfs-nginx-module.git

准备工作(每个节点)

(1) 安装依赖的软件包

yum install git gcc perl -y

(2) 创建资源目录

mkdir /usr/local/src/FastDFS
cd /usr/local/src/FastDFS
git clone https://github.com/happyfish100/libfastcommon.git
git clone https://github.com/happyfish100/fastdfs.git
git clone https://github.com/happyfish100/fastdfs-nginx-module.git

[root@node5 FastDFS]# ls
fastdfs  fastdfs-nginx-module  libfastcommon

安装FastDFS,具体步骤:

(1)安装libfastcommon

cd libfastcommon && ./make.sh && ./make.sh install

(2)安装FastDFS

cd fastdfs && ./make.sh && ./make.sh install

(3)安装完成后，配置文件在目录/etc/fdfs下：

[root@node1 FastDFS]# ls /etc/fdfs/
client.conf.sample  storage.conf.sample  storage_ids.conf.sample  tracker.conf.sample

(4)所有可执行文件在目录/usr/bin下，以fdfs_开头：

[root@node1 FastDFS]# ls /usr/bin/fdfs_*
/usr/bin/fdfs_appender_test   /usr/bin/fdfs_download_file  /usr/bin/fdfs_test1
/usr/bin/fdfs_appender_test1  /usr/bin/fdfs_file_info      /usr/bin/fdfs_trackerd
/usr/bin/fdfs_append_file     /usr/bin/fdfs_monitor        /usr/bin/fdfs_upload_appender
/usr/bin/fdfs_crc32           /usr/bin/fdfs_storaged       /usr/bin/fdfs_upload_file
/usr/bin/fdfs_delete_file     /usr/bin/fdfs_test

(5)默认配置文件有些是没有复制到/etc/fdfs下的,我们需要把没有复制的也复制下

[root@node1 fastdfs-5.10]# ls conf/
anti-steal.jpg  client.conf  http.conf  mime.types  storage.conf  storage_ids.conf  tracker.conf

cd conf && cp anti-steal.jpg http.conf  mime.types  /etc/fdfs/

(6)将FastDFS,推送到其他节点也装上

scp -rp /usr/local/src/FastDFS root@192.168.42.151:/usr/local/src/
scp -rp /usr/local/src/FastDFS root@192.168.42.152:/usr/local/src/
scp -rp /usr/local/src/FastDFS root@192.168.42.153:/usr/local/src/
scp -rp /usr/local/src/FastDFS root@192.168.42.154:/usr/local/src/

(7)其他节点安装

yum install  gcc perl -y
cd /usr/local/src/FastDFS/libfastcommon && ./make.sh install
cd /usr/local/src/FastDFS/fastdfs && ./make.sh install
cd conf && cp anti-steal.jpg http.conf  mime.types  /etc/fdfs/
echo "

”

(8)检查是否安装成功

ls /usr/bin/fdfs_* && ls /etc/fdfs/

(9)修改配置文件

mkdir -p /fastdfs
cd /etc/fdfs
cp tracker.conf.sample tracker.conf
vim tracker.conf

base_path=/fastdfs
http.server_port=80

(10)设置开机自启动

[root@node1 fdfs]# chkconfig --add fdfs_trackerd
[root@node1 fdfs]# service fdfs_trackerd start
[root@node1 fdfs]# ss -tnl
State      Recv-Q Send-Q                             Local Address:Port                                            Peer Address:Port              
LISTEN     0      128                                            *:22                                                         *:*                  
LISTEN     0      100                                    127.0.0.1:25                                                         *:*                  
LISTEN     0      1024                                           *:22122                                                      *:*                  
LISTEN     0      128                                           :::22                                                        :::*                  
LISTEN     0      100                                          ::1:25                                                        :::*   

vim http.conf
http.anti_steal.token_check_fail=/etc/fdfs/anti-steal.jpg

service fdfs_trackerd restart

配置Storage存储节点(node2,node3,node4,node5)

(1)修改配置文件

mkdir -p /data/fastdfs
cd /etc/fdfs
cp storage.conf.sample  storage.conf
vim storage.conf
base_path=/data/fastdfs
store_path0=/data/fastdfs
tracker_server=192.168.42.150:22122
http.server_port=80

(2)设置开机自启动

[root@node1 fdfs]# chkconfig --add fdfs_storaged
[root@node1 fdfs]# service fdfs_storaged start
#需要等一会儿才能看到
[root@node2 fdfs]# ss -tnl
State      Recv-Q Send-Q    Local Address:Port                   Peer Address:Port              
LISTEN     0      128                   *:22                                *:*                  
LISTEN     0      1024                  *:23000                             *:*                  
LISTEN     0      100           127.0.0.1:25                                *:*                  
LISTEN     0      128                  :::22                               :::*                  
LISTEN     0      100                 ::1:25                               :::*

(3)将配置推送到,node3,node4,node5上

scp  storage.conf root@192.168.42.152:/etc/fdfs
scp  storage.conf root@192.168.42.153:/etc/fdfs
scp  storage.conf root@192.168.42.154:/etc/fdfs

(4)其他节点需要创建目录

mkdir -p /data/fastdfs

(5)node4,node5节点我们规定的是group2,因此我们需要在node4,node5节点更改

vim /etc/fdfs/storage.conf
group_name=group2

(6)在node3,node4,node5上操作

[root@node1 fdfs]# chkconfig --add fdfs_storaged
[root@node1 fdfs]# service fdfs_storaged start

在每个节点上查看保证都能监听23000

[root@node5 conf]# ss -tnl
State      Recv-Q Send-Q                             Local Address:Port                                            Peer Address:Port              
LISTEN     0      128                                            *:22                                                         *:*                  
LISTEN     0      1024                                           *:23000                                                      *:*                  
LISTEN     0      100                                    127.0.0.1:25                                                         *:*                  
LISTEN     0      128                                           :::22                                                        :::*                  
LISTEN     0      100                                          ::1:25

(7)确认storage是否启动成功

cat /data/fastdfs/logs/storaged.log

client客户端配置

(1)在node1上配置

cd /etc/fdfs
mkdir -p /data/fastdfs/client
cp client.conf.sample client.conf

vim client.conf
base_path=/data/fastdfs/client
tracker_server=192.168.42.150:22122

FastDFS测试

(1)上传测试:

[root@node1 tmp]# fdfs_upload_file /etc/fdfs/client.conf /tmp/1.jpg
group1/M00/00/00/wKgql1lDVeeAeW1cAAIRjk6H6X0328.jpg

[root@node1 tmp]# fdfs_upload_file /etc/fdfs/client.conf /tmp/83e.gif
group1/M00/00/00/wKgqmFlDVfqAaUVGAAefBex1G80469.gif

[root@node1 tmp]# fdfs_upload_file /etc/fdfs/client.conf /tmp/zhuoku194.jpg
group1/M00/00/00/wKgql1lDVgmAQPoSAAW2iO4L3ms043.jpg

(2)查看文件信息:

fdfs_file_info /etc/fdfs/client.conf  group1/M00/00/00/wKgql1lDVeeAeW1cAAIRjk6H6X0328.jpg
[root@node1 tmp]# fdfs_file_info /etc/fdfs/client.conf  group1/M00/00/00/wKgql1lDVeeAeW1cAAIRjk6H6X0328.jpg
source storage id: 0
source ip address: 192.168.42.151
file create timestamp: 2017-06-16 11:52:07
file size: 135566
file crc32: 1317529981 (0x4E87E97D)

(3)下载测试:

cd /tmp && mkdir test && cd test
fdfs_download_file /etc/fdfs/client.conf  group1/M00/00/00/wKgql1lDVeeAeW1cAAIRjk6H6X0328.jpg
[root@node1 test]# ls
wKgql1lDVeeAeW1cAAIRjk6H6X0328.jpg

安装并配置 nginx

(1).因为FastDFS 5.0以后,默认不再支持http,因此我们需要在每个节点安装nginx,蛋疼

nginx 的fastdfs模块,我们之前已经下载好了

cd /usr/local/src/FastDFS
wget http://192.168.42.26/nginx-1.11.8.tar.gz
tar xvf nginx-1.11.8.tar.gz
cd nginx-1.11.8
yum groupinstall "Development Tools" "Server Platform Developments" 
yum install pcre-devel openssl-devel  zlib-devel -y
groupadd -r nginx
useradd -r -g nginx nginx

./configure --with-pcre  --user=nginx --group=nginx  --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module  --with-http_realip_module --add-module=../fastdfs-nginx-module/src

make && make install

复制mod_fastdfs.conf到/etc/fdfs/

cd /usr/local/src/FastDFS/fastdfs-nginx-module/src
cp mod_fastdfs.conf /etc/fdfs/
cd /etc/fdfs
cp storage_ids.conf.sample storage_ids.conf

编辑mod_fastdfs.conf

vim mod_fastdfs.conf

tracker_server=192.168.42.150:22122
#这里我们是group1的,node4,node5上需要改成group2
group_name=group1
store_path0=/data/fastdfs
base_path=/data/fastdfs/logs/
tracker_server=192.168.42.150:22122

创建软链接

cd /data/fastdfs/data
ln -s ../data/  M00

配置 nginx.conf,添加一个location

user  nginx;
worker_processes  auto;

location /M00 {
    root /data/fastdfs/data;
    ngx_fastdfs_module;
}

启动nginx

/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf

(2)node3,node4,node5上同样安装nginx

推送nginx-1.11.8
scp -rp nginx-1.11.8 root@192.168.42.152:/usr/local/src/FastDFS/
scp -rp nginx-1.11.8 root@192.168.42.153:/usr/local/src/FastDFS/
scp -rp nginx-1.11.8 root@192.168.42.154:/usr/local/src/FastDFS/

推送 mod_fastdfs.conf
cd /etc/fdfs
scp mod_fastdfs.conf root@192.168.42.152:/etc/fdfs/
scp mod_fastdfs.conf root@192.168.42.153:/etc/fdfs/
scp mod_fastdfs.conf root@192.168.42.154:/etc/fdfs/

推送nginx.conf
scp -rp /usr/local/nginx/conf/nginx.conf root@192.168.42.152:/usr/local/nginx/conf/
scp -rp /usr/local/nginx/conf/nginx.conf root@192.168.42.153:/usr/local/nginx/conf/
scp -rp /usr/local/nginx/conf/nginx.conf root@192.168.42.154:/usr/local/nginx/conf/

node4,node5,改下mod_fastdfs.conf
vim /etc/fdfs/mod_fastdfs.conf
group_name=group2

三个节点操作

cd /usr/local/src/FastDFS/nginx-1.11.8/ && make install

三个节点创建软链接

cd /data/fastdfs/data
ln -s ../data/  M00

启动nginx

/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf

在浏览器测试(之前上传的图片在group1上)

http://192.168.42.152/M00/00/00/wKgql1lDjSGAej-6AAW2iO4L3ms072.jpg
http://192.168.42.153/
http://192.168.42.154/

(3)我们用tracker节点来做反代(node1),因此也要安装nginx,因为只是用反代的,不用安装fastdfs-nginx-module

cd /usr/local/src/FastDFS
wget http://192.168.42.26/nginx-1.11.8.tar.gz
tar xvf nginx-1.11.8.tar.gz
cd nginx-1.11.8
yum install pcre-devel openssl-devel  zlib-devel -y
groupadd -r nginx
useradd -r -g nginx nginx

./configure --with-pcre  --user=nginx --group=nginx  --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module  --with-http_realip_module 

make && make install

编辑nginx.conf

user  nginx;
worker_processes  auto;


在 http {} 中添加：
upstream fdfs1 {
    server   192.168.42.151:80;
    server   192.168.42.152:80;
    server   192.168.42.153:80;
    server   192.168.42.154:80;
}

在 server{} 中添加：
location /M00 {
    proxy_pass http://fdfs1;
}

FastDFS客户端

PHP扩展安装 client主机上安装的是php v5.4.33

cd ~/fdfs/FastDFS/php_client
/usr/local/php-5.4.33/bin/phpize
./configure  --with-php-config=/usr/local/php-5.4.33/bin/php-config
make && make install
mkdir /usr/local/php-5.4.33/etc/conf.d
cp fastdfs_client.ini /usr/local/php-5.4.33/etc/conf.d
PHP扩展使用
<?php
// fastdfs.php
class Fdfs {
    private $fdfs, $tracker, $storage;
    public function __construct() {
        $this->fdfs = new FastDFS();
        // get a connected tracker server
        $this->tracker = $this->fdfs->tracker_get_connection();
        if (!$this->tracker) {
            throw new Exception('cannot connect to tracker server:[' .  
                $this->fdfs->get_last_error_no() . '] ' . 
                $this->fdfs->get_last_error_info());
        }
        // get the storage server info and connect to it
        $this->storage = $this->fdfs->tracker_query_storage_store();
        $this->server = $this->fdfs->connect_server(
            $this->storage['ip_addr'], $this->storage['port']);
        if ($this->server === false) {
            throw new Exception('cannot connect to storage server' . 
                $this->storage['ip_addr'] . ':' . 
                $this->storage['port'] .  ' :[' .  
                $this->fdfs->get_last);
        }
        $this->storage['sock'] = $this->server['sock'];
    }

    public function upload($localfile, $ext_name) {
        //$info = $this->fdfs->storage_upload_by_filename($localfile);
        $info = $this->fdfs->storage_upload_by_filename($localfile, $ext_name, 
            array(), null, $this->tracker, $this->storage);
        if (is_array($info)) {
            $group_name = $info['group_name'];
            $remote_filename = $info['filename'];
            $source_info = $this->fdfs->get_file_info($group_name, 
                $remote_filename);
            $source_ip = $source_info['source_ip_addr'];
            $file_size = $source_info['file_size'];
            return compact('group_name', 'remote_filename', 
                'source_ip', 'file_size');
        }
        return false;
    }
    public function download_to_buff($group_name, $remote_filename) {
        $content = $this->fdfs->storage_download_file_to_buff(
            $group_name, $remote_filename);
        return $content;
    }
    public function download_to_file($group_name, 
            $remote_filename, $dst_localfile) {
        return $this->fdfs->storage_download_file_to_file($group_name, 
            $remote_filename, $dst_localfile);
    }
    public function delete($group_name, $remote_filename) {
        return $this->fdfs->storage_delete_file($group_name, $remote_filename);
    }
    public function exists($group_name, $remote_filename) {
        return $this->fdfs->storage_file_exist($group_name, $remote_filename);
    }
    public function get_file_info($group_name, $remote_filename) {
        return $this->fdfs->get_file_info($group_name, $remote_filename);
    }
}

PHP客户端上传图片测试

# cat test.php
<?php
require_once('fastdfs.php');
$fdfs = new FDFS();
$localfile = './test.xml';
$fileinfo = $fdfs->upload($localfile);
if ($fileinfo) {
  // update file info in the database etc
}
var_dump($fileinfo);
# php test.php 
array(4) {
  ["group_name"]=>
  string(6) "group1"
  ["remote_filename"]=>
  string(44) "M00/00/00/wKhHf1S-qbuAISbeAAAE2uRlJkA789.xml"
  ["source_ip"]=>
  string(14) "192.168.71.127"
  ["file_size"]=>
  int(1242)
}

原创文章，作者：srayban，如若转载，请注明出处：http://www.178linux.com/78105

赞 (0)
sraybansrayban
0
上一篇 2017-06-16 19:21
下一篇 2017-06-16 20:40

相关推荐

  • lamp+nfs搭建wordpress

    一、前言 lamp是大多上公司常用的架构，本文将针对分离式的lamp+nfs来搭建一个简单的wordpress网站。 二、架构图 三、基本实现过程 3.1：dns搭建 由于这这是一个实验，故使用yum搭建dns服务器 yum install -y bind 配置如下 dns主配置文件 dns辅助配置文件 zone文件 对所有服务…

    Linux干货 2015-10-16

  • 11. vim备查の小手札

    三种模式 命令模式（默认）      command mode：移动光标，剪切/粘贴文本 插入（编辑）模式         edit mode：修改文本 扩展命令模式 extend…

    Linux干货 2016-08-18

  • shell脚本的各种循环

    For循环 For循环格式      For   变量名  in    列表    ；   do         &nb…

    Linux干货 2016-08-21
  • CentOS系统启动 Linux干货

    CentOS系统启动

    Linux组成 Linux: kernel+rootfs kernel:  进程管理、内存管理、网络管理、驱动程序、文件系统、安全功能 rootfs: 程序和glibc 库：函数集合, function,  调用接口（头文件负责描述） 过程调用：procedure ，无返回值 函数调用：function 程序：二进制执行文件 内核设计流派…

    2017-05-15

  • linux文件查找和压缩

    locate          非实查找，查找是根据全系统文件数据库进行的，                  #updatedb, 手动生成数据库，  locate查找速度快 find : &nbs…

    Linux干货 2017-03-04

  • 网络管理3

    一、知识整理 1、跨网络通信：路由 路由的分类：主机路由；网络路由；默认路由。 将主机接入网络，需要配置网络相关设置。一般包括如下内容：主机名；IP/MASK；路由：默认网关；DNS服务器：主、次、第三DNS服务器。 2、网络配置文件：IP、MASK、GW、DNS相关配置文件 /etc/sysconfig/network-scripts/ifcfg-IFAC…

    Linux干货 2016-09-13
电话咨询
在线咨询