建站
咨詢
售后
建站咨詢
新聞中心
FastDFS是一個(gè)開(kāi)源的輕量級(jí)分布式文件系統(tǒng)，由跟蹤服務(wù)器（tracker server）、存儲(chǔ)服務(wù)器（storage server）和客戶端（client）三個(gè)部分組成，主要解決了海量數(shù)據(jù)存儲(chǔ)問(wèn)題，特別適合以中小文件（建議范圍：4KB < file_size <500MB）為載體的在線服務(wù)。
成都一家集口碑和實(shí)力的網(wǎng)站建設(shè)服務(wù)商，擁有專業(yè)的企業(yè)建站團(tuán)隊(duì)和靠譜的建站技術(shù)，十多年企業(yè)及個(gè)人網(wǎng)站建設(shè)經(jīng)驗(yàn) ,為成都上1000+客戶提供網(wǎng)頁(yè)設(shè)計(jì)制作,網(wǎng)站開(kāi)發(fā),企業(yè)網(wǎng)站制作建設(shè)等服務(wù),包括成都營(yíng)銷型網(wǎng)站建設(shè)，品牌網(wǎng)站建設(shè)，同時(shí)也為不同行業(yè)的客戶提供網(wǎng)站設(shè)計(jì)、成都網(wǎng)站建設(shè)的服務(wù),包括成都電商型網(wǎng)站制作建設(shè),裝修行業(yè)網(wǎng)站制作建設(shè)，傳統(tǒng)機(jī)械行業(yè)網(wǎng)站建設(shè),傳統(tǒng)農(nóng)業(yè)行業(yè)網(wǎng)站制作建設(shè)。在成都做網(wǎng)站,選網(wǎng)站制作建設(shè)服務(wù)商就選成都創(chuàng)新互聯(lián)。
（1）每次上傳文件后都會(huì)返回一個(gè)地址，用戶需要自己保存此地址。
（2）為了支持大容量，存儲(chǔ)節(jié)點(diǎn)（服務(wù)器）采用了分卷（或分組）的組織方式。存儲(chǔ)系統(tǒng)由一個(gè)或多個(gè)卷組成，卷與卷之間的文件是相互獨(dú)立的，所有卷的文件容量累加就是整個(gè)存儲(chǔ)系統(tǒng)中的文件容量。一個(gè)卷可以由一臺(tái)或多臺(tái)存儲(chǔ)服務(wù)器組成，一個(gè)卷下的存儲(chǔ)服務(wù)器中的文件都是相同的，卷中的多臺(tái)存儲(chǔ)服務(wù)器起到了冗余備份和負(fù)載均衡的作用。
配置環(huán)境：
CentOS 7.5 x 64
tracker server: 192.168.5.71（tracker暫時(shí)一臺(tái)，后續(xù)再添加一臺(tái)做nginx+keepalived高可用）
storage server: 192.168.5.72 192.168.5.73（group1）
storage server: 192.168.5.74 192.168.5.75（group2）
#hosts文件配置
cat >> /etc/hosts << EOF
192.168.5.71 storage71.blufly.com storage71
192.168.5.72 storage72.blufly.com storage72
192.168.5.73 storage73.blufly.com storage73
192.168.5.74 storage74.blufly.com storage74
192.168.5.75 storage75.blufly.com storage75
EOF###---------------------- 安裝配置tracker server ---------------------------###
yum -y install gcc-c++ libevent#先安裝libfastcommon
cd /opt
git clone https://github.com/happyfish100/libfastcommon.git
cd libfastcommon/
./make.sh
./make.sh install
#安裝fastdfs:
cd /opt
git clone https://github.com/happyfish100/fastdfs.git
cd fastdfs
./make.sh
./make.sh install#配置文件解釋：
tracker.conf //負(fù)責(zé)均衡調(diào)度服務(wù)器配置文件
client.conf //客戶端上傳配置文件
http.conf //http服務(wù)器配置文件
storage.conf//文件存儲(chǔ)服務(wù)器配置文件
mime.types //文件類型配置文件#修改tracker server的配置文件
cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf
vi /etc/fdfs/tracker.conf
bind_addr= 改為 bind_addr=192.168.25.204
base_path=/home/yuqing/fastdfs 改為 base_path=/data/fastdfs
http.server_port=8080 改為 http.server_port=80#啟動(dòng)tracker server
mkdir -p /data/fastdfs
/etc/init.d/fdfs_trackerd start
[root@storage71 fastdfs]# netstat -tnlp|grep fdfs
tcp 0 0 192.168.5.71:22122 0.0.0.0:* LISTEN 11631/fdfs_trackerd#開(kāi)機(jī)啟動(dòng)
/sbin/chkconfig --add fdfs_trackerd
/sbin/chkconfig --level 2345 fdfs_trackerd on###--------------------------- 安裝配置storage server ----------------------###
yum -y install gcc-c++ libevent#先安裝libfastcommon
cd /opt
git clone https://github.com/happyfish100/libfastcommon.git
cd libfastcommon/
./make.sh
./make.sh install
#安裝fastdfs:
cd /opt
git clone https://github.com/happyfish100/fastdfs.git
cd fastdfs
./make.sh
./make.sh install#配置storage server
cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf
#修改storage server的配置文件：
vm /etc/fdfs/storage.conf
bind_addr= 改為 bind_addr=192.168.5.72
#base_path需要和tracker部分的base_path保持一致，如果有修改過(guò)tracker的
base_path=/home/yuqing/fastdfs 改為 base_path=/data/fastdfs
#修改storage的資源存放路徑
store_path0=/home/yuqing/fastdfs 改為 store_path0=/data/fastdfs
#如果有多個(gè)掛載磁盤(pán)則定義多個(gè)store_path，如下
#store_path1=......
#store_path2=......
修改storage的對(duì)應(yīng)的tracker_server的ip地址和端口
tracker_server=192.168.209.121:22122 改為 tracker_server=192.168.5.71:22122
#如果有多個(gè)則配置多個(gè)tracker_server
#tracker_server=......
#tracker_server=......
http.server_port=8888 改為 http.server_port=80#創(chuàng)建數(shù)據(jù)目錄，并啟動(dòng)storage server
mkdir -p /data/fastdfs
/etc/init.d/fdfs_storaged start
[root@storage72 fastdfs]# netstat -tnlp|grep fdfs
tcp 0 0 192.168.5.72:23000 0.0.0.0:* LISTEN 11624/fdfs_storaged#開(kāi)機(jī)啟動(dòng)
/sbin/chkconfig --add fdfs_storaged
/sbin/chkconfig --level 2345 fdfs_storaged on#查看tracker和storage的鏈接情況
[root@storage71 ~]# netstat -tnlpa|grep 22122
tcp 0 0 192.168.5.71:22122 0.0.0.0:* LISTEN 11631/fdfs_trackerd
tcp 0 0 192.168.5.71:22122 192.168.5.72:63755 ESTABLISHED 11631/fdfs_trackerd
tcp 0 0 192.168.5.71:22122 192.168.5.73:47697 ESTABLISHED 11631/fdfs_trackerd#在tracker server上配置客戶端上傳文件
[root@storage71 fastdfs]# cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
vim /etc/fdfs/client.conf
#base_path需要和tracker部分的base_path保持一致
base_path=/home/yuqing/fastdfs
改為 base_path=/data/fastdfs
tracker_server=192.168.0.197:22122
改為
tracker_server=192.168.5.71:22122#上傳個(gè)文件，如果上傳成功，會(huì)返回一個(gè)url，我們記住這個(gè)url，后面安裝了nginx就可以通URL來(lái)訪問(wèn)上傳的這個(gè)文件
[root@storage71 fastdfs]# /usr/bin/fdfs_test /etc/fdfs/client.conf upload /root/test.jpg
group_name=group1, ip_addr=192.168.5.72, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/wKgFSFvAem6AfpJXAAA2FXv_nRs556.jpg
source ip address: 192.168.5.72
file timestamp=2018-10-12 18:41:50
file size=13845
file crc32=2080349467
example file url: http://192.168.5.72/group1/M00/00/00/wKgFSFvAem6AfpJXAAA2FXv_nRs556.jpg
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/wKgFSFvAem6AfpJXAAA2FXv_nRs556_big.jpg
source ip address: 192.168.5.72
file timestamp=2018-10-12 18:41:51
file size=13845
file crc32=2080349467
example file url: http://192.168.5.72/group1/M00/00/00/wKgFSFvAem6AfpJXAAA2FXv_nRs556_big.jpg
通過(guò)客戶端連接并上傳文件：
#上傳"/etc/passwd"文件到fastdfs服務(wù)器:
# fdfs_upload_file /etc/fdfs/client.conf /etc/passwd
下載剛才上傳的文件：
# fdfs_download_file /etc/fdfs/client.conf group1/M00/00/00/wKgFSFvBpdSAB33ZAAI8IU3BQ48068.jpg
查看文件屬性：
# fdfs_file_info /etc/fdfs/client.conf group1/M00/00/00/wKgFSFvBpdSAB33ZAAI8IU3BQ48068.jpg
刪除上傳的文件：
# fdfs_delete_file /etc/fdfs/client.conf group1/M00/00/00/wKgFSFvBpdSAB33ZAAI8IU3BQ48068.jpg
# fdfs_delete_file /etc/fdfs/client.conf group1/M00/00/00/wKgFSFvBtTuAGz2VAAAJHTkxfCU312.txt
監(jiān)控fastdfs的狀態(tài):
# fdfs_monitor /etc/fdfs/client.conf
###------------------- 在storage server上安裝nginx -------------------------###
#安裝nginx關(guān)于fastdfs集合的擴(kuò)展模塊，注意這個(gè)模塊nginx默認(rèn)的編譯模塊中沒(méi)有，需手動(dòng)下載安裝
cd /opt
wget http://nginx.org/download/nginx-1.15.5.tar.gz
#jemalloc 優(yōu)化nginx，內(nèi)存管理
wget https://github.com/jemalloc/jemalloc/releases/download/5.1.0/jemalloc-5.1.0.tar.bz2
wget https://ftp.pcre.org/pub/pcre/pcre-8.42.tar.gz
#nginx緩存清理模塊
wget http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz
git clone https://github.com/happyfish100/fastdfs-nginx-module.gityum -y install openssl openssl-devel zlib zlib-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel ncurses ncurses-devel curl curl-develtar -jxvf jemalloc-5.1.0.tar.bz2
cd jemalloc-5.1.0
./configure --prefix=/usr/local/jemalloc --libdir=/usr/local/lib
make;make install
echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf
ldconfig
cd ../
/usr/sbin/groupadd www
/usr/sbin/useradd -g www www -s /sbin/nologin
tar -zxvf pcre-8.42.tar.gz
tar -zxvf ngx_cache_purge-2.3.tar.gz
tar -zxvf nginx-1.15.5.tar.gz
cd nginx-1.15.5
./configure --prefix=/usr/local/nginx \
--with-pcre=/opt/pcre-8.42 \
--user=www \
--group=www \
--with-http_stub_status_module \
--with-http_ssl_module \
--with-http_flv_module \
--with-http_gzip_static_module \
--with-ld-opt="-ljemalloc" \
--with-http_realip_module \
--add-module=/opt/fastdfs-nginx-module/src \
--add-module=/opt/ngx_cache_purge-2.3
make;make install#修改mod_fastdfs配置文件
cp /opt/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/
cd /opt/fastdfs/conf/ #進(jìn)入fastdfs安裝包文件夾
cp anti-steal.jpg http.conf mime.types /etc/fdfs/
vi /etc/fdfs/mod_fastdfs.conf
base_path=/tmp 改為 base_path=/data/fastdfs
tracker_server=tracker:22122 改為 tracker_server=192.168.5.71:22122
store_path0=/home/yuqing/fastdfs 改為 store_path0=/data/fastdfs
url_have_group_name = false 改為 true#配置nginx
server {
listen 80;
server_name 192.168.5.72;
location /group1/M00 {
ngx_fastdfs_module;
}
}#添加nginx啟動(dòng)腳本
cat >> /etc/init.d/nginx <&2
exit 3
;;
esac
exit 0
EOF #將nginx添加到啟動(dòng)服務(wù)中
chmod 700 /etc/init.d/nginx
/sbin/chkconfig --add nginx
/sbin/chkconfig --level 2345 nginx on#啟動(dòng)nginx
[root@storage72 opt]# /etc/init.d/nginx start
Starting nginx daemon: nginxngx_http_fastdfs_set pid=21702#查看jemalloc內(nèi)存管理
lsof -n | grep jemalloc###------------------ tracker server端做nginx反向代理 ----------------------###
#nginx.conf配置
upstream fdfs_group1 {
server 192.168.5.72 weight=1 max_fails=2 fail_timeout=30s;
server 192.168.5.73 weight=1 max_fails=2 fail_timeout=30s;
}
server {
listen 80;
server_name 192.168.5.71;
location /group1/M00 {
proxy_pass http://fdfs_group1;
}
}###---------------------------- PHP客戶端配置 ------------------------------###
#首先客戶端要安裝LNMP環(huán)境或者LANMP環(huán)境
#因?yàn)閜hp的客戶端安裝也會(huì)依賴fastdfs本身的一些庫(kù)文件，所以要安裝fastdfs
cd /opt/fastdfs/php_client
/usr/local/php/bin/phpize
./configure --with-php-config=/usr/local/php/bin/php-config
make;make installcat fastdfs_client.ini >> /usr/local/php/etc/php.ini#配置fastDFS的client.conf
mkdir -p /data/fastdfs
cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
vi /etc/fdfs/client.conf
base_path=/data/fastfds
tracker_server=192.168.5.71:22122
http.tracker_server_port=80#重新加載php
service php-fpm reload#驗(yàn)證模塊是否被正常加載
/usr/local/php/bin/php -m | grep fastdfs_client#通過(guò)http上傳測(cè)試
cat test.php
cat upload.php
getPostFilename());
$ret['file'] = $file;
$ret['fileId'] = uploadToFastdfs($curlFile, $fileSuffix);
return $ret;
}/*}}}*/
//獲取后綴
function getSuffix($fileName)
{/*{{{*/
preg_match('/\.(\w+)?$/', $fileName, $matchs);
return isset($matchs[1])?$matchs[1]:'';
}/*}}}*/
//上傳文件到fastdfs
function uploadToFastdfs(CurlFile $file, $fileSuffix)
{/*{{{*/
$fdfs = new FastDFS();
$tracker = $fdfs->tracker_get_connection();
$fileId = $fdfs->storage_upload_by_filebuff1(file_get_contents($file->getFilename()), $fileSuffix);
$fdfs->tracker_close_all_connections();
return $fileId;
}/*}}}*/
function start()
{
$ret = uploadAttach();
print_r($ret);
}
start();
?>#通過(guò)php腳本上傳測(cè)試
echo "php client is here." > /opt/upload.txtcat test.php
#執(zhí)行php腳本
/usr/local/php/bin/php test.php###------------------------- storage server宕機(jī)測(cè)試 ------------------------###
#先關(guān)閉storage73
#通過(guò)tracker可以看到storage73已經(jīng)不在線了
#然后通過(guò)客戶端上傳一張圖片
#上傳的圖片可以正常訪問(wèn)
#重啟storage73看上傳的圖片有沒(méi)有同步過(guò)來(lái)
###--------------------------- 橫向擴(kuò)容，增加group2 ------------------------###
#添加一組storage server: 192.168.5.74 192.168.5.75（group2）
#分別安裝fastdfs、nginx、fastdfs-nginx-module
#修改group2成員的storage.conf（192.168.5.74 192.168.5.75）
cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf#修改storage server的配置文件：
vi /etc/fdfs/storage.conf
group_name=group2
bind_addr=192.168.5.74
base_path=/data/fastdfs
store_path0=/data/fastdfs
tracker_server=192.168.5.71:22122
http.server_port=80
#創(chuàng)建數(shù)據(jù)目錄，并啟動(dòng)storage server
mkdir -p /data/fastdfs
/etc/init.d/fdfs_storaged start#開(kāi)機(jī)啟動(dòng)
/sbin/chkconfig --add fdfs_storaged
/sbin/chkconfig --level 2345 fdfs_storaged on#修改所有storage server的mod_fastdfs.conf配置文件
vi /etc/fdfs/mod_fastdfs.conf
base_path=/data/fastdfs
tracker_server=192.168.5.71:22122
store_path0=/data/fastdfs
url_have_group_name = true
group_name=group1/group2 # 所有組名
group_count = 2 # 組的總數(shù)
# 把所有組和目錄都添加上
[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/data/fastdfs
[group2]
group_name=group2
storage_server_port=23000
store_path_count=1
store_path0=/data/fastdfs
#在所有storage機(jī)器上，創(chuàng)建所有組的store_path目錄
mkdir -p /data/fastdfs#修改所有storage server的nginx配置文件
server {
listen 80;
server_name localhost;
location ~/group([0-9])/M00/{
ngx_fastdfs_module;
}
}#重啟storage server的nginx
/etc/init.d/nginx restart#修改tracker server的反向代理
user www www;
worker_processes 8;
error_log /usr/local/nginx/logs/nginx_error.log crit;
pid /usr/local/nginx/logs/nginx.pid;
worker_rlimit_nofile 65535;
#工作模式及連接數(shù)上限
events {
worker_connections 65535;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile off;
tcp_nopush on;
keepalive_timeout 300;
#nginx跟后端服務(wù)器連接超時(shí)時(shí)間(代理連接超時(shí))
proxy_connect_timeout 300s;
#連接成功后，后端服務(wù)器響應(yīng)時(shí)間(代理接收超時(shí))
proxy_read_timeout 300s;
proxy_send_timeout 300s;
#設(shè)置代理服務(wù)器（nginx）保存用戶頭信息的緩沖區(qū)大小
proxy_buffer_size 64k;
#proxy_buffers緩沖區(qū)，網(wǎng)頁(yè)平均在32k以下的話，這樣設(shè)置
proxy_buffers 4 32k;
#高負(fù)荷下緩沖大?。╬roxy_buffers*2）
proxy_busy_buffers_size 64k;
#設(shè)定緩存文件夾大小，大于這個(gè)值，將從upstream服務(wù)器傳遞請(qǐng)求，而不緩沖到磁盤(pán)
proxy_temp_file_write_size 64k;
#不允許代理端主動(dòng)關(guān)閉連接
proxy_ignore_client_abort on;
proxy_cache_path /data/cache1 levels=1:2 keys_zone=fastdfs_cache:200m inactive=1d max_size=20g;
fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
fastcgi_buffer_size 128k;
fastcgi_buffers 4 128k;
fastcgi_busy_buffers_size 128k;
fastcgi_temp_file_write_size 128k;
gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_types text/plain application/x-Javascript text/css application/xml;
gzip_vary on;
#nginx cache命中率統(tǒng)計(jì)日志
log_format cachelog '$remote_addr - $remote_user [$time_local] "$request" '
'$status "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'"$upstream_cache_status" $body_bytes_sent';
#設(shè)置 group1 的服務(wù)器
upstream fdfs_group1 {
server 192.168.5.72 weight=1 max_fails=2 fail_timeout=30s;
server 192.168.5.73 weight=1 max_fails=2 fail_timeout=30s;
}
#設(shè)置 group2 的服務(wù)器
upstream fdfs_group2 {
server 192.168.5.74 weight=1 max_fails=2 fail_timeout=30s;
server 192.168.5.75 weight=1 max_fails=2 fail_timeout=30s;
}
server {
listen 80;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
#設(shè)置 group 的負(fù)載均衡參數(shù)
location /group1/M00 {
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_cache fastdfs_cache;
proxy_cache_valid 200 304 12h;
proxy_cache_key $uri$is_args$args;
proxy_pass http://fdfs_group1;
expires 30d;
}
location /group2/M00 {
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_cache fastdfs_cache;
proxy_cache_valid 200 304 12h;
proxy_cache_key $uri$is_args$args;
proxy_pass http://fdfs_group2;
expires 30d;
}
#設(shè)置清除緩存的訪問(wèn)權(quán)限
location ~/purge(/.*) {
allow 127.0.0.1;
allow 192.168.5.0/24;
deny all;
proxy_cache_purge http-cache $1$is_args$args;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}#fastdfs狀態(tài)監(jiān)控
[root@storage71 /]# fdfs_monitor /etc/fdfs/client.conf
server_count=1, server_index=0
tracker server is 192.168.5.71:22122
group count: 2
Group 1:
group name = group1
Storage 1:
id = 192.168.5.72
ip_addr = 192.168.5.72 (storage72.blufly.com) ACTIV
Storage 2:
id = 192.168.5.73
ip_addr = 192.168.5.73 (storage73.blufly.com) WAIT_SYNC
Group 2:
group name = group2
Storage 1:
id = 192.168.5.74
ip_addr = 192.168.5.74 (storage74.blufly.com) ACTIVE
Storage 2:
id = 192.168.5.75
ip_addr = 192.168.5.75 (storage75.blufly.com) WAIT_SYNC#上傳圖片測(cè)試
#fastdfs的storage server的狀態(tài)查詢
#FDFS_STORAGE_STATUS：INIT:初始化，尚未得到同步已有數(shù)據(jù)的源服務(wù)器
#FDFS_STORAGE_STATUS：WAIT_SYNC:等待同步，已得到同步已有數(shù)據(jù)的源服務(wù)器
#FDFS_STORAGE_STATUS：SYNCING:同步中
#FDFS_STORAGE_STATUS：DELETED:已刪除，該服務(wù)器從本組中摘除
#FDFS_STORAGE_STATUS：OFFLINE:離線
#FDFS_STORAGE_STATUS：ONLINE:在線，尚不能提供服務(wù)
#FDFS_STORAGE_STATUS：ACTIVE:在線，可以提供服務(wù)標(biāo)題名稱：CentOS7.5下搭建高可用的FastDFS分布式文件系統(tǒng)
網(wǎng)頁(yè)網(wǎng)址：http://m.5511xx.com/article/djcisdo.html
