forked from jackdoe/sfs
-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
146 lines (125 loc) · 4.62 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
*WARNING: experimental stage, work in progress, DOES NOT USE SSL, DO NOT USE IN PRODUCTION*
very basic mysql backed fuse file system.
0: abstract
fuse based, mysql backed file system
1. requirements:
fuse, mysql, pthreads, c compiler
2. install:
download from https://github.com/jackdoe/sfs
$EDOTOR common.h #put proper mysql settings into common.h
make #DEBUG=1 if you want debug output
./sfs /mnt/fuse
3. info:
this is the table that is used:
Q("CREATE TABLE IF NOT EXISTS `entries_%s` ( \
`id` bigint(20) NOT NULL AUTO_INCREMENT, \
`path` varchar(255) NOT NULL, \
`data` longblob, \
`obj_type` int(11) NOT NULL, \
`uid` bigint(20) NOT NULL, \
`gid` bigint(20) NOT NULL, \
`mode` bigint(20) NOT NULL, \
`atime` bigint(20) NOT NULL, \
`mtime` bigint(20) NOT NULL, \
`ctime` bigint(20) NOT NULL, \
`btime` bigint(20) NOT NULL, \
PRIMARY KEY (`id`), \
UNIQUE KEY `path` (`path`))",identifier);
identifier is a copy of the mysql username (for example root)
and the table will be entries_root
table is automaticaly created at start, root directory is auto inserted as well
mysql user needs to select/update/delete/insert and maybe create if you dont have
the table created.
4. idea:
i made this as proof of concept to be used in diskless linux based routers,
where i can edit their config files remotely - for example /etc or /usr/local/etc,
with simple web based application.
5. supported calls:
sfs_oper.getattr = sfs_getattr;
sfs_oper.readdir = sfs_readdir;
sfs_oper.open = sfs_open;
sfs_oper.write = sfs_write;
sfs_oper.truncate = sfs_truncate;
sfs_oper.read = sfs_read;
sfs_oper.create = sfs_create;
sfs_oper.unlink = sfs_delete;
sfs_oper.chmod = sfs_chmod;
sfs_oper.chown = sfs_chown;
sfs_oper.statfs = sfs_statfs;
sfs_oper.mkdir = sfs_mkdir;
sfs_oper.readlink = sfs_readlink;
sfs_oper.symlink = sfs_symlink;
sfs_oper.rename = sfs_rename;
sfs_oper.rmdir = sfs_rmdir;
sfs_oper.utimens = sfs_utimens;
6. limitations:
MAX_PATH_SIZE: 256 chars
max data size: mysql's longblob 4gb
read/write is extreamly slow on big files
currently there is no locking
7. example usage:
$ mkdir ~/mysql_backed_fs
$ ./sfs ~/mysql_backed_fs/
$ cd ~/mysql_backed_fs
mysql_backed_fs $ mkdir test_directory
mysql_backed_fs $ cd test_directory/
test_directory $ ls -la
total 0
drwxr-xr-x 1 jack staff 0 Dec 27 09:24 .
drwxr-xr-x 1 jack staff 0 Dec 24 15:35 ..
test_directory $ touch empty_file
test_directory $ cat empty_file
test_directory $ echo 'not empty anymore' >> empty_file
test_directory $ cat empty_file
not empty anymore
test_directory $ cd ..
mysql_backed_fs $ ln -s test_directory/empty_file link
mysql_backed_fs $ cat link
not empty anymore
mysql_backed_fs $ ln -s test_directory link2
mysql_backed_fs $ cat link2/empty_file
not empty anymore
mysql_backed_fs $ cd ..
$ dd if=/dev/random of=blob.dd bs=1m count=1 && md5 blob.dd
1+0 records in
1+0 records out
1048576 bytes transferred in 0.080215 secs (13072073 bytes/sec)
MD5 (blob.dd) = 7d84c26f72aa5440c778014b1c9d23fe
$ cp blob.dd mysql_backed_fs/ && md5 mysql_backed_fs/blob.dd
MD5 (mysql_backed_fs/blob.dd) = 7d84c26f72aa5440c778014b1c9d23fe
$ cd mysql_backed_fs
as you can see write is very very slow
mysql_backed_fs $ dd if=/dev/zero of=./speed.blob.dd bs=1m count=5
5+0 records in
5+0 records out
5242880 bytes transferred in 14.620257 secs (358604 bytes/sec)
reading is fine on not very large files tho, because it uses SUBSTRING() and
gets only requested offset+size from the mysql
mysql_backed_fs $ time cat speed.blob.dd > /dev/null
real 0m0.005s
user 0m0.001s
sys 0m0.004s
mysql does not have a way for partial updates on blob fields so
there is download/modify (offset+size - usually 65k)/push the whole file and
the speed drops like a rock in water :)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=65k count=1
1+0 records in
1+0 records out
66560 bytes transferred in 0.004575 secs (14548589 bytes/sec)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=256k count=1
1+0 records in
1+0 records out
262144 bytes transferred in 0.015075 secs (17389082 bytes/sec)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=512k count=1
1+0 records in
1+0 records out
524288 bytes transferred in 0.046395 secs (11300513 bytes/sec)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=1024k count=1
1+0 records in
1+0 records out
1048576 bytes transferred in 0.301245 secs (3480808 bytes/sec)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=2048k count=1
1+0 records in
1+0 records out
2097152 bytes transferred in 1.876782 secs (1117419 bytes/sec)
but for small files it is fine