{"id":5000,"date":"2021-02-25T23:54:43","date_gmt":"2021-02-25T23:54:43","guid":{"rendered":"https:\/\/pariswells.com\/blog\/?p=5000"},"modified":"2021-02-25T23:54:43","modified_gmt":"2021-02-25T23:54:43","slug":"qnap-thin-pool-repair","status":"publish","type":"post","link":"https:\/\/pariswells.com\/blog\/research\/qnap-thin-pool-repair","title":{"rendered":"Qnap Thin pool repair"},"content":{"rendered":"

Stop all of services and umount all of volumes<\/strong><\/p>

Step1: Stop all of services<\/strong><\/p>

\/etc\/init.d\/services.sh stop<\/p>

\/etc\/init.d\/qsnapman.sh stop<\/p>

\/sbin\/daemon_mgr lvmetad stop “\/sbin\/lvmetad” rm \/var\/run\/lvm\/lvmetad.socket<\/p>

Step2: Confirm which volume group or pool need to repair<\/strong><\/p>

pvs # list all of volume groups<\/p>

lvs -a # list all of volumespools and volume groups<\/p>

lvs -o+time # can list the created date\/time of volumespools lvs -o+thin_id # can list the devices id of volumes<\/p>

Check Volume Groups using “pvs” command<\/strong><\/p>

PV VG Fmt Attr PSize PFree<\/p>

\/dev\/drbd1 vg1 lvm2 a– 7.24t 0 # \/dev\/drbd1 indicates md1(RAID group 1)<\/p>

\/dev\/drbd2 vg2 lvm2 a– 7.26t 0 # \/dev\/drbd2 indicates md1(RAID group 2)<\/p>

.<\/p>

.<\/p>

.<\/p>

# One Volume Group may include >= 2 RAID groups<\/p>

Check devices on the volume group\/pool which we need to repair using “lvs -a” command<\/strong><\/p>

LV<\/p><\/td>

VG<\/p><\/td>

Attr<\/p><\/td>

LSize<\/p><\/td>

Pool Origin<\/p><\/td>

Data% Meta%<\/p><\/td>

Move Log Cpy%Sync<\/p><\/td><\/tr>

Convert<\/p><\/td>

\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td><\/tr>

lv1<\/p><\/td>

vg1<\/p><\/td>

Vwi-aot—<\/p><\/td>

6.00t<\/p><\/td>

tp1<\/p><\/td>

100.00<\/p><\/td>

# Data volume 1<\/p><\/td><\/tr>

lv1312<\/p><\/td>

vg1<\/p><\/td>

-wi-ao—-<\/p><\/td>

756.00m<\/p><\/td>

# snapshots pool<\/p><\/td>

\u00a0<\/td>\u00a0<\/td><\/tr>

lv2<\/p><\/td>

vg1<\/p><\/td>

Vwi-aot— 200.00g tp1<\/p><\/td>

100.00<\/p><\/td>

# Data volume 2<\/p><\/td><\/tr>

lv544<\/p><\/td>

vg1<\/p><\/td>

-wi——- 74.14g<\/p><\/td>

# Reserved to repairing temporary<\/p><\/td>

\u00a0<\/td><\/tr>

snap10001<\/p><\/td>

vg1<\/p><\/td>

Vwi-aot—<\/p><\/td>

6.00t tp1<\/p><\/td>

lv1<\/p><\/td>

100.00<\/p><\/td>

# Snapshot volume<\/p><\/td><\/tr>

snap10002<\/p><\/td>

vg1<\/p><\/td>

Vwi-aot—<\/p><\/td>

6.00t tp1<\/p><\/td>

lv1<\/p><\/td>

100.00<\/p><\/td>

# Snapshot volume<\/p><\/td><\/tr>

.<\/p><\/td>

\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td><\/tr>

.<\/p><\/td>

\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td><\/tr>

.<\/p><\/td>

\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td><\/tr><\/tbody><\/table>

Step3: Assume vg1 need to repair, we need to umount the volumes which belong to the vg1(lv1 will be mounted in \/share\/CACHEDEV1_DATA….. and so on….)<\/strong><\/p>

If there is any volumes or snapshots which is mounted and the mounted volumes belong to the volume group which need to repair, please umount them<\/strong><\/p>

# below umount all of data volumes umount \/share\/CACHEDEV1_DATA<\/p>

umount \/share\/CACHEDEV2_DATA<\/p>

.<\/p>

.<\/p>

umount \/share\/CACHEDEV${#}_DATA<\/p>

# below umount snapshots umount \/dev\/mapper\/vg1-snap*<\/p>

if can not umount, lsof \/share\/CACHEDEV{#}_DATA to check, and “kill -9” these process. Try to umount the volumes again.<\/strong><\/p>

Remove all of cache devices and inactivate volume group<\/strong><\/p>

Step1: You can list which cache devices on the pool:<\/strong><\/p>

ls -l \/dev\/mapper\/ # will list all of devices on the pool<\/p>

Step2: The result of the above command looks like below(below assume vg1 is the volume group which need to repair)<\/strong><\/p>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

9<\/p><\/td>

2020-02-15<\/p><\/td>

10:16<\/p><\/td>

cachedev1<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

50<\/p><\/td>

2020-02-15<\/p><\/td>

10:16<\/p><\/td>

cachedev2<\/p><\/td><\/tr>

crw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

10,<\/p><\/td>

236<\/p><\/td>

2020-02-15<\/p><\/td>

18:14<\/p><\/td>

control<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

7<\/p><\/td>

2020-02-15<\/p><\/td>

10:16<\/p><\/td>

vg1-lv1<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

8<\/p><\/td>

2020-02-15<\/p><\/td>

10:16<\/p><\/td>

vg1-lv1312<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

10<\/p><\/td>

2020-02-15<\/p><\/td>

10:16<\/p><\/td>

vg1-lv2<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

11<\/p><\/td>

2020-02-18<\/p><\/td>

01:00<\/p><\/td>

vg1-snap10001<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

13<\/p><\/td>

2020-02-19<\/p><\/td>

01:00<\/p><\/td>

vg1-snap10002<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

15<\/p><\/td>

2020-02-20<\/p><\/td>

01:00<\/p><\/td>

vg1-snap10003<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

17<\/p><\/td>

2020-02-21<\/p><\/td>

01:00<\/p><\/td>

vg1-snap10004<\/p><\/td><\/tr>

brw——-<\/p><\/td>

1<\/p><\/td>

admin<\/p><\/td>

administrators<\/p><\/td>

253,<\/p><\/td>

19<\/p><\/td>

2020-02-22<\/p><\/td>

01:00<\/p><\/td>

vg1-snap10005<\/p><\/td><\/tr>

.<\/p><\/td>

\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td><\/tr>

.<\/p><\/td>

\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td><\/tr>

.<\/p><\/td>

\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td>\u00a0<\/td><\/tr><\/tbody><\/table>

Step3: Find cachedev${#} and remove them using dmsetup<\/strong><\/p>

dmsetup remove cachedev1 dmsetup remove cachedev2<\/p>

.<\/p>

.<\/p>

.<\/p>

dmsetup remove cachedev${#}<\/p>

Step4: Inactivate the volume group which need to repair<\/strong><\/p>

lvchange -an vg1 # Assume vg1 is the volume group which need to repair<\/p>

Step5: Please check again with “ls -l \/dev\/mapper” command to confirm there is no any block device of vg1. The result should be the below:<\/strong><\/p>

crw——- 1 admin administrators 10, 236 2020-02-15 18:14 control<\/p>

Collect logs of thin pool metadata and backup metadata of the thin pool<\/strong><\/p>

Step1: Download collect tools<\/strong><\/p>

wget http:\/\/download.qnap.com\/Storage\/tsd\/utility\/tp_collect.sh<\/a><\/p>

Step2: Execute collect tools to collect logs of the metadata or backup metadata of the pool<\/strong><\/p>

sh tp_collect.sh<\/p>

When the tp_collect.sh start running, please remember enter \u201cpool id\u201d. The pool id is the id of the pool which we want to repair. For examples: vg1\/tp1, please input 1vg2\/tp2, please input 2……and so on…..<\/strong><\/p>

If execute tp_collect fail, you need to let the customer plug one USB external drive(about 100G or more) in the NAS to backup metadata<\/strong><\/p>

# assume vg1\/tp1 need to collect or repair lvchange -ay vg1\/tp1_tmeta<\/p>

# Change directory to the USB external device cd \/share\/external\/DEVXXXXXX<\/p>

# backup metadata using dd command<\/p>

dd if=\/dev\/mapper\/vg1-tp1_tmeta of=tmeta.bin bs=1M<\/p>

If tp_collect is executed completely or successfully, you can skip the above flow and please backup collect.tgz.<\/strong><\/p>

Step 3: Please confirm the backup of metadata is correct<\/strong><\/p>

# A. thin check original metadata<\/p>

pdata_tools_8192(or 4096) thin_check \/dev\/mapper\/vg1-tp1_tmeta<\/p>

# B. thin check backup metadata<\/p>

pdata_tools_8192(or 4096) thin_check \/mnt\/collect\/tmeta.bin # If the metadata is backup in the USB external drive, please pdata_tools_8192(or 4096) thin_check \/share\/external\/DEVXXXXXX\/tmeta.bin<\/p>

The result of the above A, B need to be the same. If the A, B is the same, Please be sure to backup the tmeta.bin to another storage or USB<\/strong><\/p>

external drive!!! If the vg{#}-tp{#} repaired fail, the vg{#}-tp{#}_tmeta can be restored using tmeta.bin.<\/strong><\/p>

 <\/p>","protected":false},"excerpt":{"rendered":"

Stop all of services and umount all of volumesStep1: Stop all of services\/etc\/init.d\/services.sh stop\/etc\/init.d\/qsnapman.sh stop\/sbin\/daemon_mgr lvmetad stop “\/sbin\/lvmetad” rm \/var\/run\/lvm\/lvmetad.socketStep2: Confirm which volume group or pool need […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[3449,3451],"class_list":["post-5000","post","type-post","status-publish","format-standard","hentry","category-research","tag-qnap","tag-thin-pool-repair"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/posts\/5000","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/comments?post=5000"}],"version-history":[{"count":2,"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/posts\/5000\/revisions"}],"predecessor-version":[{"id":5002,"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/posts\/5000\/revisions\/5002"}],"wp:attachment":[{"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/media?parent=5000"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/categories?post=5000"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pariswells.com\/blog\/wp-json\/wp\/v2\/tags?post=5000"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}