Link to "snapshot" reserved space?

[Arctra]

Hi all.

As I posted in this thread, I added a new drive to my NAS and it allocated all the new space to the "snapshot" reserved space. Whilst I'm waiting for Thecus to get back to me on a solution I just want to run a possible work-around by you all in case one of you has experience.

Do u think creating a link (soft link) to a new folder in the reserved snapshot space will work in order to be able to use that space? I have already created a couple of links between some of my directories in the normal raid space and they work nicely. But does anyone know if there will be a problem using the snapshot area?

BTW, I have firmware 1.00.07 which I believe has the snapshot functionality disabled.

Looking forward to any of your feedback!

Cheers

[memloc]

Hi Arctra,

Did you work out this problem? I have just done the same thing.
I added a third disk to my raid, migrated Raid 0 (2disk) to Raid 0 (3 Disk), But when it finnished the free space was allocated to snapshot and the raid still has the same usable data capacity ?

[Arctra]

Na, I'm afraid I didn't fix it. Thecus support asked me to open up a way for them to log in and fix it for me, but I wasn't comfortable with that. In the end I backed all my data up on a mates machine, rebuilt the entire raid from scratch, and then copied the data back again.

Thecus really do need to get serious about providing a DIY solution to fixing this at home yourself. It's happenbed to a few users on this forum now so I struggle to believe it's that uncommon.

[ner02]

I upgraded from Raid 5 with 3 750GB drives to Raid 5 with 4 750GB drives. I did the Raid Migration, and it allocated all the space to snapshot. I analyzed the script in /img/bin (resize_raid.sh in particular) and determined:

Running 'pvdisplay' showed that the physical volume had been sized correctly, but 'lvdisplay' showed that the logical volume /dev/vg0/lv0 (I think that's it...) was still only 1.2TB. So basically the logical volume had not been properly resized.

I then copied that resize_raid.sh script over to /app, edited it to make the lvresize command take +650G as a parameter instead of the ${diffsize} parameter.

Of course, I was running over SSH, so I had to wrap the call to a script in another script that trapped any exit signals and killed the SSH process so that /raid/data could be unmounted.

All in all, a pretty big pain for something that should have "just worked."

[Arctra]

Great job ner02!

Too late for me now though

In any case, any chance you could post your modified script(s) and/or a howto on how you go about wrapping the one script in another etc?

I reckon something like that would definitely be useful to others in the future as more and more people use the N5200 and run into the same problem.

Cheers

[ner02]

This one is the wrapper. It redirects output and keeps the other script running after you kill ssh.

-nohup.sh-

#!/bin/sh
trap "" 1 15
if test -t 2>&1
then
       echo "Sending output to 'nohup.out'"
       exec "$@" >> nohup.out 2>&1
else
       exec "$@" 2>&1
fi


This script is a modified resize_raid.sh. The original is in /img/bin

-add_raid.sh-

#!/bin/sh -x

use_qt=`cat /tmp/use_qtman`

mddisk="/dev/md0"
datadisk="/dev/vg0/lv0"
minspace="2"
snap_path="/raid/snapshot"
data_path="/raid/data"
vg_path="/dev/vg0"
vgroup="vg0"
syslv="syslv"
sys_path="/raid/sys"
mount_lv=$1
#This next line you should replace 12644
#with the process ID of sshd
kill 12644

##STOP service
/img/bin/service stop

##STOP data volumne
umount ${data_path}
/sbin/lvchange -an -f ${vg_path}/lv0 > /dev/null 2>&1

#Change +200G (200 gigabytes) to however much space you
# have available and want to add to the data partition
lvresize ${datadisk} -L +200G

/sbin/lvchange -ay -f ${vg_path}/lv0 > /dev/null 2>&1
/sbin/resize2fs -f ${datadisk}

mount -o acl,rw ${datadisk} ${data_path}


##Check whether need mount other volumn
sysmount=`mount |awk '/\/raid\/sys/'`
if [ "${sysmount}" = "" ];then
       vgchange -ay ${vgroup}
       mount ${vg_path}/${syslv} ${sys_path}
fi

echo 0 > /var/tmp/raidlock
echo "Healthy" > /var/tmp/rss
echo "Busy 0" > /proc/thecus_io
echo "`date \"+%Y/%m/%d %H:%M:%S\"` `hostname` RAID resize successfully." >> /var/log/information

##START service
/img/bin/service start

#this should make the thecus beep.
echo -e "\007" > /dev/tty10


So simply executing './nohup.sh ./add_raid.sh' should do the trick.
It's a little scary and I wouldn't recommend it, as its tough to get any progress indicator of what's going on. It seems like it takes around an hour per 200Gig you're adding, but it's tough to get status. Worst of all, if the script stops prematurely due to some sort of error, you won't get any indication, and you'll have to reboot the Thecus to get sshd running again. I recommend giving it plenty of time, rather than rebooting it while it's potentially still running.

This could all definitely be cleaned up (Hello Thecus?) It should be pretty simple to have the front panel LCD echo the status of running LVM commands and give you info as to how things are going.

[slowbot]

Wow. This is exactly what I need. I wish I had the brass ones to pull it off. If I have this many questions I probably shouldn't risk my data but...

Is there anyway you could further explain how to run these scripts in even more detail. (ie software to use and step by step how to run the scripts.)

Any more clues on how to apply this fix for a newbie would be greatly appreciated.

[cstierle]

Is there any further information on resolving this issue? I have just added a 4th 500gig drive to a 3 drive raid and all of the space was allocated to snapshot. I am running version FW 1.00.08

I can follow most of the solution listed above but in the following section:

#Change +200G (200 gigabytes) to however much space you
# have available and want to add to the data partition
lvresize ${datadisk} -L +200G

what should my size be? It shows 476,106 MB as the snapshot capcity. Should I change the +200G to +476G?

[Arctra]

Sounds about right, however I would err on the side of caution and make it 470G just in case some snapshot space is needed. Good luck mate.

[cstierle]

Well I set the size to 466 just to be safe and ran the scripts. As soon as I ran them my ssh session dropped so I know at least that part worked. I left the box running for 24+ hours and then re-booted.

No change in the size of the snapshot or data partitions. Any other advice would be greatly appriciated.

As a side note, I opened a ticket with Thecus about this on 7/11 and surprisingly their crack support team still has not contacted me or assigned a priority to the ticket yet. Go figure.

[cstierle]

As an update, to anyone else that has this issue I have gone back and done some research and finally was successful in migrating the snapshot capacity to data.

After my last attemp in running the scripts I looked in /app/nohup.out to see if I could see any issues. Well I spotted the following:

+ lvresize /dev/vg0/lv0 -L +466G
Insufficient suitable allocatable extents for logical volume lv0: 270 more required

So I bumped it down to +465G and checked the HTTP GUI and nothing changed. So checked /app/nohup.out and got the following:

+ lvresize /dev/vg0/lv0 -L +465G
Insufficient suitable allocatable extents for logical volume lv0: 14 more required

So I bumped it down to +464G and checked the HTTP GUI. Status of both Data and Snapshot were N/A. I let it sit overnight and when I checked in the morning my Data capacity was the proper size. I checked /app/nohup.out and got the following:

+ lvresize /dev/vg0/lv0 -L +464G
Extending logical volume lv0 to 1.31 TB
Logical volume lv0 successfully resized

From pvdisplay and lvdisplay output it looks like everything worked out. A big thank you to both ner02 and Arctra for help on this.

And of course I am still waiting to hear back from Thecus support on my open ticket. What are the odds that no one will ever contact me?

[Arctra]

Great work cstierle. Thanks for the kudo's but I think it's ner02 is the one that did the the hard work here together with yourself.

Cheers.

[fabi]

Hi

I would also like to use the snapshot space for data. However, your script didn't fully work in my case. In nohup.out I see

...
+ umount /raid/data
umount: /raid/data: device is busy
umount: /raid/data: device is busy
+ /sbin/lvchange -an -f /dev/vg0/lv0
+ lvresize /dev/vg0/lv0 -L +50G
Extending logical volume lv0 to 670.20 GB
Logical volume lv0 successfully resized
+ /sbin/lvchange -ay -f /dev/vg0/lv0
+ /sbin/resize2fs -f /dev/vg0/lv0
resize2fs 1.38 (30-Jun-2005)
/dev/vg0/lv0 is mounted; can't resize a mounted filesystem!
...

So the resize failed because the drive couldn't be unmounted. Did it simply work for you to unmount it? I tried to kill all processes referencing /raid/data but I still have problems.

Thanks

[cstierle]

In answer to slowbot and anyone else that is looking for a step by step process, here is what I did (again the hard stuff came from ner02 so the credit should go to him). Anything between quotes is what should be entered at the command prompt:

1. ssh to your Thecus and log in.
2. “cd /app”
3. “vi nohup.sh”
4. “i” – Allows you to insert text in the file
5. Copy and paste the nohup.sh code from ner02’s message above.
6. hit the esc key – This moves you out of Insert mode
7. “:wq” – this writes the file and quits from vi. If you really screw up in vi and want to quit without saving the changes to the file you can “:q!” instead. To go back into the file just “vi nohup.sh” to open it back up.
8. “ps | grep sshd” – This will give you the process id of the ssh daemon
9. “vi add_raid.sh”
10. "i” – Allows you to insert text in the file
11. Copy and paste the add_raid.sh code from ner02’s message above.
12. hit the esc key – This moves you out of Insert mode
13. “/12644” – searches the script for 12644. The first one it finds is in the comment.
14. “n” – skips to the next 12644 which is the kill 12644 line and the one we want to change.
15. “xxxxx” – deletes the 12644.
16. “i” then hit the space bar – goes into Insert mode and puts a space after the word kill
17. Enter the PID that came back for the sshd process from step 8 above
18. hit the esc key – This moves you out of Insert mode
19. “/200G” - searches the script for 200G. The first one it finds is in the comment.
20. “n” – skips to the next 200G which is the lvresize ${datadisk} -L +200G line and the one we want to change.
21. “xxx” – deletes the 200.
22. “i” – Allows you to insert text in the file
23. Enter the size of the available space of the new drive. Mine happened to be 464 for a 500 gig drive but as can be seen above this was determined through trial and error.
24. hit the esc key – This moves you out of Insert mode
25. “:wq” – this writes the file and quits from vi. If you really screw up in vi and want to quit without saving the changes to the file you can “:q!” instead. To go back into the file just “vi add_raid.sh” to open it back up.
26. ls -l | grep .sh – This will show you the permissions on the files that were just created. They probably only have -rw-r--r— permissions.
27. “chmod 755 nohup.sh” – Gives read and execute permissions to everyone and read, write and execute to the owner.
28. “chmod 755 add_raid.sh” - Gives read and execute permissions to everyone and read, write and execute to the owner.
29. ls -l | grep .sh – This will show you the permissions on the files that were just created. They should now have -rwxr-xr-x permissions.
30. “./nohup.sh ./add_raid.sh” – runs the scripts that were just created. If everything goes well you should lose your shh session and in the HTTP GUI you should see your RAID and Snapshot capacity as N/A. After a couple of hours the resized quantities will come back and you can use “pvdisplay” and “lvdisplay” to verify everything is as it should be. I don’t know everything that can go wrong but for me my first couple of attempts were unsuccessful because I used the wrong resize value. The way I found this issue was to vi the /app/nohup.out file and search for the value I used.

I don’t know if this will be helpful to anyone or was even necessary but I hope it will help someone. Please let me know if there are any issues with this instruction list and I will update the post.

And for the normal disclaimer, I felt comfortable enough with the process and the thought of losing my data to go through this procedure. Please don’t think that by me writing this and being successful with it that I am saying you should do this or you will not lose all of your data. Use this information at your own risk.

[cstierle]

fabi,

I am at work right now so I can't look at my nohup.out file but I'll see if I can help you out once I get home. One thing that comes to mind though is if anyone has any files open or is accessing the Thecus while you are trying to umount the RAID it probably won't work. I am pretty sure you can't do a umount when a user has any files open or even if someone has an active connection to a directory.

CS