EXT3-fs Error : Raid remounting read-only

[aaronlippold]

Hello,

I have been having issues coping data to my RAID of late. Either via NFS or SSH, when I try to copy large data amounts I receive a EXT3-fs error and my raid remounts in read-only mode.

I was wondering if anyone else has encounter this problem.

I have firmware version 1.00.07

--- dmesg output ---

criticalevent_user: raid_healthy md1
RAID1 conf printout:
--- wd:5 rd:5
disk 0, wo:0, o:1, dev:sda1
disk 1, wo:0, o:1, dev:sdb1
disk 2, wo:0, o:1, dev:sdc1
disk 3, wo:0, o:1, dev:sdd1
disk 4, wo:0, o:1, dev:sde1
Adding 987896k swap on /dev/md1. Priority:-1 extents:1
Read after write, bit[35]=1
EXT3-fs error (device dm-1): ext3_new_block: Allocating block in system zone - block = 217022472
Aborting journal on device dm-1.
EXT3-fs error (device dm-1) in ext3_prepare_write: Journal has aborted
__journal_remove_journal_head: freeing b_committed_data
ext3_abort called.
EXT3-fs error (device dm-1): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only

--- end ----

--- mount output ---
root@127.0.0.1:~# mount
rootfs on / type rootfs (rw)
/dev/root on / type ext2 (rw)
/dev/hda5 on /app type ext2 (rw)
tmpfs on /tmpfs type tmpfs (rw)
/dev/hda3 on /cloopfs type ext2 (rw)
proc on /proc type proc (rw,nodiratime)
sysfs on /sys type sysfs (rw)
none on /dev/pts type devpts (rw)
/dev/cloop0 on /lib/modules type ext2 (ro)
/dev/cloop1 on /opt type ext2 (ro)
/dev/cloop2 on /img/www type ext2 (ro)
/dev/cloop3 on /img/bin type ext2 (ro)
sunrpc on /tmpfs/var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
/dev/vg0/lv0 on /raid/data type ext3 (ro,noatime)
/dev/vg0/syslv on /raid/sys type ext3 (rw)

--- end ---

After I used the Filesystem Check utility I got this output from dmesg

--- dmesg after FSCK Util in WebTool ------

md: bind<sda1>
md: nonpersistent superblock ...
md: bind<sdb1>
md: nonpersistent superblock ...
md: bind<sdc1>
md: nonpersistent superblock ...
md: bind<sdd1>
md: nonpersistent superblock ...
md: bind<sde1>
md: nonpersistent superblock ...
raid1: raid set md1 active with 5 out of 5 mirrors
criticalevent_user: raid_recovery md1
md: syncing RAID array md1
md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc.
md: using maximum available idle IO bandwith (but not more than 200000 KB/sec) for reconstruction.
md: using 128k window, over a total of 987904 blocks.
md: bind<sdb2>
md: bind<sdc2>
md: bind<sdd2>
md: bind<sde2>
md: bind<sda2>
raid5: device sda2 operational as raid disk 0
raid5: device sde2 operational as raid disk 4
raid5: device sdd2 operational as raid disk 3
raid5: device sdc2 operational as raid disk 2
raid5: device sdb2 operational as raid disk 1
raid5: allocated 5248kB for md0
raid5: raid level 5 set md0 active with 5 out of 5 devices, algorithm 2
RAID5 conf printout:
--- rd:5 wd:5 fd:0
disk 0, o:1, dev:sda2
disk 1, o:1, dev:sdb2
disk 2, o:1, dev:sdc2
disk 3, o:1, dev:sdd2
disk 4, o:1, dev:sde2
criticalevent_user: raid_healthy md0
kjournald starting. Commit interval 5 seconds
EXT3-fs warning (device dm-1): ext3_clear_journal_err: Filesystem error recorded from previous mount: IO failure
EXT3-fs warning (device dm-1): ext3_clear_journal_err: Marking fs in need of filesystem check.
EXT3-fs warning: mounting fs with errors, running e2fsck is recommended
EXT3 FS on dm-1, internal journal
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
g_file_storage gadget: N5200 USB Storage Gadget, version: 20 October 2004
g_file_storage gadget: Number of LUNs=1
g_file_storage gadget-lun0: ro=0, file: /dev/mapper/vg0-lv1
Read after write, bit[17]=1
Read after write, bit[17]=1

---- end ---

Thanks.

Aaron

[gyrfalcon]

Did you run e2fsck also?

marc.theaimsgroup.com/?l=dm-devel&m=116322666822751&w=2

[omega]

From I was reading in the Internet, this error most likely is caused by a bad block on one of the disks.

What does the SMART status says for each of the drives? Do you have installed the SSH and SYSUSER module so that you're able to login to your N5200?

The references I found about this recommend to rebuild the filesystem.... but this means that you need to save all of the data to another place first and copy it back after redoing the filesystem....

Andreas

[aaronlippold]

Hello,

The status on all the drives is 'OK'. 0 counts on all.

Yes, I have installed both the SSH and SYSUSER modules and can log into the system.

I guess it wouldn't be too much of an issue to rebuild, just a pain. I don't think I have anything on the RAID I don't have in other sources.

However, this doesn't fill me with a lot of confidence . I want to move this stuff to the raid in the end after all ...

What about running fsck myself?

A