bugGluster - Bugs: bug #24105, Files not created (self-heal) on...

 
 

bug #24105: Files not created (self-heal) on AFR volume when the file is missing from the underlying FS

Submitted by:  None
Submitted on:  Tue 19 Aug 2008 07:14:47 PM UTC  
 
Category: GlusterFSSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Improper behaviour
Status: NonePrivacy: Public
Assigned to: NoneOriginator Name: anon
Originator Email: -unavailable-Open/Closed: Closed
Release: 1.3.10 built on Aug 15 2008 09Operating System: GNU/Linux
Reproducibility: Every Time

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 21 Aug 2008 05:21:12 PM UTC, comment #2:

That's certainly fair, which is why I was unsure, but shouldn't there be some way for the system to recover from non-catastrophic disk failure? What I was attempting to simulate was a "file server power supply fails" scenario. During reboot, an fsck might move test.txt from the underlying FS to lost+found, thus mimicking my user intervention.

In a non-trivial case where you have, say, 300k files, about 100 gigs total, sitting on an AFRd server, you would then have to copy/rsync the entire file structure over. Then hopefully you can run a few recursive xattr commands to get everything back, and then hopefully a "find . " so that gluster syncs back up. Or, AFR could self heal. If N copies of the file match in every aspect and the N+1 server simply doesn't have it, that seems like a cheap and easy decision to make?

All I'm saying is that, personally, what a user is "supposed" to do versus what can actually happen is something I would love for Gluster to take care of. I would volunteer to help, but I don't think anybody wants my worthless coding skills to touch a filesystem : )

Thanks again! Fantastic work...

Wes

Wes Deviers <wdeviers>
Wed 20 Aug 2008 04:19:43 AM UTC, comment #1:

Users are not supposed to edit the backend filesystem directly. Closing the bug.

Krishna Srinivas <krishnasrinivas>
Project Member
Tue 19 Aug 2008 07:14:47 PM UTC, original submission:

This might be a bug or feature, I'm not exactly positive, but I consider it "improper behavior."

Steps:

1) Create an AFR setup. Mine is server-side, using 3 copies.
2) Mount the AFR volume using a client; works as expected.
3) Copy a file ("text.txt") onto the GlusterFS mount point on the client. File is created 3 times, as expected.
4) Shut down glusterfsd on Server0. Failover using RR DNS works as expected on the client. SSH to Server0 then remove test.txt from the underlying filesystem (in my case, ext3).
5) File still exists on Servers 1 & 2, and is viewable by the client
6) Restart glusterfsd on Server0; it rejoins the cluster.
7) test.txt is not recreated on Server0
8) Go back to the client, do a `cat test.txt`; Works as expected. test.txt is still not recreated on Server0.
9) Server0 shows the following:

2008-08-19 13:06:21 D [inode.c:577:__create_inode] localMountLocks/inode: create inode(2)
2008-08-19 13:06:21 D [inode.c:367:__active_inode] localMountLocks/inode: activating inode(2), lru=0/1024
2008-08-19 13:06:21 D [inode.c:367:__active_inode] localMountLocks/inode: activating inode(2), lru=0/1024
2008-08-19 13:06:23 D [inode.c:367:__active_inode] localMountLocks/inode: activating inode(2), lru=0/1024

10) Force client to connect to Server0 instead of the RR DNS entry, client does not show test.txt

Result is that test.txt is unavailable to 1/3 of clients connecting. I view this as a bug because in the event of a crash, it's possible that the test.txt inode may be cleared/unlinked. We would expect the gluster backend to replicate the file correctly and therefore wouldn't worry about it.

Thanks!

Wes Deviers/aka "Yhetti"
-unavailable-

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by wdeviers (Posted a comment)
  • -unavailable- added by krishnasrinivas (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 20 Aug 2008 04:19:58 AM UTCkrishnasrinivasOpen/ClosedOpen=>Closed
    Wed 20 Aug 2008 04:19:43 AM UTCkrishnasrinivasOriginator Name=>anon
      Originator Email=>-unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup1