bugGluster - Bugs: bug #21581, rsync failures and rm failures

 
 

bug #21581: rsync failures and rm failures

Submitted by:  None
Submitted on:  Wed 14 Nov 2007 07:34:11 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Improper behaviour
Status: FixedPrivacy: Public
Assigned to: Amar Tumballi <amarts>Originator Name: 
Originator Email: -unavailable-Open/Closed: Closed
Release: Operating System: GNU/Linux
Reproducibility: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sat 12 Apr 2008 01:55:21 AM UTC, comment #10:

reported to be resolved.

Amar Tumballi <amarts>
Project MemberIn charge of this item.
Fri 28 Mar 2008 02:40:09 PM UTC, comment #9:

Hello

Upgrading to 1.3.8pre4 appears to have resolved the issue.

Thanks.

Anonymous
Wed 26 Mar 2008 09:17:34 PM UTC, comment #8:

Is it possible for you to test it using latest release? 1.3.8pre4? as we have done lots of enhancements since 1.3.7

-amar

Amar Tumballi <amarts>
Project MemberIn charge of this item.
Wed 26 Mar 2008 08:54:34 PM UTC, comment #7:

Hi.

This has occured for me today as well, please see my attached glusterfs configuration.

I'm running 4 machines with AFR over two round robin volumes. Rsync complains, as well as the client blocking without any apparent explanation in the logs.

$ ./glusterfsd --version
glusterfs 1.3.7 built on Feb 25 2008

rsync error =========

rsync: mkstemp "/data/somefile.YB3jg6" failed: Function not implemented (38)
rsync error: unexplained error (code 130) at rsync.c(271) [sender=2.6.9]

client config =======

#
# Left side
#

volume media406
type protocol/client
option transport-type tcp/client
option remote-host xx.xx.xx.46
option remote-subvolume media
end-volume

volume media408
type protocol/client
option transport-type tcp/client
option remote-host xx.xx.xx.48
option remote-subvolume media
end-volume

volume media406-ns
type protocol/client
option transport-type tcp/client
option remote-host xx.xx.xx.46
option remote-subvolume media-ns
end-volume

volume left
type cluster/unify
subvolumes media406 media408
option namespace media406-ns
option scheduler rr
option rr.limits.min-disk-free 5
end-volume

#
# Right side
#

volume media407
type protocol/client
option transport-type tcp/client
option remote-host xx.xx.xx.47
option remote-subvolume media
end-volume

volume media415
type protocol/client
option transport-type tcp/client
option remote-host xx.xx.xx.51
option remote-subvolume media
end-volume

volume media407-ns
type protocol/client
option transport-type tcp/client
option remote-host xx.xx.xx.47
option remote-subvolume media-ns
end-volume

volume right
type cluster/unify
subvolumes media407 media415
option namespace media407-ns
option scheduler rr
option rr.limits.min-disk-free 5
end-volume

#
# Replicate between left and right
#

volume media
type cluster/afr
option replicate *:2
subvolumes left right
end-volume

Thanks,
-unavailable-

Anonymous
Wed 06 Feb 2008 06:54:26 PM UTC, comment #6:

It was working for me from last 2.5 months, and there is no report of bug with the latest source repository. Hence closing this bug. Request to open the new bug if the behavior persists.

Amar Tumballi <amarts>
Project MemberIn charge of this item.
Wed 30 Jan 2008 11:16:08 AM UTC, comment #5:

Hi,
can you confirm the bug still exists with the latest source? I couldn't reproduce it locally. Can we close the bug if its fixed? as there is no update from you since 2months.

Amar Tumballi <amarts>
Project MemberIn charge of this item.
Thu 06 Dec 2007 12:43:41 PM UTC, comment #4:

Hi, Did you try to reproduce the issue with latest source? Please let us know the result.

Amar Tumballi <amarts>
Project MemberIn charge of this item.
Fri 23 Nov 2007 12:25:53 AM UTC, comment #3:

hi,

I am using the latest release (1.3.7). I will check tomorrow if I can replicate the behavior with the latest source.

thanks,
cory

Anonymous
Thu 22 Nov 2007 11:33:24 AM UTC, comment #2:

Hi,
I am looking into the issue now. Can you tell me which version of GlusterFS were you trying when you hit this behavior? Also, can you confirm that with the latest source from tla, same setup results in bugs?

Thanks and Regards,
Amar

Amar Tumballi <amarts>
Project MemberIn charge of this item.
Wed 14 Nov 2007 08:48:41 PM UTC, comment #1:

One more note to add. Removing afr from the picture seems to get rid of these problems. currently rsync-ing and rm-ing with no issue.

Anonymous
Wed 14 Nov 2007 07:34:11 PM UTC, original submission:

I have a small glusterfs setup with 6 machines (the configuration for the server and client are attached).

A summary of the setup is this:

- I have 3 mirrored volumes composed to 2 machines each using AFR
- The AFR volumes are unified, and the namespace is also AFRed.
- I have tried the ALU scheduler in the attached configuration, but the error occurs regardless.
- The computers are connected via a 1000Mbit switch.

The error occurs when I try to rsync a lot of files onto the glusterfs, using rsync as follows:

rsync -av /etc /usr /bin /sbin /dbs /home-local /lib /opt /root /share /src /srv /sys /var /mnt/gluster-unify/backup/

/mnt/gluster-unify/ is empty at the start of the test.

Many files are synced properly, but many also come up with an error similar to this:

rsync: mkstemp "/mnt/gluster-unify/backup/etc/gconf/gconf.xml.schemas/schemas/apps/stickynotes_applet/settings/.%gconf.xml.b7bb0P" failed: Function not implemented (38)

The same rsync command works fine on local storage.

Now, a perhaps related error occurs when I try to delete all of the files in the glusterfs using the following command:
rm -rf /mnt/gluster-unify/*

The error returned is:
rm: FATAL: cannot ensure `/mnt/gluster-unify/backup/etc/gconf/gconf.xml.schemas/schemas/apps/gnumeric/core/sort' (returned to via ..) is safe: Input/output error

In both cases, the log files on the machines remain silent.

Last night glusterfs continually crashed with the same setup using rsync but I can't seem to reproduce it this morning.

If there is any additional information that you need, please let me know.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #14398:  glusterfs-server.vol.brick added by None (615B - application/octet-stream - gluster-server.vol on the storage-only nodes)
file #14396:  glusterfs-server.vol added by None (4KiB - application/octet-stream)
file #14397:  glusterfs-client.vol added by None (994B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by amarts (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 10 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 12 Apr 2008 01:55:21 AM UTCamartsStatusNone=>Fixed
      Open/ClosedOpen=>Closed
    Wed 26 Mar 2008 09:17:34 PM UTCamartsStatusWorks For Me=>None
      Open/ClosedClosed=>Open
    Wed 06 Feb 2008 06:54:26 PM UTCamartsOpen/ClosedOpen=>Closed
    Wed 30 Jan 2008 11:16:08 AM UTCamartsStatusNone=>Works For Me
    Thu 22 Nov 2007 11:33:24 AM UTCamartsAssigned toNone=>amarts
    Wed 14 Nov 2007 07:44:15 PM UTCNoneAttached File-=>Added glusterfs-server.vol.brick, #14398
    Wed 14 Nov 2007 07:34:11 PM UTCNoneAttached File-=>Added glusterfs-server.vol, #14396
      Attached File-=>Added glusterfs-client.vol, #14397

    Back to the top


    Powered by Savane 3.1-cleanup1