Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian User (http://www.linux-archive.org/debian-user/)
-   -   CIFS and data integrity ; Sorry for the noise (http://www.linux-archive.org/debian-user/689022-cifs-data-integrity-sorry-noise.html)

Paul E Condon 07-30-2012 08:09 PM

CIFS and data integrity ; Sorry for the noise
 
On 20120730_122543, Paul E Condon wrote:
> On 20120730_065640, Mark Fletcher wrote:
> > Joe <joe <at> jretrading.com> writes:
> >
> > >
> > > On Mon, 30 Jul 2012 01:50:14 +0900
> > > Mark Fletcher <mark27q1 <at> gmail.com> wrote:
> > >
> > > >
> > > > It looks like what got stored on the NAS is not exactly what was
> > > > originally on the host. This is a huge problem for me as it means I
> > > > can't rely on backups dumped on that device. Is there something wrong
> > > > with the way I am mounting the NAS that is leading to this?
> > > >
> > >
> > > Probably. I'd guess it is a matter of permissions. If you create the
> > > archive elsewhere, copy it to the NAS, copy it back again, presumably
> > > there is no difficulty. I also use a Buffalo NAS, but my backups are
> > > created on my server, then copied. It is possible that if the
> > > compression and expansion is done on the NAS, that a temp file involved
> > > may not have the correct permissions to write, or more likely, amend.
> > > But is your backup not running under cron as root?
> > >
> >
> > I put the exact commands I was executing in my original post. There's no job
> > involved, I am typing these commands at the command prompt. I'll bring a job
> > into it once it works reliably. If you read my original post, you can see I
> > create the archive on the host's local disk, test it to make sure it is good,
> > and then copy it to the NAS in a separate operation. I use the cp command to do
> > the copy.
> >
> > I'm inclined to rule out a bug in the cp command, which leaves something about
> > the way the data is being transferred to my NAS. Hence my concern about whether
> > my mount command (again, see details in my original post) was correct.
> >
> > And yes, to answer someone else's question, this is reproducible, reliably,
> > every time. The copy on the NAS is always the same length as the copy on the
> > host's local disk, but diff says they are two different binary files and the one
> > on the NAS cannot be extracted.
> >
>
> A quick way to by-pass the permissions issue is to log in as super-user root,
> and type your commands and tests as root. As I understand it, root is unstoppable.
> That is why it is so dangerous to use it in day-to-day mucking about. A moments
> inattention and real damaged is done. But... as a test, and you are testing,
> use root.

Having posted this, which I thought was reasonable, I went and looked at the
archives to see what OP (Mark Fletcher) had written. It turns out that all
of his investigation was done using commands typed in as root. For me, this
thread is a real puzzle. And very scary.

Mark: How old is your NAS? What brand? Is it likely that it uses Linux-based
open/free software? What vintage? (best guess).

IMHO, something bad is happening as we rush into the future. Layers of software
can cover bugs in basic functionality. Complexity beyond human comprehension.

--
Paul E Condon
pecondon@mesanetworks.net


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120730200934.GB22379@big.lan.gnu">http://lists.debian.org/20120730200934.GB22379@big.lan.gnu

Gary Dale 07-30-2012 08:15 PM

CIFS and data integrity ; Sorry for the noise
 
On 30/07/12 04:09 PM, Paul E Condon wrote:

On 20120730_122543, Paul E Condon wrote:

On 20120730_065640, Mark Fletcher wrote:

Joe<joe<at> jretrading.com> writes:


On Mon, 30 Jul 2012 01:50:14 +0900
Mark Fletcher<mark27q1<at> gmail.com> wrote:


It looks like what got stored on the NAS is not exactly what was
originally on the host. This is a huge problem for me as it means I
can't rely on backups dumped on that device. Is there something wrong
with the way I am mounting the NAS that is leading to this?


Probably. I'd guess it is a matter of permissions. If you create the
archive elsewhere, copy it to the NAS, copy it back again, presumably
there is no difficulty. I also use a Buffalo NAS, but my backups are
created on my server, then copied. It is possible that if the
compression and expansion is done on the NAS, that a temp file involved
may not have the correct permissions to write, or more likely, amend.
But is your backup not running under cron as root?


I put the exact commands I was executing in my original post. There's no job
involved, I am typing these commands at the command prompt. I'll bring a job
into it once it works reliably. If you read my original post, you can see I
create the archive on the host's local disk, test it to make sure it is good,
and then copy it to the NAS in a separate operation. I use the cp command to do
the copy.

I'm inclined to rule out a bug in the cp command, which leaves something about
the way the data is being transferred to my NAS. Hence my concern about whether
my mount command (again, see details in my original post) was correct.

And yes, to answer someone else's question, this is reproducible, reliably,
every time. The copy on the NAS is always the same length as the copy on the
host's local disk, but diff says they are two different binary files and the one
on the NAS cannot be extracted.


A quick way to by-pass the permissions issue is to log in as super-user root,
and type your commands and tests as root. As I understand it, root is unstoppable.
That is why it is so dangerous to use it in day-to-day mucking about. A moments
inattention and real damaged is done. But... as a test, and you are testing,
use root.

Having posted this, which I thought was reasonable, I went and looked at the
archives to see what OP (Mark Fletcher) had written. It turns out that all
of his investigation was done using commands typed in as root. For me, this
thread is a real puzzle. And very scary.

Mark: How old is your NAS? What brand? Is it likely that it uses Linux-based
open/free software? What vintage? (best guess).

IMHO, something bad is happening as we rush into the future. Layers of software
can cover bugs in basic functionality. Complexity beyond human comprehension.
May I suggest that it could be nothing more than a defective network
card or cable?


It may also be the result of some other hardware error. That's why I
suggested doing a file cmp on the files and using rsync. I find exactly
the same problem when backing up to rewritable optical media. You can't
always trust the writes to work properly. Always to a verify and correct
if the files are at all important.



--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 5016EB67.70906@rogers.com">http://lists.debian.org/5016EB67.70906@rogers.com

Mark Fletcher 07-31-2012 01:15 AM

CIFS and data integrity ; Sorry for the noise
 
Paul E Condon <pecondon <at> mesanetworks.net> writes:

>
> Having posted this, which I thought was reasonable, I went and looked at the
> archives to see what OP (Mark Fletcher) had written. It turns out that all
> of his investigation was done using commands typed in as root. For me, this
> thread is a real puzzle. And very scary.
>
> Mark: How old is your NAS? What brand? Is it likely that it uses Linux-based
> open/free software? What vintage? (best guess).
>
> IMHO, something bad is happening as we rush into the future. Layers of
software
> can cover bugs in basic functionality. Complexity beyond human comprehension.
>

Paul -- The NAS is a Buffalo LinkStation 4TB NAS configured to do RAID giving
me 2TB of storage. I bought myself it for Christmas from Amazon.co.jp (I live
in Japan) at Christmas 2010. I don't know what OS it will be running but doubt
it will be Linux since Buffalo will be aiming to maximise Windows
compatibility. I have heard of people wresting their NAS out of the grasp of
the OS it comes with and installing Linux on it, presumably by mucking about
with firmware etc, but I have never attempted to do any such thing as I would
probably screw it up :-) So it is running whatever it was running when it left
the factory.

I'm simultaneously glad that someone else is as unnerved by this happening as
I am, and increasingly nervous that the issue isn't inspiring "oh that old
chestnut" reaction in list readers... :-)

Mark


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: loom.20120731T030407-650@post.gmane.org">http://lists.debian.org/loom.20120731T030407-650@post.gmane.org

Henrique de Moraes Holschuh 07-31-2012 01:27 AM

CIFS and data integrity ; Sorry for the noise
 
On Tue, 31 Jul 2012, Mark Fletcher wrote:
> Paul E Condon <pecondon <at> mesanetworks.net> writes:
> > Having posted this, which I thought was reasonable, I went and looked at the
> > archives to see what OP (Mark Fletcher) had written. It turns out that all
> > of his investigation was done using commands typed in as root. For me, this
> > thread is a real puzzle. And very scary.
> >
> > Mark: How old is your NAS? What brand? Is it likely that it uses Linux-based
> > open/free software? What vintage? (best guess).
> >
> > IMHO, something bad is happening as we rush into the future. Layers of
> software
> > can cover bugs in basic functionality. Complexity beyond human comprehension.
>
> Paul -- The NAS is a Buffalo LinkStation 4TB NAS configured to do RAID giving
> me 2TB of storage. I bought myself it for Christmas from Amazon.co.jp (I live
> in Japan) at Christmas 2010. I don't know what OS it will be running but doubt
> it will be Linux since Buffalo will be aiming to maximise Windows
> compatibility. I have heard of people wresting their NAS out of the grasp of
> the OS it comes with and installing Linux on it, presumably by mucking about
> with firmware etc, but I have never attempted to do any such thing as I would
> probably screw it up :-) So it is running whatever it was running when it left
> the factory.
>
> I'm simultaneously glad that someone else is as unnerved by this happening as
> I am, and increasingly nervous that the issue isn't inspiring "oh that old
> chestnut" reaction in list readers... :-)

Well... there is an awlful lot of CIFS and NFS-related fixes in the kernel
stable queue. Check that. Also make sure it is not your NIC driver or
memory (or the NAS' memory) that went bad...

--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120731012712.GB25874@khazad-dum.debian.net">http://lists.debian.org/20120731012712.GB25874@khazad-dum.debian.net

Mark Fletcher 07-31-2012 02:45 AM

CIFS and data integrity ; Sorry for the noise
 
Henrique de Moraes Holschuh <hmh <at> debian.org> writes:

>
> Well... there is an awlful lot of CIFS and NFS-related fixes in the kernel
> stable queue. Check that. Also make sure it is not your NIC driver or
> memory (or the NAS' memory) that went bad...
>

I wondered about this too -- and the defective cable / NIC idea posted by Gary
earlier. But I would have expected that network card / driver / memory issues
would result in a failed transfer that would either end in tears (error
messages) or would cause part of the transfer to be retried and thus take
longer but eventually end with correct data having been transferred. What's
giving me the heebie-jeebies (if you'll pardon the technial term) about this
is that every software and hardware component seems to think that everything
went swimmingly as far as I can tell, but the data on the target is
nonetheless not correct.

Mark


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: loom.20120731T044059-472@post.gmane.org">http://lists.debian.org/loom.20120731T044059-472@post.gmane.org

Richard Hector 07-31-2012 03:42 AM

CIFS and data integrity ; Sorry for the noise
 
On 31/07/12 13:15, Mark Fletcher wrote:

Paul -- The NAS is a Buffalo LinkStation 4TB NAS configured to do RAID giving
me 2TB of storage. I bought myself it for Christmas from Amazon.co.jp (I live
in Japan) at Christmas 2010. I don't know what OS it will be running but doubt
it will be Linux since Buffalo will be aiming to maximise Windows
compatibility.


http://opensource.buffalo.jp/gpl_storage.html

I think it quite possibly is :-)

Oldish though - 2.4.20 by the looks of it.

Richard


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 5017541A.2020303@walnut.gen.nz">http://lists.debian.org/5017541A.2020303@walnut.gen.nz

Celejar 08-01-2012 02:01 AM

CIFS and data integrity ; Sorry for the noise
 
On Tue, 31 Jul 2012 01:15:28 +0000 (UTC)
Mark Fletcher <mark27q1@gmail.com> wrote:

...

> compatibility. I have heard of people wresting their NAS out of the grasp of
> the OS it comes with and installing Linux on it, presumably by mucking about
> with firmware etc, but I have never attempted to do any such thing as I would
> probably screw it up :-) So it is running whatever it was running when it left
> the factory.

IIUC, NAS hacking is usually done to units that ship with linux, just a
locked down / semi-proprietary version. The hacking enables the unit to
run some more or less standard distribution.

Celejar


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120731220103.0073eedd.celejar@gmail.com">http://lists.debian.org/20120731220103.0073eedd.celejar@gmail.com


All times are GMT. The time now is 11:31 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.