FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Kernel Team

 
 
LinkBack Thread Tools
 
Old 03-10-2010, 07:38 PM
Tim Gardner
 
Default Switch to jiffies for native_sched_clock() when TSC warps

On 03/10/2010 12:30 PM, Chase Douglas wrote:
> Some newer x86 processors (seen on core 2 duo, arrandale) warp their TSC
> registers after a suspend. It is believed that a microcode fix may be a
> solution, but for now we should work around the issue.
>
> This change adds an upper bound on the difference between TSC readings. It
> should be very generous (multiple year difference) so as to only catch TSC
> warping which appears to generate timestamps many years in the future.
> When a warp is found, usage of the TSC for timing is disabled. The
> kernel falls back to using the jiffies counter, which is not as precise
> but should be accurate.
>
> Signed-off-by: Chase Douglas<chase.douglas@canonical.com>
> ---
> arch/x86/kernel/tsc.c | 15 +++++++++++++++
> 1 files changed, 15 insertions(+), 0 deletions(-)
>
> diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
> index 597683a..3e2921d 100644
> --- a/arch/x86/kernel/tsc.c
> +++ b/arch/x86/kernel/tsc.c
> @@ -43,7 +43,9 @@ static int tsc_clocksource_reliable;
> u64 native_sched_clock(void)
> {
> u64 this_offset;
> + static u64 prev_offset;
>
> +jiffies:
> /*
> * Fall back to jiffies if there's no TSC available:
> * ( But note that we still use it if the TSC is marked
> @@ -60,6 +62,19 @@ u64 native_sched_clock(void)
> /* read the Time Stamp Counter: */
> rdtscll(this_offset);
>
> + /*
> + * if new time stamp is many years later, assume warping and disable
> + * TSC usage:
> + */
> + if (__cycles_2_ns(this_offset - prev_offset)> 0x100000000000000
> + && prev_offset) {
> + printk(KERN_WARNING "TSC warped, using jiffies
");
> + tsc_disabled = 1;
> + goto jiffies;
> + }
> +
> + prev_offset = this_offset;
> +
> /* return the value in ns */
> return __cycles_2_ns(this_offset);
> }

I think this is not SMP safe. Surely this function is called by more
then one CPU, therefore prev_offset must be a percpu variable at the
very least.

I am also not in favor of changing runtime behavior. Why not simply
advise the user to boot with 'notsc' using the WARN_ON_ONCE() macro?

Is this the best place to detect warping? It _is_ a fairly high
performance path. How about stashing the TSC just before suspend and
checking it upon return?

rtg
--
Tim Gardner tim.gardner@canonical.com

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-10-2010, 09:37 PM
Stefan Bader
 
Default Switch to jiffies for native_sched_clock() when TSC warps

Chase Douglas wrote:
> I took a look at the x86 code handling the clock to see what could be done
> about the TSC warping coming out of resume on some of the newer processors. The
> code includes a built-in fallback path that uses the jiffies count instead of
> the TSC register if "notsc" is used on the command line. This patch merely sets
> this option at runtime if two TSC time stamps differ by more than 6 years.
>
> I'm sending this here first because I've not touched clocking code before. I'm
> not sure whether this is a feasible approach, and I would like feedback. Note
> that the TSC warping hasn't caused any noticeable issues beyond triggering some
> oops messages, so even if there's some skew in the switch from TSC to jiffies
> it should hopefully not cause too much of an issue.
>
> The only truly negative outcome I foresee is that the clock won't be stable on
> a single CPU. Programs needing accurate clock timing can pin themselves to a
> single CPU in order to get TSC time stamps that are monotonic and accurate (The
> TSC register is per cpu, and there may be skew between CPUs). However, if the
> TSC has warped we are beyond that point anyways. If you have a warping
> processor you should run with notsc if you care about accuracy, even though
> precision would be reduced.
>
>

From my feeling, to change the sched_clock to jiffies after resume sounds not
like a good idea. What was wrong with Colin's approach of just fixing the math?



--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-11-2010, 09:58 AM
Stefan Bader
 
Default Switch to jiffies for native_sched_clock() when TSC warps

Colin Ian King wrote:
> On Wed, 2010-03-10 at 17:56 -0500, Chase Douglas wrote:
>> On Wed, Mar 10, 2010 at 5:37 PM, Stefan Bader
>> <stefan.bader@canonical.com> wrote:
>>> Chase Douglas wrote:
>>>> I took a look at the x86 code handling the clock to see what could be done
>>>> about the TSC warping coming out of resume on some of the newer processors. The
>>>> code includes a built-in fallback path that uses the jiffies count instead of
>>>> the TSC register if "notsc" is used on the command line. This patch merely sets
>>>> this option at runtime if two TSC time stamps differ by more than 6 years.
>>>>
>>>> I'm sending this here first because I've not touched clocking code before. I'm
>>>> not sure whether this is a feasible approach, and I would like feedback. Note
>>>> that the TSC warping hasn't caused any noticeable issues beyond triggering some
>>>> oops messages, so even if there's some skew in the switch from TSC to jiffies
>>>> it should hopefully not cause too much of an issue.
>>>>
>>>> The only truly negative outcome I foresee is that the clock won't be stable on
>>>> a single CPU.
>
> ..and it's hard to determine which CPUs are buggy because they may/may
> not have BIOS loaded or kernel loaded microcode fixes.
>
>> Programs needing accurate clock timing can pin themselves to a
>>>> single CPU in order to get TSC time stamps that are monotonic and accurate (The
>>>> TSC register is per cpu, and there may be skew between CPUs).
>
> ..believe me, if it can skew, it will skew.
>
>> However, if the
>>>> TSC has warped we are beyond that point anyways. If you have a warping
>>>> processor you should run with notsc if you care about accuracy, even though
>>>> precision would be reduced.
>>>>
> ..and "notsc" impacts on low-latency (see later).
>>> From my feeling, to change the sched_clock to jiffies after resume sounds not
>>> like a good idea. What was wrong with Colin's approach of just fixing the math?
>> Colin's patch fixes soft lockup bugs from being fired. That's fixing
>> merely one symptom, but not the real problem.
>
> Well, actually, it's a little more complex than that. Here are some
> extra things to throw into the discussion:
>
> 1) One some processors, the TSC can set the top 32 bits to 0xffffffff
> when coming out of S3. This is a processor issue which may be possible
> to fix on a microcode update (loaded from a new BIOS upgrade) or maybe
> by installing in the intel-microcode package. So maybe, on some systems
> we can advise users to first try the intel-microcode update. If the CPU
> is misbehaving, perhaps that's the first thing to fix.
>
> 2) While poking around I saw that we get spurious warnings from the
> softlockup detection code when the approximated seconds timing tends
> towards 0xffffffff because of a math overflow. This will happen whether
> or not we use the TSC or not. So it's good to have this fixed anyhow,
> even if the bug only happens after thousands of years uptime.
>
> 3) Disabling the use of the TSC impacts on low-latency. For example,
> when doing udelays the default is to use the TSC based delay which
> periodically yields to the scheduler rather than burning up cycles in a
> hard loop. The use of the TSC enables the delay loop to figure out how
> much delay is left after coming back from the scheduler. With the
> non-TSC mode, we burn up cycles and don't yield, so low-latency users
> may/will object to this.
>
>
>> There are other paths
>> that are causing oops messages [1]. Further bugs may be caused by TSC
>> warping that we just haven't seen yet.
>
> Here is an example of this: Doing an slow I/O operation by default uses
> writing to port 0x80 for a small delay. However, the io_delay=udelay
> kernel parameter uses a 2 microsecond udelay(), so if the TSC warps
> forward then we may pop out of the delay prematurely which could be
> problematic.
>
> If we are *really* unlucky, it is hypothetically possible for the TSC
> may warp to 0xffffffffffffffff coming out of S3 and then immediately
> wrap to zero. I believe it may be then possible for a TSC based udelay()
> to get stuck in the delay loop for possibly years/centuries/millennia.

Right so to me the best solution sound like having something similar to the
macros in the clock framework (or use that) to handle wraps in general.
Like time_after or such things.

>> Also, the TSC warping issue seems more prevalent than first thought.
>> Originally, Colin believed the issue was confined to new Arrandale
>> processors, but we're seeing the issue hit Core 2 processors as well
>> [1].
>>
>> [1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/535077
>>
>
>


--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-11-2010, 03:54 PM
Stefan Bader
 
Default Switch to jiffies for native_sched_clock() when TSC warps

Chase Douglas wrote:
> On Thu, Mar 11, 2010 at 5:58 AM, Stefan Bader
> <stefan.bader@canonical.com> wrote:
>> Colin Ian King wrote:
>>> On Wed, 2010-03-10 at 17:56 -0500, Chase Douglas wrote:
>>>> There are other paths
>>>> that are causing oops messages [1]. Further bugs may be caused by TSC
>>>> warping that we just haven't seen yet.
>>> Here is an example of this: Doing an slow I/O operation by default uses
>>> writing to port 0x80 for a small delay. However, the io_delay=udelay
>>> kernel parameter uses a 2 microsecond udelay(), so if the TSC warps
>>> forward then we may pop out of the delay prematurely which could be
>>> problematic.
>>>
>>> If we are *really* unlucky, it is hypothetically possible for the TSC
>>> may warp to 0xffffffffffffffff coming out of S3 and then immediately
>>> wrap to zero. I believe it may be then possible for a TSC based udelay()
>>> to get stuck in the delay loop for possibly years/centuries/millennia.
>> Right so to me the best solution sound like having something similar to the
>> macros in the clock framework (or use that) to handle wraps in general.
>> Like time_after or such things.
>
> So the question I have is: is the absolute value of the TSC relevant,
> or just the relative value. Having proper wrap checking would solve
> some of the issues if we only cared about the relative values.
> However, the output of native_sched_clock is supposed to be an
> absolute number of nanoseconds since system boot. I don't know myself
> whether the absolute value is expected by any calling functions to be
> correct.
>
> Beyond that though, there still may be instances where it is expected
> that the time stamps not jump years into the future. I'd be afraid
> that some protocol stack, like TCP, that depends highly on proper
> timing would go awry in such situations.
>
> I sent a message to linux-kernel last night asking about the
> possibility of switching to the jiffies count at runtime when a TSC
> warp is found [1]. No responses yet though.
>
> -- Chase
>
> [1] http://lkml.org/lkml/2010/3/10/437

What I saw in the code (delay.c)

* Since we calibrate only once at boot, this
* function should be set once at boot and not changed

and in (tsc.c)

* But note that we still use it if the TSC is marked
* unstable. We do this because unlike Time Of Day,
* the scheduler clock tolerates small errors and it's
* very important for it to be as fast as the platform
* can achive it.

makes me feel that changing it should be handled with care.

Stefan


--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 

Thread Tools




All times are GMT. The time now is 05:20 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org