FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.

» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Kernel Team

LinkBack Thread Tools
Old 03-10-2010, 08:04 PM
Chase Douglas
Default Switch to jiffies for native_sched_clock() when TSC warps

On Wed, Mar 10, 2010 at 3:38 PM, Tim Gardner <tim.gardner@canonical.com> wrote:
> On 03/10/2010 12:30 PM, Chase Douglas wrote:
>> Some newer x86 processors (seen on core 2 duo, arrandale) warp their TSC
>> registers after a suspend. It is believed that a microcode fix may be a
>> solution, but for now we should work around the issue.
>> This change adds an upper bound on the difference between TSC readings. It
>> should be very generous (multiple year difference) so as to only catch TSC
>> warping which appears to generate timestamps many years in the future.
>> When a warp is found, usage of the TSC for timing is disabled. The
>> kernel falls back to using the jiffies counter, which is not as precise
>> but should be accurate.
>> Signed-off-by: Chase Douglas<chase.douglas@canonical.com>
>> ---
>> *arch/x86/kernel/tsc.c | * 15 +++++++++++++++
>> *1 files changed, 15 insertions(+), 0 deletions(-)
>> diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
>> index 597683a..3e2921d 100644
>> --- a/arch/x86/kernel/tsc.c
>> +++ b/arch/x86/kernel/tsc.c
>> @@ -43,7 +43,9 @@ static int tsc_clocksource_reliable;
>> *u64 native_sched_clock(void)
>> *{
>> * * * *u64 this_offset;
>> + * * * static u64 prev_offset;
>> +jiffies:
>> * * * */*
>> * * * * * Fall back to jiffies if there's no TSC available:
>> * * * * * ( But note that we still use it if the TSC is marked
>> @@ -60,6 +62,19 @@ u64 native_sched_clock(void)
>> * * * */* read the Time Stamp Counter: */
>> * * * *rdtscll(this_offset);
>> + * * * /*
>> + * * * ** if new time stamp is many years later, assume warping and
>> disable
>> + * * * ** TSC usage:
>> + * * * **/
>> + * * * if (__cycles_2_ns(this_offset - prev_offset)> *0x100000000000000
>> + * * * && *prev_offset) {
>> + * * * * * * * printk(KERN_WARNING "TSC warped, using jiffies
>> + * * * * * * * tsc_disabled = 1;
>> + * * * * * * * goto jiffies;
>> + * * * }
>> +
>> + * * * prev_offset = this_offset;
>> +
>> * * * */* return the value in ns */
>> * * * *return __cycles_2_ns(this_offset);
>> *}
> I think this is not SMP safe. Surely this function is called by more then
> one CPU, therefore prev_offset must be a percpu variable at the very least.

Agreed. If it were implemented this way a percpu var would be needed.

> I am also not in favor of changing runtime behavior. Why not simply advise
> the user to boot with 'notsc' using the WARN_ON_ONCE() macro?

This is possible, but is it the best approach? I'm thinking that in
the majority of cases people aren't going to care that their timer is
lower resolution after a resume. The warning message during the switch
could mention that notsc should be used from now on. Also, thinking of
Ubuntu, using a WARN_ON_ONCE macro will create an apport message,
which would likely scare a user since it mentions that your machine
may be unsafe from now on blah blah, when in reality you should be
safe if you make it past that point.

> Is this the best place to detect warping? It _is_ a fairly high performance
> path. How about stashing the TSC just before suspend and checking it upon
> return?

I was going for a more general approach in case there are other times
where the TSC can warp. But yes, given the nature of this path it
probably makes sense to err on the safe side and check only where we
know there may be a warp until we see evidence otherwise. I'll see
what can be done for checking the TSC before and after a resume.


kernel-team mailing list
Old 03-10-2010, 09:56 PM
Chase Douglas
Default Switch to jiffies for native_sched_clock() when TSC warps

On Wed, Mar 10, 2010 at 5:37 PM, Stefan Bader
<stefan.bader@canonical.com> wrote:
> Chase Douglas wrote:
>> I took a look at the x86 code handling the clock to see what could be done
>> about the TSC warping coming out of resume on some of the newer processors. The
>> code includes a built-in fallback path that uses the jiffies count instead of
>> the TSC register if "notsc" is used on the command line. This patch merely sets
>> this option at runtime if two TSC time stamps differ by more than 6 years.
>> I'm sending this here first because I've not touched clocking code before. I'm
>> not sure whether this is a feasible approach, and I would like feedback. Note
>> that the TSC warping hasn't caused any noticeable issues beyond triggering some
>> oops messages, so even if there's some skew in the switch from TSC to jiffies
>> it should hopefully not cause too much of an issue.
>> The only truly negative outcome I foresee is that the clock won't be stable on
>> a single CPU. Programs needing accurate clock timing can pin themselves to a
>> single CPU in order to get TSC time stamps that are monotonic and accurate (The
>> TSC register is per cpu, and there may be skew between CPUs). However, if the
>> TSC has warped we are beyond that point anyways. If you have a warping
>> processor you should run with notsc if you care about accuracy, even though
>> precision would be reduced.
> From my feeling, to change the sched_clock to jiffies after resume sounds not
> like a good idea. What was wrong with Colin's approach of just fixing the math?

Colin's patch fixes soft lockup bugs from being fired. That's fixing
merely one symptom, but not the real problem. There are other paths
that are causing oops messages [1]. Further bugs may be caused by TSC
warping that we just haven't seen yet.

Also, the TSC warping issue seems more prevalent than first thought.
Originally, Colin believed the issue was confined to new Arrandale
processors, but we're seeing the issue hit Core 2 processors as well

[1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/535077

kernel-team mailing list
Old 03-11-2010, 01:27 PM
Chase Douglas
Default Switch to jiffies for native_sched_clock() when TSC warps

On Thu, Mar 11, 2010 at 5:58 AM, Stefan Bader
<stefan.bader@canonical.com> wrote:
> Colin Ian King wrote:
>> On Wed, 2010-03-10 at 17:56 -0500, Chase Douglas wrote:
>>> There are other paths
>>> that are causing oops messages [1]. Further bugs may be caused by TSC
>>> warping that we just haven't seen yet.
>> Here is an example of this: Doing an slow I/O operation by default uses
>> writing to port 0x80 for a small delay. However, the io_delay=udelay
>> kernel parameter uses a 2 microsecond udelay(), so if the TSC warps
>> forward then we may pop out of the delay prematurely which could be
>> problematic.
>> If we are *really* unlucky, it is hypothetically possible for the TSC
>> may warp to 0xffffffffffffffff coming out of S3 and then immediately
>> wrap to zero. I believe it may be then possible for a TSC based udelay()
>> to get stuck in the delay loop for possibly years/centuries/millennia.
> Right so to me the best solution sound like having something similar to the
> macros in the clock framework (or use that) to handle wraps in general.
> Like time_after or such things.

So the question I have is: is the absolute value of the TSC relevant,
or just the relative value. Having proper wrap checking would solve
some of the issues if we only cared about the relative values.
However, the output of native_sched_clock is supposed to be an
absolute number of nanoseconds since system boot. I don't know myself
whether the absolute value is expected by any calling functions to be

Beyond that though, there still may be instances where it is expected
that the time stamps not jump years into the future. I'd be afraid
that some protocol stack, like TCP, that depends highly on proper
timing would go awry in such situations.

I sent a message to linux-kernel last night asking about the
possibility of switching to the jiffies count at runtime when a TSC
warp is found [1]. No responses yet though.

-- Chase

[1] http://lkml.org/lkml/2010/3/10/437

kernel-team mailing list

Thread Tools

All times are GMT. The time now is 12:27 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org