*Re: Kernel 4.17.4 lockup
2018-07-10 2:14 ` H.J. Lu@ 2018-07-10 3:47 ` Dave Hansen
[not found] ` <CAMe9rOrHowBX06nihdRRmEqhV8v7cs+PwVY7JYQFpUFOnHC71A@mail.gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Dave Hansen @ 2018-07-10 3:47 UTC (permalink / raw)
To: H.J. Lu
Cc: H. Peter Anvin, LKML, Andy Lutomirski, Mel Gorman, Andrew Morton,
Rik van Riel, Minchan Kim
On 07/09/2018 07:14 PM, H.J. Lu wrote:
>> I'd really want to see this reproduced without KASLR to make the oops
>> easier to read. It would also be handy to try your workload with all
>> the pedantic debugging: KASAN, slab debugging, DEBUG_PAGE_ALLOC, etc...
>> and see if it still triggers.
> How can I turn them on at boot time?
The only thing you can add at boot time is slab debugging, and it's
probably the most useless of the three that I listed since you're not
actually seeing any slab corruption.
The rest are compile-time options.
^permalinkrawreply [flat|nested] 25+ messages in thread

*Re: Kernel 4.17.4 lockup
[not found] ` <CAMe9rOrHowBX06nihdRRmEqhV8v7cs+PwVY7JYQFpUFOnHC71A@mail.gmail.com>
@ 2018-07-11 15:13 ` Dave Hansen
2018-07-11 15:40 ` H.J. Lu0 siblings, 1 reply; 25+ messages in thread
From: Dave Hansen @ 2018-07-11 15:13 UTC (permalink / raw)
To: H.J. Lu
Cc: H. Peter Anvin, LKML, Andy Lutomirski, Mel Gorman, Andrew Morton,
Rik van Riel, Minchan Kim
On 07/11/2018 07:56 AM, H.J. Lu wrote:
> On Mon, Jul 9, 2018 at 8:47 PM, Dave Hansen <dave.hansen@intel.com> wrote:
>> On 07/09/2018 07:14 PM, H.J. Lu wrote:
>>>> I'd really want to see this reproduced without KASLR to make the oops
>>>> easier to read. It would also be handy to try your workload with all
>>>> the pedantic debugging: KASAN, slab debugging, DEBUG_PAGE_ALLOC, etc...
>>>> and see if it still triggers.
>>> How can I turn them on at boot time?
>> The only thing you can add at boot time is slab debugging, and it's
>> probably the most useless of the three that I listed since you're not
>> actually seeing any slab corruption.
>>
>> The rest are compile-time options.
> I enabled KASAN, slab debugging, DEBUG_PAGE_ALLOC and disabled
> KASLR. Machine locked up. Here is the last kernel message before locking
> up.
KASAN looks to have caught it, although it scrolled off the screen. I
can certainly imagine the oops you saw earlier being caused by stack
corruption.
Sounds like we need to reproduce this in an environment that can
actually capture a real oops. Can you share more about your workload?
I'll see if I can get it to reproduce in a VM.
^permalinkrawreply [flat|nested] 25+ messages in thread

*Re: Kernel 4.17.4 lockup
2018-07-11 15:13 ` Dave Hansen@ 2018-07-11 15:40 ` H.J. Lu
2018-07-11 16:24 ` Dave Hansen0 siblings, 1 reply; 25+ messages in thread
From: H.J. Lu @ 2018-07-11 15:40 UTC (permalink / raw)
To: Dave Hansen
Cc: H. Peter Anvin, LKML, Andy Lutomirski, Mel Gorman, Andrew Morton,
Rik van Riel, Minchan Kim
On Wed, Jul 11, 2018 at 8:13 AM, Dave Hansen <dave.hansen@intel.com> wrote:
> On 07/11/2018 07:56 AM, H.J. Lu wrote:
>> On Mon, Jul 9, 2018 at 8:47 PM, Dave Hansen <dave.hansen@intel.com> wrote:
>>> On 07/09/2018 07:14 PM, H.J. Lu wrote:
>>>>> I'd really want to see this reproduced without KASLR to make the oops
>>>>> easier to read. It would also be handy to try your workload with all
>>>>> the pedantic debugging: KASAN, slab debugging, DEBUG_PAGE_ALLOC, etc...
>>>>> and see if it still triggers.
>>>> How can I turn them on at boot time?
>>> The only thing you can add at boot time is slab debugging, and it's
>>> probably the most useless of the three that I listed since you're not
>>> actually seeing any slab corruption.
>>>
>>> The rest are compile-time options.
>> I enabled KASAN, slab debugging, DEBUG_PAGE_ALLOC and disabled
>> KASLR. Machine locked up. Here is the last kernel message before locking
>> up.
>
> KASAN looks to have caught it, although it scrolled off the screen. I
> can certainly imagine the oops you saw earlier being caused by stack
> corruption.
>
> Sounds like we need to reproduce this in an environment that can
> actually capture a real oops. Can you share more about your workload?
> I'll see if I can get it to reproduce in a VM.
This is a quad-core machine with HT and 6 GB RAM. The workload is
x32 GCC build and test with "make -j8". The bug is triggered during GCC
test after a couple hours. I have a script to set up my workload:
https://github.com/hjl-tools/gcc-regression
with a cron job
# It takes about 3 hour to bootstrap x86-64 GCC and 3 hour to run tests,
TIMEOUT=480
# Run it every hour,
30 * * * * /export/gnu/import/git/gcc-test-x32/gcc-build -mx32
--with-pic > /dev/null 2>&1
--
H.J.
^permalinkrawreply [flat|nested] 25+ messages in thread