Kernel panic on Intel NUC D34010WYKH and Ubuntu 14.04

[Sundtek]

No one is currently working on it, we can offer that you send your Intel NUC device to Berlin and we can have a look at it (estimate time for the investigation is around 5 days then we would ship it back).
Unfortunately we cannot get all the test hardware, we already have a lot here..
It is a Linux Kernel problem which has to be solved for everyone.

[lsr]

As already mentioned, xhci_hcd is to blame. In my configuration I use two sticks: one old MediaTV Pro and one MediaTV Pro III, connected to an ASRock Vision HT (HM87 chipset, i7-4702MQ) also with Ubuntu 14.04 and Tvheadend, getting the same Kernel Ooops constantly.

The setup worked fine on my old computer with Ubuntu 13.10 and kernel 3.11.0.19, albeit with USB 3.0 controllers from Renesas and Fresco Logic, not Intel.

I have not tried to use the aforementioned 3.11 kernel on 14.04 since I hoped for a fix in 3.15 ("xhci: delete endpoints from bandwidth list before freeing whole device"), but this one only seems to defeat an oops in resume from S4/hibernate. Consequently, installing 3.15-utopic from kernel.ubuntu.com didn't change anything.
May be worth knowing that the (one) dedicated USB 2.0 port of the new ASRock leads to the same result and that xhci-crashes are also triggered by a Seagate Backup Plus Slim with Initio INIC-3609 bridge whereas ASMedia and Fujitsu bridges (all USB 3.0) run fine.

Temporarily the sticks are now connected and net-mounted to my old Seagate Dockstar. It's not as snappy as a direct connection and prone to errors if the network is saturated so I really hope for a solid Linux USB 3.0 implementation soon. 

[Sundtek]

Can you update the driver and check /opt/bin/mediaclient --cpucheck and maybe change the CPU scheduler to performance while using the USB devices?

We figured out some systems have issues with the CPU Scheduling Governor and USB.

[lsr]

Thanks, testing now.
By the way, last line of my dmesg output:

Code: [Auswählen]

usb 3-6.4: usbfs: process 2120 (mediasrv) did not claim interface 0 before use

[Sundtek]

That does not matter, we do that on purpose for some reason.

[lsr]

That didn't last long... [dmesg truncated: "Beitrag hat die max. Länge erreicht (20000 Zeichen)"]

Code: [Auswählen]

#> mediaclient --cpucheck
Sundtek CPU Check

cpu0:
   current scheduler: performance
   available schedulers: performance powersave
   to change the scheduler you might run following command:

   echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

cpu1:
   current scheduler: performance
   available schedulers: performance powersave
   to change the scheduler you might run following command:

   echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor

cpu2:
   current scheduler: performance
   available schedulers: performance powersave
   to change the scheduler you might run following command:

   echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor

cpu3:
   current scheduler: performance
   available schedulers: performance powersave
   to change the scheduler you might run following command:

   echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor

cpu4:
   current scheduler: performance
   available schedulers: performance powersave
   to change the scheduler you might run following command:

   echo performance > /sys/devices/system/cpu/cpu4/cpufreq/scaling_governor

cpu5:
   current scheduler: performance
   available schedulers: performance powersave
   to change the scheduler you might run following command:

   echo performance > /sys/devices/system/cpu/cpu5/cpufreq/scaling_governor

cpu6:
   current scheduler: performance
   available schedulers: performance powersave
   to change the scheduler you might run following command:

   echo performance > /sys/devices/system/cpu/cpu6/cpufreq/scaling_governor

cpu7:
   current scheduler: performance
   available schedulers: performance powersave
   to change the scheduler you might run following command:

   echo performance > /sys/devices/system/cpu/cpu7/cpufreq/scaling_governor

Code: [Auswählen]

[ 1072.649867] ------------[ cut here ]------------
[ 1072.649873] WARNING: CPU: 5 PID: 4129 at /home/apw/COD/linux/drivers/usb/host/xhci-ring.c:605 xhci_find_new_dequeue_state+0x15d/0x290()
[ 1072.649874] Modules linked in: videodev arc4 md4 snd_hda_codec_hdmi snd_hda_codec_realtek x86_pkg_temp_thermal coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device ir_lirc_codec lirc_dev snd_timer ir_mce_kbd_decoder lib80211_crypt_tkip ir_sanyo_decoder ir_sony_decoder ir_jvc_decoder microcode ir_rc6_decoder ir_rc5_decoder psmouse ir_nec_decoder wl(POF) serio_raw rfcomm bnep snd i915 bluetooth rtsx_pci_ms lib80211 joydev drm_kms_helper rc_rc6_mce cfg80211 memstick mei_me nuvoton_cir lpc_ich soundcore rc_core drm video mei mac_hid i2c_algo_bit intel_smartconnect nls_utf8 parport_pc cifs ppdev fscache lp parport ext2 xfs libcrc32c rtsx_pci_sdmmc hid_generic e1000e usb_storage ahci libahci ptp rtsx_pci pps_core usbhid hid
[ 1072.649915] CPU: 5 PID: 4129 Comm: mediasrv Tainted: PF          O 3.12.20-031220-generic #201405160935
[ 1072.649916] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./HM87-HT, BIOS P1.50 01/06/2014
[ 1072.649917]  000000000000025d ffff8803d4401be8 ffffffff81745261 0000000000000007
[ 1072.649919]  0000000000000000 ffff8803d4401c28 ffffffff8106784c ffffffff8158a8c0
[ 1072.649921]  ffff880404348000 ffff8803d4401cb0 ffff8803df5e7400 00000000c739b000
[ 1072.649923] Call Trace:
[ 1072.649927]  [<ffffffff81745261>] dump_stack+0x46/0x58
[ 1072.649930]  [<ffffffff8106784c>] warn_slowpath_common+0x8c/0xc0
[ 1072.649933]  [<ffffffff8158a8c0>] ? trace_xhci_dbg_quirks+0x70/0x70
[ 1072.649935]  [<ffffffff8106789a>] warn_slowpath_null+0x1a/0x20
[ 1072.649937]  [<ffffffff8159875d>] xhci_find_new_dequeue_state+0x15d/0x290
[ 1072.649939]  [<ffffffff8158e978>] xhci_cleanup_stalled_ring+0x78/0x100
[ 1072.649941]  [<ffffffff8158eb36>] xhci_endpoint_reset+0x136/0x190
[ 1072.649943]  [<ffffffff81561f8e>] ? usb_alloc_urb+0x1e/0x50
[ 1072.649945]  [<ffffffff81561675>] usb_hcd_reset_endpoint+0x25/0x70
[ 1072.649947]  [<ffffffff815646a8>] usb_enable_endpoint+0xa8/0xb0
[ 1072.649949]  [<ffffffff815646f2>] usb_enable_interface+0x42/0x60
[ 1072.649951]  [<ffffffff81564be6>] usb_set_interface+0x1f6/0x340
[ 1072.649953]  [<ffffffff8156ee79>] usbdev_do_ioctl+0x589/0xc50
[ 1072.649956]  [<ffffffff8111323c>] ? acct_account_cputime+0x1c/0x20
[ 1072.649958]  [<ffffffff810a0c89>] ? account_user_time+0x99/0xb0
[ 1072.649960]  [<ffffffff8156f56e>] usbdev_ioctl+0xe/0x20
[ 1072.649962]  [<ffffffff811d3cda>] do_vfs_ioctl+0x7a/0x2e0
[ 1072.649964]  [<ffffffff810227f5>] ? syscall_trace_enter+0x165/0x280
[ 1072.649966]  [<ffffffff811d3fd1>] SyS_ioctl+0x91/0xb0
[ 1072.649968]  [<ffffffff8175abbf>] tracesys+0xe1/0xe6
[ 1072.649969] ---[ end trace 2c231d95ff701c85 ]---
[ 1072.649976] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
[ 1072.650718] IP: [<ffffffff815988fc>] xhci_queue_new_dequeue_state+0x6c/0xe0
[ 1072.651389] PGD 40739b067 PUD 4051c0067 PMD 0
[ 1072.651825] Oops: 0000 [#1] SMP
[ 1072.652138] Modules linked in: videodev arc4 md4 snd_hda_codec_hdmi snd_hda_codec_realtek x86_pkg_temp_thermal coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device ir_lirc_codec lirc_dev snd_timer ir_mce_kbd_decoder lib80211_crypt_tkip ir_sanyo_decoder ir_sony_decoder ir_jvc_decoder microcode ir_rc6_decoder ir_rc5_decoder psmouse ir_nec_decoder wl(POF) serio_raw rfcomm bnep snd i915 bluetooth rtsx_pci_ms lib80211 joydev drm_kms_helper rc_rc6_mce cfg80211 memstick mei_me nuvoton_cir lpc_ich soundcore rc_core drm video mei mac_hid i2c_algo_bit intel_smartconnect nls_utf8 parport_pc cifs ppdev fscache lp parport ext2 xfs libcrc32c rtsx_pci_sdmmc hid_generic e1000e usb_storage ahci libahci ptp rtsx_pci pps_core usbhid hid
[ 1072.660691] CPU: 5 PID: 4129 Comm: mediasrv Tainted: PF       W  O 3.12.20-031220-generic #201405160935
[ 1072.661559] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./HM87-HT, BIOS P1.50 01/06/2014
[ 1072.662449] task: ffff8803dad90000 ti: ffff8803d4400000 task.ti: ffff8803d4400000
[ 1072.663179] RIP: 0010:[<ffffffff815988fc>]  [<ffffffff815988fc>] xhci_queue_new_dequeue_state+0x6c/0xe0
[ 1072.664059] RSP: 0018:ffff8803d4401c38  EFLAGS: 00010046
[ 1072.664566] RAX: 0000000000000000 RBX: ffff8803d4401cb0 RCX: 0000000000000040
[ 1072.665243] RDX: 0000000000000000 RSI: ffff8803e0d34170 RDI: 0000000000000000
[ 1072.665904] RBP: ffff8803d4401c98 R08: ffff8803d4401cb0 R09: ffff8803e0d34170
[ 1072.666580] R10: 0000000000000006 R11: 000000000000047e R12: ffff880404348000
[ 1072.667259] R13: ffff880403cd01e8 R14: 0000000000000000 R15: 0000000000000002
[ 1072.667918] FS:  00007f9a82a70700(0000) GS:ffff88041e340000(0000) knlGS:0000000000000000
[ 1072.668685] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1072.669213] CR2: 0000000000000010 CR3: 0000000406d6e000 CR4: 00000000001407e0
[ 1072.669871] Stack:
[ 1072.670051]  ffff8803d4401ca8 ffff8803d4401c58 ffffffff81acb1c9 ffff8803e0d34170
[ 1072.670787]  ffff880300000006 00000000c739b001 ffff880404348000 ffff880403cd01e8
[ 1072.671525]  ffff880404348000 ffff880408767000 0000000000000002 ffff880404348048
[ 1072.672244] Call Trace:
[ 1072.672468]  [<ffffffff8158e9f9>] xhci_cleanup_stalled_ring+0xf9/0x100
[ 1072.673083]  [<ffffffff8158eb36>] xhci_endpoint_reset+0x136/0x190
[ 1072.673656]  [<ffffffff81561f8e>] ? usb_alloc_urb+0x1e/0x50
[ 1072.674171]  [<ffffffff81561675>] usb_hcd_reset_endpoint+0x25/0x70
[ 1072.674761]  [<ffffffff815646a8>] usb_enable_endpoint+0xa8/0xb0
[ 1072.675326]  [<ffffffff815646f2>] usb_enable_interface+0x42/0x60
[ 1072.675882]  [<ffffffff81564be6>] usb_set_interface+0x1f6/0x340
[ 1072.676427]  [<ffffffff8156ee79>] usbdev_do_ioctl+0x589/0xc50
[ 1072.676959]  [<ffffffff8111323c>] ? acct_account_cputime+0x1c/0x20
[ 1072.677532]  [<ffffffff810a0c89>] ? account_user_time+0x99/0xb0
[ 1072.678078]  [<ffffffff8156f56e>] usbdev_ioctl+0xe/0x20
[ 1072.678580]  [<ffffffff811d3cda>] do_vfs_ioctl+0x7a/0x2e0
[ 1072.679097]  [<ffffffff810227f5>] ? syscall_trace_enter+0x165/0x280
[ 1072.679676]  [<ffffffff811d3fd1>] SyS_ioctl+0x91/0xb0
[ 1072.680141]  [<ffffffff8175abbf>] tracesys+0xe1/0xe6
[ 1072.680596] Code: 03 84 d7 10 01 00 00 4d 8b 48 08 4c 89 c3 49 89 fc 44 89 55 c0 4c 89 f7 4c 89 ce 4c 8d 68 28 4c 89 4d b8 e8 77 ec ff ff 8b 53 10 <4d> 8b 46 10 4c 89 f1 4c 8b 4d b8 48 89 04 24 4c 89 e7 48 c7 c6
[ 1072.682934] RIP  [<ffffffff815988fc>] xhci_queue_new_dequeue_state+0x6c/0xe0
[ 1072.683615]  RSP <ffff8803d4401c38>
[ 1072.683934] CR2: 0000000000000010
[ 1072.747120] HDMI: ELD buf size is 0, force 128
[ 1072.747614] HDMI: invalid ELD data byte 0
[ 1072.748063] HDMI: ELD buf size is 0, force 128
[ 1072.748543] HDMI: invalid ELD data byte 0
[ 1073.221675] ---[ end trace 2c231d95ff701c86 ]---
[ 1093.400986] ------------[ cut here ]------------
[ 1093.400994] WARNING: CPU: 2 PID: 2186 at /home/apw/COD/linux/kernel/watchdog.c:245 watchdog_overflow_callback+0x9a/0xc0()
[ 1093.400996] Watchdog detected hard LOCKUP on cpu 2
[ 1093.400997] Modules linked in: videodev arc4 md4 snd_hda_codec_hdmi snd_hda_codec_realtek x86_pkg_temp_thermal coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device ir_lirc_codec lirc_dev snd_timer ir_mce_kbd_decoder lib80211_crypt_tkip ir_sanyo_decoder ir_sony_decoder ir_jvc_decoder microcode ir_rc6_decoder ir_rc5_decoder psmouse ir_nec_decoder wl(POF) serio_raw rfcomm bnep snd i915 bluetooth rtsx_pci_ms lib80211 joydev drm_kms_helper rc_rc6_mce cfg80211 memstick mei_me nuvoton_cir lpc_ich soundcore rc_core drm video mei mac_hid i2c_algo_bit intel_smartconnect nls_utf8 parport_pc cifs ppdev fscache lp parport ext2 xfs libcrc32c rtsx_pci_sdmmc hid_generic e1000e usb_storage ahci libahci ptp rtsx_pci pps_core usbhid hid
[ 1093.401056] CPU: 2 PID: 2186 Comm: mediasrv Tainted: PF     D W  O 3.12.20-031220-generic #201405160935
[ 1093.401057] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./HM87-HT, BIOS P1.50 01/06/2014
[ 1093.401059]  00000000000000f5 ffff88041e287ba8 ffffffff81745261 0000000000000007
[ 1093.401062]  ffff88041e287bf8 ffff88041e287be8 ffffffff8106784c 0000000000000000
[ 1093.401065]  ffff8804081f8000 0000000000000000 ffff88041e287d20 0000000000000000
[ 1093.401068] Call Trace:
[ 1093.401069]  <NMI>  [<ffffffff81745261>] dump_stack+0x46/0x58
[ 1093.401078]  [<ffffffff8106784c>] warn_slowpath_common+0x8c/0xc0
[ 1093.401081]  [<ffffffff81067936>] warn_slowpath_fmt+0x46/0x50
[ 1093.401084]  [<ffffffff8110a27a>] watchdog_overflow_callback+0x9a/0xc0
[ 1093.401088]  [<ffffffff81148d4c>] __perf_event_overflow+0x9c/0x230
[ 1093.401092]  [<ffffffff8102ae88>] ? x86_perf_event_set_period+0xd8/0x150
[ 1093.401096]  [<ffffffff81149654>] perf_event_overflow+0x14/0x20
[ 1093.401099]  [<ffffffff810326ae>] intel_pmu_handle_irq+0x1ae/0x2a0
[ 1093.401103]  [<ffffffff81753874>] perf_event_nmi_handler+0x34/0x60
[ 1093.401105]  [<ffffffff81752fca>] nmi_handle.isra.3+0x8a/0x1a0
[ 1093.401107]  [<ffffffff817538f0>] ? perf_ibs_nmi_handler+0x50/0x50
[ 1093.401110]  [<ffffffff81753259>] default_do_nmi+0xe9/0x240
[ 1093.401112]  [<ffffffff81753440>] do_nmi+0x90/0xd0
[ 1093.401115]  [<ffffffff817523c1>] end_repeat_nmi+0x1e/0x2e
[ 1093.401119]  [<ffffffff8175174d>] ? _raw_spin_lock_irqsave+0x3d/0x60
[ 1093.401122]  [<ffffffff8175174d>] ? _raw_spin_lock_irqsave+0x3d/0x60
[ 1093.401125]  [<ffffffff8175174d>] ? _raw_spin_lock_irqsave+0x3d/0x60
[ 1093.401126]  <<EOE>>  [<ffffffff8158e2cd>] xhci_urb_enqueue+0x14d/0x430
[ 1093.401133]  [<ffffffff81560ea8>] usb_hcd_submit_urb+0x88/0x1b0
[ 1093.401136]  [<ffffffff815622af>] usb_submit_urb.part.6+0x15f/0x430
[ 1093.401139]  [<ffffffff815625b5>] usb_submit_urb+0x35/0x80
[ 1093.401141]  [<ffffffff815634ca>] usb_start_wait_urb+0x6a/0x170
[ 1093.401144]  [<ffffffff81561f8e>] ? usb_alloc_urb+0x1e/0x50
[ 1093.401146]  [<ffffffff81561f8e>] ? usb_alloc_urb+0x1e/0x50
[ 1093.401149]  [<ffffffff8156385a>] usb_control_msg+0xda/0x110
[ 1093.401152]  [<ffffffff8156d22c>] proc_control+0x2bc/0x3a0
[ 1093.401155]  [<ffffffff810901e0>] ? lock_hrtimer_base.isra.23+0x30/0x60
[ 1093.401157]  [<ffffffff8156f137>] usbdev_do_ioctl+0x847/0xc50
[ 1093.401160]  [<ffffffff8111323c>] ? acct_account_cputime+0x1c/0x20
[ 1093.401163]  [<ffffffff810a0c89>] ? account_user_time+0x99/0xb0
[ 1093.401165]  [<ffffffff8156f56e>] usbdev_ioctl+0xe/0x20
[ 1093.401168]  [<ffffffff811d3cda>] do_vfs_ioctl+0x7a/0x2e0
[ 1093.401171]  [<ffffffff810227f5>] ? syscall_trace_enter+0x165/0x280
[ 1093.401174]  [<ffffffff811d3fd1>] SyS_ioctl+0x91/0xb0
[ 1093.401177]  [<ffffffff8175abbf>] tracesys+0xe1/0xe6
[ 1093.401178] ---[ end trace 2c231d95ff701c87 ]---

[Sundtek]

can you contact us via Skype-chat? Our Account is "sundtek"

[lsr]

Small update: quite a few changes * (http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.16-rc1-utopic/CHANGES) regarding xhci in Ubuntu Kernel 3.16-rc1, but the crash/oops unfortunately persists.

*

Mathias Nyman (4):
      xhci: Use command structures when queuing commands on the command ring
      xhci: Add a global command queue
      xhci: Use completion and status in global command queue
      xhci: rework command timeout and cancellation,
[etc.]

[Yoshi]

Is there any news for this issue? Bug is present very long 

[Sundtek]

Did you update your Kernel Version?

This is not our bug -- this bug is a linux bug depending on this particular usb controller used with this Intel NUC.

So unfortunately there's nothing we can do in our driver to avoid such a problem if the upper layer hardware's driver which is shipped with Linux is faulty.

All issues created by our driver can be fixed by us of course (and we really prefer to fix our problems by our-self because it's the shortest way), but the USB Controller is really out of our hands.

[Yoshi]

I'm aware this is not caused by your driver. Just asking if anyone knows if there's any progress, that's all.

No update for the kernel version. I'm on CentOS 7.

[Sundtek]

The Bios Updates are bleeding edge you really need one of the latest USB Controller Kernel drivers for this.

I see on the Intel Website they provide a separate USB 3.0 driver for Windows, so no wonder that things have to work in a different way with Linux.

Aside of that those Systems seem to have a genuine USB 2.0 Bus on the Minicard interface, there are MiniPCIe to USB Adapters available which might be able to route a genuine USB 2.0 port.