TITLE: BUG: soft lockup in ip_list_rcv
ALT: stall in ip_list_rcv
TYPE: HANG
PANICKED: Y

[  637.311457][    C0] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ksoftirqd/0:9]
[  637.387608][    C0] Modules linked in:
[  637.445248][    C0] irq event stamp: 83253
[  637.461076][    C0] hardirqs last  enabled at (83252): [<ffffa0001768a558>] _raw_spin_unlock_irqrestore+0xd8/0x110
[  637.462474][    C0] hardirqs last disabled at (83253): [<ffffa00010004ac0>] el1_irq+0x80/0x180
[  637.463788][    C0] softirqs last  enabled at (79846): [<ffffa000100029f4>] _stext+0x9f4/0x1098
[  637.465040][    C0] softirqs last disabled at (79849): [<ffffa0001018d560>] irq_exit+0x470/0x540
[  637.466599][    C0] CPU: 0 PID: 9 Comm: ksoftirqd/0 Not tainted 5.9.0-12994-gf9893351acae #15
[  637.467605][    C0] Hardware name: linux,dummy-virt (DT)
[  637.469106][    C0] pstate: 10000005 (nzcV daif -PAN -UAO -TCO BTYPE=--)
[  637.470521][    C0] pc : _raw_spin_unlock_irqrestore+0x74/0x110
[  637.471589][    C0] lr : _raw_spin_unlock_irqrestore+0xd8/0x110
[  637.472533][    C0] sp : ffff00006a238e50
[  637.473461][    C0] x29: ffff00006a238e50 x28: ffff000047654508 
[  637.475049][    C0] x27: 0000000000000002 x26: dfffa00000000000 
[  637.476402][    C0] x25: 0000000000000002 x24: ffffa0001bf00be0 
[  637.477749][    C0] x23: ffffa0001c0187d0 x22: ffffa0001b2f2000 
[  637.479178][    C0] x21: ffffa0001c0187b8 x20: 0000000000000000 
[  637.480586][    C0] x19: ffffa000127d3e1c x18: ffff00006a247448 
[  637.481959][    C0] x17: 0000000000000000 x16: 0000000000000000 
[  637.483293][    C0] x15: 0000000000000001 x14: ffff000040248000 
[  637.484603][    C0] x13: 0000000000000001 x12: ffff80000d4471b1 
[  637.485880][    C0] x11: 1fffe0000d4471b0 x10: ffff80000d4471b0 
[  637.487295][    C0] x9 : dfffa00000000000 x8 : 00000000f3000000 
[  637.488520][    C0] x7 : 00000000f3f3f3f3 x6 : dfffa00000000000 
[  637.490022][    C0] x5 : ffff000040248000 x4 : 0000000000000000 
[  637.491310][    C0] x3 : ffffa0001766a154 x2 : 0000000000000001 
[  637.492655][    C0] x1 : ffff000040248000 x0 : 0000000000014534 
[  637.494411][    C0] Call trace:
[  637.495455][    C0]  _raw_spin_unlock_irqrestore+0x74/0x110
[  637.496383][    C0]  debug_check_no_obj_freed+0x234/0x438
[  637.497375][    C0]  slab_free_freelist_hook+0x174/0x1f8
[  637.498341][    C0]  kmem_cache_free+0xac/0x4f8
[  637.499331][    C0]  kfree_skbmem+0x1b8/0x1f8
[  637.500330][    C0]  consume_skb+0x13c/0x680
[  637.501265][    C0]  __dev_kfree_skb_any+0xb0/0xd0
[  637.502164][    C0]  napi_consume_skb+0x5f0/0x850
[  637.503150][    C0]  free_old_xmit_skbs+0xf8/0x258
[  637.504026][    C0]  start_xmit+0x138/0x1530
[  637.504857][    C0]  dev_hard_start_xmit+0x204/0xde0
[  637.505756][    C0]  sch_direct_xmit+0x278/0x488
[  637.506639][    C0]  __qdisc_run+0x44c/0x1da0
[  637.507482][    C0]  __dev_queue_xmit+0x2164/0x30d0
[  637.508333][    C0]  dev_queue_xmit+0x24/0x38
[  637.509259][    C0]  ip_finish_output2+0x1010/0x2578
[  637.510224][    C0]  __ip_finish_output+0x58c/0xb20
[  637.511121][    C0]  ip_finish_output+0x40/0x1f8
[  637.512009][    C0]  ip_output+0x3a0/0x858
[  637.512863][    C0]  ip_local_out+0xbc/0x1a8
[  637.513693][    C0]  __ip_queue_xmit+0x654/0x1698
[  637.514552][    C0]  ip_queue_xmit+0x5c/0x78
[  637.515395][    C0]  __tcp_transmit_skb+0x1614/0x3688
[  637.516280][    C0]  __tcp_send_ack.part.0+0x36c/0x678
[  637.517094][    C0]  tcp_send_ack+0x8c/0xb0
[  637.517957][    C0]  __tcp_ack_snd_check+0x13c/0x8d0
[  637.518847][    C0]  tcp_rcv_established+0x1ae4/0x21a0
[  637.519789][    C0]  tcp_v4_do_rcv+0x724/0xa78
[  637.520701][    C0]  tcp_v4_rcv+0x2a68/0x33a0
[  637.521607][    C0]  ip_protocol_deliver_rcu+0x6c/0x868
[  637.522484][    C0]  ip_local_deliver_finish+0x1e0/0x350
[  637.523374][    C0]  ip_local_deliver+0x370/0x4b8
[  637.524224][    C0]  ip_sublist_rcv_finish+0x12c/0x278
[  637.525057][    C0]  ip_sublist_rcv+0x48c/0x868
[  637.525948][    C0]  ip_list_rcv+0x2c4/0x418
[  637.526879][    C0]  __netif_receive_skb_list_core+0x450/0x740
[  637.527837][    C0]  netif_receive_skb_list_internal+0x558/0xb20
[  637.528779][    C0]  gro_normal_list.part.0+0x24/0xb0
[  637.529675][    C0]  net_rx_action+0xcbc/0x1268
[  637.530599][    C0]  _stext+0x29c/0x1098
[  637.531506][    C0]  irq_exit+0x470/0x540
[  637.532402][    C0]  __handle_domain_irq+0xfc/0x1d0
[  637.533350][    C0]  gic_handle_irq+0x78/0x230
[  637.534194][    C0]  el1_irq+0xc0/0x180
[  637.534982][    C0]  kthread_should_stop+0x8c/0xd0
[  637.535900][    C0]  smpboot_thread_fn+0xa8/0x928
[  637.536734][    C0]  kthread+0x30c/0x408
[  637.537536][    C0]  ret_from_fork+0x10/0x30
[  637.539232][    C0] Kernel panic - not syncing: softlockup: hung tasks
[  637.540826][    C0] CPU: 0 PID: 9 Comm: ksoftirqd/0 Tainted: G             L    5.9.0-12994-gf9893351acae #15
[  637.541819][    C0] Hardware name: linux,dummy-virt (DT)
[  637.542714][    C0] Call trace:
[  637.543595][    C0]  dump_backtrace+0x0/0x4d0
[  637.544529][    C0]  show_stack+0x2c/0x80
[  637.545470][    C0]  dump_stack+0x1b0/0x254
[  637.546328][    C0]  panic+0x3d0/0x7fc
[  637.547177][    C0]  watchdog_timer_fn+0x730/0x740
[  637.548094][    C0]  __hrtimer_run_queues+0x764/0x1438
[  637.550025][    C0]  hrtimer_interrupt+0x300/0x718
[  637.551013][    C0]  arch_timer_handler_virt+0x84/0xb0
[  637.552014][    C0]  handle_percpu_devid_irq+0x258/0xd88
[  637.552973][    C0]  generic_handle_irq+0x88/0xb8
[  637.553894][    C0]  __handle_domain_irq+0xf4/0x1d0
[  637.554826][    C0]  gic_handle_irq+0x78/0x230
[  637.555717][    C0]  el1_irq+0xc0/0x180
[  637.556596][    C0]  _raw_spin_unlock_irqrestore+0x74/0x110
[  637.557573][    C0]  debug_check_no_obj_freed+0x234/0x438
[  637.558456][    C0]  slab_free_freelist_hook+0x174/0x1f8
[  637.559513][    C0]  kmem_cache_free+0xac/0x4f8
[  637.560288][    C0]  kfree_skbmem+0x1b8/0x1f8
[  637.561202][    C0]  consume_skb+0x13c/0x680
[  637.563062][    C0]  __dev_kfree_skb_any+0xb0/0xd0
[  637.566086][    C0]  napi_consume_skb+0x5f0/0x850
[  637.567046][    C0]  free_old_xmit_skbs+0xf8/0x258
[  637.567907][    C0]  start_xmit+0x138/0x1530
[  637.568833][    C0]  dev_hard_start_xmit+0x204/0xde0
[  637.569853][    C0]  sch_direct_xmit+0x278/0x488
[  637.570768][    C0]  __qdisc_run+0x44c/0x1da0
[  637.571669][    C0]  __dev_queue_xmit+0x2164/0x30d0
[  637.572617][    C0]  dev_queue_xmit+0x24/0x38
[  637.573525][    C0]  ip_finish_output2+0x1010/0x2578
[  637.574452][    C0]  __ip_finish_output+0x58c/0xb20
[  637.575341][    C0]  ip_finish_output+0x40/0x1f8
[  637.576161][    C0]  ip_output+0x3a0/0x858
[  637.576975][    C0]  ip_local_out+0xbc/0x1a8
[  637.577844][    C0]  __ip_queue_xmit+0x654/0x1698
[  637.578693][    C0]  ip_queue_xmit+0x5c/0x78
[  637.579566][    C0]  __tcp_transmit_skb+0x1614/0x3688
[  637.580511][    C0]  __tcp_send_ack.part.0+0x36c/0x678
[  637.581498][    C0]  tcp_send_ack+0x8c/0xb0
[  637.582375][    C0]  __tcp_ack_snd_check+0x13c/0x8d0
[  637.583294][    C0]  tcp_rcv_established+0x1ae4/0x21a0
[  637.584205][    C0]  tcp_v4_do_rcv+0x724/0xa78
[  637.585056][    C0]  tcp_v4_rcv+0x2a68/0x33a0
[  637.585944][    C0]  ip_protocol_deliver_rcu+0x6c/0x868
[  637.586822][    C0]  ip_local_deliver_finish+0x1e0/0x350
[  637.587652][    C0]  ip_local_deliver+0x370/0x4b8
[  637.588510][    C0]  ip_sublist_rcv_finish+0x12c/0x278
[  637.589454][    C0]  ip_sublist_rcv+0x48c/0x868
[  637.590367][    C0]  ip_list_rcv+0x2c4/0x418
[  637.591283][    C0]  __netif_receive_skb_list_core+0x450/0x740
[  637.592286][    C0]  netif_receive_skb_list_internal+0x558/0xb20
[  637.593232][    C0]  gro_normal_list.part.0+0x24/0xb0
[  637.594205][    C0]  net_rx_action+0xcbc/0x1268
[  637.595038][    C0]  _stext+0x29c/0x1098
[  637.595868][    C0]  irq_exit+0x470/0x540
[  637.596711][    C0]  __handle_domain_irq+0xfc/0x1d0
[  637.597612][    C0]  gic_handle_irq+0x78/0x230
[  637.598500][    C0]  el1_irq+0xc0/0x180
[  637.599405][    C0]  kthread_should_stop+0x8c/0xd0
[  637.600327][    C0]  smpboot_thread_fn+0xa8/0x928
[  637.601202][    C0]  kthread+0x30c/0x408
[  637.602055][    C0]  ret_from_fork+0x10/0x30
[  637.605644][    C0] SMP: stopping secondary CPUs
[  637.609367][    C0] Dumping ftrace buffer:
[  637.613724][    C0]    (ftrace buffer empty)
[  637.615902][    C0] Kernel Offset: disabled
[  637.617623][    C0] CPU features: 0x0240022,61002082
[  637.618754][    C0] Memory Limit: none
[  637.621412][    C0] Rebooting in 1 seconds..
