[PATCH v2 00/14] idle and soft-irq improvements and POWER9 idle optimisation

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 00/14] idle and soft-irq improvements and POWER9 idle optimisation

Nicholas Piggin-2
Since last time:

- Split out KVM parts. Technically they don't actually have
  dependencies with the Linux patches I suppose, so they could be
  merged via different trees. Logically I think they are better to
  stay together.

- Fix and simplify the KVM secondary thread management patch, thanks
  Gautham.

- Fixed a bug in the ESL=0 avoid overhead patch.

- Retested on P8 and P9 with qemu.

- Mambo simulator still doesn't handle EC=0 wakeups properly, but I
  have reported that, and managed to write a patch here to fix the
  simulator bug and test there too.

Nicholas Piggin (14):
  powerpc/64s: masked interrupt avoid branch
  powerpc/64s: masked interrupt avoid instruction
  powerpc/64s: masked interrupt returns to kernel so avoid r13 restore
  powerpc/64: cleanup __check_irq_replay
  powerpc/64s: irq replay merge HV and non-HV paths for doorbell replay
  powerpc/64s: irq replay external use the HV handler in HV mode on
    POWER9
  powerpc/64: remove redundant instruction in interrupt replay
  powerpc/64s: irq replay remove spurious irq reason
  powerpc/64: runlatch CTRL[RUN] set optimisation
  KVM: PPC: Book3S HV: POWER9 does not require secondary thread
    management
  powerpc/64s: idle POWER9 can execute stop without a sync sequence
  KVM: PPC: Book3S HV: POWER9 can execute stop without a sync sequence
  powerpc/64s: idle POWER9 can execute stop in virtual mode
  powerpc/64s: idle ESL=0 stop can avoid MSR and save/restore overhead

 arch/powerpc/include/asm/cpuidle.h        |  16 -----
 arch/powerpc/include/asm/kvm_book3s_asm.h |   4 ++
 arch/powerpc/kernel/entry_64.S            |   7 +-
 arch/powerpc/kernel/exceptions-64s.S      |  19 +++---
 arch/powerpc/kernel/idle_book3s.S         | 103 +++++++++++++++++-------------
 arch/powerpc/kernel/irq.c                 |  47 +++++++-------
 arch/powerpc/kernel/process.c             |  35 +++++++---
 arch/powerpc/kvm/book3s_hv.c              |  14 +++-
 arch/powerpc/kvm/book3s_hv_rmhandlers.S   |  32 ++++++----
 9 files changed, 154 insertions(+), 123 deletions(-)

--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 01/14] powerpc/64s: masked interrupt avoid branch

Nicholas Piggin-2
Interrupts which do not require EE to be cleared can all
be tested with a single bitwise test.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/exceptions-64s.S | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index f14f3c04ec7e..f8ad3f0eb383 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -1370,10 +1370,8 @@ masked_##_H##interrupt: \
  ori r10,r10,0xffff; \
  mtspr SPRN_DEC,r10; \
  b MASKED_DEC_HANDLER_LABEL; \
-1: cmpwi r10,PACA_IRQ_DBELL; \
- beq 2f; \
- cmpwi r10,PACA_IRQ_HMI; \
- beq 2f; \
+1: andi. r10,r10,(PACA_IRQ_DBELL|PACA_IRQ_HMI); \
+ bne 2f; \
  mfspr r10,SPRN_##_H##SRR1; \
  rldicl r10,r10,48,1; /* clear MSR_EE */ \
  rotldi r10,r10,16; \
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 02/14] powerpc/64s: masked interrupt avoid instruction

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
EE is always enabled in SRR1 for masked interrupts, so clearing
it can use xor.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/exceptions-64s.S | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index f8ad3f0eb383..c4f50a9e2ab5 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -1373,8 +1373,7 @@ masked_##_H##interrupt: \
 1: andi. r10,r10,(PACA_IRQ_DBELL|PACA_IRQ_HMI); \
  bne 2f; \
  mfspr r10,SPRN_##_H##SRR1; \
- rldicl r10,r10,48,1; /* clear MSR_EE */ \
- rotldi r10,r10,16; \
+ xori r10,r10,MSR_EE; /* clear MSR_EE */ \
  mtspr SPRN_##_H##SRR1,r10; \
 2: mtcrf 0x80,r9; \
  ld r9,PACA_EXGEN+EX_R9(r13); \
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 03/14] powerpc/64s: masked interrupt returns to kernel so avoid r13 restore

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
Places in the kernel where r13 is not the PACA pointer must have
maskable interrupts disabled, so r13 does not have to be restored
when returning from a soft-masked interrupt.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/exceptions-64s.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index c4f50a9e2ab5..67321be3122c 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -1379,7 +1379,7 @@ masked_##_H##interrupt: \
  ld r9,PACA_EXGEN+EX_R9(r13); \
  ld r10,PACA_EXGEN+EX_R10(r13); \
  ld r11,PACA_EXGEN+EX_R11(r13); \
- GET_SCRATCH0(r13); \
+ /* returns to kernel where r13 must be set up, so don't restore it */ \
  ##_H##rfid; \
  b .; \
  MASKED_DEC_HANDLER(_H)
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 04/14] powerpc/64: cleanup __check_irq_replay

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
Move the clearing of irq_happened bits into the condition where
they were found to be set. This reduces instruction count slightly,
and reduces stores into irq_happened.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/irq.c | 45 +++++++++++++++++++++++----------------------
 1 file changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/irq.c b/arch/powerpc/kernel/irq.c
index f291f7826abc..7c46e0cce054 100644
--- a/arch/powerpc/kernel/irq.c
+++ b/arch/powerpc/kernel/irq.c
@@ -143,9 +143,10 @@ notrace unsigned int __check_irq_replay(void)
  */
  unsigned char happened = local_paca->irq_happened;
 
- /* Clear bit 0 which we wouldn't clear otherwise */
- local_paca->irq_happened &= ~PACA_IRQ_HARD_DIS;
  if (happened & PACA_IRQ_HARD_DIS) {
+ /* Clear bit 0 which we wouldn't clear otherwise */
+ local_paca->irq_happened &= ~PACA_IRQ_HARD_DIS;
+
  /*
  * We may have missed a decrementer interrupt if hard disabled.
  * Check the decrementer register in case we had a rollover
@@ -173,39 +174,39 @@ notrace unsigned int __check_irq_replay(void)
  * This is a higher priority interrupt than the others, so
  * replay it first.
  */
- local_paca->irq_happened &= ~PACA_IRQ_HMI;
- if (happened & PACA_IRQ_HMI)
+ if (happened & PACA_IRQ_HMI) {
+ local_paca->irq_happened &= ~PACA_IRQ_HMI;
  return 0xe60;
+ }
 
- /*
- * We may have missed a decrementer interrupt. We check the
- * decrementer itself rather than the paca irq_happened field
- * in case we also had a rollover while hard disabled
- */
- local_paca->irq_happened &= ~PACA_IRQ_DEC;
- if (happened & PACA_IRQ_DEC)
+ if (happened & PACA_IRQ_DEC) {
+ local_paca->irq_happened &= ~PACA_IRQ_DEC;
  return 0x900;
+ }
 
- /* Finally check if an external interrupt happened */
- local_paca->irq_happened &= ~PACA_IRQ_EE;
- if (happened & PACA_IRQ_EE)
+ if (happened & PACA_IRQ_EE) {
+ local_paca->irq_happened &= ~PACA_IRQ_EE;
  return 0x500;
+ }
 
 #ifdef CONFIG_PPC_BOOK3E
- /* Finally check if an EPR external interrupt happened
- * this bit is typically set if we need to handle another
- * "edge" interrupt from within the MPIC "EPR" handler
+ /*
+ * Check if an EPR external interrupt happened this bit is typically
+ * set if we need to handle another "edge" interrupt from within the
+ * MPIC "EPR" handler.
  */
- local_paca->irq_happened &= ~PACA_IRQ_EE_EDGE;
- if (happened & PACA_IRQ_EE_EDGE)
+ if (happened & PACA_IRQ_EE_EDGE) {
+ local_paca->irq_happened &= ~PACA_IRQ_EE_EDGE;
  return 0x500;
+ }
 
- local_paca->irq_happened &= ~PACA_IRQ_DBELL;
- if (happened & PACA_IRQ_DBELL)
+ if (happened & PACA_IRQ_DBELL) {
+ local_paca->irq_happened &= ~PACA_IRQ_DBELL;
  return 0x280;
+ }
 #else
- local_paca->irq_happened &= ~PACA_IRQ_DBELL;
  if (happened & PACA_IRQ_DBELL) {
+ local_paca->irq_happened &= ~PACA_IRQ_DBELL;
  if (cpu_has_feature(CPU_FTR_HVMODE))
  return 0xe80;
  return 0xa00;
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 05/14] powerpc/64s: irq replay merge HV and non-HV paths for doorbell replay

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
This results in smaller code, and fewer branches.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/entry_64.S       | 6 +-----
 arch/powerpc/kernel/exceptions-64s.S | 2 +-
 arch/powerpc/kernel/irq.c            | 2 --
 3 files changed, 2 insertions(+), 8 deletions(-)

diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 49d8422767b4..ec67f67dafab 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -990,11 +990,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
 #ifdef CONFIG_PPC_BOOK3E
  cmpwi cr0,r3,0x280
 #else
- BEGIN_FTR_SECTION
- cmpwi cr0,r3,0xe80
- FTR_SECTION_ELSE
- cmpwi cr0,r3,0xa00
- ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE)
+ cmpwi cr0,r3,0xa00
 #endif /* CONFIG_PPC_BOOK3E */
  bne 1f
  addi r3,r1,STACK_FRAME_OVERHEAD;
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 67321be3122c..f9d0796fb2c9 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -1674,7 +1674,7 @@ _GLOBAL(__replay_interrupt)
  cmpwi r3,0x500
  beq hardware_interrupt_common
 BEGIN_FTR_SECTION
- cmpwi r3,0xe80
+ cmpwi r3,0xa00
  beq h_doorbell_common_msgclr
  cmpwi r3,0xea0
  beq h_virt_irq_common
diff --git a/arch/powerpc/kernel/irq.c b/arch/powerpc/kernel/irq.c
index 7c46e0cce054..60ee6d7251b8 100644
--- a/arch/powerpc/kernel/irq.c
+++ b/arch/powerpc/kernel/irq.c
@@ -207,8 +207,6 @@ notrace unsigned int __check_irq_replay(void)
 #else
  if (happened & PACA_IRQ_DBELL) {
  local_paca->irq_happened &= ~PACA_IRQ_DBELL;
- if (cpu_has_feature(CPU_FTR_HVMODE))
- return 0xe80;
  return 0xa00;
  }
 #endif /* CONFIG_PPC_BOOK3E */
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 06/14] powerpc/64s: irq replay external use the HV handler in HV mode on POWER9

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
POWER9 host external interrupts use the h_virt_irq_common handler, so
use that to replay them rather than using the hardware_interrupt_common
handler. Both call do_IRQ, but using the correct handler reduces i-cache
footprint.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/exceptions-64s.S | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index f9d0796fb2c9..29253cecf713 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -1672,7 +1672,11 @@ _GLOBAL(__replay_interrupt)
  cmpwi r3,0x900
  beq decrementer_common
  cmpwi r3,0x500
+BEGIN_FTR_SECTION
+ beq h_virt_irq_common
+FTR_SECTION_ELSE
  beq hardware_interrupt_common
+ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_300)
 BEGIN_FTR_SECTION
  cmpwi r3,0xa00
  beq h_doorbell_common_msgclr
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 07/14] powerpc/64: remove redundant instruction in interrupt replay

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/entry_64.S | 1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index ec67f67dafab..3f2666d24a7e 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -995,7 +995,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
  bne 1f
  addi r3,r1,STACK_FRAME_OVERHEAD;
  bl doorbell_exception
- b ret_from_except
 #endif /* CONFIG_PPC_DOORBELL */
 1: b ret_from_except /* What else to do here ? */
 
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 08/14] powerpc/64s: irq replay remove spurious irq reason

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
HVI interrupts have always used 0x500, so remove the dead branch.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/exceptions-64s.S | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 29253cecf713..566cf126c13b 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -1680,8 +1680,6 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_300)
 BEGIN_FTR_SECTION
  cmpwi r3,0xa00
  beq h_doorbell_common_msgclr
- cmpwi r3,0xea0
- beq h_virt_irq_common
  cmpwi r3,0xe60
  beq hmi_exception_common
 FTR_SECTION_ELSE
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 09/14] powerpc/64: runlatch CTRL[RUN] set optimisation

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
The CTRL register is read-only except bit 63 which is the run latch
control. This means it can be updated with a mtspr rather than
mfspr/mtspr.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/process.c | 35 +++++++++++++++++++++++++++--------
 1 file changed, 27 insertions(+), 8 deletions(-)

diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 9f3e2c932dcc..75306b6e1812 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1994,11 +1994,25 @@ void show_stack(struct task_struct *tsk, unsigned long *stack)
 void notrace __ppc64_runlatch_on(void)
 {
  struct thread_info *ti = current_thread_info();
- unsigned long ctrl;
 
- ctrl = mfspr(SPRN_CTRLF);
- ctrl |= CTRL_RUNLATCH;
- mtspr(SPRN_CTRLT, ctrl);
+ if (cpu_has_feature(CPU_FTR_ARCH_206)) {
+ /*
+ * Least significant bit (RUN) is the only writable bit of
+ * the CTRL register, so we can avoid mfspr. 2.06 is not the
+ * earliest ISA where this is the case, but it's convenient.
+ */
+ mtspr(SPRN_CTRLT, CTRL_RUNLATCH);
+ } else {
+ unsigned long ctrl;
+
+ /*
+ * Some architectures (e.g., Cell) have writable fields other
+ * than RUN, so do the read-modify-write.
+ */
+ ctrl = mfspr(SPRN_CTRLF);
+ ctrl |= CTRL_RUNLATCH;
+ mtspr(SPRN_CTRLT, ctrl);
+ }
 
  ti->local_flags |= _TLF_RUNLATCH;
 }
@@ -2007,13 +2021,18 @@ void notrace __ppc64_runlatch_on(void)
 void notrace __ppc64_runlatch_off(void)
 {
  struct thread_info *ti = current_thread_info();
- unsigned long ctrl;
 
  ti->local_flags &= ~_TLF_RUNLATCH;
 
- ctrl = mfspr(SPRN_CTRLF);
- ctrl &= ~CTRL_RUNLATCH;
- mtspr(SPRN_CTRLT, ctrl);
+ if (cpu_has_feature(CPU_FTR_ARCH_206)) {
+ mtspr(SPRN_CTRLT, 0);
+ } else {
+ unsigned long ctrl;
+
+ ctrl = mfspr(SPRN_CTRLF);
+ ctrl &= ~CTRL_RUNLATCH;
+ mtspr(SPRN_CTRLT, ctrl);
+ }
 }
 #endif /* CONFIG_PPC64 */
 
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 10/14] KVM: PPC: Book3S HV: POWER9 does not require secondary thread management

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
POWER9 CPUs have independent MMU contexts per thread, so KVM does not
need to quiesce secondary threads, so the hwthread_req/hwthread_state
protocol does not have to be used. So patch it away on POWER9, and patch
away the branch from the Linux idle wakeup to kvm_start_guest that is
never used.

Add a warning and error out of kvmppc_grab_hwthread in case it is ever
called on POWER9.

This avoids a hwsync in the idle wakeup path on POWER9.

Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/include/asm/kvm_book3s_asm.h |  4 ++++
 arch/powerpc/kernel/idle_book3s.S         | 30 +++++++++++++++---------------
 arch/powerpc/kvm/book3s_hv.c              | 14 +++++++++++++-
 arch/powerpc/kvm/book3s_hv_rmhandlers.S   |  8 ++++++++
 4 files changed, 40 insertions(+), 16 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 7cea76f11c26..83596f32f50b 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -104,6 +104,10 @@ struct kvmppc_host_state {
  u8 napping;
 
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+ /*
+ * hwthread_req/hwthread_state pair is used to pull sibling threads
+ * out of guest on pre-ISAv3.0B CPUs where threads share MMU.
+ */
  u8 hwthread_req;
  u8 hwthread_state;
  u8 host_ipi;
diff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S
index e6252c5a57a4..9a9a28f0758d 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -243,12 +243,6 @@ enter_winkle:
  * r3 - PSSCR value corresponding to the requested stop state.
  */
 power_enter_stop:
-#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
- /* Tell KVM we're entering idle */
- li r4,KVM_HWTHREAD_IN_IDLE
- /* DO THIS IN REAL MODE!  See comment above. */
- stb r4,HSTATE_HWTHREAD_STATE(r13)
-#endif
 /*
  * Check if we are executing the lite variant with ESL=EC=0
  */
@@ -411,6 +405,18 @@ pnv_powersave_wakeup_mce:
 
  b pnv_powersave_wakeup
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+kvm_start_guest_check:
+ li r0,KVM_HWTHREAD_IN_KERNEL
+ stb r0,HSTATE_HWTHREAD_STATE(r13)
+ /* Order setting hwthread_state vs. testing hwthread_req */
+ sync
+ lbz r0,HSTATE_HWTHREAD_REQ(r13)
+ cmpwi r0,0
+ beqlr
+ b kvm_start_guest
+#endif
+
 /*
  * Called from reset vector for powersave wakeups.
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
@@ -435,15 +441,9 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_ARCH_300)
  mr r3,r12
 
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
- li r0,KVM_HWTHREAD_IN_KERNEL
- stb r0,HSTATE_HWTHREAD_STATE(r13)
- /* Order setting hwthread_state vs. testing hwthread_req */
- sync
- lbz r0,HSTATE_HWTHREAD_REQ(r13)
- cmpwi r0,0
- beq 1f
- b kvm_start_guest
-1:
+BEGIN_FTR_SECTION
+ bl kvm_start_guest_check
+END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)
 #endif
 
  /* Return SRR1 from power7_nap() */
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 359c79cdf0cc..e34cd6fb947b 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2111,6 +2111,16 @@ static int kvmppc_grab_hwthread(int cpu)
  struct paca_struct *tpaca;
  long timeout = 10000;
 
+ /*
+ * ISA v3.0 idle routines do not set hwthread_state or test
+ * hwthread_req, so they can not grab idle threads.
+ */
+ if (cpu_has_feature(CPU_FTR_ARCH_300)) {
+ WARN_ON(1);
+ pr_err("KVM: can not control sibling threads\n");
+ return -EBUSY;
+ }
+
  tpaca = &paca[cpu];
 
  /* Ensure the thread won't go into the kernel if it wakes */
@@ -2145,10 +2155,12 @@ static void kvmppc_release_hwthread(int cpu)
  struct paca_struct *tpaca;
 
  tpaca = &paca[cpu];
- tpaca->kvm_hstate.hwthread_req = 0;
  tpaca->kvm_hstate.kvm_vcpu = NULL;
  tpaca->kvm_hstate.kvm_vcore = NULL;
  tpaca->kvm_hstate.kvm_split_mode = NULL;
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ tpaca->kvm_hstate.hwthread_req = 0;
+
 }
 
 static void radix_flush_cpu(struct kvm *kvm, int cpu, struct kvm_vcpu *vcpu)
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index c52184a8efdf..3e024fd71fe8 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -149,9 +149,11 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
  subf r4, r4, r3
  mtspr SPRN_DEC, r4
 
+BEGIN_FTR_SECTION
  /* hwthread_req may have got set by cede or no vcpu, so clear it */
  li r0, 0
  stb r0, HSTATE_HWTHREAD_REQ(r13)
+END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)
 
  /*
  * For external interrupts we need to call the Linux
@@ -314,6 +316,7 @@ kvm_novcpu_exit:
  * Relocation is off and most register values are lost.
  * r13 points to the PACA.
  * r3 contains the SRR1 wakeup value, SRR1 is trashed.
+ * This is not used by ISAv3.0B processors.
  */
  .globl kvm_start_guest
 kvm_start_guest:
@@ -432,6 +435,9 @@ kvm_secondary_got_guest:
  * While waiting we also need to check if we get given a vcpu to run.
  */
 kvm_no_guest:
+BEGIN_FTR_SECTION
+ twi 31,0,0
+END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
  lbz r3, HSTATE_HWTHREAD_REQ(r13)
  cmpwi r3, 0
  bne 53f
@@ -2509,8 +2515,10 @@ kvm_do_nap:
  clrrdi r0, r0, 1
  mtspr SPRN_CTRLT, r0
 
+BEGIN_FTR_SECTION
  li r0,1
  stb r0,HSTATE_HWTHREAD_REQ(r13)
+END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)
  mfspr r5,SPRN_LPCR
  ori r5,r5,LPCR_PECE0 | LPCR_PECE1
 BEGIN_FTR_SECTION
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 11/14] powerpc/64s: idle POWER9 can execute stop without a sync sequence

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
Reviewed-by: Gautham R. Shenoy <[hidden email]>
Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/include/asm/cpuidle.h | 16 ----------------
 arch/powerpc/kernel/idle_book3s.S  | 26 ++++++++++++++++++++------
 2 files changed, 20 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h b/arch/powerpc/include/asm/cpuidle.h
index 52586f9956bb..6853a3741338 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -90,20 +90,4 @@ static inline void report_invalid_psscr_val(u64 psscr_val, int err)
 
 #endif
 
-/* Idle state entry routines */
-#ifdef CONFIG_PPC_P7_NAP
-#define IDLE_STATE_ENTER_SEQ(IDLE_INST)                         \
- /* Magic NAP/SLEEP/WINKLE mode enter sequence */ \
- std r0,0(r1); \
- ptesync; \
- ld r0,0(r1); \
-236: cmpd cr0,r0,r0; \
- bne 236b; \
- IDLE_INST; \
-
-#define IDLE_STATE_ENTER_SEQ_NORET(IDLE_INST) \
- IDLE_STATE_ENTER_SEQ(IDLE_INST)                         \
- b .
-#endif /* CONFIG_PPC_P7_NAP */
-
 #endif
diff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S
index 9a9a28f0758d..3b701b1a5e87 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -151,6 +151,19 @@ pnv_powersave_common:
  mtmsrd r7,0
  bctr
 
+/*
+ * This is the sequence required to execute idle instructions, as
+ * specified in ISA v2.07. MSR[IR] and MSR[DR] must be 0.
+ */
+#define ARCH207_IDLE_STATE_ENTER_SEQ_NORET(IDLE_INST) \
+ /* Magic NAP/SLEEP/WINKLE mode enter sequence */ \
+ std r0,0(r1); \
+ ptesync; \
+ ld r0,0(r1); \
+236: cmpd cr0,r0,r0; \
+ bne 236b; \
+ IDLE_INST;
+
  .globl pnv_enter_arch207_idle_mode
 pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
@@ -176,7 +189,7 @@ pnv_enter_arch207_idle_mode:
  stb r3,PACA_THREAD_IDLE_STATE(r13)
  cmpwi cr3,r3,PNV_THREAD_SLEEP
  bge cr3,2f
- IDLE_STATE_ENTER_SEQ_NORET(PPC_NAP)
+ ARCH207_IDLE_STATE_ENTER_SEQ_NORET(PPC_NAP)
  /* No return */
 2:
  /* Sleep or winkle */
@@ -215,7 +228,7 @@ pnv_fastsleep_workaround_at_entry:
 
 common_enter: /* common code for all the threads entering sleep or winkle */
  bgt cr3,enter_winkle
- IDLE_STATE_ENTER_SEQ_NORET(PPC_SLEEP)
+ ARCH207_IDLE_STATE_ENTER_SEQ_NORET(PPC_SLEEP)
 
 fastsleep_workaround_at_entry:
  oris r15,r15,PNV_CORE_IDLE_LOCK_BIT@h
@@ -237,7 +250,7 @@ fastsleep_workaround_at_entry:
 enter_winkle:
  bl save_sprs_to_stack
 
- IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
+ ARCH207_IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
 
 /*
  * r3 - PSSCR value corresponding to the requested stop state.
@@ -249,7 +262,7 @@ power_enter_stop:
  andis.   r4,r3,PSSCR_EC_ESL_MASK_SHIFTED
  clrldi   r3,r3,60 /* r3 = Bits[60:63] = Requested Level (RL) */
  bne .Lhandle_esl_ec_set
- IDLE_STATE_ENTER_SEQ(PPC_STOP)
+ PPC_STOP
  li r3,0  /* Since we didn't lose state, return 0 */
 
  /*
@@ -282,7 +295,8 @@ power_enter_stop:
  ld r4,ADDROFF(pnv_first_deep_stop_state)(r5)
  cmpd r3,r4
  bge .Lhandle_deep_stop
- IDLE_STATE_ENTER_SEQ_NORET(PPC_STOP)
+ PPC_STOP /* Does not return (system reset interrupt) */
+
 .Lhandle_deep_stop:
 /*
  * Entering deep idle state.
@@ -304,7 +318,7 @@ lwarx_loop_stop:
 
  bl save_sprs_to_stack
 
- IDLE_STATE_ENTER_SEQ_NORET(PPC_STOP)
+ PPC_STOP /* Does not return (system reset interrupt) */
 
 /*
  * Entered with MSR[EE]=0 and no soft-masked interrupts pending.
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 12/14] KVM: PPC: Book3S HV: POWER9 can execute stop without a sync sequence

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
Reviewed-by: Gautham R. Shenoy <[hidden email]>
Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kvm/book3s_hv_rmhandlers.S | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 3e024fd71fe8..edb47738a686 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -2527,7 +2527,17 @@ BEGIN_FTR_SECTION
 END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
 
 kvm_nap_sequence: /* desired LPCR value in r5 */
-BEGIN_FTR_SECTION
+BEGIN_FTR_SECTION /* nap sequence */
+ mtspr SPRN_LPCR,r5
+ isync
+ li r0, 0
+ std r0, HSTATE_SCRATCH0(r13)
+ ptesync
+ ld r0, HSTATE_SCRATCH0(r13)
+1: cmpd r0, r0
+ bne 1b
+ nap
+FTR_SECTION_ELSE /* stop sequence */
  /*
  * PSSCR bits: exit criterion = 1 (wakeup based on LPCR at sreset)
  * enable state loss = 1 (allow SMT mode switch)
@@ -2539,18 +2549,8 @@ BEGIN_FTR_SECTION
  li r4, LPCR_PECE_HVEE@higher
  sldi r4, r4, 32
  or r5, r5, r4
-END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
  mtspr SPRN_LPCR,r5
- isync
- li r0, 0
- std r0, HSTATE_SCRATCH0(r13)
- ptesync
- ld r0, HSTATE_SCRATCH0(r13)
-1: cmpd r0, r0
- bne 1b
-BEGIN_FTR_SECTION
- nap
-FTR_SECTION_ELSE
+
  PPC_STOP
 ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300)
  b .
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 13/14] powerpc/64s: idle POWER9 can execute stop in virtual mode

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
The hardware can execute stop in any context, and KVM does not
require real mode because siblings do not share MMU state. This
saves a switch to real-mode when going idle.

Acked-by: Gautham R. Shenoy <[hidden email]>
Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/idle_book3s.S | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S
index 3b701b1a5e87..fc5145339277 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -141,7 +141,16 @@ pnv_powersave_common:
  std r5,_CCR(r1)
  std r1,PACAR1(r13)
 
+BEGIN_FTR_SECTION
+ /*
+ * POWER9 does not require real mode to stop, and does not set
+ * hwthread_state for KVM (threads don't share MMU context), so
+ * we can remain in virtual mode for this.
+ */
+ bctr
+END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
  /*
+ * POWER8
  * Go to real mode to do the nap, as required by the architecture.
  * Also, we need to be in real mode before setting hwthread_state,
  * because as soon as we do that, another thread can switch
--
2.13.3

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH v2 14/14] powerpc/64s: idle ESL=0 stop can avoid MSR and save/restore overhead

Nicholas Piggin-2
In reply to this post by Nicholas Piggin-2
When stop is executed with EC=ESL=0, it appears to execute like a
normal instruction (resuming from NIP when woken by interrupt).
So all the save/restore handling can be avoided completely. In
particular NV GPRs do not have to be saved, and MSR does not have
to be switched back to kernel MSR.

So move the test for "lite" sleep states out to power9_idle_stop.

Reviewed-by: Gautham R. Shenoy <[hidden email]>
Signed-off-by: Nicholas Piggin <[hidden email]>
---
 arch/powerpc/kernel/idle_book3s.S | 40 ++++++++++++++-------------------------
 1 file changed, 14 insertions(+), 26 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S
index fc5145339277..0d8dd9823bd3 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -264,31 +264,8 @@ enter_winkle:
 /*
  * r3 - PSSCR value corresponding to the requested stop state.
  */
-power_enter_stop:
-/*
- * Check if we are executing the lite variant with ESL=EC=0
- */
- andis.   r4,r3,PSSCR_EC_ESL_MASK_SHIFTED
+power_enter_stop_esl:
  clrldi   r3,r3,60 /* r3 = Bits[60:63] = Requested Level (RL) */
- bne .Lhandle_esl_ec_set
- PPC_STOP
- li r3,0  /* Since we didn't lose state, return 0 */
-
- /*
- * pnv_wakeup_noloss() expects r12 to contain the SRR1 value so
- * it can determine if the wakeup reason is an HMI in
- * CHECK_HMI_INTERRUPT.
- *
- * However, when we wakeup with ESL=0, SRR1 will not contain the wakeup
- * reason, so there is no point setting r12 to SRR1.
- *
- * Further, we clear r12 here, so that we don't accidentally enter the
- * HMI in pnv_wakeup_noloss() if the value of r12[42:45] == WAKE_HMI.
- */
- li r12, 0
- b pnv_wakeup_noloss
-
-.Lhandle_esl_ec_set:
  /*
  * POWER9 DD2 can incorrectly set PMAO when waking up after a
  * state-loss idle. Saving and restoring MMCR0 over idle is a
@@ -361,9 +338,20 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66); \
  * r3 contains desired PSSCR register value.
  */
 _GLOBAL(power9_idle_stop)
- std r3, PACA_REQ_PSSCR(r13)
+ /*
+ * Check if we are executing the lite variant with ESL=EC=0
+ * This case resumes execution after the stop instruction without
+ * losing any state, so nothing has to be saved.
+ */
  mtspr SPRN_PSSCR,r3
- LOAD_REG_ADDR(r4,power_enter_stop)
+ andis. r4,r3,PSSCR_EC_ESL_MASK_SHIFTED
+ bne 1f
+ PPC_STOP
+ li r3,0  /* Since we didn't lose state, return 0 */
+ blr
+1: /* state-loss idle */
+ std r3, PACA_REQ_PSSCR(r13)
+ LOAD_REG_ADDR(r4,power_enter_stop_esl)
  b pnv_powersave_common
  /* No return */
 
--
2.13.3

Loading...