Commit Graph

984095 Commits

Author SHA1 Message Date
Kalesh Singh
437bfe2d64 UPSTREAM: tracing/selftests: Add tests for hist trigger expression parsing
Add tests for the parsing of hist trigger expressions; and to
validate expression evaluation.

Link: https://lkml.kernel.org/r/20211029183339.3216491-5-kaleshsingh@google.com

Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Tom Zanussi <zanussi@kernel.org>
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 4e9f63c9e5)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I4c5f12675ab6ec2641a45ab1852e07b8adcd827a
2021-11-04 17:33:48 +00:00
Kalesh Singh
bb73ccd853 UPSTREAM: tracing/histogram: Document hist trigger variables
Update the tracefs README to describe how hist trigger variables
can be created.

Link: https://lkml.kernel.org/r/20211029183339.3216491-4-kaleshsingh@google.com

Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Tom Zanussi <zanussi@kernel.org>
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 6a6e5ef2b2)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I8ea0a4c44fe3496d13a09bcf4922e8aa5cdb78ba
2021-11-04 17:33:38 +00:00
Kalesh Singh
47347e1917 UPSTREAM: tracing/histogram: Update division by 0 documentation
If the divisor is a constant and zero, the undeifned case can be
detected and an error returned instead of -1.

Link: https://lkml.kernel.org/r/20211029183339.3216491-3-kaleshsingh@google.com

Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Tom Zanussi <zanussi@kernel.org>
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 0ca6d12c97)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I5bf188056aa42bee69f0e4439a23b004295322db
2021-11-04 17:33:15 +00:00
Kalesh Singh
43484db804 BACKPORT: tracing/histogram: Optimize division by constants
If the divisor is a constant use specific division functions to
avoid extra branches when the trigger is hit.

If the divisor constant but not a power of 2, the division can be
replaced with a multiplication and shift in the following case:

Let X = dividend and Y = divisor.

Choose Z = some power of 2. If Y <= Z, then:
    X / Y = (X * (Z / Y)) / Z

(Z / Y) is a constant (mult) which is calculated at parse time, so:
    X / Y = (X * mult) / Z

The division by Z can be replaced by a shift since Z is a power of 2:
    X / Y = (X * mult) >> shift

As long, as X < Z the results will not be off by more than 1.

Link: https://lkml.kernel.org/r/20211029232410.3494196-1-kaleshsingh@google.com

Suggested-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 8b5d46fd7a)

[ Kalesh Singh: Resolve conflicts in kernel/trace/trace_events_hist.c ]
Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I700b7fdfa461f9ca5d8c49aecbf8ea19c5c74a18
2021-11-04 17:32:55 +00:00
kernel test robot
34cb12b5d3 UPSTREAM: tracing/histogram: Fix semicolon.cocci warnings
kernel/trace/trace_events_hist.c:6039:2-3: Unneeded semicolon

 Remove unneeded semicolon.

Generated by: scripts/coccinelle/misc/semicolon.cocci

Link: https://lkml.kernel.org/r/20211030005615.GA41257@3074f0d39c61

Fixes: c5eac6ee8b ("tracing/histogram: Simplify handling of .sym-offset in expressions")
CC: Kalesh Singh <kaleshsingh@google.com>
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: kernel test robot <lkp@intel.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit feea69ec12)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: Id6929c9ea0cc4e9f8e3c7ac1279410db1ae8f413
2021-11-04 17:30:46 +00:00
Kalesh Singh
572d42d1b0 UPSTREAM: tracing/histogram: Fix documentation inline emphasis warning
This fixes the warning:

Documentation/trace/histogram.rst:1766: WARNING: Inline emphasis
start-string without end-string

The issue was caused by an unescaped '*' character.

Link: https://lore.kernel.org/all/20211028170548.2597449-1-kaleshsingh@google.com/T/#m77da47432f5cc6521d4294ffdb9621949cc35d04
Link: https://lkml.kernel.org/r/20211028170548.2597449-1-kaleshsingh@google.com

Fixes: 2d2f6d4b8c ("tracing/histogram: Document expression arithmetic and constants")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 93d76e4a0e)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: Ie9d0738d8bb47ea7fd84917a79cad99fdefe767f
2021-11-04 17:30:06 +00:00
Kalesh Singh
0036c192be UPSTREAM: tracing/histogram: Document expression arithmetic and constants
Histogram expressions now support division, and multiplication in
addition to the already supported subtraction and addition operators.

Numeric constants can also be used in a hist trigger expressions
or assigned to a variable and used by refernce in an expression.

Link: https://lkml.kernel.org/r/20211025200852.3002369-9-kaleshsingh@google.com

Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Reviewed-by: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 2d2f6d4b8c)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: Iaf9609e3b0373414280f6c8aae638c034797bbf2
2021-11-04 17:29:26 +00:00
Kalesh Singh
fe8d0a93e7 UPSTREAM: tracing/histogram: Optimize division by a power of 2
The division is a slow operation. If the divisor is a power of 2, use a
shift instead.

Results were obtained using Android's version of perf (simpleperf[1]) as
described below:

1. hist_field_div() is modified to call 2 test functions:
   test_hist_field_div_[not]_optimized(); passing them the
   same args. Use noinline and volatile to ensure these are
   not optimized out by the compiler.
2. Create a hist event trigger that uses division:
      events/kmem/rss_stat$ echo 'hist:keys=common_pid:x=size/<divisor>'
         >> trigger
      events/kmem/rss_stat$ echo 'hist:keys=common_pid:vals=$x'
         >> trigger
3. Run Android's lmkd_test[2] to generate rss_stat events, and
   record CPU samples with Android's simpleperf:
      simpleperf record -a --exclude-perf --post-unwind=yes -m 16384 -g
         -f 2000 -o perf.data

== Results ==

Divisor is a power of 2 (divisor == 32):

   test_hist_field_div_not_optimized  | 8,717,091 cpu-cycles
   test_hist_field_div_optimized      | 1,643,137 cpu-cycles

If the divisor is a power of 2, the optimized version is ~5.3x faster.

Divisor is not a power of 2 (divisor == 33):

   test_hist_field_div_not_optimized  | 4,444,324 cpu-cycles
   test_hist_field_div_optimized      | 5,497,958 cpu-cycles

If the divisor is not a power of 2, as expected, the optimized version is
slightly slower (~24% slower).

[1] https://android.googlesource.com/platform/system/extras/+/master/simpleperf/doc/README.md
[2] https://cs.android.com/android/platform/superproject/+/master:system/memory/lmkd/tests/lmkd_test.cpp

Link: https://lkml.kernel.org/r/20211025200852.3002369-7-kaleshsingh@google.com

Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Suggested-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 722eddaa40)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: Iffa3aa6beba6c0f6e1761047e14a69486bf0753d
2021-11-04 17:28:53 +00:00
Kalesh Singh
1f8f3e1fff BACKPORT: tracing/histogram: Covert expr to const if both operands are constants
If both operands of a hist trigger expression are constants, convert the
expression to a constant. This optimization avoids having to perform the
same calculation multiple times and also saves on memory since the
merged constants are represented by a single struct hist_field instead
or multiple.

Link: https://lkml.kernel.org/r/20211025200852.3002369-6-kaleshsingh@google.com

Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Suggested-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit f47716b7a9)

[ Kalesh Singh: Resolve conflicts in kernel/trace/trace_events_hist.c ]
Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: Iaa33bfd20808cee2fe62b65ab81b5aa01cb9a8dd
2021-11-04 17:28:10 +00:00
Kalesh Singh
fa428eedd3 UPSTREAM: tracing/histogram: Simplify handling of .sym-offset in expressions
The '-' in .sym-offset can confuse the hist trigger arithmetic
expression parsing. Simplify the handling of this by replacing the
'sym-offset' with 'symXoffset'. This allows us to correctly evaluate
expressions where the user may have inadvertently added a .sym-offset
modifier to one of the operands in an expression, instead of bailing
out. In this case the .sym-offset has no effect on the evaluation of the
expression. The only valid use of the .sym-offset is as a hist key
modifier.

Link: https://lkml.kernel.org/r/20211025200852.3002369-5-kaleshsingh@google.com

Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Suggested-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit c5eac6ee8b)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I31debbf1ad5d14d2b43a7894ec8703b9a9bf1d74
2021-11-04 17:26:41 +00:00
Kalesh Singh
3c296c22ea UPSTREAM: tracing: Fix operator precedence for hist triggers expression
The current histogram expression evaluation logic evaluates the
expression from right to left. This can lead to incorrect results
if the operations are not associative (as is the case for subtraction
and, the now added, division operators).
	e.g. 16-8-4-2 should be 2 not 10 --> 16-8-4-2 = ((16-8)-4)-2
	     64/8/4/2 should be 1 not 16 --> 64/8/4/2 = ((64/8)/4)/2

Division and multiplication are currently limited to single operation
expression due to operator precedence support not yet implemented.

Rework the expression parsing to support the correct evaluation of
expressions containing operators of different precedences; and fix
the associativity error by evaluating expressions with operators of
the same precedence from left to right.

Examples:
        (1) echo 'hist:keys=common_pid:a=8,b=4,c=2,d=1,w=$a-$b-$c-$d' \
                  >> event/trigger
        (2) echo 'hist:keys=common_pid:x=$a/$b/3/2' >> event/trigger
        (3) echo 'hist:keys=common_pid:y=$a+10/$c*1024' >> event/trigger
        (4) echo 'hist:keys=common_pid:z=$a/$b+$c*$d' >> event/trigger

Link: https://lkml.kernel.org/r/20211025200852.3002369-4-kaleshsingh@google.com

Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Reviewed-by: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 9710b2f341)

Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I9f5fa30eb9b9473e6ddfe7de8d6d75a6840a6f9e
2021-11-04 17:25:35 +00:00
Kalesh Singh
b421a7c2b7 BACKPORT: tracing: Add division and multiplication support for hist triggers
Adds basic support for division and multiplication operations for
hist trigger variable expressions.

For simplicity this patch only supports, division and multiplication
for a single operation expression (e.g. x=$a/$b), as currently
expressions are always evaluated right to left. This can lead to some
incorrect results:

	e.g. echo 'hist:keys=common_pid:x=8-4-2' >> event/trigger

	     8-4-2 should evaluate to 2 i.e. (8-4)-2
	     but currently x evaluate to  6 i.e. 8-(4-2)

Multiplication and division in sub-expressions will work correctly, once
correct operator precedence support is added (See next patch in this
series).

For the undefined case of division by 0, the histogram expression
evaluates to (u64)(-1). Since this cannot be detected when the
expression is created, it is the responsibility of the user to be
aware and account for this possibility.

Examples:
	echo 'hist:keys=common_pid:a=8,b=4,x=$a/$b' \
                   >> event/trigger

	echo 'hist:keys=common_pid:y=5*$b' \
                   >> event/trigger

Link: https://lkml.kernel.org/r/20211025200852.3002369-3-kaleshsingh@google.com

Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit bcef044150)

[ Kalesh Singh: Resolve minor conflicts in kernel/trace/trace_events_hist.c ]
Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I3b57e8bf353992e1ef052174bec16754d14f2127
2021-11-04 17:24:36 +00:00
Kalesh Singh
6ffd928653 BACKPORT: tracing: Add support for creating hist trigger variables from literal
Currently hist trigger expressions don't support the use of numeric
literals:
	e.g. echo 'hist:keys=common_pid:x=$y-1234'
		--> is not valid expression syntax

Having the ability to use numeric constants in hist triggers supports
a wider range of expressions for creating variables.

Add support for creating trace event histogram variables from numeric
literals.

	e.g. echo 'hist:keys=common_pid:x=1234,y=size-1024' >> event/trigger

A negative numeric constant is created, using unary minus operator
(parentheses are required).

	e.g. echo 'hist:keys=common_pid:z=-(2)' >> event/trigger

Constants can be used with division/multiplication (added in the
next patch in this series) to implement granularity filters for frequent
trace events. For instance we can limit emitting the rss_stat
trace event to when there is a 512KB cross over in the rss size:

  # Create a synthetic event to monitor instead of the high frequency
  # rss_stat event
  echo 'rss_stat_throttled unsigned int mm_id; unsigned int curr;
	int member; long size' >> tracing/synthetic_events

  # Create a hist trigger that emits the synthetic rss_stat_throttled
  # event only when the rss size crosses a 512KB boundary.
  echo 'hist:keys=keys=mm_id,member:bucket=size/0x80000:onchange($bucket)
      .rss_stat_throttled(mm_id,curr,member,size)'
        >> events/kmem/rss_stat/trigger

A use case for using constants with addition/subtraction is not yet
known, but for completeness the use of constants are supported for all
operators.

Link: https://lkml.kernel.org/r/20211025200852.3002369-2-kaleshsingh@google.com

Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 52cfb37353)

[ Kalesh Singh: Resolve minor conflicts in kernel/trace/trace_events_hist.c ]
Bug: 146055070
Bug: 145972256
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I800274b26a6319a8c4af8e9e732235a5cc5bcacf
2021-11-04 17:22:35 +00:00
Kalesh Singh
1d30c0a162 UPSTREAM: tracing/cfi: Fix cmp_entries_* functions signature mismatch
If CONFIG_CFI_CLANG=y, attempting to read an event histogram will cause
the kernel to panic due to failed CFI check.

    1. echo 'hist:keys=common_pid' >> events/sched/sched_switch/trigger
    2. cat events/sched/sched_switch/hist
    3. kernel panics on attempting to read hist

This happens because the sort() function expects a generic
int (*)(const void *, const void *) pointer for the compare function.
To prevent this CFI failure, change tracing map cmp_entries_* function
signatures to match this.

Also, fix the build error reported by the kernel test robot [1].

[1] https://lore.kernel.org/r/202110141140.zzi4dRh4-lkp@intel.com/

Link: https://lkml.kernel.org/r/20211014045217.3265162-1-kaleshsingh@google.com

Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
(cherry picked from commit 7ce1bb83a1)

Bug: 146055070
Bug: 145972256
Bug: 200805622
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I9e0dc6544f6464868fa1c5dbf90666ecdab01ece
2021-11-04 17:15:25 +00:00
Kalesh Singh
7163abc9e4 ANDROID: GKI: Enable HIST_TRIGGERS (x86)
Histogram triggers (already enabled for arm64) will be used
to throttle frequent trace events on Android.

Bug: 146055070
Bug: 145972256
Change-Id: Ia186a79867866e5b4ed37f2cc14be194505f3fba
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
2021-11-04 17:15:17 +00:00
David Brazdil
6d44858773 ANDROID: Enable KVM_S2MPU in gki_defconfig
Enable the KVM S2MPU driver in GKI.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I24285cc85af63b090c9bd0191b3841e44c2d8a0c
2021-11-04 13:38:17 +00:00
David Brazdil
8f23406153 ANDROID: KVM: arm64: Unmap S2MPU MMIO registers from host stage-2
The S2MPU driver needs to protect its MMIO registers from the host.
Implement the host_stage2_adjust_mmio_range callback and restrict
the address range that is about to be mapped in to avoid the known
S2MPU MMIO regions.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Id30eb44c23a153fc3aaabebff8b33d1c7192e95e
2021-11-04 13:38:17 +00:00
David Brazdil
03d4db5339 ANDROID: KVM: arm64: Implement MMIO handler in S2MPU driver
The host should not have access to the vast majority of S2MPU MMIO
registers. Currently it only needs access to fault information, in
the future maybe also performance registers.

Implement an MMIO trap handler for the S2MPU, allowing read-only
access to FAULT_* registers, and a write-only access to
INTERRUPT_CLEAR.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I14d37bb0d5ad591ac3ac372701afd5597e0b3435
2021-11-04 13:38:17 +00:00
David Brazdil
533c59945d ANDROID: KVM: arm64: Unmap S2MPU MMIO regions in MPT
Host stage-2 is used for bookkeeping of ownership of physical pages, but
only for memory and not MMIO. Page tables covering device memory are
created on demand. That is in contrast to the S2MPU MPTs, which are
never discarded and cover the entire physical address space.

The S2MPU MPT should therefore unmap MMIO regions which the host should
not have access to. Currently those are only the S2MPU MMIO registers
themselves.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I85af31d5337459c4877eb90b1c5a2d94f94ab890
2021-11-04 13:38:17 +00:00
David Brazdil
6b75c0e0de ANDROID: KVM: arm64: Add S2MPU kselftest
Add a kselftest which exercises the S2MPU's MPT logic. The functions are
included into a kernel module and exercised in EL1. This is because
testing the EL2 driver itself would require generating DMA traffic and
probing the S2MPU without crashing the system. Instead, the logic is
used on a local FMPT/SMPT and the results are inspected.

Test: run kvm/aarch64/s2mpu kselftest
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ib1e572e3247d864e59fad8a4960e38956237ef8c
2021-11-04 13:38:17 +00:00
David Brazdil
4475d993aa ANDROID: KVM: arm64: Modify S2MPU MPT in 'host_stage2_set_owner'
The 'host_stage2_set_owner' callback indicates that a range of
PA-contiguous pages changed owner. With all devices owned by the host,
the driver sets the protection bits in the corresponding FMPT/SMPT to
either MPT_PROT_RW if owned by the host or MPT_PROT_NONE otherwise.

For each gigabyte region, the implementation will select between 1G and
4K/64K (depending on PAGE_SIZE) mappings and populate the L1ENTRY_ATTR
register or SMPT bitmap, respectivelly.

The driver never dynamically switches between two granularities which
both require a SMPT. This is because the L1ENTRY_ATTR and
L1ENTRY_L2TABLE_ADDR registers would need to be set atomically.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I9d21df4c615e436285a08720216ac5119e1cfeef
2021-11-04 13:38:17 +00:00
David Brazdil
f0e1de52ef ANDROID: KVM: arm64: Set up S2MPU Memory Protection Table
S2MPU Second-level Memory Protection Table is a PA-contiguous buffer
containing an array of 2-bit read/write entries at given granularity
for a given gigabyte physical address space region. The size of SMPT
varies per granularity but at the finest 4K granularity it is 64KB
PA-contiguous, aligned to 64KB.

Allocate sufficient number of SMPT buffers for the S2MPU driver assuming
4K granularity for 4K/16K PAGE_SIZE, and 64K granularity for 64K
PAGE_SIZE. We also assume that all S2MPUs share SMPTs for a given
gigabyte region. There are 34 gigabyte regions that can be set by the
driver (GBs 4-33 always block all traffic).

Hyp takes ownership of the memory in s2mpu_init and assigns pointers to
the buffers to L1ENTRY_L2TABLE_ADDR registers on init and power-on
events. The pointers remain static as the driver will only change
granularity between 1G and 4K/64K (depending on PAGE_SIZE).

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: If27436087ebba1dd0a977960d960d5eaff4279fd
2021-11-04 13:38:17 +00:00
David Brazdil
8ca0b34fe4 ANDROID: KVM: arm64: Reprogram S2MPUs in 'host_smc_handler'
Intercept SMCs known to be used by the host to inform EL3 about power
events, either powering SoC blocks on or off.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I936f7aaecf3ec77ec252f067a3c1571aadba4cdb
2021-11-04 13:38:16 +00:00
David Brazdil
8aa6c440da ANDROID: KVM: arm64: Enable S2MPUs in __pkvm_init_stage2_iommu
Initialize the S2MPU driver in __pkvm_init_stage2_iommu if requested by
the host. The driver sets kvm_iommu_ops and configures all S2MPUs which
are powered on at that point (ie. all S2MPUs on currently supported
devices).

The S2MPU L1ENTRY registers are set to 1G granularity and R/W access.
CTRL0/CTRL1/CFG as set to reasonable defaults, though the code relies on
the reset state blocking all traffic as well.

On fault the S2MPUs are configured to return SLVERR/DECERR (v8/9) to the
master. Interrupts are enabled for all VIDs and trigger an IRQ handler
if EL1 init registered a handler as a result of a DT interrupts entry.

Because the host can configure the SSMTs freely, all permission bits are
configured for all VIDs. For v9 CONTEXT_CFG_VALID_VIDS is set to the
value precomputed at EL1, allocating a context ID to each VID.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I20e658a77c93fa18a07e388d13a639dc67e600d8
2021-11-04 13:38:16 +00:00
David Brazdil
bc80f81582 ANDROID: KVM: arm64: Copy S2MPU configuration to hyp
Create variables in hyp that will hold the DT information about S2MPUs
to use by hyp at runtime. Copy the information from EL1 to EL2.

The EL1 code computes the size of the data and allocates a sufficient
number of pages, which hyp will later take ownership of.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I45ff22ea2049dd285bfcedbad3390baba9fc59a0
2021-11-04 13:38:16 +00:00
David Brazdil
2517c4e5f0 ANDROID: KVM: arm64: Implement IRQ handler for S2MPU faults
The S2MPU can be configured to trigger an interrupt on faults: access
permission (both regular and during page table walks) and if no matching
context ID is found for request's VID (v9 only).

When interrupt information is provided in the S2MPU's DT node, parse the
information and enable an IRQ handler. Later patch will enable the
functionality in the S2MPU.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I5e26a64fc2f09f96f36a93ea9bc5bf3035a71077
2021-11-04 13:38:16 +00:00
David Brazdil
923353be1e ANDROID: KVM: arm64: Allocate context IDs for valid VIDs
S2MPU_CONTEXT_CFG_VALID_VID register must be configured on v9,
allocating a context ID in range 0 to S2MPU_NUM_CONTEXT to each valid
VID. For now assume that all 8 VIDs are valid. This will change once
the hypervisor takes control over SSMT configuration as well.

If there are more VIDs than available context IDs, the driver prints
a warning that DMA may be blocked and continues.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I7a78999b73c4ddf4ff78900a7c91eabe57aad572
2021-11-04 13:38:16 +00:00
David Brazdil
4a7da93bdb ANDROID: KVM: arm64: Read and check S2MPU_VERSION
Read S2MPU_VERSION during driver init and check it against list of
supported versions. The register fields are as follows:
  - MAJOR_ARCH_VER,
  - MINOR_ARCH_VER,
  - REV_ARCH_VER,
  - RTL_VER.
Their exact use is not documented. For now, we mask out RTL_VER and
expect a match on MAJOR_, MINOR_ and REV_ARCH_VER. This may be tweaked
in the future.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I2fbd20ab78a992c8bdb3574a6d480012260c9ded
2021-11-04 13:38:16 +00:00
David Brazdil
4e91a00153 ANDROID: KVM: arm64: Parse S2MPU MMIO region
Start EL1 portion of the S2MPU driver with an init function which
probes the Device tree for nodes compatible with 'google,s2mpu'.
Parse and check the base, size and power domain ID.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ic8b421e20b40b4a9fc5fb268dece00a11e35e3eb
2021-11-04 13:38:16 +00:00
David Brazdil
b2de5483b7 ANDROID: KVM: arm64: Create empty S2MPU driver
Create a skeleton driver for the S2MPU - an EL1 portion called during
KVM init which will parse the DT and configure the kernel, and an EL2
portion which will program the S2MPUs later at runtime. The code is
behind CONFIG_KVM_S2MPU.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ic6a3460cad69fba673754cc1926a5bac88f1fa17
2021-11-04 13:38:15 +00:00
David Brazdil
ff91ee259c ANDROID: dt-bindings: iommu: Add Google S2MPU
Add DeviceTree bindings for Google S2MPU.

Test: n/a
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I002b5df1cac73766179d73ed56fe4945e8e7c2e6
2021-11-04 13:38:15 +00:00
David Brazdil
cc1ad46fb2 ANDROID: KVM: arm64: Add 'host_stage2_adjust_mmio_range' to kvm_iommu_ops
Add a new kvm_iommu_ops hook to the lower-EL instruction/data abort
handler, which allows the IOMMU driver to restrict the region of device
memory that is about to be mapped in the host stage-2.

This can be used by the IOMMU driver to restrict access to the MMIO
registers of the IOMMU itself.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I58f7b7a78795a3bfc4d77f7128e4fdc1110fde58
2021-11-04 13:38:15 +00:00
David Brazdil
25f81ec77b ANDROID: KVM: arm64: Add 'host_mmio_dabt_handler' to kvm_iommu_ops
Add a new kvm_iommu_ops hook which allows the IOMMU driver to handle
data aborts in unmapped device memory regions. If the abort is handled
by the driver, the global abort handler will not attempt to map in the
page.

For example, this enables the IOMMU driver to virtualize access to
the underlying IOMMU hardware, or to allow access to a subset of the
functionality, eg. performance counters.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ie2e38652f0568c27d30190fcc3879592872863ae
2021-11-04 13:37:53 +00:00
David Brazdil
3cd8b5b00b ANDROID: KVM: arm64: Add 'host_stage2_set_owner' to kvm_iommu_ops
Add a new hook to kvm_iommu_ops that is invoked whenever a range of
pages changes their owner in the host stage2. This is currently limited
to finalize_host_mappings, which changes the owner of EL2-mapped pages
from host to hyp.

The driver is expected to apply corresponding changes in the IOMMU it
controls, so that only the new owner can access the page range.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ic5ca01a56344cd0253bf7b71560f057ba0e54d6b
2021-11-04 13:18:30 +00:00
David Brazdil
d2efcdcb2b ANDROID: KVM: arm64: Add 'host_smc_handler' to kvm_iommu_ops
IOMMU drivers need to intercept power management SMCs between the host
and EL3. Add a hook to hyp's 'handle_host_smc'.

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Ief42370ac371f6b3b87a0d46c00acfffece47052
2021-11-04 13:18:30 +00:00
David Brazdil
79775d0225 ANDROID: KVM: arm64: Introduce IOMMU driver infrastructure
Bootstrap infrastructure for IOMMU drivers by introducing kvm_iommu_ops
struct in EL2 that is populated based on a iommu_driver parameter to
__pkvm_init hypercall and selected in EL1 early init.

An 'init' operation is called in __pkvm_init_finalise, giving the driver
an opportunity to initialize itself in EL2 and create any EL2 mappings
that it will need. 'init' is specifically called before
'finalize_host_mappings' so that:
  (a) pages mapped by the driver change owner to hyp,
  (b) ownership changes in 'finalize_host_mappings' get reflected in
      IOMMU mappings (added in a future patch).

Test: builds, boots
Bug: 190463801
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: Icd496d193c0bbf811406c3a8d4b94610e2a03197
2021-11-04 13:18:30 +00:00
Woody Lin
fddf813941 ANDROID: debug_symbols: Add show_mem
Add show_mem symbol which will be used by the hard-lockup
debugging module to debug_symbols driver.

Bug: 199478662
Signed-off-by: Woody Lin <woodylin@google.com>
Change-Id: I479700e9f1428b4e1192881b4e3b67c9e43afbeb
2021-11-04 08:42:54 +00:00
Bart Van Assche
4295353099 ANDROID: scsi: ufs: Align the HPB calls with the upstream code
The Android UFS driver calls the HPB functions in a different way than
the upstream UFS driver. Align the Android UFS driver with the upstream
driver.

Bug: 204438323
Change-Id: Ia98f96c86768db8f995ba31cbac20e76e7201e2b
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:37 -07:00
Bart Van Assche
bdb2117c28 ANDROID: scsi: ufs: Re-enable runtime power management
Android commit e7d848c921 ("FROMLIST: scsi: ufs: Fix deadlock while
suspending ufs host") disabled runtime power management. Re-enable
runtime power management since runtime power management support has
been fixed after that commit was merged.

Bug: 204438323
Change-Id: I4d473262e90f242bc55eaa07b8a3bf7ab47c1a65
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:37 -07:00
Bart Van Assche
568dd97549 ANDROID: scsi: ufs: Remove the ufs_rpmb_wlun_template
The upstream commit edc0596cc0 ("scsi: ufs: core: Stop clearing UNIT
ATTENTIONS") removed the ufs_rpmb_wlun_template but its Android backport
not. Hence this patch that removes the ufs_rpmb_wlun_template.

Bug: 204438323
Change-Id: I86206399a9745e26ad5f84c2cac2179a37ba80e7
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:37 -07:00
Bart Van Assche
6e41c820f6 ANDROID: scsi: ufs: Remove a local variable from ufshcd_abort()
Bring the Android code in sync with the upstream code. This patch does
not change any functionality.

Bug: 204438323
Change-Id: I8fe9f9f3c7d39feb1e211a6b6dc00d6e638ef1b4
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:37 -07:00
Bart Van Assche
69d296ea60 ANDROID: scsi: ufs: Remove a blank line
There is no blank line in the upstream code above the luns_avail
declaration. Hence this patch.

Bug: 204438323
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: I79927cc14cc011c87eb96ca8847f0bd6a046bb40
2021-11-03 13:31:37 -07:00
Adrian Hunter
23dd19247d UPSTREAM: scsi: ufs: core: Do not exit ufshcd_err_handler() unless operational or dead
Callers of ufshcd_err_handler() expect it to return in an operational
state. However, the code does not check the state before exiting.

Add a check for the state and perform retries until either success or the
maximum number of retries is reached.

Link: https://lore.kernel.org/r/20211002154550.128511-3-adrian.hunter@intel.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Change-Id: Ib3818dd4fd956837418218b37f39ee837438a2be
Bug: 204438323
(cherry picked from commit 87bf6a6bbe)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:37 -07:00
Adrian Hunter
ccba7d5b8e UPSTREAM: scsi: ufs: core: Do not exit ufshcd_reset_and_restore() unless operational or dead
Callers of ufshcd_reset_and_restore() expect it to return in an operational
state. However, the code only checks direct errors and so the ufshcd_state
may not be UFSHCD_STATE_OPERATIONAL due to error interrupts.

Fix by also checking ufshcd_state, still allowing non-fatal errors which
are left for the error handler to deal with.

Link: https://lore.kernel.org/r/20211002154550.128511-2-adrian.hunter@intel.com
Reviewed-by: Avri altman <avri.altman@wdc.com>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit 54a4045342)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: Ibfa8f43aaa256c96efbfcb8e185067beb655d603
2021-11-03 13:31:36 -07:00
Bean Huo
e2919fba76 UPSTREAM: scsi: ufs: core: Remove return statement in void function
Return statement is not useful at the end of "void" function.

Link: https://lore.kernel.org/r/20210929200640.828611-4-huobean@gmail.com
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit f44abcfc3f)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: I33a7a6dfbecb50baf017185b30e97089bb266e71
2021-11-03 13:31:36 -07:00
Bean Huo
1da235a555 UPSTREAM: scsi: ufs: core: Fix ufshcd_probe_hba() prototype to match the definition
Since commit 568dd99596 ("scsi: ufs: Rename the second ufshcd_probe_hba()
argument"), the second ufshcd_probe_hba() argument has been changed to
init_dev_params.

Link: https://lore.kernel.org/r/20210929200640.828611-3-huobean@gmail.com
Fixes: 568dd99596 ("scsi: ufs: Rename the second ufshcd_probe_hba() argument")
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Change-Id: I04365efcd6f0e4154f280eb66bd01590100009ad
Bug: 204438323
(cherry picked from commit 68444d73d6)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2021-11-03 13:31:36 -07:00
Anders Roxell
558e6162fe UPSTREAM: scsi: ufs: core: SCSI_UFS_HWMON depends on HWMON=y
When building an allmodconfig kernel, the following build error shows up:

aarch64-linux-gnu-ld: drivers/scsi/ufs/ufs-hwmon.o: in function `ufs_hwmon_probe':
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:177: undefined reference to `hwmon_device_register_with_info'
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:177:(.text+0x510): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol `hwmon_device_register_with_info'
aarch64-linux-gnu-ld: drivers/scsi/ufs/ufs-hwmon.o: in function `ufs_hwmon_remove':
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:195: undefined reference to `hwmon_device_unregister'
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:195:(.text+0x5c8): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol `hwmon_device_unregister'
aarch64-linux-gnu-ld: drivers/scsi/ufs/ufs-hwmon.o: in function `ufs_hwmon_notify_event':
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:206: undefined reference to `hwmon_notify_event'
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:206:(.text+0x64c): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol `hwmon_notify_event'
aarch64-linux-gnu-ld: /home/anders/src/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:209: undefined reference to `hwmon_notify_event'
/kernel/next/drivers/scsi/ufs/ufs-hwmon.c:209:(.text+0x66c): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol `hwmon_notify_event'

Since SCSI_UFS_HWMON can't be built as a module, SCSI_UFS_HWMON has to
depend on HWMON=y.

Link: https://lore.kernel.org/r/20210927084615.1938432-1-anders.roxell@linaro.org
Fixes: e88e2d3220 ("scsi: ufs: core: Probe for temperature notification support")
Also-reported-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org> # build-tested
Acked-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit 60c98a87fc)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: I233fa50a3b5ba54b609c05b3b271276db6339afb
2021-11-03 13:31:36 -07:00
Krzysztof Kozlowski
61a867f350 UPSTREAM: scsi: ufs: exynos: Unify naming
Use "Samsung" and "Exynos", not the uppercase versions.

Link: https://lore.kernel.org/r/20210924132658.109814-2-krzysztof.kozlowski@canonical.com
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit ce580e47e8)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: I8baf39fa0c17a5a3760836067e993a8fbbdf5398
2021-11-03 13:31:36 -07:00
Avri Altman
061f867509 UPSTREAM: scsi: ufs: core: Add temperature notification exception handling
The device may notify the host of an extreme temperature by using the
exception event mechanism. The exception can be raised when the device’s
Tcase temperature is either too high or too low.

It is essentially up to the platform to decide what further actions need to
be taken. leave a placeholder for a designated vop for that.

Link: https://lore.kernel.org/r/20210915060407.40-3-avri.altman@wdc.com
Reviewed-by: Guenter Roeck <linux@roeck-us.net>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Daejun Park <daejun7.park@samsung.com>
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit 322c4b29ee)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: Iaf049a274fb4be5c3bffd92fa2651ed50b16ca29
2021-11-03 13:31:36 -07:00
Avri Altman
bbce73fc60 UPSTREAM: scsi: ufs: core: Probe for temperature notification support
Probe the dExtendedUFSFeaturesSupport register for the device's temperature
notification support and, if supported, add a hardware monitor device.

Link: https://lore.kernel.org/r/20210915060407.40-2-avri.altman@wdc.com
Reviewed-by: Guenter Roeck <linux@roeck-us.net>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Daejun Park <daejun7.park@samsung.com>
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 204438323
(cherry picked from commit e88e2d3220)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Change-Id: I40639256cf1d2e70d5e80b09b03def001a9a5487
2021-11-03 13:31:36 -07:00