shinys000114/linux
1
0
Fork 0
mirror of https://github.com/hardkernel/linux.git synced 2026-06-10 21:07:02 +09:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'android12-5.10' into 'android13-5.10'

Browse Source 
Big sync after the KMI freeze of July 14.

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I231ed34f3772d3d2741cfbdcaec97eb5b5eb238c
This commit is contained in:
Greg Kroah-Hartman
2021-07-21 14:24:16 +02:00
parent f932f5456f fdb1cfe2d3
commit 2b034856a5
357 changed files with 23659 additions and 2984 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
Documentation/ABI/testing/sysfs-devices-system-cpu
View File

@@ -648,3 +648,21 @@ Description: SPURR ticks for cpuX when it was idle.
This sysfs interface exposes the number of SPURR ticks
for cpuX when it was idle.
What: /sys/devices/system/cpu/cpuX/mte_tcf_preferred
Date: July 2021
Contact: Linux ARM Kernel Mailing list <linux-arm-kernel@lists.infradead.org>
Description: Preferred MTE tag checking mode
When a user program specifies more than one MTE tag checking
mode, this sysfs node is used to specify which mode should
be preferred when running on that CPU. Possible values:
================ ==============================================
"sync" Prefer synchronous mode
"async" Prefer asynchronous mode
================ ==============================================
Changes to this sysfs node may not take effect immediately.
See also: Documentation/arm64/memory-tagging-extension.rst

362
Documentation/ABI/testing/sysfs-driver-ufs
View File

@@ -987,6 +987,132 @@ Description: This entry shows the target state of an UFS UIC link
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/monitor_enable
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the status of performance monitor enablement
and it can be used to start/stop the monitor. When the monitor
is stopped, the performance data collected is also cleared.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/monitor_chunk_size
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file tells the monitor to focus on requests transferring
data of specific chunk size (in Bytes). 0 means any chunk size.
It can only be changed when monitor is disabled.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/read_total_sectors
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows how many sectors (in 512 Bytes) have been
sent from device to host after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/read_total_busy
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows how long (in micro seconds) has been spent
sending data from device to host after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/read_nr_requests
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows how many read requests have been sent after
monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/read_req_latency_max
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the maximum latency (in micro seconds) of
read requests after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/read_req_latency_min
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the minimum latency (in micro seconds) of
read requests after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/read_req_latency_avg
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the average latency (in micro seconds) of
read requests after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/read_req_latency_sum
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the total latency (in micro seconds) of
read requests sent after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/write_total_sectors
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows how many sectors (in 512 Bytes) have been sent
from host to device after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/write_total_busy
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows how long (in micro seconds) has been spent
sending data from host to device after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/write_nr_requests
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows how many write requests have been sent after
monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/write_req_latency_max
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the maximum latency (in micro seconds) of write
requests after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/write_req_latency_min
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the minimum latency (in micro seconds) of write
requests after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/write_req_latency_avg
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the average latency (in micro seconds) of write
requests after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/monitor/write_req_latency_sum
Date: January 2021
Contact: Can Guo <cang@codeaurora.org>
Description: This file shows the total latency (in micro seconds) of write
requests after monitor gets started.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/device_descriptor/wb_presv_us_en
Date: June 2020
Contact: Asutosh Das <asutoshd@codeaurora.org>
@@ -1153,3 +1279,239 @@ Description: This entry shows the configured size of WriteBooster buffer.
0400h corresponds to 4GB.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/device_descriptor/hpb_version
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the HPB specification version.
The full information about the descriptor could be found at UFS
HPB (Host Performance Booster) Extension specifications.
Example: version 1.2.3 = 0123h
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/device_descriptor/hpb_control
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows an indication of the HPB control mode.
00h: Host control mode
01h: Device control mode
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_region_size
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the bHPBRegionSize which can be calculated
as in the following (in bytes):
HPB Region size = 512B * 2^bHPBRegionSize
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_number_lu
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the maximum number of HPB LU supported by
the device.
00h: HPB is not supported by the device.
01h ~ 20h: Maximum number of HPB LU supported by the device
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_subregion_size
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the bHPBSubRegionSize, which can be
calculated as in the following (in bytes) and shall be a multiple of
logical block size:
HPB Sub-Region size = 512B x 2^bHPBSubRegionSize
bHPBSubRegionSize shall not exceed bHPBRegionSize.
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_max_active_regions
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the maximum number of active HPB regions that
is supported by the device.
The file is read only.
What: /sys/class/scsi_device/*/device/unit_descriptor/hpb_lu_max_active_regions
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the maximum number of HPB regions assigned to
the HPB logical unit.
The file is read only.
What: /sys/class/scsi_device/*/device/unit_descriptor/hpb_pinned_region_start_offset
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the start offset of HPB pinned region.
The file is read only.
What: /sys/class/scsi_device/*/device/unit_descriptor/hpb_number_pinned_regions
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the number of HPB pinned regions assigned to
the HPB logical unit.
The file is read only.
What: /sys/class/scsi_device/*/device/hpb_stats/hit_cnt
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the number of reads that changed to HPB read.
The file is read only.
What: /sys/class/scsi_device/*/device/hpb_stats/miss_cnt
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the number of reads that cannot be changed to
HPB read.
The file is read only.
What: /sys/class/scsi_device/*/device/hpb_stats/rb_noti_cnt
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the number of response UPIUs that has
recommendations for activating sub-regions and/or inactivating region.
The file is read only.
What: /sys/class/scsi_device/*/device/hpb_stats/rb_active_cnt
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the number of active sub-regions recommended by
response UPIUs.
The file is read only.
What: /sys/class/scsi_device/*/device/hpb_stats/rb_inactive_cnt
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the number of inactive regions recommended by
response UPIUs.
The file is read only.
What: /sys/class/scsi_device/*/device/hpb_stats/map_req_cnt
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the number of read buffer commands for
activating sub-regions recommended by response UPIUs.
The file is read only.
What: /sys/class/scsi_device/*/device/hpb_params/requeue_timeout_ms
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the requeue timeout threshold for write buffer
command in ms. This value can be changed by writing proper integer to
this entry.
What: /sys/bus/platform/drivers/ufshcd/*/attributes/max_data_size_hpb_single_cmd
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the maximum HPB data size for using single HPB
command.
=== ========
00h 4KB
01h 8KB
02h 12KB
...
FFh 1024KB
=== ========
The file is read only.
What: /sys/bus/platform/drivers/ufshcd/*/flags/hpb_enable
Date: June 2021
Contact: Daejun Park <daejun7.park@samsung.com>
Description: This entry shows the status of HPB.
== ============================
0 HPB is not enabled.
1 HPB is enabled
== ============================
The file is read only.
What: /sys/class/scsi_device/*/device/hpb_param_sysfs/activation_thld
Date: February 2021
Contact: Avri Altman <avri.altman@wdc.com>
Description: In host control mode, reads are the major source of activation
trials. once this threshold hs met, the region is added to the
"to-be-activated" list. Since we reset the read counter upon
write, this include sending a rb command updating the region
ppn as well.
What: /sys/class/scsi_device/*/device/hpb_param_sysfs/normalization_factor
Date: February 2021
Contact: Avri Altman <avri.altman@wdc.com>
Description: In host control mode, We think of the regions as "buckets".
Those buckets are being filled with reads, and emptied on write.
We use entries_per_srgn - the amount of blocks in a subregion as
our bucket size. This applies because HPB1.0 only concern a
single-block reads. Once the bucket size is crossed, we trigger
a normalization work - not only to avoid overflow, but mainly
because we want to keep those counters normalized, as we are
using those reads as a comparative score, to make various decisions.
The normalization is dividing (shift right) the read counter by
the normalization_factor. If during consecutive normalizations
an active region has exhaust its reads - inactivate it.
What: /sys/class/scsi_device/*/device/hpb_param_sysfs/eviction_thld_enter
Date: February 2021
Contact: Avri Altman <avri.altman@wdc.com>
Description: Region deactivation is often due to the fact that eviction took
place: a region become active on the expense of another. This is
happening when the max-active-regions limit has crossed.
In host mode, eviction is considered an extreme measure. We
want to verify that the entering region has enough reads, and
the exiting region has much less reads. eviction_thld_enter is
the min reads that a region must have in order to be considered
as a candidate to evict other region.
What: /sys/class/scsi_device/*/device/hpb_param_sysfs/eviction_thld_exit
Date: February 2021
Contact: Avri Altman <avri.altman@wdc.com>
Description: same as above for the exiting region. A region is consider to
be a candidate to be evicted, only if it has less reads than
eviction_thld_exit.
What: /sys/class/scsi_device/*/device/hpb_param_sysfs/read_timeout_ms
Date: February 2021
Contact: Avri Altman <avri.altman@wdc.com>
Description: In order not to hang on to “cold” regions, we shall inactivate
a region that has no READ access for a predefined amount of
time - read_timeout_ms. If read_timeout_ms has expired, and the
region is dirty - it is less likely that we can make any use of
HPB-READing it. So we inactivate it. Still, deactivation has
its overhead, and we may still benefit from HPB-READing this
region if it is clean - see read_timeout_expiries.
What: /sys/class/scsi_device/*/device/hpb_param_sysfs/read_timeout_expiries
Date: February 2021
Contact: Avri Altman <avri.altman@wdc.com>
Description: if the region read timeout has expired, but the region is clean,
just re-wind its timer for another spin. Do that as long as it
is clean and did not exhaust its read_timeout_expiries threshold.
What: /sys/class/scsi_device/*/device/hpb_param_sysfs/timeout_polling_interval_ms
Date: February 2021
Contact: Avri Altman <avri.altman@wdc.com>
Description: the frequency in which the delayed worker that checks the
read_timeouts is awaken.
What: /sys/class/scsi_device/*/device/hpb_param_sysfs/inflight_map_req
Date: February 2021
Contact: Avri Altman <avri.altman@wdc.com>
Description: in host control mode the host is the originator of map requests.
To not flood the device with map requests, use a simple throttling
mechanism that limits the number of inflight map requests.

43
Documentation/ABI/testing/sysfs-fs-f2fs
View File

@@ -203,7 +203,34 @@ Description: Shows total written kbytes issued to disk.
What: /sys/fs/f2fs/<disk>/features
Date: July 2017
Contact: "Jaegeuk Kim" <jaegeuk@kernel.org>
Description: Shows all enabled features in current device.
Description: <deprecated: should use /sys/fs/f2fs/<disk>/feature_list/
Shows all enabled features in current device.
Supported features:
encryption, blkzoned, extra_attr, projquota, inode_checksum,
flexible_inline_xattr, quota_ino, inode_crtime, lost_found,
verity, sb_checksum, casefold, readonly, compression, pin_file.
What: /sys/fs/f2fs/<disk>/feature_list/
Date: June 2021
Contact: "Jaegeuk Kim" <jaegeuk@kernel.org>
Description: Expand /sys/fs/f2fs/<disk>/features to meet sysfs rule.
Supported on-disk features:
encryption, block_zoned (aka blkzoned), extra_attr,
project_quota (aka projquota), inode_checksum,
flexible_inline_xattr, quota_ino, inode_crtime, lost_found,
verity, sb_checksum, casefold, readonly, compression.
Note that, pin_file is moved into /sys/fs/f2fs/features/.
What: /sys/fs/f2fs/features/
Date: July 2017
Contact: "Jaegeuk Kim" <jaegeuk@kernel.org>
Description: Shows all enabled kernel features.
Supported features:
encryption, block_zoned, extra_attr, project_quota,
inode_checksum, flexible_inline_xattr, quota_ino,
inode_crtime, lost_found, verity, sb_checksum,
casefold, readonly, compression, test_dummy_encryption_v2,
atomic_write, pin_file, encrypted_casefold.
What: /sys/fs/f2fs/<disk>/inject_rate
Date: May 2016
@@ -466,3 +493,17 @@ Contact: "Chao Yu" <yuchao0@huawei.com>
Description: When ATGC is on, it controls age threshold to bypass GCing young
candidates whose age is not beyond the threshold, by default it was
initialized as 604800 seconds (equals to 7 days).
What: /sys/fs/f2fs/<disk>/gc_reclaimed_segments
Date: July 2021
Contact: "Daeho Jeong" <daehojeong@google.com>
Description: Show how many segments have been reclaimed by GC during a specific
GC mode (0: GC normal, 1: GC idle CB, 2: GC idle greedy,
3: GC idle AT, 4: GC urgent high, 5: GC urgent low)
You can re-initialize this value to "0".
What: /sys/fs/f2fs/<disk>/gc_segment_mode
Date: July 2021
Contact: "Daeho Jeong" <daehojeong@google.com>
Description: You can control for which gc mode the "gc_reclaimed_segments" node shows.
Refer to the description of the modes in "gc_reclaimed_segments".

35
Documentation/ABI/testing/sysfs-kernel-dmabuf-buffers
View File

@@ -22,38 +22,3 @@ KernelVersion: v5.12
Contact: Hridya Valsaraju <hridya@google.com>
Description: This file is read-only and specifies the size of the DMA-BUF in
bytes.
What: /sys/kernel/dmabuf/buffers/<inode_number>/attachments
Date: January 2021
KernelVersion: v5.12
Contact: Hridya Valsaraju <hridya@google.com>
Description: This directory will contain subdirectories representing every
attachment of the DMA-BUF.
What: /sys/kernel/dmabuf/buffers/<inode_number>/attachments/<attachment_uid>
Date: January 2021
KernelVersion: v5.12
Contact: Hridya Valsaraju <hridya@google.com>
Description: This directory will contain information on the attaching device
and the number of current distinct device mappings.
What: /sys/kernel/dmabuf/buffers/<inode_number>/attachments/<attachment_uid>/device
Date: January 2021
KernelVersion: v5.12
Contact: Hridya Valsaraju <hridya@google.com>
Description: This file is read-only and is a symlink to the attaching devices's
sysfs entry.
What: /sys/kernel/dmabuf/buffers/<inode_number>/attachments/<attachment_uid>/map_counter
Date: January 2021
KernelVersion: v5.12
Contact: Hridya Valsaraju <hridya@google.com>
Description: This file is read-only and contains a map_counter indicating the
number of distinct device mappings of the attachment.
What: /sys/kernel/dmabuf/buffers/<inode_number>/mmap_count
Date: January 2021
KernelVersion: v5.10
Contact: Kalesh Singh <kaleshsingh@google.com>
Description: This file is read-only and contains a counter indicating the
number of times the buffer has been mmap().

55
Documentation/admin-guide/cgroup-v2.rst
View File

@@ -54,6 +54,7 @@ v1 is available under :ref:`Documentation/admin-guide/cgroup-v1/index.rst <cgrou
5-3-3. IO Latency
5-3-3-1. How IO Latency Throttling Works
5-3-3-2. IO Latency Interface Files
5-3-4. IO Priority
5-4. PID
5-4-1. PID Interface Files
5-5. Cpuset
@@ -1848,6 +1849,60 @@ IO Latency Interface Files
duration of time between evaluation events. Windows only elapse
with IO activity. Idle periods extend the most recent window.
IO Priority
~~~~~~~~~~~
A single attribute controls the behavior of the I/O priority cgroup policy,
namely the blkio.prio.class attribute. The following values are accepted for
that attribute:
no-change
Do not modify the I/O priority class.
none-to-rt
For requests that do not have an I/O priority class (NONE),
change the I/O priority class into RT. Do not modify
the I/O priority class of other requests.
restrict-to-be
For requests that do not have an I/O priority class or that have I/O
priority class RT, change it into BE. Do not modify the I/O priority
class of requests that have priority class IDLE.
idle
Change the I/O priority class of all requests into IDLE, the lowest
I/O priority class.
The following numerical values are associated with the I/O priority policies:
+-------------+---+
| no-change | 0 |
+-------------+---+
| none-to-rt | 1 |
+-------------+---+
| rt-to-be | 2 |
+-------------+---+
| all-to-idle | 3 |
+-------------+---+
The numerical value that corresponds to each I/O priority class is as follows:
+-------------------------------+---+
| IOPRIO_CLASS_NONE | 0 |
+-------------------------------+---+
| IOPRIO_CLASS_RT (real-time) | 1 |
+-------------------------------+---+
| IOPRIO_CLASS_BE (best effort) | 2 |
+-------------------------------+---+
| IOPRIO_CLASS_IDLE | 3 |
+-------------------------------+---+
The algorithm to set the I/O priority class for a request is as follows:
- Translate the I/O priority class policy into a number.
- Change the request I/O priority class into the maximum of the I/O priority
class policy number and the numerical I/O priority class.
PID
---

9
Documentation/admin-guide/kernel-parameters.txt
View File

@@ -513,16 +513,21 @@
ccw_timeout_log [S390]
See Documentation/s390/common_io.rst for details.
cgroup_disable= [KNL] Disable a particular controller
Format: {name of the controller(s) to disable}
cgroup_disable= [KNL] Disable a particular controller or optional feature
Format: {name of the controller(s) or feature(s) to disable}
The effects of cgroup_disable=foo are:
- foo isn't auto-mounted if you mount all cgroups in
a single hierarchy
- foo isn't visible as an individually mountable
subsystem
- if foo is an optional feature then the feature is
disabled and corresponding cgroup files are not
created
{Currently only "memory" controller deal with this and
cut the overhead, others just disable the usage. So
only cgroup_disable=memory is actually worthy}
Specifying "pressure" disables per-cgroup pressure
stall information accounting feature
cgroup_no_v1= [KNL] Disable cgroup controllers and named hierarchies in v1
Format: { { controller | "all" | "named" }

3
Documentation/admin-guide/sysctl/vm.rst
View File

@@ -127,7 +127,8 @@ compaction_proactiveness
This tunable takes a value in the range [0, 100] with a default value of
20. This tunable determines how aggressively compaction is done in the
background. Setting it to 0 disables proactive compaction.
background. On write of non zero value to this tunable will immediately
trigger the proactive compaction. Setting it to 0 disables proactive compaction.
Note that compaction has a non-trivial system-wide impact as pages
belonging to different processes are moved around, which could also lead

48
Documentation/arm64/memory-tagging-extension.rst
View File

@@ -77,14 +77,20 @@ configurable behaviours:
address is unknown).
The user can select the above modes, per thread, using the
``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call where
``flags`` contain one of the following values in the ``PR_MTE_TCF_MASK``
``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call where ``flags``
contains any number of the following values in the ``PR_MTE_TCF_MASK``
bit-field:
- ``PR_MTE_TCF_NONE`` - *Ignore* tag check faults
- ``PR_MTE_TCF_NONE``  - *Ignore* tag check faults
(ignored if combined with other options)
- ``PR_MTE_TCF_SYNC`` - *Synchronous* tag check fault mode
- ``PR_MTE_TCF_ASYNC`` - *Asynchronous* tag check fault mode
If no modes are specified, tag check faults are ignored. If a single
mode is specified, the program will run in that mode. If multiple
modes are specified, the mode is selected as described in the "Per-CPU
preferred tag checking modes" section below.
The current tag check fault mode can be read using the
``prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0)`` system call.
@@ -120,13 +126,39 @@ in the ``PR_MTE_TAG_MASK`` bit-field.
interface provides an include mask. An include mask of ``0`` (exclusion
mask ``0xffff``) results in the CPU always generating tag ``0``.
Per-CPU preferred tag checking mode
-----------------------------------
On some CPUs the performance of MTE in stricter tag checking modes
is similar to that of less strict tag checking modes. This makes it
worthwhile to enable stricter checks on those CPUs when a less strict
checking mode is requested, in order to gain the error detection
benefits of the stricter checks without the performance downsides. To
support this scenario, a privileged user may configure a stricter
tag checking mode as the CPU's preferred tag checking mode.
The preferred tag checking mode for each CPU is controlled by
``/sys/devices/system/cpu/cpu<N>/mte_tcf_preferred``, to which a
privileged user may write the value ``async`` or ``sync``. The default
preferred mode for each CPU is ``async``.
To allow a program to potentially run in the CPU's preferred tag
checking mode, the user program may set multiple tag check fault mode
bits in the ``flags`` argument to the ``prctl(PR_SET_TAGGED_ADDR_CTRL,
flags, 0, 0, 0)`` system call. If the CPU's preferred tag checking
mode is in the task's set of provided tag checking modes (this will
always be the case at present because the kernel only supports two
tag checking modes, but future kernels may support more modes), that
mode will be selected. Otherwise, one of the modes in the task's mode
set will be selected in a currently unspecified manner.
Initial process state
---------------------
On ``execve()``, the new process has the following configuration:
- ``PR_TAGGED_ADDR_ENABLE`` set to 0 (disabled)
- Tag checking mode set to ``PR_MTE_TCF_NONE``
- No tag checking modes are selected (tag check faults ignored)
- ``PR_MTE_TAG_MASK`` set to 0 (all tags excluded)
- ``PSTATE.TCO`` set to 0
- ``PROT_MTE`` not set on any of the initial memory maps
@@ -251,11 +283,13 @@ Example of correct usage
return EXIT_FAILURE;
/*
* Enable the tagged address ABI, synchronous MTE tag check faults and
* allow all non-zero tags in the randomly generated set.
* Enable the tagged address ABI, synchronous or asynchronous MTE
* tag check faults (based on per-CPU preference) and allow all
* non-zero tags in the randomly generated set.
*/
if (prctl(PR_SET_TAGGED_ADDR_CTRL,
PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | (0xfffe << PR_MTE_TAG_SHIFT),
PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | PR_MTE_TCF_ASYNC |
(0xfffe << PR_MTE_TAG_SHIFT),
0, 0, 0)) {
perror("prctl() failed");
return EXIT_FAILURE;

34
Documentation/arm64/pointer-authentication.rst
View File

@@ -107,3 +107,37 @@ filter out the Pointer Authentication system key registers from
KVM_GET/SET_REG_* ioctls and mask those features from cpufeature ID
register. Any attempt to use the Pointer Authentication instructions will
result in an UNDEFINED exception being injected into the guest.
Enabling and disabling keys
---------------------------
The prctl PR_PAC_SET_ENABLED_KEYS allows the user program to control which
PAC keys are enabled in a particular task. It takes two arguments, the
first being a bitmask of PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY
and PR_PAC_APDBKEY specifying which keys shall be affected by this prctl,
and the second being a bitmask of the same bits specifying whether the key
should be enabled or disabled. For example::
prctl(PR_PAC_SET_ENABLED_KEYS,
PR_PAC_APIAKEY | PR_PAC_APIBKEY | PR_PAC_APDAKEY | PR_PAC_APDBKEY,
PR_PAC_APIBKEY, 0, 0);
disables all keys except the IB key.
The main reason why this is useful is to enable a userspace ABI that uses PAC
instructions to sign and authenticate function pointers and other pointers
exposed outside of the function, while still allowing binaries conforming to
the ABI to interoperate with legacy binaries that do not sign or authenticate
pointers.
The idea is that a dynamic loader or early startup code would issue this
prctl very early after establishing that a process may load legacy binaries,
but before executing any PAC instructions.
For compatibility with previous kernel versions, processes start up with IA,
IB, DA and DB enabled, and are reset to this state on exec(). Processes created
via fork() and clone() inherit the key enabled state from the calling process.
It is recommended to avoid disabling the IA key, as this has higher performance
overhead than disabling any of the other keys.

6
Documentation/devicetree/bindings/arm/ete.yaml
View File

@@ -37,7 +37,11 @@ properties:
out-ports:
description: |
Output connections from the ETE to legacy CoreSight trace bus.
$ref: /schemas/graph.yaml#/properties/port
$ref: /schemas/graph.yaml#/properties/ports
properties:
port:
description: Output connection from the ETE to legacy CoreSight Trace bus.
$ref: /schemas/graph.yaml#/properties/port required:
required:
- compatible

128
Documentation/devicetree/bindings/usb/dwc3.txt
View File

@@ -1,128 +0,0 @@
synopsys DWC3 CORE
DWC3- USB3 CONTROLLER. Complies to the generic USB binding properties
as described in 'usb/generic.txt'
Required properties:
- compatible: must be "snps,dwc3"
- reg : Address and length of the register set for the device
- interrupts: Interrupts used by the dwc3 controller.
- clock-names: list of clock names. Ideally should be "ref",
"bus_early", "suspend" but may be less or more.
- clocks: list of phandle and clock specifier pairs corresponding to
entries in the clock-names property.
Exception for clocks:
clocks are optional if the parent node (i.e. glue-layer) is compatible to
one of the following:
"cavium,octeon-7130-usb-uctl"
"qcom,dwc3"
"samsung,exynos5250-dwusb3"
"samsung,exynos5433-dwusb3"
"samsung,exynos7-dwusb3"
"sprd,sc9860-dwc3"
"st,stih407-dwc3"
"ti,am437x-dwc3"
"ti,dwc3"
"ti,keystone-dwc3"
"rockchip,rk3399-dwc3"
"xlnx,zynqmp-dwc3"
Optional properties:
- usb-phy : array of phandle for the PHY device. The first element
in the array is expected to be a handle to the USB2/HS PHY and
the second element is expected to be a handle to the USB3/SS PHY
- phys: from the *Generic PHY* bindings
- phy-names: from the *Generic PHY* bindings; supported names are "usb2-phy"
or "usb3-phy".
- resets: set of phandle and reset specifier pairs
- snps,usb2-lpm-disable: indicate if we don't want to enable USB2 HW LPM
- snps,usb3_lpm_capable: determines if platform is USB3 LPM capable
- snps,dis-start-transfer-quirk: when set, disable isoc START TRANSFER command
failure SW work-around for DWC_usb31 version 1.70a-ea06
and prior.
- snps,disable_scramble_quirk: true when SW should disable data scrambling.
Only really useful for FPGA builds.
- snps,has-lpm-erratum: true when DWC3 was configured with LPM Erratum enabled
- snps,lpm-nyet-threshold: LPM NYET threshold
- snps,u2exit_lfps_quirk: set if we want to enable u2exit lfps quirk
- snps,u2ss_inp3_quirk: set if we enable P3 OK for U2/SS Inactive quirk
- snps,req_p1p2p3_quirk: when set, the core will always request for
P1/P2/P3 transition sequence.
- snps,del_p1p2p3_quirk: when set core will delay P1/P2/P3 until a certain
amount of 8B10B errors occur.
- snps,del_phy_power_chg_quirk: when set core will delay PHY power change
from P0 to P1/P2/P3.
- snps,lfps_filter_quirk: when set core will filter LFPS reception.
- snps,rx_detect_poll_quirk: when set core will disable a 400us delay to start
Polling LFPS after RX.Detect.
- snps,tx_de_emphasis_quirk: when set core will set Tx de-emphasis value.
- snps,tx_de_emphasis: the value driven to the PHY is controlled by the
LTSSM during USB3 Compliance mode.
- snps,dis_u3_susphy_quirk: when set core will disable USB3 suspend phy.
- snps,dis_u2_susphy_quirk: when set core will disable USB2 suspend phy.
- snps,dis_enblslpm_quirk: when set clears the enblslpm in GUSB2PHYCFG,
disabling the suspend signal to the PHY.
- snps,dis-u1-entry-quirk: set if link entering into U1 needs to be disabled.
- snps,dis-u2-entry-quirk: set if link entering into U2 needs to be disabled.
- snps,dis_rxdet_inp3_quirk: when set core will disable receiver detection
in PHY P3 power state.
- snps,dis-u2-freeclk-exists-quirk: when set, clear the u2_freeclk_exists
in GUSB2PHYCFG, specify that USB2 PHY doesn't provide
a free-running PHY clock.
- snps,dis-del-phy-power-chg-quirk: when set core will change PHY power
from P0 to P1/P2/P3 without delay.
- snps,dis-tx-ipgap-linecheck-quirk: when set, disable u2mac linestate check
during HS transmit.
- snps,parkmode-disable-ss-quirk: when set, all SuperSpeed bus instances in
park mode are disabled.
- snps,dis_metastability_quirk: when set, disable metastability workaround.
CAUTION: use only if you are absolutely sure of it.
- snps,dis-split-quirk: when set, change the way URBs are handled by the
driver. Needed to avoid -EPROTO errors with usbhid
on some devices (Hikey 970).
- snps,is-utmi-l1-suspend: true when DWC3 asserts output signal
utmi_l1_suspend_n, false when asserts utmi_sleep_n
- snps,hird-threshold: HIRD threshold
- snps,hsphy_interface: High-Speed PHY interface selection between "utmi" for
UTMI+ and "ulpi" for ULPI when the DWC_USB3_HSPHY_INTERFACE has value 3.
- snps,quirk-frame-length-adjustment: Value for GFLADJ_30MHZ field of GFLADJ
register for post-silicon frame length adjustment when the
fladj_30mhz_sdbnd signal is invalid or incorrect.
- snps,rx-thr-num-pkt-prd: periodic ESS RX packet threshold count - host mode
only. Set this and rx-max-burst-prd to a valid,
non-zero value 1-16 (DWC_usb31 programming guide
section 1.2.4) to enable periodic ESS RX threshold.
- snps,rx-max-burst-prd: max periodic ESS RX burst size - host mode only. Set
this and rx-thr-num-pkt-prd to a valid, non-zero value
1-16 (DWC_usb31 programming guide section 1.2.4) to
enable periodic ESS RX threshold.
- snps,tx-thr-num-pkt-prd: periodic ESS TX packet threshold count - host mode
only. Set this and tx-max-burst-prd to a valid,
non-zero value 1-16 (DWC_usb31 programming guide
section 1.2.3) to enable periodic ESS TX threshold.
- snps,tx-max-burst-prd: max periodic ESS TX burst size - host mode only. Set
this and tx-thr-num-pkt-prd to a valid, non-zero value
1-16 (DWC_usb31 programming guide section 1.2.3) to
enable periodic ESS TX threshold.
- <DEPRECATED> tx-fifo-resize: determines if the FIFO *has* to be reallocated.
- snps,incr-burst-type-adjustment: Value for INCR burst type of GSBUSCFG0
register, undefined length INCR burst type enable and INCRx type.
When just one value, which means INCRX burst mode enabled. When
more than one value, which means undefined length INCR burst type
enabled. The values can be 1, 4, 8, 16, 32, 64, 128 and 256.
- in addition all properties from usb-xhci.txt from the current directory are
supported as well
This is usually a subnode to DWC3 glue to which it is connected.
dwc3@4a030000 {
compatible = "snps,dwc3";
reg = <0x4a030000 0xcfff>;
interrupts = <0 92 4>
usb-phy = <&usb2_phy>, <&usb3,phy>;
snps,incr-burst-type-adjustment = <1>, <4>, <8>, <16>;
};

323
Documentation/devicetree/bindings/usb/snps,dwc3.yaml Normal file
View File

@@ -0,0 +1,323 @@
# SPDX-License-Identifier: GPL-2.0
%YAML 1.2
---
$id: http://devicetree.org/schemas/usb/snps,dwc3.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Synopsys DesignWare USB3 Controller
maintainers:
- Felipe Balbi <balbi@kernel.org>
description:
This is usually a subnode to DWC3 glue to which it is connected, but can also
be presented as a standalone DT node with an optional vendor-specific
compatible string.
allOf:
- $ref: usb-drd.yaml#
- if:
properties:
dr_mode:
const: peripheral
required:
- dr_mode
then:
$ref: usb.yaml#
else:
$ref: usb-xhci.yaml#
properties:
compatible:
contains:
const: snps,dwc3
interrupts:
minItems: 1
maxItems: 3
clocks:
description:
In general the core supports three types of clocks. bus_early is a
SoC Bus Clock (AHB/AXI/Native). ref generates ITP when the UTMI/ULPI
PHY is suspended. suspend clocks a small part of the USB3 core when
SS PHY in P3. But particular cases may differ from that having less
or more clock sources with another names.
clock-names:
contains:
anyOf:
- enum: [bus_early, ref, suspend]
- true
usb-phy:
minItems: 1
items:
- description: USB2/HS PHY
- description: USB3/SS PHY
phys:
minItems: 1
items:
- description: USB2/HS PHY
- description: USB3/SS PHY
phy-names:
minItems: 1
items:
- const: usb2-phy
- const: usb3-phy
resets:
minItems: 1
snps,usb2-lpm-disable:
description: Indicate if we don't want to enable USB2 HW LPM
type: boolean
snps,usb3_lpm_capable:
description: Determines if platform is USB3 LPM capable
type: boolean
snps,dis-start-transfer-quirk:
description:
When set, disable isoc START TRANSFER command failure SW work-around
for DWC_usb31 version 1.70a-ea06 and prior.
type: boolean
snps,disable_scramble_quirk:
description:
True when SW should disable data scrambling. Only really useful for FPGA
builds.
type: boolean
snps,has-lpm-erratum:
description: True when DWC3 was configured with LPM Erratum enabled
type: boolean
snps,lpm-nyet-threshold:
description: LPM NYET threshold
$ref: /schemas/types.yaml#/definitions/uint8
snps,u2exit_lfps_quirk:
description: Set if we want to enable u2exit lfps quirk
type: boolean
snps,u2ss_inp3_quirk:
description: Set if we enable P3 OK for U2/SS Inactive quirk
type: boolean
snps,req_p1p2p3_quirk:
description:
When set, the core will always request for P1/P2/P3 transition sequence.
type: boolean
snps,del_p1p2p3_quirk:
description:
When set core will delay P1/P2/P3 until a certain amount of 8B10B errors
occur.
type: boolean
snps,del_phy_power_chg_quirk:
description: When set core will delay PHY power change from P0 to P1/P2/P3.
type: boolean
snps,lfps_filter_quirk:
description: When set core will filter LFPS reception.
type: boolean
snps,rx_detect_poll_quirk:
description:
when set core will disable a 400us delay to start Polling LFPS after
RX.Detect.
type: boolean
snps,tx_de_emphasis_quirk:
description: When set core will set Tx de-emphasis value
type: boolean
snps,tx_de_emphasis:
description:
The value driven to the PHY is controlled by the LTSSM during USB3
Compliance mode.
$ref: /schemas/types.yaml#/definitions/uint8
snps,dis_u3_susphy_quirk:
description: When set core will disable USB3 suspend phy
type: boolean
snps,dis_u2_susphy_quirk:
description: When set core will disable USB2 suspend phy
type: boolean
snps,dis_enblslpm_quirk:
description:
When set clears the enblslpm in GUSB2PHYCFG, disabling the suspend signal
to the PHY.
type: boolean
snps,dis-u1-entry-quirk:
description: Set if link entering into U1 needs to be disabled
type: boolean
snps,dis-u2-entry-quirk:
description: Set if link entering into U2 needs to be disabled
type: boolean
snps,dis_rxdet_inp3_quirk:
description:
When set core will disable receiver detection in PHY P3 power state.
type: boolean
snps,dis-u2-freeclk-exists-quirk:
description:
When set, clear the u2_freeclk_exists in GUSB2PHYCFG, specify that USB2
PHY doesn't provide a free-running PHY clock.
type: boolean
snps,dis-del-phy-power-chg-quirk:
description:
When set core will change PHY power from P0 to P1/P2/P3 without delay.
type: boolean
snps,dis-tx-ipgap-linecheck-quirk:
description: When set, disable u2mac linestate check during HS transmit
type: boolean
snps,parkmode-disable-ss-quirk:
description:
When set, all SuperSpeed bus instances in park mode are disabled.
type: boolean
snps,dis_metastability_quirk:
description:
When set, disable metastability workaround. CAUTION! Use only if you are
absolutely sure of it.
type: boolean
snps,dis-split-quirk:
description:
When set, change the way URBs are handled by the driver. Needed to
avoid -EPROTO errors with usbhid on some devices (Hikey 970).
type: boolean
snps,is-utmi-l1-suspend:
description:
True when DWC3 asserts output signal utmi_l1_suspend_n, false when
asserts utmi_sleep_n.
type: boolean
snps,hird-threshold:
description: HIRD threshold
$ref: /schemas/types.yaml#/definitions/uint8
snps,hsphy_interface:
description:
High-Speed PHY interface selection between UTMI+ and ULPI when the
DWC_USB3_HSPHY_INTERFACE has value 3.
$ref: /schemas/types.yaml#/definitions/uint8
enum: [utmi, ulpi]
snps,quirk-frame-length-adjustment:
description:
Value for GFLADJ_30MHZ field of GFLADJ register for post-silicon frame
length adjustment when the fladj_30mhz_sdbnd signal is invalid or
incorrect.
$ref: /schemas/types.yaml#/definitions/uint32
snps,rx-thr-num-pkt-prd:
description:
Periodic ESS RX packet threshold count (host mode only). Set this and
snps,rx-max-burst-prd to a valid, non-zero value 1-16 (DWC_usb31
programming guide section 1.2.4) to enable periodic ESS RX threshold.
$ref: /schemas/types.yaml#/definitions/uint8
minimum: 1
maximum: 16
snps,rx-max-burst-prd:
description:
Max periodic ESS RX burst size (host mode only). Set this and
snps,rx-thr-num-pkt-prd to a valid, non-zero value 1-16 (DWC_usb31
programming guide section 1.2.4) to enable periodic ESS RX threshold.
$ref: /schemas/types.yaml#/definitions/uint8
minimum: 1
maximum: 16
snps,tx-thr-num-pkt-prd:
description:
Periodic ESS TX packet threshold count (host mode only). Set this and
snps,tx-max-burst-prd to a valid, non-zero value 1-16 (DWC_usb31
programming guide section 1.2.3) to enable periodic ESS TX threshold.
$ref: /schemas/types.yaml#/definitions/uint8
minimum: 1
maximum: 16
snps,tx-max-burst-prd:
description:
Max periodic ESS TX burst size (host mode only). Set this and
snps,tx-thr-num-pkt-prd to a valid, non-zero value 1-16 (DWC_usb31
programming guide section 1.2.3) to enable periodic ESS TX threshold.
$ref: /schemas/types.yaml#/definitions/uint8
minimum: 1
maximum: 16
tx-fifo-resize:
description: Determines if the TX fifos can be dynamically resized depending
on the number of IN endpoints used and if bursting is supported. This
may help improve bandwidth on platforms with higher system latencies, as
increased fifo space allows for the controller to prefetch data into its
internal memory.
type: boolean
tx-fifo-max-num:
description: Specifies the max number of packets the txfifo resizing logic
can account for when higher endpoint bursting is used. (bMaxBurst > 6) The
higher the number, the more fifo space the txfifo resizing logic will
allocate for that endpoint.
$ref: /schemas/types.yaml#/definitions/uint8
minimum: 3
snps,incr-burst-type-adjustment:
description:
Value for INCR burst type of GSBUSCFG0 register, undefined length INCR
burst type enable and INCRx type. A single value means INCRX burst mode
enabled. If more than one value specified, undefined length INCR burst
type will be enabled with burst lengths utilized up to the maximum
of the values passed in this property.
$ref: /schemas/types.yaml#/definitions/uint32-array
minItems: 1
maxItems: 8
uniqueItems: true
items:
enum: [1, 4, 8, 16, 32, 64, 128, 256]
unevaluatedProperties: false
required:
- compatible
- reg
- interrupts
examples:
- |
usb@4a030000 {
compatible = "snps,dwc3";
reg = <0x4a030000 0xcfff>;
interrupts = <0 92 4>;
usb-phy = <&usb2_phy>, <&usb3_phy>;
snps,incr-burst-type-adjustment = <1>, <4>, <8>, <16>;
};
- |
usb@4a000000 {
compatible = "snps,dwc3";
reg = <0x4a000000 0xcfff>;
interrupts = <0 92 4>;
clocks = <&clk 1>, <&clk 2>, <&clk 3>;
clock-names = "bus_early", "ref", "suspend";
phys = <&usb2_phy>, <&usb3_phy>;
phy-names = "usb2-phy", "usb3-phy";
snps,dis_u2_susphy_quirk;
snps,dis_enblslpm_quirk;
};
...

3
Documentation/filesystems/f2fs.rst
View File

@@ -289,6 +289,9 @@ compress_mode=%s Control file compression mode. This supports "fs" and "user"
choosing the target file and the timing. The user can do manual
compression/decompression on the compression enabled files using
ioctls.
compress_cache Support to use address space of a filesystem managed inode to
cache compressed block, in order to improve cache hit ratio of
random read.
inlinecrypt When possible, encrypt/decrypt the contents of encrypted
files using the blk-crypto framework rather than
filesystem-layer encryption. This allows the use of

314
android/abi_gki_aarch64_db845c
View File

@@ -2,19 +2,25 @@
# commonly used symbols
add_uevent_var
alloc_io_pgtable_ops
__alloc_skb
alloc_workqueue
__arch_copy_from_user
__arch_copy_to_user
arm64_const_caps_ready
arm64_use_ng_mappings
bcmp
blocking_notifier_call_chain
blocking_notifier_chain_register
blocking_notifier_chain_unregister
bpf_trace_run1
bpf_trace_run2
bpf_trace_run3
bpf_trace_run4
bpf_trace_run5
bpf_trace_run6
bus_register
bus_unregister
cancel_delayed_work
cancel_delayed_work_sync
cancel_work_sync
capable
@@ -23,6 +29,8 @@
cdev_init
__cfi_slowpath
__check_object_size
__class_register
class_unregister
clk_bulk_disable
clk_bulk_enable
clk_bulk_prepare
@@ -54,7 +62,9 @@
cpumask_next
cpu_number
__cpu_online_mask
__cpu_possible_mask
crc32_le
_ctype
debugfs_create_dir
debugfs_create_file
debugfs_create_u32
@@ -62,8 +72,10 @@
debugfs_remove
default_llseek
delayed_work_timer_fn
del_timer
del_timer_sync
destroy_workqueue
dev_close
dev_coredumpv
dev_driver_string
_dev_err
@@ -108,6 +120,7 @@
devm_pinctrl_register
devm_platform_ioremap_resource
devm_regmap_add_irq_chip
devm_regmap_field_alloc
__devm_regmap_init
__devm_regmap_init_i2c
__devm_regmap_init_mmio_clk
@@ -175,15 +188,20 @@
drm_helper_probe_single_connector_modes
drm_mode_vrefresh
enable_irq
eth_mac_addr
eth_platform_get_mac_address
ethtool_op_get_link
eth_type_trans
eth_validate_addr
event_triggers_call
find_next_bit
find_next_zero_bit
finish_wait
flush_work
flush_workqueue
free_io_pgtable_ops
free_irq
gcd
generic_handle_irq
generic_mii_ioctl
get_device
@@ -232,12 +250,13 @@
idr_alloc_cyclic
idr_destroy
idr_find
idr_for_each
idr_get_next
idr_remove
ieee80211_get_channel_khz
init_net
__init_swait_queue_head
init_timer_key
init_uts_ns
init_wait_entry
__init_waitqueue_head
iomem_resource
@@ -276,6 +295,8 @@
irq_to_desc
is_vmalloc_addr
jiffies
jiffies_to_msecs
jiffies_to_usecs
kasan_flag_enabled
kasprintf
kernel_connect
@@ -284,6 +305,7 @@
kernel_sendmsg
kfree
kfree_const
kfree_sensitive
kfree_skb
__kmalloc
kmalloc_caches
@@ -297,8 +319,12 @@
ktime_get
ktime_get_mono_fast_ns
ktime_get_real_ts64
kvfree
kvfree_call_rcu
kvmalloc_node
__list_add_valid
__list_del_entry_valid
__local_bh_enable_ip
__log_post_read_mmio
__log_read_mmio
__log_write_mmio
@@ -313,10 +339,12 @@
memremap
memset
memstart_addr
memunmap
mii_ethtool_gset
mii_nway_restart
misc_deregister
misc_register
mod_delayed_work_on
mod_timer
module_layout
__msecs_to_jiffies
@@ -333,10 +361,15 @@
netdev_err
netdev_info
netdev_warn
netif_carrier_on
netif_napi_add
__netif_napi_del
__nla_parse
nla_put
no_llseek
nr_cpu_ids
nvmem_cell_get
nvmem_cell_put
nvmem_cell_read
of_address_to_resource
of_alias_get_id
@@ -348,6 +381,7 @@
of_device_is_compatible
of_device_uevent_modalias
of_dma_configure_id
of_find_device_by_node
of_find_property
of_fwnode_ops
of_genpd_add_provider_onecell
@@ -380,8 +414,11 @@
of_property_read_u32_index
of_property_read_variable_u32_array
of_property_read_variable_u8_array
of_prop_next_u32
of_reserved_mem_lookup
param_ops_bool
param_ops_charp
param_ops_int
param_ops_uint
__pci_register_driver
pci_unregister_driver
@@ -419,10 +456,12 @@
__pm_runtime_set_status
__pm_runtime_suspend
__pm_runtime_use_autosuspend
preempt_schedule
preempt_schedule_notrace
prepare_to_wait_event
printk
pskb_expand_head
__pskb_pull_tail
put_device
__put_task_struct
qcom_smem_state_register
@@ -451,10 +490,13 @@
regcache_cache_only
regcache_mark_dirty
regcache_sync
register_netdevice_notifier
register_reboot_notifier
__register_rpmsg_driver
regmap_bulk_read
regmap_bulk_write
regmap_field_read
regmap_field_update_bits_base
__regmap_init
regmap_irq_get_virq
regmap_multi_reg_write
@@ -480,6 +522,7 @@
reset_control_assert
reset_control_deassert
reset_control_reset
round_jiffies_up
rpmsg_register_device
rpmsg_send
rpmsg_unregister_device
@@ -490,6 +533,9 @@
rproc_del
rproc_free
rproc_remove_subdev
rtnl_is_locked
rtnl_lock
rtnl_unlock
schedule
schedule_timeout
scnprintf
@@ -507,10 +553,14 @@
single_open
single_release
skb_clone
skb_copy
skb_copy_bits
skb_copy_expand
skb_dequeue
skb_pull
skb_push
skb_put
skb_queue_head
skb_queue_purge
skb_queue_tail
skb_trim
@@ -557,16 +607,22 @@
strncpy
strpbrk
strsep
__sw_hweight16
__sw_hweight32
__sw_hweight64
__sw_hweight8
synchronize_irq
synchronize_net
synchronize_rcu
syscon_node_to_regmap
syscon_regmap_lookup_by_phandle
sysfs_create_link
sysfs_remove_link
sysrq_mask
system_power_efficient_wq
system_wq
tasklet_init
tasklet_kill
__tasklet_schedule
thermal_cooling_device_unregister
trace_event_buffer_commit
@@ -597,6 +653,7 @@
uart_write_wakeup
__udelay
unregister_chrdev_region
unregister_netdevice_notifier
unregister_reboot_notifier
unregister_rpmsg_driver
usb_deregister
@@ -623,6 +680,7 @@
usbnet_write_cmd_async
usbnet_write_cmd_nopm
usb_register_driver
__usecs_to_jiffies
usleep_range
vabits_actual
vfree
@@ -657,12 +715,10 @@
iommu_group_ref_get
iommu_put_dma_cookie
of_dma_is_coherent
param_ops_int
pci_bus_type
pci_device_group
# required by asix.ko
eth_mac_addr
genphy_resume
mdiobus_alloc_size
mdiobus_free
@@ -679,7 +735,6 @@
phy_print_status
phy_start
phy_stop
skb_copy_expand
usbnet_change_mtu
usbnet_get_drvinfo
usbnet_get_link
@@ -687,98 +742,23 @@
usbnet_set_link_ksettings
usbnet_unlink_rx_urbs
# required by ath.ko
freq_reg_info
reg_initiator_name
wiphy_apply_custom_regulatory
# required by ath10k_core.ko
bcmp
cancel_delayed_work
__cfg80211_alloc_event_skb
__cfg80211_alloc_reply_skb
cfg80211_calculate_bitrate
cfg80211_find_elem_match
cfg80211_find_vendor_elem
cfg80211_get_bss
cfg80211_put_bss
__cfg80211_send_event_skb
cfg80211_vendor_cmd_reply
cpu_latency_qos_add_request
cpu_latency_qos_remove_request
device_get_mac_address
device_set_wakeup_enable
firmware_request_nowarn
guid_gen
idr_for_each
ieee80211_alloc_hw_nm
ieee80211_beacon_cntdwn_is_complete
ieee80211_beacon_get_template
ieee80211_beacon_get_tim
ieee80211_beacon_loss
ieee80211_beacon_update_cntdwn
ieee80211_bss_get_elem
ieee80211_channel_to_freq_khz
ieee80211_connection_loss
ieee80211_csa_finish
ieee80211_find_sta
ieee80211_find_sta_by_ifaddr
ieee80211_free_hw
ieee80211_free_txskb
ieee80211_hdrlen
ieee80211_iterate_active_interfaces_atomic
ieee80211_iterate_stations_atomic
ieee80211_iter_chan_contexts_atomic
ieee80211_manage_rx_ba_offl
ieee80211_next_txq
ieee80211_proberesp_get
ieee80211_queue_delayed_work
ieee80211_queue_work
ieee80211_radar_detected
ieee80211_ready_on_channel
ieee80211_register_hw
ieee80211_remain_on_channel_expired
ieee80211_report_low_ack
ieee80211_restart_hw
ieee80211_rx_napi
ieee80211_scan_completed
__ieee80211_schedule_txq
ieee80211_sta_register_airtime
ieee80211_stop_queue
ieee80211_stop_queues
ieee80211_tdls_oper_request
ieee80211_tx_dequeue
ieee80211_txq_get_depth
ieee80211_txq_may_transmit
ieee80211_txq_schedule_start
ieee80211_tx_rate_update
ieee80211_tx_status
ieee80211_tx_status_irqsafe
ieee80211_unregister_hw
ieee80211_wake_queue
ieee80211_wake_queues
init_dummy_netdev
init_uts_ns
__kfifo_alloc
__kfifo_free
__local_bh_enable_ip
__nla_parse
nla_put
param_ops_ulong
regulatory_hint
skb_copy
skb_dequeue_tail
skb_queue_head
skb_realloc_headroom
strlcat
strscpy
__sw_hweight16
__sw_hweight8
thermal_cooling_device_register
vzalloc
wiphy_read_of_freq_limits
wiphy_rfkill_set_hw_state
wiphy_to_ieee80211_hw
# required by ath10k_pci.ko
pci_clear_master
@@ -801,11 +781,9 @@
iommu_map
# required by ax88179_178a.ko
ethtool_op_get_link
ethtool_op_get_ts_info
mii_ethtool_get_link_ksettings
mii_ethtool_set_link_ksettings
netif_carrier_on
# required by bam_dma.ko
dma_async_device_register
@@ -815,7 +793,6 @@
of_dma_controller_free
of_dma_controller_register
pm_runtime_irq_safe
tasklet_kill
tasklet_setup
vchan_dma_desc_free_list
vchan_find_desc
@@ -823,6 +800,59 @@
vchan_tx_desc_free
vchan_tx_submit
# required by cfg80211.ko
bpf_trace_run10
bpf_trace_run7
debugfs_rename
dev_change_net_namespace
__dev_get_by_index
dev_get_by_index
device_add
device_del
device_rename
genlmsg_multicast_allns
genlmsg_put
genl_register_family
genl_unregister_family
get_net_ns_by_fd
get_net_ns_by_pid
inet_csk_get_port
key_create_or_update
key_put
keyring_alloc
ktime_get_coarse_with_offset
memcmp
netif_rx_ni
netlink_broadcast
netlink_register_notifier
netlink_unicast
netlink_unregister_notifier
net_ns_type_operations
nla_find
nla_memcpy
nla_put_64bit
nla_reserve
__nla_validate
__put_net
rb_erase
rb_insert_color
register_pernet_device
request_firmware_nowait
rfkill_alloc
rfkill_blocked
rfkill_destroy
rfkill_pause_polling
rfkill_register
rfkill_resume_polling
rfkill_set_hw_state
rfkill_unregister
skb_add_rx_frag
__sock_create
trace_print_array_seq
unregister_pernet_device
verify_pkcs7_signature
wireless_nlevent_flush
# required by clk-qcom.ko
__clk_determine_rate
clk_fixed_factor_ops
@@ -836,7 +866,6 @@
__clk_mux_determine_rate_closest
divider_ro_round_rate_parent
of_find_node_opts_by_path
of_prop_next_u32
pm_genpd_remove_subdomain
# required by clk-rpmh.ko
@@ -858,7 +887,6 @@
gpiod_get_value_cansleep
gpiod_set_debounce
gpiod_to_irq
system_power_efficient_wq
# required by fastrpc.ko
dma_buf_attach
@@ -925,9 +953,6 @@
i2c_put_dma_safe_msg_buf
of_machine_is_compatible
# required by i2c-qup.ko
__usecs_to_jiffies
# required by i2c-rk3x.ko
clk_notifier_register
clk_notifier_unregister
@@ -953,12 +978,88 @@
mipi_dsi_device_unregister
of_find_mipi_dsi_host_by_node
# required by mac80211.ko
alloc_netdev_mqs
__alloc_percpu_gfp
arc4_crypt
arc4_setkey
call_rcu
crc32_be
crypto_aead_decrypt
crypto_aead_encrypt
crypto_aead_setauthsize
crypto_aead_setkey
crypto_alloc_aead
crypto_alloc_shash
crypto_alloc_skcipher
crypto_destroy_tfm
__crypto_memneq
crypto_shash_digest
crypto_shash_finup
crypto_shash_setkey
crypto_shash_update
crypto_skcipher_decrypt
crypto_skcipher_encrypt
crypto_skcipher_setkey
__crypto_xor
dev_alloc_name
dev_fetch_sw_netstats
dev_printk
dev_queue_xmit
ether_setup
flush_delayed_work
free_netdev
free_percpu
get_random_u32
__hw_addr_init
__hw_addr_sync
__hw_addr_unsync
kernel_param_lock
kernel_param_unlock
kfree_skb_list
ktime_get_seconds
ktime_get_with_offset
napi_gro_receive
netdev_set_default_ethtool_ops
netif_carrier_off
netif_receive_skb
netif_receive_skb_list
netif_rx
netif_tx_stop_all_queues
netif_tx_wake_queue
net_ratelimit
__per_cpu_offset
prandom_bytes
prandom_u32
___pskb_trim
rcu_barrier
register_inet6addr_notifier
register_inetaddr_notifier
register_netdevice
rhashtable_free_and_destroy
rhashtable_insert_slow
rhltable_init
__rht_bucket_nested
rht_bucket_nested
rht_bucket_nested_insert
round_jiffies
round_jiffies_relative
sg_init_one
skb_checksum_help
skb_clone_sk
skb_complete_wifi_ack
skb_ensure_writable
__skb_get_hash
__skb_gso_segment
system_freezable_wq
unregister_inet6addr_notifier
unregister_inetaddr_notifier
unregister_netdevice_many
unregister_netdevice_queue
# required by msm.ko
__bitmap_andnot
__bitmap_weight
bpf_trace_run1
bpf_trace_run2
bpf_trace_run6
bpf_trace_run8
clk_get_parent
__clk_hw_register_divider
@@ -975,10 +1076,8 @@
component_master_add_with_match
component_master_del
component_unbind_all
_ctype
debugfs_create_bool
debugfs_create_u64
del_timer
dev_coredumpm
devfreq_recommended_opp
devfreq_resume_device
@@ -1198,12 +1297,9 @@
kthread_create_worker
kthread_destroy_worker
kthread_queue_work
kvfree
kvmalloc_node
llist_add_batch
memdup_user_nul
memparse
memunmap
mipi_dsi_create_packet
mipi_dsi_host_register
mipi_dsi_host_unregister
@@ -1211,20 +1307,16 @@
mutex_lock_interruptible
mutex_trylock_recursive
nsecs_to_jiffies
nvmem_cell_get
nvmem_cell_put
of_clk_hw_onecell_get
of_device_is_available
of_drm_find_bridge
of_drm_find_panel
of_find_device_by_node
of_find_matching_node_and_match
of_get_compatible_child
of_graph_get_endpoint_by_regs
of_graph_get_next_endpoint
of_graph_get_remote_port_parent
of_icc_get
param_ops_charp
phy_calibrate
phy_configure
pid_task
@@ -1240,7 +1332,6 @@
regulator_get
regulator_put
reservation_ww_class
round_jiffies_up
sched_set_fifo
schedule_timeout_interruptible
__sg_page_iter_dma_next
@@ -1280,7 +1371,6 @@
dma_pool_create
dma_pool_destroy
dma_pool_free
flush_work
free_pages
gen_pool_dma_alloc_align
gen_pool_dma_zalloc_align
@@ -1391,7 +1481,6 @@
cpufreq_get_driver_data
cpufreq_register_driver
cpufreq_unregister_driver
__cpu_possible_mask
dev_pm_opp_adjust_voltage
dev_pm_opp_disable
dev_pm_opp_enable
@@ -1450,9 +1539,6 @@
# required by qcom_hwspinlock.ko
devm_hwspin_lock_register
devm_regmap_field_alloc
regmap_field_read
regmap_field_update_bits_base
# required by qcom_pil_info.ko
__memset_io
@@ -1476,7 +1562,6 @@
__num_online_cpus
# required by qcom_spmi-regulator.ko
jiffies_to_usecs
regulator_disable_regmap
regulator_enable_regmap
regulator_is_enabled_regmap
@@ -1486,14 +1571,16 @@
rproc_get_by_child
try_wait_for_completion
# required by qrtr-smd.ko
__pskb_pull_tail
# required by qcom_tsens.ko
debugfs_lookup
devm_thermal_zone_of_sensor_register
thermal_zone_device_update
thermal_zone_get_slope
# required by qrtr-tun.ko
_copy_to_iter
# required by qrtr.ko
__alloc_skb
autoremove_wake_function
datagram_poll
do_wait_intr_irq
@@ -1507,7 +1594,6 @@
refcount_dec_and_mutex_lock
release_sock
sk_alloc
skb_copy_bits
skb_copy_datagram_iter
skb_free_datagram
__skb_pad
@@ -1526,7 +1612,6 @@
sock_queue_rcv_skb
sock_register
sock_unregister
synchronize_rcu
# required by reboot-mode.ko
devres_add
@@ -1545,8 +1630,6 @@
# required by rmtfs_mem.ko
alloc_chrdev_region
__class_register
class_unregister
# required by rtc-pm8xxx.ko
devm_request_any_context_irq
@@ -1605,9 +1688,6 @@
snd_soc_of_parse_aux_devs
snd_soc_of_parse_card_name
# required by snd-soc-rl6231.ko
gcd
# required by snd-soc-rt5663.ko
regcache_cache_bypass
snd_soc_add_component_controls
@@ -1666,7 +1746,6 @@
spi_delay_exec
spi_finalize_current_message
spi_get_next_queued_message
tasklet_init
# required by spmi-pmic-arb.ko
irq_domain_set_info
@@ -1686,7 +1765,6 @@
dma_sync_sg_for_cpu
dma_sync_sg_for_device
__free_pages
preempt_schedule
__sg_page_iter_next
# required by ufs_qcom.ko

549
android/abi_gki_aarch64_exynos
View File

File diff suppressed because it is too large Load Diff

1478
android/abi_gki_aarch64_exynosauto Normal file
View File

File diff suppressed because it is too large Load Diff

11
android/abi_gki_aarch64_fips140
View File

@@ -1,12 +1,17 @@
[abi_symbol_list]
# commonly used symbols
module_layout
__put_task_struct
# required by fips140.ko
add_random_ready_callback
aead_register_instance
arm64_const_caps_ready
bcmp
cancel_work_sync
__cfi_slowpath
cpu_have_feature
cpu_hwcap_keys
crypto_aead_decrypt
crypto_aead_encrypt
crypto_aead_setauthsize
@@ -16,11 +21,14 @@
crypto_alg_list
crypto_alg_mod_lookup
crypto_alg_sem
crypto_alloc_aead
crypto_alloc_base
crypto_alloc_rng
crypto_alloc_shash
crypto_alloc_skcipher
crypto_attr_alg_name
crypto_check_attr_type
crypto_cipher_decrypt_one
crypto_cipher_encrypt_one
crypto_cipher_setkey
crypto_destroy_tfm
@@ -43,6 +51,7 @@
crypto_remove_final
crypto_remove_spawns
crypto_req_done
crypto_rng_reset
crypto_shash_alg_has_setkey
crypto_shash_digest
crypto_shash_final
@@ -77,6 +86,7 @@
kmalloc_caches
kmalloc_order_trace
kmem_cache_alloc_trace
kmemdup
__list_add_valid
__list_del_entry_valid
memcpy
@@ -89,6 +99,7 @@
preempt_schedule_notrace
printk
queue_work_on
refcount_warn_saturate
scatterwalk_ffwd
scatterwalk_map_and_copy
sg_init_one

4374
android/abi_gki_aarch64_galaxy Normal file
View File

File diff suppressed because it is too large Load Diff

152
android/abi_gki_aarch64_generic
View File

@@ -24,6 +24,8 @@
amba_driver_unregister
android_rvh_probe_register
anon_inode_getfd
arc4_crypt
arc4_setkey
__arch_clear_user
__arch_copy_from_user
__arch_copy_in_user
@@ -105,6 +107,7 @@
bus_unregister
bus_unregister_notifier
cache_line_size
call_rcu
cancel_delayed_work
cancel_delayed_work_sync
cancel_work_sync
@@ -115,35 +118,6 @@
cdev_device_add
cdev_device_del
cdev_init
__cfg80211_alloc_event_skb
__cfg80211_alloc_reply_skb
cfg80211_chandef_create
cfg80211_ch_switch_notify
cfg80211_connect_done
cfg80211_del_sta_sinfo
cfg80211_disconnected
cfg80211_external_auth_request
cfg80211_find_elem_match
cfg80211_get_bss
cfg80211_ibss_joined
cfg80211_inform_bss_frame_data
cfg80211_mgmt_tx_status
cfg80211_michael_mic_failure
cfg80211_new_sta
cfg80211_port_authorized
cfg80211_put_bss
cfg80211_ready_on_channel
cfg80211_remain_on_channel_expired
cfg80211_roamed
cfg80211_rx_mgmt_khz
cfg80211_scan_done
cfg80211_sched_scan_results
cfg80211_sched_scan_stopped
cfg80211_sched_scan_stopped_rtnl
__cfg80211_send_event_skb
cfg80211_unlink_bss
cfg80211_unregister_wdev
cfg80211_vendor_cmd_reply
__cfi_slowpath
__check_object_size
__class_create
@@ -254,6 +228,8 @@
cpus_read_lock
cpus_read_unlock
cpu_subsys
crc32_be
crc32_le
crc8
crc8_populate_msb
crypto_aead_decrypt
@@ -263,19 +239,24 @@
crypto_alloc_aead
crypto_alloc_base
crypto_alloc_shash
crypto_alloc_skcipher
crypto_alloc_sync_skcipher
crypto_comp_compress
crypto_comp_decompress
crypto_destroy_tfm
__crypto_memneq
crypto_register_alg
crypto_register_scomp
crypto_shash_digest
crypto_shash_finup
crypto_shash_setkey
crypto_shash_update
crypto_skcipher_decrypt
crypto_skcipher_encrypt
crypto_skcipher_setkey
crypto_unregister_alg
crypto_unregister_scomp
__crypto_xor
csum_ipv6_magic
csum_tcpudp_nofold
_ctype
@@ -293,6 +274,7 @@
debugfs_create_x32
debugfs_lookup
debugfs_remove
debugfs_rename
dec_zone_page_state
default_llseek
deferred_free
@@ -302,11 +284,14 @@
del_timer_sync
desc_to_gpio
destroy_workqueue
dev_alloc_name
dev_change_net_namespace
dev_close
_dev_crit
dev_driver_string
_dev_emerg
_dev_err
dev_fetch_sw_netstats
devfreq_add_device
devfreq_add_governor
devfreq_monitor_resume
@@ -319,12 +304,16 @@
devfreq_unregister_opp_notifier
devfreq_update_interval
dev_fwnode
__dev_get_by_index
dev_get_by_index
dev_get_by_name
device_add
device_add_disk
device_add_groups
device_create
device_create_file
device_create_with_groups
device_del
device_destroy
device_find_child
device_for_each_child
@@ -340,6 +329,7 @@
device_register
device_remove_file
device_remove_groups
device_rename
device_set_wakeup_capable
device_set_wakeup_enable
device_unregister
@@ -419,6 +409,8 @@
devm_thermal_zone_of_sensor_unregister
devm_usb_get_phy_by_phandle
_dev_notice
dev_pm_domain_attach_by_name
dev_pm_domain_detach
dev_pm_opp_add
dev_pm_opp_disable
dev_pm_opp_find_freq_ceil
@@ -435,6 +427,7 @@
dev_pm_opp_set_regulators
dev_pm_qos_read_value
dev_pm_qos_update_request
dev_printk
dev_printk_emit
dev_queue_xmit
devres_add
@@ -552,6 +545,7 @@
__drm_atomic_helper_crtc_destroy_state
__drm_atomic_helper_crtc_duplicate_state
__drm_atomic_helper_crtc_reset
drm_atomic_helper_disable_all
drm_atomic_helper_disable_plane
drm_atomic_helper_duplicate_state
drm_atomic_helper_fake_vblank
@@ -572,8 +566,11 @@
drm_atomic_normalize_zpos
drm_atomic_private_obj_fini
drm_atomic_private_obj_init
drm_atomic_set_crtc_for_connector
drm_atomic_set_crtc_for_plane
drm_atomic_set_fb_for_plane
drm_atomic_set_mode_for_crtc
drm_atomic_set_mode_prop_for_crtc
drm_atomic_state_alloc
drm_atomic_state_clear
__drm_atomic_state_free
@@ -641,6 +638,7 @@
drmm_mode_config_init
drm_mode_config_reset
drm_mode_convert_to_umode
drm_mode_copy
drm_mode_duplicate
drm_mode_equal
drm_mode_equal_no_clocks
@@ -714,6 +712,9 @@
emergency_restart
enable_irq
enable_percpu_irq
ether_setup
eth_mac_addr
ethtool_op_get_link
eth_type_trans
eventfd_ctx_fdget
eventfd_ctx_put
@@ -743,6 +744,7 @@
find_vma
finish_wait
flush_dcache_page
flush_delayed_work
flush_work
flush_workqueue
fput
@@ -769,6 +771,7 @@
generic_file_llseek
generic_handle_irq
generic_iommu_put_resv_regions
genlmsg_multicast_allns
genlmsg_put
genl_register_family
genl_unregister_family
@@ -787,6 +790,8 @@
get_device
__get_free_pages
get_governor_parent_kobj
get_net_ns_by_fd
get_net_ns_by_pid
get_pid_task
get_random_bytes
get_random_bytes_arch
@@ -854,6 +859,9 @@
hrtimer_sleeper_start_expires
hrtimer_start_range_ns
hrtimer_try_to_cancel
__hw_addr_init
__hw_addr_sync
__hw_addr_unsync
hwrng_register
hwrng_unregister
i2c_adapter_type
@@ -882,16 +890,15 @@
idr_destroy
idr_find
idr_for_each
idr_get_next
idr_preload
idr_remove
ieee80211_channel_to_freq_khz
ieee80211_freq_khz_to_channel
ieee80211_get_channel_khz
iio_device_unregister
import_iovec
in6_pton
in_aton
inc_zone_page_state
inet_csk_get_port
init_dummy_netdev
init_net
init_pseudo
@@ -899,6 +906,7 @@
__init_swait_queue_head
init_task
init_timer_key
init_uts_ns
init_wait_entry
__init_waitqueue_head
input_allocate_device
@@ -987,9 +995,14 @@
kasprintf
kernel_cpustat
kernel_kobj
kernel_param_lock
kernel_param_unlock
kernel_restart
kern_mount
kern_unmount
key_create_or_update
key_put
keyring_alloc
__kfifo_alloc
__kfifo_free
__kfifo_in
@@ -998,6 +1011,7 @@
kfree
kfree_sensitive
kfree_skb
kfree_skb_list
kill_anon_super
kill_fasync
kimage_voffset
@@ -1053,11 +1067,13 @@
kthread_stop
kthread_worker_fn
ktime_get
ktime_get_coarse_with_offset
ktime_get_mono_fast_ns
ktime_get_raw
ktime_get_raw_ts64
ktime_get_real_seconds
ktime_get_real_ts64
ktime_get_seconds
ktime_get_ts64
ktime_get_with_offset
kvfree
@@ -1076,6 +1092,8 @@
log_threaded_irq_wakeup_reason
__log_write_mmio
loops_per_jiffy
lru_cache_disable
lru_cache_enable
lzo1x_1_compress
lzo1x_decompress_safe
lzorle1x_1_compress
@@ -1143,19 +1161,31 @@
__napi_schedule
napi_schedule_prep
__netdev_alloc_skb
netdev_err
netdev_info
netdev_set_default_ethtool_ops
netdev_state_change
netdev_update_features
netif_carrier_off
netif_carrier_on
netif_napi_add
__netif_napi_del
netif_receive_skb
netif_receive_skb_list
netif_rx
netif_rx_ni
netif_tx_stop_all_queues
netif_tx_wake_queue
netlink_broadcast
__netlink_kernel_create
netlink_kernel_release
netlink_register_notifier
netlink_unicast
netlink_unregister_notifier
net_ns_type_operations
net_ratelimit
nla_append
nla_find
nla_memcpy
__nla_parse
nla_put
@@ -1163,6 +1193,7 @@
nla_put_nohdr
nla_reserve
nla_strlcpy
__nla_validate
__nlmsg_put
no_llseek
nonseekable_open
@@ -1397,8 +1428,10 @@
proc_remove
pskb_expand_head
__pskb_pull_tail
___pskb_trim
put_device
put_disk
__put_net
__put_page
put_sg_io_hdr
__put_task_struct
@@ -1436,6 +1469,7 @@
rb_next
rb_prev
rb_replace_node
rcu_barrier
__rcu_read_lock
__rcu_read_unlock
rdev_get_drvdata
@@ -1457,6 +1491,7 @@
register_netdev
register_netdevice
register_netdevice_notifier
register_pernet_device
register_pernet_subsys
register_pm_notifier
register_reboot_notifier
@@ -1500,7 +1535,6 @@
regulator_set_voltage
regulator_set_voltage_sel_regmap
regulator_unregister
regulatory_hint
release_firmware
__release_region
remap_pfn_range
@@ -1517,11 +1551,24 @@
return_address
revalidate_disk_size
rfkill_alloc
rfkill_blocked
rfkill_destroy
rfkill_init_sw_state
rfkill_pause_polling
rfkill_register
rfkill_resume_polling
rfkill_set_hw_state
rfkill_unregister
rhashtable_free_and_destroy
rhashtable_insert_slow
rhltable_init
__rht_bucket_nested
rht_bucket_nested
rht_bucket_nested_insert
root_task_group
round_jiffies
round_jiffies_relative
round_jiffies_up
rps_needed
rtc_class_close
rtc_class_open
@@ -1611,10 +1658,20 @@
single_open
single_open_size
single_release
skb_add_rx_frag
skb_checksum
skb_checksum_help
skb_clone
skb_clone_sk
skb_complete_wifi_ack
skb_copy
skb_copy_bits
skb_copy_expand
skb_dequeue
skb_dequeue_tail
skb_ensure_writable
__skb_get_hash
__skb_gso_segment
skb_pull
skb_push
skb_put
@@ -1704,6 +1761,8 @@
snd_soc_unregister_component
snprintf
soc_device_register
__sock_create
sock_release
sock_wfree
softnet_data
sort
@@ -1729,6 +1788,7 @@
sscanf
__stack_chk_fail
__stack_chk_guard
static_key_disable
static_key_slow_dec
static_key_slow_inc
stop_machine
@@ -1761,14 +1821,17 @@
submit_bio
submit_bio_wait
subsys_system_register
__sw_hweight16
__sw_hweight32
__sw_hweight64
__sw_hweight8
sync_file_create
sync_file_get_fence
synchronize_irq
synchronize_net
synchronize_rcu
syscon_regmap_lookup_by_phandle
sysctl_sched_features
sysctl_sched_latency
sysfs_add_file_to_group
sysfs_create_file_ns
@@ -1786,6 +1849,7 @@
sysfs_streq
sysfs_update_group
sysrq_mask
system_freezable_wq
system_freezing_cnt
system_highpri_wq
system_long_wq
@@ -1846,6 +1910,7 @@
__traceiter_android_rvh_post_init_entity_util_avg
__traceiter_android_rvh_preempt_disable
__traceiter_android_rvh_preempt_enable
__traceiter_android_rvh_sched_fork
__traceiter_android_rvh_select_task_rq_rt
__traceiter_android_rvh_set_iowait
__traceiter_android_rvh_typec_tcpci_chk_contaminant
@@ -1865,6 +1930,9 @@
__traceiter_android_vh_of_i2c_get_board_info
__traceiter_android_vh_pagecache_get_page
__traceiter_android_vh_rmqueue
__traceiter_android_vh_setscheduler_uclamp
__traceiter_android_vh_snd_compr_use_pause_in_drain
__traceiter_android_vh_sound_usb_support_cpu_suspend
__traceiter_android_vh_thermal_pm_notify_suspend
__traceiter_android_vh_timerfd_create
__traceiter_android_vh_typec_store_partner_src_caps
@@ -1887,6 +1955,8 @@
__traceiter_dwc3_readl
__traceiter_dwc3_writel
__traceiter_gpu_mem_total
__traceiter_mm_vmscan_direct_reclaim_begin
__traceiter_mm_vmscan_direct_reclaim_end
__traceiter_pelt_cfs_tp
__traceiter_pelt_dl_tp
__traceiter_pelt_irq_tp
@@ -1913,6 +1983,7 @@
__tracepoint_android_rvh_post_init_entity_util_avg
__tracepoint_android_rvh_preempt_disable
__tracepoint_android_rvh_preempt_enable
__tracepoint_android_rvh_sched_fork
__tracepoint_android_rvh_select_task_rq_rt
__tracepoint_android_rvh_set_iowait
__tracepoint_android_rvh_typec_tcpci_chk_contaminant
@@ -1932,6 +2003,9 @@
__tracepoint_android_vh_of_i2c_get_board_info
__tracepoint_android_vh_pagecache_get_page
__tracepoint_android_vh_rmqueue
__tracepoint_android_vh_setscheduler_uclamp
__tracepoint_android_vh_snd_compr_use_pause_in_drain
__tracepoint_android_vh_sound_usb_support_cpu_suspend
__tracepoint_android_vh_thermal_pm_notify_suspend
__tracepoint_android_vh_timerfd_create
__tracepoint_android_vh_typec_store_partner_src_caps
@@ -1954,6 +2028,8 @@
__tracepoint_dwc3_readl
__tracepoint_dwc3_writel
__tracepoint_gpu_mem_total
__tracepoint_mm_vmscan_direct_reclaim_begin
__tracepoint_mm_vmscan_direct_reclaim_end
__tracepoint_pelt_cfs_tp
__tracepoint_pelt_dl_tp
__tracepoint_pelt_irq_tp
@@ -2023,8 +2099,10 @@
unregister_inet6addr_notifier
unregister_inetaddr_notifier
unregister_netdev
unregister_netdevice_many
unregister_netdevice_notifier
unregister_netdevice_queue
unregister_pernet_device
unregister_pernet_subsys
unregister_pm_notifier
unregister_reboot_notifier
@@ -2132,6 +2210,7 @@
vb2_streamon
vb2_vmalloc_memops
vb2_wait_for_all_buffers
verify_pkcs7_signature
vfree
video_devdata
video_device_alloc
@@ -2183,11 +2262,7 @@
watchdog_register_device
watchdog_set_restart_priority
watchdog_unregister_device
wiphy_apply_custom_regulatory
wiphy_free
wiphy_new_nm
wiphy_register
wiphy_unregister
wireless_nlevent_flush
woken_wake_function
work_busy
__xfrm_state_destroy
@@ -2202,6 +2277,7 @@
xhci_handle_event
xhci_init_driver
xhci_resume
xhci_ring_alloc
xhci_ring_free
xhci_run
xhci_suspend

49
android/abi_gki_aarch64_hikey960
View File

@@ -166,12 +166,6 @@
ida_alloc_range
ida_destroy
ida_free
ieee80211_channel_to_freq_khz
ieee80211_connection_loss
ieee80211_find_sta
ieee80211_get_hdrlen_from_skb
ieee80211_queue_delayed_work
ieee80211_stop_rx_ba_session
__init_swait_queue_head
init_timer_key
init_wait_entry
@@ -1225,17 +1219,10 @@
tcpci_unregister_port
# required by wl18xx.ko
__cfg80211_alloc_event_skb
__cfg80211_send_event_skb
ieee80211_radar_detected
kstrtou8_from_user
# required by wlcore.ko
bcmp
__cfg80211_alloc_reply_skb
cfg80211_find_elem_match
cfg80211_find_vendor_elem
cfg80211_vendor_cmd_reply
complete_all
consume_skb
device_create_bin_file
@@ -1244,40 +1231,6 @@
dev_pm_set_dedicated_wake_irq
disable_irq_nosync
get_random_u32
ieee80211_alloc_hw_nm
ieee80211_ap_probereq_get
ieee80211_beacon_get_tim
ieee80211_chswitch_done
ieee80211_cqm_beacon_loss_notify
ieee80211_cqm_rssi_notify
ieee80211_csa_finish
ieee80211_free_hw
ieee80211_free_txskb
ieee80211_freq_khz_to_channel
ieee80211_hdrlen
ieee80211_iterate_active_interfaces_atomic
ieee80211_iterate_interfaces
ieee80211_nullfunc_get
ieee80211_probereq_get
ieee80211_proberesp_get
ieee80211_pspoll_get
ieee80211_queue_work
ieee80211_ready_on_channel
ieee80211_register_hw
ieee80211_remain_on_channel_expired
ieee80211_report_low_ack
ieee80211_restart_hw
ieee80211_rx_napi
ieee80211_scan_completed
ieee80211_sched_scan_results
ieee80211_sched_scan_stopped
ieee80211_sta_ps_transition
ieee80211_stop_queue
ieee80211_stop_queues
ieee80211_tx_status
ieee80211_unregister_hw
ieee80211_wake_queue
ieee80211_wake_queues
jiffies_to_msecs
jiffies_to_usecs
__local_bh_enable_ip
@@ -1286,14 +1239,12 @@
no_seek_end_llseek
_raw_spin_trylock
request_firmware_nowait
rfc1042_header
skb_dequeue
skb_push
skb_put
skb_queue_head
skb_trim
vscnprintf
wiphy_to_ieee80211_hw
# required by wlcore_sdio.ko
platform_device_add

735
android/abi_gki_aarch64_mtk
View File

File diff suppressed because it is too large Load Diff

2023
android/abi_gki_aarch64_oplus Normal file
View File

File diff suppressed because it is too large Load Diff

166
android/abi_gki_aarch64_qcom
View File

@@ -24,16 +24,20 @@
alloc_netdev_mqs
__alloc_pages_nodemask
__alloc_percpu
__alloc_percpu_gfp
__alloc_skb
alloc_skb_with_frags
alloc_workqueue
amba_bustype
amba_driver_register
amba_driver_unregister
android_debug_for_each_module
android_debug_per_cpu_symbol
android_debug_symbol
android_rvh_probe_register
anon_inode_getfile
arc4_crypt
arc4_setkey
__arch_clear_user
__arch_copy_from_user
__arch_copy_in_user
@@ -121,37 +125,6 @@
cdev_device_add
cdev_device_del
cdev_init
__cfg80211_alloc_event_skb
__cfg80211_alloc_reply_skb
cfg80211_calculate_bitrate
cfg80211_chandef_create
cfg80211_ch_switch_notify
cfg80211_connect_done
cfg80211_del_sta_sinfo
cfg80211_disconnected
cfg80211_external_auth_request
cfg80211_ft_event
cfg80211_get_bss
cfg80211_gtk_rekey_notify
cfg80211_inform_bss_frame_data
cfg80211_mgmt_tx_status
cfg80211_michael_mic_failure
cfg80211_new_sta
cfg80211_pmksa_candidate_notify
cfg80211_put_bss
cfg80211_ready_on_channel
cfg80211_remain_on_channel_expired
cfg80211_roamed
cfg80211_rx_mgmt_khz
cfg80211_rx_unprot_mlme_mgmt
cfg80211_scan_done
cfg80211_sched_scan_results
__cfg80211_send_event_skb
cfg80211_stop_iface
cfg80211_tdls_oper_request
cfg80211_unlink_bss
cfg80211_update_owe_info_event
cfg80211_vendor_cmd_reply
__cfi_slowpath
cgroup_path_ns
cgroup_taskset_first
@@ -242,6 +215,7 @@
copy_from_kernel_nofault
__cpu_active_mask
cpu_bit_bitmap
cpufreq_cooling_unregister
cpufreq_cpu_get
cpufreq_cpu_get_raw
cpufreq_cpu_put
@@ -293,8 +267,11 @@
cpus_read_unlock
cpu_subsys
cpu_topology
crc32_be
crc32_le
crc8
crc8_populate_msb
crypto_aead_decrypt
crypto_aead_encrypt
crypto_aead_setauthsize
crypto_aead_setkey
@@ -315,6 +292,7 @@
crypto_enqueue_request
crypto_has_alg
crypto_init_queue
__crypto_memneq
crypto_register_aead
crypto_register_ahash
crypto_register_alg
@@ -322,7 +300,9 @@
crypto_register_rngs
crypto_register_scomp
crypto_register_skcipher
crypto_shash_digest
crypto_shash_final
crypto_shash_finup
crypto_shash_setkey
crypto_shash_update
crypto_skcipher_decrypt
@@ -335,6 +315,7 @@
crypto_unregister_rngs
crypto_unregister_scomp
crypto_unregister_skcipher
__crypto_xor
css_next_child
csum_ipv6_magic
csum_partial
@@ -365,6 +346,7 @@
debugfs_lookup
debugfs_print_regs32
debugfs_remove
debugfs_rename
dec_zone_page_state
default_llseek
deferred_free
@@ -375,6 +357,8 @@
desc_to_gpio
destroy_workqueue
dev_alloc_name
dev_change_net_namespace
dev_close
dev_coredumpv
_dev_crit
__dev_direct_xmit
@@ -382,6 +366,7 @@
_dev_emerg
_dev_err
dev_err_probe
dev_fetch_sw_netstats
devfreq_add_device
devfreq_add_governor
devfreq_cooling_unregister
@@ -421,6 +406,7 @@
device_property_read_u8_array
device_register
device_remove_file
device_rename
device_set_wakeup_capable
device_show_int
device_store_int
@@ -879,6 +865,7 @@
enable_percpu_irq
ether_setup
eth_mac_addr
ethtool_op_get_link
eth_type_trans
eventfd_ctx_fdget
eventfd_ctx_fileget
@@ -895,6 +882,7 @@
extcon_register_notifier
extcon_set_state_sync
extcon_unregister_notifier
failure_tracking
fasync_helper
__fdget
fd_install
@@ -945,6 +933,7 @@
generic_file_llseek
generic_handle_irq
generic_iommu_put_resv_regions
genlmsg_multicast_allns
genlmsg_put
genl_register_family
genl_unregister_family
@@ -967,6 +956,8 @@
get_each_object_track
__get_free_pages
get_governor_parent_kobj
get_net_ns_by_fd
get_net_ns_by_pid
get_option
get_page_owner_handle
get_pfnblock_flags_mask
@@ -1043,6 +1034,9 @@
hvc_kick
hvc_poll
hvc_remove
__hw_addr_init
__hw_addr_sync
__hw_addr_unsync
hwrng_register
hwrng_unregister
hwspin_lock_free
@@ -1059,6 +1053,20 @@
i2c_register_driver
i2c_transfer
i2c_transfer_buffer_flags
i3c_generic_ibi_alloc_pool
i3c_generic_ibi_free_pool
i3c_generic_ibi_get_free_slot
i3c_generic_ibi_recycle_slot
i3c_master_add_i3c_dev_locked
i3c_master_disec_locked
i3c_master_do_daa
i3c_master_enec_locked
i3c_master_entdaa_locked
i3c_master_get_free_addr
i3c_master_queue_ibi
i3c_master_register
i3c_master_set_info
i3c_master_unregister
icc_disable
icc_enable
icc_get
@@ -1076,6 +1084,7 @@
ida_free
idr_alloc
idr_alloc_cyclic
idr_alloc_u32
idr_destroy
idr_find
idr_for_each
@@ -1083,9 +1092,6 @@
idr_preload
idr_remove
idr_replace
ieee80211_freq_khz_to_channel
ieee80211_get_channel_khz
ieee80211_hdrlen
iio_channel_get_all
iio_read_channel_processed
import_iovec
@@ -1094,6 +1100,7 @@
inc_node_page_state
inc_zone_page_state
in_egroup_p
inet_csk_get_port
inet_proto_csum_replace4
init_dummy_netdev
init_iova_domain
@@ -1243,6 +1250,8 @@
kernel_connect
kernel_getsockname
kernel_kobj
kernel_param_lock
kernel_param_unlock
kernel_power_off
kernel_recvmsg
kernel_restart
@@ -1252,6 +1261,9 @@
kernfs_put
kern_mount
kern_unmount
key_create_or_update
key_put
keyring_alloc
__kfifo_alloc
__kfifo_free
__kfifo_in
@@ -1260,6 +1272,7 @@
kfree_const
kfree_sensitive
kfree_skb
kfree_skb_list
kick_all_cpus_sync
kill_anon_super
kill_fasync
@@ -1333,6 +1346,7 @@
kthread_unpark
kthread_worker_fn
ktime_get
ktime_get_coarse_with_offset
ktime_get_mono_fast_ns
ktime_get_real_seconds
ktime_get_real_ts64
@@ -1422,6 +1436,7 @@
mmc_send_tuning
mmput
mod_delayed_work_on
__mod_node_page_state
mod_node_page_state
mod_timer
__module_get
@@ -1448,8 +1463,11 @@
neigh_lookup
neigh_xmit
__netdev_alloc_skb
netdev_err
netdev_info
netdev_rx_handler_register
netdev_rx_handler_unregister
netdev_set_default_ethtool_ops
netdev_update_features
netif_carrier_off
netif_carrier_on
@@ -1458,6 +1476,7 @@
netif_napi_add
__netif_napi_del
netif_receive_skb
netif_receive_skb_core
netif_receive_skb_list
netif_rx
netif_rx_ni
@@ -1468,7 +1487,11 @@
netlink_capable
__netlink_kernel_create
netlink_kernel_release
netlink_register_notifier
netlink_unicast
netlink_unregister_notifier
net_ns_type_operations
net_ratelimit
__next_zones_zonelist
nla_find
nla_memcpy
@@ -1784,6 +1807,7 @@
power_supply_set_property
power_supply_unregister
power_supply_unreg_notifier
prandom_bytes
prandom_u32
preempt_schedule
preempt_schedule_notrace
@@ -1809,6 +1833,7 @@
put_device
put_disk
put_iova_domain
__put_net
__put_page
put_pid
put_sg_io_hdr
@@ -1892,6 +1917,7 @@
rcuwait_wake_up
rdev_get_drvdata
reboot_mode
reclaim_shmem_address_space
refcount_dec_and_lock
refcount_dec_not_one
refcount_warn_saturate
@@ -1916,6 +1942,7 @@
register_netdevice_notifier
register_netevent_notifier
register_oom_notifier
register_pernet_device
register_pm_notifier
register_reboot_notifier
register_restart_handler
@@ -1958,7 +1985,6 @@
regulator_set_mode
regulator_set_voltage
regulator_unregister_notifier
regulatory_set_wiphy_regd
release_firmware
__release_region
release_sock
@@ -1985,10 +2011,20 @@
return_address
revalidate_disk_size
rfkill_alloc
rfkill_blocked
rfkill_destroy
rfkill_init_sw_state
rfkill_pause_polling
rfkill_register
rfkill_resume_polling
rfkill_set_hw_state
rfkill_unregister
rhashtable_free_and_destroy
rhashtable_insert_slow
rhltable_init
__rht_bucket_nested
rht_bucket_nested
rht_bucket_nested_insert
rndis_deregister
rndis_free_response
rndis_get_next_response
@@ -2001,7 +2037,9 @@
rndis_signal_connect
rndis_uninit
root_task_group
round_jiffies
round_jiffies_relative
round_jiffies_up
rpmsg_get_signals
rpmsg_poll
rpmsg_register_device
@@ -2013,8 +2051,10 @@
rproc_add_subdev
rproc_alloc
rproc_boot
rproc_coredump
rproc_coredump_add_custom_segment
rproc_coredump_add_segment
rproc_coredump_cleanup
rproc_coredump_set_elf_info
rproc_coredump_using_sections
rproc_del
@@ -2053,6 +2093,7 @@
sched_setscheduler
sched_setscheduler_nocheck
sched_show_task
sched_trace_rd_span
sched_uclamp_used
schedule
schedule_hrtimeout
@@ -2129,7 +2170,9 @@
sg_pcopy_from_buffer
sg_pcopy_to_buffer
sg_scsi_ioctl
shmem_file_setup
shmem_mark_page_lazyfree
shmem_read_mapping_page_gfp
shmem_truncate_range
show_rcu_gp_kthreads
show_regs
@@ -2150,8 +2193,11 @@
skb_add_rx_frag
skb_append_pagefrags
skb_checksum
skb_checksum_help
skb_clone
skb_clone_sk
skb_coalesce_rx_frag
skb_complete_wifi_ack
skb_copy
skb_copy_bits
skb_copy_datagram_iter
@@ -2242,6 +2288,7 @@
soc_device_register
soc_device_unregister
sock_alloc_send_skb
__sock_create
sock_create_kern
sock_gettstamp
sock_init_data
@@ -2320,6 +2367,7 @@
strscpy
strsep
strstr
__sw_hweight16
__sw_hweight32
__sw_hweight64
__sw_hweight8
@@ -2427,6 +2475,8 @@
__traceiter_android_rvh_find_busiest_queue
__traceiter_android_rvh_find_lowest_rq
__traceiter_android_rvh_flush_task
__traceiter_android_rvh_force_compatible_post
__traceiter_android_rvh_force_compatible_pre
__traceiter_android_rvh_gic_v3_set_affinity
__traceiter_android_rvh_irqs_disable
__traceiter_android_rvh_irqs_enable
@@ -2473,8 +2523,6 @@
__traceiter_android_vh_cpuidle_psci_enter
__traceiter_android_vh_cpuidle_psci_exit
__traceiter_android_vh_dump_throttled_rt_tasks
__traceiter_android_vh_force_compatible_post
__traceiter_android_vh_force_compatible_pre
__traceiter_android_vh_freq_table_limits
__traceiter_android_vh_ftrace_dump_buffer
__traceiter_android_vh_ftrace_format_check
@@ -2487,12 +2535,24 @@
__traceiter_android_vh_ipi_stop
__traceiter_android_vh_jiffies_update
__traceiter_android_vh_logbuf
__traceiter_android_vh_logbuf_pr_cont
__traceiter_android_vh_printk_hotplug
__traceiter_android_vh_rproc_recovery
__traceiter_android_vh_scheduler_tick
__traceiter_android_vh_show_max_freq
__traceiter_android_vh_show_resume_epoch_val
__traceiter_android_vh_show_suspend_epoch_val
__traceiter_android_vh_timer_calc_index
__traceiter_android_vh_ufs_check_int_errors
__traceiter_android_vh_ufs_clock_scaling
__traceiter_android_vh_ufs_compl_command
__traceiter_android_vh_ufs_fill_prdt
__traceiter_android_vh_ufs_prepare_command
__traceiter_android_vh_ufs_send_command
__traceiter_android_vh_ufs_send_tm_command
__traceiter_android_vh_ufs_send_uic_command
__traceiter_android_vh_ufs_update_sdev
__traceiter_android_vh_ufs_update_sysfs
__traceiter_android_vh_update_topology_flags_workfn
__traceiter_binder_transaction_received
__traceiter_cpu_frequency
@@ -2505,6 +2565,7 @@
__traceiter_rwmmio_post_read
__traceiter_rwmmio_read
__traceiter_rwmmio_write
__traceiter_sched_overutilized_tp
__traceiter_sched_switch
__traceiter_suspend_resume
__tracepoint_android_rvh_account_irq
@@ -2520,6 +2581,8 @@
__tracepoint_android_rvh_find_busiest_queue
__tracepoint_android_rvh_find_lowest_rq
__tracepoint_android_rvh_flush_task
__tracepoint_android_rvh_force_compatible_post
__tracepoint_android_rvh_force_compatible_pre
__tracepoint_android_rvh_gic_v3_set_affinity
__tracepoint_android_rvh_irqs_disable
__tracepoint_android_rvh_irqs_enable
@@ -2557,6 +2620,7 @@
__tracepoint_android_rvh_update_misfit_status
__tracepoint_android_rvh_wake_up_new_task
__tracepoint_android_vh_allow_domain_state
__tracepoint_android_vh_alter_rwsem_list_add
__tracepoint_android_vh_binder_restore_priority
__tracepoint_android_vh_binder_set_priority
__tracepoint_android_vh_binder_transaction_init
@@ -2568,8 +2632,6 @@
__tracepoint_android_vh_cpuidle_psci_enter
__tracepoint_android_vh_cpuidle_psci_exit
__tracepoint_android_vh_dump_throttled_rt_tasks
__tracepoint_android_vh_force_compatible_post
__tracepoint_android_vh_force_compatible_pre
__tracepoint_android_vh_freq_table_limits
__tracepoint_android_vh_ftrace_dump_buffer
__tracepoint_android_vh_ftrace_format_check
@@ -2582,15 +2644,29 @@
__tracepoint_android_vh_ipi_stop
__tracepoint_android_vh_jiffies_update
__tracepoint_android_vh_logbuf
__tracepoint_android_vh_logbuf_pr_cont
__tracepoint_android_vh_oom_check_panic
__tracepoint_android_vh_printk_hotplug
__tracepoint_android_vh_process_killed
__tracepoint_android_vh_psi_event
__tracepoint_android_vh_psi_group
__tracepoint_android_vh_rproc_recovery
__tracepoint_android_vh_scheduler_tick
__tracepoint_android_vh_shmem_alloc_page
__tracepoint_android_vh_show_max_freq
__tracepoint_android_vh_show_resume_epoch_val
__tracepoint_android_vh_show_suspend_epoch_val
__tracepoint_android_vh_timer_calc_index
__tracepoint_android_vh_ufs_check_int_errors
__tracepoint_android_vh_ufs_clock_scaling
__tracepoint_android_vh_ufs_compl_command
__tracepoint_android_vh_ufs_fill_prdt
__tracepoint_android_vh_ufs_prepare_command
__tracepoint_android_vh_ufs_send_command
__tracepoint_android_vh_ufs_send_tm_command
__tracepoint_android_vh_ufs_send_uic_command
__tracepoint_android_vh_ufs_update_sdev
__tracepoint_android_vh_ufs_update_sysfs
__tracepoint_android_vh_update_topology_flags_workfn
__tracepoint_binder_transaction_received
__tracepoint_cpu_frequency
@@ -2606,6 +2682,7 @@
__tracepoint_rwmmio_post_read
__tracepoint_rwmmio_read
__tracepoint_rwmmio_write
__tracepoint_sched_overutilized_tp
__tracepoint_sched_switch
__tracepoint_suspend_resume
trace_print_array_seq
@@ -2702,6 +2779,7 @@
unregister_netdevice_queue
unregister_netevent_notifier
unregister_oom_notifier
unregister_pernet_device
unregister_pm_notifier
unregister_reboot_notifier
unregister_restart_handler
@@ -2846,6 +2924,7 @@
vchan_init
vchan_tx_desc_free
vchan_tx_submit
verify_pkcs7_signature
vfree
video_devdata
video_device_alloc
@@ -2860,6 +2939,7 @@
vmalloc_to_pfn
vmap
vmemdup_user
vm_event_states
vmf_insert_mixed
vmf_insert_pfn
vm_get_page_prot
@@ -2890,10 +2970,7 @@
wakeup_source_register
wakeup_source_unregister
__warn_printk
wiphy_free
wiphy_new_nm
wiphy_register
wiphy_unregister
wireless_nlevent_flush
wireless_send_event
woken_wake_function
work_busy
@@ -2906,6 +2983,9 @@
xa_find_after
xa_load
xa_store
__xfrm_decode_session
xfrm_lookup
xfrm_output
xhci_alloc_command
xhci_alloc_erst
xhci_free_command

37
android/abi_gki_aarch64_unisoc
View File

@@ -317,9 +317,6 @@
idr_find
idr_for_each
idr_remove
ieee80211_channel_to_freq_khz
ieee80211_freq_khz_to_channel
ieee80211_get_channel_khz
iget_failed
iget5_locked
ignore_console_lock_warning
@@ -1956,40 +1953,11 @@
# required by sprdwl_ng.ko
bcmp
build_skb
__cfg80211_alloc_event_skb
__cfg80211_alloc_reply_skb
cfg80211_chandef_create
cfg80211_ch_switch_notify
cfg80211_connect_done
cfg80211_cqm_rssi_notify
cfg80211_del_sta_sinfo
cfg80211_disconnected
cfg80211_find_elem_match
cfg80211_get_bss
cfg80211_ibss_joined
cfg80211_inform_bss_data
cfg80211_mgmt_tx_status
cfg80211_michael_mic_failure
cfg80211_new_sta
cfg80211_put_bss
cfg80211_ready_on_channel
cfg80211_remain_on_channel_expired
cfg80211_roamed
cfg80211_rx_mgmt
cfg80211_rx_unprot_mlme_mgmt
cfg80211_scan_done
cfg80211_sched_scan_results
__cfg80211_send_event_skb
cfg80211_tdls_oper_request
cfg80211_unlink_bss
cfg80211_unregister_wdev
cfg80211_vendor_cmd_reply
console_printk
consume_skb
_ctype
dev_get_by_index
down_timeout
freq_reg_info
genlmsg_put
jiffies_to_usecs
kfree_skb_list
@@ -2007,7 +1975,6 @@
register_inet6addr_notifier
register_inetaddr_notifier
register_netdevice
regulatory_hint
rtnl_lock
rtnl_unlock
simple_open
@@ -2017,10 +1984,6 @@
unregister_inet6addr_notifier
unregister_inetaddr_notifier
unregister_netdevice_queue
wiphy_free
wiphy_new_nm
wiphy_register
wiphy_unregister
# required by sunwave_fp.ko
input_unregister_device

66
android/abi_gki_aarch64_virtual_device
View File

@@ -34,8 +34,6 @@
cancel_delayed_work_sync
cancel_work_sync
capable
cfg80211_inform_bss_data
cfg80211_put_bss
__cfi_slowpath
__check_object_size
__class_create
@@ -565,10 +563,6 @@
# required by mac80211_hwsim.ko
alloc_netdev_mqs
__cfg80211_alloc_event_skb
__cfg80211_alloc_reply_skb
__cfg80211_send_event_skb
cfg80211_vendor_cmd_reply
debugfs_attr_read
debugfs_attr_write
dev_alloc_name
@@ -583,28 +577,6 @@
hrtimer_forward
hrtimer_init
hrtimer_start_range_ns
ieee80211_alloc_hw_nm
ieee80211_beacon_cntdwn_is_complete
ieee80211_beacon_get_tim
ieee80211_csa_finish
ieee80211_free_hw
ieee80211_free_txskb
ieee80211_get_buffered_bc
ieee80211_get_tx_rates
ieee80211_iterate_active_interfaces_atomic
ieee80211_probereq_get
ieee80211_queue_delayed_work
ieee80211_radar_detected
ieee80211_ready_on_channel
ieee80211_register_hw
ieee80211_remain_on_channel_expired
ieee80211_rx_irqsafe
ieee80211_scan_completed
ieee80211_stop_queues
ieee80211_stop_tx_ba_cb_irqsafe
ieee80211_tx_status_irqsafe
ieee80211_unregister_hw
ieee80211_wake_queues
init_net
__netdev_alloc_skb
netif_rx
@@ -619,7 +591,6 @@
nla_put
param_ops_ushort
register_pernet_device
regulatory_hint
rhashtable_destroy
rhashtable_init
rhashtable_insert_slow
@@ -635,7 +606,6 @@
skb_trim
skb_unlink
unregister_pernet_device
wiphy_apply_custom_regulatory
# required by md-mod.ko
ack_all_badblocks
@@ -940,9 +910,6 @@
devm_mfd_add_devices
# required by virt_wifi.ko
cfg80211_connect_done
cfg80211_disconnected
cfg80211_scan_done
__dev_get_by_index
dev_printk
__module_get
@@ -952,13 +919,8 @@
rtnl_link_unregister
skb_clone
unregister_netdevice_many
wiphy_free
wiphy_new_nm
wiphy_register
wiphy_unregister
# required by virt_wifi_sim.ko
ieee80211_get_channel_khz
release_firmware
request_firmware
@@ -1233,6 +1195,8 @@
# required by virtio_pci.ko
irq_set_affinity_hint
pci_alloc_irq_vectors_affinity
pci_disable_sriov
pci_enable_sriov
pci_find_capability
pci_find_ext_capability
pci_find_next_capability
@@ -1242,6 +1206,7 @@
pci_irq_vector
pci_release_selected_regions
pci_request_selected_regions
pci_vfs_assigned
synchronize_irq
virtio_device_freeze
virtio_device_restore
@@ -1322,3 +1287,28 @@
_raw_read_unlock
_raw_write_lock
_raw_write_unlock
# required by gs_usb.ko
usb_kill_anchored_urbs
alloc_candev_mqs
register_candev
free_candev
can_change_mtu
open_candev
usb_anchor_urb
usb_unanchor_urb
alloc_can_skb
can_get_echo_skb
alloc_can_err_skb
close_candev
can_put_echo_skb
can_free_echo_skb
unregister_candev
# required by vcan.ko
sock_efree
# required by slcan.ko
tty_mode_ioctl
tty_hangup
hex_asc_upper

1997
android/abi_gki_aarch64_vivo Normal file
View File

File diff suppressed because it is too large Load Diff

1
arch/arm/Kconfig
View File

@@ -51,6 +51,7 @@ config ARM
select GENERIC_ATOMIC64 if CPU_V7M || CPU_V6 || !CPU_32v6K || !AEABI
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
select GENERIC_IRQ_IPI if SMP
select ARCH_WANTS_IRQ_RAW if GENERIC_IRQ_IPI
select GENERIC_CPU_AUTOPROBE
select GENERIC_EARLY_IOREMAP
select GENERIC_IDLE_POLL_SETUP

4
arch/arm/kernel/smp.c
View File

@@ -741,6 +741,10 @@ void __init set_smp_ipi_range(int ipi_base, int n)
ipi_desc[i] = irq_to_desc(ipi_base + i);
irq_set_status_flags(ipi_base + i, IRQ_HIDDEN);
/* The recheduling IPI is special... */
if (i == IPI_RESCHEDULE)
__irq_modify_status(ipi_base + i, 0, IRQ_RAW, ~0);
}
ipi_irq_base = ipi_base;

1
arch/arm64/Kconfig
View File

@@ -111,6 +111,7 @@ config ARM64
select GENERIC_EARLY_IOREMAP
select GENERIC_IDLE_POLL_SETUP
select GENERIC_IRQ_IPI
select ARCH_WANTS_IRQ_RAW
select GENERIC_IRQ_MULTI_HANDLER
select GENERIC_IRQ_PROBE
select GENERIC_IRQ_SHOW

5
arch/arm64/configs/db845c_gki.fragment
View File

@@ -1,3 +1,8 @@
CONFIG_CFG80211=m
CONFIG_NL80211_TESTMODE=y
# CONFIG_CFG80211_DEFAULT_PS is not set
# CONFIG_CFG80211_CRDA_SUPPORT is not set
CONFIG_MAC80211=m
CONFIG_QRTR=m
CONFIG_QRTR_TUN=m
CONFIG_SCSI_UFS_QCOM=m

14
arch/arm64/configs/gki_defconfig
View File

@@ -15,6 +15,7 @@ CONFIG_IKHEADERS=y
CONFIG_UCLAMP_TASK=y
CONFIG_UCLAMP_BUCKETS_COUNT=20
CONFIG_CGROUPS=y
CONFIG_MEMCG=y
CONFIG_BLK_CGROUP=y
CONFIG_CGROUP_SCHED=y
CONFIG_UCLAMP_TASK_GROUP=y
@@ -48,6 +49,7 @@ CONFIG_PROFILING=y
CONFIG_ARCH_SUNXI=y
CONFIG_ARCH_HISI=y
CONFIG_ARCH_QCOM=y
CONFIG_ARCH_SPRD=y
CONFIG_SCHED_MC=y
CONFIG_NR_CPUS=32
CONFIG_PARAVIRT=y
@@ -59,7 +61,7 @@ CONFIG_CP15_BARRIER_EMULATION=y
CONFIG_SETEND_EMULATION=y
CONFIG_RANDOMIZE_BASE=y
# CONFIG_RANDOMIZE_MODULE_REGION_FULL is not set
CONFIG_CMDLINE="stack_depot_disable=on kasan.stacktrace=off kvm-arm.mode=protected"
CONFIG_CMDLINE="stack_depot_disable=on kasan.stacktrace=off kvm-arm.mode=protected cgroup_disable=pressure cgroup.memory=nokmem"
CONFIG_CMDLINE_EXTEND=y
# CONFIG_DMI is not set
CONFIG_PM_WAKELOCKS=y
@@ -96,6 +98,7 @@ CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODVERSIONS=y
CONFIG_MODULE_SCMVERSION=y
CONFIG_BLK_CGROUP_IOCOST=y
CONFIG_BLK_INLINE_ENCRYPTION=y
CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK=y
CONFIG_IOSCHED_BFQ=y
@@ -255,6 +258,7 @@ CONFIG_NET_ACT_GACT=y
CONFIG_NET_ACT_MIRRED=y
CONFIG_NET_ACT_SKBEDIT=y
CONFIG_VSOCKETS=y
CONFIG_CGROUP_NET_PRIO=y
CONFIG_BPF_JIT=y
CONFIG_CAN=y
CONFIG_BT=y
@@ -266,15 +270,11 @@ CONFIG_BT_HCIUART=y
CONFIG_BT_HCIUART_LL=y
CONFIG_BT_HCIUART_BCM=y
CONFIG_BT_HCIUART_QCA=y
CONFIG_CFG80211=y
CONFIG_NL80211_TESTMODE=y
# CONFIG_CFG80211_DEFAULT_PS is not set
# CONFIG_CFG80211_CRDA_SUPPORT is not set
CONFIG_MAC80211=y
CONFIG_RFKILL=y
CONFIG_PCI=y
CONFIG_PCIEPORTBUS=y
CONFIG_PCIEAER=y
CONFIG_PCI_IOV=y
CONFIG_PCI_HOST_GENERIC=y
CONFIG_PCIE_DW_PLAT_EP=y
CONFIG_PCIE_QCOM=y
@@ -438,7 +438,6 @@ CONFIG_SND=y
CONFIG_SND_HRTIMER=y
CONFIG_SND_DYNAMIC_MINORS=y
# CONFIG_SND_SUPPORT_OLD_API is not set
# CONFIG_SND_VERBOSE_PROCFS is not set
# CONFIG_SND_DRIVERS is not set
CONFIG_SND_USB_AUDIO=y
CONFIG_SND_SOC=y
@@ -520,6 +519,7 @@ CONFIG_STAGING=y
CONFIG_ASHMEM=y
CONFIG_DEBUG_KINFO=y
CONFIG_COMMON_CLK_SCPI=y
# CONFIG_SPRD_COMMON_CLK is not set
# CONFIG_CLK_SUNXI is not set
# CONFIG_SUNXI_CCU is not set
CONFIG_HWSPINLOCK=y

5
arch/arm64/configs/hikey960_gki.fragment
View File

@@ -50,3 +50,8 @@ CONFIG_PHY_HI3660_USB=m
CONFIG_PINCTRL_SINGLE=m
CONFIG_DMABUF_HEAPS_CMA=m
CONFIG_DMABUF_HEAPS_SYSTEM=m
CONFIG_CFG80211=m
CONFIG_NL80211_TESTMODE=y
# CONFIG_CFG80211_DEFAULT_PS is not set
# CONFIG_CFG80211_CRDA_SUPPORT is not set
CONFIG_MAC80211=m

20
arch/arm64/include/asm/asm_pointer_auth.h
View File

@@ -13,30 +13,12 @@
* so use the base value of ldp as thread.keys_user and offset as
* thread.keys_user.ap*.
*/
.macro ptrauth_keys_install_user tsk, tmp1, tmp2, tmp3
.macro __ptrauth_keys_install_user tsk, tmp1, tmp2, tmp3
mov \tmp1, #THREAD_KEYS_USER
add \tmp1, \tsk, \tmp1
alternative_if_not ARM64_HAS_ADDRESS_AUTH
b .Laddr_auth_skip_\@
alternative_else_nop_endif
ldp \tmp2, \tmp3, [\tmp1, #PTRAUTH_USER_KEY_APIA]
msr_s SYS_APIAKEYLO_EL1, \tmp2
msr_s SYS_APIAKEYHI_EL1, \tmp3
ldp \tmp2, \tmp3, [\tmp1, #PTRAUTH_USER_KEY_APIB]
msr_s SYS_APIBKEYLO_EL1, \tmp2
msr_s SYS_APIBKEYHI_EL1, \tmp3
ldp \tmp2, \tmp3, [\tmp1, #PTRAUTH_USER_KEY_APDA]
msr_s SYS_APDAKEYLO_EL1, \tmp2
msr_s SYS_APDAKEYHI_EL1, \tmp3
ldp \tmp2, \tmp3, [\tmp1, #PTRAUTH_USER_KEY_APDB]
msr_s SYS_APDBKEYLO_EL1, \tmp2
msr_s SYS_APDBKEYHI_EL1, \tmp3
.Laddr_auth_skip_\@:
alternative_if ARM64_HAS_GENERIC_AUTH
ldp \tmp2, \tmp3, [\tmp1, #PTRAUTH_USER_KEY_APGA]
msr_s SYS_APGAKEYLO_EL1, \tmp2
msr_s SYS_APGAKEYHI_EL1, \tmp3
alternative_else_nop_endif
.endm
.macro __ptrauth_keys_install_kernel_nosync tsk, tmp1, tmp2, tmp3

2
arch/arm64/include/asm/lse.h
View File

@@ -4,7 +4,7 @@
#include <asm/atomic_ll_sc.h>
#ifdef CONFIG_ARM64_LSE_ATOMICS
#if defined(CONFIG_ARM64_LSE_ATOMICS) && !defined(BUILD_FIPS140_KO)
#define __LSE_PREAMBLE ".arch_extension lse\n"

8
arch/arm64/include/asm/mte.h
View File

@@ -37,9 +37,10 @@ void mte_free_tag_storage(char *storage);
/* track which pages have valid allocation tags */
#define PG_mte_tagged PG_arch_2
void mte_zero_clear_page_tags(void *addr);
void mte_sync_tags(pte_t old_pte, pte_t pte);
void mte_copy_page_tags(void *kto, const void *kfrom);
void flush_mte_state(void);
void mte_thread_init_user(void);
void mte_thread_switch(struct task_struct *next);
void mte_suspend_enter(void);
void mte_suspend_exit(void);
@@ -53,13 +54,16 @@ int mte_ptrace_copy_tags(struct task_struct *child, long request,
/* unused if !CONFIG_ARM64_MTE, silence the compiler */
#define PG_mte_tagged 0
static inline void mte_zero_clear_page_tags(void *addr)
{
}
static inline void mte_sync_tags(pte_t old_pte, pte_t pte)
{
}
static inline void mte_copy_page_tags(void *kto, const void *kfrom)
{
}
static inline void flush_mte_state(void)
static inline void mte_thread_init_user(void)
{
}
static inline void mte_thread_switch(struct task_struct *next)

8
arch/arm64/include/asm/page.h
View File

@@ -13,6 +13,7 @@
#ifndef __ASSEMBLY__
#include <linux/personality.h> /* for READ_IMPLIES_EXEC */
#include <linux/types.h> /* for gfp_t */
#include <asm/pgtable-types.h>
struct page;
@@ -28,10 +29,13 @@ void copy_user_highpage(struct page *to, struct page *from,
void copy_highpage(struct page *to, struct page *from);
#define __HAVE_ARCH_COPY_HIGHPAGE
#define alloc_zeroed_user_highpage_movable(vma, vaddr) \
alloc_page_vma(GFP_HIGHUSER_MOVABLE | __GFP_ZERO | __GFP_CMA, vma, vaddr)
struct page *alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma,
unsigned long vaddr);
#define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE
void tag_clear_highpage(struct page *to);
#define __HAVE_ARCH_TAG_CLEAR_HIGHPAGE
#define clear_user_page(page, vaddr, pg) clear_page(page)
#define copy_user_page(to, from, vaddr, pg) copy_page(to, from)

61
arch/arm64/include/asm/pointer_auth.h
View File

@@ -3,12 +3,16 @@
#define __ASM_POINTER_AUTH_H
#include <linux/bitops.h>
#include <linux/prctl.h>
#include <linux/random.h>
#include <asm/cpufeature.h>
#include <asm/memory.h>
#include <asm/sysreg.h>
#define PR_PAC_ENABLED_KEYS_MASK \
(PR_PAC_APIAKEY | PR_PAC_APIBKEY | PR_PAC_APDAKEY | PR_PAC_APDBKEY)
#ifdef CONFIG_ARM64_PTR_AUTH
/*
* Each key is a 128-bit quantity which is split across a pair of 64-bit
@@ -34,6 +38,25 @@ struct ptrauth_keys_kernel {
struct ptrauth_key apia;
};
#define __ptrauth_key_install_nosync(k, v) \
do { \
struct ptrauth_key __pki_v = (v); \
write_sysreg_s(__pki_v.lo, SYS_ ## k ## KEYLO_EL1); \
write_sysreg_s(__pki_v.hi, SYS_ ## k ## KEYHI_EL1); \
} while (0)
static inline void ptrauth_keys_install_user(struct ptrauth_keys_user *keys)
{
if (system_supports_address_auth()) {
__ptrauth_key_install_nosync(APIB, keys->apib);
__ptrauth_key_install_nosync(APDA, keys->apda);
__ptrauth_key_install_nosync(APDB, keys->apdb);
}
if (system_supports_generic_auth())
__ptrauth_key_install_nosync(APGA, keys->apga);
}
static inline void ptrauth_keys_init_user(struct ptrauth_keys_user *keys)
{
if (system_supports_address_auth()) {
@@ -45,14 +68,9 @@ static inline void ptrauth_keys_init_user(struct ptrauth_keys_user *keys)
if (system_supports_generic_auth())
get_random_bytes(&keys->apga, sizeof(keys->apga));
}
#define __ptrauth_key_install_nosync(k, v) \
do { \
struct ptrauth_key __pki_v = (v); \
write_sysreg_s(__pki_v.lo, SYS_ ## k ## KEYLO_EL1); \
write_sysreg_s(__pki_v.hi, SYS_ ## k ## KEYHI_EL1); \
} while (0)
ptrauth_keys_install_user(keys);
}
static __always_inline void ptrauth_keys_init_kernel(struct ptrauth_keys_kernel *keys)
{
@@ -71,6 +89,10 @@ static __always_inline void ptrauth_keys_switch_kernel(struct ptrauth_keys_kerne
extern int ptrauth_prctl_reset_keys(struct task_struct *tsk, unsigned long arg);
extern int ptrauth_set_enabled_keys(struct task_struct *tsk, unsigned long keys,
unsigned long enabled);
extern int ptrauth_get_enabled_keys(struct task_struct *tsk);
static inline unsigned long ptrauth_strip_insn_pac(unsigned long ptr)
{
return ptrauth_clear_pac(ptr);
@@ -85,8 +107,23 @@ static __always_inline void ptrauth_enable(void)
isb();
}
#define ptrauth_thread_init_user(tsk) \
ptrauth_keys_init_user(&(tsk)->thread.keys_user)
#define ptrauth_suspend_exit() \
ptrauth_keys_install_user(&current->thread.keys_user)
#define ptrauth_thread_init_user() \
do { \
ptrauth_keys_init_user(&current->thread.keys_user); \
\
/* enable all keys */ \
if (system_supports_address_auth()) \
ptrauth_set_enabled_keys(current, \
PR_PAC_ENABLED_KEYS_MASK, \
PR_PAC_ENABLED_KEYS_MASK); \
} while (0)
#define ptrauth_thread_switch_user(tsk) \
ptrauth_keys_install_user(&(tsk)->thread.keys_user)
#define ptrauth_thread_init_kernel(tsk) \
ptrauth_keys_init_kernel(&(tsk)->thread.keys_kernel)
#define ptrauth_thread_switch_kernel(tsk) \
@@ -95,9 +132,13 @@ static __always_inline void ptrauth_enable(void)
#else /* CONFIG_ARM64_PTR_AUTH */
#define ptrauth_enable()
#define ptrauth_prctl_reset_keys(tsk, arg) (-EINVAL)
#define ptrauth_set_enabled_keys(tsk, keys, enabled) (-EINVAL)
#define ptrauth_get_enabled_keys(tsk) (-EINVAL)
#define ptrauth_strip_insn_pac(lr) (lr)
#define ptrauth_thread_init_user(tsk)
#define ptrauth_suspend_exit()
#define ptrauth_thread_init_user()
#define ptrauth_thread_init_kernel(tsk)
#define ptrauth_thread_switch_user(tsk)
#define ptrauth_thread_switch_kernel(tsk)
#endif /* CONFIG_ARM64_PTR_AUTH */

19
arch/arm64/include/asm/processor.h
View File

@@ -19,6 +19,12 @@
*/
#define NET_IP_ALIGN 0
#define MTE_CTRL_GCR_USER_EXCL_SHIFT 0
#define MTE_CTRL_GCR_USER_EXCL_MASK 0xffff
#define MTE_CTRL_TCF_SYNC (1UL << 16)
#define MTE_CTRL_TCF_ASYNC (1UL << 17)
#ifndef __ASSEMBLY__
#include <linux/build_bug.h>
@@ -156,12 +162,14 @@ struct thread_struct {
struct ptrauth_keys_kernel keys_kernel;
#endif
#ifdef CONFIG_ARM64_MTE
u64 gcr_user_excl;
u64 mte_ctrl;
#endif
u64 sctlr_user;
};
#define SCTLR_USER_MASK SCTLR_EL1_TCF0_MASK
#define SCTLR_USER_MASK \
(SCTLR_ELx_ENIA | SCTLR_ELx_ENIB | SCTLR_ELx_ENDA | SCTLR_ELx_ENDB | \
SCTLR_EL1_TCF0_MASK)
static inline void arch_thread_struct_whitelist(unsigned long *offset,
unsigned long *size)
@@ -254,7 +262,7 @@ extern void release_thread(struct task_struct *);
unsigned long get_wchan(struct task_struct *p);
void set_task_sctlr_el1(u64 sctlr);
void update_sctlr_el1(u64 sctlr);
/* Thread switching */
extern struct task_struct *cpu_switch_to(struct task_struct *prev,
@@ -310,6 +318,11 @@ extern void __init minsigstksz_setup(void);
/* PR_PAC_RESET_KEYS prctl */
#define PAC_RESET_KEYS(tsk, arg) ptrauth_prctl_reset_keys(tsk, arg)
/* PR_PAC_{SET,GET}_ENABLED_KEYS prctl */
#define PAC_SET_ENABLED_KEYS(tsk, keys, enabled) \
ptrauth_set_enabled_keys(tsk, keys, enabled)
#define PAC_GET_ENABLED_KEYS(tsk) ptrauth_get_enabled_keys(tsk)
#ifdef CONFIG_ARM64_TAGGED_ADDR_ABI
/* PR_{SET,GET}_TAGGED_ADDR_CTRL prctl */
long set_tagged_addr_ctrl(struct task_struct *task, unsigned long arg);

4
arch/arm64/include/asm/sysreg.h
View File

@@ -612,8 +612,10 @@
#define SCTLR_ELx_TCF_ASYNC (UL(0x2) << SCTLR_ELx_TCF_SHIFT)
#define SCTLR_ELx_TCF_MASK (UL(0x3) << SCTLR_ELx_TCF_SHIFT)
#define SCTLR_ELx_ENIA_SHIFT 31
#define SCTLR_ELx_ITFSB (BIT(37))
#define SCTLR_ELx_ENIA (BIT(31))
#define SCTLR_ELx_ENIA (BIT(SCTLR_ELx_ENIA_SHIFT))
#define SCTLR_ELx_ENIB (BIT(30))
#define SCTLR_ELx_ENDA (BIT(27))
#define SCTLR_ELx_EE (BIT(25))

7
arch/arm64/kernel/asm-offsets.c
View File

@@ -44,12 +44,13 @@ int main(void)
#endif
BLANK();
DEFINE(THREAD_CPU_CONTEXT, offsetof(struct task_struct, thread.cpu_context));
DEFINE(THREAD_SCTLR_USER, offsetof(struct task_struct, thread.sctlr_user));
#ifdef CONFIG_ARM64_PTR_AUTH
DEFINE(THREAD_KEYS_USER, offsetof(struct task_struct, thread.keys_user));
DEFINE(THREAD_KEYS_KERNEL, offsetof(struct task_struct, thread.keys_kernel));
#endif
#ifdef CONFIG_ARM64_MTE
DEFINE(THREAD_GCR_EL1_USER, offsetof(struct task_struct, thread.gcr_user_excl));
DEFINE(THREAD_MTE_CTRL, offsetof(struct task_struct, thread.mte_ctrl));
#endif
BLANK();
DEFINE(S_X0, offsetof(struct pt_regs, regs[0]));
@@ -164,10 +165,6 @@ int main(void)
#endif
#ifdef CONFIG_ARM64_PTR_AUTH
DEFINE(PTRAUTH_USER_KEY_APIA, offsetof(struct ptrauth_keys_user, apia));
DEFINE(PTRAUTH_USER_KEY_APIB, offsetof(struct ptrauth_keys_user, apib));
DEFINE(PTRAUTH_USER_KEY_APDA, offsetof(struct ptrauth_keys_user, apda));
DEFINE(PTRAUTH_USER_KEY_APDB, offsetof(struct ptrauth_keys_user, apdb));
DEFINE(PTRAUTH_USER_KEY_APGA, offsetof(struct ptrauth_keys_user, apga));
DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia));
BLANK();
#endif

97
arch/arm64/kernel/entry.S
View File

@@ -148,42 +148,46 @@ alternative_cb_end
.endm
/* Check for MTE asynchronous tag check faults */
.macro check_mte_async_tcf, tmp, ti_flags
.macro check_mte_async_tcf, tmp, ti_flags, thread_sctlr
#ifdef CONFIG_ARM64_MTE
.arch_extension lse
alternative_if_not ARM64_MTE
b 1f
alternative_else_nop_endif
/*
* Asynchronous tag check faults are only possible in ASYNC (2) or
* ASYM (3) modes. In each of these modes bit 1 of SCTLR_EL1.TCF0 is
* set, so skip the check if it is unset.
*/
tbz \thread_sctlr, #(SCTLR_EL1_TCF0_SHIFT + 1), 1f
mrs_s \tmp, SYS_TFSRE0_EL1
tbz \tmp, #SYS_TFSR_EL1_TF0_SHIFT, 1f
/* Asynchronous TCF occurred for TTBR0 access, set the TI flag */
mov \tmp, #_TIF_MTE_ASYNC_FAULT
add \ti_flags, tsk, #TSK_TI_FLAGS
stset \tmp, [\ti_flags]
msr_s SYS_TFSRE0_EL1, xzr
1:
#endif
.endm
/* Clear the MTE asynchronous tag check faults */
.macro clear_mte_async_tcf
.macro clear_mte_async_tcf thread_sctlr
#ifdef CONFIG_ARM64_MTE
alternative_if ARM64_MTE
/* See comment in check_mte_async_tcf above. */
tbz \thread_sctlr, #(SCTLR_EL1_TCF0_SHIFT + 1), 1f
dsb ish
msr_s SYS_TFSRE0_EL1, xzr
1:
alternative_else_nop_endif
#endif
.endm
.macro mte_set_gcr, tmp, tmp2
.macro mte_set_gcr, mte_ctrl, tmp
#ifdef CONFIG_ARM64_MTE
/*
* Calculate and set the exclude mask preserving
* the RRND (bit[16]) setting.
*/
mrs_s \tmp2, SYS_GCR_EL1
bfi \tmp2, \tmp, #0, #16
msr_s SYS_GCR_EL1, \tmp2
ubfx \tmp, \mte_ctrl, #MTE_CTRL_GCR_USER_EXCL_SHIFT, #16
orr \tmp, \tmp, #SYS_GCR_EL1_RRND
msr_s SYS_GCR_EL1, \tmp
#endif
.endm
@@ -195,7 +199,6 @@ alternative_else_nop_endif
ldr_l \tmp, gcr_kernel_excl
mte_set_gcr \tmp, \tmp2
isb
1:
#endif
.endm
@@ -205,7 +208,7 @@ alternative_else_nop_endif
alternative_if_not ARM64_MTE
b 1f
alternative_else_nop_endif
ldr \tmp, [\tsk, #THREAD_GCR_EL1_USER]
ldr \tmp, [\tsk, #THREAD_MTE_CTRL]
mte_set_gcr \tmp, \tmp2
1:
@@ -246,13 +249,48 @@ alternative_else_nop_endif
disable_step_tsk x19, x20
/* Check for asynchronous tag check faults in user space */
check_mte_async_tcf x22, x23
ldr x0, [tsk, THREAD_SCTLR_USER]
check_mte_async_tcf x22, x23, x0
#ifdef CONFIG_ARM64_PTR_AUTH
alternative_if ARM64_HAS_ADDRESS_AUTH
/*
* Enable IA for in-kernel PAC if the task had it disabled. Although
* this could be implemented with an unconditional MRS which would avoid
* a load, this was measured to be slower on Cortex-A75 and Cortex-A76.
*
* Install the kernel IA key only if IA was enabled in the task. If IA
* was disabled on kernel exit then we would have left the kernel IA
* installed so there is no need to install it again.
*/
tbz x0, SCTLR_ELx_ENIA_SHIFT, 1f
__ptrauth_keys_install_kernel_nosync tsk, x20, x22, x23
b 2f
1:
mrs x0, sctlr_el1
orr x0, x0, SCTLR_ELx_ENIA
msr sctlr_el1, x0
2:
alternative_else_nop_endif
#endif
apply_ssbd 1, x22, x23
ptrauth_keys_install_kernel tsk, x20, x22, x23
mte_set_kernel_gcr x22, x23
/*
* Any non-self-synchronizing system register updates required for
* kernel entry should be placed before this point.
*/
alternative_if ARM64_MTE
isb
b 1f
alternative_else_nop_endif
alternative_if ARM64_HAS_ADDRESS_AUTH
isb
alternative_else_nop_endif
1:
scs_load tsk, x20
.else
add x21, sp, #S_FRAME_SIZE
@@ -367,8 +405,29 @@ alternative_else_nop_endif
3:
scs_save tsk, x0
/* No kernel C function calls after this as user keys are set. */
ptrauth_keys_install_user tsk, x0, x1, x2
/* Ignore asynchronous tag check faults in the uaccess routines */
ldr x0, [tsk, THREAD_SCTLR_USER]
clear_mte_async_tcf x0
#ifdef CONFIG_ARM64_PTR_AUTH
alternative_if ARM64_HAS_ADDRESS_AUTH
/*
* IA was enabled for in-kernel PAC. Disable it now if needed, or
* alternatively install the user's IA. All other per-task keys and
* SCTLR bits were updated on task switch.
*
* No kernel C function calls after this.
*/
tbz x0, SCTLR_ELx_ENIA_SHIFT, 1f
__ptrauth_keys_install_user tsk, x0, x1, x2
b 2f
1:
mrs x0, sctlr_el1
bic x0, x0, SCTLR_ELx_ENIA
msr sctlr_el1, x0
2:
alternative_else_nop_endif
#endif
mte_set_user_gcr tsk, x0, x1
@@ -784,8 +843,6 @@ SYM_CODE_START_LOCAL(ret_to_user)
cbnz x2, work_pending
finish_ret_to_user:
user_enter_irqoff
/* Ignore asynchronous tag check faults in the uaccess routines */
clear_mte_async_tcf
enable_step_tsk x19, x2
#ifdef CONFIG_GCC_PLUGIN_STACKLEAK
bl stackleak_erase

139
arch/arm64/kernel/mte.c
View File

@@ -4,6 +4,7 @@
*/
#include <linux/bitops.h>
#include <linux/cpu.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/prctl.h>
@@ -26,6 +27,8 @@ u64 gcr_kernel_excl __ro_after_init;
static bool report_fault_once = true;
static DEFINE_PER_CPU_READ_MOSTLY(u64, mte_tcf_preferred);
#ifdef CONFIG_KASAN_HW_TAGS
/* Whether the MTE asynchronous mode is enabled. */
DEFINE_STATIC_KEY_FALSE(mte_async_mode);
@@ -205,17 +208,29 @@ static void update_gcr_el1_excl(u64 excl)
sysreg_clear_set_s(SYS_GCR_EL1, SYS_GCR_EL1_EXCL_MASK, excl);
}
static void set_gcr_el1_excl(u64 excl)
static void mte_update_sctlr_user(struct task_struct *task)
{
current->thread.gcr_user_excl = excl;
/*
* SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value
* by mte_set_user_gcr() in kernel_exit,
* This must be called with preemption disabled and can only be called
* on the current or next task since the CPU must match where the thread
* is going to run. The caller is responsible for calling
* update_sctlr_el1() later in the same preemption disabled block.
*/
unsigned long sctlr = task->thread.sctlr_user;
unsigned long mte_ctrl = task->thread.mte_ctrl;
unsigned long pref, resolved_mte_tcf;
pref = __this_cpu_read(mte_tcf_preferred);
resolved_mte_tcf = (mte_ctrl & pref) ? pref : mte_ctrl;
sctlr &= ~SCTLR_EL1_TCF0_MASK;
if (resolved_mte_tcf & MTE_CTRL_TCF_ASYNC)
sctlr |= SCTLR_EL1_TCF0_ASYNC;
else if (resolved_mte_tcf & MTE_CTRL_TCF_SYNC)
sctlr |= SCTLR_EL1_TCF0_SYNC;
task->thread.sctlr_user = sctlr;
}
void flush_mte_state(void)
void mte_thread_init_user(void)
{
if (!system_supports_mte())
return;
@@ -224,15 +239,14 @@ void flush_mte_state(void)
dsb(ish);
write_sysreg_s(0, SYS_TFSRE0_EL1);
clear_thread_flag(TIF_MTE_ASYNC_FAULT);
/* disable tag checking */
set_task_sctlr_el1((current->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK) |
SCTLR_EL1_TCF0_NONE);
/* reset tag generation mask */
set_gcr_el1_excl(SYS_GCR_EL1_EXCL_MASK);
/* disable tag checking and reset tag generation mask */
set_mte_ctrl(current, 0);
}
void mte_thread_switch(struct task_struct *next)
{
mte_update_sctlr_user(next);
/*
* Check if an async tag exception occurred at EL1.
*
@@ -270,33 +284,23 @@ void mte_suspend_exit(void)
long set_mte_ctrl(struct task_struct *task, unsigned long arg)
{
u64 sctlr = task->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK;
u64 gcr_excl = ~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) &
SYS_GCR_EL1_EXCL_MASK;
u64 mte_ctrl = (~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) &
SYS_GCR_EL1_EXCL_MASK) << MTE_CTRL_GCR_USER_EXCL_SHIFT;
if (!system_supports_mte())
return 0;
switch (arg & PR_MTE_TCF_MASK) {
case PR_MTE_TCF_NONE:
sctlr |= SCTLR_EL1_TCF0_NONE;
break;
case PR_MTE_TCF_SYNC:
sctlr |= SCTLR_EL1_TCF0_SYNC;
break;
case PR_MTE_TCF_ASYNC:
sctlr |= SCTLR_EL1_TCF0_ASYNC;
break;
default:
return -EINVAL;
}
if (arg & PR_MTE_TCF_ASYNC)
mte_ctrl |= MTE_CTRL_TCF_ASYNC;
if (arg & PR_MTE_TCF_SYNC)
mte_ctrl |= MTE_CTRL_TCF_SYNC;
if (task != current) {
task->thread.sctlr_user = sctlr;
task->thread.gcr_user_excl = gcr_excl;
} else {
set_task_sctlr_el1(sctlr);
set_gcr_el1_excl(gcr_excl);
task->thread.mte_ctrl = mte_ctrl;
if (task == current) {
preempt_disable();
mte_update_sctlr_user(task);
update_sctlr_el1(task->thread.sctlr_user);
preempt_enable();
}
return 0;
@@ -305,24 +309,18 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg)
long get_mte_ctrl(struct task_struct *task)
{
unsigned long ret;
u64 incl = ~task->thread.gcr_user_excl & SYS_GCR_EL1_EXCL_MASK;
u64 mte_ctrl = task->thread.mte_ctrl;
u64 incl = (~mte_ctrl >> MTE_CTRL_GCR_USER_EXCL_SHIFT) &
SYS_GCR_EL1_EXCL_MASK;
if (!system_supports_mte())
return 0;
ret = incl << PR_MTE_TAG_SHIFT;
switch (task->thread.sctlr_user & SCTLR_EL1_TCF0_MASK) {
case SCTLR_EL1_TCF0_NONE:
ret |= PR_MTE_TCF_NONE;
break;
case SCTLR_EL1_TCF0_SYNC:
ret |= PR_MTE_TCF_SYNC;
break;
case SCTLR_EL1_TCF0_ASYNC:
if (mte_ctrl & MTE_CTRL_TCF_ASYNC)
ret |= PR_MTE_TCF_ASYNC;
break;
}
if (mte_ctrl & MTE_CTRL_TCF_SYNC)
ret |= PR_MTE_TCF_SYNC;
return ret;
}
@@ -461,3 +459,54 @@ int mte_ptrace_copy_tags(struct task_struct *child, long request,
return ret;
}
static ssize_t mte_tcf_preferred_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
switch (per_cpu(mte_tcf_preferred, dev->id)) {
case MTE_CTRL_TCF_ASYNC:
return sysfs_emit(buf, "async\n");
case MTE_CTRL_TCF_SYNC:
return sysfs_emit(buf, "sync\n");
default:
return sysfs_emit(buf, "???\n");
}
}
static ssize_t mte_tcf_preferred_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t count)
{
u64 tcf;
if (sysfs_streq(buf, "async"))
tcf = MTE_CTRL_TCF_ASYNC;
else if (sysfs_streq(buf, "sync"))
tcf = MTE_CTRL_TCF_SYNC;
else
return -EINVAL;
device_lock(dev);
per_cpu(mte_tcf_preferred, dev->id) = tcf;
device_unlock(dev);
return count;
}
static DEVICE_ATTR_RW(mte_tcf_preferred);
static int register_mte_tcf_preferred_sysctl(void)
{
unsigned int cpu;
if (!system_supports_mte())
return 0;
for_each_possible_cpu(cpu) {
per_cpu(mte_tcf_preferred, cpu) = MTE_CTRL_TCF_ASYNC;
device_create_file(get_cpu_device(cpu),
&dev_attr_mte_tcf_preferred);
}
return 0;
}
subsys_initcall(register_mte_tcf_preferred_sysctl);

65
arch/arm64/kernel/pointer_auth.c
View File

@@ -43,6 +43,71 @@ int ptrauth_prctl_reset_keys(struct task_struct *tsk, unsigned long arg)
get_random_bytes(&keys->apdb, sizeof(keys->apdb));
if (arg & PR_PAC_APGAKEY)
get_random_bytes(&keys->apga, sizeof(keys->apga));
ptrauth_keys_install_user(keys);
return 0;
}
static u64 arg_to_enxx_mask(unsigned long arg)
{
u64 sctlr_enxx_mask = 0;
WARN_ON(arg & ~PR_PAC_ENABLED_KEYS_MASK);
if (arg & PR_PAC_APIAKEY)
sctlr_enxx_mask |= SCTLR_ELx_ENIA;
if (arg & PR_PAC_APIBKEY)
sctlr_enxx_mask |= SCTLR_ELx_ENIB;
if (arg & PR_PAC_APDAKEY)
sctlr_enxx_mask |= SCTLR_ELx_ENDA;
if (arg & PR_PAC_APDBKEY)
sctlr_enxx_mask |= SCTLR_ELx_ENDB;
return sctlr_enxx_mask;
}
int ptrauth_set_enabled_keys(struct task_struct *tsk, unsigned long keys,
unsigned long enabled)
{
u64 sctlr;
if (!system_supports_address_auth())
return -EINVAL;
if (is_compat_thread(task_thread_info(tsk)))
return -EINVAL;
if ((keys & ~PR_PAC_ENABLED_KEYS_MASK) || (enabled & ~keys))
return -EINVAL;
preempt_disable();
sctlr = tsk->thread.sctlr_user;
sctlr &= ~arg_to_enxx_mask(keys);
sctlr |= arg_to_enxx_mask(enabled);
tsk->thread.sctlr_user = sctlr;
if (tsk == current)
update_sctlr_el1(sctlr);
preempt_enable();
return 0;
}
int ptrauth_get_enabled_keys(struct task_struct *tsk)
{
int retval = 0;
if (!system_supports_address_auth())
return -EINVAL;
if (is_compat_thread(task_thread_info(tsk)))
return -EINVAL;
if (tsk->thread.sctlr_user & SCTLR_ELx_ENIA)
retval |= PR_PAC_APIAKEY;
if (tsk->thread.sctlr_user & SCTLR_ELx_ENIB)
retval |= PR_PAC_APIBKEY;
if (tsk->thread.sctlr_user & SCTLR_ELx_ENDA)
retval |= PR_PAC_APDAKEY;
if (tsk->thread.sctlr_user & SCTLR_ELx_ENDB)
retval |= PR_PAC_APDBKEY;
return retval;
}

32
arch/arm64/kernel/process.c
View File

@@ -337,7 +337,6 @@ void flush_thread(void)
tls_thread_flush();
flush_ptrace_hw_breakpoint(current);
flush_tagged_addr_state();
flush_mte_state();
}
void release_thread(struct task_struct *dead_task)
@@ -538,27 +537,24 @@ static void erratum_1418040_thread_switch(struct task_struct *prev,
write_sysreg(val, cntkctl_el1);
}
static void update_sctlr_el1(u64 sctlr)
/*
* __switch_to() checks current->thread.sctlr_user as an optimisation. Therefore
* this function must be called with preemption disabled and the update to
* sctlr_user must be made in the same preemption disabled block so that
* __switch_to() does not see the variable update before the SCTLR_EL1 one.
*/
void update_sctlr_el1(u64 sctlr)
{
sysreg_clear_set(sctlr_el1, SCTLR_USER_MASK, sctlr);
/*
* EnIA must not be cleared while in the kernel as this is necessary for
* in-kernel PAC. It will be cleared on kernel exit if needed.
*/
sysreg_clear_set(sctlr_el1, SCTLR_USER_MASK & ~SCTLR_ELx_ENIA, sctlr);
/* ISB required for the kernel uaccess routines when setting TCF0. */
isb();
}
void set_task_sctlr_el1(u64 sctlr)
{
/*
* __switch_to() checks current->thread.sctlr as an
* optimisation. Disable preemption so that it does not see
* the variable update before the SCTLR_EL1 one.
*/
preempt_disable();
current->thread.sctlr_user = sctlr;
update_sctlr_el1(sctlr);
preempt_enable();
}
/*
* Thread switching.
*/
@@ -575,6 +571,7 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev,
uao_thread_switch(next);
ssbs_thread_switch(next);
erratum_1418040_thread_switch(prev, next);
ptrauth_thread_switch_user(next);
/*
* vendor hook is needed before the dsb(),
* because MPAM is related to cache maintenance.
@@ -666,7 +663,8 @@ void arch_setup_new_exec(void)
}
current->mm->context.flags = mmflags;
ptrauth_thread_init_user(current);
ptrauth_thread_init_user();
mte_thread_init_user();
if (task_spec_ssb_noexec(current)) {
arch_prctl_spec_ctrl_set(current, PR_SPEC_STORE_BYPASS,

41
arch/arm64/kernel/ptrace.c
View File

@@ -908,6 +908,38 @@ static int pac_mask_get(struct task_struct *target,
return membuf_write(&to, &uregs, sizeof(uregs));
}
static int pac_enabled_keys_get(struct task_struct *target,
const struct user_regset *regset,
struct membuf to)
{
long enabled_keys = ptrauth_get_enabled_keys(target);
if (IS_ERR_VALUE(enabled_keys))
return enabled_keys;
return membuf_write(&to, &enabled_keys, sizeof(enabled_keys));
}
static int pac_enabled_keys_set(struct task_struct *target,
const struct user_regset *regset,
unsigned int pos, unsigned int count,
const void *kbuf, const void __user *ubuf)
{
int ret;
long enabled_keys = ptrauth_get_enabled_keys(target);
if (IS_ERR_VALUE(enabled_keys))
return enabled_keys;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &enabled_keys, 0,
sizeof(long));
if (ret)
return ret;
return ptrauth_set_enabled_keys(target, PR_PAC_ENABLED_KEYS_MASK,
enabled_keys);
}
#ifdef CONFIG_CHECKPOINT_RESTORE
static __uint128_t pac_key_to_user(const struct ptrauth_key *key)
{
@@ -1073,6 +1105,7 @@ enum aarch64_regset {
#endif
#ifdef CONFIG_ARM64_PTR_AUTH
REGSET_PAC_MASK,
REGSET_PAC_ENABLED_KEYS,
#ifdef CONFIG_CHECKPOINT_RESTORE
REGSET_PACA_KEYS,
REGSET_PACG_KEYS,
@@ -1159,6 +1192,14 @@ static const struct user_regset aarch64_regsets[] = {
.regset_get = pac_mask_get,
/* this cannot be set dynamically */
},
[REGSET_PAC_ENABLED_KEYS] = {
.core_note_type = NT_ARM_PAC_ENABLED_KEYS,
.n = 1,
.size = sizeof(long),
.align = sizeof(long),
.regset_get = pac_enabled_keys_get,
.set = pac_enabled_keys_set,
},
#ifdef CONFIG_CHECKPOINT_RESTORE
[REGSET_PACA_KEYS] = {
.core_note_type = NT_ARM_PACA_KEYS,

4
arch/arm64/kernel/smp.c
View File

@@ -1008,6 +1008,10 @@ void __init set_smp_ipi_range(int ipi_base, int n)
ipi_desc[i] = irq_to_desc(ipi_base + i);
irq_set_status_flags(ipi_base + i, IRQ_HIDDEN);
/* The recheduling IPI is special... */
if (i == IPI_RESCHEDULE)
__irq_modify_status(ipi_base + i, 0, IRQ_RAW, ~0);
}
ipi_irq_base = ipi_base;

3
arch/arm64/kernel/suspend.c
View File

@@ -75,8 +75,9 @@ void notrace __cpu_suspend_exit(void)
*/
spectre_v4_enable_mitigation(NULL);
/* Restore additional MTE-specific configuration */
/* Restore additional feature-specific configuration */
mte_suspend_exit();
ptrauth_suspend_exit();
}
/*

8
arch/arm64/kvm/debug.c
View File

@@ -141,7 +141,13 @@ void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu)
* @vcpu: the vcpu pointer
*
* This is called before each entry into the hypervisor to setup any
* debug related registers.
* debug related registers. Currently this just ensures we will trap
* access to:
* - Performance monitors (MDCR_EL2_TPM/MDCR_EL2_TPMCR)
* - Debug ROM Address (MDCR_EL2_TDRA)
* - OS related registers (MDCR_EL2_TDOSA)
* - Statistical profiler (MDCR_EL2_TPMS/MDCR_EL2_E2PB)
* - Self-hosted Trace (MDCR_EL2_TTRF/MDCR_EL2_E2TB)
*
* Additionally, KVM only traps guest accesses to the debug registers if
* the guest is not actively using them (see the KVM_ARM64_DEBUG_DIRTY

11
arch/arm64/kvm/hyp/nvhe/switch.c
View File

@@ -192,12 +192,11 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_save_state_nvhe(host_ctxt);
/*
* For nVHE, we must save and disable any SPE
* buffers, as the translation regime is going
* to be loaded with that of the guest. And we must
* save host context for SPE, before we change the
* ownership to EL2 (via MDCR_EL2_E2PB == 0) and before
* we load guest Stage1.
* We must flush and disable the SPE buffer for nVHE, as
* the translation regime(EL1&0) is going to be loaded with
* that of the guest. And we must do this before we change the
* translation regime to EL2 (via MDCR_EL2_E2PB == 0) and
* before we load guest Stage1.
*/
__debug_save_host_buffers_nvhe(vcpu);

20
arch/arm64/lib/mte.S
View File

@@ -36,6 +36,26 @@ SYM_FUNC_START(mte_clear_page_tags)
ret
SYM_FUNC_END(mte_clear_page_tags)
/*
* Zero the page and tags at the same time
*
* Parameters:
* x0 - address to the beginning of the page
*/
SYM_FUNC_START(mte_zero_clear_page_tags)
mrs x1, dczid_el0
and w1, w1, #0xf
mov x2, #4
lsl x1, x2, x1
and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag
1: dc gzva, x0
add x0, x0, x1
tst x0, #(PAGE_SIZE - 1)
b.ne 1b
ret
SYM_FUNC_END(mte_zero_clear_page_tags)
/*
* Copy the tags from the source page to the destination one
* x0 - address of the destination page

26
arch/arm64/mm/fault.c
View File

@@ -968,3 +968,29 @@ void do_debug_exception(unsigned long addr_if_watchpoint, unsigned int esr,
debug_exception_exit(regs);
}
NOKPROBE_SYMBOL(do_debug_exception);
/*
* Used during anonymous page fault handling.
*/
struct page *alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma,
unsigned long vaddr)
{
gfp_t flags = GFP_HIGHUSER_MOVABLE | __GFP_ZERO | __GFP_CMA;
/*
* If the page is mapped with PROT_MTE, initialise the tags at the
* point of allocation and page zeroing as this is usually faster than
* separate DC ZVA and STGM.
*/
if (vma->vm_flags & VM_MTE)
flags |= __GFP_ZEROTAGS;
return alloc_page_vma(flags, vma, vaddr);
}
void tag_clear_highpage(struct page *page)
{
mte_zero_clear_page_tags(page_address(page));
page_kasan_tag_reset(page);
set_bit(PG_mte_tagged, &page->flags);
}

10
arch/arm64/mm/proc.S
View File

@@ -46,9 +46,13 @@
#endif
#ifdef CONFIG_KASAN_HW_TAGS
#define TCR_KASAN_HW_FLAGS SYS_TCR_EL1_TCMA1 | TCR_TBI1 | TCR_TBID1
#define TCR_MTE_FLAGS SYS_TCR_EL1_TCMA1 | TCR_TBI1 | TCR_TBID1
#else
#define TCR_KASAN_HW_FLAGS 0
/*
* The mte_zero_clear_page_tags() implementation uses DC GZVA, which relies on
* TBI being enabled at EL1.
*/
#define TCR_MTE_FLAGS TCR_TBI1 | TCR_TBID1
#endif
/*
@@ -462,7 +466,7 @@ SYM_FUNC_START(__cpu_setup)
msr_s SYS_TFSRE0_EL1, xzr
/* set the TCR_EL1 bits */
mov_q mte_tcr, TCR_KASAN_HW_FLAGS
mov_q mte_tcr, TCR_MTE_FLAGS
1:
#endif
msr mair_el1, x5

12
arch/x86/configs/gki_defconfig
View File

@@ -17,6 +17,7 @@ CONFIG_IKHEADERS=y
CONFIG_UCLAMP_TASK=y
CONFIG_UCLAMP_BUCKETS_COUNT=20
CONFIG_CGROUPS=y
CONFIG_MEMCG=y
CONFIG_BLK_CGROUP=y
CONFIG_CGROUP_SCHED=y
CONFIG_UCLAMP_TASK_GROUP=y
@@ -53,7 +54,7 @@ CONFIG_PARAVIRT=y
CONFIG_NR_CPUS=32
CONFIG_EFI=y
CONFIG_CMDLINE_BOOL=y
CONFIG_CMDLINE="stack_depot_disable=on"
CONFIG_CMDLINE="stack_depot_disable=on cgroup_disable=pressure cgroup.memory=nokmem"
CONFIG_PM_WAKELOCKS=y
CONFIG_PM_WAKELOCKS_LIMIT=0
# CONFIG_PM_WAKELOCKS_GC is not set
@@ -73,6 +74,7 @@ CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODVERSIONS=y
CONFIG_MODULE_SCMVERSION=y
CONFIG_BLK_CGROUP_IOCOST=y
CONFIG_BLK_INLINE_ENCRYPTION=y
CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK=y
CONFIG_IOSCHED_BFQ=y
@@ -232,6 +234,7 @@ CONFIG_NET_ACT_GACT=y
CONFIG_NET_ACT_MIRRED=y
CONFIG_NET_ACT_SKBEDIT=y
CONFIG_VSOCKETS=y
CONFIG_CGROUP_NET_PRIO=y
CONFIG_BPF_JIT=y
CONFIG_CAN=y
CONFIG_BT=y
@@ -243,16 +246,12 @@ CONFIG_BT_HCIUART=y
CONFIG_BT_HCIUART_LL=y
CONFIG_BT_HCIUART_BCM=y
CONFIG_BT_HCIUART_QCA=y
CONFIG_CFG80211=y
CONFIG_NL80211_TESTMODE=y
# CONFIG_CFG80211_DEFAULT_PS is not set
# CONFIG_CFG80211_CRDA_SUPPORT is not set
CONFIG_MAC80211=y
CONFIG_RFKILL=y
CONFIG_PCI=y
CONFIG_PCIEPORTBUS=y
CONFIG_PCIEAER=y
CONFIG_PCI_MSI=y
CONFIG_PCI_IOV=y
CONFIG_PCIE_DW_PLAT_EP=y
CONFIG_PCI_ENDPOINT=y
CONFIG_FW_LOADER_USER_HELPER=y
@@ -391,7 +390,6 @@ CONFIG_SND=y
CONFIG_SND_HRTIMER=y
CONFIG_SND_DYNAMIC_MINORS=y
# CONFIG_SND_SUPPORT_OLD_API is not set
# CONFIG_SND_VERBOSE_PROCFS is not set
# CONFIG_SND_DRIVERS is not set
CONFIG_SND_USB_AUDIO=y
CONFIG_SND_SOC=y

19
block/Kconfig
View File

@@ -133,6 +133,13 @@ config BLK_WBT
dynamically on an algorithm loosely based on CoDel, factoring in
the realtime performance of the disk.
config BLK_WBT_MQ
bool "Enable writeback throttling by default"
default y
depends on BLK_WBT
help
Enable writeback throttling by default for request-based block devices.
config BLK_CGROUP_IOLATENCY
bool "Enable support for latency based cgroup IO protection"
depends on BLK_CGROUP=y
@@ -155,12 +162,14 @@ config BLK_CGROUP_IOCOST
distributes IO capacity between different groups based on
their share of the overall weight distribution.
config BLK_WBT_MQ
bool "Multiqueue writeback throttling"
default y
depends on BLK_WBT
config BLK_CGROUP_IOPRIO
bool "Cgroup I/O controller for assigning an I/O priority class"
depends on BLK_CGROUP
help
Enable writeback throttling by default on multiqueue devices.
Enable the .prio interface for assigning an I/O priority class to
requests. The I/O priority class affects the order in which an I/O
scheduler and block devices process requests. Only some I/O schedulers
and some block devices support I/O priorities.
config BLK_DEBUG_FS
bool "Block layer debugging information in debugfs"

6
block/Kconfig.iosched
View File

@@ -9,6 +9,12 @@ config MQ_IOSCHED_DEADLINE
help
MQ version of the deadline IO scheduler.
config MQ_IOSCHED_DEADLINE_CGROUP
tristate
default y
depends on MQ_IOSCHED_DEADLINE
depends on BLK_CGROUP
config MQ_IOSCHED_KYBER
tristate "Kyber I/O scheduler"
default y

3
block/Makefile
View File

@@ -17,9 +17,12 @@ obj-$(CONFIG_BLK_DEV_BSGLIB) += bsg-lib.o
obj-$(CONFIG_BLK_CGROUP) += blk-cgroup.o
obj-$(CONFIG_BLK_CGROUP_RWSTAT) += blk-cgroup-rwstat.o
obj-$(CONFIG_BLK_DEV_THROTTLING) += blk-throttle.o
obj-$(CONFIG_BLK_CGROUP_IOPRIO) += blk-ioprio.o
obj-$(CONFIG_BLK_CGROUP_IOLATENCY) += blk-iolatency.o
obj-$(CONFIG_BLK_CGROUP_IOCOST) += blk-iocost.o
obj-$(CONFIG_MQ_IOSCHED_DEADLINE) += mq-deadline.o
mq-deadline-y += mq-deadline-main.o
mq-deadline-$(CONFIG_MQ_IOSCHED_DEADLINE_CGROUP)+= mq-deadline-cgroup.o
obj-$(CONFIG_MQ_IOSCHED_KYBER) += kyber-iosched.o
bfq-y := bfq-iosched.o bfq-wf2q.o bfq-cgroup.o
obj-$(CONFIG_IOSCHED_BFQ) += bfq.o

7
block/bfq-iosched.c
View File

@@ -4499,7 +4499,7 @@ check_queue:
bfqq = bfqq->bic->bfqq[0];
else if (bfq_bfqq_has_waker(bfqq) &&
bfq_bfqq_busy(bfqq->waker_bfqq) &&
bfqq->next_rq &&
bfqq->waker_bfqq->next_rq &&
bfq_serv_to_charge(bfqq->waker_bfqq->next_rq,
bfqq->waker_bfqq) <=
bfq_bfqq_budget_left(bfqq->waker_bfqq)
@@ -4640,9 +4640,6 @@ static bool bfq_has_work(struct blk_mq_hw_ctx *hctx)
{
struct bfq_data *bfqd = hctx->queue->elevator->elevator_data;
if (!atomic_read(&hctx->elevator_queued))
return false;
/*
* Avoiding lock: a race on bfqd->busy_queues should cause at
* most a call to dispatch for nothing
@@ -5557,7 +5554,6 @@ static void bfq_insert_requests(struct blk_mq_hw_ctx *hctx,
rq = list_first_entry(list, struct request, queuelist);
list_del_init(&rq->queuelist);
bfq_insert_request(hctx, rq, at_head);
atomic_inc(&hctx->elevator_queued);
}
}
@@ -5925,7 +5921,6 @@ static void bfq_finish_requeue_request(struct request *rq)
bfq_completed_request(bfqq, bfqd);
bfq_finish_requeue_request_body(bfqq);
atomic_dec(&rq->mq_hctx->elevator_queued);
spin_unlock_irqrestore(&bfqd->lock, flags);
} else {

14
block/blk-cgroup.c
View File

@@ -31,6 +31,7 @@
#include <linux/tracehook.h>
#include <linux/psi.h>
#include "blk.h"
#include "blk-ioprio.h"
#define MAX_KEY_LEN 100
@@ -1181,15 +1182,18 @@ int blkcg_init_queue(struct request_queue *q)
if (preloaded)
radix_tree_preload_end();
ret = blk_iolatency_init(q);
if (ret)
goto err_destroy_all;
ret = blk_ioprio_init(q);
if (ret)
goto err_destroy_all;
ret = blk_throtl_init(q);
if (ret)
goto err_destroy_all;
ret = blk_iolatency_init(q);
if (ret) {
blk_throtl_exit(q);
goto err_destroy_all;
}
return 0;
err_destroy_all:

7
block/blk-core.c
View File

@@ -18,7 +18,6 @@
#include <linux/bio.h>
#include <linux/blkdev.h>
#include <linux/blk-mq.h>
#include <linux/blk-pm.h>
#include <linux/highmem.h>
#include <linux/mm.h>
#include <linux/pagemap.h>
@@ -441,8 +440,7 @@ int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags)
* responsible for ensuring that that counter is
* globally visible before the queue is unfrozen.
*/
if ((pm && queue_rpm_status(q) != RPM_SUSPENDED) ||
!blk_queue_pm_only(q)) {
if (pm || !blk_queue_pm_only(q)) {
success = true;
} else {
percpu_ref_put(&q->q_usage_counter);
@@ -467,7 +465,8 @@ int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags)
wait_event(q->mq_freeze_wq,
(!q->mq_freeze_depth &&
blk_pm_resume_queue(pm, q)) ||
(pm || (blk_pm_request_resume(q),
!blk_queue_pm_only(q)))) ||
blk_queue_dying(q));
if (blk_queue_dying(q))
return -ENODEV;

262
block/blk-ioprio.c Normal file
View File

@@ -0,0 +1,262 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Block rq-qos policy for assigning an I/O priority class to requests.
*
* Using an rq-qos policy for assigning I/O priority class has two advantages
* over using the ioprio_set() system call:
*
* - This policy is cgroup based so it has all the advantages of cgroups.
* - While ioprio_set() does not affect page cache writeback I/O, this rq-qos
* controller affects page cache writeback I/O for filesystems that support
* assiociating a cgroup with writeback I/O. See also
* Documentation/admin-guide/cgroup-v2.rst.
*/
#include <linux/blk-cgroup.h>
#include <linux/blk-mq.h>
#include <linux/blk_types.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include "blk-ioprio.h"
#include "blk-rq-qos.h"
/**
* enum prio_policy - I/O priority class policy.
* @POLICY_NO_CHANGE: (default) do not modify the I/O priority class.
* @POLICY_NONE_TO_RT: modify IOPRIO_CLASS_NONE into IOPRIO_CLASS_RT.
* @POLICY_RESTRICT_TO_BE: modify IOPRIO_CLASS_NONE and IOPRIO_CLASS_RT into
* IOPRIO_CLASS_BE.
* @POLICY_ALL_TO_IDLE: change the I/O priority class into IOPRIO_CLASS_IDLE.
*
* See also <linux/ioprio.h>.
*/
enum prio_policy {
POLICY_NO_CHANGE = 0,
POLICY_NONE_TO_RT = 1,
POLICY_RESTRICT_TO_BE = 2,
POLICY_ALL_TO_IDLE = 3,
};
static const char *policy_name[] = {
[POLICY_NO_CHANGE] = "no-change",
[POLICY_NONE_TO_RT] = "none-to-rt",
[POLICY_RESTRICT_TO_BE] = "restrict-to-be",
[POLICY_ALL_TO_IDLE] = "idle",
};
static struct blkcg_policy ioprio_policy;
/**
* struct ioprio_blkg - Per (cgroup, request queue) data.
* @pd: blkg_policy_data structure.
*/
struct ioprio_blkg {
struct blkg_policy_data pd;
};
/**
* struct ioprio_blkcg - Per cgroup data.
* @cpd: blkcg_policy_data structure.
* @prio_policy: One of the IOPRIO_CLASS_* values. See also <linux/ioprio.h>.
*/
struct ioprio_blkcg {
struct blkcg_policy_data cpd;
enum prio_policy prio_policy;
};
static inline struct ioprio_blkg *pd_to_ioprio(struct blkg_policy_data *pd)
{
return pd ? container_of(pd, struct ioprio_blkg, pd) : NULL;
}
static struct ioprio_blkcg *blkcg_to_ioprio_blkcg(struct blkcg *blkcg)
{
return container_of(blkcg_to_cpd(blkcg, &ioprio_policy),
struct ioprio_blkcg, cpd);
}
static struct ioprio_blkcg *
ioprio_blkcg_from_css(struct cgroup_subsys_state *css)
{
return blkcg_to_ioprio_blkcg(css_to_blkcg(css));
}
static struct ioprio_blkcg *ioprio_blkcg_from_bio(struct bio *bio)
{
struct blkg_policy_data *pd = blkg_to_pd(bio->bi_blkg, &ioprio_policy);
if (!pd)
return NULL;
return blkcg_to_ioprio_blkcg(pd->blkg->blkcg);
}
static int ioprio_show_prio_policy(struct seq_file *sf, void *v)
{
struct ioprio_blkcg *blkcg = ioprio_blkcg_from_css(seq_css(sf));
seq_printf(sf, "%s\n", policy_name[blkcg->prio_policy]);
return 0;
}
static ssize_t ioprio_set_prio_policy(struct kernfs_open_file *of, char *buf,
size_t nbytes, loff_t off)
{
struct ioprio_blkcg *blkcg = ioprio_blkcg_from_css(of_css(of));
int ret;
if (off != 0)
return -EIO;
/* kernfs_fop_write_iter() terminates 'buf' with '\0'. */
ret = sysfs_match_string(policy_name, buf);
if (ret < 0)
return ret;
blkcg->prio_policy = ret;
return nbytes;
}
static struct blkg_policy_data *
ioprio_alloc_pd(gfp_t gfp, struct request_queue *q, struct blkcg *blkcg)
{
struct ioprio_blkg *ioprio_blkg;
ioprio_blkg = kzalloc(sizeof(*ioprio_blkg), gfp);
if (!ioprio_blkg)
return NULL;
return &ioprio_blkg->pd;
}
static void ioprio_free_pd(struct blkg_policy_data *pd)
{
struct ioprio_blkg *ioprio_blkg = pd_to_ioprio(pd);
kfree(ioprio_blkg);
}
static struct blkcg_policy_data *ioprio_alloc_cpd(gfp_t gfp)
{
struct ioprio_blkcg *blkcg;
blkcg = kzalloc(sizeof(*blkcg), gfp);
if (!blkcg)
return NULL;
blkcg->prio_policy = POLICY_NO_CHANGE;
return &blkcg->cpd;
}
static void ioprio_free_cpd(struct blkcg_policy_data *cpd)
{
struct ioprio_blkcg *blkcg = container_of(cpd, typeof(*blkcg), cpd);
kfree(blkcg);
}
#define IOPRIO_ATTRS \
{ \
.name = "prio.class", \
.seq_show = ioprio_show_prio_policy, \
.write = ioprio_set_prio_policy, \
}, \
{ } /* sentinel */
/* cgroup v2 attributes */
static struct cftype ioprio_files[] = {
IOPRIO_ATTRS
};
/* cgroup v1 attributes */
static struct cftype ioprio_legacy_files[] = {
IOPRIO_ATTRS
};
static struct blkcg_policy ioprio_policy = {
.dfl_cftypes = ioprio_files,
.legacy_cftypes = ioprio_legacy_files,
.cpd_alloc_fn = ioprio_alloc_cpd,
.cpd_free_fn = ioprio_free_cpd,
.pd_alloc_fn = ioprio_alloc_pd,
.pd_free_fn = ioprio_free_pd,
};
struct blk_ioprio {
struct rq_qos rqos;
};
static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq,
struct bio *bio)
{
struct ioprio_blkcg *blkcg = ioprio_blkcg_from_bio(bio);
/*
* Except for IOPRIO_CLASS_NONE, higher I/O priority numbers
* correspond to a lower priority. Hence, the max_t() below selects
* the lower priority of bi_ioprio and the cgroup I/O priority class.
* If the cgroup policy has been set to POLICY_NO_CHANGE == 0, the
* bio I/O priority is not modified. If the bio I/O priority equals
* IOPRIO_CLASS_NONE, the cgroup I/O priority is assigned to the bio.
*/
bio->bi_ioprio = max_t(u16, bio->bi_ioprio,
IOPRIO_PRIO_VALUE(blkcg->prio_policy, 0));
}
static void blkcg_ioprio_exit(struct rq_qos *rqos)
{
struct blk_ioprio *blkioprio_blkg =
container_of(rqos, typeof(*blkioprio_blkg), rqos);
blkcg_deactivate_policy(rqos->q, &ioprio_policy);
kfree(blkioprio_blkg);
}
static struct rq_qos_ops blkcg_ioprio_ops = {
.track = blkcg_ioprio_track,
.exit = blkcg_ioprio_exit,
};
int blk_ioprio_init(struct request_queue *q)
{
struct blk_ioprio *blkioprio_blkg;
struct rq_qos *rqos;
int ret;
blkioprio_blkg = kzalloc(sizeof(*blkioprio_blkg), GFP_KERNEL);
if (!blkioprio_blkg)
return -ENOMEM;
ret = blkcg_activate_policy(q, &ioprio_policy);
if (ret) {
kfree(blkioprio_blkg);
return ret;
}
rqos = &blkioprio_blkg->rqos;
rqos->id = RQ_QOS_IOPRIO;
rqos->ops = &blkcg_ioprio_ops;
rqos->q = q;
/*
* Registering the rq-qos policy after activating the blk-cgroup
* policy guarantees that ioprio_blkcg_from_bio(bio) != NULL in the
* rq-qos callbacks.
*/
rq_qos_add(q, rqos);
return 0;
}
static int __init ioprio_init(void)
{
return blkcg_policy_register(&ioprio_policy);
}
static void __exit ioprio_exit(void)
{
blkcg_policy_unregister(&ioprio_policy);
}
module_init(ioprio_init);
module_exit(ioprio_exit);

19
block/blk-ioprio.h Normal file
View File

@@ -0,0 +1,19 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _BLK_IOPRIO_H_
#define _BLK_IOPRIO_H_
#include <linux/kconfig.h>
struct request_queue;
#ifdef CONFIG_BLK_CGROUP_IOPRIO
int blk_ioprio_init(struct request_queue *q);
#else
static inline int blk_ioprio_init(struct request_queue *q)
{
return 0;
}
#endif
#endif /* _BLK_IOPRIO_H_ */

15
block/blk-mq-debugfs.c
View File

@@ -939,6 +939,21 @@ void blk_mq_debugfs_unregister_sched(struct request_queue *q)
q->sched_debugfs_dir = NULL;
}
static const char *rq_qos_id_to_name(enum rq_qos_id id)
{
switch (id) {
case RQ_QOS_WBT:
return "wbt";
case RQ_QOS_LATENCY:
return "latency";
case RQ_QOS_COST:
return "cost";
case RQ_QOS_IOPRIO:
return "ioprio";
}
return "unknown";
}
void blk_mq_debugfs_unregister_rqos(struct rq_qos *rqos)
{
debugfs_remove_recursive(rqos->debugfs_dir);

2
block/blk-mq-tag.h
View File

@@ -26,6 +26,8 @@ struct blk_mq_tags {
* request pool
*/
spinlock_t lock;
ANDROID_OEM_DATA(1);
};
extern struct blk_mq_tags *blk_mq_init_tags(unsigned int nr_tags,

71
block/blk-mq.c
View File

@@ -41,6 +41,8 @@
#include "blk-mq-sched.h"
#include "blk-rq-qos.h"
#include <trace/hooks/block.h>
static DEFINE_PER_CPU(struct list_head, blk_cpu_done);
static void blk_mq_poll_stats_start(struct request_queue *q);
@@ -341,6 +343,7 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data,
}
data->hctx->queued++;
trace_android_vh_blk_rq_ctx_init(rq, tags, data, alloc_time_ns);
return rq;
}
@@ -1665,6 +1668,42 @@ void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async)
}
EXPORT_SYMBOL(blk_mq_run_hw_queue);
/*
* Is the request queue handled by an IO scheduler that does not respect
* hardware queues when dispatching?
*/
static bool blk_mq_has_sqsched(struct request_queue *q)
{
struct elevator_queue *e = q->elevator;
if (e && e->type->ops.dispatch_request &&
!(e->type->elevator_features & ELEVATOR_F_MQ_AWARE))
return true;
return false;
}
/*
* Return prefered queue to dispatch from (if any) for non-mq aware IO
* scheduler.
*/
static struct blk_mq_hw_ctx *blk_mq_get_sq_hctx(struct request_queue *q)
{
struct blk_mq_hw_ctx *hctx;
/*
* If the IO scheduler does not respect hardware queues when
* dispatching, we just don't bother with multiple HW queues and
* dispatch from hctx for the current CPU since running multiple queues
* just causes lock contention inside the scheduler and pointless cache
* bouncing.
*/
hctx = blk_mq_map_queue_type(q, HCTX_TYPE_DEFAULT,
raw_smp_processor_id());
if (!blk_mq_hctx_stopped(hctx))
return hctx;
return NULL;
}
/**
* blk_mq_run_hw_queues - Run all hardware queues in a request queue.
* @q: Pointer to the request queue to run.
@@ -1672,14 +1711,23 @@ EXPORT_SYMBOL(blk_mq_run_hw_queue);
*/
void blk_mq_run_hw_queues(struct request_queue *q, bool async)
{
struct blk_mq_hw_ctx *hctx;
struct blk_mq_hw_ctx *hctx, *sq_hctx;
int i;
sq_hctx = NULL;
if (blk_mq_has_sqsched(q))
sq_hctx = blk_mq_get_sq_hctx(q);
queue_for_each_hw_ctx(q, hctx, i) {
if (blk_mq_hctx_stopped(hctx))
continue;
blk_mq_run_hw_queue(hctx, async);
/*
* Dispatch from this hctx either if there's no hctx preferred
* by IO scheduler or if it has requests that bypass the
* scheduler.
*/
if (!sq_hctx || sq_hctx == hctx ||
!list_empty_careful(&hctx->dispatch))
blk_mq_run_hw_queue(hctx, async);
}
}
EXPORT_SYMBOL(blk_mq_run_hw_queues);
@@ -1691,14 +1739,23 @@ EXPORT_SYMBOL(blk_mq_run_hw_queues);
*/
void blk_mq_delay_run_hw_queues(struct request_queue *q, unsigned long msecs)
{
struct blk_mq_hw_ctx *hctx;
struct blk_mq_hw_ctx *hctx, *sq_hctx;
int i;
sq_hctx = NULL;
if (blk_mq_has_sqsched(q))
sq_hctx = blk_mq_get_sq_hctx(q);
queue_for_each_hw_ctx(q, hctx, i) {
if (blk_mq_hctx_stopped(hctx))
continue;
blk_mq_delay_run_hw_queue(hctx, msecs);
/*
* Dispatch from this hctx either if there's no hctx preferred
* by IO scheduler or if it has requests that bypass the
* scheduler.
*/
if (!sq_hctx || sq_hctx == hctx ||
!list_empty_careful(&hctx->dispatch))
blk_mq_delay_run_hw_queue(hctx, msecs);
}
}
EXPORT_SYMBOL(blk_mq_delay_run_hw_queues);
@@ -2426,6 +2483,7 @@ int blk_mq_alloc_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags,
*/
rq_size = round_up(sizeof(struct request) + set->cmd_size,
cache_line_size());
trace_android_vh_blk_alloc_rqs(&rq_size, set, tags);
left = rq_size * depth;
for (i = 0; i < depth; ) {
@@ -2737,7 +2795,6 @@ blk_mq_alloc_hctx(struct request_queue *q, struct blk_mq_tag_set *set,
goto free_hctx;
atomic_set(&hctx->nr_active, 0);
atomic_set(&hctx->elevator_queued, 0);
if (node == NUMA_NO_NODE)
node = set->numa_node;
hctx->numa_node = node;

2
block/blk-mq.h
View File

@@ -35,6 +35,8 @@ struct blk_mq_ctx {
struct request_queue *queue;
struct blk_mq_ctxs *ctxs;
struct kobject kobj;
ANDROID_OEM_DATA_ARRAY(1, 2);
} ____cacheline_aligned_in_smp;
void blk_mq_exit_queue(struct request_queue *q);

14
block/blk-pm.h
View File

@@ -6,14 +6,11 @@
#include <linux/pm_runtime.h>
#ifdef CONFIG_PM
static inline int blk_pm_resume_queue(const bool pm, struct request_queue *q)
static inline void blk_pm_request_resume(struct request_queue *q)
{
if (!q->dev || !blk_queue_pm_only(q))
return 1; /* Nothing to do */
if (pm && q->rpm_status != RPM_SUSPENDED)
return 1; /* Request allowed */
pm_request_resume(q->dev);
return 0;
if (q->dev && (q->rpm_status == RPM_SUSPENDED ||
q->rpm_status == RPM_SUSPENDING))
pm_request_resume(q->dev);
}
static inline void blk_pm_mark_last_busy(struct request *rq)
@@ -47,9 +44,8 @@ static inline void blk_pm_put_request(struct request *rq)
--rq->q->nr_pending;
}
#else
static inline int blk_pm_resume_queue(const bool pm, struct request_queue *q)
static inline void blk_pm_request_resume(struct request_queue *q)
{
return 1;
}
static inline void blk_pm_mark_last_busy(struct request *rq)

14
block/blk-rq-qos.h
View File

@@ -17,6 +17,7 @@ enum rq_qos_id {
RQ_QOS_WBT,
RQ_QOS_LATENCY,
RQ_QOS_COST,
RQ_QOS_IOPRIO,
};
struct rq_wait {
@@ -79,19 +80,6 @@ static inline struct rq_qos *blkcg_rq_qos(struct request_queue *q)
return rq_qos_id(q, RQ_QOS_LATENCY);
}
static inline const char *rq_qos_id_to_name(enum rq_qos_id id)
{
switch (id) {
case RQ_QOS_WBT:
return "wbt";
case RQ_QOS_LATENCY:
return "latency";
case RQ_QOS_COST:
return "cost";
}
return "unknown";
}
static inline void rq_wait_init(struct rq_wait *rq_wait)
{
atomic_set(&rq_wait->inflight, 0);

1
block/kyber-iosched.c
View File

@@ -1030,6 +1030,7 @@ static struct elevator_type kyber_sched = {
#endif
.elevator_attrs = kyber_sched_attrs,
.elevator_name = "kyber",
.elevator_features = ELEVATOR_F_MQ_AWARE,
.elevator_owner = THIS_MODULE,
};

126
block/mq-deadline-cgroup.c Normal file
View File

@@ -0,0 +1,126 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/blk-cgroup.h>
#include <linux/ioprio.h>
#include "mq-deadline-cgroup.h"
static struct blkcg_policy dd_blkcg_policy;
static struct blkcg_policy_data *dd_cpd_alloc(gfp_t gfp)
{
struct dd_blkcg *pd;
pd = kzalloc(sizeof(*pd), gfp);
if (!pd)
return NULL;
pd->stats = alloc_percpu_gfp(typeof(*pd->stats),
GFP_KERNEL | __GFP_ZERO);
if (!pd->stats) {
kfree(pd);
return NULL;
}
return &pd->cpd;
}
static void dd_cpd_free(struct blkcg_policy_data *cpd)
{
struct dd_blkcg *dd_blkcg = container_of(cpd, typeof(*dd_blkcg), cpd);
free_percpu(dd_blkcg->stats);
kfree(dd_blkcg);
}
static struct dd_blkcg *dd_blkcg_from_pd(struct blkg_policy_data *pd)
{
return container_of(blkcg_to_cpd(pd->blkg->blkcg, &dd_blkcg_policy),
struct dd_blkcg, cpd);
}
/*
* Convert an association between a block cgroup and a request queue into a
* pointer to the mq-deadline information associated with a (blkcg, queue) pair.
*/
struct dd_blkcg *dd_blkcg_from_bio(struct bio *bio)
{
struct blkg_policy_data *pd;
pd = blkg_to_pd(bio->bi_blkg, &dd_blkcg_policy);
if (!pd)
return NULL;
return dd_blkcg_from_pd(pd);
}
static size_t dd_pd_stat(struct blkg_policy_data *pd, char *buf, size_t size)
{
static const char *const prio_class_name[] = {
[IOPRIO_CLASS_NONE] = "NONE",
[IOPRIO_CLASS_RT] = "RT",
[IOPRIO_CLASS_BE] = "BE",
[IOPRIO_CLASS_IDLE] = "IDLE",
};
struct dd_blkcg *blkcg = dd_blkcg_from_pd(pd);
int res = 0;
u8 prio;
for (prio = 0; prio < ARRAY_SIZE(blkcg->stats->stats); prio++)
res += scnprintf(buf + res, size - res,
" [%s] dispatched=%u inserted=%u merged=%u",
prio_class_name[prio],
ddcg_sum(blkcg, dispatched, prio) +
ddcg_sum(blkcg, merged, prio) -
ddcg_sum(blkcg, completed, prio),
ddcg_sum(blkcg, inserted, prio) -
ddcg_sum(blkcg, completed, prio),
ddcg_sum(blkcg, merged, prio));
return res;
}
static struct blkg_policy_data *dd_pd_alloc(gfp_t gfp, struct request_queue *q,
struct blkcg *blkcg)
{
struct dd_blkg *pd;
pd = kzalloc(sizeof(*pd), gfp);
if (!pd)
return NULL;
return &pd->pd;
}
static void dd_pd_free(struct blkg_policy_data *pd)
{
struct dd_blkg *dd_blkg = container_of(pd, typeof(*dd_blkg), pd);
kfree(dd_blkg);
}
static struct blkcg_policy dd_blkcg_policy = {
.cpd_alloc_fn = dd_cpd_alloc,
.cpd_free_fn = dd_cpd_free,
.pd_alloc_fn = dd_pd_alloc,
.pd_free_fn = dd_pd_free,
.pd_stat_fn = dd_pd_stat,
};
int dd_activate_policy(struct request_queue *q)
{
return blkcg_activate_policy(q, &dd_blkcg_policy);
}
void dd_deactivate_policy(struct request_queue *q)
{
blkcg_deactivate_policy(q, &dd_blkcg_policy);
}
int __init dd_blkcg_init(void)
{
return blkcg_policy_register(&dd_blkcg_policy);
}
void __exit dd_blkcg_exit(void)
{
blkcg_policy_unregister(&dd_blkcg_policy);
}

114
block/mq-deadline-cgroup.h Normal file
View File

@@ -0,0 +1,114 @@
/* SPDX-License-Identifier: GPL-2.0 */
#if !defined(_MQ_DEADLINE_CGROUP_H_)
#define _MQ_DEADLINE_CGROUP_H_
#include <linux/blk-cgroup.h>
struct request_queue;
/**
* struct io_stats_per_prio - I/O statistics per I/O priority class.
* @inserted: Number of inserted requests.
* @merged: Number of merged requests.
* @dispatched: Number of dispatched requests.
* @completed: Number of I/O completions.
*/
struct io_stats_per_prio {
local_t inserted;
local_t merged;
local_t dispatched;
local_t completed;
};
/* I/O statistics per I/O cgroup per I/O priority class (IOPRIO_CLASS_*). */
struct blkcg_io_stats {
struct io_stats_per_prio stats[4];
};
/**
* struct dd_blkcg - Per cgroup data.
* @cpd: blkcg_policy_data structure.
* @stats: I/O statistics.
*/
struct dd_blkcg {
struct blkcg_policy_data cpd; /* must be the first member */
struct blkcg_io_stats __percpu *stats;
};
/*
* Count one event of type 'event_type' and with I/O priority class
* 'prio_class'.
*/
#define ddcg_count(ddcg, event_type, prio_class) do { \
if (ddcg) { \
struct blkcg_io_stats *io_stats = get_cpu_ptr((ddcg)->stats); \
\
BUILD_BUG_ON(!__same_type((ddcg), struct dd_blkcg *)); \
BUILD_BUG_ON(!__same_type((prio_class), u8)); \
local_inc(&io_stats->stats[(prio_class)].event_type); \
put_cpu_ptr(io_stats); \
} \
} while (0)
/*
* Returns the total number of ddcg_count(ddcg, event_type, prio_class) calls
* across all CPUs. No locking or barriers since it is fine if the returned
* sum is slightly outdated.
*/
#define ddcg_sum(ddcg, event_type, prio) ({ \
unsigned int cpu; \
u32 sum = 0; \
\
BUILD_BUG_ON(!__same_type((ddcg), struct dd_blkcg *)); \
BUILD_BUG_ON(!__same_type((prio), u8)); \
for_each_present_cpu(cpu) \
sum += local_read(&per_cpu_ptr((ddcg)->stats, cpu)-> \
stats[(prio)].event_type); \
sum; \
})
#ifdef CONFIG_BLK_CGROUP
/**
* struct dd_blkg - Per (cgroup, request queue) data.
* @pd: blkg_policy_data structure.
*/
struct dd_blkg {
struct blkg_policy_data pd; /* must be the first member */
};
struct dd_blkcg *dd_blkcg_from_bio(struct bio *bio);
int dd_activate_policy(struct request_queue *q);
void dd_deactivate_policy(struct request_queue *q);
int __init dd_blkcg_init(void);
void __exit dd_blkcg_exit(void);
#else /* CONFIG_BLK_CGROUP */
static inline struct dd_blkcg *dd_blkcg_from_bio(struct bio *bio)
{
return NULL;
}
static inline int dd_activate_policy(struct request_queue *q)
{
return 0;
}
static inline void dd_deactivate_policy(struct request_queue *q)
{
}
static inline int dd_blkcg_init(void)
{
return 0;
}
static inline void dd_blkcg_exit(void)
{
}
#endif /* CONFIG_BLK_CGROUP */
#endif /* _MQ_DEADLINE_CGROUP_H_ */

1170
block/mq-deadline-main.c Normal file
View File

File diff suppressed because it is too large Load Diff

822
block/mq-deadline.c
View File

@@ -1,822 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
/*
* MQ Deadline i/o scheduler - adaptation of the legacy deadline scheduler,
* for the blk-mq scheduling framework
*
* Copyright (C) 2016 Jens Axboe <axboe@kernel.dk>
*/
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/blkdev.h>
#include <linux/blk-mq.h>
#include <linux/elevator.h>
#include <linux/bio.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/init.h>
#include <linux/compiler.h>
#include <linux/rbtree.h>
#include <linux/sbitmap.h>
#include "blk.h"
#include "blk-mq.h"
#include "blk-mq-debugfs.h"
#include "blk-mq-tag.h"
#include "blk-mq-sched.h"
/*
* See Documentation/block/deadline-iosched.rst
*/
static const int read_expire = HZ / 2; /* max time before a read is submitted. */
static const int write_expire = 5 * HZ; /* ditto for writes, these limits are SOFT! */
static const int writes_starved = 2; /* max times reads can starve a write */
static const int fifo_batch = 16; /* # of sequential requests treated as one
by the above parameters. For throughput. */
struct deadline_data {
/*
* run time data
*/
/*
* requests (deadline_rq s) are present on both sort_list and fifo_list
*/
struct rb_root sort_list[2];
struct list_head fifo_list[2];
/*
* next in sort order. read, write or both are NULL
*/
struct request *next_rq[2];
unsigned int batching; /* number of sequential requests made */
unsigned int starved; /* times reads have starved writes */
/*
* settings that change how the i/o scheduler behaves
*/
int fifo_expire[2];
int fifo_batch;
int writes_starved;
int front_merges;
spinlock_t lock;
spinlock_t zone_lock;
struct list_head dispatch;
};
static inline struct rb_root *
deadline_rb_root(struct deadline_data *dd, struct request *rq)
{
return &dd->sort_list[rq_data_dir(rq)];
}
/*
* get the request after `rq' in sector-sorted order
*/
static inline struct request *
deadline_latter_request(struct request *rq)
{
struct rb_node *node = rb_next(&rq->rb_node);
if (node)
return rb_entry_rq(node);
return NULL;
}
static void
deadline_add_rq_rb(struct deadline_data *dd, struct request *rq)
{
struct rb_root *root = deadline_rb_root(dd, rq);
elv_rb_add(root, rq);
}
static inline void
deadline_del_rq_rb(struct deadline_data *dd, struct request *rq)
{
const int data_dir = rq_data_dir(rq);
if (dd->next_rq[data_dir] == rq)
dd->next_rq[data_dir] = deadline_latter_request(rq);
elv_rb_del(deadline_rb_root(dd, rq), rq);
}
/*
* remove rq from rbtree and fifo.
*/
static void deadline_remove_request(struct request_queue *q, struct request *rq)
{
struct deadline_data *dd = q->elevator->elevator_data;
list_del_init(&rq->queuelist);
/*
* We might not be on the rbtree, if we are doing an insert merge
*/
if (!RB_EMPTY_NODE(&rq->rb_node))
deadline_del_rq_rb(dd, rq);
elv_rqhash_del(q, rq);
if (q->last_merge == rq)
q->last_merge = NULL;
}
static void dd_request_merged(struct request_queue *q, struct request *req,
enum elv_merge type)
{
struct deadline_data *dd = q->elevator->elevator_data;
/*
* if the merge was a front merge, we need to reposition request
*/
if (type == ELEVATOR_FRONT_MERGE) {
elv_rb_del(deadline_rb_root(dd, req), req);
deadline_add_rq_rb(dd, req);
}
}
static void dd_merged_requests(struct request_queue *q, struct request *req,
struct request *next)
{
/*
* if next expires before rq, assign its expire time to rq
* and move into next position (next will be deleted) in fifo
*/
if (!list_empty(&req->queuelist) && !list_empty(&next->queuelist)) {
if (time_before((unsigned long)next->fifo_time,
(unsigned long)req->fifo_time)) {
list_move(&req->queuelist, &next->queuelist);
req->fifo_time = next->fifo_time;
}
}
/*
* kill knowledge of next, this one is a goner
*/
deadline_remove_request(q, next);
}
/*
* move an entry to dispatch queue
*/
static void
deadline_move_request(struct deadline_data *dd, struct request *rq)
{
const int data_dir = rq_data_dir(rq);
dd->next_rq[READ] = NULL;
dd->next_rq[WRITE] = NULL;
dd->next_rq[data_dir] = deadline_latter_request(rq);
/*
* take it off the sort and fifo list
*/
deadline_remove_request(rq->q, rq);
}
/*
* deadline_check_fifo returns 0 if there are no expired requests on the fifo,
* 1 otherwise. Requires !list_empty(&dd->fifo_list[data_dir])
*/
static inline int deadline_check_fifo(struct deadline_data *dd, int ddir)
{
struct request *rq = rq_entry_fifo(dd->fifo_list[ddir].next);
/*
* rq is expired!
*/
if (time_after_eq(jiffies, (unsigned long)rq->fifo_time))
return 1;
return 0;
}
/*
* For the specified data direction, return the next request to
* dispatch using arrival ordered lists.
*/
static struct request *
deadline_fifo_request(struct deadline_data *dd, int data_dir)
{
struct request *rq;
unsigned long flags;
if (WARN_ON_ONCE(data_dir != READ && data_dir != WRITE))
return NULL;
if (list_empty(&dd->fifo_list[data_dir]))
return NULL;
rq = rq_entry_fifo(dd->fifo_list[data_dir].next);
if (data_dir == READ || !blk_queue_is_zoned(rq->q))
return rq;
/*
* Look for a write request that can be dispatched, that is one with
* an unlocked target zone.
*/
spin_lock_irqsave(&dd->zone_lock, flags);
list_for_each_entry(rq, &dd->fifo_list[WRITE], queuelist) {
if (blk_req_can_dispatch_to_zone(rq))
goto out;
}
rq = NULL;
out:
spin_unlock_irqrestore(&dd->zone_lock, flags);
return rq;
}
/*
* For the specified data direction, return the next request to
* dispatch using sector position sorted lists.
*/
static struct request *
deadline_next_request(struct deadline_data *dd, int data_dir)
{
struct request *rq;
unsigned long flags;
if (WARN_ON_ONCE(data_dir != READ && data_dir != WRITE))
return NULL;
rq = dd->next_rq[data_dir];
if (!rq)
return NULL;
if (data_dir == READ || !blk_queue_is_zoned(rq->q))
return rq;
/*
* Look for a write request that can be dispatched, that is one with
* an unlocked target zone.
*/
spin_lock_irqsave(&dd->zone_lock, flags);
while (rq) {
if (blk_req_can_dispatch_to_zone(rq))
break;
rq = deadline_latter_request(rq);
}
spin_unlock_irqrestore(&dd->zone_lock, flags);
return rq;
}
/*
* deadline_dispatch_requests selects the best request according to
* read/write expire, fifo_batch, etc
*/
static struct request *__dd_dispatch_request(struct deadline_data *dd)
{
struct request *rq, *next_rq;
bool reads, writes;
int data_dir;
if (!list_empty(&dd->dispatch)) {
rq = list_first_entry(&dd->dispatch, struct request, queuelist);
list_del_init(&rq->queuelist);
goto done;
}
reads = !list_empty(&dd->fifo_list[READ]);
writes = !list_empty(&dd->fifo_list[WRITE]);
/*
* batches are currently reads XOR writes
*/
rq = deadline_next_request(dd, WRITE);
if (!rq)
rq = deadline_next_request(dd, READ);
if (rq && dd->batching < dd->fifo_batch)
/* we have a next request are still entitled to batch */
goto dispatch_request;
/*
* at this point we are not running a batch. select the appropriate
* data direction (read / write)
*/
if (reads) {
BUG_ON(RB_EMPTY_ROOT(&dd->sort_list[READ]));
if (deadline_fifo_request(dd, WRITE) &&
(dd->starved++ >= dd->writes_starved))
goto dispatch_writes;
data_dir = READ;
goto dispatch_find_request;
}
/*
* there are either no reads or writes have been starved
*/
if (writes) {
dispatch_writes:
BUG_ON(RB_EMPTY_ROOT(&dd->sort_list[WRITE]));
dd->starved = 0;
data_dir = WRITE;
goto dispatch_find_request;
}
return NULL;
dispatch_find_request:
/*
* we are not running a batch, find best request for selected data_dir
*/
next_rq = deadline_next_request(dd, data_dir);
if (deadline_check_fifo(dd, data_dir) || !next_rq) {
/*
* A deadline has expired, the last request was in the other
* direction, or we have run out of higher-sectored requests.
* Start again from the request with the earliest expiry time.
*/
rq = deadline_fifo_request(dd, data_dir);
} else {
/*
* The last req was the same dir and we have a next request in
* sort order. No expired requests so continue on from here.
*/
rq = next_rq;
}
/*
* For a zoned block device, if we only have writes queued and none of
* them can be dispatched, rq will be NULL.
*/
if (!rq)
return NULL;
dd->batching = 0;
dispatch_request:
/*
* rq is the selected appropriate request.
*/
dd->batching++;
deadline_move_request(dd, rq);
done:
/*
* If the request needs its target zone locked, do it.
*/
blk_req_zone_write_lock(rq);
rq->rq_flags |= RQF_STARTED;
return rq;
}
/*
* One confusing aspect here is that we get called for a specific
* hardware queue, but we may return a request that is for a
* different hardware queue. This is because mq-deadline has shared
* state for all hardware queues, in terms of sorting, FIFOs, etc.
*/
static struct request *dd_dispatch_request(struct blk_mq_hw_ctx *hctx)
{
struct deadline_data *dd = hctx->queue->elevator->elevator_data;
struct request *rq;
spin_lock(&dd->lock);
rq = __dd_dispatch_request(dd);
spin_unlock(&dd->lock);
if (rq)
atomic_dec(&rq->mq_hctx->elevator_queued);
return rq;
}
static void dd_exit_queue(struct elevator_queue *e)
{
struct deadline_data *dd = e->elevator_data;
BUG_ON(!list_empty(&dd->fifo_list[READ]));
BUG_ON(!list_empty(&dd->fifo_list[WRITE]));
kfree(dd);
}
/*
* initialize elevator private data (deadline_data).
*/
static int dd_init_queue(struct request_queue *q, struct elevator_type *e)
{
struct deadline_data *dd;
struct elevator_queue *eq;
eq = elevator_alloc(q, e);
if (!eq)
return -ENOMEM;
dd = kzalloc_node(sizeof(*dd), GFP_KERNEL, q->node);
if (!dd) {
kobject_put(&eq->kobj);
return -ENOMEM;
}
eq->elevator_data = dd;
INIT_LIST_HEAD(&dd->fifo_list[READ]);
INIT_LIST_HEAD(&dd->fifo_list[WRITE]);
dd->sort_list[READ] = RB_ROOT;
dd->sort_list[WRITE] = RB_ROOT;
dd->fifo_expire[READ] = read_expire;
dd->fifo_expire[WRITE] = write_expire;
dd->writes_starved = writes_starved;
dd->front_merges = 1;
dd->fifo_batch = fifo_batch;
spin_lock_init(&dd->lock);
spin_lock_init(&dd->zone_lock);
INIT_LIST_HEAD(&dd->dispatch);
q->elevator = eq;
return 0;
}
static int dd_request_merge(struct request_queue *q, struct request **rq,
struct bio *bio)
{
struct deadline_data *dd = q->elevator->elevator_data;
sector_t sector = bio_end_sector(bio);
struct request *__rq;
if (!dd->front_merges)
return ELEVATOR_NO_MERGE;
__rq = elv_rb_find(&dd->sort_list[bio_data_dir(bio)], sector);
if (__rq) {
BUG_ON(sector != blk_rq_pos(__rq));
if (elv_bio_merge_ok(__rq, bio)) {
*rq = __rq;
return ELEVATOR_FRONT_MERGE;
}
}
return ELEVATOR_NO_MERGE;
}
static bool dd_bio_merge(struct request_queue *q, struct bio *bio,
unsigned int nr_segs)
{
struct deadline_data *dd = q->elevator->elevator_data;
struct request *free = NULL;
bool ret;
spin_lock(&dd->lock);
ret = blk_mq_sched_try_merge(q, bio, nr_segs, &free);
spin_unlock(&dd->lock);
if (free)
blk_mq_free_request(free);
return ret;
}
/*
* add rq to rbtree and fifo
*/
static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq,
bool at_head)
{
struct request_queue *q = hctx->queue;
struct deadline_data *dd = q->elevator->elevator_data;
const int data_dir = rq_data_dir(rq);
/*
* This may be a requeue of a write request that has locked its
* target zone. If it is the case, this releases the zone lock.
*/
blk_req_zone_write_unlock(rq);
if (blk_mq_sched_try_insert_merge(q, rq))
return;
blk_mq_sched_request_inserted(rq);
if (at_head || blk_rq_is_passthrough(rq)) {
if (at_head)
list_add(&rq->queuelist, &dd->dispatch);
else
list_add_tail(&rq->queuelist, &dd->dispatch);
} else {
deadline_add_rq_rb(dd, rq);
if (rq_mergeable(rq)) {
elv_rqhash_add(q, rq);
if (!q->last_merge)
q->last_merge = rq;
}
/*
* set expire time and add to fifo list
*/
rq->fifo_time = jiffies + dd->fifo_expire[data_dir];
list_add_tail(&rq->queuelist, &dd->fifo_list[data_dir]);
}
}
static void dd_insert_requests(struct blk_mq_hw_ctx *hctx,
struct list_head *list, bool at_head)
{
struct request_queue *q = hctx->queue;
struct deadline_data *dd = q->elevator->elevator_data;
spin_lock(&dd->lock);
while (!list_empty(list)) {
struct request *rq;
rq = list_first_entry(list, struct request, queuelist);
list_del_init(&rq->queuelist);
dd_insert_request(hctx, rq, at_head);
atomic_inc(&hctx->elevator_queued);
}
spin_unlock(&dd->lock);
}
/*
* Nothing to do here. This is defined only to ensure that .finish_request
* method is called upon request completion.
*/
static void dd_prepare_request(struct request *rq)
{
}
/*
* For zoned block devices, write unlock the target zone of
* completed write requests. Do this while holding the zone lock
* spinlock so that the zone is never unlocked while deadline_fifo_request()
* or deadline_next_request() are executing. This function is called for
* all requests, whether or not these requests complete successfully.
*
* For a zoned block device, __dd_dispatch_request() may have stopped
* dispatching requests if all the queued requests are write requests directed
* at zones that are already locked due to on-going write requests. To ensure
* write request dispatch progress in this case, mark the queue as needing a
* restart to ensure that the queue is run again after completion of the
* request and zones being unlocked.
*/
static void dd_finish_request(struct request *rq)
{
struct request_queue *q = rq->q;
if (blk_queue_is_zoned(q)) {
struct deadline_data *dd = q->elevator->elevator_data;
unsigned long flags;
spin_lock_irqsave(&dd->zone_lock, flags);
blk_req_zone_write_unlock(rq);
if (!list_empty(&dd->fifo_list[WRITE]))
blk_mq_sched_mark_restart_hctx(rq->mq_hctx);
spin_unlock_irqrestore(&dd->zone_lock, flags);
}
}
static bool dd_has_work(struct blk_mq_hw_ctx *hctx)
{
struct deadline_data *dd = hctx->queue->elevator->elevator_data;
if (!atomic_read(&hctx->elevator_queued))
return false;
return !list_empty_careful(&dd->dispatch) ||
!list_empty_careful(&dd->fifo_list[0]) ||
!list_empty_careful(&dd->fifo_list[1]);
}
/*
* sysfs parts below
*/
static ssize_t
deadline_var_show(int var, char *page)
{
return sprintf(page, "%d\n", var);
}
static void
deadline_var_store(int *var, const char *page)
{
char *p = (char *) page;
*var = simple_strtol(p, &p, 10);
}
#define SHOW_FUNCTION(__FUNC, __VAR, __CONV) \
static ssize_t __FUNC(struct elevator_queue *e, char *page) \
{ \
struct deadline_data *dd = e->elevator_data; \
int __data = __VAR; \
if (__CONV) \
__data = jiffies_to_msecs(__data); \
return deadline_var_show(__data, (page)); \
}
SHOW_FUNCTION(deadline_read_expire_show, dd->fifo_expire[READ], 1);
SHOW_FUNCTION(deadline_write_expire_show, dd->fifo_expire[WRITE], 1);
SHOW_FUNCTION(deadline_writes_starved_show, dd->writes_starved, 0);
SHOW_FUNCTION(deadline_front_merges_show, dd->front_merges, 0);
SHOW_FUNCTION(deadline_fifo_batch_show, dd->fifo_batch, 0);
#undef SHOW_FUNCTION
#define STORE_FUNCTION(__FUNC, __PTR, MIN, MAX, __CONV) \
static ssize_t __FUNC(struct elevator_queue *e, const char *page, size_t count) \
{ \
struct deadline_data *dd = e->elevator_data; \
int __data; \
deadline_var_store(&__data, (page)); \
if (__data < (MIN)) \
__data = (MIN); \
else if (__data > (MAX)) \
__data = (MAX); \
if (__CONV) \
*(__PTR) = msecs_to_jiffies(__data); \
else \
*(__PTR) = __data; \
return count; \
}
STORE_FUNCTION(deadline_read_expire_store, &dd->fifo_expire[READ], 0, INT_MAX, 1);
STORE_FUNCTION(deadline_write_expire_store, &dd->fifo_expire[WRITE], 0, INT_MAX, 1);
STORE_FUNCTION(deadline_writes_starved_store, &dd->writes_starved, INT_MIN, INT_MAX, 0);
STORE_FUNCTION(deadline_front_merges_store, &dd->front_merges, 0, 1, 0);
STORE_FUNCTION(deadline_fifo_batch_store, &dd->fifo_batch, 0, INT_MAX, 0);
#undef STORE_FUNCTION
#define DD_ATTR(name) \
__ATTR(name, 0644, deadline_##name##_show, deadline_##name##_store)
static struct elv_fs_entry deadline_attrs[] = {
DD_ATTR(read_expire),
DD_ATTR(write_expire),
DD_ATTR(writes_starved),
DD_ATTR(front_merges),
DD_ATTR(fifo_batch),
__ATTR_NULL
};
#ifdef CONFIG_BLK_DEBUG_FS
#define DEADLINE_DEBUGFS_DDIR_ATTRS(ddir, name) \
static void *deadline_##name##_fifo_start(struct seq_file *m, \
loff_t *pos) \
__acquires(&dd->lock) \
{ \
struct request_queue *q = m->private; \
struct deadline_data *dd = q->elevator->elevator_data; \
\
spin_lock(&dd->lock); \
return seq_list_start(&dd->fifo_list[ddir], *pos); \
} \
\
static void *deadline_##name##_fifo_next(struct seq_file *m, void *v, \
loff_t *pos) \
{ \
struct request_queue *q = m->private; \
struct deadline_data *dd = q->elevator->elevator_data; \
\
return seq_list_next(v, &dd->fifo_list[ddir], pos); \
} \
\
static void deadline_##name##_fifo_stop(struct seq_file *m, void *v) \
__releases(&dd->lock) \
{ \
struct request_queue *q = m->private; \
struct deadline_data *dd = q->elevator->elevator_data; \
\
spin_unlock(&dd->lock); \
} \
\
static const struct seq_operations deadline_##name##_fifo_seq_ops = { \
.start = deadline_##name##_fifo_start, \
.next = deadline_##name##_fifo_next, \
.stop = deadline_##name##_fifo_stop, \
.show = blk_mq_debugfs_rq_show, \
}; \
\
static int deadline_##name##_next_rq_show(void *data, \
struct seq_file *m) \
{ \
struct request_queue *q = data; \
struct deadline_data *dd = q->elevator->elevator_data; \
struct request *rq = dd->next_rq[ddir]; \
\
if (rq) \
__blk_mq_debugfs_rq_show(m, rq); \
return 0; \
}
DEADLINE_DEBUGFS_DDIR_ATTRS(READ, read)
DEADLINE_DEBUGFS_DDIR_ATTRS(WRITE, write)
#undef DEADLINE_DEBUGFS_DDIR_ATTRS
static int deadline_batching_show(void *data, struct seq_file *m)
{
struct request_queue *q = data;
struct deadline_data *dd = q->elevator->elevator_data;
seq_printf(m, "%u\n", dd->batching);
return 0;
}
static int deadline_starved_show(void *data, struct seq_file *m)
{
struct request_queue *q = data;
struct deadline_data *dd = q->elevator->elevator_data;
seq_printf(m, "%u\n", dd->starved);
return 0;
}
static void *deadline_dispatch_start(struct seq_file *m, loff_t *pos)
__acquires(&dd->lock)
{
struct request_queue *q = m->private;
struct deadline_data *dd = q->elevator->elevator_data;
spin_lock(&dd->lock);
return seq_list_start(&dd->dispatch, *pos);
}
static void *deadline_dispatch_next(struct seq_file *m, void *v, loff_t *pos)
{
struct request_queue *q = m->private;
struct deadline_data *dd = q->elevator->elevator_data;
return seq_list_next(v, &dd->dispatch, pos);
}
static void deadline_dispatch_stop(struct seq_file *m, void *v)
__releases(&dd->lock)
{
struct request_queue *q = m->private;
struct deadline_data *dd = q->elevator->elevator_data;
spin_unlock(&dd->lock);
}
static const struct seq_operations deadline_dispatch_seq_ops = {
.start = deadline_dispatch_start,
.next = deadline_dispatch_next,
.stop = deadline_dispatch_stop,
.show = blk_mq_debugfs_rq_show,
};
#define DEADLINE_QUEUE_DDIR_ATTRS(name) \
{#name "_fifo_list", 0400, .seq_ops = &deadline_##name##_fifo_seq_ops}, \
{#name "_next_rq", 0400, deadline_##name##_next_rq_show}
static const struct blk_mq_debugfs_attr deadline_queue_debugfs_attrs[] = {
DEADLINE_QUEUE_DDIR_ATTRS(read),
DEADLINE_QUEUE_DDIR_ATTRS(write),
{"batching", 0400, deadline_batching_show},
{"starved", 0400, deadline_starved_show},
{"dispatch", 0400, .seq_ops = &deadline_dispatch_seq_ops},
{},
};
#undef DEADLINE_QUEUE_DDIR_ATTRS
#endif
static struct elevator_type mq_deadline = {
.ops = {
.insert_requests = dd_insert_requests,
.dispatch_request = dd_dispatch_request,
.prepare_request = dd_prepare_request,
.finish_request = dd_finish_request,
.next_request = elv_rb_latter_request,
.former_request = elv_rb_former_request,
.bio_merge = dd_bio_merge,
.request_merge = dd_request_merge,
.requests_merged = dd_merged_requests,
.request_merged = dd_request_merged,
.has_work = dd_has_work,
.init_sched = dd_init_queue,
.exit_sched = dd_exit_queue,
},
#ifdef CONFIG_BLK_DEBUG_FS
.queue_debugfs_attrs = deadline_queue_debugfs_attrs,
#endif
.elevator_attrs = deadline_attrs,
.elevator_name = "mq-deadline",
.elevator_alias = "deadline",
.elevator_features = ELEVATOR_F_ZBD_SEQ_WRITE,
.elevator_owner = THIS_MODULE,
};
MODULE_ALIAS("mq-deadline-iosched");
static int __init deadline_init(void)
{
return elv_register(&mq_deadline);
}
static void __exit deadline_exit(void)
{
elv_unregister(&mq_deadline);
}
module_init(deadline_init);
module_exit(deadline_exit);
MODULE_AUTHOR("Jens Axboe");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("MQ deadline IO scheduler");

5
build.config.gki.aarch64
View File

@@ -7,6 +7,7 @@ Image.lz4
"
ABI_DEFINITION=android/abi_gki_aarch64.xml
TIDY_ABI=1
KMI_SYMBOL_LIST=android/abi_gki_aarch64
ADDITIONAL_KMI_SYMBOL_LISTS="
android/abi_gki_aarch64_core
@@ -14,11 +15,15 @@ android/abi_gki_aarch64_db845c
android/abi_gki_aarch64_qcom
android/abi_gki_aarch64_virtual_device
android/abi_gki_aarch64_hikey960
android/abi_gki_aarch64_galaxy
android/abi_gki_aarch64_generic
android/abi_gki_aarch64_exynos
android/abi_gki_aarch64_exynosauto
android/abi_gki_aarch64_mtk
android/abi_gki_aarch64_xiaomi
android/abi_gki_aarch64_fips140
android/abi_gki_aarch64_vivo
android/abi_gki_aarch64_oplus
"
FILES="${FILES}

24
crypto/Kconfig
View File

@@ -32,13 +32,35 @@ config CRYPTO_FIPS
certification. You should say no unless you know what
this is.
# CRYPTO_FIPS140 just enables the support in the kernel for loading fips140.ko.
# The module still needs to be built and loaded if you need FIPS 140 compliance.
config CRYPTO_FIPS140
def_bool y
depends on MODULES && ARM64 && ARM64_MODULE_PLTS
config CRYPTO_FIPS140_MOD
bool "Enable FIPS140 integrity self-checked loadable module"
bool "Enable FIPS 140 cryptographic module"
depends on LTO_CLANG && CRYPTO_FIPS140
help
This option enables building a loadable module fips140.ko, which
contains various crypto algorithms that are also built into vmlinux.
At load time, this module overrides the built-in implementations of
these algorithms with its implementations. It also runs self-tests on
these algorithms and verifies the integrity of its code and data. If
either of these steps fails, the kernel will panic.
This module is intended to be loaded at early boot time in order to
meet FIPS 140 and NIAP FPT_TST_EXT.1 requirements. It shouldn't be
used if you don't need to meet these requirements.
config CRYPTO_FIPS140_MOD_ERROR_INJECTION
bool "Support injecting failures into the FIPS 140 self-tests"
depends on CRYPTO_FIPS140_MOD
help
This option adds a module parameter "broken_alg" to the fips140 module
which can be used to fail the self-tests for a particular algorithm,
causing a kernel panic. This option is for FIPS lab testing only, and
it shouldn't be enabled on production systems.
config CRYPTO_ALGAPI
tristate

2
crypto/Makefile
View File

@@ -228,7 +228,7 @@ $(obj)/lib-crypto-%-fips.o: $(srctree)/lib/crypto/%.c FORCE
$(obj)/crypto-fips.a: $(addprefix $(obj)/,$(crypto-fips-objs)) FORCE
$(call if_changed,ar_and_symver)
fips140-objs := fips140-module.o crypto-fips.a
fips140-objs := fips140-module.o fips140-selftests.o crypto-fips.a
obj-m += fips140.o
CFLAGS_fips140-module.o += $(FIPS140_CFLAGS)

65
crypto/fips140-generated-testvecs.h Normal file
View File

@@ -0,0 +1,65 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright 2021 Google LLC */
/*
* This header was automatically generated by gen_fips140_testvecs.py.
* Don't edit it directly.
*/
static const u8 fips_message[32] __initconst =
"This is a 32-byte test message.";
static const u8 fips_aes_key[16] __initconst = "128-bit AES key";
static const u8 fips_aes_iv[16] __initconst = "ABCDEFGHIJKL";
static const u8 fips_aes_cbc_ciphertext[32] __initconst =
"\xc4\x6d\xad\xa4\x04\x52\x11\x5a\x7a\xb3\x7c\x68\x85\x8d\x90\xf0"
"\x55\xc3\xd3\x35\xc1\x75\x31\x90\xdf\x90\x4b\x5a\x56\xfd\xa7\x89";
static const u8 fips_aes_ecb_ciphertext[32] __initconst =
"\xc1\x9d\xe6\xb8\xb2\x90\xff\xfe\xf2\x77\x18\xb0\x55\xd3\xee\xa9"
"\xe2\x6f\x4a\x32\x67\xfd\xb7\xa5\x2f\x4b\x6e\x1a\x86\x2b\x6e\x3a";
static const u8 fips_aes_ctr_ciphertext[32] __initconst =
"\x92\xbe\x23\xa1\x80\x88\x5d\x31\x27\xb3\x9c\x40\x58\x57\x1d\xde"
"\xc1\x8d\x5b\xe7\x42\x93\x09\xf8\xd4\xf7\x49\x42\xcf\x40\x62\x7e";
static const u8 fips_aes_gcm_assoc[22] __initconst = "associated data string";
static const u8 fips_aes_gcm_ciphertext[48] __initconst =
"\x37\x88\x3e\x1d\x58\x50\xda\x10\x07\xeb\x52\xdf\xea\x0a\x54\xd4"
"\x44\xbf\x88\x2a\xf3\x03\x03\x84\xaf\x8b\x96\xbd\xea\x65\x60\x6f"
"\x82\xfa\x51\xf4\x28\xad\x0c\xf1\xce\x0f\x91\xdd\x1a\x4c\x77\x5f";
static const u8 fips_aes_xts_key[32] __initconst =
"This is an AES-128-XTS key.";
static const u8 fips_aes_xts_ciphertext[32] __initconst =
"\x5e\xb9\x98\xd6\x26\xb3\x55\xbf\x44\xab\x3e\xae\x73\xc0\x81\xc9"
"\xf4\x29\x0e\x17\x1e\xc5\xc8\x90\x79\x99\xf1\x43\x3a\x23\x08\x5a";
static const u8 fips_hmac_key[16] __initconst = "128-bit HMAC key";
static const u8 fips_sha1_digest[20] __initconst =
"\x1b\x78\xc7\x4b\xd5\xd4\x83\xb1\x58\xc5\x96\x83\x4f\x16\x8d\x15"
"\xb4\xaa\x22\x8c";
static const u8 fips_sha256_digest[32] __initconst =
"\x4e\x11\x83\x0c\x53\x80\x1e\x5f\x9b\x38\x33\x38\xe8\x74\x43\xb0"
"\xc1\x3a\xbe\xbf\x75\xf0\x12\x0f\x21\x33\xf5\x16\x33\xf1\xb0\x81";
static const u8 fips_hmac_sha256_digest[32] __initconst =
"\x63\x0e\xb5\x73\x79\xfc\xaf\x5f\x86\xe3\xaf\xf0\xc8\x36\xef\xd5"
"\x35\x8d\x40\x25\x38\xb3\x65\x72\x98\xf3\x59\xd8\x1e\x54\x4c\xa1";
static const u8 fips_sha512_digest[64] __initconst =
"\x32\xe0\x44\x23\xbd\xe3\xec\x28\xbf\xf1\x34\x11\xd5\xae\xbf\xd5"
"\xc0\x8e\xb5\xa1\x04\xef\x2f\x07\x84\xf1\xd9\x83\x0f\x6c\x31\xab"
"\xf7\xe7\x57\xfa\xf7\xae\xf0\x6f\xb2\x16\x08\x32\xcf\xc7\xef\x35"
"\xb3\x3b\x51\xb9\xfd\xe7\xff\x5e\xb2\x8b\xc6\x79\xe6\x14\x04\xb4";
/*
* This header was automatically generated by gen_fips140_testvecs.py.
* Don't edit it directly.
*/

143
crypto/fips140-module.c
View File

@@ -3,16 +3,17 @@
* Copyright 2021 Google LLC
* Author: Ard Biesheuvel <ardb@google.com>
*
* This file is the core of the fips140.ko, which carries a number of crypto
* algorithms and chaining mode templates that are also built into vmlinux.
* This modules performs a load time integrity check, as mandated by FIPS 140,
* and replaces registered crypto algorithms that appear on the FIPS 140 list
* with ones provided by this module. This meets the FIPS 140 requirements for
* a cryptographic software module.
* This file is the core of fips140.ko, which contains various crypto algorithms
* that are also built into vmlinux. At load time, this module overrides the
* built-in implementations of these algorithms with its implementations. It
* also runs self-tests on these algorithms and verifies the integrity of its
* code and data. If either of these steps fails, the kernel will panic.
*
* This module is intended to be loaded at early boot time in order to meet
* FIPS 140 and NIAP FPT_TST_EXT.1 requirements. It shouldn't be used if you
* don't need to meet these requirements.
*/
#define pr_fmt(fmt) "fips140: " fmt
#include <linux/ctype.h>
#include <linux/module.h>
#include <crypto/aead.h>
@@ -23,8 +24,18 @@
#include <crypto/rng.h>
#include <trace/hooks/fips140.h>
#include "fips140-module.h"
#include "internal.h"
/*
* This option allows deliberately failing the self-tests for a particular
* algorithm. This is for FIPS lab testing only.
*/
#ifdef CONFIG_CRYPTO_FIPS140_MOD_ERROR_INJECTION
char *fips140_broken_alg;
module_param_named(broken_alg, fips140_broken_alg, charp, 0);
#endif
/*
* FIPS 140-2 prefers the use of HMAC with a public key over a plain hash.
*/
@@ -52,7 +63,13 @@ const u32 *__initcall_start = &__initcall_start_marker;
const u8 *__text_start = &__fips140_text_start;
const u8 *__rodata_start = &__fips140_rodata_start;
static const char fips140_algorithms[][22] __initconst = {
/*
* The list of the crypto API algorithms (by cra_name) that will be unregistered
* by this module, in preparation for the module registering its own
* implementation(s) of them. When adding a new algorithm here, make sure to
* consider whether it needs a self-test added to fips140_selftests[] as well.
*/
static const char * const fips140_algorithms[] __initconst = {
"aes",
"gcm(aes)",
@@ -73,28 +90,7 @@ static const char fips140_algorithms[][22] __initconst = {
"sha384",
"sha512",
"drbg_nopr_ctr_aes256",
"drbg_nopr_ctr_aes192",
"drbg_nopr_ctr_aes128",
"drbg_nopr_hmac_sha512",
"drbg_nopr_hmac_sha384",
"drbg_nopr_hmac_sha256",
"drbg_nopr_hmac_sha1",
"drbg_nopr_sha512",
"drbg_nopr_sha384",
"drbg_nopr_sha256",
"drbg_nopr_sha1",
"drbg_pr_ctr_aes256",
"drbg_pr_ctr_aes192",
"drbg_pr_ctr_aes128",
"drbg_pr_hmac_sha512",
"drbg_pr_hmac_sha384",
"drbg_pr_hmac_sha256",
"drbg_pr_hmac_sha1",
"drbg_pr_sha512",
"drbg_pr_sha384",
"drbg_pr_sha256",
"drbg_pr_sha1",
"stdrng",
};
static bool __init is_fips140_algo(struct crypto_alg *alg)
@@ -116,6 +112,38 @@ static bool __init is_fips140_algo(struct crypto_alg *alg)
static LIST_HEAD(unchecked_fips140_algos);
/*
* Release a list of algorithms which have been removed from crypto_alg_list.
*
* Note that even though the list is a private list, we have to hold
* crypto_alg_sem while iterating through it because crypto_unregister_alg() may
* run concurrently (as we haven't taken a reference to the algorithms on the
* list), and crypto_unregister_alg() will remove the algorithm from whichever
* list it happens to be on, while holding crypto_alg_sem. That's okay, since
* in that case crypto_unregister_alg() will handle the crypto_alg_put().
*/
static void fips140_remove_final(struct list_head *list)
{
struct crypto_alg *alg;
struct crypto_alg *n;
/*
* We need to take crypto_alg_sem to safely traverse the list (see
* comment above), but we have to drop it when doing each
* crypto_alg_put() as that may take crypto_alg_sem again.
*/
down_write(&crypto_alg_sem);
list_for_each_entry_safe(alg, n, list, cra_list) {
list_del_init(&alg->cra_list);
up_write(&crypto_alg_sem);
crypto_alg_put(alg);
down_write(&crypto_alg_sem);
}
up_write(&crypto_alg_sem);
}
static void __init unregister_existing_fips140_algos(void)
{
struct crypto_alg *alg, *tmp;
@@ -152,25 +180,17 @@ static void __init unregister_existing_fips140_algos(void)
* transformations. We will swap these out
* later with integrity checked versions.
*/
pr_info("found already-live algorithm '%s' ('%s')\n",
alg->cra_name, alg->cra_driver_name);
list_move(&alg->cra_list,
&unchecked_fips140_algos);
}
}
}
/*
* We haven't taken a reference to the algorithms on the remove_list,
* so technically, we may be competing with a concurrent invocation of
* crypto_unregister_alg() here. Fortunately, crypto_unregister_alg()
* just gives up with a warning if the algo that is being unregistered
* has already disappeared, so this happens to be safe. That does mean
* we need to hold on to the lock, to ensure that the algo is either on
* the list or it is not, and not in some limbo state.
*/
crypto_remove_final(&remove_list);
crypto_remove_final(&spawns);
up_write(&crypto_alg_sem);
fips140_remove_final(&remove_list);
fips140_remove_final(&spawns);
}
static void __init unapply_text_relocations(void *section, int section_size,
@@ -250,8 +270,8 @@ static bool __init check_fips140_module_hmac(void)
textsize = &__fips140_text_end - &__fips140_text_start;
rodatasize = &__fips140_rodata_end - &__fips140_rodata_start;
pr_warn("text size : 0x%x\n", textsize);
pr_warn("rodata size: 0x%x\n", rodatasize);
pr_info("text size : 0x%x\n", textsize);
pr_info("rodata size: 0x%x\n", rodatasize);
textcopy = kmalloc(textsize + rodatasize, GFP_KERNEL);
if (!textcopy) {
@@ -283,7 +303,7 @@ static bool __init check_fips140_module_hmac(void)
return false;
}
pr_warn("using '%s' for integrity check\n",
pr_info("using '%s' for integrity check\n",
crypto_shash_driver_name(desc->tfm));
err = crypto_shash_setkey(desc->tfm, fips140_integ_hmac_key,
@@ -523,11 +543,12 @@ static bool update_fips140_library_routines(void)
* let's disable CFI locally when handling the initcall array, to avoid
* surpises.
*/
int __init __attribute__((__no_sanitize__("cfi"))) fips140_init(void)
static int __init __attribute__((__no_sanitize__("cfi")))
fips140_init(void)
{
const u32 *initcall;
pr_info("Loading FIPS 140 module\n");
pr_info("loading module\n");
unregister_existing_fips140_algos();
@@ -536,8 +557,17 @@ int __init __attribute__((__no_sanitize__("cfi"))) fips140_init(void)
initcall < &__initcall_end_marker;
initcall++) {
int (*init)(void) = offset_to_ptr(initcall);
int err = init();
init();
/*
* ENODEV is expected from initcalls that only register
* algorithms that depend on non-present CPU features. Besides
* that, errors aren't expected here.
*/
if (err && err != -ENODEV) {
pr_err("initcall %ps() failed: %d\n", init, err);
goto panic;
}
}
if (!update_live_fips140_algos())
@@ -553,22 +583,25 @@ int __init __attribute__((__no_sanitize__("cfi"))) fips140_init(void)
*/
synchronize_rcu_tasks();
/* insert self tests here */
if (!fips140_run_selftests())
goto panic;
/*
* It may seem backward to perform the integrity check last, but this
* is intentional: the check itself uses hmac(sha256) which is one of
* the algorithms that are replaced with versions from this module, and
* the integrity check must use the replacement version.
* the integrity check must use the replacement version. Also, to be
* ready for FIPS 140-3, the integrity check algorithm must have already
* been self-tested.
*/
if (!check_fips140_module_hmac()) {
pr_crit("FIPS 140 integrity check failed -- giving up!\n");
pr_crit("integrity check failed -- giving up!\n");
goto panic;
}
pr_info("integrity check passed\n");
pr_info("FIPS 140 integrity check successful\n");
pr_info("FIPS 140 module successfully loaded\n");
pr_info("module successfully loaded\n");
return 0;
panic:

20
crypto/fips140-module.h Normal file
View File

@@ -0,0 +1,20 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright 2021 Google LLC
*/
#ifndef _CRYPTO_FIPS140_MODULE_H
#define _CRYPTO_FIPS140_MODULE_H
#include <linux/module.h>
#undef pr_fmt
#define pr_fmt(fmt) "fips140: " fmt
#ifdef CONFIG_CRYPTO_FIPS140_MOD_ERROR_INJECTION
extern char *fips140_broken_alg;
#endif
bool __init __must_check fips140_run_selftests(void);
#endif /* _CRYPTO_FIPS140_MODULE_H */

867
crypto/fips140-selftests.c Normal file
View File

@@ -0,0 +1,867 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright 2021 Google LLC
*
* Authors: Elena Petrova <lenaptr@google.com>,
* Eric Biggers <ebiggers@google.com>
*
* Self-tests of fips140.ko cryptographic functionality. These are run at
* module load time to fulfill FIPS 140 and NIAP FPT_TST_EXT.1 requirements.
*
* The actual requirements for these self-tests are somewhat vague, but
* section 9 ("Self-Tests") of the FIPS 140-2 Implementation Guidance document
* (https://csrc.nist.gov/csrc/media/projects/cryptographic-module-validation-program/documents/fips140-2/fips1402ig.pdf)
* is somewhat helpful. Basically, all implementations of all FIPS approved
* algorithms (including modes of operation) must be tested. However:
*
* - If an implementation won't be used, it doesn't have to be tested. So
* when multiple implementations of the same algorithm are registered with
* the crypto API, we only have to test the default (highest-priority) one.
*
* - There are provisions for skipping tests that are already sufficiently
* covered by other tests. E.g., HMAC-SHA256 may cover SHA-256.
*
* - Only one test vector is required per algorithm, and it can be generated
* by any known-good implementation or taken from any official document.
*
* - For ciphers, both encryption and decryption must be tested.
*
* - Only one key size per algorithm needs to be tested.
*
* See fips140_selftests[] for the list of tests we've selected. Currently, all
* our test vectors except the DRBG ones were generated by the script
* tools/crypto/gen_fips140_testvecs.py, using the known-good implementations in
* the Python packages hashlib, pycryptodome, and cryptography. The DRBG test
* vectors were manually extracted from
* https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Algorithm-Validation-Program/documents/drbg/drbgtestvectors.zip.
*
* Note that we don't reuse the upstream crypto API's self-tests
* (crypto/testmgr.{c,h}), for several reasons:
*
* - To meet FIPS requirements, the self-tests must be located within the FIPS
* module boundary (fips140.ko). But testmgr is integrated into the crypto
* API framework and can't be extracted into the module.
*
* - testmgr is much more heavyweight than required for FIPS and NIAP; it
* tests more algorithms and does more tests per algorithm, as it's meant to
* do proper testing and not just meet certification requirements. We need
* tests that can run with minimal overhead on every boot-up.
*
* - Despite being more heavyweight in general, testmgr doesn't test the
* SHA-256 and AES library APIs, despite that being needed here.
*/
#include <crypto/aead.h>
#include <crypto/aes.h>
#include <crypto/drbg.h>
#include <crypto/hash.h>
#include <crypto/internal/cipher.h>
#include <crypto/rng.h>
#include <crypto/sha.h>
#include <crypto/skcipher.h>
#include "fips140-module.h"
/* Test vector for a block cipher algorithm */
struct blockcipher_testvec {
const u8 *key;
size_t key_size;
const u8 *plaintext;
const u8 *ciphertext;
size_t block_size;
};
/* Test vector for an AEAD algorithm */
struct aead_testvec {
const u8 *key;
size_t key_size;
const u8 *iv;
size_t iv_size;
const u8 *assoc;
size_t assoc_size;
const u8 *plaintext;
size_t plaintext_size;
const u8 *ciphertext;
size_t ciphertext_size;
};
/* Test vector for a length-preserving encryption algorithm */
struct skcipher_testvec {
const u8 *key;
size_t key_size;
const u8 *iv;
size_t iv_size;
const u8 *plaintext;
const u8 *ciphertext;
size_t message_size;
};
/* Test vector for a hash algorithm */
struct hash_testvec {
const u8 *key;
size_t key_size;
const u8 *message;
size_t message_size;
const u8 *digest;
size_t digest_size;
};
/* Test vector for a DRBG algorithm */
struct drbg_testvec {
const u8 *entropy;
size_t entropy_size;
const u8 *pers;
size_t pers_size;
const u8 *entpr_a;
const u8 *entpr_b;
size_t entpr_size;
const u8 *add_a;
const u8 *add_b;
size_t add_size;
const u8 *output;
size_t out_size;
};
/*
* A struct which specifies an algorithm name (using crypto API syntax), a test
* function for that algorithm, and a test vector used by that test function.
*/
struct fips_test {
const char *alg;
int __must_check (*func)(const struct fips_test *test);
union {
struct blockcipher_testvec blockcipher;
struct aead_testvec aead;
struct skcipher_testvec skcipher;
struct hash_testvec hash;
struct drbg_testvec drbg;
};
};
/* Maximum IV size (in bytes) among any algorithm tested here */
#define MAX_IV_SIZE 16
static int __init __must_check
fips_check_result(const struct fips_test *test, u8 *result,
const u8 *expected_result, size_t result_size,
const char *operation)
{
#ifdef CONFIG_CRYPTO_FIPS140_MOD_ERROR_INJECTION
/* Inject a failure (via corrupting the result) if requested. */
if (fips140_broken_alg && strcmp(test->alg, fips140_broken_alg) == 0)
result[0] ^= 0xff;
#endif
if (memcmp(result, expected_result, result_size) != 0) {
pr_err("wrong result from %s %s\n", test->alg, operation);
return -EBADMSG;
}
return 0;
}
/*
* None of the algorithms should be ASYNC, as the FIPS module doesn't register
* any ASYNC algorithms. (The ASYNC flag is only declared by hardware
* algorithms, which would need their own FIPS certification.)
*
* Ideally we would verify alg->cra_module == THIS_MODULE here as well, but that
* doesn't work because the files are compiled as built-in code.
*/
static int __init __must_check
fips_validate_alg(const struct crypto_alg *alg)
{
if (alg->cra_flags & CRYPTO_ALG_ASYNC) {
pr_err("unexpectedly got async implementation of %s (%s)\n",
alg->cra_name, alg->cra_driver_name);
return -EINVAL;
}
return 0;
}
/* Test a block cipher using the crypto_cipher API. */
static int __init __must_check
fips_test_blockcipher(const struct fips_test *test)
{
const struct blockcipher_testvec *vec = &test->blockcipher;
struct crypto_cipher *tfm;
u8 block[MAX_CIPHER_BLOCKSIZE];
int err;
if (WARN_ON(vec->block_size > MAX_CIPHER_BLOCKSIZE))
return -EINVAL;
tfm = crypto_alloc_cipher(test->alg, 0, 0);
if (IS_ERR(tfm)) {
err = PTR_ERR(tfm);
pr_err("failed to allocate %s tfm: %d\n", test->alg, err);
return err;
}
err = fips_validate_alg(tfm->base.__crt_alg);
if (err)
goto out;
if (crypto_cipher_blocksize(tfm) != vec->block_size) {
pr_err("%s has wrong block size\n", test->alg);
err = -EINVAL;
goto out;
}
err = crypto_cipher_setkey(tfm, vec->key, vec->key_size);
if (err) {
pr_err("failed to set %s key: %d\n", test->alg, err);
goto out;
}
/* Encrypt the plaintext, then verify the resulting ciphertext. */
memcpy(block, vec->plaintext, vec->block_size);
crypto_cipher_encrypt_one(tfm, block, block);
err = fips_check_result(test, block, vec->ciphertext, vec->block_size,
"encryption");
if (err)
goto out;
/* Decrypt the ciphertext, then verify the resulting plaintext. */
crypto_cipher_decrypt_one(tfm, block, block);
err = fips_check_result(test, block, vec->plaintext, vec->block_size,
"decryption");
out:
crypto_free_cipher(tfm);
return err;
}
/*
* Test for plain AES (no mode of operation). We test this separately from the
* AES modes because the implementation of AES which is used by the "aes"
* crypto_cipher isn't necessarily the same as that used by the AES modes such
* as "ecb(aes)". Similarly, the aes_{encrypt,decrypt}() library functions may
* use a different implementation as well, so we test them separately too.
*/
static int __init __must_check
fips_test_aes(const struct fips_test *test)
{
const struct blockcipher_testvec *vec = &test->blockcipher;
struct crypto_aes_ctx ctx;
u8 block[AES_BLOCK_SIZE];
int err;
if (WARN_ON(vec->block_size != AES_BLOCK_SIZE))
return -EINVAL;
err = fips_test_blockcipher(test);
if (err)
return err;
err = aes_expandkey(&ctx, vec->key, vec->key_size);
if (err) {
pr_err("aes_expandkey() failed: %d\n", err);
return err;
}
aes_encrypt(&ctx, block, vec->plaintext);
err = fips_check_result(test, block, vec->ciphertext, AES_BLOCK_SIZE,
"encryption (library API)");
if (err)
return err;
aes_decrypt(&ctx, block, block);
return fips_check_result(test, block, vec->plaintext, AES_BLOCK_SIZE,
"decryption (library API)");
}
/* Test a length-preserving symmetric cipher using the crypto_skcipher API. */
static int __init __must_check
fips_test_skcipher(const struct fips_test *test)
{
const struct skcipher_testvec *vec = &test->skcipher;
struct crypto_skcipher *tfm;
struct skcipher_request *req = NULL;
u8 *message = NULL;
struct scatterlist sg;
u8 iv[MAX_IV_SIZE];
int err;
if (WARN_ON(vec->iv_size > MAX_IV_SIZE))
return -EINVAL;
tfm = crypto_alloc_skcipher(test->alg, 0, 0);
if (IS_ERR(tfm)) {
err = PTR_ERR(tfm);
pr_err("failed to allocate %s tfm: %d\n", test->alg, err);
return err;
}
err = fips_validate_alg(&crypto_skcipher_alg(tfm)->base);
if (err)
goto out;
if (crypto_skcipher_ivsize(tfm) != vec->iv_size) {
pr_err("%s has wrong IV size\n", test->alg);
err = -EINVAL;
goto out;
}
req = skcipher_request_alloc(tfm, GFP_KERNEL);
message = kmemdup(vec->plaintext, vec->message_size, GFP_KERNEL);
if (!req || !message) {
err = -ENOMEM;
goto out;
}
sg_init_one(&sg, message, vec->message_size);
skcipher_request_set_callback(req, CRYPTO_TFM_REQ_MAY_SLEEP,
NULL, NULL);
skcipher_request_set_crypt(req, &sg, &sg, vec->message_size, iv);
err = crypto_skcipher_setkey(tfm, vec->key, vec->key_size);
if (err) {
pr_err("failed to set %s key: %d\n", test->alg, err);
goto out;
}
/* Encrypt the plaintext, then verify the resulting ciphertext. */
memcpy(iv, vec->iv, vec->iv_size);
err = crypto_skcipher_encrypt(req);
if (err) {
pr_err("%s encryption failed: %d\n", test->alg, err);
goto out;
}
err = fips_check_result(test, message, vec->ciphertext,
vec->message_size, "encryption");
if (err)
goto out;
/* Decrypt the ciphertext, then verify the resulting plaintext. */
memcpy(iv, vec->iv, vec->iv_size);
err = crypto_skcipher_decrypt(req);
if (err) {
pr_err("%s decryption failed: %d\n", test->alg, err);
goto out;
}
err = fips_check_result(test, message, vec->plaintext,
vec->message_size, "decryption");
out:
kfree(message);
skcipher_request_free(req);
crypto_free_skcipher(tfm);
return err;
}
/* Test an AEAD using the crypto_aead API. */
static int __init __must_check
fips_test_aead(const struct fips_test *test)
{
const struct aead_testvec *vec = &test->aead;
const int tag_size = vec->ciphertext_size - vec->plaintext_size;
struct crypto_aead *tfm;
struct aead_request *req = NULL;
u8 *assoc = NULL;
u8 *message = NULL;
struct scatterlist sg[2];
int sg_idx = 0;
u8 iv[MAX_IV_SIZE];
int err;
if (WARN_ON(vec->iv_size > MAX_IV_SIZE))
return -EINVAL;
if (WARN_ON(vec->ciphertext_size <= vec->plaintext_size))
return -EINVAL;
tfm = crypto_alloc_aead(test->alg, 0, 0);
if (IS_ERR(tfm)) {
err = PTR_ERR(tfm);
pr_err("failed to allocate %s tfm: %d\n", test->alg, err);
return err;
}
err = fips_validate_alg(&crypto_aead_alg(tfm)->base);
if (err)
goto out;
if (crypto_aead_ivsize(tfm) != vec->iv_size) {
pr_err("%s has wrong IV size\n", test->alg);
err = -EINVAL;
goto out;
}
req = aead_request_alloc(tfm, GFP_KERNEL);
assoc = kmemdup(vec->assoc, vec->assoc_size, GFP_KERNEL);
message = kzalloc(vec->ciphertext_size, GFP_KERNEL);
if (!req || !assoc || !message) {
err = -ENOMEM;
goto out;
}
memcpy(message, vec->plaintext, vec->plaintext_size);
sg_init_table(sg, ARRAY_SIZE(sg));
if (vec->assoc_size)
sg_set_buf(&sg[sg_idx++], assoc, vec->assoc_size);
sg_set_buf(&sg[sg_idx++], message, vec->ciphertext_size);
aead_request_set_ad(req, vec->assoc_size);
aead_request_set_callback(req, CRYPTO_TFM_REQ_MAY_SLEEP, NULL, NULL);
err = crypto_aead_setkey(tfm, vec->key, vec->key_size);
if (err) {
pr_err("failed to set %s key: %d\n", test->alg, err);
goto out;
}
err = crypto_aead_setauthsize(tfm, tag_size);
if (err) {
pr_err("failed to set %s authentication tag size: %d\n",
test->alg, err);
goto out;
}
/*
* Encrypt the plaintext, then verify the resulting ciphertext (which
* includes the authentication tag).
*/
memcpy(iv, vec->iv, vec->iv_size);
aead_request_set_crypt(req, sg, sg, vec->plaintext_size, iv);
err = crypto_aead_encrypt(req);
if (err) {
pr_err("%s encryption failed: %d\n", test->alg, err);
goto out;
}
err = fips_check_result(test, message, vec->ciphertext,
vec->ciphertext_size, "encryption");
if (err)
goto out;
/*
* Decrypt the ciphertext (which includes the authentication tag), then
* verify the resulting plaintext.
*/
memcpy(iv, vec->iv, vec->iv_size);
aead_request_set_crypt(req, sg, sg, vec->ciphertext_size, iv);
err = crypto_aead_decrypt(req);
if (err) {
pr_err("%s decryption failed: %d\n", test->alg, err);
goto out;
}
err = fips_check_result(test, message, vec->plaintext,
vec->plaintext_size, "decryption");
out:
kfree(message);
kfree(assoc);
aead_request_free(req);
crypto_free_aead(tfm);
return err;
}
/*
* Test a hash algorithm using the crypto_shash API.
*
* Note that we don't need to test the crypto_ahash API too, since none of the
* hash algorithms in the FIPS module have the ASYNC flag, and thus there will
* be no hash algorithms that can be accessed only through crypto_ahash.
*/
static int __init __must_check
fips_test_hash(const struct fips_test *test)
{
const struct hash_testvec *vec = &test->hash;
struct crypto_shash *tfm;
u8 digest[HASH_MAX_DIGESTSIZE];
int err;
if (WARN_ON(vec->digest_size > HASH_MAX_DIGESTSIZE))
return -EINVAL;
tfm = crypto_alloc_shash(test->alg, 0, 0);
if (IS_ERR(tfm)) {
err = PTR_ERR(tfm);
pr_err("failed to allocate %s tfm: %d\n", test->alg, err);
return err;
}
err = fips_validate_alg(&crypto_shash_alg(tfm)->base);
if (err)
goto out;
if (crypto_shash_digestsize(tfm) != vec->digest_size) {
pr_err("%s has wrong digest size\n", test->alg);
err = -EINVAL;
goto out;
}
if (vec->key) {
err = crypto_shash_setkey(tfm, vec->key, vec->key_size);
if (err) {
pr_err("failed to set %s key: %d\n", test->alg, err);
goto out;
}
}
err = crypto_shash_tfm_digest(tfm, vec->message, vec->message_size,
digest);
if (err) {
pr_err("%s digest computation failed: %d\n", test->alg, err);
goto out;
}
err = fips_check_result(test, digest, vec->digest, vec->digest_size,
"digest");
out:
crypto_free_shash(tfm);
return err;
}
/*
* Test the sha256() library function, as it may not be covered by the "sha256"
* crypto_shash, and thus may not be covered by the "hmac(sha256)" test we do.
*/
static int __init __must_check
fips_test_sha256_library(const struct fips_test *test)
{
const struct hash_testvec *vec = &test->hash;
u8 digest[SHA256_DIGEST_SIZE];
if (WARN_ON(vec->digest_size != SHA256_DIGEST_SIZE))
return -EINVAL;
sha256(vec->message, vec->message_size, digest);
return fips_check_result(test, digest, vec->digest, vec->digest_size,
"digest (library API)");
}
/* Test a DRBG using the crypto_rng API. */
static int __init __must_check
fips_test_drbg(const struct fips_test *test)
{
const struct drbg_testvec *vec = &test->drbg;
struct crypto_rng *rng;
u8 *output = NULL;
struct drbg_test_data test_data;
struct drbg_string addtl, pers, testentropy;
int err;
rng = crypto_alloc_rng(test->alg, 0, 0);
if (IS_ERR(rng)) {
err = PTR_ERR(rng);
pr_err("failed to allocate %s tfm: %d\n", test->alg, err);
return PTR_ERR(rng);
}
err = fips_validate_alg(&crypto_rng_alg(rng)->base);
if (err)
goto out;
output = kzalloc(vec->out_size, GFP_KERNEL);
if (!output) {
err = -ENOMEM;
goto out;
}
/*
* Initialize the DRBG with the entropy and personalization string given
* in the test vector.
*/
test_data.testentropy = &testentropy;
drbg_string_fill(&testentropy, vec->entropy, vec->entropy_size);
drbg_string_fill(&pers, vec->pers, vec->pers_size);
err = crypto_drbg_reset_test(rng, &pers, &test_data);
if (err) {
pr_err("failed to reset %s\n", test->alg);
goto out;
}
/*
* Generate some random bytes using the additional data string provided
* in the test vector. Also use the additional entropy if provided
* (relevant for the prediction-resistant DRBG variants only).
*/
drbg_string_fill(&addtl, vec->add_a, vec->add_size);
if (vec->entpr_size) {
drbg_string_fill(&testentropy, vec->entpr_a, vec->entpr_size);
err = crypto_drbg_get_bytes_addtl_test(rng, output,
vec->out_size, &addtl,
&test_data);
} else {
err = crypto_drbg_get_bytes_addtl(rng, output, vec->out_size,
&addtl);
}
if (err) {
pr_err("failed to get bytes from %s (try 1): %d\n",
test->alg, err);
goto out;
}
/*
* Do the same again, using a second additional data string, and (when
* applicable) a second additional entropy string.
*/
drbg_string_fill(&addtl, vec->add_b, vec->add_size);
if (test->drbg.entpr_size) {
drbg_string_fill(&testentropy, vec->entpr_b, vec->entpr_size);
err = crypto_drbg_get_bytes_addtl_test(rng, output,
vec->out_size, &addtl,
&test_data);
} else {
err = crypto_drbg_get_bytes_addtl(rng, output, vec->out_size,
&addtl);
}
if (err) {
pr_err("failed to get bytes from %s (try 2): %d\n",
test->alg, err);
goto out;
}
/* Check that the DRBG generated the expected output. */
err = fips_check_result(test, output, vec->output, vec->out_size,
"get_bytes");
out:
kfree(output);
crypto_free_rng(rng);
return err;
}
/* Include the test vectors generated by the Python script. */
#include "fips140-generated-testvecs.h"
/* List of all self-tests. Keep this in sync with fips140_algorithms[]. */
static const struct fips_test fips140_selftests[] __initconst = {
/*
* Tests for AES and AES modes.
*
* The full list of AES algorithms we potentially need to test are AES
* by itself, AES-CBC, AES-CTR, AES-ECB, AES-GCM, and AES-XTS. We can
* follow the FIPS 140-2 Implementation Guidance (IG) document to try to
* reduce this list, but we run into the issue that the architecture-
* specific implementations of these algorithms in Linux often don't
* share the "same" underlying AES implementation. E.g., the ARMv8 CE
* optimized implementations issue ARMv8 CE instructions directly rather
* than going through a separate AES implementation. In this case,
* separate tests are needed according to section 9.2 of the IG.
*/
{
.alg = "aes",
.func = fips_test_aes,
.blockcipher = {
.key = fips_aes_key,
.key_size = sizeof(fips_aes_key),
.plaintext = fips_message,
.ciphertext = fips_aes_ecb_ciphertext,
.block_size = 16,
}
}, {
.alg = "cbc(aes)",
.func = fips_test_skcipher,
.skcipher = {
.key = fips_aes_key,
.key_size = sizeof(fips_aes_key),
.iv = fips_aes_iv,
.iv_size = sizeof(fips_aes_iv),
.plaintext = fips_message,
.ciphertext = fips_aes_cbc_ciphertext,
.message_size = sizeof(fips_message),
}
}, {
.alg = "ctr(aes)",
.func = fips_test_skcipher,
.skcipher = {
.key = fips_aes_key,
.key_size = sizeof(fips_aes_key),
.iv = fips_aes_iv,
.iv_size = sizeof(fips_aes_iv),
.plaintext = fips_message,
.ciphertext = fips_aes_ctr_ciphertext,
.message_size = sizeof(fips_message),
}
}, {
.alg = "ecb(aes)",
.func = fips_test_skcipher,
.skcipher = {
.key = fips_aes_key,
.key_size = sizeof(fips_aes_key),
.plaintext = fips_message,
.ciphertext = fips_aes_ecb_ciphertext,
.message_size = sizeof(fips_message)
}
}, {
.alg = "gcm(aes)",
.func = fips_test_aead,
.aead = {
.key = fips_aes_key,
.key_size = sizeof(fips_aes_key),
.iv = fips_aes_iv,
/* The GCM implementation assumes an IV size of 12. */
.iv_size = 12,
.assoc = fips_aes_gcm_assoc,
.assoc_size = sizeof(fips_aes_gcm_assoc),
.plaintext = fips_message,
.plaintext_size = sizeof(fips_message),
.ciphertext = fips_aes_gcm_ciphertext,
.ciphertext_size = sizeof(fips_aes_gcm_ciphertext),
}
}, {
.alg = "xts(aes)",
.func = fips_test_skcipher,
.skcipher = {
.key = fips_aes_xts_key,
.key_size = sizeof(fips_aes_xts_key),
.iv = fips_aes_iv,
.iv_size = sizeof(fips_aes_iv),
.plaintext = fips_message,
.ciphertext = fips_aes_xts_ciphertext,
.message_size = sizeof(fips_message),
}
/*
* Tests for SHA-1, SHA-256, HMAC-SHA256, and SHA-512.
*
* The selection of these specific tests follows the guidance from
* section 9 of the FIPS 140-2 Implementation Guidance (IG) document to
* achieve a minimal list of tests, rather than testing all of
* SHA-{1,224,256,384,512} and HMAC-SHA{1,224,256,384,512}. As per the
* IG, testing SHA-224 is only required if SHA-256 isn't implemented,
* and testing SHA-384 is only required if SHA-512 isn't implemented.
* Also, HMAC only has to be tested with one underlying SHA, and the
* HMAC test also fulfills the test for its underlying SHA. That would
* result in a test list of e.g. SHA-1, HMAC-SHA256, and SHA-512.
*
* However we also need to take into account cases where implementations
* aren't shared in the "natural" way assumed by the IG. Currently the
* only known exception w.r.t. SHA-* and HMAC-* is the sha256() library
* function which may not be covered by the test of the "hmac(sha256)"
* crypto_shash. So, we test sha256() separately.
*/
}, {
.alg = "sha1",
.func = fips_test_hash,
.hash = {
.message = fips_message,
.message_size = sizeof(fips_message),
.digest = fips_sha1_digest,
.digest_size = sizeof(fips_sha1_digest)
}
}, {
.alg = "sha256",
.func = fips_test_sha256_library,
.hash = {
.message = fips_message,
.message_size = sizeof(fips_message),
.digest = fips_sha256_digest,
.digest_size = sizeof(fips_sha256_digest)
}
}, {
.alg = "hmac(sha256)",
.func = fips_test_hash,
.hash = {
.key = fips_hmac_key,
.key_size = sizeof(fips_hmac_key),
.message = fips_message,
.message_size = sizeof(fips_message),
.digest = fips_hmac_sha256_digest,
.digest_size = sizeof(fips_hmac_sha256_digest)
}
}, {
.alg = "sha512",
.func = fips_test_hash,
.hash = {
.message = fips_message,
.message_size = sizeof(fips_message),
.digest = fips_sha512_digest,
.digest_size = sizeof(fips_sha512_digest)
}
/*
* Tests for DRBG algorithms.
*
* Only the default variant (the one that users get when they request
* "stdrng") is required to be tested, as we don't consider the other
* variants to be used / usable in the FIPS security policy. This is
* similar to how e.g. we don't test both "xts(aes-generic)" and
* "xts-aes-ce" but rather just "xts(aes)".
*
* Currently the default one is "drbg_nopr_hmac_sha256"; however, just
* in case we also test the prediction-resistant enabled variant too.
*/
}, {
.alg = "drbg_nopr_hmac_sha256",
.func = fips_test_drbg,
.drbg = {
.entropy =
"\xf9\x7a\x3c\xfd\x91\xfa\xa0\x46\xb9\xe6\x1b\x94"
"\x93\xd4\x36\xc4\x93\x1f\x60\x4b\x22\xf1\x08\x15"
"\x21\xb3\x41\x91\x51\xe8\xff\x06\x11\xf3\xa7\xd4"
"\x35\x95\x35\x7d\x58\x12\x0b\xd1\xe2\xdd\x8a\xed",
.entropy_size = 48,
.output =
"\xc6\x87\x1c\xff\x08\x24\xfe\x55\xea\x76\x89\xa5"
"\x22\x29\x88\x67\x30\x45\x0e\x5d\x36\x2d\xa5\xbf"
"\x59\x0d\xcf\x9a\xcd\x67\xfe\xd4\xcb\x32\x10\x7d"
"\xf5\xd0\x39\x69\xa6\x6b\x1f\x64\x94\xfd\xf5\xd6"
"\x3d\x5b\x4d\x0d\x34\xea\x73\x99\xa0\x7d\x01\x16"
"\x12\x6d\x0d\x51\x8c\x7c\x55\xba\x46\xe1\x2f\x62"
"\xef\xc8\xfe\x28\xa5\x1c\x9d\x42\x8e\x6d\x37\x1d"
"\x73\x97\xab\x31\x9f\xc7\x3d\xed\x47\x22\xe5\xb4"
"\xf3\x00\x04\x03\x2a\x61\x28\xdf\x5e\x74\x97\xec"
"\xf8\x2c\xa7\xb0\xa5\x0e\x86\x7e\xf6\x72\x8a\x4f"
"\x50\x9a\x8c\x85\x90\x87\x03\x9c",
.out_size = 128,
.add_a =
"\x51\x72\x89\xaf\xe4\x44\xa0\xfe\x5e\xd1\xa4\x1d"
"\xbb\xb5\xeb\x17\x15\x00\x79\xbd\xd3\x1e\x29\xcf"
"\x2f\xf3\x00\x34\xd8\x26\x8e\x3b",
.add_b =
"\x88\x02\x8d\x29\xef\x80\xb4\xe6\xf0\xfe\x12\xf9"
"\x1d\x74\x49\xfe\x75\x06\x26\x82\xe8\x9c\x57\x14"
"\x40\xc0\xc9\xb5\x2c\x42\xa6\xe0",
.add_size = 32,
}
}, {
.alg = "drbg_pr_hmac_sha256",
.func = fips_test_drbg,
.drbg = {
.entropy =
"\xc7\xcc\xbc\x67\x7e\x21\x66\x1e\x27\x2b\x63\xdd"
"\x3a\x78\xdc\xdf\x66\x6d\x3f\x24\xae\xcf\x37\x01"
"\xa9\x0d\x89\x8a\xa7\xdc\x81\x58\xae\xb2\x10\x15"
"\x7e\x18\x44\x6d\x13\xea\xdf\x37\x85\xfe\x81\xfb",
.entropy_size = 48,
.entpr_a =
"\x7b\xa1\x91\x5b\x3c\x04\xc4\x1b\x1d\x19\x2f\x1a"
"\x18\x81\x60\x3c\x6c\x62\x91\xb7\xe9\xf5\xcb\x96"
"\xbb\x81\x6a\xcc\xb5\xae\x55\xb6",
.entpr_b =
"\x99\x2c\xc7\x78\x7e\x3b\x88\x12\xef\xbe\xd3\xd2"
"\x7d\x2a\xa5\x86\xda\x8d\x58\x73\x4a\x0a\xb2\x2e"
"\xbb\x4c\x7e\xe3\x9a\xb6\x81\xc1",
.entpr_size = 32,
.output =
"\x95\x6f\x95\xfc\x3b\xb7\xfe\x3e\xd0\x4e\x1a\x14"
"\x6c\x34\x7f\x7b\x1d\x0d\x63\x5e\x48\x9c\x69\xe6"
"\x46\x07\xd2\x87\xf3\x86\x52\x3d\x98\x27\x5e\xd7"
"\x54\xe7\x75\x50\x4f\xfb\x4d\xfd\xac\x2f\x4b\x77"
"\xcf\x9e\x8e\xcc\x16\xa2\x24\xcd\x53\xde\x3e\xc5"
"\x55\x5d\xd5\x26\x3f\x89\xdf\xca\x8b\x4e\x1e\xb6"
"\x88\x78\x63\x5c\xa2\x63\x98\x4e\x6f\x25\x59\xb1"
"\x5f\x2b\x23\xb0\x4b\xa5\x18\x5d\xc2\x15\x74\x40"
"\x59\x4c\xb4\x1e\xcf\x9a\x36\xfd\x43\xe2\x03\xb8"
"\x59\x91\x30\x89\x2a\xc8\x5a\x43\x23\x7c\x73\x72"
"\xda\x3f\xad\x2b\xba\x00\x6b\xd1",
.out_size = 128,
.add_a =
"\x18\xe8\x17\xff\xef\x39\xc7\x41\x5c\x73\x03\x03"
"\xf6\x3d\xe8\x5f\xc8\xab\xe4\xab\x0f\xad\xe8\xd6"
"\x86\x88\x55\x28\xc1\x69\xdd\x76",
.add_b =
"\xac\x07\xfc\xbe\x87\x0e\xd3\xea\x1f\x7e\xb8\xe7"
"\x9d\xec\xe8\xe7\xbc\xf3\x18\x25\x77\x35\x4a\xaa"
"\x00\x99\x2a\xdd\x0a\x00\x50\x82",
.add_size = 32,
.pers =
"\xbc\x55\xab\x3c\xf6\x52\xb0\x11\x3d\x7b\x90\xb8"
"\x24\xc9\x26\x4e\x5a\x1e\x77\x0d\x3d\x58\x4a\xda"
"\xd1\x81\xe9\xf8\xeb\x30\x8f\x6f",
.pers_size = 32,
}
}
};
bool __init fips140_run_selftests(void)
{
int i;
pr_info("running self-tests\n");
for (i = 0; i < ARRAY_SIZE(fips140_selftests); i++) {
const struct fips_test *test = &fips140_selftests[i];
int err;
err = test->func(test);
if (err) {
pr_emerg("self-tests failed for algorithm %s: %d\n",
test->alg, err);
/* The caller is responsible for calling panic(). */
return false;
}
}
pr_info("all self-tests passed\n");
return true;
}

22
drivers/android/binder.c
View File

@@ -738,7 +738,7 @@ static void binder_transaction_priority(struct task_struct *task,
t->saved_priority.sched_policy = task->policy;
t->saved_priority.prio = task->normal_prio;
trace_android_vh_binder_transaction_priority_skip(task, &skip);
trace_android_vh_binder_priority_skip(task, &skip);
if (skip)
return;
@@ -2851,9 +2851,28 @@ static void binder_transaction(struct binder_proc *proc,
if (target_node && target_node->txn_security_ctx) {
u32 secid;
size_t added_size;
int max_retries = 100;
security_task_getsecid(proc->tsk, &secid);
retry_alloc:
ret = security_secid_to_secctx(secid, &secctx, &secctx_sz);
if (ret == -ENOMEM && max_retries-- > 0) {
struct page *dummy_page;
/*
* security_secid_to_secctx() can fail because of a
* GFP_ATOMIC allocation in which case -ENOMEM is
* returned. This needs to be retried, but there is
* currently no way to tell userspace to retry so we
* do it here. We make sure there is still available
* memory first and then retry.
*/
dummy_page = alloc_page(GFP_KERNEL);
if (dummy_page) {
__free_page(dummy_page);
goto retry_alloc;
}
}
if (ret) {
return_error = BR_FAILED_REPLY;
return_error_param = ret;
@@ -5481,6 +5500,7 @@ static void print_binder_transaction_ilocked(struct seq_file *m,
struct binder_buffer *buffer = t->buffer;
spin_lock(&t->lock);
trace_android_vh_binder_print_transaction_info(m, proc, prefix, t);
to_proc = t->to_proc;
seq_printf(m,
"%s %d: %pK from %d:%d to %d:%d code %x flags %x pri %d:%d r%d",

1
drivers/android/binder_internal.h
View File

@@ -572,6 +572,7 @@ struct binder_transaction {
*/
spinlock_t lock;
ANDROID_VENDOR_DATA(1);
ANDROID_OEM_DATA_ARRAY(1, 2);
};
/**

9
drivers/android/debug_symbols.c
View File

@@ -16,6 +16,8 @@
#include <linux/memblock.h>
#include <linux/page_owner.h>
#include <linux/swap.h>
#include <linux/mm.h>
#include <linux/security.h>
struct ads_entry {
char *name;
@@ -59,6 +61,13 @@ static const struct ads_entry ads_entries[ADS_END] = {
#ifdef CONFIG_SWAP
ADS_ENTRY(ADS_NR_SWAP_PAGES, &nr_swap_pages),
#endif
#ifdef CONFIG_MMU
ADS_ENTRY(ADS_MMAP_MIN_ADDR, &mmap_min_addr),
#endif
ADS_ENTRY(ADS_STACK_GUARD_GAP, &stack_guard_gap),
#ifdef CONFIG_SYSCTL
ADS_ENTRY(ADS_SYSCTL_LEGACY_VA_LAYOUT, &sysctl_legacy_va_layout),
#endif
};
/*

50
drivers/android/vendor_hooks.c
View File

@@ -40,6 +40,7 @@
#include <trace/hooks/iommu.h>
#include <trace/hooks/thermal.h>
#include <trace/hooks/ufshcd.h>
#include <trace/hooks/block.h>
#include <trace/hooks/cgroup.h>
#include <trace/hooks/workqueue.h>
#include <trace/hooks/sys.h>
@@ -63,6 +64,14 @@
#include <trace/hooks/user.h>
#include <trace/hooks/cpuidle_psci.h>
#include <trace/hooks/fips140.h>
#include <trace/hooks/remoteproc.h>
#include <trace/hooks/shmem_fs.h>
#include <trace/hooks/net.h>
#include <trace/hooks/syscall_check.h>
#include <trace/hooks/usb.h>
#include <trace/hooks/ipv6.h>
#include <trace/hooks/sound.h>
#include <trace/hooks/snd_compr.h>
/*
* Export tracepoints that act as a bare tracehook (ie: have no trace event
@@ -82,10 +91,14 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_prepare_prio_fork);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_finish_prio_fork);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_set_user_nice);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_setscheduler);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_sk_alloc);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_sk_free);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_nf_conn_alloc);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_nf_conn_free);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_arch_set_freq_scale);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_is_fpsimd_save);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_binder_transaction_init);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_binder_transaction_priority_skip);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_binder_priority_skip);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_binder_set_priority);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_binder_restore_priority);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_binder_wakeup_ilocked);
@@ -105,6 +118,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_rwsem_read_wait_finish);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_rwsem_write_wait_start);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_rwsem_write_wait_finish);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_sched_show_task);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_shmem_alloc_page);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_cpu_idle_enter);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_cpu_idle_exit);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_mpam_set);
@@ -193,6 +207,8 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_cma_alloc_start);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_cma_alloc_finish);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_rmqueue);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_pagecache_get_page);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_filemap_fault_get_page);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_filemap_fault_cache_page);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_enable_thermal_genl_check);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_thermal_pm_notify_suspend);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_ufs_fill_prdt);
@@ -204,6 +220,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_ufs_send_uic_command);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_ufs_send_tm_command);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_ufs_check_int_errors);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_ufs_update_sdev);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_ufs_clock_scaling);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_do_wake_up_sync);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_set_wake_flags);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_uclamp_eff_get);
@@ -228,6 +245,8 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_selinux_avc_insert);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_selinux_avc_node_delete);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_selinux_avc_node_replace);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_selinux_avc_lookup);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_blk_alloc_rqs);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_blk_rq_ctx_init);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_commit_creds);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_exit_creds);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_override_creds);
@@ -241,9 +260,9 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_set_module_permit_after_init);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_util_est_update);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_meminfo_proc_show);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_exit_mm);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_get_unmapped_area_from_anti_fragment_pool);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_get_from_fragment_pool);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_exclude_reserved_zone);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_get_unmapped_area_include_reserved_zone);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_include_reserved_zone);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_alloc_pages_slowpath);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_show_mem);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_print_slabinfo_header);
@@ -269,6 +288,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_post_init_entity_util_avg);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_typec_tcpm_get_timer);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_typec_tcpm_adj_current_limit);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_logbuf);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_logbuf_pr_cont);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_tune_scan_type);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_tune_swappiness);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_shrink_slab_bypass);
@@ -306,6 +326,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_mmc_gpio_cd_irqt);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_save_vmalloc_stack);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_show_stack_hash);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_save_track_hash);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_vmpressure);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_set_task_comm);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_cpufreq_acct_update_power);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_typec_tcpm_log);
@@ -313,12 +334,16 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_media_device_setup_link);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_clear_reserved_fmt_fields);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_fill_ext_fmtdesc);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_clear_mask_adjust);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_v4l2subdev_set_selection);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_v4l2subdev_set_fmt);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_v4l2subdev_set_frame_interval);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_scmi_timeout_sync);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_find_new_ilb);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_force_compatible_pre);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_force_compatible_post);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_alloc_uid);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_free_user);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_freq_qos_add_request);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_freq_qos_update_request);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_freq_qos_remove_request);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_set_balance_anon_file_reclaim);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_cpuidle_psci_enter);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_cpuidle_psci_exit);
@@ -326,3 +351,18 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_sha256);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_aes_expandkey);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_aes_encrypt);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_aes_decrypt);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_force_compatible_pre);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_rvh_force_compatible_post);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_binder_print_transaction_info);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_setscheduler_uclamp);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_rproc_recovery);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_ptype_head);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_kfree_skb);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_check_mmap_file);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_check_file_open);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_check_bpf_syscall);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_usb_dev_suspend);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_usb_dev_resume);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_ipv6_gen_linklocal_addr);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_sound_usb_support_cpu_suspend);
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_snd_compr_use_pause_in_drain);

12
drivers/base/power/main.c
View File

@@ -1234,8 +1234,8 @@ Run:
error = dpm_run_callback(callback, dev, state, info);
if (error) {
async_error = error;
log_suspend_abort_reason("Callback failed on %s in %pS returned %d",
dev_name(dev), callback, error);
log_suspend_abort_reason("Device %s failed to %s noirq: error %d",
dev_name(dev), pm_verb(state.event), error);
goto Complete;
}
@@ -1428,8 +1428,8 @@ Run:
error = dpm_run_callback(callback, dev, state, info);
if (error) {
async_error = error;
log_suspend_abort_reason("Callback failed on %s in %pS returned %d",
dev_name(dev), callback, error);
log_suspend_abort_reason("Device %s failed to %s late: error %d",
dev_name(dev), pm_verb(state.event), error);
goto Complete;
}
dpm_propagate_wakeup_to_parent(dev);
@@ -1701,8 +1701,8 @@ static int __device_suspend(struct device *dev, pm_message_t state, bool async)
dpm_propagate_wakeup_to_parent(dev);
dpm_clear_superiors_direct_complete(dev);
} else {
log_suspend_abort_reason("Callback failed on %s in %pS returned %d",
dev_name(dev), callback, error);
log_suspend_abort_reason("Device %s failed to %s: error %d",
dev_name(dev), pm_verb(state.event), error);
}
device_unlock(dev);

5
drivers/cpuidle/cpuidle.c
View File

@@ -212,6 +212,9 @@ int cpuidle_enter_state(struct cpuidle_device *dev, struct cpuidle_driver *drv,
* broadcast must be assigned after the vendor hook.
*/
trace_android_vh_cpu_idle_enter(&index, dev);
if (index < 0)
return index;
target_state = &drv->states[index];
broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
@@ -428,7 +431,7 @@ void cpuidle_uninstall_idle_handler(void)
{
if (enabled_devices) {
initialized = 0;
wake_up_all_idle_cpus();
wake_up_all_online_idle_cpus();
}
/*

139
drivers/dma-buf/dma-buf-sysfs-stats.c
View File

@@ -52,13 +52,6 @@ static ssize_t exporter_name_show(struct dma_buf *dmabuf,
return sysfs_emit(buf, "%s\n", dmabuf->exp_name);
}
static ssize_t mmap_count_show(struct dma_buf *dmabuf,
struct dma_buf_stats_attribute *attr,
char *buf)
{
return sysfs_emit(buf, "%d\n", dmabuf->mmap_count);
}
static ssize_t size_show(struct dma_buf *dmabuf,
struct dma_buf_stats_attribute *attr,
char *buf)
@@ -69,13 +62,10 @@ static ssize_t size_show(struct dma_buf *dmabuf,
static struct dma_buf_stats_attribute exporter_name_attribute =
__ATTR_RO(exporter_name);
static struct dma_buf_stats_attribute size_attribute = __ATTR_RO(size);
static struct dma_buf_stats_attribute mmap_count_attribute =
__ATTR_RO(mmap_count);
static struct attribute *dma_buf_stats_default_attrs[] = {
&exporter_name_attribute.attr,
&size_attribute.attr,
&mmap_count_attribute.attr,
NULL,
};
ATTRIBUTE_GROUPS(dma_buf_stats_default);
@@ -94,120 +84,6 @@ static struct kobj_type dma_buf_ktype = {
.default_groups = dma_buf_stats_default_groups,
};
#define to_dma_buf_attach_entry_from_kobj(x) container_of(x, struct dma_buf_attach_sysfs_entry, kobj)
struct dma_buf_attach_stats_attribute {
struct attribute attr;
ssize_t (*show)(struct dma_buf_attach_sysfs_entry *sysfs_entry,
struct dma_buf_attach_stats_attribute *attr, char *buf);
};
#define to_dma_buf_attach_stats_attr(x) container_of(x, struct dma_buf_attach_stats_attribute, attr)
static ssize_t dma_buf_attach_stats_attribute_show(struct kobject *kobj,
struct attribute *attr,
char *buf)
{
struct dma_buf_attach_stats_attribute *attribute;
struct dma_buf_attach_sysfs_entry *sysfs_entry;
attribute = to_dma_buf_attach_stats_attr(attr);
sysfs_entry = to_dma_buf_attach_entry_from_kobj(kobj);
if (!attribute->show)
return -EIO;
return attribute->show(sysfs_entry, attribute, buf);
}
static const struct sysfs_ops dma_buf_attach_stats_sysfs_ops = {
.show = dma_buf_attach_stats_attribute_show,
};
static ssize_t map_counter_show(struct dma_buf_attach_sysfs_entry *sysfs_entry,
struct dma_buf_attach_stats_attribute *attr,
char *buf)
{
return sysfs_emit(buf, "%u\n", sysfs_entry->map_counter);
}
static struct dma_buf_attach_stats_attribute map_counter_attribute =
__ATTR_RO(map_counter);
static struct attribute *dma_buf_attach_stats_default_attrs[] = {
&map_counter_attribute.attr,
NULL,
};
ATTRIBUTE_GROUPS(dma_buf_attach_stats_default);
static void dma_buf_attach_sysfs_release(struct kobject *kobj)
{
struct dma_buf_attach_sysfs_entry *sysfs_entry;
sysfs_entry = to_dma_buf_attach_entry_from_kobj(kobj);
kfree(sysfs_entry);
}
static struct kobj_type dma_buf_attach_ktype = {
.sysfs_ops = &dma_buf_attach_stats_sysfs_ops,
.release = dma_buf_attach_sysfs_release,
.default_groups = dma_buf_attach_stats_default_groups,
};
void dma_buf_attach_stats_teardown(struct dma_buf_attachment *attach)
{
struct dma_buf_attach_sysfs_entry *sysfs_entry;
sysfs_entry = attach->sysfs_entry;
if (!sysfs_entry)
return;
sysfs_delete_link(&sysfs_entry->kobj, &attach->dev->kobj, "device");
kobject_del(&sysfs_entry->kobj);
kobject_put(&sysfs_entry->kobj);
}
int dma_buf_attach_stats_setup(struct dma_buf_attachment *attach,
unsigned int uid)
{
struct dma_buf_attach_sysfs_entry *sysfs_entry;
int ret;
struct dma_buf *dmabuf;
if (!attach)
return -EINVAL;
dmabuf = attach->dmabuf;
sysfs_entry = kzalloc(sizeof(struct dma_buf_attach_sysfs_entry),
GFP_KERNEL);
if (!sysfs_entry)
return -ENOMEM;
sysfs_entry->kobj.kset = dmabuf->sysfs_entry->attach_stats_kset;
attach->sysfs_entry = sysfs_entry;
ret = kobject_init_and_add(&sysfs_entry->kobj, &dma_buf_attach_ktype,
NULL, "%u", uid);
if (ret)
goto kobj_err;
ret = sysfs_create_link(&sysfs_entry->kobj, &attach->dev->kobj,
"device");
if (ret)
goto link_err;
return 0;
link_err:
kobject_del(&sysfs_entry->kobj);
kobj_err:
kobject_put(&sysfs_entry->kobj);
attach->sysfs_entry = NULL;
return ret;
}
void dma_buf_stats_teardown(struct dma_buf *dmabuf)
{
struct dma_buf_sysfs_entry *sysfs_entry;
@@ -216,7 +92,6 @@ void dma_buf_stats_teardown(struct dma_buf *dmabuf)
if (!sysfs_entry)
return;
kset_unregister(sysfs_entry->attach_stats_kset);
kobject_del(&sysfs_entry->kobj);
kobject_put(&sysfs_entry->kobj);
}
@@ -264,7 +139,6 @@ int dma_buf_stats_setup(struct dma_buf *dmabuf)
{
struct dma_buf_sysfs_entry *sysfs_entry;
int ret;
struct kset *attach_stats_kset;
if (!dmabuf || !dmabuf->file)
return -EINVAL;
@@ -289,21 +163,8 @@ int dma_buf_stats_setup(struct dma_buf *dmabuf)
if (ret)
goto err_sysfs_dmabuf;
/* create the directory for attachment stats */
attach_stats_kset = kset_create_and_add("attachments",
&dmabuf_sysfs_no_uevent_ops,
&sysfs_entry->kobj);
if (!attach_stats_kset) {
ret = -ENOMEM;
goto err_sysfs_attach;
}
sysfs_entry->attach_stats_kset = attach_stats_kset;
return 0;
err_sysfs_attach:
kobject_del(&sysfs_entry->kobj);
err_sysfs_dmabuf:
kobject_put(&sysfs_entry->kobj);
dmabuf->sysfs_entry = NULL;

27
drivers/dma-buf/dma-buf-sysfs-stats.h
View File

@@ -14,23 +14,8 @@ int dma_buf_init_sysfs_statistics(void);
void dma_buf_uninit_sysfs_statistics(void);
int dma_buf_stats_setup(struct dma_buf *dmabuf);
int dma_buf_attach_stats_setup(struct dma_buf_attachment *attach,
unsigned int uid);
static inline void dma_buf_update_attachment_map_count(struct dma_buf_attachment *attach,
int delta)
{
struct dma_buf_attach_sysfs_entry *entry = attach->sysfs_entry;
entry->map_counter += delta;
}
void dma_buf_stats_teardown(struct dma_buf *dmabuf);
void dma_buf_attach_stats_teardown(struct dma_buf_attachment *attach);
static inline unsigned int dma_buf_update_attach_uid(struct dma_buf *dmabuf)
{
struct dma_buf_sysfs_entry *entry = dmabuf->sysfs_entry;
return entry->attachment_uid++;
}
#else
static inline int dma_buf_init_sysfs_statistics(void)
@@ -44,19 +29,7 @@ static inline int dma_buf_stats_setup(struct dma_buf *dmabuf)
{
return 0;
}
static inline int dma_buf_attach_stats_setup(struct dma_buf_attachment *attach,
unsigned int uid)
{
return 0;
}
static inline void dma_buf_stats_teardown(struct dma_buf *dmabuf) {}
static inline void dma_buf_attach_stats_teardown(struct dma_buf_attachment *attach) {}
static inline void dma_buf_update_attachment_map_count(struct dma_buf_attachment *attach,
int delta) {}
static inline unsigned int dma_buf_update_attach_uid(struct dma_buf *dmabuf)
{
return 0;
}
#endif
#endif // _DMA_BUF_SYSFS_STATS_H

67
drivers/dma-buf/dma-buf.c
View File

@@ -98,12 +98,12 @@ static void dma_buf_release(struct dentry *dentry)
*/
BUG_ON(dmabuf->cb_shared.active || dmabuf->cb_excl.active);
dma_buf_stats_teardown(dmabuf);
dmabuf->ops->release(dmabuf);
if (dmabuf->resv == (struct dma_resv *)&dmabuf[1])
dma_resv_fini(dmabuf->resv);
dma_buf_stats_teardown(dmabuf);
module_put(dmabuf->owner);
kfree(dmabuf->name);
kfree(dmabuf);
@@ -149,54 +149,6 @@ static struct file_system_type dma_buf_fs_type = {
.kill_sb = kill_anon_super,
};
#ifdef CONFIG_DMABUF_SYSFS_STATS
static void dma_buf_vma_open(struct vm_area_struct *vma)
{
struct dma_buf *dmabuf = vma->vm_file->private_data;
dmabuf->mmap_count++;
/* call the heap provided vma open() op */
if (dmabuf->exp_vm_ops->open)
dmabuf->exp_vm_ops->open(vma);
}
static void dma_buf_vma_close(struct vm_area_struct *vma)
{
struct dma_buf *dmabuf = vma->vm_file->private_data;
if (dmabuf->mmap_count)
dmabuf->mmap_count--;
/* call the heap provided vma close() op */
if (dmabuf->exp_vm_ops->close)
dmabuf->exp_vm_ops->close(vma);
}
static int dma_buf_do_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
{
/* call this first because the exporter might override vma->vm_ops */
int ret = dmabuf->ops->mmap(dmabuf, vma);
if (ret)
return ret;
/* save the exporter provided vm_ops */
dmabuf->exp_vm_ops = vma->vm_ops;
dmabuf->vm_ops = *(dmabuf->exp_vm_ops);
/* override open() and close() to provide buffer mmap count */
dmabuf->vm_ops.open = dma_buf_vma_open;
dmabuf->vm_ops.close = dma_buf_vma_close;
vma->vm_ops = &dmabuf->vm_ops;
dmabuf->mmap_count++;
return ret;
}
#else /* CONFIG_DMABUF_SYSFS_STATS */
static int dma_buf_do_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
{
return dmabuf->ops->mmap(dmabuf, vma);
}
#endif /* CONFIG_DMABUF_SYSFS_STATS */
static int dma_buf_mmap_internal(struct file *file, struct vm_area_struct *vma)
{
struct dma_buf *dmabuf;
@@ -215,7 +167,7 @@ static int dma_buf_mmap_internal(struct file *file, struct vm_area_struct *vma)
dmabuf->size >> PAGE_SHIFT)
return -EINVAL;
return dma_buf_do_mmap(dmabuf, vma);
return dmabuf->ops->mmap(dmabuf, vma);
}
static loff_t dma_buf_llseek(struct file *file, loff_t offset, int whence)
@@ -778,7 +730,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
{
struct dma_buf_attachment *attach;
int ret;
unsigned int attach_uid;
if (WARN_ON(!dmabuf || !dev))
return ERR_PTR(-EINVAL);
@@ -804,13 +755,8 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
}
dma_resv_lock(dmabuf->resv, NULL);
list_add(&attach->node, &dmabuf->attachments);
attach_uid = dma_buf_update_attach_uid(dmabuf);
dma_resv_unlock(dmabuf->resv);
ret = dma_buf_attach_stats_setup(attach, attach_uid);
if (ret)
goto err_sysfs;
/* When either the importer or the exporter can't handle dynamic
* mappings we cache the mapping here to avoid issues with the
* reservation object lock.
@@ -837,7 +783,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
dma_resv_unlock(attach->dmabuf->resv);
attach->sgt = sgt;
attach->dir = DMA_BIDIRECTIONAL;
dma_buf_update_attachment_map_count(attach, 1 /* delta */);
}
return attach;
@@ -854,7 +799,6 @@ err_unlock:
if (dma_buf_is_dynamic(attach->dmabuf))
dma_resv_unlock(attach->dmabuf->resv);
err_sysfs:
dma_buf_detach(dmabuf, attach);
return ERR_PTR(ret);
}
@@ -893,7 +837,6 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
dma_resv_lock(attach->dmabuf->resv, NULL);
dmabuf->ops->unmap_dma_buf(attach, attach->sgt, attach->dir);
dma_buf_update_attachment_map_count(attach, -1 /* delta */);
if (dma_buf_is_dynamic(attach->dmabuf)) {
dma_buf_unpin(attach);
@@ -907,7 +850,6 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
if (dmabuf->ops->detach)
dmabuf->ops->detach(dmabuf, attach);
dma_buf_attach_stats_teardown(attach);
kfree(attach);
}
EXPORT_SYMBOL_GPL(dma_buf_detach);
@@ -1013,9 +955,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
attach->dir = direction;
}
if (!IS_ERR(sg_table))
dma_buf_update_attachment_map_count(attach, 1 /* delta */);
return sg_table;
}
EXPORT_SYMBOL_GPL(dma_buf_map_attachment);
@@ -1053,8 +992,6 @@ void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
if (dma_buf_is_dynamic(attach->dmabuf) &&
!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY))
dma_buf_unpin(attach);
dma_buf_update_attachment_map_count(attach, -1 /* delta */);
}
EXPORT_SYMBOL_GPL(dma_buf_unmap_attachment);

6
drivers/dma-buf/heaps/cma_heap.c
View File

@@ -99,9 +99,10 @@ static struct sg_table *cma_heap_map_dma_buf(struct dma_buf_attachment *attachme
{
struct dma_heap_attachment *a = attachment->priv;
struct sg_table *table = &a->table;
int attrs = attachment->dma_map_attrs;
int ret;
ret = dma_map_sgtable(attachment->dev, table, direction, 0);
ret = dma_map_sgtable(attachment->dev, table, direction, attrs);
if (ret)
return ERR_PTR(-ENOMEM);
a->mapped = true;
@@ -113,9 +114,10 @@ static void cma_heap_unmap_dma_buf(struct dma_buf_attachment *attachment,
enum dma_data_direction direction)
{
struct dma_heap_attachment *a = attachment->priv;
int attrs = attachment->dma_map_attrs;
a->mapped = false;
dma_unmap_sgtable(attachment->dev, table, direction, 0);
dma_unmap_sgtable(attachment->dev, table, direction, attrs);
}
static int cma_heap_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,

3
drivers/dma-buf/heaps/deferred-free-helper.c
View File

@@ -62,7 +62,7 @@ static size_t free_one_item(enum df_reason reason)
return nr_pages;
}
static unsigned long get_freelist_nr_pages(void)
unsigned long get_freelist_nr_pages(void)
{
unsigned long nr_pages;
unsigned long flags;
@@ -72,6 +72,7 @@ static unsigned long get_freelist_nr_pages(void)
spin_unlock_irqrestore(&free_list_lock, flags);
return nr_pages;
}
EXPORT_SYMBOL_GPL(get_freelist_nr_pages);
static unsigned long freelist_shrink_count(struct shrinker *shrinker,
struct shrink_control *sc)

2
drivers/dma-buf/heaps/deferred-free-helper.h
View File

@@ -52,4 +52,6 @@ void deferred_free(struct deferred_freelist_item *item,
void (*free)(struct deferred_freelist_item *i,
enum df_reason reason),
size_t nr_pages);
unsigned long get_freelist_nr_pages(void);
#endif

8
drivers/dma-buf/heaps/system_heap.c
View File

@@ -140,11 +140,11 @@ static struct sg_table *system_heap_map_dma_buf(struct dma_buf_attachment *attac
{
struct dma_heap_attachment *a = attachment->priv;
struct sg_table *table = a->table;
int attr = 0;
int attr = attachment->dma_map_attrs;
int ret;
if (a->uncached)
attr = DMA_ATTR_SKIP_CPU_SYNC;
attr |= DMA_ATTR_SKIP_CPU_SYNC;
ret = dma_map_sgtable(attachment->dev, table, direction, attr);
if (ret)
@@ -159,10 +159,10 @@ static void system_heap_unmap_dma_buf(struct dma_buf_attachment *attachment,
enum dma_data_direction direction)
{
struct dma_heap_attachment *a = attachment->priv;
int attr = 0;
int attr = attachment->dma_map_attrs;
if (a->uncached)
attr = DMA_ATTR_SKIP_CPU_SYNC;
attr |= DMA_ATTR_SKIP_CPU_SYNC;
a->mapped = false;
dma_unmap_sgtable(attachment->dev, table, direction, attr);
}

2
drivers/hwtracing/coresight/coresight-core.c
View File

@@ -23,7 +23,7 @@
#include "coresight-priv.h"
static DEFINE_MUTEX(coresight_mutex);
DEFINE_PER_CPU(struct coresight_device *, csdev_sink);
static DEFINE_PER_CPU(struct coresight_device *, csdev_sink);
/**
* struct coresight_node - elements of a path, from source to sink

4
drivers/hwtracing/coresight/coresight-etm4x-core.c
View File

@@ -115,7 +115,7 @@ void etm4x_sysreg_write(u64 val, u32 offset, bool _relaxed, bool _64bit)
}
}
u64 ete_sysreg_read(u32 offset, bool _relaxed, bool _64bit)
static u64 ete_sysreg_read(u32 offset, bool _relaxed, bool _64bit)
{
u64 res = 0;
@@ -132,7 +132,7 @@ u64 ete_sysreg_read(u32 offset, bool _relaxed, bool _64bit)
return res;
}
void ete_sysreg_write(u64 val, u32 offset, bool _relaxed, bool _64bit)
static void ete_sysreg_write(u64 val, u32 offset, bool _relaxed, bool _64bit)
{
if (!_relaxed)
__iowmb(); /* Imitate the !relaxed I/O helpers */

18
drivers/hwtracing/coresight/coresight-trbe.c
View File

@@ -5,6 +5,8 @@
* device (ETE) thus generating required trace data. Trace can be enabled
* via the perf framework.
*
* The AUX buffer handling is inspired from Arm SPE PMU driver.
*
* Copyright (C) 2020 ARM Ltd.
*
* Author: Anshuman Khandual <anshuman.khandual@arm.com>
@@ -515,7 +517,7 @@ static void *arm_trbe_alloc_buffer(struct coresight_device *csdev,
if (!buf->trbe_base) {
kfree(pglist);
kfree(buf);
return ERR_PTR(buf->trbe_base);
return ERR_PTR(-ENOMEM);
}
buf->trbe_limit = buf->trbe_base + nr_pages * PAGE_SIZE;
buf->trbe_write = buf->trbe_base;
@@ -614,9 +616,10 @@ static unsigned long arm_trbe_update_buffer(struct coresight_device *csdev,
/*
* Otherwise, the buffer is full and the write pointer
* has reached base. Adjust this back to the Limit pointer
* for correct size.
* for correct size. Also, mark the buffer truncated.
*/
write = get_trbe_limit_pointer();
perf_aux_output_flag(handle, PERF_AUX_FLAG_TRUNCATED);
}
offset = write - base;
@@ -703,7 +706,12 @@ static void trbe_handle_overflow(struct perf_output_handle *handle)
if (buf->snapshot)
handle->head += size;
perf_aux_output_flag(handle, PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW);
/*
* Mark the buffer as truncated, as we have stopped the trace
* collection upon the WRAP event, without stopping the source.
*/
perf_aux_output_flag(handle, PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW |
PERF_AUX_FLAG_TRUNCATED);
perf_aux_output_end(handle, size);
event_data = perf_aux_output_begin(handle, event);
if (!event_data) {
@@ -863,7 +871,7 @@ static void arm_trbe_register_coresight_cpu(struct trbe_drvdata *drvdata, int cp
dev = &cpudata->drvdata->pdev->dev;
desc.name = devm_kasprintf(dev, GFP_KERNEL, "trbe%d", cpu);
if (IS_ERR(desc.name))
if (!desc.name)
goto cpu_clear;
desc.type = CORESIGHT_DEV_TYPE_SINK;
@@ -1038,7 +1046,7 @@ static int arm_trbe_probe_irq(struct platform_device *pdev,
if (irq_get_percpu_devid_partition(drvdata->irq, &drvdata->supported_cpus))
return -EINVAL;
drvdata->handle = alloc_percpu(typeof(*drvdata->handle));
drvdata->handle = alloc_percpu(struct perf_output_handle *);
if (!drvdata->handle)
return -ENOMEM;

3
drivers/hwtracing/coresight/coresight-trbe.h
View File

@@ -128,8 +128,7 @@ static inline void set_trbe_write_pointer(unsigned long addr)
static inline unsigned long get_trbe_limit_pointer(void)
{
u64 trblimitr = read_sysreg_s(SYS_TRBLIMITR_EL1);
unsigned long limit = (trblimitr >> TRBLIMITR_LIMIT_SHIFT) & TRBLIMITR_LIMIT_MASK;
unsigned long addr = limit << TRBLIMITR_LIMIT_SHIFT;
unsigned long addr = trblimitr & (TRBLIMITR_LIMIT_MASK << TRBLIMITR_LIMIT_SHIFT);
WARN_ON(!IS_ALIGNED(addr, PAGE_SIZE));
return addr;

6
drivers/iommu/dma-iommu.c
View File

@@ -436,9 +436,9 @@ static int dma_info_to_prot(enum dma_data_direction dir, bool coherent,
if (attrs & DMA_ATTR_PRIVILEGED)
prot |= IOMMU_PRIV;
if (attrs & DMA_ATTR_SYS_CACHE_ONLY)
prot |= IOMMU_SYS_CACHE_ONLY;
prot |= IOMMU_SYS_CACHE;
if (attrs & DMA_ATTR_SYS_CACHE_ONLY_NWA)
prot |= IOMMU_SYS_CACHE_ONLY_NWA;
prot |= IOMMU_SYS_CACHE_NWA;
switch (dir) {
case DMA_BIDIRECTIONAL:
@@ -489,7 +489,7 @@ static dma_addr_t iommu_dma_alloc_iova(struct iommu_domain *domain,
iova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift,
true);
trace_android_vh_iommu_alloc_iova(dev, iova, size);
trace_android_vh_iommu_alloc_iova(dev, (dma_addr_t)iova << shift, size);
return (dma_addr_t)iova << shift;
}

38
drivers/iommu/io-pgtable-arm.c
View File

@@ -112,18 +112,20 @@
#define ARM_LPAE_VTCR_PS_SHIFT 16
#define ARM_LPAE_VTCR_PS_MASK 0x7
#define ARM_LPAE_MAIR_ATTR_SHIFT(n) ((n) << 3)
#define ARM_LPAE_MAIR_ATTR_MASK 0xff
#define ARM_LPAE_MAIR_ATTR_DEVICE 0x04ULL
#define ARM_LPAE_MAIR_ATTR_NC 0x44ULL
#define ARM_LPAE_MAIR_ATTR_INC_OWBRANWA 0xe4ULL
#define ARM_LPAE_MAIR_ATTR_INC_OWBRWA 0xf4ULL
#define ARM_LPAE_MAIR_ATTR_WBRWA 0xffULL
#define ARM_LPAE_MAIR_ATTR_IDX_NC 0
#define ARM_LPAE_MAIR_ATTR_IDX_CACHE 1
#define ARM_LPAE_MAIR_ATTR_IDX_DEV 2
#define ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE 3
#define ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE_NWA 4
#define ARM_LPAE_MAIR_ATTR_SHIFT(n) ((n) << 3)
#define ARM_LPAE_MAIR_ATTR_MASK 0xff
#define ARM_LPAE_MAIR_ATTR_DEVICE 0x04ULL
#define ARM_LPAE_MAIR_ATTR_NC 0x44ULL
#define ARM_LPAE_MAIR_ATTR_INC_OWBRANWA 0xe4ULL
#define ARM_LPAE_MAIR_ATTR_IWBRWA_OWBRANWA 0xefULL
#define ARM_LPAE_MAIR_ATTR_INC_OWBRWA 0xf4ULL
#define ARM_LPAE_MAIR_ATTR_WBRWA 0xffULL
#define ARM_LPAE_MAIR_ATTR_IDX_NC 0
#define ARM_LPAE_MAIR_ATTR_IDX_CACHE 1
#define ARM_LPAE_MAIR_ATTR_IDX_DEV 2
#define ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE 3
#define ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE_NWA 4
#define ARM_LPAE_MAIR_ATTR_IDX_ICACHE_OCACHE_NWA 5
#define ARM_MALI_LPAE_TTBR_ADRMODE_TABLE (3u << 0)
#define ARM_MALI_LPAE_TTBR_READ_INNER BIT(2)
@@ -435,13 +437,17 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
if (prot & IOMMU_MMIO)
pte |= (ARM_LPAE_MAIR_ATTR_IDX_DEV
<< ARM_LPAE_PTE_ATTRINDX_SHIFT);
else if ((prot & IOMMU_CACHE) && (prot & IOMMU_SYS_CACHE_NWA))
pte |= (ARM_LPAE_MAIR_ATTR_IDX_ICACHE_OCACHE_NWA
<< ARM_LPAE_PTE_ATTRINDX_SHIFT);
/* IOMMU_CACHE + IOMMU_SYS_CACHE equivalent to IOMMU_CACHE */
else if (prot & IOMMU_CACHE)
pte |= (ARM_LPAE_MAIR_ATTR_IDX_CACHE
<< ARM_LPAE_PTE_ATTRINDX_SHIFT);
else if (prot & IOMMU_SYS_CACHE_ONLY)
else if (prot & IOMMU_SYS_CACHE)
pte |= (ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE
<< ARM_LPAE_PTE_ATTRINDX_SHIFT);
else if (prot & IOMMU_SYS_CACHE_ONLY_NWA)
else if (prot & IOMMU_SYS_CACHE_NWA)
pte |= (ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE_NWA
<< ARM_LPAE_PTE_ATTRINDX_SHIFT);
}
@@ -904,7 +910,9 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie)
(ARM_LPAE_MAIR_ATTR_INC_OWBRWA
<< ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE)) |
(ARM_LPAE_MAIR_ATTR_INC_OWBRANWA
<< ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE_NWA));
<< ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE_NWA)) |
(ARM_LPAE_MAIR_ATTR_IWBRWA_OWBRANWA
<< ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_ICACHE_OCACHE_NWA));
cfg->arm_lpae_s1_cfg.mair = reg;

16
drivers/irqchip/irq-gic-v3.c
View File

@@ -50,20 +50,6 @@ struct redist_region {
bool single_redist;
};
struct gic_chip_data {
struct fwnode_handle *fwnode;
void __iomem *dist_base;
struct redist_region *redist_regions;
struct rdists rdists;
struct irq_domain *domain;
u64 redist_stride;
u32 nr_redist_regions;
u64 flags;
bool has_rss;
unsigned int ppi_nr;
struct partition_desc **ppi_descs;
};
static struct gic_chip_data gic_data __read_mostly;
static DEFINE_STATIC_KEY_TRUE(supports_deactivate_key);
@@ -1303,7 +1289,7 @@ static inline void gic_cpu_pm_init(void) { }
#ifdef CONFIG_PM
void gic_resume(void)
{
trace_android_vh_gic_resume(gic_data.domain, gic_data.dist_base);
trace_android_vh_gic_resume(&gic_data);
}
EXPORT_SYMBOL_GPL(gic_resume);

20
drivers/media/v4l2-core/v4l2-subdev.c
View File

@@ -22,6 +22,9 @@
#include <media/v4l2-ioctl.h>
#include <media/v4l2-fh.h>
#include <media/v4l2-event.h>
#ifndef __GENKSYMS__
#include <trace/hooks/v4l2core.h>
#endif
#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
static int subdev_fh_init(struct v4l2_subdev_fh *fh, struct v4l2_subdev *sd)
@@ -487,10 +490,16 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
case VIDIOC_SUBDEV_S_FMT: {
struct v4l2_subdev_format *format = arg;
int ret = 0;
if (format->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
return -EPERM;
trace_android_vh_v4l2subdev_set_fmt(sd, subdev_fh->pad,
format, &ret);
if (ret)
return ret;
memset(format->reserved, 0, sizeof(format->reserved));
memset(format->format.reserved, 0, sizeof(format->format.reserved));
return v4l2_subdev_call(sd, pad, set_fmt, subdev_fh->pad, format);
@@ -561,10 +570,15 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
case VIDIOC_SUBDEV_S_FRAME_INTERVAL: {
struct v4l2_subdev_frame_interval *fi = arg;
int ret = 0;
if (ro_subdev)
return -EPERM;
trace_android_vh_v4l2subdev_set_frame_interval(sd, fi, &ret);
if (ret)
return ret;
memset(fi->reserved, 0, sizeof(fi->reserved));
return v4l2_subdev_call(sd, video, s_frame_interval, arg);
}
@@ -587,10 +601,16 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
case VIDIOC_SUBDEV_S_SELECTION: {
struct v4l2_subdev_selection *sel = arg;
int ret = 0;
if (sel->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
return -EPERM;
trace_android_vh_v4l2subdev_set_selection(sd, subdev_fh->pad,
sel, &ret);
if (ret)
return ret;
memset(sel->reserved, 0, sizeof(sel->reserved));
return v4l2_subdev_call(
sd, pad, set_selection, subdev_fh->pad, sel);

Some files were not shown because too many files have changed in this diff Show More