Cortex-A510's erratum #2658417 causes two BF16 instructions to return the
wrong result in rare circumstances when a pair of A510 CPUs are using
shared neon hardware.
The two instructions affected are BFMMLA and VMMLA, support for these is
indicated by the BF16 HWCAP. Remove it on affected platforms.
Signed-off-by: James Morse <james.morse@arm.com>
Link: https://lore.kernel.org/r/20220909165938.3931307-4-james.morse@arm.com
[catalin.marinas@arm.com: add revision to the Kconfig help; remove .type]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Conflicts:
arch/arm64/include/asm/sysreq.h
1. Added definition for ID_AA64ISAR1_EL1_BF16_MASK
Bug: 261510586
Change-Id: I83a5dd577fc8c0edd83c40b21f3fe54c54b6a9fa
(cherry picked from commit 1bdb0fbb2e)
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
arm64 advertises hardware features to user-space via HWCAPs, and by
emulating access to the CPUs id registers. The cpufeature code has a
sanitised system-wide view of an id register, and a sanitised user-space
view of an id register, where some features use their 'safe' value
instead of the hardware value.
It is currently possible for a HWCAP to be advertised where the user-space
view of the id register does not show the feature as supported.
Erratum workaround need to remove both the HWCAP, and the feature from
the user-space view of the id register. This involves duplicating the
code, and spreading it over cpufeature.c and cpu_errata.c.
Make the HWCAP code use the user-space view of id registers. This ensures
the values never diverge, and allows erratum workaround to remove HWCAP
by modifying the user-space view of the id register.
Signed-off-by: James Morse <james.morse@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20220909165938.3931307-2-james.morse@arm.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Bug: 261510586
Change-Id: I86bac3f501bf72ac9ee8fa6e9755ae9fbbdc81e6
(cherry picked from commit 237405ebef)
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Following "ANDROID: fips140: eliminate crypto-fips.a build step", the
only remaining dependency on LTO is the fact that the part of the module
linker script that merges the text and rodata sections and adds some
symbols is guarded by '#ifdef CONFIG_LTO_CLANG'.
This doesn't actually need to be the case, though. So guard it by
alternatively IS_ENABLED(CONFIG_CRYPTO_FIPS140_MOD).
Then, remove the dependency of CRYPTO_FIPS140_MOD on LTO_CLANG.
(Note: the android14-6.1 kernel currently has LTO disabled, which is
part of the motivation for this change. I don't know whether it will
stay that way, though.)
Bug: 188620248
Change-Id: I1aa7b293ac7a793721275e06e3ae40628e26bbc4
Signed-off-by: Eric Biggers <ebiggers@google.com>
To trick the build system into compiling some source files as built-in
code despite their actual destination being fips140.ko, a layer of
indirection was being used where the files were first built into a
static library crypto-fips.a, and then that static library was linked
into fips140.o before the final link of fips140.ko.
The problem with that approach is that it is incompatible with the usual
behavior of linking, where linking to a static library incorporates only
the needed parts of the library, not the whole library. The only reason
that it happened to work anyway is due to the dependency of the fips140
module on LTO, combined with a peculiarity of the way that the kernel
build system built LTO modules: the build system actually created
${modname}.o as a static library (despite the .o suffix), and used the
--whole-archive linker flag when linking ${modname}.ko.
commit c25e1c5582 ("kbuild: do not create *.prelink.o for Clang LTO or
IBT") in Linux v5.19 changed that. Now, ${modname}.o is an object file,
and the --whole-archive flag isn't used when linking ${modname}.ko.
Therefore, the crypto-fips.a hack no longer works, as things from this
static library (such as the initcalls) get lost during linking.
Replace it with a different hack that eliminates the dependency on LTO
and should be less fragile: undefine MODULE in fips140-defs.h, and
re-define it in the one file where it is needed. (For consistency, also
move the definition of __DISABLE_EXPORTS into fips140-defs.h.)
Bug: 188620248
Change-Id: I4a6a5f68381a7540bf37ba610216442dae0d2a7a
Signed-off-by: Eric Biggers <ebiggers@google.com>
KCFI generates ABS32 relocations in the .text section. In the temporary
copy of the .text section that the FIPS integrity check is done on,
these relocations need to be unapplied for the integrity check to pass.
Example from 'llvm-readelf --relocs --wide fips140.ko':
Relocation section '.rela.text' at offset 0x5b4a8 contains 2008 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend
[...]
0000000000000c80 0000092900000102 R_AARCH64_ABS32 0000000050e29065 __kcfi_typeid_pmull_ghash_update_p64 + 0
0000000000000e08 0000092800000102 R_AARCH64_ABS32 0000000050e29065 __kcfi_typeid_pmull_ghash_update_p8 + 0
Bug: 188620248
Change-Id: I85c01641114a66b2603abce467977823469f50c8
Signed-off-by: Eric Biggers <ebiggers@google.com>
CONFIG_CRYPTO_FIPS140_MOD builds a loadable kernel module when set to
'y'. That's very unusual, as it doesn't follow the convention of
loadable modules being 'm'.
I'm guessing that the reason that a bool was used instead of a tristate
is because this functionality is not supported built-in, so there are
only two allowed settings: disabled or modular.
However, there's actually a way to express that in the kconfig language:
a tristate that depends on 'm'. Let's do it that way.
This also eliminates the need to explicitly depend on MODULES.
(Note: I decided to keep MOD in the name, since the word "module" in
"FIPS 140 cryptographic module" is a different meaning of "module", and
I didn't want to bother renaming CRYPTO_FIPS140_MOD_EVAL_TESTING too.)
Bug: 188620248
Change-Id: Ib195d64d68c23ca93dd244d9ac77255992870424
Signed-off-by: Eric Biggers <ebiggers@google.com>
There is no good reason for the CRYPTO_FIPS140 kconfig option to exist
separately from CRYPTO_FIPS140_MOD. It existed mainly to guard some
code in the module loader that was needed only for loading the fips140
module. However, that code has been removed, since an alternate
solution that doesn't require changes to the module loader was found.
The remaining references to CRYPTO_FIPS140 are in:
- scripts/module.lds.S. But the guarded code only affects building the
fips140 module, so CRYPTO_FIPS140_MOD should be used here instead.
- lib/crypto/, for guarding the Android vendor hooks required by the
fips140 module. However, Android vendor hooks are already guarded by
ANDROID_VENDOR_HOOKS. The extra guard by CRYPTO_FIPS140 isn't useful,
especially since CRYPTO_FIPS140 was effectively hardcoded to y anyway.
It did have the side effect of making the hooks be guarded by arm64,
which excluded them from builds of arch/x86/purgatory/. However, a
cleaner way to accomplish that is to check for __DISABLE_EXPORTS,
which handles both arch/x86/purgatory/ and fips140.ko itself.
Bug: 188620248
Change-Id: Ic6141cd2a553540c2bf95774e71de7310926e3ce
Signed-off-by: Eric Biggers <ebiggers@google.com>
Kernel code is expected to support both in-tree and out-of-tree builds.
Fix in-tree builds of the fips140 module by using the full include path
for fips140-defs.h.
Bug: 188620248
Change-Id: Iba5ed888bc6a316bd4d0a97479768f50094d63d0
Signed-off-by: Eric Biggers <ebiggers@google.com>
The customizations to the module linker script required by the fips140
module are currently split between scripts/module.lds.S and a file it
includes, arch/arm64/include/asm/module.lds.h. There isn't actually
anything architecture-specific about the part that was put into the
architecture-specific file, though. Let's move that part into
scripts/module.lds.S so that the fips140 changes are in the same file.
Bug: 188620248
Change-Id: If559a216e026d2107cdf3291b487686fdb21a5e8
Signed-off-by: Eric Biggers <ebiggers@google.com>
This reverts commit 9cb398bd05.
The '__hyprel_{start,end}' symbols are no longer used, so don't bother
generating them.
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Will Deacon <willdeacon@google.com>
Bug: 261855285
Change-Id: I8e8dc5c94a9e67400e73e362e4377032328d86d4
Resolving the addresses of the hypervisor sections within a loadable
module using symbol assignment is fragile, particularly in the face of
mergeable sections (i.e. those emitted with SHF_MERGE by the compiler).
Instead, parse the ELF .hyp.* sections directly and remove the need for
global symbols in the hypervisor module linker script.
Signed-off-by: Will Deacon <will@kernel.org>
[will: Preserve function_nocfi() calls which aren't needed in android14-6.1]
Signed-off-by: Will Deacon <willdeacon@google.com>
Bug: 261855285
Change-Id: I91d88e1a341b91ffe52ffc770dddc9b46ccb3aa4
Currently, only HS descriptors will be updated with endpoint address
during binding process. According to current max_speed in configfs,
this patch will also update SS/SSP descriptors with endpoint address.
Bug: 162562782
Signed-off-by: Ray Chi <raychi@google.com>
Change-Id: I67983ef47df7ac567ec1d3af80921c39c98a545d
(cherry picked from commit 41fe558317)
(cherry picked from commit ba3ec687b7)
There shouldn't be any restriction of the descriptor size (not the
descriptor id for that matter) up to QUERY_DESC_MAX_SIZE. According to the
spec, the caller can use any descriptor size, and it is up to the device to
return the actual size. Therefore there shouldn't be any sizes hardcoded
in the kernel, nor any need to cache it, hence the
ufshcd_map_desc_id_to_length() function is redundant. Always read the
descriptors with QUERY_DESC_MAX_SIZE size.
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Suggested-by: Bean Huo <beanhuo@micron.com>
Change-Id: I7933da0b7ec3771b9faccb2c8fe05bb83f741162
Signed-off-by: Arthur Simchaev <Arthur.Simchaev@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 16ed9d312b git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
len argument is not used anymore in ufshcd_set_active_icc_lvl() function.
Change-Id: I351a36061dbc0f2f61bae371892c2de2fed3dcc1
Signed-off-by: Arthur Simchaev <Arthur.Simchaev@wdc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 01a0d515b7 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Always read the descriptor with QUERY_DESC_MAX_SIZE. According to the
spec, the device returns the actual size.
Change-Id: I29053a66d1f65f674fba6d7e1a6c6e7f546eec74
Signed-off-by: Arthur Simchaev <Arthur.Simchaev@wdc.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit f2a89b071b git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
We used to use the extended-feature field in the device descriptor as an
indication that the device supported UFS 2.2 or later. Remove that as this
check is specifically done few lines above.
Change-Id: I3291740eb846215c87cefe36c60fe09a7f4f2856
Signed-off-by: Arthur Simchaev <Arthur.Simchaev@wdc.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 358ae02f47 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
dev_err() statements are better to use than pr_err(), so switch to those.
In a similar vein, the check on the dev_req_params pointer here is not
needed, the two places this function is called never pass in a NULL
pointer, so instead of using dev_err() there just remove it.
Change-Id: I012f6a6d02f00a4295dab59ffb6772dce3f77149
Signed-off-by: Andrew Halaney <ahalaney@redhat.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Asutosh Das <quic_asutoshd@quicinc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 1026f7d366 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
This bitmask is unconditionally set in the current driver, so all
conditionals using it can be considered bit rot.
Let's take the current default conditional path everywhere and remove
dbg_print_en from the driver.
Change-Id: I7b83c8a11829963efacadd9969b50d3eeb4ec177
Signed-off-by: Andrew Halaney <ahalaney@redhat.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Asutosh Das <quic_asutoshd@quicinc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit e4ce23fba3 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
The current implementation has abstractions that don't give any benefits.
The print_fn callback (and its only callback implementation,
ufs_qcom_dump_regs_wrapper()) was only used by
ufs_qcom_print_hw_debug_reg_all() and just multiplies len by 4 before
calling ufshcd_dump_regs().
ufs_qcom_print_hw_debug_reg_all() is only called by
ufs_qcom_dump_dbg_regs().
There's no real gain in those abstractions, so let's just do the work
directly in ufs_qcom_dump_dbg_regs() (the dbg_register_dump callback).
Change-Id: I5085c094e56319ea519c2e6e0a28f97be49e4a30
Signed-off-by: Andrew Halaney <ahalaney@redhat.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Asutosh Das <quic_asutoshd@quicinc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 50a427a00c git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
This code path is only called through one function, and the HBA struct is
already accessed in ufshcd_vops_dbg_register_dump() prior to calling so
there is no way for it to be NULL.
Likewise, the print_fn callback is always supplied within this driver and
is always provided.
Change-Id: I7c0b84767d094568e998bcfa11008a2e19c0554b
Signed-off-by: Andrew Halaney <ahalaney@redhat.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Asutosh Das <quic_asutoshd@quicinc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 921a880827 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Add advanced RPMB support in ufs_bsg:
1. According to the UFS specification, only one RPMB operation can be
performed at any time. We can ensure this by using reserved slot and
its dev_cmd sync operation protection mechanism.
2. For Advanced RPMB, RPMB metadata is packaged in an EHS (Extra Header
Segment) of a command UPIU, and the corresponding reply EHS (from the
device) should also be returned to the user space. bsg_job->request
and bsg_job->reply allow us to pass and return EHS from/back to
userspace.
Compared to normal/legacy RPMB, the advantages of advanced RPMB are:
1. The data length in the Advanced RPMB data read/write command can be
larger than 4KB. For the legacy RPMB, the data length in a single RPMB
data transfer is 256 bytes.
2. All of the advanced RPMB operations will be a single command. For
legacy RPMB, take the read write-counter value as an example, you need
two commands (first SECURITY PROTOCOL OUT, then second SECURITY
PROTOCOL IN).
Change-Id: Iefadc64abb18c1dd6ac5e416ed692be9b7019d71
Signed-off-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 6ff265fc5e git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
We need to fill in the total EHS length in UTP Transfer Request Descriptor.
Add this functionality to ufshcd_prepare_req_desc_hdr().
Change-Id: Ifdc93e0877226d437daae95e7385b26d899188d4
Signed-off-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit a4b1c9b9b3 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Check UFS Advanced RPMB LU enablement during ufshcd_lu_init().
Change-Id: Ib923917dd9144f824bc08e4b600e6fd54e1716a5
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit f6b9d0fe5c git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Take out the "map scatter-gather list to prdt" part of the code in
ufshcd_map_sg() and split it into a new function ufshcd_sgl_to_prdt().
Change-Id: Idd40c405ab09cb1154eae17abcd921b0ea0013c9
Signed-off-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 7a4df79d0b git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Remove checks on job->request_len and job->reply_len because msgcode checks
in a subseqent commit will rule out malicious requests.
Change-Id: I62580fb6e8facf3983d51664e69c821db0bc5684
Signed-off-by: Bean Huo <beanhuo@micron.com>
Acked-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 64d4864714 git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
When running the clean target, kbuild is supposed to remove dtbs and
dtbos. However, kbuild only searches for those devicetree build
artifacts in the directory that the kernel binaries or kernel modules
are output to. This is not sufficient in cases where an external
devicetree is used, so include the external devicetree path when
searching for dtb and dtbo build artifacts.
Bug: 264602319
Fixes: 3d42cc9e75 ("ANDROID: kbuild: add support for compiling external device trees")
Change-Id: I45fdfdef09c3d57401d98e5db731273147d7d265
Signed-off-by: Isaac J. Manjarres <isaacmanjarres@google.com>
As S2MPU driver was moved to an EL2 module, we need to export init
and register functions to be used from the module.
Bug: 261857213
Change-Id: I819dff0e6164051c464533d80eb68ea003b34b21
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Add functions needed by S2MPU el2 module:
-pkvm_host_donate_hyp/pkvm_hyp_donate_host: get and return SMPT from
host kernel.
-memcpy/memset: common functions used by s2mpu module.
-hyp_pa: convert hyp address to PA, to get DMA address of SMPT.
-kern_hyp_va: convert kernel address to hyp VA, used by the
module to get the VA of SMPTs allocated from the kernel.
Bug: 261857213
Change-Id: Idceccf0ac38444e7f267f50c476b99f01a613372
Signed-off-by: Mostafa Saleh <smostafa@google.com>
As S2MPU driver is removed from common kernel, remove it's symbols.
Bug: 261857213
Change-Id: I7c70f656ffe9c8555fba1085dae29ac8c5de0f5e
Signed-off-by: Mostafa Saleh <smostafa@google.com>
As the the s2mpu el2 driver is loaded at run time, we can't convert
its address from kernel to hyp using kern_hyp_va.
The kernel module will be responsible of sending the address of the
struct as hyp address in s2mpu hypercalls, currently the load address
of the modules is returned as the token of the module.
Although the passed arguments is the driver struct address, but the
address passed is not a kernel address, so pass it as u64 to avoid
confusing it as a kernel address.
Bug: 261857213
Change-Id: Ieb787f83d7c07ada2c8565f0e34cdc6de71aff6f
Signed-off-by: Mostafa Saleh <smostafa@google.com>
As S2MPU driver is used by a few devices and EL2 loadable modules
code is merged.
S2MPU pKVM driver will be moved to be an EL2 module.
This commit removes all the code and configuration for the driver.
Bug: 261857213
Change-Id: I53da660f8503c4bedf2fc48a90d1e8ba73329c05
Signed-off-by: Mostafa Saleh <smostafa@google.com>
crypto_memneq() is one of the utility functions that was intentionally
included in the fips140 module, out of concerns that it would be seen as
"cryptographic" and thus would be required to be included the module for
the FIPS certification. It should not have been removed from the
module, so add it back.
Bug: 188620248
Fixes: 916c8c836f ("ANDROID: FIPS: remove memneq.o from the list of objects for the fips module")
Change-Id: I8a19dfd73390f8c1348885f97fa42d900e47b82b
Signed-off-by: Eric Biggers <ebiggers@google.com>
This reverts commit aeac190b3b.
Bug: 264333547
Test: /data/local/tmp/sebastianene/tests/test_host_app
Change-Id: Id88b705dd725cc8720913fd2909030c2f2fb597f
Signed-off-by: Sebastian Ene <sebastianene@google.com>
This reverts commit 8a57e65659.
Reason for revert: This symbol "PageMovable" should not be used by
any driver and is being removed from upstream.
Bug: 264353898
Bug: 261718474
Cc: Jörg Wagner <jorwag@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I368f30bd4ab1f28480ee37e3ee10df9dbb2f5744
For the entropy analysis, we must provide some output from the Jitter
RNG: a large amount of output from one instance, and a smaller amount of
output from each of a certain number of instances.
The original plan was to use a build of the userspace jitterentropy
library that matches the kernel's jitterentropy_rng as closely as
possible. However, it's now being requested that the output be gotten
from the kernel instead.
Now that fips140_lab_util depends on AF_ALG anyway, it's straightforward
to dump output from jitterentropy_rng instances using AF_ALG.
Therefore, add a command dump_jitterentropy which supports this.
Bug: 188620248
Change-Id: I78eb26250e88f2fc28fc44aa201acbe5b84df8bb
Signed-off-by: Eric Biggers <ebiggers@google.com>
(cherry picked from commit dc01503266)
There is a race window between page_pinner_inited set and the pp_buffer
initialization which cause accessing the pp_buffer->lock. Avoid this by
moving the pp_buffer initialization to page_ext_ops->init() which sets
the page_pinner_inited only after the pp_buffer is initialized.
Race scenario:
1) init_page_pinner is called --> page_pinner_inited is set.
2) __alloc_contig_migrate_range --> __page_pinner_failure_detect()
accesses the pp_buffer->lock(yet to be initialized).
3) Then the pp_buffer is allocated and initialized.
Below is the issue call stack:
spin_bug+0x0
_raw_spin_lock_irqsave+0x3c
__page_pinner_failure_detect+0x110
__alloc_contig_migrate_range+0x1c4
alloc_contig_range+0x130
cma_alloc+0x170
dma_alloc_contiguous+0xa0
__dma_direct_alloc_pages+0x16c
dma_direct_alloc+0x88
Bug: 259024332
Change-Id: I6849ac4d944498b9a431b47cad7adc7903c9bbaa
Signed-off-by: Charan Teja Kalla <quic_charante@quicinc.com>
Get direct reclaim info.
Bug: 190795589
Signed-off-by: Martin Liu <liumartin@google.com>
Change-Id: Ie66a3c87484a364a918c19b8e044c82f1afd6749
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit 12902c9996)
Do not account __GFP_NORETRY allocation failure as cma_alloc_fail
since it's not critical failure(i.e., the caller with __GFP_NORETRY
should always carry on the fallback plan). It's also good for
compatibility POV with upstream since upstream cma_alloc_fail
only counts cma_alloc_fail with !__GFP_NORETRY since upstream
doesn't support __GFP_NORTRY yet.
Bug: 220669548
Signed-off-by: Minchan Kim <minchan@google.com>
Change-Id: I377e6b033c3786e10b6b1c814037a4fc40e20a73
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit 8ffc7ff817)
