acrn-kernel/lib
Gabriel Krisman Bertazi ff9571a4de sbitmap: Try each queue to wake up at least one waiter
commit 26edb30dd1 upstream.

Jan reported the new algorithm as merged might be problematic if the
queue being awaken becomes empty between the waitqueue_active inside
sbq_wake_ptr check and the wake up.  If that happens, wake_up_nr will
not wake up any waiter and we loose too many wake ups.  In order to
guarantee progress, we need to wake up at least one waiter here, if
there are any.  This now requires trying to wake up from every queue.

Instead of walking through all the queues with sbq_wake_ptr, this call
moves the wake up inside that function.  In a previous version of the
patch, I found that updating wake_index several times when walking
through queues had a measurable overhead.  This ensures we only update
it once, at the end.

Fixes: 4f8126bb23 ("sbitmap: Use single per-bitmap counting to wake up queued tags")
Reported-by: Jan Kara <jack@suse.cz>
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
Reviewed-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20221115224553.23594-4-krisman@suse.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-03-10 09:34:34 +01:00
..
842
crypto crypto: lib - remove unneeded selection of XOR_BLOCKS 2022-08-26 18:40:14 +08:00
dim
fonts lib/fonts: fix undefined behavior in bit shift for get_default_font 2022-12-31 13:31:56 +01:00
kunit kunit: alloc_string_stream_fragment error handling bug fix 2023-01-12 12:02:41 +01:00
livepatch selftests/livepatch: better synchronize test_klp_callbacks_busy 2022-06-15 10:29:10 +02:00
lz4 lib: make LZ4_decompress_safe_forceExtDict() static 2022-07-17 17:31:39 -07:00
lzo lib/lzo/lzo1x_compress.c: replace ternary operator with min() and min_t() 2022-07-29 18:12:34 -07:00
math
mpi lib/mpi: Fix buffer overrun when SG is too long 2023-03-10 09:32:52 +01:00
pldmfw
raid6
reed_solomon treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
test_fortify
vdso lib/vdso: use "grep -E" instead of "egrep" 2022-11-23 19:50:15 +01:00
xz
zlib_deflate
zlib_dfltcc
zlib_inflate
zstd zstd: Fixing mixed module-builtin objects 2022-10-03 03:52:58 +09:00
.gitignore
Kconfig This update includes the following changes: 2022-10-10 13:04:25 -07:00
Kconfig.debug test_kprobes: Fix implicit declaration error of test_kprobes 2023-01-07 11:11:55 +01:00
Kconfig.kasan kasan: drop CONFIG_KASAN_TAGS_IDENTIFY 2022-10-03 14:02:57 -07:00
Kconfig.kcsan
Kconfig.kfence
Kconfig.kgdb parisc: Convert PDC console to an early console 2022-10-11 12:01:24 +02:00
Kconfig.kmsan kmsan: make sure PREEMPT_RT is off 2022-11-08 15:57:24 -08:00
Kconfig.ubsan ubsan: disable UBSAN_DIV_ZERO for clang 2022-07-14 15:45:26 -07:00
Makefile maple_tree: reorganize testing to restore module testing 2022-11-08 15:57:22 -08:00
argv_split.c
ashldi3.c
ashrdi3.c
asn1_decoder.c
asn1_encoder.c
assoc_array.c assoc_array: Fix BUG_ON during garbage collect 2022-06-01 18:29:06 -07:00
atomic64.c
atomic64_test.c
audit.c
base64.c lib/base64: RFC4648-compliant base64 encoding 2022-08-02 17:14:47 -06:00
bcd.c
bch.c
bitfield_kunit.c
bitmap.c lib/bitmap: remove bitmap_ord_to_pos 2022-09-26 12:19:12 -07:00
bitrev.c
bootconfig-data.S
bootconfig.c
bsearch.c
btree.c lib/btree: simplify btree_{lookup|update} 2022-06-16 19:58:21 -07:00
bucket_locks.c
bug.c cpuidle: lib/bug: Disable rcu_is_watching() during WARN/BUG 2023-03-10 09:33:47 +01:00
build_OID_registry
buildid.c
bust_spinlocks.c kernel/panic: Drop unblank_screen call 2022-09-01 16:55:35 +02:00
check_signature.c
checksum.c
clz_ctz.c
clz_tab.c
cmdline.c lib/cmdline: avoid page fault in next_arg 2022-09-11 21:55:06 -07:00
cmdline_kunit.c treewide: use get_random_{u8,u16}() when possible, part 1 2022-10-11 17:42:58 -06:00
cmpdi2.c
compat_audit.c
cpu_rmap.c
cpumask.c lib/find_bit: add find_next{,_and}_bit_wrap 2022-10-01 10:22:57 -07:00
cpumask_kunit.c lib/test_cpumask: Add for_each_cpu_and(not) tests 2022-10-06 05:57:36 -07:00
crc-ccitt.c
crc-itu-t.c crc-itu-t: fix typo in CRC ITU-T polynomial comment 2022-06-07 10:27:38 +02:00
crc-t10dif.c
crc4.c
crc7.c
crc8.c
crc16.c
crc32.c
crc32defs.h
crc32test.c
crc64-rocksoft.c
crc64.c
ctype.c
debug_info.c
debug_locks.c
debugobjects.c lib/debugobjects: fix stat count and optimize debug_objects_mem_init 2022-12-31 13:31:56 +01:00
dec_and_lock.c
decompress.c
decompress_bunzip2.c
decompress_inflate.c
decompress_unlz4.c
decompress_unlzma.c
decompress_unlzo.c
decompress_unxz.c
decompress_unzstd.c
devmem_is_allowed.c
devres.c devres: remove devm_ioremap_np 2022-09-01 18:04:43 +02:00
digsig.c
dump_stack.c
dynamic_debug.c dyndbg: add drm.debug style (drm/parameters/debug) bitmap support 2022-09-07 17:04:49 +02:00
dynamic_queue_limits.c
earlycpio.c lib: move from strlcpy with unused retval to strscpy 2022-09-11 21:55:10 -07:00
errname.c printf: fix errname.c list 2023-03-10 09:33:27 +01:00
error-inject.c lib/error-inject: traverse list with mutex 2022-07-17 17:31:38 -07:00
errseq.c
extable.c
fault-inject-usercopy.c
fault-inject.c mm: fix unexpected changes to {failslab|fail_page_alloc}.attr 2022-11-22 18:50:44 -08:00
fdt.c
fdt_addresses.c
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
find_bit.c lib/find_bit: Introduce find_next_andnot_bit() 2022-10-06 05:57:36 -07:00
find_bit_benchmark.c treewide: use prandom_u32_max() when possible, part 1 2022-10-11 17:42:55 -06:00
flex_proportions.c flex_proportions: Disable preemption entering the write section. 2022-09-19 14:35:08 +02:00
fortify_kunit.c fortify: Adjust KUnit test for modular build 2022-09-14 07:04:15 -07:00
gen_crc32table.c
gen_crc64table.c
genalloc.c
generic-radix-tree.c
glob.c
globtest.c
hexdump.c
hweight.c
idr.c ida: don't use BUG_ON() for debugging 2022-07-10 13:55:49 -07:00
inflate.c
interval_tree.c
interval_tree_test.c
iomap.c kmsan: add iomap support 2022-10-03 14:03:21 -07:00
iomap_copy.c
iommu-helper.c
iov_iter.c instrumented.h: allow instrumenting both sides of copy_from_user() 2022-10-03 14:03:18 -07:00
irq_poll.c
irq_regs.c
is_signed_type_kunit.c lib: Improve the is_signed_type() kunit test 2022-09-07 16:37:27 -07:00
is_single_threaded.c
kasprintf.c
kfifo.c
klist.c
kobject.c kobject: Fix slab-out-of-bounds in fill_kobj_path() 2023-03-10 09:33:30 +01:00
kobject_uevent.c
kstrtox.c
kstrtox.h
libcrc32c.c
linear_ranges.c
list-test.c
list_debug.c lib/list_debug.c: Detect uninitialized lists 2022-06-16 19:58:20 -07:00
list_sort.c
llist.c llist: use try_cmpxchg in llist_add_batch and llist_del_first 2022-09-11 21:55:06 -07:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-rtmutex.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c
lockref.c lockref: stop doing cpu_relax in the cmpxchg loop 2023-02-01 08:34:34 +01:00
logic_iomem.c
logic_pio.c
lru_cache.c lib/lru_cache: fix error free handing in lc_create 2022-07-17 17:31:37 -07:00
lshrdi3.c
maple_tree.c maple_tree: fix mas_empty_area_rev() lower bound validation 2023-02-09 11:28:23 +01:00
memcat_p.c
memcpy_kunit.c kunit/memcpy: Avoid pathological compile-time string size 2022-09-07 16:37:48 -07:00
memory-notifier-error-inject.c
memregion.c
memweight.c
muldi3.c
net_utils.c
netdev-notifier-error-inject.c
nlattr.c netlink: prevent potential spectre v1 gadgets 2023-02-01 08:34:43 +01:00
nmi_backtrace.c
notifier-error-inject.c lib/notifier-error-inject: fix error when writing -errno to debugfs file 2022-12-31 13:31:58 +01:00
notifier-error-inject.h
objagg.c
of-reconfig-notifier-error-inject.c
oid_registry.c
once.c once: rename _SLOW to _SLEEPABLE 2022-10-03 17:34:32 -07:00
overflow_kunit.c overflow: Refactor test skips for Clang-specific issues 2022-10-25 14:57:42 -07:00
packing.c
parman.c
parser.c
pci_iomap.c
percpu-refcount.c
percpu_counter.c
percpu_test.c
plist.c
pm-notifier-error-inject.c
polynomial.c
radix-tree.c lib/radix-tree: remove unused argument of insert_entries 2022-07-17 17:31:38 -07:00
random32.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
ratelimit.c ratelimit: Fix data-races in ___ratelimit(). 2022-08-24 13:46:57 +01:00
rbtree.c
rbtree_test.c
ref_tracker.c
refcount.c
rhashtable.c
sbitmap.c sbitmap: Try each queue to wake up at least one waiter 2023-03-10 09:34:34 +01:00
scatterlist.c lib/scatterlist: use matched parameter type when calling __sg_free_table() 2022-07-17 17:31:39 -07:00
seq_buf.c
sg_pool.c lib/sg_pool: change module_init(sg_pool_init) to subsys_initcall 2022-09-23 16:46:19 +02:00
sg_split.c
show_mem.c mm: reduce noise in show_mem for lowmem allocations 2022-09-26 19:46:29 -07:00
siphash.c SPDX changes for 5.19-rc1 2022-06-03 10:34:34 -07:00
slub_kunit.c
smp_processor_id.c lib/smp_processor_id: fix imbalanced instrumentation_end() call 2022-07-17 17:31:41 -07:00
sort.c
stackdepot.c stackdepot: reserve 5 extra bits in depot_stack_handle_t 2022-10-03 14:03:18 -07:00
stackinit_kunit.c lib: stackinit: update reference to kunit-tool 2022-09-30 13:21:22 -06:00
stmp_device.c
string.c kmsan: disable strscpy() optimization under KMSAN 2022-10-03 14:03:22 -07:00
string_helpers.c lib/string_helpers: Introduce parse_int_array_user() 2022-09-05 14:51:46 +01:00
strncpy_from_user.c
strnlen_user.c
syscall.c
test-kstrtox.c
test-string_helpers.c treewide: use prandom_u32_max() when possible, part 1 2022-10-11 17:42:55 -06:00
test_bitmap.c lib/bitmap: add tests for for_each() loops 2022-10-01 10:22:58 -07:00
test_bitops.c
test_bits.c
test_blackhole_dev.c
test_bpf.c test_bpf: fix incorrect netdev features 2022-06-22 19:20:20 -07:00
test_debug_virtual.c
test_dynamic_debug.c dyndbg: test DECLARE_DYNDBG_CLASSMAP, sysfs nodes 2022-09-07 17:04:49 +02:00
test_firmware.c test_firmware: fix memory leak in test_firmware_init() 2022-12-31 13:32:40 +01:00
test_fprobe.c treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
test_fpu.c
test_free_pages.c lib/test_free_pages.c: pass a pointer to virt_to_page() 2022-07-17 17:14:36 -07:00
test_hash.c
test_hexdump.c treewide: use prandom_u32_max() when possible, part 1 2022-10-11 17:42:55 -06:00
test_hmm.c hmm-tests: add test for migrate_device_range() 2022-10-12 18:51:50 -07:00
test_hmm_uapi.h hmm-tests: add test for migrate_device_range() 2022-10-12 18:51:50 -07:00
test_ida.c
test_kmod.c
test_kprobes.c treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
test_linear_ranges.c
test_list_sort.c treewide: use prandom_u32_max() when possible, part 1 2022-10-11 17:42:55 -06:00
test_lockup.c
test_maple_tree.c maple_tree: fix mas_empty_area_rev() lower bound validation 2023-02-09 11:28:23 +01:00
test_memcat_p.c
test_meminit.c lib/test_meminit: add checks for the allocation functions 2022-10-12 18:51:49 -07:00
test_min_heap.c treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
test_module.c
test_objagg.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
test_parman.c
test_printf.c lib/test_printf.c: fix clang -Wformat warnings 2022-07-28 10:38:30 +02:00
test_ref_tracker.c
test_rhashtable.c rhashtable: make test actually random 2022-10-26 13:39:09 +01:00
test_scanf.c
test_siphash.c
test_sort.c
test_static_key_base.c
test_static_keys.c
test_string.c
test_strscpy.c
test_sysctl.c
test_ubsan.c
test_user_copy.c
test_uuid.c
test_vmalloc.c treewide: use get_random_{u8,u16}() when possible, part 2 2022-10-11 17:42:58 -06:00
test_xarray.c
textsearch.c
timerqueue.c
trace_readwrite.c lib: Add register read/write tracing support 2022-06-15 17:41:12 +02:00
ts_bm.c lib/ts_bm.c: remove redundant store to variable consumed after addition 2022-07-17 17:31:39 -07:00
ts_fsm.c
ts_kmp.c
ubsan.c panic: Consolidate open-coded panic_on_warn checks 2023-01-24 07:24:41 +01:00
ubsan.h
ucmpdi2.c
ucs2_string.c
usercopy.c uaccess: Add speculation barrier to copy_from_user() 2023-02-25 11:25:41 +01:00
uuid.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
vsprintf.c printk changes for 6.1 2022-10-10 11:24:19 -07:00
win_minmax.c
xarray.c mm/huge_memory: Fix xarray node memory leak 2022-06-09 16:24:25 -04:00
xxhash.c