Notable upstream pull request merges:
#16853894edd084 Add TXG timestamp database
#17004 -multiple Implement dynamic gang header sizes
#17405dea0fc969 ZVOL: Return early, if volmode is ZFS_VOLMODE_NONE
on FreeBSD side
#17455cf146460c Default to zfs_bclone_wait_dirty=1
#17481523d9d600 Validate mountpoint on path-based unmount using statx
#1748292da9e0e9 ZVOL: Implement zvol_alloc() function on FreeBSD side
#17489dee62e074 spa: ZIO_TASKQ_ISSUE: Use symbolic priority
#17496bf846dcb7 Release topology restrictions on special/dedup
#174974e92aee23 Relax special_small_blocks restrictions
#17503ea38787f2 Revert "Fix incorrect expected error in ztest"
#17505be1e991a1 Allow and prefer special vdevs as ZIL
#17506ee0cb4cb8 ztest: Fix false positive of ENOSPC handling
#1750792d3b4ee2 zio: rename `io_reexecute` as `io_post`;
use it for the direct IO checksum error flag
#175086af8db61b metaslab: don't pass whole zio to throttle reserve APIs
#175184c2a7f85d FreeBSD: Add support for _PC_HAS_HIDDENSYSTEM
#17521d7ab07dfb ZIL: Force writing of open LWB on suspend
#17524b6e8db509 zpool/zfs: Add '-a|--all' option to scrub, trim,
initialize
#17531c1e51c55f Correct weight recalculation of space-based metaslabs
#17533d323fbf49 FreeBSD: zfs_putpages: don't undirty pages until
after write completes
#17536b21e04e8d Fix zdb pool/ with -k
#17537 -multiple Userspace tunables
#175402957eabbe Add support for FreeBSD's Solaris style extended
attribute interface
#175474bd7a2eaa zdb: fix checksum calculation for decompressed blocks
#17551 -multiple cleanup: remove var init/update
#175610f8a1105e Skip dbuf_evict_one() from dbuf_evict_notify()
for reclaim thread
#1756310a78e264 Faster checksum benchmark on system boot
#1756400ce064d8 spa: update blkptr diagram to include vdev padding
on encrypted blocks
#17565 -multiple Physical rewrite
#17566fc885f308 Don't use wrong weight when passivating group
#17572f70c85086 BRT: Fix ZAP entry endianness
#175750b6fd024a ZVOL: Unify zvol minors operations and improve error
handling
#17581cb5e7e097 range_tree: Provide more debug details upon unexpected
add/remove
#17587 -multiple Fix two issues with dynamic gang headers
Obtained from: OpenZFS
OpenZFS commit: 8302b6e32b
If the system administrator does "zpool offline", he's doing it for a
reason. zfsd shouldn't consider an offline disk to be an event that
requires automatic healing. Don't online it in response to a GEOM
event, and don't try to activate a hotspare to take over from it.
MFC after: 2 weeks
Sponsored by: ConnectWise
Symbols defined using assembler directives lack type info, but in this
case one ought to be able to cast a pointer to the symbol and
dereference the pointer to get a value. Without this change, D
disallows this trick since it requires all identifiers to have a type.
Relax the rules slightly and allow an identifier to have type "void" if
we know we're just taking its address.
As a result, the following dtrace invocation works:
dtrace -n 'tick-1s {printf("%d", *(int *)&`ticks);}'
In particular, since commit b2b974f7ef ("clock: Simplify subr_ticks
and rename"), "ticks" does not have any type info associated with it, so
its value couldn't be printed. This trick provides a workaround and is
probably generally useful.
Add a regression test which exercises this functionality.
PR: 287752
Reviewed by: avg
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D51417
Dtrace assumes only ELF sections of type SHT_PROGBITS or SHT_NOBITS
occupy memory space. However, sections with SHF_ALLOC flag also consume
memory space. Moreover, the symbol address initialization skips symbols
at the very beginning of a section in ET_REL KLDs.
Fix: Check section flag for calculating section offset, and disable the
skipping at the beginning of a section.
PR: 288000
Reviewed by: markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D51188
Commit 5c1ba994a8 inadvertently moved the installed location of a
shared library. Move it back.
Reported by: olivier
Tested by: olivier
Fixes: 5c1ba994a8
Sponsored by: ConnectWise
I occasionally see failures due to pthread_barrier_wait() not returning
0. There is another possible non-error return value, so allow that too.
While here, check the result in tp_delay() as well.
Fixes: 5c1ba994a8 ("Add a regression test for a libtpool bug")
Differential Revision: https://reviews.freebsd.org/D50967
Test that tpool_dispatch returns an error if it cannot start even one
worker. Previously, it would hang. The test must reside here rather
than in the OpenZFS repo because the latter has no infrastructure for
writing libtpool tests.
https://github.com/openzfs/zfs/issues/16172
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D45587
Alphabetize option table to improve first glance access.
Markup semantics to differentiate them and add them to apropos.
MFC after: 3 days
Reviewed by: Pau Amma <pauamma@gundo.com>
Approved by: mhorne (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D49883
Commit 1cbb58886a (shipped in 12.0.0) removed all lint infrastructure.
A bunch of NO_LINT definitions remained (perhaps as a bootstrapping
measture). Remove them.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D50704
Notable upstream pull request merges:
#14876b048bfa9c Allow opt-in of zvol blocks in special class
#16486b6916f995 ARC: parallel eviction
#171691a8f5ad3b zvol: Enable zvol threading functionality on FreeBSD
#17209c17bdc491 More aggressively assert that db_mtx protects db.db_data
#17218734eba251 Wire O_DIRECT also to Uncached I/O
#1724649fbdd453 Introduce zfs rewrite subcommand
#17255b1ccab172 ARC: Avoid overflows in arc_evict_adj()
#17280848794503 zcp: get_prop: fix encryptionroot and encryption
#17301086105f4c Cause zpool scan resume commands to get logged in history
#17306246e5883b zfs_valstr: update zio_flag strings for
ZIO_FLAG_PREALLOCATED
#1730978628a5c1 FreeBSD: Use new SYSCTL_SIZEOF()
#1731489a8a9158 ARC: Notify dbuf cache about target size reduction
#173198b9c4e643 spa: clear checkpoint information during retry
#17339d5616ad34 Increase meta-dnode redundancy in "some" mode
#17340ea74cdedd Fix 2 bugs in non-raw send with encryption
#17342e55225be3 Add explicit DMU_DIRECTIO checks
#17343d8a33bc0a icp: Use explicit_memset() exclusively in gcm_clear_ctx()
#17344 -multiple libzfs_core: add ZFS_IOC_TRACE envvar to enable ioctl
tracing
#17348f0baaa329 arcstat: prevent ZeroDivisionError when L2ARC becomes
empty
#1735383fa80a55 dmu_objset_hold_flags() should call
dsl_dataset_rele_flags() on error
#17355 -multiple Allow txg_wait_synced_flags() and dmu_tx_assign()
to return when the pool suspends
#173615c30b2438 Fix null dereference in spa_vdev_remove_cancel_sync()
#17363ddf28f27c Fix off-by-one bug in range tree code
#1736706fa8f3f6 zfs_cmd: reorganise zfs_cmd_t to match original size
#173682a91d577b Expose dataset encryption status via fast stat path
#17372c464f1d01 Only interrupt active disk I/Os in failmode=continue
#173739d76950d6 ZIL: Improve write log size accounting
#17377 -multiple tunables: general code cleanup
#17379fa697b94e FreeBSD: Add posix_fadvise(POSIX_FADV_WILLNEED) support
#173843dfa98d01 ZVOL: Make zvol_inhibit_dev module parameter
platform-independent
Obtained from: OpenZFS
OpenZFS commit: 3084336ae4
currently FreeBSD-utilities depends on both FreeBSD-ufs and FreeBSD-zfs.
this is not desirable, because those are both relatively large packages
and the user may want to remove one or the other – or perhaps both, e.g.
in a jail or embedded system.
the reason for this dependency is that fstyp(8), which is in
FreeBSD-utilities, links both libufs and libzfs. FreeBSD-utilities is
the correct place for fstyp, so we don't want to move that.
instead, add two new packages: libufs contains libufs, and libzfs
contains libzfs plus the ZFS libraries it depends on: libavl, libnvpair,
libspl, libtpool, libumem, libuutil, libzfs_core and libzutil.
with this change, it is possible to remove FreeBSD-ufs and/or
FreeBSD-zfs while leaving FreeBSD-libufs, FreeBSD-libzfs and
FreeBSD-utilities installed.
Reviewed by: manu, des
Approved by: des (mentor)
Differential Revision: https://reviews.freebsd.org/D50148
dt_oformat_drop() should only be called when in structured output mode.
Reviewed by: Domagoj Stolfa
Fixes: 93f27766a7 ("dtrace: Add the 'oformat' libdtrace option")
The illumos-specific _SC_CPUID_MAX is the largest CPU ID in the system. This
was mapped to _SC_NPROCESSORS_CONF, which is the total number of CPUs recognized
by the kernel. If CPU IDs are contiguous, as is the case on amd64 and arm64,
this value is one greater than the maximum ID. As a result, when consuming
per-CPU dtrace buffers, libdtrace tries to fetch from a non-existent CPU. This
is mostly harmless in practice, but still wrong.
As we don't have a sysconf value for the maximum CPU ID, add a wrapper which
fetches it using the kern.smp.maxid sysctl.
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D49243
The zero'th index in the array is unused, so a priority queue of N elements
needs N+1 array slots. Fix the allocation.
Also fix the assertion in dt_pq_insert(): the assertion needs to be checked
after incrementing the count of items in the priority queue, otherwise it can
miss an overflow.
Reported by: CHERI
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D49242
Notable upstream pull request merges:
#16857387ed5ca4 Add recursive dataset mounting and unmounting support
to pam_zfs_key
#16929c2458ba92 optimize recv_fix_encryption_hierarchy()
#1698012f0baf34 Make the vfs.zfs.vdev.raidz_impl sysctl cross-platform
#1698640496514b Expand fragmentation table to reflect larger possibile
allocation sizes
#1700388020b993 Add kstats tracking gang allocations
#1701321205f648 Avoid ARC buffer transfrom operations in prefetch
#17016390f6c119 zio: lock parent zios when updating wait counts on
reexecute
#17029b8c73ab78 zio: do no-op injections just before handing off to vdevs
#170376a2f7b384 Fix metaslab group fragmentation math
#17040b901d4a0b Update the dataset name in handle after zfs_rename
Obtained from: OpenZFS
OpenZFS commit: 6a2f7b3844
Make it easier to support data models other than ILP32 and LP64 by
avoiding constructs which assume that it must be one or the other.
No functional change intended.
MFC after: 2 weeks
Sponsored by: Innovate UK
If one of two pointers refers to a forward declaration, let the pointers
be compatible so long as the referred types have the same name.
Otherwise we can get spurious errors.
To give a specific example, this can happen when ipfw_nat.ko is loaded
before ipfw.ko and /usr/lib/dtrace/ipfw.d is processed. Currently,
ipfw_nat.ko does not have a definition for struct inpcb (i.e., none of
its files include in_pcb.h), so in the CTF type graph, struct
ip_fw_args' "inp" member refers to a forward declaration, represented in
CTF with CTF_K_FORWARD.
Then, when libdtrace processes the ipfw_match_info_t translator in
ipfw.d, it decides that the "inp" field assignment is incorrect because
the two pointers are incompatible. However, there's no harm in allowing
this assignment. Add some logic to dt_node_is_ptrcompat() to detect
this case and declare the pointers as compatible so long as the name of
the thing they refer to is the same, similar to how any pointer is
compatible with a void *.
Reported by: marck
Reviewed by: Domagoj Stolfa <domagoj.stolfa@gmail.com>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D48254
Remove unneeded th_x2 initalization, use named constants
instead of magic numbers (fixing one oversight) and add
some line breaks. Expand one man page slightly.
No functional change intended.
Reviewed By: tuexen, cc
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D48065
Notable upstream pull request merges:
#16244acb6e71ed Added output to `zpool online` and `offline`
#1668494a03dd1e Pack dmu_buf_impl_t by 16 bytes
#166906187b1943 On the first vdev open ignore impossible ashift hints
#16692673efbbf5 zdb: add extra -T flag to show histograms of BRT refcounts
#166932bf152021 Fix gcc uninitialized warning in FreeBSD zio_crypt.c
#16694b16e09619 Reduce dirty records memory usage
#167015945676bc ZFS send should use spill block prefetched from
send_reader_thread
#167341c9a4c8cb Fix user properties output for zpool list
Obtained from: OpenZFS
OpenZFS commit: 1c9a4c8cb4
It was disabled by default in fe52b7f60e. We planned to (but did not)
remove the option before FreeBSD 14. Remove it now, for FreeBSD 15.
Relnotes: Yes
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31558
Notable upstream pull request merges:
#9416 -multiple zio_compress: introduce max size threshold
#10018a10e552b9 Adding Direct IO Support
#15147e419a63bf xattr dataset prop: change defaults to sa
#154547e957fde7 send/recv: open up additional stream feature flags
#158100d77e738e Defer resilver only when progress is above a threshold
#159213cf2bfa57 Allocate zap_attribute_t from kmem instead of stack
#16483 -multiple dmu_objset: replace dnode_hash impl with cityhash4
#164858be2f4c3d zio_resume: log when unsuspending the pool
#1649188433e640 sys/types32.h: Remove struct timeval32 from libspl header
#16496f245541e2 zfs_file: implement zfs_file_deallocate for FreeBSD 14
#16511308f7c2f1 Fix an uninitialized data access
#1652929c9e6c32 Fix handling of DNS names with '-' in them for sharenfs
#16531ddf5f34f0 Avoid fault diagnosis if multiple vdevs have errors
#165396f50f8e16 zfs_log: add flex array fields to log record structs
#16546d40d40913 Evicting too many bytes from MFU metadata
#165513014dcb76 Reduce and handle EAGAIN errors on AIO label reads
#1655480645d658 FreeBSD: restore zfs_znode_update_vfs()
#16565832f66b21 FreeBSD: Sync taskq_cancel_id() returns with Linux
#1656748d1be254 Properly release key in spa_keystore_dsl_key_hold_dd()
#16569141368a4b Restrict raidz faulted vdev count
#16583c84a37ae9 lua: add flex array field to TString type
#1658486737c592 Avoid computing strlen() inside loops
#16587d34d4f97a snapdir: add 'disabled' value to make .zfs inaccessible
#16593224393a32 feature: large_microzap
#16597412105977 Temporarily disable Direct IO by default
#166054ebe674d9 ARC: Cache arc_c value during arc_evict()
Backported pull request merges:
#16613ab777f436 Return boolean_t in inline functions of
lib/libspl/include/sys/uio.h
#16616efeb60b86 FreeBSD: ignore some includes when not building kernel
#16635 ---TBD--- zdb: fix printf format in dump_zap()
Obtained from: OpenZFS
OpenZFS commit: b109925820
OpenZFS tag: 2.3.0-rc1
These were reported by `mandoc -T lint ...` as errors; this commit only
handles unnecessary .El commands.
The rendered output (in ascii and html) is not affected by this commit.
Signed-off-by: Graham Percival <gperciva@tarsnap.com>
Reviewed by: mhorne
MFC after: 3 days
Sponsored by: Tarsnap Backup Inc.
Pull Request: https://github.com/freebsd/freebsd-src/pull/1447
Without this patch, vdevs faulted via AUX state would not be replaced
once the appropriate drive is replaced. ZFS does not internally use
that state, but a drive can be manually forced into such a state with a
command like zinject.
Submitted by: Goran Mekić <meka@tilda.center>
Sponsored by: ConnectWise
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D46866
Notable upstream pull request merges:
#15892 -multiple Fast Dedup: Introduce the FDT on-disk format and feature flag
#15893 -multiple Fast Dedup: “flat” DDT entry format
#15895 -multiple Fast Dedup: FDT-log feature
#162396be8bf555 zpool: Provide GUID to zpool-reguid(8) with -g
#16277 -multiple Fast Dedup: prune unique entries
#163165807de90a Fix null ptr deref when renaming a zvol with snaps and snapdev=visible
#1634377a797a38 Enable L2 cache of all (MRU+MFU) metadata but MFU data only
#1644683f359245 FreeBSD: fix build without kernel option MAC
#16449963e6c9f3 Fix incorrect error report on vdev attach/replace
#16505b10992582 spa_prop_get: require caller to supply output nvlist
Obtained from: OpenZFS
OpenZFS commit: b109925820
The zfsd tests rely on this having the same layout as the real
zpool_handle, which changed in the last OpenZFS import.
Fixes: 62e7d3c89e ("ddt: add support for prefetching tables into the ARC")
Reported by: Jenkins
Notable upstream pull request merges:
#158175536c0dee Sync AUX label during pool import
#15889c7ada64bb ddt: dedup table quota enforcement
#1589062e7d3c89 ddt: add support for prefetching tables into the ARC
#15894e26b3771e spa_preferred_class: pass the entire zio
#15894d54d0fff3 dnode: allow storage class to be overridden by object type
#1619755427add3 Several improvements to ARC shrinking
#16217 -multiple JSON output for various zfs and zpool subcommands
#1624824e6585e7 libzfs.h: Set ZFS_MAXPROPLEN and ZPOOL_MAXPROPLEN
to ZAP_MAXVALUELEN
#162649dfc5c4a0 Fix long_free_dirty accounting for small files
#16268ed0db1cc8 Make txg_wait_synced conditional in zfsvfs_teardown,
for FreeBSD
#16288d60debbf5 Fix sa_add_projid to lookup and update SA_ZPL_DXATTR
#16308ec580bc52 zfs: add bounds checking to zil_parse
#16310c21dc56ea Fix zdb_dump_block for little endian
#163157ddc1f737 zil: add stats for commit failure/fallback
#16326b0bf14cdb abd: lift ABD zero scan from zio_compress_data()
to abd_cmp_zero()
#16337c8184d714 Block cloning conditionally destroy ARC buffer
#16338dbe07928b Add support for multiple lines to the sharenfs property
for FreeBSD
#163741a3e32e6a Cleanup DB_DNODE() macros usage
#16374ed87d456e Skip dnode handles use when not needed
#16346fb6d8cf22 Add some missing vdev properties
#16364670147be5 zvol: ensure device minors are properly cleaned up
#16382dea8fabf7 FreeBSD: Fix RLIMIT_FSIZE handling for block cloning
#16387aef452f10 Improve zfs_blkptr_verify()
#16395cbcb52243 Fix the names of some FreeBSD sysctls in
include/tunables.cfg
#164015b9f3b766 Soften pruning threshold on not evictable metadata
#16404cdd53fea1 FreeBSD: Add missing memory reclamation accounting
#164041fdcb653b Once more refactor arc_summary output
#164191f5bf91a8 Fix memory corruption during parallel zpool import
with -o cachefile
#16426cf6e8b218 zstream: remove duplicate highbit64 definition
Obtained from: OpenZFS
OpenZFS commit: 9c56b8ec78
Notable upstream pull request merges:
#16209 --multi-- icp: rip out everything we don't use
#1623020c8bdd85 FreeBSD: Update use of UMA-related symbols in
arc_available_memory
#16242121a2d335 FreeBSD: unregister mountroot eventhandler on unload
#162585de3ac223 vdev_open: clear async fault flag after reopen
#16270436731276 zvol: Fix suspend lock leaks
#16273c87cb22ba head_errlog: fix use-after-free
#16284f72e081fb FreeBSD: Use a statement expression to implement
SET_ERROR()
#16300a10faf5ce FreeBSD: Use the new freeuio() helper to free dynamically
allocated UIOs
#16302a7fc4c85e zstd: don't call zstd_mempool_reap if there are no buffers
#16334dc91e7452 zdb: dump ZAP_FLAG_UINT64_KEY ZAPs properly
Obtained from: OpenZFS
OpenZFS commit: 1147a27978
ZFS' libspl needs to be made aware that we have strlcat(3) and
strlcpy(3) to avoid some more complicated declaration duplication, so
go ahead and define these HAVE_ macros now.
libprocstat has to define `_KERNEL` and include kernel headers in order
to get what it wants, but this results in sys/cdefs.h being included too
late and we pick up the build breaking version of the __RENAME
definition. Just explicitly include sys/cdefs.h earlier rather than
disabling _FORTIFY_SOURCE. The zfs/ subdir only builds an object that
holds some structures and sizes, so just disable _FORTIFY_SOURCE there
entirely rather than trying to move #define _KERNEL into the file..
While we're here, make sure that we disable _FORTIFY_SOURCE in the
bootloader because we don't have the symbol renaming support today to do
it as cleanly as we'd like. ssp/ssp.h needs to be pulled into the libsa
environment so that other bits can understand that ssp is disabled in
the consistent __SSP_FORTIFY_LEVEL way that we try to do.
Reviewed by: allanjude (previous version), markj
Sponsored by: Klara, Inc.
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D45676
Martin Matuska