c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2024-11-18 18:31:00 +03:00
Code Issues Packages Projects Releases Wiki Activity
f94b3cbf43
mirror_zfs/config/kernel-shrink.m4

261 lines
6.8 KiB
Plaintext
Raw Normal View History

Linux 3.1 compat, super_block->s_shrink The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #466 Closes #292
2011-12-23 00:20:43 +04:00
dnl #
dnl # 3.1 API change
dnl # The super_block structure now stores a per-filesystem shrinker.
dnl # This interface is preferable because it can be used to specifically
dnl # target only the zfs filesystem for pruning.
dnl #
AC_DEFUN([ZFS_AC_KERNEL_SHRINK], [
AC_MSG_CHECKING([whether super_block has s_shrink])
ZFS_LINUX_TRY_COMPILE([
#include <linux/fs.h>
Eliminate runtime function pointer mods in autotools checks PaX/GrSecurity patched kernels implement a dialect of C that relies on a GCC plugin for enforcement. A basic idea in this dialect is that function pointers in structures should not change during runtime. This causes code that modifies function pointers at runtime to fail to compile in many instances. The autotools checks rely on whether or not small test cases compile against a given kernel. Some autotools checks assume some default case if other cases fail. When one of these autotools checks tests a PaX/GrSecurity patched kernel by modifying a function pointer at runtime, the default case will be used. Early detection of such situations is possible by relying on compiler warnings, which are compiler errors when --enable-debug is used. Unfortunately, very few people build ZFS with --enable-debug. The more common situation is that these issues manifest themselves as runtime failures in the form of NULL pointer exceptions. Previous patches that addressed such issues with PaX/GrSecurity compatibility largely relied on rewriting autotools checks to avoid runtime function pointer modification or the addition of PaX/GrSecurity specific checks. This patch takes the previous work to its logical conclusion by eliminating the use of runtime function pointer modification. This permits the removal of PaX-specific autotools checks in favor of ones that work across all supported kernels. This should resolve issues that were reported to occur with PaX/GrSecurity-patched Linux 3.7.5 kernels on Gentoo Linux. https://bugs.gentoo.org/show_bug.cgi?id=457176 We should be able to prevent future regressions in PaX/GrSecurity compatibility by ensuring that all changes to ZFSOnLinux avoid runtime function pointer modification. At the same time, this does not solve the issue of silent failures triggering default cases in the autotools check, which is what permitted these regressions to become runtime failures in the first place. This will need to be addressed in a future patch. Reported-by: Marcin Mirosław <bug@mejor.pl> Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #1300
2013-02-15 03:54:04 +04:00
int shrink(struct shrinker *s, struct shrink_control *sc)
{ return 0; }
static const struct super_block
sb __attribute__ ((unused)) = {
Linux 3.1 compat, super_block->s_shrink The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #466 Closes #292
2011-12-23 00:20:43 +04:00
.s_shrink.shrink = shrink,
.s_shrink.seeks = DEFAULT_SEEKS,
.s_shrink.batch = 0,
};
Eliminate runtime function pointer mods in autotools checks PaX/GrSecurity patched kernels implement a dialect of C that relies on a GCC plugin for enforcement. A basic idea in this dialect is that function pointers in structures should not change during runtime. This causes code that modifies function pointers at runtime to fail to compile in many instances. The autotools checks rely on whether or not small test cases compile against a given kernel. Some autotools checks assume some default case if other cases fail. When one of these autotools checks tests a PaX/GrSecurity patched kernel by modifying a function pointer at runtime, the default case will be used. Early detection of such situations is possible by relying on compiler warnings, which are compiler errors when --enable-debug is used. Unfortunately, very few people build ZFS with --enable-debug. The more common situation is that these issues manifest themselves as runtime failures in the form of NULL pointer exceptions. Previous patches that addressed such issues with PaX/GrSecurity compatibility largely relied on rewriting autotools checks to avoid runtime function pointer modification or the addition of PaX/GrSecurity specific checks. This patch takes the previous work to its logical conclusion by eliminating the use of runtime function pointer modification. This permits the removal of PaX-specific autotools checks in favor of ones that work across all supported kernels. This should resolve issues that were reported to occur with PaX/GrSecurity-patched Linux 3.7.5 kernels on Gentoo Linux. https://bugs.gentoo.org/show_bug.cgi?id=457176 We should be able to prevent future regressions in PaX/GrSecurity compatibility by ensuring that all changes to ZFSOnLinux avoid runtime function pointer modification. At the same time, this does not solve the issue of silent failures triggering default cases in the autotools check, which is what permitted these regressions to become runtime failures in the first place. This will need to be addressed in a future patch. Reported-by: Marcin Mirosław <bug@mejor.pl> Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #1300
2013-02-15 03:54:04 +04:00
],[
Linux 3.1 compat, super_block->s_shrink The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #466 Closes #292
2011-12-23 00:20:43 +04:00
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_SHRINK, 1, [struct super_block has s_shrink])
],[
AC_MSG_RESULT(no)
Fix arc_adapt() spinning in iterate_supers_type() The iterate_supers_type() function which was introduced in the 3.0 kernel was supposed to provide a safe way to call an arbitrary function on all super blocks of a specific type. Unfortunately, because a list_head was used a bug was introduced which made it possible for iterate_supers_type() to get stuck spinning on a super block which was just deactivated. This can occur because when the list head is removed from the fs_supers list it is reinitialized to point to itself. If the iterate_supers_type() function happened to be processing the removed list_head it will get stuck spinning on that list_head. The bug was fixed in the 3.3 kernel by converting the list_head to an hlist_node. However, to resolve the issue for existing 3.0 - 3.2 kernels we detect when a list_head is used. Then to prevent the spinning from occurring the .next pointer is set to the fs_supers list_head which ensures the iterate_supers_type() function will always terminate. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #1045 Closes #861 Closes #790
2013-07-16 00:37:51 +04:00
])
])
dnl #
dnl # 3.3 API change
dnl # The super_block structure was changed to use an hlist_node instead
dnl # of a list_head for the .s_instance linkage.
dnl #
dnl # This was done in part to resolve a race in the iterate_supers_type()
dnl # function which was introduced in Linux 3.0 kernel. The iterator
dnl # was supposed to provide a safe way to call an arbitrary function on
dnl # all super blocks of a specific type. Unfortunately, because a
dnl # list_head was used it was possible for iterate_supers_type() to
dnl # get stuck spinning a super block which was just deactivated.
dnl #
dnl # This can occur because when the list head is removed from the
dnl # fs_supers list it is reinitialized to point to itself. If the
dnl # iterate_supers_type() function happened to be processing the
dnl # removed list_head it will get stuck spinning on that list_head.
dnl #
dnl # To resolve the issue for existing 3.0 - 3.2 kernels we detect when
dnl # a list_head is used. Then to prevent the spinning from occurring
dnl # the .next pointer is set to the fs_supers list_head which ensures
dnl # the iterate_supers_type() function will always terminate.
dnl #
AC_DEFUN([ZFS_AC_KERNEL_S_INSTANCES_LIST_HEAD], [
AC_MSG_CHECKING([whether super_block has s_instances list_head])
ZFS_LINUX_TRY_COMPILE([
#include <linux/fs.h>
],[
struct super_block sb __attribute__ ((unused));
INIT_LIST_HEAD(&sb.s_instances);
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_S_INSTANCES_LIST_HEAD, 1,
[struct super_block has s_instances list_head])
],[
AC_MSG_RESULT(no)
Linux 3.1 compat, super_block->s_shrink The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #466 Closes #292
2011-12-23 00:20:43 +04:00
])
])
AC_DEFUN([ZFS_AC_KERNEL_NR_CACHED_OBJECTS], [
AC_MSG_CHECKING([whether sops->nr_cached_objects() exists])
ZFS_LINUX_TRY_COMPILE([
#include <linux/fs.h>
Eliminate runtime function pointer mods in autotools checks PaX/GrSecurity patched kernels implement a dialect of C that relies on a GCC plugin for enforcement. A basic idea in this dialect is that function pointers in structures should not change during runtime. This causes code that modifies function pointers at runtime to fail to compile in many instances. The autotools checks rely on whether or not small test cases compile against a given kernel. Some autotools checks assume some default case if other cases fail. When one of these autotools checks tests a PaX/GrSecurity patched kernel by modifying a function pointer at runtime, the default case will be used. Early detection of such situations is possible by relying on compiler warnings, which are compiler errors when --enable-debug is used. Unfortunately, very few people build ZFS with --enable-debug. The more common situation is that these issues manifest themselves as runtime failures in the form of NULL pointer exceptions. Previous patches that addressed such issues with PaX/GrSecurity compatibility largely relied on rewriting autotools checks to avoid runtime function pointer modification or the addition of PaX/GrSecurity specific checks. This patch takes the previous work to its logical conclusion by eliminating the use of runtime function pointer modification. This permits the removal of PaX-specific autotools checks in favor of ones that work across all supported kernels. This should resolve issues that were reported to occur with PaX/GrSecurity-patched Linux 3.7.5 kernels on Gentoo Linux. https://bugs.gentoo.org/show_bug.cgi?id=457176 We should be able to prevent future regressions in PaX/GrSecurity compatibility by ensuring that all changes to ZFSOnLinux avoid runtime function pointer modification. At the same time, this does not solve the issue of silent failures triggering default cases in the autotools check, which is what permitted these regressions to become runtime failures in the first place. This will need to be addressed in a future patch. Reported-by: Marcin Mirosław <bug@mejor.pl> Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #1300
2013-02-15 03:54:04 +04:00
int nr_cached_objects(struct super_block *sb) { return 0; }
static const struct super_operations
sops __attribute__ ((unused)) = {
Linux 3.1 compat, super_block->s_shrink The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #466 Closes #292
2011-12-23 00:20:43 +04:00
.nr_cached_objects = nr_cached_objects,
};
Eliminate runtime function pointer mods in autotools checks PaX/GrSecurity patched kernels implement a dialect of C that relies on a GCC plugin for enforcement. A basic idea in this dialect is that function pointers in structures should not change during runtime. This causes code that modifies function pointers at runtime to fail to compile in many instances. The autotools checks rely on whether or not small test cases compile against a given kernel. Some autotools checks assume some default case if other cases fail. When one of these autotools checks tests a PaX/GrSecurity patched kernel by modifying a function pointer at runtime, the default case will be used. Early detection of such situations is possible by relying on compiler warnings, which are compiler errors when --enable-debug is used. Unfortunately, very few people build ZFS with --enable-debug. The more common situation is that these issues manifest themselves as runtime failures in the form of NULL pointer exceptions. Previous patches that addressed such issues with PaX/GrSecurity compatibility largely relied on rewriting autotools checks to avoid runtime function pointer modification or the addition of PaX/GrSecurity specific checks. This patch takes the previous work to its logical conclusion by eliminating the use of runtime function pointer modification. This permits the removal of PaX-specific autotools checks in favor of ones that work across all supported kernels. This should resolve issues that were reported to occur with PaX/GrSecurity-patched Linux 3.7.5 kernels on Gentoo Linux. https://bugs.gentoo.org/show_bug.cgi?id=457176 We should be able to prevent future regressions in PaX/GrSecurity compatibility by ensuring that all changes to ZFSOnLinux avoid runtime function pointer modification. At the same time, this does not solve the issue of silent failures triggering default cases in the autotools check, which is what permitted these regressions to become runtime failures in the first place. This will need to be addressed in a future patch. Reported-by: Marcin Mirosław <bug@mejor.pl> Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #1300
2013-02-15 03:54:04 +04:00
],[
Linux 3.1 compat, super_block->s_shrink The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #466 Closes #292
2011-12-23 00:20:43 +04:00
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_NR_CACHED_OBJECTS, 1,
[sops->nr_cached_objects() exists])
],[
AC_MSG_RESULT(no)
])
])
AC_DEFUN([ZFS_AC_KERNEL_FREE_CACHED_OBJECTS], [
AC_MSG_CHECKING([whether sops->free_cached_objects() exists])
ZFS_LINUX_TRY_COMPILE([
#include <linux/fs.h>
Eliminate runtime function pointer mods in autotools checks PaX/GrSecurity patched kernels implement a dialect of C that relies on a GCC plugin for enforcement. A basic idea in this dialect is that function pointers in structures should not change during runtime. This causes code that modifies function pointers at runtime to fail to compile in many instances. The autotools checks rely on whether or not small test cases compile against a given kernel. Some autotools checks assume some default case if other cases fail. When one of these autotools checks tests a PaX/GrSecurity patched kernel by modifying a function pointer at runtime, the default case will be used. Early detection of such situations is possible by relying on compiler warnings, which are compiler errors when --enable-debug is used. Unfortunately, very few people build ZFS with --enable-debug. The more common situation is that these issues manifest themselves as runtime failures in the form of NULL pointer exceptions. Previous patches that addressed such issues with PaX/GrSecurity compatibility largely relied on rewriting autotools checks to avoid runtime function pointer modification or the addition of PaX/GrSecurity specific checks. This patch takes the previous work to its logical conclusion by eliminating the use of runtime function pointer modification. This permits the removal of PaX-specific autotools checks in favor of ones that work across all supported kernels. This should resolve issues that were reported to occur with PaX/GrSecurity-patched Linux 3.7.5 kernels on Gentoo Linux. https://bugs.gentoo.org/show_bug.cgi?id=457176 We should be able to prevent future regressions in PaX/GrSecurity compatibility by ensuring that all changes to ZFSOnLinux avoid runtime function pointer modification. At the same time, this does not solve the issue of silent failures triggering default cases in the autotools check, which is what permitted these regressions to become runtime failures in the first place. This will need to be addressed in a future patch. Reported-by: Marcin Mirosław <bug@mejor.pl> Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #1300
2013-02-15 03:54:04 +04:00
void free_cached_objects(struct super_block *sb, int x)
{ return; }
static const struct super_operations
sops __attribute__ ((unused)) = {
Linux 3.1 compat, super_block->s_shrink The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #466 Closes #292
2011-12-23 00:20:43 +04:00
.free_cached_objects = free_cached_objects,
};
Eliminate runtime function pointer mods in autotools checks PaX/GrSecurity patched kernels implement a dialect of C that relies on a GCC plugin for enforcement. A basic idea in this dialect is that function pointers in structures should not change during runtime. This causes code that modifies function pointers at runtime to fail to compile in many instances. The autotools checks rely on whether or not small test cases compile against a given kernel. Some autotools checks assume some default case if other cases fail. When one of these autotools checks tests a PaX/GrSecurity patched kernel by modifying a function pointer at runtime, the default case will be used. Early detection of such situations is possible by relying on compiler warnings, which are compiler errors when --enable-debug is used. Unfortunately, very few people build ZFS with --enable-debug. The more common situation is that these issues manifest themselves as runtime failures in the form of NULL pointer exceptions. Previous patches that addressed such issues with PaX/GrSecurity compatibility largely relied on rewriting autotools checks to avoid runtime function pointer modification or the addition of PaX/GrSecurity specific checks. This patch takes the previous work to its logical conclusion by eliminating the use of runtime function pointer modification. This permits the removal of PaX-specific autotools checks in favor of ones that work across all supported kernels. This should resolve issues that were reported to occur with PaX/GrSecurity-patched Linux 3.7.5 kernels on Gentoo Linux. https://bugs.gentoo.org/show_bug.cgi?id=457176 We should be able to prevent future regressions in PaX/GrSecurity compatibility by ensuring that all changes to ZFSOnLinux avoid runtime function pointer modification. At the same time, this does not solve the issue of silent failures triggering default cases in the autotools check, which is what permitted these regressions to become runtime failures in the first place. This will need to be addressed in a future patch. Reported-by: Marcin Mirosław <bug@mejor.pl> Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #1300
2013-02-15 03:54:04 +04:00
],[
Linux 3.1 compat, super_block->s_shrink The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #466 Closes #292
2011-12-23 00:20:43 +04:00
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_FREE_CACHED_OBJECTS, 1,
[sops->free_cached_objects() exists])
],[
AC_MSG_RESULT(no)
])
])
3.12 compat, NUMA-aware per-superblock shrinker Kernels >= 3.12 have a NUMA-aware superblock shrinker which is used in ZoL by zfs_sb_prune(). This patch calls the shrinker for each on-line NUMA node in order that memory be freed for each one. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #3495
2015-06-14 19:19:40 +03:00
dnl #
dnl # 3.12 API change
dnl # The nid member was added to struct shrink_control to support
dnl # NUMA-aware shrinkers.
dnl #
AC_DEFUN([ZFS_AC_KERNEL_SHRINK_CONTROL_HAS_NID], [
AC_MSG_CHECKING([whether shrink_control has nid])
ZFS_LINUX_TRY_COMPILE([
#include <linux/fs.h>
],[
struct shrink_control sc __attribute__ ((unused));
unsigned long scnidsize __attribute__ ((unused)) =
sizeof(sc.nid);
],[
AC_MSG_RESULT(yes)
AC_DEFINE(SHRINK_CONTROL_HAS_NID, 1,
[struct shrink_control has nid])
],[
AC_MSG_RESULT(no)
])
])
Update build system and packaging Minimal changes required to integrate the SPL sources in to the ZFS repository build infrastructure and packaging. Build system and packaging: * Renamed SPL_* autoconf m4 macros to ZFS_*. * Removed redundant SPL_* autoconf m4 macros. * Updated the RPM spec files to remove SPL package dependency. * The zfs package obsoletes the spl package, and the zfs-kmod package obsoletes the spl-kmod package. * The zfs-kmod-devel* packages were updated to add compatibility symlinks under /usr/src/spl-x.y.z until all dependent packages can be updated. They will be removed in a future release. * Updated copy-builtin script for in-kernel builds. * Updated DKMS package to include the spl.ko. * Updated stale AUTHORS file to include all contributors. * Updated stale COPYRIGHT and included the SPL as an exception. * Renamed README.markdown to README.md * Renamed OPENSOLARIS.LICENSE to LICENSE. * Renamed DISCLAIMER to NOTICE. Required code changes: * Removed redundant HAVE_SPL macro. * Removed _BOOT from nvpairs since it doesn't apply for Linux. * Initial header cleanup (removal of empty headers, refactoring). * Remove SPL repository clone/build from zimport.sh. * Use of DEFINE_RATELIMIT_STATE and DEFINE_SPINLOCK removed due to build issues when forcing C99 compilation. * Replaced legacy ACCESS_ONCE with READ_ONCE. * Include needed headers for `current` and `EXPORT_SYMBOL`. Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Olaf Faaland <faaland1@llnl.gov> Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: Pavel Zakharov <pavel.zakharov@delphix.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> TEST_ZIMPORT_SKIP="yes" Closes #7556
2018-02-16 04:53:18 +03:00
AC_DEFUN([ZFS_AC_KERNEL_SHRINKER_CALLBACK],[
tmp_flags="$EXTRA_KCFLAGS"
EXTRA_KCFLAGS="-Werror"
dnl #
dnl # 2.6.23 to 2.6.34 API change
dnl # ->shrink(int nr_to_scan, gfp_t gfp_mask)
dnl #
AC_MSG_CHECKING([whether old 2-argument shrinker exists])
ZFS_LINUX_TRY_COMPILE([
#include <linux/mm.h>
int shrinker_cb(int nr_to_scan, gfp_t gfp_mask);
],[
struct shrinker cache_shrinker = {
.shrink = shrinker_cb,
.seeks = DEFAULT_SEEKS,
};
register_shrinker(&cache_shrinker);
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_2ARGS_OLD_SHRINKER_CALLBACK, 1,
[old shrinker callback wants 2 args])
],[
AC_MSG_RESULT(no)
dnl #
dnl # 2.6.35 - 2.6.39 API change
dnl # ->shrink(struct shrinker *,
dnl # int nr_to_scan, gfp_t gfp_mask)
dnl #
AC_MSG_CHECKING([whether old 3-argument shrinker exists])
ZFS_LINUX_TRY_COMPILE([
#include <linux/mm.h>
int shrinker_cb(struct shrinker *, int nr_to_scan,
gfp_t gfp_mask);
],[
struct shrinker cache_shrinker = {
.shrink = shrinker_cb,
.seeks = DEFAULT_SEEKS,
};
register_shrinker(&cache_shrinker);
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_3ARGS_SHRINKER_CALLBACK, 1,
[old shrinker callback wants 3 args])
],[
AC_MSG_RESULT(no)
dnl #
dnl # 3.0 - 3.11 API change
dnl # ->shrink(struct shrinker *,
dnl # struct shrink_control *sc)
dnl #
AC_MSG_CHECKING(
[whether new 2-argument shrinker exists])
ZFS_LINUX_TRY_COMPILE([
#include <linux/mm.h>
int shrinker_cb(struct shrinker *,
struct shrink_control *sc);
],[
struct shrinker cache_shrinker = {
.shrink = shrinker_cb,
.seeks = DEFAULT_SEEKS,
};
register_shrinker(&cache_shrinker);
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_2ARGS_NEW_SHRINKER_CALLBACK, 1,
[new shrinker callback wants 2 args])
],[
AC_MSG_RESULT(no)
dnl #
dnl # 3.12 API change,
dnl # ->shrink() is logically split in to
dnl # ->count_objects() and ->scan_objects()
dnl #
AC_MSG_CHECKING(
[whether ->count_objects callback exists])
ZFS_LINUX_TRY_COMPILE([
#include <linux/mm.h>
unsigned long shrinker_cb(
struct shrinker *,
struct shrink_control *sc);
],[
struct shrinker cache_shrinker = {
.count_objects = shrinker_cb,
.scan_objects = shrinker_cb,
.seeks = DEFAULT_SEEKS,
};
register_shrinker(&cache_shrinker);
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_SPLIT_SHRINKER_CALLBACK,
1, [->count_objects exists])
],[
AC_MSG_ERROR(error)
])
])
])
])
EXTRA_KCFLAGS="$tmp_flags"
])
dnl #
dnl # 2.6.39 API change,
dnl # Shrinker adjust to use common shrink_control structure.
dnl #
AC_DEFUN([ZFS_AC_KERNEL_SHRINK_CONTROL_STRUCT], [
AC_MSG_CHECKING([whether struct shrink_control exists])
ZFS_LINUX_TRY_COMPILE([
#include <linux/mm.h>
],[
struct shrink_control sc __attribute__ ((unused));
sc.nr_to_scan = 0;
sc.gfp_mask = GFP_KERNEL;
],[
AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_SHRINK_CONTROL_STRUCT, 1,
[struct shrink_control exists])
],[
AC_MSG_RESULT(no)
])
])
Reference in New Issue Copy Permalink