What is: “Wasted Memory”? (Zend OpCache & Memcached)

What is: “Wasted Memory”? (Zend OpCache & Memcached)

As I have been trying to fine tune all the performance settings on our new Linux box there was a new term we came across: “Wasted Memory”. Zend OpCache has a setting that will cause the restart of your OpCache if the wasted memory exceeds a threshold. I am no expert but that would result in losing all your cached information in OpCache.

So, what is “Wasted Memory”? From what I have read it is memory that has been allocated to the service but is not in use. As your memory needs increase so does the memory allocated to you. However, once you release information from the cache that memory is still in use. Wasted Memory is therefore memory you used in the past that right now contains information that is of no use therefore it is considered wasted.

Same goes for Memcache. Wasted memory is memory allocated in RAM so it cannot be used by another application, but it still is available for Memcache although not used at the moment.

So why would there be a function to restart the service you ask? Well, say you had limited memory, so by restarting the service you would free all the memory again. The downside is that your currently cached information is lost but the advantage is that you start fresh. Theoretically this would not happen often. You can adjust the value of % of Wasted Memory to better suit your needs, perhaps you want to be more tolerant than the 5% that comes out of the box or perhaps you don’t want to use it at all. It’s all up to you!

Observed Behavior

I decided to play a little with OpCache to see if a restart is such a bad thing or not. Apparently about 60% of the cache I had got re-established in an instant and another 20% in the next few seconds. Probably it tries to preload as much as possible when it starts, not sure, but this is a good sign. I was wondering why it was set to 5% waste memory by default but a restart seems not a bad thing now. I am going to see how it behaves once there is no free memory and it needs to start using the wasted memory.

UPDATE 1: I already managed to deplete the available memory but still no dice. Wasted memory continues to slowly increase while used memory is decreasing which I assume is not a good sign. To further indicate this might be a problem the % of hits has gone down. I went ahead and further restricted the amount of memory available to OpCache. The Memory quickly filled and the waste memory slowly grew.

I researched a little bit more and found the following information:

memory_usage – array; contains information about Optimizer+ memory usage, with the following keys:
used_memory – integer; bytes of memory used
free_memory – integer; bytes of memory available for cache
wasted_memory – integer; bytes of memory used by invalid or outdated code
Wasted memory is reclaimed when the accelerator is reset, or when the percentage of wasted memory reaches the value of the max_wasted_percentage directive
current_wasted_percentage – float; The percentage of wasted opcode cache memory out of total memory available

As noted Wasted memory is reclaimed when the accelerator is reset, or when the percentage of wasted memory reaches the value of the max_wasted_percentage directive (which triggers a reset). I don’t understand why it works that way but who am I to argue. Back to opcache.max_wasted_percentage = 5.  Also another thing I noted is the opcache.use_cwd (default “1”) setting. I recommend leaving it enabled as having multiple WordPress installations on your server would result in collisions (or multiple anything on server as the file names would be the same.) I’ve included below the most common directives, their definitions and default values:

Configuration Directives
------------------------

opcache.enable (default "1")
	OPcache On/Off switch. When set to Off, code is not optimized and cached.

opcache.enable_cli (default "0")
	Enables the OPcache for the CLI version of PHP. It's mostly for testing
	and debugging.

opcache.memory_consumption (default "64")
	The OPcache shared memory storage size. The amount of memory for storing
	precompiled PHP code in Mbytes.

opcache.interned_strings_buffer (default "4")
	The amount of memory for interned strings in Mbytes.

opcache.max_accelerated_files (default "2000")
	The maximum number of keys (scripts) in the OPcache hash table.
	The number is actually the first one in the following set of prime
	numbers that is bigger than the one supplied: { 223, 463, 983, 1979, 3907,
	7963, 16229, 32531, 65407, 130987, 262237, 524521, 1048793 }. Only numbers
	between 200 and 1000000 are allowed.

opcache.max_wasted_percentage (default "5")
	The maximum percentage of "wasted" memory until a restart is scheduled.

opcache.use_cwd (default "1")
	When this directive is enabled, the OPcache appends the current working
	directory to the script key, thus eliminating possible collisions between
	files with the same name (basename). Disabling the directive improves
	performance, but may break existing applications.

opcache.validate_timestamps (default "1")
	When disabled, you must reset the OPcache manually or restart the
	webserver for changes to the filesystem to take effect.
	The frequency of the check is controlled by the directive
	"opcache.revalidate_freq".

opcache.revalidate_freq (default "2")
	How often (in seconds) to check file timestamps for changes to the shared
	memory storage allocation. ("1" means validate once per second, but only
	once per request. "0" means always validate)

opcache.file_update_protection (default "2")
	Prevents caching files that are less than this number of seconds old.
	It protects from caching of incompletely updated files. In case all file
	updates on your site are atomic, you may increase performance setting it
	to "0".

opcache.revalidate_path (default "0")
	Enables or disables file search in include_path optimization
	If the file search is disabled and a cached file is found that uses
	the same include_path, the file is not searched again. Thus, if a file
	with the same name appears somewhere else in include_path, it
	won't be found. Enable this directive if this optimization has an effect on
	your applications. The default for this directive is disabled, which means
	that optimization is active.

opcache.save_comments (default "1")
	If disabled, all PHPDoc comments are dropped from the code to reduce the
	size of the optimized code. Disabling "Doc Comments" may break some
	existing applications and frameworks (e.g. Doctrine, ZF2, PHPUnit)

opcache.load_comments (default "1")
	If disabled, PHPDoc comments are not loaded from SHM, so "Doc Comments"
	may be always stored (save_comments=1), but not loaded by applications
	that don't need them anyway.

opcache.fast_shutdown (default "0")
	If enabled, a fast shutdown sequence is used for the accelerated code
	The fast shutdown sequence doesn't free each allocated block, but lets
	the Zend Engine Memory Manager do the work.

opcache.enable_file_override (default "0")
	Allow file existence override (file_exists, etc.) performance feature.

opcache.optimization_level (default "0xffffffff")
	A bitmask, where each bit enables or disables the appropriate OPcache
	passes

opcache.inherited_hack (default "1")
	Enable this hack as a workaround for "can't redeclare class" errors.
	The OPcache stores the places where DECLARE_CLASS opcodes use
	inheritance (These are the only opcodes that can be executed by PHP,
	but which may not be executed because the parent class is missing due to
	optimization). When the file is loaded, OPcache tries to bind the
	inherited classes by using the current environment. The problem with this
	scenario is that, while the DECLARE_CLASS opcode may not be needed for the
	current script, if the script requires that the opcode at least be defined,
	it may not run. The default for this directive is disabled, which means
	that optimization is active. In php-5.3 and above this hack is not needed
	anymore and this setting has no effect.

opcache.dups_fix (default "0")
	Enable this hack as a workaround for "Cannot redeclare class" errors.

opcache.blacklist_filename
	The location of the OPcache blacklist file (wildcards allowed).
	Each OPcache blacklist file is a text file that holds the names of files
	that should not be accelerated. The file format is to add each filename
	to a new line. The filename may be a full path or just a file prefix
	(i.e., /var/www/x  blacklists all the files and directories in /var/www
	that start with 'x'). Line starting with a ; are ignored (comments).
	Files are usually triggered by one of the following three reasons:
	1) Directories that contain auto generated code, like Smarty or ZFW cache.
	2) Code that does not work well when accelerated, due to some delayed
	   compile time evaluation.
	3) Code that triggers an OPcache bug.

opcache.max_file_size (default "0")
	Allows exclusion of large files from being cached. By default all files
	are cached.

opcache.consistency_checks (default "0")
	Check the cache checksum each N requests.
	The default value of "0" means that the checks are disabled.
	Because calculating the checksum impairs performance, this directive should
	be enabled only as part of a debugging process.

opcache.force_restart_timeout (default "180")
	How long to wait (in seconds) for a scheduled restart to begin if the cache
	is not being accessed.
	The OPcache uses this directive to identify a situation where there may
	be a problem with a process. After this time period has passed, the
	OPcache assumes that something has happened and starts killing the
	processes that still hold the locks that are preventing a restart.
	If the log level is 3 or above, a "killed locker" error is recorded
	in the Apache logs when this happens.

opcache.error_log
	OPcache error_log file name. Empty string assumes "stderr".

opcache.log_verbosity_level (default "1")
	All OPcache errors go to the Web server log.
	By default, only fatal errors (level 0) or errors (level 1) are logged.
	You can also enable warnings (level 2), info messages (level 3) or
	debug messages (level 4).

opcache.preferred_memory_model
	Preferred Shared Memory back-end. Leave empty and let the system decide.

opcache.protect_memory (default "0")
	Protect the shared memory from unexpected writing during script execution.
	Useful for internal debugging only.

opcache.restrict_api (default "")
	Allows calling OPcache API functions only from PHP scripts which path is
	started from specified string. The default "" means no restriction.

opcache.mmap_base
	Mapping base of shared memory segments (for Windows only). All the PHP
	processes have to map shared memory into the same address space. This
	directive allows to manually fix the "Unable to reattach to base address"
	errors.