to dynamically optimize the in-memory machine
code of a running process.
modern English: from a combination of
dynamic and optimize.
Dynimize is a user mode program that quickly optimizes the in-memory machine
code of target processes based on profiling information it gathers at run-time,
thereby improving performance for some workloads. Dynimize typically runs as a background
system process and does not require the source code of target programs that it optimizes.
2. Installation & Quick Start
To install Dynimize, run the following commands in a Linux terminal.
wget https://dynimize.com/install -O install
wget https://dynimizecloud.com/install.sha256 -O install.sha256
sha256sum -c install.sha256; if [ $? -eq 0 ]; then sudo bash ./install; fi
Use your access token to start a subscription license for your host.
sudo dyni -license=start -token=<access token>
To optimize any CPU intensive process who's exe is listed in the [exeList] section of /etc/dyni.conf, run:
sudo dyni -start
The command dyni -status will show target processes progressing from the "profiling",
to "dynimizing", and then "dynimized" states. A process has been fully optimized once
in the "dynimized" state:
$ sudo dyni -status
Dynimize is running
mysqld, pid: 8375, dynimizing
$ sudo dyni -status
Dynimize is running
mysqld, pid: 8375, dynimized
Dynimize can be shutdown with the following command:
$ sudo dyni -stop
If Dynimize is shutdown, all dynimized processes will remain dynimized.
To "undynimize" all dynimized processes, run
sudo dyni -undo
or alternatively restart the dynimized process. For example, the following command can restart the mysqld process for MySQL:
service mysql restart
To cancel a subscription license for a given host, run the following command on that host:
sudo dyni -license=stop -token=<access token>
To uninstall Dynimize, run:
3. System Requirements
• x86-64 Linux Kernel version 2.6.32 or later
• 4GB virtual memory (swap + RAM) per target process being
dynimized in parallel. This is because Dynimize requires 4 GB per target process in the
"dynimizing" phase, releasing the memory after that.
• Above virtual memory requirement increased to 6GB (swap + RAM)
per target process being dynimized when the target is a MySQL 8 mysqld process.
3.1 VM specific requirements
There are no specific requirements for running inside KVM,
Xen or VMware virtual machines. Therefore Dynimize can be run in most cloud environments.
3.2 Docker specific requiremets
Dynimize must be run in the same container as the target process being optimized.
When run, the dyni process requires the ability to use the ptrace(),
and perf_event_open() system calls to profile and interface with
the target process being optimized. This requires the dyni process to be run with the
CAP_SYS_ADMIN and CAP_SYS_PTRACE
capabilities. Docker as of the time of this writing (Docker version 18) disables these system calls
and capabilities by default. The simplest way to enable them is by starting a docker container
with the --privileged flag. For example,
$run -it --privileged my_image.
Alternatively, see Docker's documentation on seccomp profiles for specifically enabling just
these system calls.
4. Command Line Options
Print the help message.
Start Dynimize as a background daemon process.
Stop any invocation of Dynimize that is currently running.
Print the state or phase of Dynimize and its optimized processes.
States will be listed in the following format:
target process executable name, pid, state.
Stop any other invocation of Dynimize that is already running, remove all optimizations already performed by Dynimize on target processes, and then exit.
Dynimize can then be started at a later time and used to dynimize the "undynimized" processes again under different workload conditions.
The same operation as -undo, except that this invocation of Dynimize will continue running,
and may now proceed to automatically optimize the same processes again, along with other processes.
Allow Dynimize to automatically redynimize a process if it detects that its workload has drastically changed. This option
is disabled by default. Reoptimization can only be performed if the previous optimization was done by the same Dynimize process without Dynimize being restarted.
See the -pid=<PID> option on how -reoptimize affects it.
Print the Dynimize version.
Display product information.
Specify a full pathname for the output log.
Reduce the amount of time required to dynimize a process.
This may limit the final speedup of any processes dynimized this way.
Reduce peak profiling overhead for target processes. This may limit the final
speedup of any processes dynimized this way.
The current Dynimize command will only apply to processes run from this executable file name. Do not specify the full pathname.
The current Dynimize command will only apply to the process specified by this PID. This will cause Dynimize
to exit after dynimizing a process unless used in conjunction with
Download and apply any Dynimize updates.
Report if there are any Dynimize updates to apply.
Skip any prompts verifying the actions to be performed.
• -token=<access token>
Specify your account access token for use with any licensing actions.
Start a subscription license for the current host.
Stop the subscription license for the current host.
Create a permanent license for the current host.
The account used will be billed for this license at the moment this command is issued,
rather than at the end of the month like subscription licenses.
Download a copy of the license key into /opt/dynimize/license-key
Display the local license information of the current host system.
Display the license information of every host system associated with this account.
• -name=<license name>
Specify a name for the license associated with this host.
Include this option on the client and server side if you are running your own Dynimize license server, to specify the port number used by your Dynimize license server.
If not specified then the default port used is 443.
See Licensing isolated hosts.
Include this option on the client side if you are running your own Dynimize license server to specify the IP address or hostname of your Dynimize license server.
If not specified then the default license host used is license.dynimize.com.
See Licensing isolated hosts.
*Note that the output log will list all options enabled at Dynimize startup and can be used to verify which options are in effect.
5. Configuration File
On startup, Dynimize is configured based on the settings in /etc/dyni.conf.
The following is a description of these settings. Many of these settings can be
overridden by command line options. Note that starting a line with the # character
will cause that line to be skipped.
All options follow the line in the .conf file containing the string
[options]. If any of those following options are missing,
their default settings are used. Note that the < | >
characters should not actually be placed in the log file:
Same as the command line option ‑reoptimize
Same as the command line option -log
The max total size of both log files at <path> and <path>.old combined.
Size is in bytes unless specified by MB, M, GB, G, KB, K. The defaults is 1 MB.
Same as the command line option ‑fastCompile
Same as the command line option ‑lowOverhead
If present, Dynimize will be enabled as service in /etc/init.d and launched on system startup.
If present, Dynimize will automatically check for Dynimize updates and apply them.
Same as the command line option ‑skipPrompt
Same as the command line option ‑token=<access token>
If expired, automatically update the license key in /opt/dynimize/license-key.
Same as the command line option ‑licenseServer
Same as the command line option ‑licensePort=<port>
Same as the command line option ‑licenseHost=<host>
The lines following [exeList] in dyni.conf
denote the whitelist of possible optimization targets. These are the names of
the executables that are used to launch the processes that can be targeted by
Dynimize, one name per line. So for example, the MySQL server process would
be denoted as mysqld. A process must have been started by an executable on
this list to be dynimized. Note that if a symbolic or hard link is used to call
an executable, the real executable file name must be specified on the
The lines following [users] list the possible user
names of the owners of processes that can be targeted by Dynimize, one name per line.
If present, a process must match both the [exeList] and
[users] list in order to be dynimized.
Default: no user name requirement.
The output log specified by -log=<pathname> records all errors and Dynimize status updates
along with their timestamps. It will also list all options enabled at Dynimize startup which can
be used to verify the options that are in effect. On installation the log path is set to /var/log/dyni.log
in /etc/dyni.conf. Once the log file reaches half the maxLogSize, it is moved to the
same path name appended with .old, and a new log is started as the original log file name.
Logging is disabled if the log option is removed from dyni.conf.
6. Dynimize Usage Examples
Start Dynimize as a background daemon process, dynimize any processes encountered,
and then continue to run and target new processes that are CPU intensive and on the [exeList] in /etc/dyni.conf.
$ sudo dyni -start
Dynimize a process with pid 9618 and exit once dynimized:
$ sudo dyni -start -pid=9618
Launch Dynimize as a foreground process, only dynimizing process 9618
and then exit. Note that the -start option is excluded here so that
Dynimize is not launched in the background, which can be useful for
running Dynimize from shell scripts that wait for Dynimize to finish before performing the next task:
$ sudo dyni -pid=9618
Launch Dynimize, only dynimizing process 9618, however
in this case Dynimize will continue to run after the process has entered
the "dynimized" state so that it can reoptimize the process if its workload
$ sudo dyni -start -pid=9618 -reoptimize
The following example will stop any other running instances of Dynimize, remove any optimizations performed by previous Dynimize
invocations, and then continue running. Dynimize may now proceed to optimize these processes again, along
with any other processes. This could be useful if you think that your workload has drastically changed and want
it reoptimized using current profiling information:
$ sudo dyni -start -redo
Only dynimize processes run from the myprog executable:
$ sudo dyni -start -exe=myprog
Dynimize processes and shorten the amount of time spent in the
$ sudo dyni -start -fastCompile
Start Dynimize with a new log path directory /tmp/log:
$ sudo dyni -start -log=/tmp/log
7. Licensing & Billing
One of the primary purposes of Dynimize is to reduce the financial costs of CPU resources consumed by target workloads for a given level of performance.
Therefore to help users reduce billing costs in the cloud, elastic, on-demand subscriptions can be generated on the command line to license Dynimize
with per second billing granularity, thereby matching the billing granularity of CPU resources in many public clouds and allowing for easy licensing automation.
To cancel a host's subscription license from the command line,
Dynimize must be shut down and there cannot be any dynimized processes running on that host.
To stop running instances of Dynimize and "undynimize" all dynimized processes, run
sudo dyni -undo
or you can simply stop Dynimize and restart any dynimized processes to "naturally" undynimize them. For example, the following commands can restart the mysqld process for MySQL after stopping Dynimize.
sudo dyni -stop
service mysql restart
To then cancel a subscription license for a given host, run the following command on that host.
sudo dyni -license=stop -token=<access token>
By default, time is accounted for using per second granularity, but only when the license is later correctly cancelled
from the command line. See Billing time granularity for exceptions.
Time accounting for any licenses used only begins 7 days after your account was activated.
This initial period is your free trial, after which each license is then priced at $1 per
CPU core per 30 days, prorated per second of usage when cancelled
from the command line on the licensed host.
For example, if you only have 1 hour and 48 minutes of use over the course of a month on a
4 core host, you will only be invoiced $0.01. Your account will be billed at the end of each
month for subscription license time used during that month.
7.1 Fixed licenses
A fixed license is only billed once per host, is priced at $24 per CPU core, and will support a single host indefinitely.
You can start a fixed license with the following command.
sudo dyni -license=fixed -token=<access token>
A fixed license can also be created on a host anytime after a subscription license was started.
In that case, the subscription license is implicitly stopped and no subscription time will be
billed past that point for that host. A fixed license is then created at the standard price.
7.2 Viewing licenses
To view all the active licenses associated with your account, run
sudo dyni -info=all -token=<access token>
To only view the license on the current host, run
sudo dyni -info=this -token=<access token>
7.3 Storing your access token
Your access token in the option -token=<access token> acts as the credentials for accessing
the Dynimize licensing system. To prevent it from being stored in your bash history, you may want to
change your access token and then use that new value in the
configuration file /etc/dyni.conf by placing it in the token=<access token> field.
You can then omit -token=<access token> from the command line. Note that any use of
-token=<access token> on the command line will automatically update the value in /etc/dyni.conf.
7.4 License key expiration
Licensing commands will store the current license key in /opt/dynimize/license-key.
While fixed license keys never expire, subscription license keys will report that Dynimize
is expired 30 days after license key creation, and prevent Dynimize from starting or performing further
optimizations (if already running) after 60 days from license key creation. If the flag autoRenew is set in /etc/dyni.conf
and the server has internet access or network access to a
self hosted license server,
Dynimize will automatically update the license key every 30 days.
This provides an additional 30 day window to address any issues encountered renewing the license key.
7.5 Anonymous bookkeeping
The licensing system creates a relatively unique identifier for each licensed host.
This identifier combines system values such as the MAC address and the number of CPU cores.
This information is combined into a 32-bit identifier that contains a fraction of
the total bits of the original system values, in such a way that those original values cannot be extracted
from this unique identifier. This allows Dynimize to associate a license with a specific host without
transmitting any of the real system values of that host, maintaining a degree of anonymity and privacy.
This identifier is then used to allow the licensing system to ignore redundant licensing commands.
For example, starting a subscription license on a host that already has a fixed or subscription
license will have no effect.
Note that all communication between a host and a license server is encrypted through TLS.
This communication only happens when a licensing command is issued by the user, or a subscription license
key is automatically updated once expired, which is once every 30 days only if autoRenew is enabled and a
subscription license is in use. Dynimize does not require a network connection to the licensing system
outside of those times.
7.6 Billing time granularity
Dynimize time accounting is performed with per second billing granularity only when a subscription
license is stopped from the command line. However, if /opt/dynimize/dyni-node is modified on the host at
any point after license key creation, per second billing granularity will not be possible for the
next licensing command and the host may continue to accrue billable time for an additional 30 days
after the subscription license is stopped from the command line. Additionally, if the MAC address
or number of CPU cores on the licensed host has changed, Dynimize licensing commands will fail to
recognize the host system and the subscription license will need to be stopped from the web portal,
where per second billing granularity cannot be used. In this case, Dynimize will continue to accrue
billable time for an additional 30 days after the subscription license is stopped from the web portal.
IMPORTANT: For the above reasons, it is important to stop a
subscription license from the command line of any cloud instance before terminating that instance.
This is because any new instance will have a different MAC address, requiring the
original instance license to be stopped from the web portal and thereby forfeiting per second
7.7 Licensing isolated hosts
Some security policies do not allow specific servers access to the internet, but allow them to reach
internal servers that do have internet access. In this case you can run Dynimize as a licensing
server on your own system that does have internet access and is reachable by these isolated servers
you wish to license.
To run Dynimize as a licensing server background process, run the following command on the host with internet access.
sudo dyni -start -licenseServer
Alternatively, you can set licenseServer in /etc/dyni.conf
You can then run any licensing command on
the isolated servers that can only reach the above host using the
-licenseHost=<IP|host> flag. For example, to start a subscription
license on an internet isolated host assuming the above self-hosted license server has an IP address of 10.30.1.72,
run the following command.
You can also set licenseHost=<IP|host>
in /etc/dyni.conf on the isolated servers.
Note that the hostname can be used in place of the IP address in the above example.
By default, all licensing commands use port 443. To change this in the above example, on both systems use the command line option
-licensePort=<port number> or set
licensePort=<port number> in /etc/dyni.conf.
8. Workload Requirements
To obtain benefit from the current version of Dynimize,
all of the following workload conditions must be met:
A small number of CPU intensive processes
On a given OS host where the workload is running, the workload must be comprised of
one or a few CPU intensive processes. Optimizing a large number of processes at once is
Long running programs
The processes being optimized have long lifetimes, and their workloads are long running
in order to amortize the warmup time associated with optimization.
Optimized processes must be 64-bit, derived from x86-64 executables and shared
libraries, which must comply with the x86-64 ABI and ELF-64 formats.
Most statically compiled applications on Linux meet this requirement.
Target processes must be dynamically linked to its shared libraries. Statically
linked processes are not yet supported. Most Linux programs are dynamically linked.
No self modifying code
The target application must not be running its own Just-In-Time compiler such as
those found in Java virtual machines. This therefore excludes Java Applications.
Front-end CPU stalls
The workload wastes a lot of time in CPU instruction cache misses, instruction TLB misses,
and to a lesser extent branch mispredictions.
User mode execution
Much of that wasted time is spent in user mode execution (as opposed to kernel mode),
as Dynimize only optimizes user mode machine code.
Because of these requirements, Dynimize takes a whitelist approach when determining
if programs are allowed to be optimized, with MySQL and its variants being the currently
supported optimization targets on that list. Other programs are
not currently supported, and while many can be used with Dynimize, they should be very
thoroughly tested with Dynimize before being deployed in a
Future versions of Dynimize may eliminate many of these workload requirements,
broadening the variety of applicable scenarios as well as further increasing the performance
delivered in previously beneficial cases.
9. Miscellaneous Notes
Sequential CPU speedup
Dynimize usage may result in the speedup of the target application and/or a
reduction in CPU resources consumed by that application. Speedup typically occurs
when sequential CPU performance is a bottleneck. When sequential CPU performance is
not a bottleneck, a reduction in CPU resources used by the optimized process is
experienced which can be observed by an increase in CPU idling. All improvements
are generally limited to time spent in user mode execution.
Dynimize performs work in response to CPU usage
When a process whose executable is listed in the exeList consumes large amounts of
CPU resources, Dynimize will automatically begin to optimize the in-memory
machine code of that process. Dynimize only performs work in response to CPU
resources consumed by a target process. The more CPU resources consumed by a target process,
the more intensely Dynimize will work to dynimize it and the more quickly the process
will become dynimized. A target process that consumes little CPU resources may never be targeted by Dynimize, or will
take a long time to progress to the dynimized state.
Dynimize can respond to changes in workload
Once all target applications
are sufficiently optimized, Dynimize will enter idle mode.
If Dynimize was originally started with the reoptimize option,
it may then respond at a later time to a significant change in the
CPU workload of an already dynimized process by redynimizing it.
It will also dynimize any new CPU intensive undynimized process that was started from
an executable on the exeList.
Target applications do NOT need to be restarted in order to be dynimized.
Once Dynimize is started it will automatically detect and begin dynimizing them immediately.
One Dynimize instance per OS host
Dynimize is not designed to be run as multiple instances on the same host OS.
If a second instance is launched in parallel, it will detect an already running
instance of Dynimize and exit.
Dynimize is currently single threaded and will only consume the resources of at
most one CPU core, even when optimizing multiple target processes in parallel.
No handoffs between Dynimize instances
A new Dynimize process will not dynimize a target process that has already been
dynimized by a previous Dynimize process, unless that target process has been undynimized
using the -undo or -redo flags.
No stale shared libraries
Dynimize will not dynimize a running process if its shared libraries on disk have changed
after they were loaded into that running process. dyni -status will report "stale shared libraries"
for that target process and it must be restarted before Dynimize can target it.
Dynimized processes stay dynimized
Once Dynimize is shut down, any dynimized processes will continue to run in the dynimized state.
However if a dynimized process is then restarted, any previous optimizations will have disappeared and
Dynimize will need to be started again to target the application.
10. Example Life Cycle of Dynimize and a Target Application
Dynimize is started as a background process and begins in idle mode, monitoring system processes.
This state consumes virtually no CPU resources:
$ dyni -start
A new target application process such as mysqld is then launched and becomes
CPU intensive, or an already running target application becomes CPU intensive.
Dynimize detects this and begins to dynimize the target application. Incremental,
atomic updates to the target application's machine code are made. If the target
application remains CPU intensive then this optimization typically takes around 60
seconds to complete:
$ dyni -status
Dynimize is running
mysqld, pid: 8375, dynimizing
Dynimize has completed its current batch of optimization work and enters idle mode:
$ dyni -status
Dynimize is running
mysqld, pid: 8375, dynimized
A new target application is launched and Dynimize returns to step 3. Additionally, if the workload of the previously dynimized
process has drastically changed and Dynimize was started with the reoptimize option, it also returns to step 3.
Although Dynimize is single threaded, it can apply these steps to multiple
target processes at the same time.
11. Dynimize Overheads
Dynimize introduces both CPU and memory overheads when ramping up performance during the
dynimizing phase, with plans to reduce these overheads in future releases. The following section addresses these overheads.
11.1 Dynimize CPU overhead
CPU overhead exists during the warm-up phase when a process is being dynimized.
There are two components of CPU overhead during this phase.
The most obvious is the amount of CPU cycles that the dyni process is actually consuming.
While at first spiking to 100% utilization of a single core for less than a second,
dyni typically fluctuates at around 20% CPU utilization (of a single core) for the remainder
of the warm-up phase. Because it is quite rare to be fully utilizing all hardware threads
on a large multi-core system, the single threaded dyni process is unlikely to result
in significant overheads in this regards. The second CPU performance overhead that takes
place is that of application profiling, and although brief, it is typically far more drastic.
Both these overheads are offset by the gradual machine code optimizations that take effect,
and are completely eliminated once the process reaches the dynimized phase.
Therefore an initial warmup period lasting anywhere from 30 seconds to several minutes should
be set aside for workloads using Dynimize.
11.2 Dynimize Memory overhead
Memory overhead also exists, where the dyni process typically requires around 4 GB of
virtual memory for each target process being dynimized, while it is dynimizing them.
Most of this virtual memory consumption is used for book keeping purposes when dynimizing,
and so the dynamic range of memory accesses is quite limited and in our experience does
little to trigger additional page faults in memory constrained workloads.
That said, at least 4GB total of free swap space + RAM is required for each dynimized process during
the dynimizing phase. Once a process is dynimized, most of that memory is freed. The dyni process then typically
consumes 50-150 MB of virtual memory per dynimized process. This is used for book keeping
purposes in order to react to drastic workload changes in dynimized processes. This memory
also undergoes a very small range of dynamic accesses, and should therefore have negligible
impact on system paging.
Note that when using -reoptimize, if a drastic workload change occurs for a specific process
then that target process may be redynimized and will return to the
dynimizing phase. In this case Dynimize will briefly incur the 4 GB virtual memory
overhead per target process again.
An additional use of memory is the 35 MB code cache used by the just-in-time compiler that
is loaded into the target process at the start of the first dynimizing phase. That being said, the actual access patterns of
machine code instruction memory with the addition of the code cache are more constrained
than that of the original undynimized process, and so the resident memory pages used there
should be less in most cases. Note that the flags -undo
and -redo will leave behind this 35 MB code cache in a target
process, and any subsequent reoptimization of a target process by Dynimize will then create a new code cache when that same process
first enters the dynimizing state again.
On the other hand, when the -reoptimize flag is used and dyni detects a drastic workload change in a process
already dynimized by the current dyni invocation and subsequently reoptimizes it, Dynimize will continue to use the same code cache
initially setup by the current dyni invocation.
Overall, because of the limited range of dynamic memory accesses, these memory overheads
should not affect performance in memory constrained environments. Therefore more RAM is
typically not required. However one should make sure that the system swap space can accommodate
the increased virtual memory used during the dynimizing phase in memory constrained environments.
12. Using Dynimize on Applications Other than MySQL
Many have asked why MySQL is the main target for the initial release of Dynimize.
Database workloads can be IO bound and may not always benefit from improved CPU performance to
the same extent as other workloads. Extreme levels of instruction cache misses are common
in many other CPU intensive enterprise software workloads, which leaves many places where the current crop of optimizations in Dynimize
could be put
to productive use in the future. However a lot of these other examples such as Redis or Nginx
spend most of this CPU time in system mode execution (in the Linux kernel). The machine code
underlying this system mode execution cannot be optimized by Dynimize in the current release
and therefore these applications benefit to a lesser extent than MySQL. For example, at most around
a 10% speedup with Nginx in a CPU bound setup has been observed.
Other programs such as Apache HTTP Server are often
configured in a multiprocess configuration where there may be too many processes for Dynimize
to handle effectively, or where the processes have short lifetimes. For these reasons,
little time has been spent investigating these other use cases up to now.
Future versions of Dynimize will be able to optimize workloads with these characteristics as well.
However at the moment, MySQL and its variants have been the main focus, with other
single process, multithreaded relational databases on Linux such as MongoDB being likely candidates to
benefit from this current release. Stay tuned for expanded scope in future releases.
If you find Dynimize useful in other workloads outside of MySQL, we'd love to hear about it.
Let us know at firstname.lastname@example.org.
Questions & Feedback
We love answering questions and your feedback is extremely valuable! Please use the discussion
form below for anything related to this documentation.