1. Overview

Dynimize is a user mode program that quickly optimizes the in-memory machine code of target programs at run-time, thereby improving the performance of many compute intensive workloads. It does this by applying profile guided compiler optimizations to live, in-memory executable code, by taking in live machine code, live profiling information, live process data, and the system environment as input to an optimizing compiler, and then outputting more optimal machine code back into the live target process or in-memory Linux Kernel executable code.

Dynimize does this without having to recompile, reconfigure or restart target programs, making it a run-time binary optimizer. It does not modify a target program's object code or data on disk. By targeting a workload's in-memory executable, shared libraries and kernel machine code, it can seamlessly and instantly perform full stack profile guided optimizations across the entire system, with minimal effort, utilizing the most relevant live profiling information available. This makes it superior to traditional profile guided compiler optimization which is challenging to apply across entire systems, suffers from stale profiling data and lack of per-process specialization. Dynimize typically runs as a background system process.

---

2. Installation & Quick Start

To install Dynimize, run the following commands in a Linux terminal.




Use your access token to start a subscription license for your host.




To optimize the Linux Kernel or any CPU intensive process who's exe is listed in the [exeList] section of /etc/dyni.conf, run:




The command dyni -status will show target processes or the Kernel progressing from the "profiling", to "dynimizing", and then "dynimized" states. A process has been fully optimized once in the "dynimized" state:











Dynimize can be shutdown with the following command:



If Dynimize is shutdown, all dynimized processes or the Kernel will remain dynimized.


To "undynimize" all dynimized processes or the Kernel, run:



or alternatively restart the dynimized process or reboot your system to undynimize the Kernel.


To cancel a subscription license for a given host, run the following command on that host:




To uninstall Dynimize, run:

---

3. System Requirements

3.1 OS requirements

x86-64 Linux Kernel version 2.6.32 or later is required.

---

3.2 Memory requirements

On average, approximately 4GB of virtual memory (swap + RAM) per target process being dynimized in parallel is required. This is because Dynimize requires 4 GB per target process in the "dynimizing" phase, releasing this memory after that. Note that this is the average and varies by the target process executable.

Dynimize targets multi-process programs with the same memory requirements as single process programs. For example, NGINX and most other web servers use multiple worker processes with a common parent process. Here Dynimize does not need to target each process individually.

---

3.3 VM specific requirements

There are no specific requirements for running inside KVM, Xen or VMware virtual machines. Therefore Dynimize can be run in most cloud environments.

---

3.4 Container specific requirements

Dynimize can target containerized processes, such as processes running inside Docker, by running Dynimize outside of the container directly on the host OS. Dynimize cannot itself be launched from inside a container.

---

4. Command Line Options

---

5. Configuration File

On startup, Dynimize is configured based on the settings in /etc/dyni.conf. The following is a description of these settings. Many of these settings can be overridden by command line options. Note that starting a line with the # character will cause that line to be skipped.

All options follow the line in the dyni.conf file containing the string [options]. If any of those following options are missing, their default settings are used. Note that the < > characters should not actually be placed in the log file:



The lines following [exeList] in dyni.conf denote the list of allowed optimization targets. These are the names of the executables that are used to launch the processes or process groups that can be targeted by Dynimize, one name per line. So for example, the NGINX server process group would be denoted as nginx. vmlinux is always used to denote the Linux Kernel and associated kernel modules or device drivers. A process or process group must have been started by an executable on this list to be dynimized. Note that if a symbolic or hard link is used to call an executable, the real executable file name must be specified on the [exeList].

Beside each target name under the [exeList], one can optionally specify any of the following: -window=<seconds> -delay=<seconds> -cpu-usage=<percentage>. These have the same effect as their associated command line options, except they are only applied to each target that matches the target name. These options override their equivalents on the command line.

For example:



The above example inside dyni.conf will apply the default window, delay and cpu-usage options to mysqld processes, unless otherwise specified on the command line. For nginx process groups, it will use a window and delay of 60 and 120 seconds respectively, and a cpu-usage threshold of 30%. It will use a window of 60 seconds and the default delay and cpu-usage thresholds when targeting the Linux Kernel and other kernel modules along with it.

The lines following [users] list the possible user names of the owners of processes that can be targeted by Dynimize, one name per line. If present, a process must match both the [exeList] and [users] list in order to be dynimized. By default, no user name is requirement.

The output log specified by -log=<pathname> records all errors and Dynimize status updates along with their timestamps. It will also list all options enabled at Dynimize startup which can be used to verify the options that are in effect. On installation the log path is set to /var/log/dyni.log in /etc/dyni.conf. Once the log file reaches half the maxLogSize, it is moved to the same path name appended with .old, and a new log is started as the original log file name. Logging is disabled if the log option is removed from dyni.conf.

---

6. Dynimize States

A process, process group or Linux Kernel that is being targeted by Dynimize for optimization may be in one of the following states, as indicated by the dyni -status command. For example:





The follow are the 6 possible states:

waiting

profiling

dynimizing

dynimized

undynimized

partly dynimized

---

7. Dynimize Usage Examples

8. Licensing & Billing

One of the primary purposes of Dynimize is to reduce the financial costs of CPU resources consumed by target workloads for a given level of performance. Therefore to help users reduce billing costs in the cloud, elastic, on-demand subscriptions can be generated on the command line to license Dynimize with per second billing granularity, thereby matching the billing granularity of CPU resources in many public clouds and allowing for easy licensing automation.

Use your access token to start a subscription license for your host.



To cancel a host's subscription license from the command line, Dynimize must be shut down and there cannot be any dynimized programs running on that host.

To stop running instances of Dynimize and "undynimize" all dynimized programs, run



or you can simply stop Dynimize and restart any dynimized processes to "naturally" undynimize them. Rebooting the system does the same to the Linux Kernel. For example, the following commands can restart the mysqld process for MySQL after stopping Dynimize.





To then cancel a subscription license for a given host, run the following command on that host.



By default, time is accounted for using per second granularity, but only when the license is later correctly cancelled from the command line. See Billing time granularity for exceptions.

Time accounting for any licenses used begins 7 days after your account was activated. This initial wait period is your free trial, after which each license is then priced at $1 per CPU core per 30 days, prorated per second of usage when cancelled from the command line on the licensed host.

For example, if you only have 1 hour and 48 minutes of use over the course of a month on a 4 core host, you will only be invoiced $0.01. Your account will be billed at the end of each month for subscription license time used during that month.

---

Workload Requirements

To obtain benefit from the current version of Dynimize, all of the following workload conditions must be met:

Long running processes

Linux x86-64

Dynamically Linked

No self modifying code

No GC-mapped code

Front-end CPU stalls

Because of these requirements, Dynimize uses a configurable allowlist approach when determining if programs are allowed to be optimized. The processes currently supported are as follows, named by the executable they are launched from:

  •  nginx (NGINX)
  •  httpd or apache2 (Apache HTTP Server)
  •  litespeed (LiteSpeed Web Server)
  •  mongod (MongoDB Server)
  •  mysqld (MySQL Server, MariaDB Server or Percona Server for MySQL)
  •  mariadbd (MariaDB Server)
  •  postgres (PostgreSQL Server)
  •  db2sysc (IBM Db2 Server)
  •  redis or redis-check-rdb (Redis)
  •  memcached (Memcached)
  •  node (Node.js)
  •  php-fpm (PHP processor)
  •  python, python2, or python3 (Python runtime)
  •  ruby (Ruby runtime)
  •  java (Java Virtual Machine)

Note that when targeting processes launched from one of the above exes, both the in-memory executable and shared library code of these processes are optimized.

In addition, the Linux Kernel and other kernel modules are also supported optimization targets.

Other programs are not currently supported, and while many can be used with Dynimize, they should be very thoroughly tested with Dynimize before being deployed in a production environment. If you find Dynimize useful for other workloads outside of the ones listed above, we'd love to hear about it. Let us know at info@dynimize.com.

Future versions of Dynimize may eliminate many of these workload requirements, broadening the variety of applicable scenarios as well as further increasing the performance delivered in previously beneficial cases.

---

10. Dynimizing the Linux Kernel

The Linux Kernel and other CPU intensive kernel modules can be dynimized by including vmlinux under [exeList] in the /etc/dyni.conf file.

---

10.1 Signing the Dynimize Kernel Module

The following section is only relevant if Secure Boot is enabled in the system BIOS settings.

When Dynimize targets the Linux Kernel, it will utilize the Dynimize kernel module /opt/dynimize/dyni0000.ko as a code cache for the optimized machine code Dynimize generates. Subsequent calls to dyni -undo followed by dynimizing vmlinux again will load additional copies of dyni0000.ko, renamed anything from dyni0001.koto dyni<nnnn>.ko, corresponding to the number of times vmlinux is re-dynimized. If Secure Boot is enabled, these kernel modules must all be generated and signed before use. This can be done with any of the following approaches.


1. If Secure Boot is enabled when Dynimize is installed, the installer will detect this and then ask if the secure boot Dynimize installation step should be performed. This step will install any missing signing tools, generate and enroll a new signing key, generate 10 copies of dyni0000.ko (dyni0000.ko to dyni0009.ko), each signed with the new signing key and then archived as .tar.gz files in /opt/dynimize/mods/. This entire Secure Boot step for Dynimize can also be invoked any time by calling:



This script will leave behind the enrolled signing key at /opt/dynimize/.sign/MOK.priv. It is recommended that this key be removed from the host and stored securely for future use. This script can be called in the future if a new signing key needs to be generated and enrolled.


2. If the signing tools are already installed and there is already a signing key at /opt/dynimize/.sign/MOK.priv, invoking



will generate the signed dyni<nnnn>.ko.tar.gz files in /opt/dynimize/mods/ using that key. If using a different signing key, place it at /opt/dynimize/.sign/MOK.priv before invoking this script. Note that this signing method is attempted automatically by dyni if the Linux Kernel refuses to load dyni<nnnn>.ko, such as after the Linux Kernel is updated and the Dynimize Kernel modules have not yet been re-signed.


3. To specify a custom signing command to be used as a template to sign all the generated dyni<nnnn>.ko.tar.gz files, call



For example,



This command generates all the versions of dyni<nnnn>.ko needed, signed using that custom signing command as a template with the appropriate paths to each kernel module automatically substituted for the one in the example command.


Note that these kernel modules must be re-signed each time the Linux Kernel is updated if Secure Boot is enabled.

---

10.2 Dynimize & Livepatching The Linux Kernel

A dynimized Linux Kernel must be undynimized before a livepatching service is used to successfully apply updates. Some examples of livepatching services are KernelCare, Canonical Livepatch, kpatch, ksplice and kGraft.
For example:





Because the Linux kernel is dynimized, it should be undynimized before applying the livepatch updates:





Now the livepatch updates can be applied, after which the updated Linux Kernel can be re-dynimized again:

---

11. Miscellaneous Notes

Sequential CPU speedup

Dynimize can respond to changes in workload

Zero downtime

One Dynimize instance per OS host

Single threaded

No handoffs between Dynimize instances

No stale shared libraries

Dynimized processes stay dynimized

---

12. Example Life Cycle of Dynimize and a Target Application

1. Dynimize is started as a background process and begins in idle mode, monitoring system processes. This state consumes virtually no CPU resources:
   
   
      $ dyni -start
   
   


2. A new target application process such as mysqld is then launched and becomes CPU intensive, or an already running target application becomes CPU intensive.


3. Dynimize detects this and begins to dynimize the target application. Profiling data is gathered, and incremental, atomic updates to the target application's machine code are made. This takes about 90 seconds by default, unless the -window=<seconds> option changes that setting:
   
   
      $ dyni -status
   
   
   
     Dynimize is running
   mysqld, pid: 8375, dynimizing
   
   


4. Dynimize has completed its current batch of optimization work and enters idle mode:
   
   
      $ dyni -status
   
   
   
     Dynimize is running
   mysqld, pid: 8375, dynimized
   
   


5. A new target application is launched and Dynimize returns to step 3. Additionally, if the workload of the previously dynimized process has drastically changed and Dynimize was started with the reoptimize option, it also returns to step 3. Although Dynimize is single threaded, it can apply these steps to multiple target processes at the same time.

---

13. Dynimize Overheads

Dynimize introduces both CPU and memory overheads when ramping up performance during the dynimizing phase, with plans to reduce these overheads in future releases. The following section addresses these overheads.

---

13.1 Dynimize CPU overhead

CPU overhead exists during the warm-up phase when a process is being dynimized. There are two components of CPU overhead during this phase. The most obvious is the amount of CPU cycles that the dyni process is actually consuming. While at first spiking to 100% utilization of a single core for less than a second, dyni typically fluctuates at around 20% CPU utilization (of a single core) for the remainder of the dynimizing phase. Because it is quite rare to be fully utilizing all hardware threads on a large multi-core system, the single threaded dyni process is unlikely to result in significant overheads in this regards. The second CPU performance overhead that takes place is that of application profiling, and although brief, it typically makes a larger impact. Both these overheads are offset by the gradual machine code optimizations that take effect, and are completely eliminated once the process reaches the dynimized phase. Therefore an initial warmup period lasting the equivalent of the -window=<seconds> setting (default 90 seconds) should be set aside for programs using Dynimize.

---

12.2 Dynimize Memory overhead

Memory overhead also exists, where the dyni process typically requires on average around 4 GB of virtual memory for each target program being dynimized, while it is dynimizing them. However this can vary depending on the program being targeted. The Linux Kernel, or an entire process group such as the parent and worker processes of NGINX each count as one single program for this purpose. Each worker process in a process group will not incur separate memory overheads by Dynimize.

Most of this virtual memory consumption is used for book keeping purposes when dynimizing, and so the dynamic range of memory accesses is limited and does little to trigger additional page faults in memory constrained workloads. However, at least 4GB total of free swap space + RAM is typically required for each dynimized process during the dynimizing phase. Once a process is dynimized, most of that memory is freed. The dyni process then typically consumes 50-150 MB of virtual memory per dynimized process. This is used for book keeping purposes in order to react to drastic workload changes in dynimized processes. This memory also undergoes a very small range of dynamic accesses, and should therefore have negligible impact on system paging.

Note that when using -reoptimize, if a drastic workload change occurs for a specific process then that target process may be redynimized and will return to the dynimizing phase. In this case Dynimize will briefly incur the 4 GB virtual memory overhead per target process again.

An additional use of memory is the 35 MB code cache used by the just-in-time compiler that is loaded into the target process at the start of the first dynimizing phase. That code cache is shared by the parent and children of a process group such as NGINX. That being said, the actual access patterns of machine code instruction memory with the addition of the code cache are more constrained than that of the original undynimized process, and so the resident memory pages used there should be less in most cases. Note that the flags -undo and -redo will leave behind this 35 MB code cache in a target process, and any subsequent reoptimization of a target process by Dynimize will then create a new code cache when that same process first enters the dynimizing state again. On the other hand, when the -reoptimize flag is used and dyni detects a drastic workload change in a process already dynimized by the current dyni invocation and subsequently reoptimizes it, Dynimize will continue to use the same code cache initially setup by the current dyni invocation.

Overall, because of the limited range of dynamic memory accesses, these memory overheads should not affect performance in memory constrained environments. Therefore more RAM is typically not required. However one should make sure that the system swap space can accommodate the increased virtual memory used during the dynimizing phase in memory constrained environments.

---

Questions & Feedback

We love answering questions and your feedback is extremely valuable! Please use the discussion form below for anything related to this documentation.