Appendix D - RLM Cloud Performance Notes

Reprise Software has run performance testing on the RLM Cloud servers. The initial tests were run on the first release of the RLM Cloud product.

Testing Environment

All testing is done on a standard RLM Cloud ISV server virtual machine. The virtual machine used for this test is in the Orlando data center.

The server machine is configured with a single customer (“perf”), which has 400-450 ISV servers provisioned and running.. Each server serves 3 different licenses, none of which have the “share” attribute enabled.

Customer simulators are run against the ISV servers. Each customer simulator does the following:

  • Check out a variable number of licenses, using 3 separate processes, in a continuous loop:

  • 20% of the licenses are checked out for 2 hours, then checked back in for 5 minutes.

  • 35% of the licenses are checked out for 15 minutes, then checked in for 1 minute.

  • 45% of the licenses are checked out for 2 minutes, then checked in for 2 seconds.

  • Each simulation process runs until stopped, or until it has registered 50 checkout errors. Each checkout is retried and if both the original and retry fail, an error is recorded.

  • Simulators are run on multiple server systems, no more than 100 per server, with a staggered start; each subsequent customer simulator (on a particular simulation machine) is delayed by one additional second from the last.

  • The customer simulators run between 40 and 238 checkouts each (for customer “perfNNN”, the simulator runs 40 + (NNN mod 100)*2 license checkouts).

Results

Our tests indicate:

  • 450 ISV servers is the maximum number which can be run on a server VM. Any more than this and various system limits are exceeded.

  • With 400 ISV servers running, the server VM can handle 350 simulated customers (with the parameters indicated above). With 350 simulated customers running for 12+ hours, interactive checkouts showed no slowdown, and all simulated customers run without error. Similarly, 375 simulated customers run without error for several hours. With 400 simulated customers, 9 of the 1200 client processes registered 50 errors and exited.

  • With 450 ISV servers running and 350-400 simulated customers, some customer simulators get errors and stop. Running for a few hours with 400 simulated customers, 40 of the 1200 simulation processes had > 50 errors and stopped.

  • In all cases above, interactive checkouts continue to run without delay.

  • Independent of the numbers above, you should note that these performance tests were run without updates to licenses and status requests from the RLM Cloud GUI. When you get to the higher number of servers, license updates cause the redirector’s license file on all servers to be re-written, and status requests cause a lot of process creation and database update operations on the server, which makes the supportable number of servers lower.

Reprise Recommendation

Configure up to 300 customers per server maximum, or 150-175 if you are using pairs of servers to failover for each other. You should be able to use the higher end of this range if your customers have low numbers of licenses or long checkout times.

Conclusion

Your mileage will vary. Test in your environment to see how many customers you can support.