Grind your web application! Load testing web apps with Grinder.
What’s Grinder? It’s a free distributed load testing framework written in Java. It allows you to write test scripts in Jython and Clojure. I’m not going to advertise it nor describe all its features, so for more detailed information go to the Grinder website. In this post I’m going to present load testing web applications with Grinder.
Grinder runs an agent process on a machine which can create/stop worker processes. Each worker process can run tests in many threads. The console is the process that coordinates it and lets you control it using the GUI. The console also gathers some statistics and allows scripts editing and distribution. You don’t need the console to run tests on a single machine. These are the three main elements of Grinder. You can create script from scratch on your own, or use a proxy recorder. I’ll use a proxy to record script and modify it later to fit my needs. I’ll use the most recent version of Grinder which is 3.11, it also supports Jython 2.5.3 pretty wel.
First thing to do is download the proper version of Grinder from https://www.sourceforge.net/projects/grinder. Extract it to some dir – if you want it in your repo as a dependency, then put it in the vendor dir (in case of RoR project). If not, it’s up to you where it will land (just don’t forget the path). To use Grinder 3 you just need to have at least java 6 on your machine.
Next, you probably would like to prepare a test script. You can write it from scratch (which might be quite a tedious task) or use the grinder proxy to record it. The second option is more convenient, at least it was for me. I had a couple of complex scenarios to script. But before you do that prepare a set of small shell scripts which will help you with starting proxy, console or with running a test. The first one named set_grinder_env.sh just sets and exports couple of required variables.
If Grinder is added to the CLASSPATH, then you can start recording a proxy. I’ve also created a small script, as recommended in the starting guide on the Grinder website.
#!/bin/bash
. . /set_grinder_env.sh
java net.grinder.TCPProxy -console -http > grinder.py
Recording proxy is started with a -console option which displays a control window that allows terminating a proxy process cleanly, -http option enables filters which allow the recording HTTP traffic. By default proxy sends output to terminal, so it’s redirected to grinder.py file. Proxy listens on port 8001. You can record additional headers if the basic set is not enough for you – go to the Grinder additional headers section for a description.
Before recording script you need to configure your browser to use this proxy. In recent Firefox (v. 23) it’s Edit -> Preferences -> Advanced -> Network -> Settings.
Now the script can be recorded. Let’s say that you want to simulate 100 users signing in to your application. After starting the proxy just point your browser to your application and sign in as some existing user. You may do some more actions, it depends on the scenarios that you want to automate. When you finish recording, terminate the proxy process using the console window. This guarantees proper saving of the script. You should have something like that in the grinder.py file.
The next step is to parametrize and clean this up a bit, so it will be more readable and easier to maintain. I removed requests, methods and constants referring to them that were done to external services. I also got rid of requests for images, stylesheets etc. I also corrected the methods’ name and requests’ numbering. Now each request had a separate page method which was separately instrumented where statistics were gathered for each of them. I also added a couple of helper methods for: loading users from file, getting a random user, writing response to file (sometimes helps in debugging your test). After all the modifications the script looked more or less like this:
Before you run the test, there’s one thing that you have to do and that is to prepare the properties file. It is the file that tells grinder which script to run plus how many processes and threads to create, how many times run the script, how long are the sleep times and much more. You can even set a path to a different jython version there. I had to set the grinder.jvm.arguments property to stop Grinder complaining about caches. I also turned off the console. In my test I wanted 1 process to be created and 3 threads that will be repeating the login scenario 5 times (of course each thread will run it 5 times). initialSleepTime property says each thread to wait before start random number between 0 and 10000 ms. All times are specified in milliseconds.
Grinder gives you many ways of configuring creation of processes. You can run couple of processes on different machines. Processes on different machines can create different number of threads (each load injector has different properties file). All processes might be started at the same time or you can set how many processes will be started after every period of time, even how many of them will be initially created. Here’s my grinder.properties file.
Finally you can run the test. Use that bash script, to do it in an easier way. Just remember to create all the necessary test data, like users, etc…
#!/bin/bash
. ./set_grinder_env.sh
java net.grinder.Grinder $GRINDERPROPERTIES
The results you will see on your screen summary report will look like this.
It shows the number of successful tests and errors. Number of all executed tests = tests number * processes number * threads number * runs number. Total, mean, standard deviation for each of the tests is calculated only for successful ones. All data is saved to log files. Grinder organizes log files in a way, that each process saves data to a separate file called $hostname-$process_number.log. Grinder also gathers data about test executions in files called $hostname-$process_number-data.log.
When running a test with the console turned on, you’ll see a window similar to this:
Grinder does not generate any graph files. You can do it with other tools, that are listed on the Grinder’s links page, like Grinder Analyzer, or Ground Report. Thanks for reading and let me know how grinding your app went!