- Apache Ab Windows Download
- Apache Ab Post Example
- Apache Ab Cookie Example
- Apache Ab Download
- Apache Ab Tool Benchmark Example
- Apache Ab Upload File
Browse other questions tagged linux apache-2.2 yum ab or ask your own question. The Overflow Blog Level Up: Creative Coding with p5.js – parts 4 and 5. Well as long as the apache sees the POST from ab it means ab does it's job and it sends the postfile content. Other than that i do not know how your script catcher is made and how it understands what ab POSTS. Time: How long ab spent handling the connection, from the beginning of its attempt to contact the server to when the connection closed (i.e., done - start) From there, ab uses the data object to calculate the remaining latency-related metrics: aggregated connection times, percentiles, and, if you use the -g flag, data for individual requests.
- Apache Bench Tutorial
Example if nc when ab starts will be opened immediately c parallel connections and will not reopened new when one connection is closed. Content of POST file (-p post-soap.data) could be obtained by a TCP dump of a working request. Ab (Apache Banchmark) time flow diagram. AB is Apache's HTTP service testing tool, which can test the performance of your HTTP server, especially the number of requests processed per second. AB can not only do stress tests on Apache servers, but also on other types of servers. For example, nginx, Tomcat, IIS, etc.
- Apache Bench Useful Resources
- Selected Reading
Performance testing has proved itself to be crucial for the success of a business. Not only does a poor performing site face financial losses, it can also lead to legal repercussions at times.
No one wants to put up with a slow performing, unreliable site in important online interactions such as purchasing, online test taking, bill payment, etc. With the Internet being so widely available, the range of alternatives is immense. It is easier to lose clientele than gain them and performance is a key game changer.
Need for a Load Testing Tool
If we can understand what is the need for a load testing tool, it will give us the reason and motivation to use it. Some famous business sites have suffered serious downtimes when they get large number of visitors. E-commerce websites invest heavily in advertising campaigns, but not in Load Testing. Therefore, they fail to ensure optimal system performance, when that marketing brings in traffic.
Another familiar example of ignoring load testing is that of 'error establishing connection' in WordPress websites. Therefore, it is a good idea to load test a website or application before its deployment in production. It is nice to quickly establish a best-case scenario for a project before running more detailed tests down the road.
What is Apache Bench?
Apache Bench (ab) is a tool from the Apache organization for benchmarking a Hypertext Transfer Protocol (HTTP) web server. Although it is designed to measure the performance of Apache web server, yet it can also be used to test any other web server that is equally good. With this tool, you can quickly know how many requests per second your web server is capable of serving.
Features of Apache Bench
Let us see the important features and limitations of Apache Bench. The features and limitations are listed below −
Being an open source software, it is freely available.
It is a simple command line computer program.
It is a platform-independent tool. It means that it can be invoked on Linux/Unix or on Windows server equally well.
It can conduct load and performance test for only web server - HTTP or HTTPS.
It is not extensible.
Apache Bench uses only one operating system thread irrespective of the concurrency level (specified by the -c flag). Therefore, when benchmarking high-capacity servers, a single instance of Apache Bench can itself be a bottleneck. To completely saturate the target URL, it is better to use additional instances of Apache Bench in parallel, if your server has multiple processor cores.
Precaution
You need to be aware that there is no directive in the Apache Bench to increase concurrency in particular intervals while running tests. Therefore, running load tests using ab is equivalent to a denial-of-service (DOS) attack. It is recommended that you inform and take prior permission from your VPS service provider if you are going to do heavy load testing for a long period of time. They will allot you an appropriate time interval or shift your node for the load testing task.
Second, if you are testing a third person's website continuously and for a long time just for learning Apache Bench from your VPS (which becomes the testing node), there is a remote possibility that your VPS public IP can be blocked by the third person's website permanently. In that case, you will not be able to connect to that website with the same IP. But if you really want to connect to the website in future, the only solution will be to talk to the system administrator of the target website, or create a new instance of the server with a different IP with the help of your VPS service provider.
Having warned you, let me assure you that all tests in this tutorial are safe enough and out of what system administrators generally call 'system abuse' practices.
In this chapter, we will guide you how to set up your environment for Apache Bench on your VPS.
System Requirement
Memory − 128 MB
Disk Space − No minimum requirement
Operating System − No minimum requirement
Installing Apache Bench
Apache Bench is a stand-alone application, and has no dependencies on the Apache web server installation. The following is a two-step process to install Apache Bench.
Step 1 − Update package database.
Please note that symbol # before a terminal command means that root user is issuing that command.
Step 2 − Install apache2 utils package to get access to Apache Bench.
Apache Bench is now installed. If you want to test a web application hosted on the same VPS, then it is enough to install the Apache web server only −
Being an Apache utility, Apache Bench is automatically installed on installation of the Apache web server.
Verifying Apache Bench Installation
Let us now see how to verify Apache Bench Installation. The following code will help verify the installation −
Output
When you see the above terminal output, it means you have successfully installed Apache Bench.
Creating a Privileged Sudo User
From the safety point of view, it is considered a good practice for system administrator to create a sudo user instead of working as root. We will create a test user, named test, for the purpose −
Let us set the password for the new user −
System will prompt for a new password for the user test. You can enter a simple password as we are just testing, and not deploying to the production server. Usually the sudo command will prompt you to provide the sudo user password; it is recommended not to use complicated password as the process becomes cumbersome.
Output
Testing Apache.org Website
In this section, we will test the Apache.org Website. Let us first switch to the sudo user test −
To begin with, we will test the website of Apache organization, https://www.apache.org/. We will first run the command, and then understand the output −
Here -n is the number of requests to perform for the benchmarking session. The default is to just perform a single request which usually leads to non-representative benchmarking results.
And -c is the concurrency and denotes the number of multiple requests to perform at a time. Default is one request at a time.
So in this test, Apache Bench will make 100 requests with concurrency 10 to the Apache organization server.
Output
Having run our first test, it will be easy to recognize the pattern of use for this command which is as follows −
where,
ab − Apache Bench command
options − flags for particular task we want to perform
URL − path url we want to test
Understanding the Output Values
We need to understand the different metrics to understand the various output values returned by ab. Here goes the list −
Server Software − It is the name of the web server returned in the HTTP header of the first successful return.
Server Hostname − It is the DNS or IP address given on the command line.
Server Port − It is the port to which ab is connecting. If no port is given on the command line, this will default to 80 for http and 443 for https.
SSL/TLS Protocol − This is the protocol parameter negotiated between the client and server. This will only be printed if SSL is used.
Document Path − This is the request URI parsed from the command line string.
Document Length − It is the size in bytes of the first successfully returned document. If the document length changes during testing, the response is considered an error.
Concurrency Level − This is the number of concurrent clients (equivalent to web browsers) used during the test.
Time Taken for Tests − This is the time taken from the moment the first socket connection is created to the moment the last response is received.
Complete Requests − The number of successful responses received.
Failed Requests − The number of requests that were considered a failure. If the number is greater than zero, another line will be printed showing the number of requests that failed due to connecting, reading, incorrect content length, or exceptions.
Total Transferred − The total number of bytes received from the server. This number is essentially the number of bytes sent over the wire.
HTML Transferred − The total number of document bytes received from the server. This number excludes bytes received in HTTP headers
Requests per second − This is the number of requests per second. This value is the result of dividing the number of requests by the total time taken.
Time per request − The average time spent per request. The first value is calculated with the formula concurrency * timetaken * 1000 / done while the second value is calculated with the formula timetaken * 1000 / done
Transfer rate − The rate of transfer as calculated by the formula totalread / 1024 / timetaken.
Quick Analysis of the Load Testing Output
Having learned about the headings of the output values from the ab command, let us try to analyze and understand the output values for our initial test −
Apache organisation is using their own web Server Software − Apache (version 2.4.7)
Server is listening on Port 443 because of https. Had it been http, it would have been 80 (default).
Total data transferred is 58769 bytes for 100 requests.
Test completed in 1.004 seconds. There are no failed requests.
Requests per seconds − 99.56. This is considered a pretty good number.
Time per request − 100.444 ms (for 10 concurrent requests). So across all requests, it is 100.444 ms/10 = 10.044 ms.
Transfer rate − 1338.39 [Kbytes/sec] received.
In connection time statistics, you can observe that many requests had to wait for few seconds. This may be due to apache web server putting requests in wait queue.
In our first test, we had tested an application (i.e., www.apache.org) hosted on a different server. In the later part of the tutorial, we will be testing our sample web-applications hosted on the same server from which we will be running the ab tests. This is for the ease of learning and demonstration purpose. Ideally, the host node and testing node should be different for accurate measurement.
To better learn ab, you should compare and observe how the output values vary for different cases as we move forward in this tutorial.
Plotting the Output of Apache Bench
Here we will plot the relevant outcome to see how much time the server takes as the number of requests increases. For that, we will add the -g option in the previous command followed by the file name (here out.data) in which the ab output data will be saved −
Let us now see the out.data before we create a plot −
Output
Let us now understand the column headers in the out.data file −
starttime − This is the date and time at which the call started.
seconds − Same as starttime but in the Unix timestamp format (date -d @1496160697 returns starttime output).
ctime − This is the Connection Time.
dtime − This is the Processing Time.
ttime − This is the Total Time (it is the sum of ctime and dtime, mathematically ttime = ctime + dtime).
wait − This is the Waiting Time.
For a pictorial visualization of how these multiple items are related to each other, take a look at the following image −
If we are working over terminal or where graphics are not available, gnuplot is a great option. We will quickly understand it by going through the following steps.
Let us install and launch gnuplot −
Output
As we are working over terminal and supposing that graphics are not available, we can choose the dumb terminal which will give output in ASCII over the terminal itself. This helps us get an idea what our plot looks like with this quick tool. Let us now prepare the terminal for ASCII plot.
Output
As, our gnuplot terminal is now ready for ASCII plot, let us plot the data from the out.data file −
Output
We have plotted the ttime, total time (in ms) from column 9, with respect to the number of requests. We can notice that for the initial ten requests, the total time was in the nearly 100 ms, for next 30 requests (from 10th to 40th), it increased to 1100 ms, and so on. Your plot must be different depending on your out.data.
In the previous chapter, we understood the basic use of the Apache Bench to test a third party website. In this section, we will use this tool to test a web application on our own server. To keep the tutorial self-contained to the extent possible, we have chosen to install a python application for the demonstration purpose; you can choose any other language like PHP or Ruby depending on your expertise level.
Installing Python
Generally, Python is installed by default on Linux servers.
Installing Bottle Framework and Creating a Simple Application
Bottle is a micro-framework written in python for creating web applications, and pip is a python package manager. Type the following command in terminal to install Bottle −
Let us now create a small Bottle application. For that, create a directory and move inside it −
We will create a new python script, app.py, inside the webapp directory −
Now, write the following code in the app.py file −
When you have added the above lines, save and close the file. Having saved the file, we can run the python script to launch the application −
Output
This output shows that our application is running on the local machine at the host http://localhost and listening on the port 8080.
Let us check if our app is responding properly to the HTTP requests. As this terminal cannot take any input without quitting serving the Bottle application, we need to login to our VPS with another terminal. After logging into the VPS with another terminal, you can navigate to your application by typing the following code in the new terminal.
Lynx is a command line browser and is usually installed by default in various Linux distributions like Debian and Ubuntu. If you see the following output, it means your app is working fine.
Output
If you see the above output, that means our application is live and ready for testing.
Testing the Application with Developmental Web Server
Please note that there is a bug in ab, and it is not able to test the application on the localhost. So we will change the host from localhost to 127.0.0.1 in the app.py file. So the file will change to the following −
Let us now test our app by typing the following command on the same terminal on which ran the lynx command −
Output
While the output on the first terminal will be (100 times) as follows −
You can observe how the various values of the ab outcome have changed as compared to the initial test.
Testing the Application with a Multi-Threaded Web Server
In the previous tests of ab, we have used the default web server bundled in the Bottle framework.
Now we will change the single-threaded default web server with a multi-threaded one. Therefore, let us install a multi-threaded web server library like cherrypy or gunicorn and tell Bottle to use it. We have chosen gunicorn for the demonstration purpose here (you can choose some other one too) −
And modify the file, that is change from the default web server to gunicorn −
Let us test the app in the second terminal.
Output
Observe how the Requests per second increased from 493 to 3252. It means gunicorn is suitable as a production server for python apps.
In this chapter, we will learn how to test multiple URLs concurrently. For that, we will need to edit our application file, app.py to include two URLs −
Creating a Simple Shell Script
You can do this by creating a shell script, with multiple ab calls. Create a file test.sh and add the following lines to it −
When you have added the above lines, Save and Close the file. Make the file executable −
Let us now run the script −
To avoid repetition and purpose of clarity, we will show only the relevant of the ab output, indicating by dots what portion has been omitted, as in the following.
Output
Shell Script to Save the Apache Bench Output to a File
You can save the Apache Bench Output to file by creating a shell script, with multiple ab calls. At the end of each line, place an &; this makes the command run in the background, and lets the next command start its execution. You will also want to redirect the output to a file for each url using . For example, our file test.sh will look like the following after modification −
Here, test1.txt and test2.txt are the files to save the output data.
You can check that the above script has created two files, test1.txt and test2.txt which contains the ab output for the respective URLs −
Output
Watch-out Situation
While using ab, you should be alert to the failed test without warning. For example, if you check a wrong URL, you may get something similar to the following (we have deliberately changed the port here).
Output
In this chapter, we will understand the preparation required for testing dynamic pages. A server-side dynamic web page is a web page the construction of which is controlled by an application server processing server-side scripts. The apache bench can only load test the server-side dynamic web page.
Concurrency Level and the Total Number of Requests
Concurrency level should be lower than the total number of requests.
Output
Use of Flags
In this section, we will describe the use of some important flags with the ab command. We will use the terms, options and flags, interchangeably.
Verbose -v
The verbose option can be used to analyze and debug if there exist multiple number of failed requests. A common indication of failure of the load test is that the test finishes very fast and it gives a good number for request per second value. But it will be a wrong benchmark. To identify the success or failure, you can use the -v 2 option which will dump each response's body and header to the terminal output. Following command depicts a use case −
Output
Of course, if you are testing variable responses or returning non-200 HTTP codes in the event of any error, you should simply ignore length checking with the -l option. We will soon see non-200 HTTP when we will launch a web2py application in the subsequent chapters.
Keep-alive -k
When the client sends HTTP request, the connection is made to the server, the server sends the response, and the connection is closed after it has sent the request. This cycle continues with each request. However, with the keep-alive setting (also known as persistent connections), the client maintains an underlying TCP connection open to facilitate multiple requests and response; this eliminates the slow and costly connection initialization time that would otherwise be present.
Variable document length -l
If the web page is of variable length, then you should make use of the option -l. Apache Bench does not report errors if the length of the responses is not constant. This can be useful for dynamic pages.
Use of option -r
How to force ab not to exit on receiving errors? You should use the option -r. Without this option, your test may break as soon as any request hits the socket error. However, with this option, errors will be reported in the failed errors heading, but the test will continue till the end.
Use of option -H
This option is used to add arbitrary header line. The argument is typically in the form of a valid header line, containing a colon-separated field-value pair (i.e., 'Accept-Encoding: zip/zop;8bit').
Use of option -C
In the following section, we will learn in detail how to use the above options in combination with the option to use the cookie value, i.e., the -C option. The -C option is typically in the form of a name = value pair. This field can be repeated.
Using Session Cookie with Apache Bench
To understand how to use the cookie with Apache Bench, we need a web page that tries to set a cookie. A very good example is the web2py application which is a python web framework.
Installing web2py
We are going to quickly install another python app web2py. You can read more on how to use it on Web2py Framework Overview.
Python is generally installed by default on the Ubuntu and Debian server. Therefore, one requirement is already met to run web2py successfully.
However, we need to install the unzip package to extract the source files of web2py from the zip file which we will be downloading −
Let us get the web2py framework from the project's website. We will download this to our home folder −
Now, we can unzip the file we just downloaded and move inside −
To run the web2py, you do not need to install it. Once you are inside the web2py directory, you can run it by typing the following command −
If everything is successful, you will see the following output where you will be asked to choose a password for the administrative UI −
However, you need to be aware of the fact that the launched web interface is accessible on the local machine only.
From the output, you can understand that to stop the web server, you will have to type 'CTRL-C' in the instant terminal. On the other hand, to stop the web2py server on the other terminal related to the same VPS, you can insert the command kill -SIGTERM , where is the process ID for the web2py server, which in this case is 23904.
Session Cookie from web2py
If a page is only accessible by a logged in user, not directly accessible from the login page, in that case you can use the -C flag. This flag defines a cookie for the ab command. But you have to get the value of the session identifier cookie from a valid session. How to get that? Various online tutorials will guide you towards Chrome (or Mozilla) browser developer tools. But in our test case, as the application is available only on the command line, we will use the lynx browser to obtain the value.
Let us get the cookie value of a session first. Open another terminal and type the following command −
In response to the above command, lynx will ask your permission to accept the cookie from the web2py server as shown in the image below.
Note down the cookie value before typing y to accept the cookie. Now the terminal will look similar to the following image – website on the terminal!
Having obtained the cookie value, we will now run the ab test. For that, we will have to open the third terminal (see the image below) −
Now, let us use the -C flag in the third terminal −
Output
From the output above, we note several points. First, web2py uses Rocket web server. We also note that we are getting ‘Non-2xx responses' in addition to previously discussed headings in the output. In general, Http protocol responds to a request using a response code, and anything within the 200s range means ‘okay', and the rest corresponds to some problem. For example, 400s are resource related errors such as 404 File Not Found. 500s correspond to server errors. In our instant case, there is no error anywhere except when we are using the -C option. It can be suppressed using the -l option as already described.
Checking Admin Page
In this section, we will understand how to check the admin page. For the purpose of comparison, let us test another URL of the web2py application −
Output
You should in particular note the respective statistics in section 'Connection Times' and 'Percentage of the requests served …' of http://127.0.0.1:8000/ and http://127.0.0.1:8000/admin. There is a huge difference.
Using Timelimit Option
Generally, Timelimit option is a tricky one. Let us understand this from the manual of ab, which is quite explanatory −
Let us run a test with this option. We will note our observations after the going through the output −
Output
Notice that the output shows this option overrides the number of requests specified by the -n option and continues upto the 50K requests. However, as the requests were handled very fast, ab has terminated as soon as 50k mark was achieved – within 22 seconds (see heading Time taken for tests) in the instant case.
You can test the same command replacing http://127.0.0.1:8000/ with http://127.0.0.1:8000/admin (assuming it is our web2py application) or a third party website like https://www.apache.org/, notice the difference in statistics.
Checklist Before Performing the Load Test
Apache Ab Windows Download
There are a few checks which will help you successfully run the test, and measure the performance accurately. Consider the following conditions before performing the load test −
Ensure that no extra python module is loaded.
To avoid TCP/IP Port Exhaustion, you should typically wait 2-3 minutes before you move to another ab test.
Ensure that the number of concurrent connections are lower than Apache Worker Threads.
You should reboot the server before performing another test, if Apache or python crashes.
In this chapter, we will describe the various combinations of -n and -c with the important flags to gradually increase the load on your webserver.
You should mainly focus on how the following metrics change as you increase the load −
- Requests per second
- Connection Times (ms)
- Percentage of the requests served within a certain time (ms)
You should also notice for the threshold value when server starts getting stuck and you start getting failed requests.
1 Concurrent User Doing 100 Page Hits
Let us do 100 sequential page-loads by a single user −
Output
5 Concurrent Users Each Doing 10 Page Hits
This case corresponds to a peak load on a website that gets around 50,000+ hits a month.
In the following subsequent outputs, we will be omitting the common header for clarity purpose.
Output
10 Concurrent Users Each Doing 10 Page Hits
This test corresponds to 100 page loads by 10 different concurrent users, each user is doing 10 sequential pages loads.
Output
20 Concurrent Users Each Doing 20 Page Hits
This test corresponds to 400 page loads by 20 different concurrent users, each user is doing 20 sequential pages loads.
Output
30 Concurrent Users Each Doing 30 Page Hits
This test corresponds to 900 page loads by 30 different concurrent users, each user is doing 30 sequential pages loads.
Output
We have now learned how to increase the load gradually on the website and test its performance.
Quick Analysis of the Load Testing Output
Having learned about the headings of the output values from the ab command, let us try to analyze and understand the output values for our initial test −
Apache organisation is using their own web Server Software − Apache (version 2.4.7)
Server is listening on Port 443 because of https. Had it been http, it would have been 80 (default).
Total data transferred is 58769 bytes for 100 requests.
Test completed in 1.004 seconds. There are no failed requests.
Requests per seconds − 99.56. This is considered a pretty good number.
Time per request − 100.444 ms (for 10 concurrent requests). So across all requests, it is 100.444 ms/10 = 10.044 ms.
Transfer rate − 1338.39 [Kbytes/sec] received.
In connection time statistics, you can observe that many requests had to wait for few seconds. This may be due to apache web server putting requests in wait queue.
In our first test, we had tested an application (i.e., www.apache.org) hosted on a different server. In the later part of the tutorial, we will be testing our sample web-applications hosted on the same server from which we will be running the ab tests. This is for the ease of learning and demonstration purpose. Ideally, the host node and testing node should be different for accurate measurement.
To better learn ab, you should compare and observe how the output values vary for different cases as we move forward in this tutorial.
Plotting the Output of Apache Bench
Here we will plot the relevant outcome to see how much time the server takes as the number of requests increases. For that, we will add the -g option in the previous command followed by the file name (here out.data) in which the ab output data will be saved −
Let us now see the out.data before we create a plot −
Output
Let us now understand the column headers in the out.data file −
starttime − This is the date and time at which the call started.
seconds − Same as starttime but in the Unix timestamp format (date -d @1496160697 returns starttime output).
ctime − This is the Connection Time.
dtime − This is the Processing Time.
ttime − This is the Total Time (it is the sum of ctime and dtime, mathematically ttime = ctime + dtime).
wait − This is the Waiting Time.
For a pictorial visualization of how these multiple items are related to each other, take a look at the following image −
If we are working over terminal or where graphics are not available, gnuplot is a great option. We will quickly understand it by going through the following steps.
Let us install and launch gnuplot −
Output
As we are working over terminal and supposing that graphics are not available, we can choose the dumb terminal which will give output in ASCII over the terminal itself. This helps us get an idea what our plot looks like with this quick tool. Let us now prepare the terminal for ASCII plot.
Output
As, our gnuplot terminal is now ready for ASCII plot, let us plot the data from the out.data file −
Output
We have plotted the ttime, total time (in ms) from column 9, with respect to the number of requests. We can notice that for the initial ten requests, the total time was in the nearly 100 ms, for next 30 requests (from 10th to 40th), it increased to 1100 ms, and so on. Your plot must be different depending on your out.data.
In the previous chapter, we understood the basic use of the Apache Bench to test a third party website. In this section, we will use this tool to test a web application on our own server. To keep the tutorial self-contained to the extent possible, we have chosen to install a python application for the demonstration purpose; you can choose any other language like PHP or Ruby depending on your expertise level.
Installing Python
Generally, Python is installed by default on Linux servers.
Installing Bottle Framework and Creating a Simple Application
Bottle is a micro-framework written in python for creating web applications, and pip is a python package manager. Type the following command in terminal to install Bottle −
Let us now create a small Bottle application. For that, create a directory and move inside it −
We will create a new python script, app.py, inside the webapp directory −
Now, write the following code in the app.py file −
When you have added the above lines, save and close the file. Having saved the file, we can run the python script to launch the application −
Output
This output shows that our application is running on the local machine at the host http://localhost and listening on the port 8080.
Let us check if our app is responding properly to the HTTP requests. As this terminal cannot take any input without quitting serving the Bottle application, we need to login to our VPS with another terminal. After logging into the VPS with another terminal, you can navigate to your application by typing the following code in the new terminal.
Lynx is a command line browser and is usually installed by default in various Linux distributions like Debian and Ubuntu. If you see the following output, it means your app is working fine.
Output
If you see the above output, that means our application is live and ready for testing.
Testing the Application with Developmental Web Server
Please note that there is a bug in ab, and it is not able to test the application on the localhost. So we will change the host from localhost to 127.0.0.1 in the app.py file. So the file will change to the following −
Let us now test our app by typing the following command on the same terminal on which ran the lynx command −
Output
While the output on the first terminal will be (100 times) as follows −
You can observe how the various values of the ab outcome have changed as compared to the initial test.
Testing the Application with a Multi-Threaded Web Server
In the previous tests of ab, we have used the default web server bundled in the Bottle framework.
Now we will change the single-threaded default web server with a multi-threaded one. Therefore, let us install a multi-threaded web server library like cherrypy or gunicorn and tell Bottle to use it. We have chosen gunicorn for the demonstration purpose here (you can choose some other one too) −
And modify the file, that is change from the default web server to gunicorn −
Let us test the app in the second terminal.
Output
Observe how the Requests per second increased from 493 to 3252. It means gunicorn is suitable as a production server for python apps.
In this chapter, we will learn how to test multiple URLs concurrently. For that, we will need to edit our application file, app.py to include two URLs −
Creating a Simple Shell Script
You can do this by creating a shell script, with multiple ab calls. Create a file test.sh and add the following lines to it −
When you have added the above lines, Save and Close the file. Make the file executable −
Let us now run the script −
To avoid repetition and purpose of clarity, we will show only the relevant of the ab output, indicating by dots what portion has been omitted, as in the following.
Output
Shell Script to Save the Apache Bench Output to a File
You can save the Apache Bench Output to file by creating a shell script, with multiple ab calls. At the end of each line, place an &; this makes the command run in the background, and lets the next command start its execution. You will also want to redirect the output to a file for each url using . For example, our file test.sh will look like the following after modification −
Here, test1.txt and test2.txt are the files to save the output data.
You can check that the above script has created two files, test1.txt and test2.txt which contains the ab output for the respective URLs −
Output
Watch-out Situation
While using ab, you should be alert to the failed test without warning. For example, if you check a wrong URL, you may get something similar to the following (we have deliberately changed the port here).
Output
In this chapter, we will understand the preparation required for testing dynamic pages. A server-side dynamic web page is a web page the construction of which is controlled by an application server processing server-side scripts. The apache bench can only load test the server-side dynamic web page.
Concurrency Level and the Total Number of Requests
Concurrency level should be lower than the total number of requests.
Output
Use of Flags
In this section, we will describe the use of some important flags with the ab command. We will use the terms, options and flags, interchangeably.
Verbose -v
The verbose option can be used to analyze and debug if there exist multiple number of failed requests. A common indication of failure of the load test is that the test finishes very fast and it gives a good number for request per second value. But it will be a wrong benchmark. To identify the success or failure, you can use the -v 2 option which will dump each response's body and header to the terminal output. Following command depicts a use case −
Output
Of course, if you are testing variable responses or returning non-200 HTTP codes in the event of any error, you should simply ignore length checking with the -l option. We will soon see non-200 HTTP when we will launch a web2py application in the subsequent chapters.
Keep-alive -k
When the client sends HTTP request, the connection is made to the server, the server sends the response, and the connection is closed after it has sent the request. This cycle continues with each request. However, with the keep-alive setting (also known as persistent connections), the client maintains an underlying TCP connection open to facilitate multiple requests and response; this eliminates the slow and costly connection initialization time that would otherwise be present.
Variable document length -l
If the web page is of variable length, then you should make use of the option -l. Apache Bench does not report errors if the length of the responses is not constant. This can be useful for dynamic pages.
Use of option -r
How to force ab not to exit on receiving errors? You should use the option -r. Without this option, your test may break as soon as any request hits the socket error. However, with this option, errors will be reported in the failed errors heading, but the test will continue till the end.
Use of option -H
This option is used to add arbitrary header line. The argument is typically in the form of a valid header line, containing a colon-separated field-value pair (i.e., 'Accept-Encoding: zip/zop;8bit').
Use of option -C
In the following section, we will learn in detail how to use the above options in combination with the option to use the cookie value, i.e., the -C option. The -C option is typically in the form of a name = value pair. This field can be repeated.
Using Session Cookie with Apache Bench
To understand how to use the cookie with Apache Bench, we need a web page that tries to set a cookie. A very good example is the web2py application which is a python web framework.
Installing web2py
We are going to quickly install another python app web2py. You can read more on how to use it on Web2py Framework Overview.
Python is generally installed by default on the Ubuntu and Debian server. Therefore, one requirement is already met to run web2py successfully.
However, we need to install the unzip package to extract the source files of web2py from the zip file which we will be downloading −
Let us get the web2py framework from the project's website. We will download this to our home folder −
Now, we can unzip the file we just downloaded and move inside −
To run the web2py, you do not need to install it. Once you are inside the web2py directory, you can run it by typing the following command −
If everything is successful, you will see the following output where you will be asked to choose a password for the administrative UI −
However, you need to be aware of the fact that the launched web interface is accessible on the local machine only.
From the output, you can understand that to stop the web server, you will have to type 'CTRL-C' in the instant terminal. On the other hand, to stop the web2py server on the other terminal related to the same VPS, you can insert the command kill -SIGTERM , where is the process ID for the web2py server, which in this case is 23904.
Session Cookie from web2py
If a page is only accessible by a logged in user, not directly accessible from the login page, in that case you can use the -C flag. This flag defines a cookie for the ab command. But you have to get the value of the session identifier cookie from a valid session. How to get that? Various online tutorials will guide you towards Chrome (or Mozilla) browser developer tools. But in our test case, as the application is available only on the command line, we will use the lynx browser to obtain the value.
Let us get the cookie value of a session first. Open another terminal and type the following command −
In response to the above command, lynx will ask your permission to accept the cookie from the web2py server as shown in the image below.
Note down the cookie value before typing y to accept the cookie. Now the terminal will look similar to the following image – website on the terminal!
Having obtained the cookie value, we will now run the ab test. For that, we will have to open the third terminal (see the image below) −
Now, let us use the -C flag in the third terminal −
Output
From the output above, we note several points. First, web2py uses Rocket web server. We also note that we are getting ‘Non-2xx responses' in addition to previously discussed headings in the output. In general, Http protocol responds to a request using a response code, and anything within the 200s range means ‘okay', and the rest corresponds to some problem. For example, 400s are resource related errors such as 404 File Not Found. 500s correspond to server errors. In our instant case, there is no error anywhere except when we are using the -C option. It can be suppressed using the -l option as already described.
Checking Admin Page
In this section, we will understand how to check the admin page. For the purpose of comparison, let us test another URL of the web2py application −
Output
You should in particular note the respective statistics in section 'Connection Times' and 'Percentage of the requests served …' of http://127.0.0.1:8000/ and http://127.0.0.1:8000/admin. There is a huge difference.
Using Timelimit Option
Generally, Timelimit option is a tricky one. Let us understand this from the manual of ab, which is quite explanatory −
Let us run a test with this option. We will note our observations after the going through the output −
Output
Notice that the output shows this option overrides the number of requests specified by the -n option and continues upto the 50K requests. However, as the requests were handled very fast, ab has terminated as soon as 50k mark was achieved – within 22 seconds (see heading Time taken for tests) in the instant case.
You can test the same command replacing http://127.0.0.1:8000/ with http://127.0.0.1:8000/admin (assuming it is our web2py application) or a third party website like https://www.apache.org/, notice the difference in statistics.
Checklist Before Performing the Load Test
Apache Ab Windows Download
There are a few checks which will help you successfully run the test, and measure the performance accurately. Consider the following conditions before performing the load test −
Ensure that no extra python module is loaded.
To avoid TCP/IP Port Exhaustion, you should typically wait 2-3 minutes before you move to another ab test.
Ensure that the number of concurrent connections are lower than Apache Worker Threads.
You should reboot the server before performing another test, if Apache or python crashes.
In this chapter, we will describe the various combinations of -n and -c with the important flags to gradually increase the load on your webserver.
You should mainly focus on how the following metrics change as you increase the load −
- Requests per second
- Connection Times (ms)
- Percentage of the requests served within a certain time (ms)
You should also notice for the threshold value when server starts getting stuck and you start getting failed requests.
1 Concurrent User Doing 100 Page Hits
Let us do 100 sequential page-loads by a single user −
Output
5 Concurrent Users Each Doing 10 Page Hits
This case corresponds to a peak load on a website that gets around 50,000+ hits a month.
In the following subsequent outputs, we will be omitting the common header for clarity purpose.
Output
10 Concurrent Users Each Doing 10 Page Hits
This test corresponds to 100 page loads by 10 different concurrent users, each user is doing 10 sequential pages loads.
Output
20 Concurrent Users Each Doing 20 Page Hits
This test corresponds to 400 page loads by 20 different concurrent users, each user is doing 20 sequential pages loads.
Output
30 Concurrent Users Each Doing 30 Page Hits
This test corresponds to 900 page loads by 30 different concurrent users, each user is doing 30 sequential pages loads.
Output
We have now learned how to increase the load gradually on the website and test its performance.
In this chapter, we will compare the outputs with and without flags. Let us see how the use of appropriate flags can increase the performance of your web application. Before that, we need to understand how if your application is simple then you may not notice the difference. As is the case with our simple application, with flags and without flags. Then we will perform the same test with https://www.apache.org/ URL, and see the difference.
Testing our Application without Flags
In this section, we will understand how to test our application without flags.
Output
Testing our Application with Flags
In this section, we will understand how to test our application with flags.
Output
We can simply note that there is not much difference between the output statistics.
Apache Ab Post Example
Testing Apache Organisation Website without Flags
Let us now see how to test the Apache Organisation Website without flags.
Output
Testing Apache Organisation Website with Flags
Let us now test the Apache Organisation Website with Flags.
Output
You can simply note how the request per second increased with the use of flags. In the instant case, it is particularly due to use of -H 'Accept-Encoding: gzip, deflate because this flag tells the Apache server to serve requests in gzipped format.
Considering the Apache Bench Results
A few important points need to be considered when it comes to the Apache Bench results. This will help us design our overall strategy to remove the bottlenecks in our application and improve its performance.
Apache Ab Cookie Example
We need to Requests Per Second. This gives us an idea of how well our web server set-up is working; the larger the number, the better the performance. Then comes the Connection Times (ms) and the Percentage of the requests served. You may have to tweak the settings of your web server to change these metrics to your desired performance.
Check if there are errors in the Apache's or the used web server error logs or (general) logs. As you will increase your load, things will start to choke: memory issues will start coming up. A lot of python scripts will begin to crash if they are not written with concurrency in mind.
You need to find out what is the critical concurrency value above which your web server crashes and/or times-out? Normally this should happen at a fairly high concurrency level. If this value is low, something is wrong and you need to adjust these settings lower/higher.
Conclusion
In this tutorial we learned how Apache Bench can be used to load test any web site or web application. Apache Bench can be a very valuable tool for determining how your web application server setup should be improved, to reduce bottlenecks and increase performance. Now that you are familiar with the basic usage of Apache Bench, you can start by creating new test plans to measure the performance of your applications in various scenarios.
Apache Bench
httpd and IHS ship with a cool little command line utility called Apache Bench (ab). At its simplest, you pass the number of requests you want to send (-n), at what concurrency (-c) and the URL to benchmark. ab will return various statistics on the responses (mean, median, max, standard deviation, etc.). This is really useful when you want to 'spot check' backend server performance or compare two different environments, because you do not need to install complex load testing software, and since IHS usually has direct access to WAS, you do not have to worry about firewalls, etc.
Below is an example execution.
The key things to look at are:
- Time taken for tests: This is how long it took for all requests finish. When comparing two environments, if your mean and median are similar but total time is worse in one case, this may suggest queueing effects.
- Failed requests, write errors, and Non-2xx responses: These may indicate some problem. See below for a caveat on 'Failed requests.'
- Requests per second: Throughput.
- Total: Look at min, mean, median, max and sd (standard deviation). Usually the mean is the place to start.
- Percentage of the requests served within a certain time: Response times on a percentile basis. Many customers look at 95%, but this is arbitrary and usually based on what percentage of requests are expected to have errors or do weird behavior.
Apache Ab Download
Some important notes:
Apache Ab Tool Benchmark Example
ab has odd behavior in that it counts requests with varying Content-Length headers as 'Failed requests' due to 'length;' for example:
It is common to have different content lengths, so usually this can be disregarded (only if the 'Length' number counts all the 'failed' requests). There is a patch for this, but it has never made it into the core code: https://issues.apache.org/bugzilla/show_bug.cgi?id=27888.
Non-2xx responses may or may not be okay. HTTP status codes are usually okay if they are 304 (Not Modified), for example. They are usually not okay if they are 4xx or 5xx. To get details on the response code, use '-v 2' which will print a warning for non-2xx response codes and list what the code was.
ab is not a browser, so when you request a page, ab will not fetch any resources within the page such as images, scripts, iframes, etc.
Apache Ab Upload File
Previous Section (R Project) | Next Section (awk) | Back to Table of Contents
