Get the Number of IP Address Occurrences in the Access Log

July 19th, 2011 - Posted by Steve Marks to Server Management, Web Development.

I had a scenario recently where bandwidth usage on a server was becoming an issue. As a result the need arose to find out if any third party websites or scripts were excessively referencing the server.

To do this, one of the steps taken was to identify and locate any IP addresses repeatedly appearing in the access log. Once logged onto the command line and having navigated to the access log directory the following command was ran:

time awk '{!a[$1]++}END{for(i in a) if ( a[i] >100 ) print a[i],i }' access_log

The above gets all IP addresses that appear in the access log more than 100 times and outputs something similar to the below:


real    0m0.006s
user    0m0.002s
sys     0m0.004s

Note: IP addresses shown in the above example have been made up

This entry was posted on Tuesday, July 19th, 2011 at 10:21 pm by +Steve Marks and is filed under Server Management, Web Development. You can follow any responses to this entry through the RSS 2.0 feed.

Fear not, we won't publish this

Comments (0)

No comments have been left yet. Be the first