aws-shell linux | unix

Find 504s in access logs, grep timestamps and correlate with error and php5-fpm logs

Web-Server throwing too many HTTP 504s? Use this script to drill down and find the cause

Script finds the 504s in the access logs (across zipped access logs as well), greps the timestamp of the log files and then finds the error logs (Across zipped error logs as well) corresponding to the timestamps of 504’s access log entries found.

  • OS – Linux 3.2.0-4-amd64 x86_64
  • Memory/RAM – 4GB
  • MySQL DB is running on a different server
  • WebServer – nginx/1.2.1 PHP – PHP 5.4.44-0+deb7u1 (cli)
  • PHP5-FPM – PHP-FPM is PHP’s FastCGI implementation and runs as a standalone FastCGI server. So, in simple words, nginx will pass any PHP request for dynamic content to the socket on which the FastCGI server is listening governed by fastcgi_pass  directive in nginx configuration file.

You can find the same script on my github link along with other useful scripts.


#Put whatever date range you would like to search the 504 errors between and do not forget to replace the Month's name in the following script as well.
for date in {20..26}
echo -e "\e[1;32mAccess logs 504s on $date/Sep - Location /var/log/nginx/ e[0m"

#Finding in access logs now
zgrep "$date/Sep/2018" /var/log/nginx/access.lo* | grep " 504 "
IFS=" "
for times in `zgrep "$date/Sep/2018" /var/log/nginx/access.lo* | grep " 504 " | awk '{print $4}' | awk -F: '{print $2" "$3}' | tr " " ":" | tr "\n" " "`

#Finding in error logs now
echo -e "\e[1;31mError logs on $date/Sep $times - Location (/var/log/nginx/)\e[0m"
zgrep "2018/09/$date $times" /var/log/nginx/error.lo*

#Finding in php5-fpm logs now
echo -e "\e[1;36mphp5-fpm logs on $date/Sep - Location /var/log/ \e[0m"
zgrep "$date-Sep-2018" /var/log/php5-fpm.lo*

#Pretty print
printf '%*s\n' "${COLUMNS:-$(tput cols)}" '' | tr ' ' -


Leave a Reply

Your email address will not be published. Required fields are marked *

five − 3 =