Monday, January 26, 2015

How to add a Custom NAT instance in AWS VPC?

In this tutorial I am assuming , you must be running VPC in AWS.

NAT Instances

Instances that you launch into a private subnet in a virtual private cloud (VPC) can't communicate with the Internet. You can optionally use a network address translation (NAT) instance in a public subnet in your VPC to enable instances in the private subnet to initiate outbound traffic to the Internet, but prevent the instances from receiving inbound traffic initiated by someone on the Internet.


To launch NAT instance in AWS , search for NAT in community AMI section , AWS provides lots of NAT instances AMI .

On the Choose an Instance Type page, select the instance type, then click Next: Configure Instance Details.

On the Configure Instance Details page, select the VPC you created from the Network list, and select your public subnet from the Subnet list.

Once NAT instance launch disable the SrcDestCheck attribute for the NAT instance.




Click on “Yes,Disable




Connect to the NAT instance using terminal emulation software (i.e. putty), and allow the ip forwarding on it:

vi /etc/sysctl.conf

Uncomment the below line

net.ipv4.ip_forward=1

Issue the Iptables command for  MASQUERADE:

 iptables -t nat -A POSTROUTING  -s  10.0.0.0/16 -o eth0 -j MASQUERADE

Modify the NAT instance security group to allow all or desired inbound traffic from private subnet (In my case, 10.100.20.0/24) or desired server.

Create a custom route, associate your private subnet(s) to it and make a default route to use the NAT instance as a gateway:

Tuesday, April 29, 2014

How To Analyze HAProxy Logs

Sometimes it is very difficult to analyse the HaProxy Logs manually . Very few people know about the small tool name halog , it gets shipped with HaProxy itself.
HALog is a small and very powerful tool to analyze HaProxy log lines.Installation is pretty simple, as described bellow:
  • cd /usr/src
  • wget http://haproxy.1wt.eu/download/1.5/src/devel/haproxy-1.5-dev11.tar.gz
  • tar xzf haproxy-1.5-dev11.tar.gz
  • cd haproxy-1.5-dev11/contrib/halog
  • make
  • cp halog /usr/bin/
and we are done and ready to analyse the HaProxy Logs. Below are few examples 
  1. List servers by number of requests treated
The command below lists the servers by the number of requests they treated. The server name is
prefixed by the backend name.
 
The eighth columns "tot_req" gives the number of requests treated by the server.
 
cat filename.log | halog -srv -H -q |awk ’NR==1; NR > 1 {print $0 | "sort -n -r -k 9"}’ |column -t
 
       2. List servers by response time
 
The command below lists the servers by response time. The server name is prefixed by the backend
name.
The response time is in milliseconds and the latest columns "avg_rt" gives the average response time
for all the URLs forwarded to this server in this backend.
 
cat filename.log | halog -srv -H -q |awk ’NR==1; NR > 1 {print $0 | "sort -n -r -k 12"}’ |column -t
 
      3. List servers by application errors: HTTP status code 5xx
 
The command below lists the servers by number of application errors. The server name is prefixed by
the backend name.
The sixth column "5xx" gives the number of application errors generated by the server.
 
cat filename.log |halog -srv -H -q | awk ’NR==1; NR > 1 {print $0 | "sort -n -r -k 6"}’ | column -t
 
      4. List servers by errors
 
The command below lists the servers by number of errors not related to the application. The server
name is prefixed by the backend name.
 
cat filename.log |halog -srv -H -q | awk ’NR==1; NR > 1 {print $0 | "sort -n -r -k 5"}’ | column -t
 
    5. List URLs by server computation time
The command below lists the URLs by the average computation time, whatever the server which treated
it.
The sixth column "okavg" provides the URL average computation time in milliseconds.
 
cat filename.log | halog -ut -H -q | column -t
 
    6. List URLs by errors
 
The command below lists the URLs by the number of errors they have generated, whatever the server
which treated it or the type of error.
The second column "err" provides the number of errors generated by the given URL (latest column).
 
cat filename.log | halog -ue -H -q | column -t
 
   7 . List URLs by missing files: HTTP status code 404
The command below lists the URLs by the number of missing files error they have generated, whatever
the server which treated it.
The first column "req" provides the number of 404 returned for the given URL (latest column).
 
cat filename.log | halog -u -H -q -hs 404 | column -t
 
    8. List URLs by number of request
The command below lists the URLs by the number of time they have been requested on the platform.
The first column "req" provides the number of time the URLs was called.
 
cat filename.log | halog -u -H -q | awk ’NR==1; NR > 1 {print $0 | "sort -n -r -k 1"}’ | column -t