My guess is it is the time for DNS to resolve, since you are using the
constructor:

public TcpClient(
  string hostname,
  int port
);

if you know the IP address, then use the constructor:
[C#]
public TcpClient(
  IPEndPoint localEP
);

The following example demonstrates how to create an instance of the
TcpClient class using a local endpoint.
//Creates a TCPClient using a localend point.
IPAddress ipAddress = ipaddress;
IPEndPoint ipLocalEndPoint = new IPEndPoint(ipAddress, port);
try{
        TcpClient tcpClientA = new TcpClient(ipLocalEndPoint);
}
catch (Exception e ) {
         Console.WriteLine(e.ToString());
 }

(Sample is from MSDN)