Holy on Dev

Posts Tagged ‘tool’

Simulating network timeouts with toxiproxy

Posted by Jakub Holý on May 9, 2017

Goal: Simulate how a Node.js application reacts to timeouts.

Solution: Use toxiproxy and its timeout “toxic” with the value of 0, i.e. the connection won’t close, and data will be delayed until the toxic is removed.

The steps:

1. Start toxiproxy, exposing the port 6666 that we intend to use as localhost:6666:

docker pull shopify/toxiproxy
docker run --name=toxiproxy --rm --expose 6666 -p 6666:6666 -it shopify/toxiproxy

(If I was on Linux and not OSX then I could use --net=host and wouldn’t need to expose and/or map the port.)

2. Tell toxiproxy to serve request att 6666  via an upstream service:

docker exec -it toxiproxy /bin/sh
/ # cd /go/bin/
/go/bin # ./toxiproxy-cli create upstream -l -u google.com:443

3. Modify your code to access the local port 6666 and test that everything works.

Since we want to access Google via HTTPS, we would get a certificate error when accessing it via localhost:6666 (e.g. “SSLHandshakeException: PKIX path building failed: [..] unable to find valid certification path to requested target” in Java or (much better) “(51) SSL: no alternative certificate subject name matches target host name ‘localhost’” in curl) so we will add an alias to our local s /etc/hosts: proxied.google.com

and use
https://proxied.google.com:6666 in our connecting code (instead of the https://google.com:443 we had there before). Verify that it works and the code gets a response as expected.

Note: google.com is likely a bad choice here since it will return 404 as you must specify the header “Host: http://www.google.com” to get 200 OK back; without it you will get 404.

4. Tell toxiproxy to have an infinite timeout for this service

Continuing our toxiproxy configuration from step 2:

./toxiproxy-cli toxic add -t timeout -a timeout=0 upstream

(Alternatively, e.g. timeout=100; then the connection will be closed after 100 ms.)

5. Trigger your code again. You should get a timeout now.

Tip: You can simulate the service being down via disabling the proxy:

./toxiproxy-cli toggle upstream

Aside: Challenges when proxying through Toxiproxy

The host header

Servers (e.g. google.com, example.com) don’t like it when the Host header (derived normally from the URL) differs from what they expect. So you either need to make it possible to access localhost:<toxiproxy port> via the upstream server’s hostname by adding it as an alias to /etc/hosts (but how do you then access the actual service?) or you need to override the host header. In curl that is easy with -H "Host: http://www.google.com" but not so in Java.

In Java (openjdk 11.0.1 2018-10-16) you need to pass -Dsun.net.http.allowRestrictedHeaders=true to the JVM at startup to enable overriding the Host header (Oracle JVM might allow to do that at runtime) and then:

(doto ^HttpURLConnection (.openConnection (URL. "https://proxied.google.com:6666/"))
(.setRequestProperty "Host" "www.google.com")
SSL certificate issues

As described above, when talking to HTTPS via Toxiproxy, you need to ensure that the hostname you use in your request is covered by the server’s certificate, otherwise you will get SSL errors. To apply the solution described here, i.e. adding e.g. proxied.<server name, e.g. google.com> to your /etc/hosts works, provided the certificate is valid also for subdomains, i.e. is issued for <server> and *.<server>, which is not always the case.

Alternatively, you can disable certificate validation – trivial in curl with -k but much more typing in Java.

Posted in Tools, Uncategorized | Tagged: , , | Comments Off on Simulating network timeouts with toxiproxy

OS X: Using scutils to discover whether/what a web proxy is in use

Posted by Jakub Holý on May 7, 2015

When looking for ways to discover whether a proxy is being used by OS X, you will be typically pointed to

networksetup -getwebproxy

However that does not always work – for example when using “Auto Proxy Discovery” and/or “Automatic Proxy Configuration” with a proxy.pac file. scutils --proxy seems to detect all these cases (though it cannot give you the proxy when using auto config, I suppose):
Read the rest of this entry »

Posted in General | Tagged: , , | Comments Off on OS X: Using scutils to discover whether/what a web proxy is in use