Translating an enterprise Spring webapp to Clojure

How to translate the concepts and patterns we know from enterprise Java applications to Clojure? It is not just a different syntax, the whole philosophy of the language is different. The thing is, many concepts and patterns do not translate - you just do things differently. We will look shortly at how we can solve common enterprise concerns in Clojure, compared to Java.

This post is intended for an experienced Java developer curious about how his object-oriented, enterprise know-how would translate into the world of functional programming.

If you are short on time then just scan the Summary table and read Basic principles perhaps together with Clojure primer to make sense of it.


Continue reading →

Clojure vs Java: Troubleshooting an application in production

I have just gone through the painful experience of troubleshooting a remote Java webapp in a production-like environment and longed for Clojure’s explore-and-edit-running-app REPL. I want to demonstrate and contrast the tools the two languages offer for this case.


Continue reading →

Clojure vs Java: The benefit of Few Data Structures, Many Functions over Many Unique Classes

In Clojure we use again and again the same data structures and have many functions operating on them. Java programmers, on the other hand, create a unique class for every grouping of data, with its own “API” (getters, setters, return types, …) for accessing and manipulating the data. Having been forced to translate between two such “class APIs” I want to share my experience and thus demonstrate in practical terms the truth in the maxim

It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures.
- Alan Perils in Epigrams on Programming (1982)


Continue reading →

Solution design in Java/OOP vs. Clojure/FP - I/O anywhere or at the boundaries? - experience

As a Clojure developer thrown into an “enterprise” Java/Spring/Groovy application, I have a unique opportunity to experience and think about the differences between functional (FP) and object-oriented programming (OOP) and approach to design. Today I want to compare how the solution would differ for a small subsystem responsible for checking for and progressing the process of fixing data discrepancies. The main question we will explore will be where do we deal with external effects, i.e. I/O.

(We are going to explore here an application of the Functional Core, Imperative Shell architecture. You can learn more about it in the Related resources section.)


Continue reading →

It will only take one hour… (On why programmers suck at estimating and the perils of software development)

“It will only take about an hour,” I said to her. Two days later, a pull request awaits review. Where has all that time gone? What are the sources of delay in software development and how can we make it faster?


Continue reading →

Java/Spring App Troubleshooting on Steroids with Clojure REPL

(Published originally at the Telia Engineering blog.)

We have a Java/Groovy Spring Boot webapp, mainly running a bunch of batch jobs fetching, transforming and combining data. It is challenging to troubleshoot production issues because some production APIs are only accessible from the production servers and it is difficult and possibly dangerous to run the application in full production setup locally. Fortunately, we can now connect a REPL to the running application, get hold of its Spring beans, and interact with it (invoking remote calls, checking the returned data, …), which is a real life-saver and something I want to demonstrate and describe here.


Continue reading →

How to use Clojure 1.10 pREPL to connect to a remote server (WIP)

Clojure 1.10 includes a new, program-friendly REPL or prepl (pronounced as “preppy,” not p-repl). However there is still very little documentation about it, though it is reportedly in making (it is alpha, after all). Here I want to demonstrate how to start it and how to connect to it in a primitive way (I hope to improve the user experience of the client eventually).

Update 22/3: Check out O. Caldwell’s Clojure socket prepl cookbook.


Continue reading →

AWS RDS: How to find out login credentials to the database

To log in to your AWS RDS database (Oracle in my case) you need login credentials, but what are these for a newly created DB? The password is the master user password you entered during DB creation and which you can change via the Console.

To find out the master user name:


Continue reading →

Migrating from Wordpress.com to a static site generated by GatsbyJS

I am moving my blog over from Wordpress.com to a statically generated site using Gatsby. Wordpress has served me well in many years but it isn’t really fit for writing (about) code and the latest updates have made it even more difficult for me. With Gatsby I get a quick site and full control over everything (using JavaScript, React, and any of the tons of plugins for Gatsby).

The content from my old blog is coming soon, it is work in progress :-)


Continue reading →

Java: Simulating various connection problems with Toxiproxy

Java: Simulating various connection problems with Toxiproxy

Simulate various connection issues with Toxiproxy and Java's HttpURLConnection to see what kind of errors get produced: connect timed out vs. read timed out vs. connection refused ... .

Results:

System: openjdk 11.0.1 2018-10-16

  1. (.setConnectTimeout 1) => java.net.SocketTimeoutException: connect timed out
  2. (.setReadTimeout 1) => javax.net.ssl.SSLProtocolException: Read timed out on HTTPS, java.net.SocketTimeoutException: Read timed out on HTTP (or Toxiproxy with 5s latency or timeout )
  3. Nothing listening at the port => java.net.ConnectException: Connection refused
  4. Toxiproxy with no upstream configured (i.e. the port is open, but nothing happesn with the connection) => javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake on HTTPS, java.net.SocketTimeoutException: Read timed out on HTTP
  5. limit_data_downstream => java.io.IOException: Premature EOF

(What I haven't been able to simulate (yet) is "connection interrupted/broken", i.e. java.net.SocketException Connection reset (perhaps you closed it and try to write to it anyway?) and java.net.SocketException Connection reset by peer (perhaps when dropped by a firewall/the server/...?).)

The setup

Prerequisities

To /etc/hosts add:

127.0.0.1       proxied.google.com

The toxiproxy setup

Start toxiproxy:

docker pull shopify/toxiproxy
# BEFORE we `run` it: case #3
docker run --rm -p 5555:5555 -p 6666:6666 -p 8474:8474 --name toxiproxy -it shopify/toxiproxy

Configure it (we could just POST to :8474 but using the CLI is easier):

$ docker exec -it toxiproxy /bin/sh
/ # cd /go/bin/
# ./toxiproxy-cli create google -l 0.0.0.0:6666 -u www.google.com:443 # BEFORE this is run: case #4
# ./toxiproxy-cli toxic add google -t latency -a latency=5000 # case #2
Added downstream latency toxic 'latency_downstream' on proxy 'google
# ./toxiproxy-cli toxic remove google -n latency_downstream
Removed toxic 'latency_downstream' on proxy 'google'

# ./toxiproxy-cli toxic add google -t  timeout -a timeout=2000 # case #2
Added downstream timeout toxic 'timeout_downstream' on proxy 'google'
# ./toxiproxy-cli toxic remove google -n timeout_downstream
Removed toxic 'timeout_downstream' on proxy 'google'

# ./toxiproxy-cli toxic add google -t limit_data -a bytes=5000 # case #5
Added downstream limit_data toxic 'limit_data_downstream' on proxy 'google'

The test code

(import '[java.net URL HttpURLConnection])
(->
  (doto ^HttpURLConnection (.openConnection (URL. "https://proxied.google.com:6666/"))
    ;; BEWARE: JVM *must* be started with `-Dsun.net.http.allowRestrictedHeaders=true` to allow setting the Host:
    (.setRequestProperty "Host" "www.google.com")
    (.setConnectTimeout 1000)
    (.setReadTimeout 1000))
  (.getInputStream)
  slurp)

Background

Read my Simulating network timeouts with toxiproxy to learn why we need to bother with /etc/hosts and the Host header.

view rawREADME.md hosted with ❤ by GitHub

Continue reading →

Clojure - comparison of gnuplot, Incanter, oz/vega-lite for plotting usage data

What is the best way to plot memory and CPU usage data (mainly) in Clojure? I will compare gnuplot, Incanter with JFreeChart, and vega-lite (via Oz). (Spoiler: I like Oz/vega-lite most but still use Incanter to prepare the data.)

The data looks like this:

;; sec.ns | memory | CPU %
1541052937.882172509 59m 0.0
1541052981.122419892 78m 58.0
1541052981.625876498 199m 85.9
1541053011.489811184 1.2g 101.8


The data has been produced by monitor-usage.sh.

The tools



Gnuplot 5



Gnuplot is the simplest, with a lot available out of the box. But it is also somewhat archaic and little flexible.


Continue reading →

How I got fired and learned the importance of communication and play time

When I came to the office one late autumn morning in 2005, I have been shocked to find out that - without any warning signs whatsoever - I hd been fired. That day I have learned the importance of communication. Their criticism was justified but the thing is, nobody bothered to tell me anything during my 11 months in the company. I received exactly 0 feedback about my behaviour or work. The company ended up at court with its client - which both explains why they were stressed and was also caused by bad communication. So communication - even, or especially under stress - is really important. It must be open, transparent, and broad.



The funny thing is that I still do the things they fired me for.




Continue reading →

Beware the performance cost of async_hooks (Node 8)

I was excited about async_hooks
Continue reading →

How good monitoring saved our ass ... again

You know how it goes - suddenly people complain your app does not work, your are getting plenty of timeouts or other errors in your error tracking tool, you find the backend app that is misbehaving and finally "fix" the problem by restarting it. Phew!

Continue reading →

Monitoring process memory/CPU usage with top and plotting it with gnuplot

siege-c3e2

If you want to monitor the memory and CPU usage of a particular Linux process for a few minutes, perhaps during a performance test, you can capture the data with top and plot them with gnuplot. Here is how:


Continue reading →

Copyright © 2021 Jakub Holý
Powered by Cryogen
Theme by KingMob