Category Archives: Tips and Tricks

Throttling & Tuning Spring Boot

One of my Spring-Boot projects was battling with overloaded CPU and unresponsive / slow server response at times when there is more traffic. I have explored and implemented caching but my problem was with excessive connections coming in and server itself becoming slow. I could have setup multiple instances and do some kind of auto-scaling but given limited budget and hardware I wanted to put in some hard limits on my Spring-Boot app as to how much traffic it can take in and when it can give up gracefully (there is no shame in rejecting traffic with HTTP status 503 if the server infrastructure is overloaded).

I found a blog post entry from Netflix on how to tune Apache Tomcat and another article on how to tune Rest Controller code itself to implement a rudimentary Rate Limiter. I was glad to find the RateLimiter implementation in Goggle Guava library which I ultimately ended up using (for now). However I think the annotation driven RateLimiter is also a very good solution which is certainly very powerful and I will take it out for a spin sometime in near future.

The basic lesson learnt from this exercise:
– Tweak Tomcat and keep a watch on the acceptCount parameter which technically puts in a limit of how much traffic reaches your Rest controller.
– Use a RateLimiter on your hot APIs (which have higher latency) and don’t let your application get abused beyond a limit.
– Scale horizontally if the limits set above result in lot of traffic getting rejected.

Block a shell script till a server boots up

This small script blocks a shell script / docker compose command script till a dependent server boots up.

#!/usr/bin/env bash

while :
do
  response=$(curl --write-out %{http_code} --silent --output /dev/null -X GET "$1")
  if [[ "$response" -ne 200 ]] ; then
    echo "Server is not yet up >> $1 >> $response"
    sleep 1
  else
    echo "Server is up >> $1 >> $response"
    exit 1
  fi
done

Taking backup zip from multi-module Java project

I like to keep an archive of my code on a monthly basis. I wrote a small bash script to create zip out of my multi module Java projects. It gets the job done and can be improved to include resources and test folders as well. As of now I am only interested in the Java code zip backup.

!/bin/bash
 if [ -d "combined" ]; then
     echo "Removing existing zip files in combined .."
     rm combined/*.zip
 else
     echo "Creating combined folder.."
     mkdir combined
 fi
 for d in */ ; do
     if [ -d "$d/src/main/java" ]; then
         var="$(echo $d | sed 's/.$//')"
         echo "Processing $var .."
         cd $d/src/main/java
         zip -r ../../../../combined/$var.zip com
         cd ../../../..
     else
         echo "Ignoring .. $d"
     fi
 done

ssh_exchange_identification: read: Connection reset by peer

I recently upgraded my Mac Desktop and I noticed a strange problem that I was not able to SSH into my desktop from my laptop. I ultimately found out that the SSH setup that came with MacOS was having some issue as when I did ssh user@localhost on the desktop I got the same error:

ssh_exchange_identification: read: Connection reset by peer

I ended up tailing the system log using the following command:

sudo tail -f /var/log/system.log

I observed the following entries in the log whenever a SSH was attempted. 

com.apple.xpc.launchd[1] (com.openssh.sshd.[UUID][NUM]): Service exited with abnormal code: 1

It was clear that there was setup issue with the SSH that came with default MacOS installation. So I decided to spawn a separate SSH instance and watch it’s log:

sudo /usr/sbin/sshd -d -p 2222

This command showed up a lot of issues related to file permissions.

Permissions 0644 for '/etc/ssh/ssh_host_dsa_key' are too open.
Permissions 0644 for '/etc/ssh/ssh_host_ecdsa_key' are too open.
Permissions 0644 for '/etc/ssh/ssh_host_ed25519_key' are too open.

I fixed these permission issues by changing their permission to 400:

sudo chmod 400 /etc/ssh/ssh_host_dsa_key
sudo chmod 400 /etc/ssh/ssh_host_ecdsa_key
sudo chmod 400 /etc/ssh/ssh_host_ed25519_key

After this change the following command succeeded and I was able to do successful SSH connection to port 2222.

sudo /usr/sbin/sshd -d -p 2222

So I killed this process and decided to restart SSH:

sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist 
sudo lsof -i:22
echo $?
sudo launchctl load /System/Library/LaunchDaemons/ssh.plist 

Once SSHD was restarted I could successfully do logins using ssh user@localhost from Desktop as well as remote login via SSH from my laptop. 

Prettify JSON on Terminal

I love using curl command on my Mac terminal to debug my REST endpoints. However the REST call JSON output used to come in a blob of text which required further formatting in an Editor like Visual Studio Code. To allay this problem I ended up installing “jsonpp” using homebrew.

brew install jsonpp

So now I just pipe the output of my curl command to the jsonpp program and I get a fully formatted JSON.

$ curl http://localhost:8080/test | jsonpp
{
"year": 2018,
"month": 2,
"worked": 18,
"leaves": 2
}

 

 

Curl Command New Line Post Output

I like to use curl instead of UI tools like Postman for debugging my RESTful web services traffic whenever possible. I however didn’t like my output being messed up by the bash prompt being suffixed to the output. Something like the following:

$ curl -H "$auth_token" http://localhost:8080/xyz/abc-efg
["-","A","B","C","D","E"]$

So basically what I needed was to have a new line forced after the curl output. A quick search on internet yielded this article. So I executed the following command on my terminal.

$ echo '-w "\n"' >> ~/.curlrc

After doing this when I execute the same curl command I get the following output.

$ curl -H "$auth_token" http://localhost:8080/xyz/abc-efg
["-","A","B","C","D","E"]
$

So now the bash prompt is actually coming on a new line by default!

Validating Signature in PDF documents in Acrobat Reader

I received a digitally signed document from a trusted source. However when I proceeded to take a print it came out with “Signature Not Verified” in place of the signature field. This was not going to work so I did some googling and found out this link. It basically allowed me to validate the signature and take a printout with “Signature Valid” in place of the digital signature.

Uninstalling and reinstalling brew

I use brew utility for all my terminal based installation needs. However recently I had to deal with a corrupted brew install. To fix this I followed this link. The uninstallation and reinstallation worked fine and I now have a working brew instance.

The commands as mentioned in the above link are:

$ cd `brew –prefix`
$ rm -rf Cellar
$ brew prune
$ rm -rf Library .git .gitignore bin/brew README.md share/man/man1/brew
$ rm -rf ~/Library/Caches/Homebrew
$ ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”